[libcamera-devel] [RFC PATCH 1/6] utils: ipc: Add script to extract doxygen docs from mojom files

[paul.elder at ideasonboard.com]

Hi Umang,

On Mon, May 24, 2021 at 03:12:06PM +0530, Umang Jain wrote:
> Hi Paul,
> 
> Thanks for this work. The script, essentially the meat of the series, looks
> great!

\o/ thanks!

> 
> On 5/24/21 2:10 PM, Paul Elder wrote:
> > Add a script to extract doxygen documentation comments from mojom files.
> > It matches based on ^\/\*\*$ for start of block and ^ \*\/$ for end of
> > block, and simply copies the comments to the output file along with a
> > header and the libcamera namespace.
> > 
> > Also add it to the meson file so it is usable by other meson files.
> > 
> > Signed-off-by: Paul Elder <paul.elder at ideasonboard.com>
> Acked-by: Umang Jain <umang.jain at ideasonboard.com>
> > ---
> >   utils/ipc/extract-docs.py | 77 +++++++++++++++++++++++++++++++++++++++
> >   utils/ipc/meson.build     |  2 +
> >   2 files changed, 79 insertions(+)
> >   create mode 100755 utils/ipc/extract-docs.py
> > 
> > diff --git a/utils/ipc/extract-docs.py b/utils/ipc/extract-docs.py
> > new file mode 100755
> > index 00000000..b6667057
> > --- /dev/null
> > +++ b/utils/ipc/extract-docs.py
> > @@ -0,0 +1,77 @@
> > +#!/usr/bin/env python3
> > +# SPDX-License-Identifier: GPL-2.0-or-later
> > +# Copyright (C) 2021, Google Inc.
> > +#
> > +# Author: Paul Elder <paul.elder at ideasonboard.com>
> > +#
> > +# extract-docs.py - Extract doxygen documentation from mojom files
> > +
> > +import argparse
> > +import re
> > +import sys
> > +
> > +regex_block_start = re.compile('^\/\*\*$')
> > +regex_block_end = re.compile('^ \*\/$')
> > +
> > +
> > +def main(argv):
> > +
> > +    # Parse command line arguments
> > +    parser = argparse.ArgumentParser()
> > +    parser.add_argument('-o', dest='output', metavar='file', type=str,
> > +                        help='Output file name. Defaults to standard output if not specified.')
> > +    parser.add_argument('input', type=str,
> > +                        help='Input file name.')
> > +    args = parser.parse_args(argv[1:])
> > +
> > +    lines = open(args.input, 'r').readlines()
> > +    pipeline = args.input.split('/')[-1].replace('.mojom', '')
> > +    data = f'''\
> > +/* SPDX-License-Identifier: LGPL-2.1-or-later */
> > +/*
> > + * Copyright (C) 2021, Google Inc.
> > + *
> > + * {pipeline}_ipa_interface.cpp - Docs file for generated {pipeline}.mojom
> > + *
> > + * This file is auto-generated. Do not edit.
> > + */
> > +
> > +namespace libcamera {{
> > +
> > +'''
> > +
> > +    in_block = False
> > +    comment = ''
> > +    for lineno, line in enumerate(lines, start=1):
> > +        if regex_block_start.match(line):
> > +            if in_block:
> > +                raise SyntaxError('Expected end of comment',
> > +                                  (args.input, lineno, 1, line))
> > +            in_block = True
> > +            comment = line
> > +            continue
> > +
> > +        if regex_block_end.match(line):
> > +            if in_block:
> > +                comment += line
> > +                data += comment + '\n'
> > +            in_block = False
> > +            continue
> > +
> > +        if in_block:
> > +            comment += line
> > +
> 
> The only bit I was wondering, if we needed a '\n' after every block is
> extracted. But I guess, it makes /no/ difference to the doxygen if a \n is
> present between blocks.

It's not really necessary, but I like generated code to be at least
readable, and I think a blank line in between blocks helps with that.

I have a bunch of those in the generated code for IPC too.


Paul

> > +    data += '} /* namespace libcamera */\n'
> > +
> > +    if args.output:
> > +        output = open(args.output, 'wb')
> > +        output.write(data.encode('utf-8'))
> > +        output.close()
> > +    else:
> > +        sys.stdout.write(data)
> > +
> > +    return 0
> > +
> > +
> > +if __name__ == '__main__':
> > +    sys.exit(main(sys.argv))
> > diff --git a/utils/ipc/meson.build b/utils/ipc/meson.build
> > index 4a41b9c1..973a5417 100644
> > --- a/utils/ipc/meson.build
> > +++ b/utils/ipc/meson.build
> > @@ -8,6 +8,8 @@ mojom_parser = find_program('./parser.py')
> >   mojom_generator = find_program('./generate.py')
> > +mojom_docs_extractor = find_program('./extract-docs.py')
> > +
> >   mojom_templates = custom_target('mojom_templates',
> >                                   input : mojom_template_files,
> >                                   output : 'libcamera_templates.zip',
>