[libcamera-devel] [PATCH 00/38] IPA isolation implementation
Paul Elder
paul.elder at ideasonboard.com
Tue Sep 22 15:34:59 CEST 2020
This patch series depends on the patch series "Unify utils locations".
This patchset implements IPA isolation, and is fully plumbed and
runnable on all pipeline handlers. This also includes code
generators for the IPA proxies, and headers and serializers for custom
data structures and functions. Unlike the RFC, this patch series is
structured like a normal patch series, in order that should be merged.
The only exception is patches 16/38-36/38, which must be squashed to
avoid bisection breakage, and are only kept separate to ease review.
To see samples of generated code (without running this), see the RFC
"IPA isolation example, with IPC, fully plumbed".
To restate the problem, we have two goals:
- to be able to run IPAs isolated in a separate process
- the isolation must be transparent to both the pipeline handler and
the IPA
During design of the IPC mechanism, we realized that we could support
both custom fuctions and custom data structures, which would be a lot
nicer than the tedious manual de/serialization that we had before with
IPAOperationData.
The architecture of the whole thing is as follows:
pipeline handler -> IPAProxyRPi --thread--> IPARPi
\
\-> IPAIPC --IPC--> IPAProxyRPiWorker -> IPARPi
Where the pipeline author defines the interface between the pipeline
handler and the IPA, as well as any data structures. Based on this,
headers, de/serializers, and the proxy (including the worker) will be
generated.
Patches 1/38~04/38 will be fast tracked, and I plan to merge them
separately from the rest of this series after they are approved in this
round.
Patch 6/38 imports mojo directly from the chromium repos, with some
big things pruned. Patches 8/38 to 15/38 add the meat of this IPC and
custom IPA interface mechanisms. Patches 16/38 to 36/38 are modifications
in the rest of libcamera to make the whole mechanism work, and must be
squashed together to avoid bisection breakage. Patches 37/38 and 38/38
add tests for the serializer and IPC components of this series.
Patch 27/38 may be of interest, as it shows how one would write the
interface and data definition. In v2 I have customized the raspberrypi
mojom file to take advantage of the freedom in definition the IPA
interface and data structures (as opposed to v1, where it was
practically a direct translation of IPAOperationData).
In v2 I have added some rules to the mojom file, such as start(),
stop(), and init() are required, along with the main and callback
interfaces. These are documented in core.mojom in patch 26/38 and
enforced in the code generator in patch 08/38. core.mojom is a new
addition in v2, and is where common mojom "structs" (which represent
libcamera objects) and IPA interface definition documentaion are.
Patch 28/38 is also noteworthy, as it shows how one would use the
interface that has been defined. Since the headers and proxies are
autogenerated, there isn't a patch where the generated code can be seen,
however.
Patches 29/38 and 30/38 also define mojom files and use them, but are
for vimc and rkisp1, respectively. These will also give an idea on how
the mojom definition looks and how it will be used.
Changes in v2:
- upgrade documentation
- fix documentation compiling
- plumb the new IPC system through the other pipelines
- fix the issue where editing jinja templates wouldn't trigger
recompile
- enforce IPA interface rules in code generator
- fix previously-untriggered code generation bugs
- add de/serializers for all primitive types, string, and const
ControlList
- customized the RPi IPA interface
- add licenses
- add tests
Paul Elder (38):
Documentation: coding-style: Document global variable guidelines
libcamera: ProcessManager: make ProcessManager lifetime explicitly
managed
libcamera: PipelineHandler: Move printing pipeline names to
CameraManager
libcamera: process: forward-declare EventNotifier
libcamera: meson: Add internal headers to sources
utils: ipc: import mojo
libcamera: Update dep5 to specify license for mojo
utils: ipc: add templates for code generation for IPC mechanism
utils: ipc: add generator script
Documentation: skip generating documentation for generated code
meson: ipa, proxy: Generate headers and proxy with mojo
ipa: raspberrypi: meson: Add dependency on generated headers
libcamera: Add IPADataSerializer
libcamera: Add IPAIPC
libcamera: Add IPAIPC implementation based on unix socket
libcamera: IPAModule: Replace ipa_context with IPAInterface
libcamera: ipa_context_wrapper: Remove ipa_context_wrapper
libcamera: IPAProxy: Remove stop() override
libcamera: IPAProxy: Add isolate parameter to create()
libcamera: PipelineHandler: Remove IPA from base class
libcamera: IPAInterface: Remove all functions from IPAInterface
libcamera: IPAInterface: make ipaCreate return IPAInterface
libcamera: IPAInterface: remove ipa_context and functions from
documentation
libcamera: IPAManager: Fetch IPAProxy corresponding to pipeline
libcamera: IPAManager: add isolation flag to proxy creation
ipa: Add core.mojom
ipa: raspberrypi: Add mojom data definition file
libcamera: pipeline, ipa: raspberrypi: Use new data definition
libcamera: pipeline, ipa: vimc: Support the new IPC mechanism
libcamera: pipeline, ipa: rkisp1: Support the new IPC mechanism
libcamera: IPAProxy: Remove registration mechanism
libcamera: proxy: Remove IPAProxyLinux and IPAProxyThread
libcamera: IPAManager: Make createIPA return proxy directly
ipa: remove libipa
tests: ipa_interface_test: Update to use new createIPA
tests: Remove IPA wrappers test
tests: Add IPADataSerializer test
tests: Add test for IPAIPCUnixSocket
.reuse/dep5 | 6 +
Documentation/Doxyfile.in | 7 +-
Documentation/coding-style.rst | 15 +
Documentation/meson.build | 2 -
.../libcamera/internal/ipa_context_wrapper.h | 52 -
.../libcamera/internal/ipa_data_serializer.h | 1220 +++++++++++
include/libcamera/internal/ipa_ipc.h | 41 +
.../libcamera/internal/ipa_ipc_unixsocket.h | 115 +
include/libcamera/internal/ipa_manager.h | 31 +-
include/libcamera/internal/ipa_module.h | 4 +-
include/libcamera/internal/ipa_proxy.h | 31 -
include/libcamera/internal/meson.build | 1 -
include/libcamera/internal/pipeline_handler.h | 1 -
include/libcamera/internal/process.h | 29 +
include/libcamera/ipa/core.mojom | 74 +
include/libcamera/ipa/ipa_interface.h | 126 +-
include/libcamera/ipa/meson.build | 89 +
include/libcamera/ipa/raspberrypi.h | 55 +-
include/libcamera/ipa/raspberrypi.mojom | 157 ++
include/libcamera/ipa/rkisp1.h | 14 +-
include/libcamera/ipa/rkisp1.mojom | 42 +
include/libcamera/ipa/{ipa_vimc.h => vimc.h} | 4 +
include/libcamera/ipa/vimc.mojom | 12 +
src/ipa/libipa/ipa_interface_wrapper.cpp | 285 ---
src/ipa/libipa/ipa_interface_wrapper.h | 61 -
src/ipa/libipa/meson.build | 15 -
src/ipa/meson.build | 2 -
src/ipa/raspberrypi/meson.build | 4 +-
src/ipa/raspberrypi/raspberrypi.cpp | 154 +-
src/ipa/rkisp1/meson.build | 5 +-
src/ipa/rkisp1/rkisp1.cpp | 55 +-
src/ipa/vimc/meson.build | 5 +-
src/ipa/vimc/vimc.cpp | 20 +-
src/libcamera/camera_manager.cpp | 5 +
src/libcamera/ipa_context_wrapper.cpp | 297 ---
src/libcamera/ipa_data_serializer.cpp | 154 ++
src/libcamera/ipa_interface.cpp | 517 +----
src/libcamera/ipa_ipc.cpp | 110 +
src/libcamera/ipa_ipc_unixsocket.cpp | 175 ++
src/libcamera/ipa_manager.cpp | 47 +-
src/libcamera/ipa_module.cpp | 10 +-
src/libcamera/ipa_proxy.cpp | 101 -
src/libcamera/meson.build | 6 +-
.../pipeline/raspberrypi/raspberrypi.cpp | 170 +-
src/libcamera/pipeline/rkisp1/rkisp1.cpp | 44 +-
src/libcamera/pipeline/vimc/vimc.cpp | 8 +-
src/libcamera/pipeline_handler.cpp | 11 -
src/libcamera/process.cpp | 46 +-
src/libcamera/proxy/ipa_proxy_linux.cpp | 103 -
src/libcamera/proxy/ipa_proxy_thread.cpp | 172 --
src/libcamera/proxy/meson.build | 19 +-
.../proxy/worker/ipa_proxy_linux_worker.cpp | 90 -
src/libcamera/proxy/worker/meson.build | 22 +-
test/ipa/ipa_interface_test.cpp | 7 +-
test/ipa/ipa_wrappers_test.cpp | 452 ----
test/ipa/meson.build | 5 +-
test/ipc/meson.build | 3 +-
test/ipc/unixsocket_ipc.cpp | 238 ++
test/log/log_process.cpp | 2 +
test/process/process_test.cpp | 2 +
.../ipa_data_serializer_test.cpp | 463 ++++
test/serialization/meson.build | 3 +-
utils/ipc/generate.py | 21 +
.../libcamera_templates/meson.build | 11 +
.../module_generated.h.tmpl | 96 +
.../module_ipa_proxy.cpp.tmpl | 222 ++
.../module_ipa_proxy.h.tmpl | 107 +
.../module_ipa_proxy_worker.cpp.tmpl | 178 ++
.../module_serializer.h.tmpl | 44 +
.../libcamera_templates/proxy_functions.tmpl | 185 ++
.../libcamera_templates/serializer.tmpl | 268 +++
utils/ipc/generators/meson.build | 3 +
.../generators/mojom_libcamera_generator.py | 452 ++++
utils/ipc/meson.build | 14 +
utils/ipc/mojo/public/LICENSE | 27 +
utils/ipc/mojo/public/tools/.style.yapf | 6 +
utils/ipc/mojo/public/tools/BUILD.gn | 18 +
utils/ipc/mojo/public/tools/bindings/BUILD.gn | 108 +
.../ipc/mojo/public/tools/bindings/README.md | 816 +++++++
.../chromium_bindings_configuration.gni | 51 +
.../tools/bindings/compile_typescript.py | 27 +
.../tools/bindings/concatenate-files.py | 54 +
...concatenate_and_replace_closure_exports.py | 73 +
.../bindings/format_typemap_generator_args.py | 36 +
.../tools/bindings/gen_data_files_list.py | 52 +
.../tools/bindings/generate_type_mappings.py | 187 ++
.../ipc/mojo/public/tools/bindings/mojom.gni | 1941 +++++++++++++++++
.../bindings/mojom_bindings_generator.py | 390 ++++
.../mojom_bindings_generator_unittest.py | 62 +
.../tools/bindings/mojom_types_downgrader.py | 119 +
.../tools/bindings/validate_typemap_config.py | 57 +
utils/ipc/mojo/public/tools/mojom/README.md | 14 +
.../mojom/check_stable_mojom_compatibility.py | 170 ++
...eck_stable_mojom_compatibility_unittest.py | 260 +++
.../mojo/public/tools/mojom/const_unittest.py | 90 +
.../mojo/public/tools/mojom/enum_unittest.py | 92 +
.../mojo/public/tools/mojom/mojom/BUILD.gn | 43 +
.../mojo/public/tools/mojom/mojom/__init__.py | 0
.../mojo/public/tools/mojom/mojom/error.py | 28 +
.../mojo/public/tools/mojom/mojom/fileutil.py | 45 +
.../tools/mojom/mojom/fileutil_unittest.py | 40 +
.../tools/mojom/mojom/generate/__init__.py | 0
.../mojom/mojom/generate/constant_resolver.py | 93 +
.../tools/mojom/mojom/generate/generator.py | 325 +++
.../mojom/generate/generator_unittest.py | 74 +
.../tools/mojom/mojom/generate/module.py | 1635 ++++++++++++++
.../mojom/mojom/generate/module_unittest.py | 31 +
.../public/tools/mojom/mojom/generate/pack.py | 258 +++
.../mojom/mojom/generate/pack_unittest.py | 225 ++
.../mojom/mojom/generate/template_expander.py | 83 +
.../tools/mojom/mojom/generate/translate.py | 854 ++++++++
.../mojom/generate/translate_unittest.py | 73 +
.../tools/mojom/mojom/parse/__init__.py | 0
.../public/tools/mojom/mojom/parse/ast.py | 427 ++++
.../tools/mojom/mojom/parse/ast_unittest.py | 121 +
.../mojom/mojom/parse/conditional_features.py | 82 +
.../parse/conditional_features_unittest.py | 233 ++
.../public/tools/mojom/mojom/parse/lexer.py | 251 +++
.../tools/mojom/mojom/parse/lexer_unittest.py | 198 ++
.../public/tools/mojom/mojom/parse/parser.py | 488 +++++
.../mojom/mojom/parse/parser_unittest.py | 1390 ++++++++++++
.../mojo/public/tools/mojom/mojom_parser.py | 361 +++
.../tools/mojom/mojom_parser_test_case.py | 73 +
.../tools/mojom/mojom_parser_unittest.py | 171 ++
.../tools/mojom/stable_attribute_unittest.py | 127 ++
.../mojom/version_compatibility_unittest.py | 397 ++++
.../public/tools/run_all_python_unittests.py | 28 +
utils/ipc/tools/diagnosis/crbug_1001171.py | 51 +
utils/meson.build | 1 +
129 files changed, 17795 insertions(+), 2678 deletions(-)
delete mode 100644 include/libcamera/internal/ipa_context_wrapper.h
create mode 100644 include/libcamera/internal/ipa_data_serializer.h
create mode 100644 include/libcamera/internal/ipa_ipc.h
create mode 100644 include/libcamera/internal/ipa_ipc_unixsocket.h
create mode 100644 include/libcamera/ipa/core.mojom
create mode 100644 include/libcamera/ipa/raspberrypi.mojom
create mode 100644 include/libcamera/ipa/rkisp1.mojom
rename include/libcamera/ipa/{ipa_vimc.h => vimc.h} (87%)
create mode 100644 include/libcamera/ipa/vimc.mojom
delete mode 100644 src/ipa/libipa/ipa_interface_wrapper.cpp
delete mode 100644 src/ipa/libipa/ipa_interface_wrapper.h
delete mode 100644 src/ipa/libipa/meson.build
delete mode 100644 src/libcamera/ipa_context_wrapper.cpp
create mode 100644 src/libcamera/ipa_data_serializer.cpp
create mode 100644 src/libcamera/ipa_ipc.cpp
create mode 100644 src/libcamera/ipa_ipc_unixsocket.cpp
delete mode 100644 src/libcamera/proxy/ipa_proxy_linux.cpp
delete mode 100644 src/libcamera/proxy/ipa_proxy_thread.cpp
delete mode 100644 src/libcamera/proxy/worker/ipa_proxy_linux_worker.cpp
delete mode 100644 test/ipa/ipa_wrappers_test.cpp
create mode 100644 test/ipc/unixsocket_ipc.cpp
create mode 100644 test/serialization/ipa_data_serializer_test.cpp
create mode 100755 utils/ipc/generate.py
create mode 100644 utils/ipc/generators/libcamera_templates/meson.build
create mode 100644 utils/ipc/generators/libcamera_templates/module_generated.h.tmpl
create mode 100644 utils/ipc/generators/libcamera_templates/module_ipa_proxy.cpp.tmpl
create mode 100644 utils/ipc/generators/libcamera_templates/module_ipa_proxy.h.tmpl
create mode 100644 utils/ipc/generators/libcamera_templates/module_ipa_proxy_worker.cpp.tmpl
create mode 100644 utils/ipc/generators/libcamera_templates/module_serializer.h.tmpl
create mode 100644 utils/ipc/generators/libcamera_templates/proxy_functions.tmpl
create mode 100644 utils/ipc/generators/libcamera_templates/serializer.tmpl
create mode 100644 utils/ipc/generators/meson.build
create mode 100644 utils/ipc/generators/mojom_libcamera_generator.py
create mode 100644 utils/ipc/meson.build
create mode 100644 utils/ipc/mojo/public/LICENSE
create mode 100644 utils/ipc/mojo/public/tools/.style.yapf
create mode 100644 utils/ipc/mojo/public/tools/BUILD.gn
create mode 100644 utils/ipc/mojo/public/tools/bindings/BUILD.gn
create mode 100644 utils/ipc/mojo/public/tools/bindings/README.md
create mode 100644 utils/ipc/mojo/public/tools/bindings/chromium_bindings_configuration.gni
create mode 100644 utils/ipc/mojo/public/tools/bindings/compile_typescript.py
create mode 100755 utils/ipc/mojo/public/tools/bindings/concatenate-files.py
create mode 100755 utils/ipc/mojo/public/tools/bindings/concatenate_and_replace_closure_exports.py
create mode 100755 utils/ipc/mojo/public/tools/bindings/format_typemap_generator_args.py
create mode 100644 utils/ipc/mojo/public/tools/bindings/gen_data_files_list.py
create mode 100755 utils/ipc/mojo/public/tools/bindings/generate_type_mappings.py
create mode 100644 utils/ipc/mojo/public/tools/bindings/mojom.gni
create mode 100755 utils/ipc/mojo/public/tools/bindings/mojom_bindings_generator.py
create mode 100644 utils/ipc/mojo/public/tools/bindings/mojom_bindings_generator_unittest.py
create mode 100755 utils/ipc/mojo/public/tools/bindings/mojom_types_downgrader.py
create mode 100755 utils/ipc/mojo/public/tools/bindings/validate_typemap_config.py
create mode 100644 utils/ipc/mojo/public/tools/mojom/README.md
create mode 100755 utils/ipc/mojo/public/tools/mojom/check_stable_mojom_compatibility.py
create mode 100755 utils/ipc/mojo/public/tools/mojom/check_stable_mojom_compatibility_unittest.py
create mode 100644 utils/ipc/mojo/public/tools/mojom/const_unittest.py
create mode 100644 utils/ipc/mojo/public/tools/mojom/enum_unittest.py
create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/BUILD.gn
create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/__init__.py
create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/error.py
create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/fileutil.py
create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/fileutil_unittest.py
create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/__init__.py
create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/constant_resolver.py
create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/generator.py
create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/generator_unittest.py
create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/module.py
create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/module_unittest.py
create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/pack.py
create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/pack_unittest.py
create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/template_expander.py
create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/translate.py
create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/translate_unittest.py
create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/parse/__init__.py
create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/parse/ast.py
create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/parse/ast_unittest.py
create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/parse/conditional_features.py
create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/parse/conditional_features_unittest.py
create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/parse/lexer.py
create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/parse/lexer_unittest.py
create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/parse/parser.py
create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/parse/parser_unittest.py
create mode 100755 utils/ipc/mojo/public/tools/mojom/mojom_parser.py
create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom_parser_test_case.py
create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom_parser_unittest.py
create mode 100644 utils/ipc/mojo/public/tools/mojom/stable_attribute_unittest.py
create mode 100644 utils/ipc/mojo/public/tools/mojom/version_compatibility_unittest.py
create mode 100755 utils/ipc/mojo/public/tools/run_all_python_unittests.py
create mode 100644 utils/ipc/tools/diagnosis/crbug_1001171.py
--
2.27.0
More information about the libcamera-devel
mailing list