[libcamera-devel] [PATCH 3/8] libcamera: ipa_module_info: update struct to allow IPA matching
Laurent Pinchart
laurent.pinchart at ideasonboard.com
Tue Jun 4 11:53:14 CEST 2019
Hi Niklas,
On Tue, Jun 04, 2019 at 01:08:48AM +0200, Niklas Söderlund wrote:
> On 2019-05-27 18:35:35 -0400, Paul Elder wrote:
> > We need a way to match pipelines with IPA modules, so add fields in
> > IPAModuleInfo to hold the IPA API version number, the pipeline name, and
> > the pipeline major and minor version.
>
> I'm not sure I like that the IPA API have a version number, I know now
> you think I'm crazy but please hear me out ;-)
>
> If we put API version numbers in both the PipelineHandler's and IPA's
> that means that libcamera will need some logic to match the API version
> requested by a pipeline handler to the version number reported by the
> IPA. On the surface this seems sane, but then we enter the fun world of
> compatibility and 3rd party closed source IPA's.
>
> - Case 1 - Open source IPA shipped with libcamera
>
> Here we have no issue as both PipelineHanlder and IPA lives in the
> same source tree and are build and distributed from the same
> sources.
>
> Recording API versions in both PipelineHandler and IPA make sens. In
> this case the version could be equal to the libcamera version.
>
> - Case 2 - Closed source IPA shipped as 3rd party component
>
> If anything have changed in the API between pipeline handlers and
> IPA's between libcamera version A and B they are no longer
> compatible. This change can be a pipe <-> IPA specific API change or
> a change in any of the libcamera common objects which are
> sent/received between the pipe and IPA.
>
> For example a new control is added which potentially could change
> the size of libcamera::Request::Controls (theoretical object).
>
> Now we the 3rd party vendor have a few options on how to handle
> this. The ones I can think of on the top of my head is:
>
> 1. Rebuild and distribute a binary IPA for every new released
> version of libcamera.
> 2. Try to upstream some compatibility conversion to the pipeline
> handler which uses "old" libcamera objects.
> 3. Try to convince libcamera to provide a versioned LTS API between
> pipeline and IPA (both for pipe<->IPA and objects).
>
> All of the solutions for closed source IPAs have drawbacks, either from
> a IPA provider point of view (1) or for libcamera (2, 3). I totally
> agree that we should not change any libcamera objects involved in the
> IPA without a good reason, but we should not be blocked form solving
> problems in libcamera due to closed source components.
>
> There fore I propose we think about a solution where all compatibility
> conversions and decisions are pushed to the IPA implementations. The IPA
> would be initialized with two version numbers when the so is loaded.
>
> - A libcamera API version which control the version of all common
> libcamera objects which can be transmitted/received to/from a IPA.
> - A pipeline API version which control the API of which the pipeline
> expects to interact with the IPA.
The libcamera API version number stored in the info structure describes
the layout of that structure. It can't be passed by libcamera to the
IPA, as the structure is read by libcamera before the IPA init()
function is called. I thus think we need to keep that version number, in
order to interpret the content of the structure.
> Then it's up to the IPA to accept if it can function with these two
> versions, or not. Then if a IPA implementation wish to function whit
> both a newer and older version of one of these components it can
> implement it's own compatibility layer. And we can still move forward
> with libcamera core API and open source IPA in the libcamera tree
> without needing a huge verification program for 3rd party IPAs.
It's not a bad idea, but it won't allow an old IPA to work with a newer
version of libcamera, which is the main issue we're trying to solve with
versioning. I expect IPAs to be updated way less frequently than
libcamera.
> Once more; I'm not advocating an API which is changed lightly as we
> really should do our most to make IPA designers life easy. But if we
> need to do so to fix bugs in libcamera we should not be blocked by
> anything out of tree.
>
> I don't feel strongly about this topic but thought it worth bringing up
> when we design this. So feel free to ignore me ;-)
>
> >
> > Also update IPA module tests and Doxygen accordingly. Doxygen needs to
> > be updated to accomodate __attribute__((packed)).
> >
> > Signed-off-by: Paul Elder <paul.elder at ideasonboard.com>
> > ---
> > Documentation/Doxyfile.in | 7 ++--
> > include/libcamera/ipa/ipa_module_info.h | 11 ++++--
> > src/libcamera/ipa_module.cpp | 48 +++++++++++++++++++++----
> > test/ipa/ipa_test.cpp | 45 ++++++++++++++++++-----
> > test/ipa/shared_test.cpp | 4 ++-
> > 5 files changed, 94 insertions(+), 21 deletions(-)
> >
> > diff --git a/Documentation/Doxyfile.in b/Documentation/Doxyfile.in
> > index 67eaded..ac70efb 100644
> > --- a/Documentation/Doxyfile.in
> > +++ b/Documentation/Doxyfile.in
> > @@ -2016,7 +2016,7 @@ ENABLE_PREPROCESSING = YES
> > # The default value is: NO.
> > # This tag requires that the tag ENABLE_PREPROCESSING is set to YES.
> >
> > -MACRO_EXPANSION = NO
> > +MACRO_EXPANSION = YES
> >
> > # If the EXPAND_ONLY_PREDEF and MACRO_EXPANSION tags are both set to YES then
> > # the macro expansion is limited to the macros specified with the PREDEFINED and
> > @@ -2024,7 +2024,7 @@ MACRO_EXPANSION = NO
> > # The default value is: NO.
> > # This tag requires that the tag ENABLE_PREPROCESSING is set to YES.
> >
> > -EXPAND_ONLY_PREDEF = NO
> > +EXPAND_ONLY_PREDEF = YES
> >
> > # If the SEARCH_INCLUDES tag is set to YES, the include files in the
> > # INCLUDE_PATH will be searched if a #include is found.
> > @@ -2057,7 +2057,8 @@ INCLUDE_FILE_PATTERNS = *.h
> > # This tag requires that the tag ENABLE_PREPROCESSING is set to YES.
> >
> > PREDEFINED = __DOXYGEN__ \
> > - __cplusplus
> > + __cplusplus \
> > + __attribute__(x)=
> >
> > # If the MACRO_EXPANSION and EXPAND_ONLY_PREDEF tags are set to YES then this
> > # tag can be used to specify a list of macro names that should be expanded. The
> > diff --git a/include/libcamera/ipa/ipa_module_info.h b/include/libcamera/ipa/ipa_module_info.h
> > index 4e0d668..84492ed 100644
> > --- a/include/libcamera/ipa/ipa_module_info.h
> > +++ b/include/libcamera/ipa/ipa_module_info.h
> > @@ -7,14 +7,21 @@
> > #ifndef __LIBCAMERA_IPA_MODULE_INFO_H__
> > #define __LIBCAMERA_IPA_MODULE_INFO_H__
> >
> > +#include <stdint.h>
> > +
> > +#define IPA_MODULE_API_VERSION 1
> > +
> > #ifdef __cplusplus
> > namespace libcamera {
> > #endif
> >
> > struct IPAModuleInfo {
> > + int moduleAPIVersion;
> > + int pipelineVersionMajor;
> > + int pipelineVersionMinor;
> > + char pipelineName[256];
> > char name[256];
> > - unsigned int version;
> > -};
> > +} __attribute__((packed));
> >
> > #ifdef __cplusplus
> > extern "C" {
> > diff --git a/src/libcamera/ipa_module.cpp b/src/libcamera/ipa_module.cpp
> > index 86cbe71..6e68934 100644
> > --- a/src/libcamera/ipa_module.cpp
> > +++ b/src/libcamera/ipa_module.cpp
> > @@ -169,6 +169,17 @@ int elfLoadSymbol(void *dst, size_t size, void *map, size_t soSize,
> >
> > } /* namespace */
> >
> > +/**
> > + * \def IPA_MODULE_API_VERSION
> > + * \brief The IPA API version
> > + *
> > + * This version number specifies the version for the layout of
> > + * struct IPAModuleInfo, and the API between pipeline handlers and the IPA.
> > + * The IPA module should use this macro to define its moduleAPIVersion field.
> > + *
> > + * \sa libcamera::IPAModuleInfo::moduleAPIVersion
> > + */
> > +
> > /**
> > * \struct IPAModuleInfo
> > * \brief Information of an IPA module
> > @@ -176,14 +187,39 @@ int elfLoadSymbol(void *dst, size_t size, void *map, size_t soSize,
> > * This structure contains the information of an IPA module. It is loaded,
> > * read, and validated before anything else is loaded from the shared object.
> > *
> > - * \var IPAModuleInfo::name
> > - * \brief The name of the IPA module
> > + * \var IPAModuleInfo::moduleAPIVersion
> > + * \brief The IPA API version that the IPA module was made with
> > + *
> > + * This version number specifies the version for the layout of
> > + * struct IPAModuleInfo, and the API between pipeline handlers and the IPA.
> > + * The IPA module shall report here the version that it was built for,
> > + * using the macro IPA_MODULE_API_VERSION.
> > *
> > - * \var IPAModuleInfo::version
> > - * \brief The version of the IPA module
> > + * \sa IPA_MODULE_API_VERSION
> > *
> > - * \todo abi compatability version
> > - * \todo pipeline compatability matcher
> > + * \var IPAModuleInfo::pipelineVersionMajor
> > + * \brief The pipeline handler version (major) that the IPA module is for
> > + * \todo actually match the pipeline handler against this
> > + *
> > + * The major version of the module and pipeline handler must be equal in
> > + * order for the pipeline handler and the module to be matched.
> > + *
> > + * \var IPAModuleInfo::pipelineVersionMinor
> > + * \brief The pipeline handler version (minor) that the IPA module is for
> > + *
> > + * The minor version of the module must be equal to or greater than
> > + * the pipeline handler minor version in order for the pipeline handler
> > + * and the module to be matched.
> > + *
> > + * \var IPAModuleInfo::pipelineName
> > + * \brief The name of the pipeline handler that the IPA module is for
> > + *
> > + * This name is used to match a pipeline handler with the module.
> > + * The name of the pipeline handler is declared in the pipeline handler
> > + * with the macro REGISTER_PIPELINE_HANDLER.
> > + *
> > + * \var IPAModuleInfo::name
> > + * \brief The name of the IPA module
> > */
> >
> > /**
> > diff --git a/test/ipa/ipa_test.cpp b/test/ipa/ipa_test.cpp
> > index 2dbc702..f50880e 100644
> > --- a/test/ipa/ipa_test.cpp
> > +++ b/test/ipa/ipa_test.cpp
> > @@ -33,20 +33,45 @@ protected:
> >
> > const struct IPAModuleInfo &info = ll->info();
> >
> > - if (strcmp(info.name, testInfo.name)) {
> > - cerr << "test IPA module has incorrect name" << endl;
> > - cerr << "expected \"" << testInfo.name << "\", got \""
> > - << info.name << "\"" << endl;
> > + stringstream errString;
> > +
> > + if (info.moduleAPIVersion != testInfo.moduleAPIVersion) {
> > + errString << "incorrect name: expected \""
> > + << testInfo.moduleAPIVersion << "\", got \""
> > + << info.moduleAPIVersion << "\"" << endl;
> > ret = -1;
> > }
> >
> > - if (info.version != testInfo.version) {
> > - cerr << "test IPA module has incorrect version" << endl;
> > - cerr << "expected \"" << testInfo.version << "\", got \""
> > - << info.version << "\"" << endl;
> > + if (info.pipelineVersionMajor != testInfo.pipelineVersionMajor) {
> > + errString << "incorrect name: expected \""
> > + << testInfo.pipelineVersionMajor << "\", got \""
> > + << info.pipelineVersionMajor << "\"" << endl;
> > ret = -1;
> > }
> >
> > + if (info.pipelineVersionMinor != testInfo.pipelineVersionMinor) {
> > + errString << "incorrect name: expected \""
> > + << testInfo.pipelineVersionMinor << "\", got \""
> > + << info.pipelineVersionMinor << "\"" << endl;
> > + ret = -1;
> > + }
> > +
> > + if (strcmp(info.pipelineName, testInfo.pipelineName)) {
> > + errString << "incorrect name: expected \""
> > + << testInfo.pipelineName << "\", got \""
> > + << info.pipelineName << "\"" << endl;
> > + ret = -1;
> > + }
> > +
> > + if (strcmp(info.name, testInfo.name)) {
> > + errString << "incorrect name: expected \""
> > + << testInfo.name << "\", got \""
> > + << info.name << "\"" << endl;
> > + ret = -1;
> > + }
> > +
> > + cerr << errString.str();
> > +
> > delete ll;
> > return ret;
> > }
> > @@ -56,8 +81,10 @@ protected:
> > int count = 0;
> >
> > const struct IPAModuleInfo testInfo = {
> > + 1,
> > + 0, 9001,
> > + "bleep",
> > "It's over nine thousand!",
> > - 9001,
> > };
> >
> > count += runTest("test/ipa/ipa-dummy-cpp.so", testInfo);
> > diff --git a/test/ipa/shared_test.cpp b/test/ipa/shared_test.cpp
> > index 4e5c976..d932a9b 100644
> > --- a/test/ipa/shared_test.cpp
> > +++ b/test/ipa/shared_test.cpp
> > @@ -4,8 +4,10 @@ namespace libcamera {
> >
> > extern "C" {
> > const struct libcamera::IPAModuleInfo ipaModuleInfo = {
> > + 1,
> > + 0, 9001,
> > + "bleep",
> > "It's over nine thousand!",
> > - 9001,
> > };
> > };
> >
> > --
> > 2.20.1
> >
> > _______________________________________________
> > libcamera-devel mailing list
> > libcamera-devel at lists.libcamera.org
> > https://lists.libcamera.org/listinfo/libcamera-devel
>
> --
> Regards,
> Niklas Söderlund
> _______________________________________________
> libcamera-devel mailing list
> libcamera-devel at lists.libcamera.org
> https://lists.libcamera.org/listinfo/libcamera-devel
--
Regards,
Laurent Pinchart
More information about the libcamera-devel
mailing list