[libcamera-devel] Intel IPU3 IPA module

[Kieran Bingham]

Initial release of the Intel IA Imaging based IPA for ChromeOS
--------------------------------------------------------------

A functional prototype of the Intel IPU3 IPA is available at the
following location:

    https://git.libcamera.org/libcamera/ipu3-ipa.git/

This release provides a demonstration of the integration of the Intel IA
Imaging libraries as an IPA module suitable for use with libcamera.

The IPA module is dependant upon the pre-installation of the
closed-source Intel imaging libraries and tuning data files, which are
not distributed here. They are packaged as part of ChromeOS, and as such
this IPA module has only been tested on ChromeOS.


Sources
-------

To test this release, the sources must be manually integrated to a
ChromeOS build.

Within your ChromeOS development environment, add the IPU3 IPA to your
source tree:

    (cr) $ cd trunk/src/third_party/
    (cr) $ git clone https://git.libcamera.org/libcamera/ipu3-ipa.git/

An overlay suitable for building the IPA is provided as part of the
source tree, which can be added to your chromiumos-overlay to use:

    (cr) $ cp -r ./ipu3-ipa/package/gentoo/media-libs/ipu3-ipa/ \
		./chromiumos-overlay/media-libs/
    (cr) $ cros-workon-${BOARD} start media-libs/ipu3-ipa


Build changes for libcamera
-----------------------------

There is currently no explicit mechanism for libcamera to choose which
IPA module will be used for a pipeline handler if there are multiple IPA
modules available for a given pipeline handler.

The existing behaviour is such that it will identify libraries from the
filesystem, and will match first based on alphabetical sort order, which
will default to the Intel IPA module; however this behaviour is not
specified and should not be relied upon.

Installing the Intel IPA module should be sufficient to use it, however
you may wish to prevent the build of the open-source IPU3 IPA module by
disabling the IPA's within the libcamera ebuild:

Apply the following change to

$cros/trunk/src/third_party/chromiumos-overlay/media-libs/libcamera/libcamera-9999.ebuild

    @@ -53,6 +52,7 @@ src_configure() {
                    $(meson_feature doc documentation)
                    -Dandroid="enabled"
                    -Dandroid_platform="cros"
    +               -Dipas=""
                    -Dpipelines="$(pipeline_list "${pipelines[@]}")"
                    --buildtype "$(usex debug debug plain)"
                    --sysconfdir /etc/camera


You may wish to ensure any existing IPA modules are removed from the target:

    (cr) ssh $REMOTE_IP -tC "rm -rf /usr/lib64/libcamera/*"

IPA Isolation
-------------

The Intel IPA is built externally, and as such is treated by default as
an untrusted IPA. This leads to IPA isolation which has not been fully
validated for performance on ChromeOS and may be undesirable.

For temporary integration as a non-isolated IPA, modify the libcamera
sources to disable isolation:

    (cr) $ cros-workon-${BOARD} start media-libs/libcamera

    diff --git a/src/libcamera/ipa_manager.cpp
b/src/libcamera/ipa_manager.cpp
    index b4606c6159e5..6b5f30a6b564 100644
    --- a/src/libcamera/ipa_manager.cpp
    +++ b/src/libcamera/ipa_manager.cpp
    @@ -258,6 +258,8 @@ unsigned int IPAManager::addDir(const char
*libDir, unsigned int maxDepth)

     bool IPAManager::isSignatureValid([[maybe_unused]] IPAModule *ipa)
const
     {
    +       return true;
    +
     #if HAVE_IPA_PUBKEY
            File file{ ipa->path() };
            if (!file.open(File::ReadOnly))




Build and deploy
----------------

Ensure that the libcamera package is built and up to date:

    (cr) $ USE="ipu3" emerge-${BOARD} media-libs/libcamera
    (cr) $ cros deploy $REMOTE_IP media-libs/libcamera

Then build and deploy the IPU3 IPA.

    (cr) $ emerge-${BOARD} media-libs/ipu3-ipa
    (cr) $ cros deploy $REMOTE_IP media-libs/ipu3-ipa


Testing
-------

This IPA module has been validated with the CTS test suite running the
libcamera subplan. The results match expectations for the current
development state, and are comparable to the test results on the Open
IPA for the Soraka Platform.

Image quality has been monitored using the Chrome Camera Application,
and video conferencing web services.


Known limitations
-----------------

- This IPA does not make use of all features and algorithms available in
  the Intel imaging libraries.
- The prototype does not have optimised control loops, and to prevent
  oscillations algorithms do not run for every frame.
- The GBCE algorithm does not run successfully on the Nautilus platform,
  leading to darker images.
- AWB FilterResponse configuration is explicitly disabled.
- Currently supported tuning data files are installed by the IPA
  package, and selected based upon the sensor name.