[libcamera-devel] [PATCH v5 0/4] Implement IPA algorithms and demo with IPU3

[Jean-Michel Hautbois]

Hi Jacopo,

On 21/04/2021 16:15, Jacopo Mondi wrote:
> Hello,
> 
> On Fri, Apr 16, 2021 at 09:49:05AM +0200, Jean-Michel Hautbois wrote:
>> Here is v5 of the IPA demonstration on IPU3.
>> It demonstrates a conjunction of automatic exposure (AGC)
>> and automatic white balance (AWB).
>>
>> The skeleton are extracted from RPi controller and probably subject to
>> modifications and discussions :
>> - are the mandatory methods really those ?
>> - how to exchange data between algorithms ?
>>   - metadata like RPi?
>>   - IPC ?
>>   - public methods?
>>   - controller API ?
>> - is the controller needed at all ?
>>
>> The implementation in IPU3 is tested on SGo2.
>> It is not perfect but shows a really better behaviour than default.
>> Right now, the AGC algorithm is part of the RPi one, adapted to the
>> stats and parameters we have in IPU3 on SGo2.
> 
> I've run CTS on master with this series applied on top (testing
> LIMITED mode then).
> 
> I have an additional test that fails
> - android.hardware.camera2.cts.AllocationTest#testAllocationFromCameraFlexibleYuv
> 
> It occasionally fails on master too.
> 
> The test tries to capture in all supported YUV formats, and check if
> the "input buffer" (in my understanding what would be sent to display
> if any surface was allocated) is not all 0s

One thing we are not doing right now, but would make sense, is dropping
frames while AGC has not converged. It would make the first frame ok for
the test, if everything is fine :-).

> It fails when testing the result from 320x240 and I'm tempted to postpone
> this to when 320x240 is produced by post-processing as we know the
> ImgU has not been validated for resolutions < VGA.
> 
> I would say
> Tested-by: Jacopo Mondi <jacopo at jmondi.org>
> 
> Thanks
>   j
> 
>> Changes in v5:
>> - adress multiple issues found at review time
>> - use RPi grey-world AWB algorithm and correct gains
>> - prepare for a more generic ISP stat usage
>>
>> Changes in v4:
>> - removed unneeded patch setting exposure and gains
>> - improve documentation of Histogram class
>> - take review comments
>>
>> Changes in v3:
>> - introduce histogram class (for AGC)
>> - add a patch to introduce parameter context saving
>> - changes in AGC to simplify it using part of RPi work
>> - changes in the AWB loop
>> - calculate the grid for the parameters using the BDS size from pipeline
>>   handler
>>
>> In v2:
>> - splits AWB and AEC/AGC algorithms.
>> - remove unused Controller
>> - remove unused variables/methods
>> - rename most of the class members
>>
>> Jean-Michel Hautbois (4):
>>   ipa: Add a common interface for algorithm objects
>>   ipa: ipu3: Add a histogram class
>>   ipa: ipu3: Add support for IPU3 AWB algorithm
>>   ipa: ipu3: Add support for IPU3 AEC/AGC algorithm
>>
>>  src/ipa/ipu3/ipu3.cpp        |  96 +++++++++-
>>  src/ipa/ipu3/ipu3_agc.cpp    | 206 ++++++++++++++++++++
>>  src/ipa/ipu3/ipu3_agc.h      |  62 +++++++
>>  src/ipa/ipu3/ipu3_awb.cpp    | 351 +++++++++++++++++++++++++++++++++++
>>  src/ipa/ipu3/ipu3_awb.h      |  91 +++++++++
>>  src/ipa/ipu3/meson.build     |   8 +-
>>  src/ipa/libipa/algorithm.cpp |  39 ++++
>>  src/ipa/libipa/algorithm.h   |  24 +++
>>  src/ipa/libipa/histogram.cpp | 150 +++++++++++++++
>>  src/ipa/libipa/histogram.h   |  40 ++++
>>  src/ipa/libipa/meson.build   |   6 +-
>>  11 files changed, 1063 insertions(+), 10 deletions(-)
>>  create mode 100644 src/ipa/ipu3/ipu3_agc.cpp
>>  create mode 100644 src/ipa/ipu3/ipu3_agc.h
>>  create mode 100644 src/ipa/ipu3/ipu3_awb.cpp
>>  create mode 100644 src/ipa/ipu3/ipu3_awb.h
>>  create mode 100644 src/ipa/libipa/algorithm.cpp
>>  create mode 100644 src/ipa/libipa/algorithm.h
>>  create mode 100644 src/ipa/libipa/histogram.cpp
>>  create mode 100644 src/ipa/libipa/histogram.h
>>
>> --
>> 2.27.0
>>
>> _______________________________________________
>> libcamera-devel mailing list
>> libcamera-devel at lists.libcamera.org
>> https://lists.libcamera.org/listinfo/libcamera-devel
> _______________________________________________
> libcamera-devel mailing list
> libcamera-devel at lists.libcamera.org
> https://lists.libcamera.org/listinfo/libcamera-devel
>