[libcamera-devel] [PATCH v3 01/19] ipa: ipu3: Document IPAIPU3 class interface

[Kieran Bingham]

Quoting Jean-Michel Hautbois (2021-10-22 16:12:00)
> The IPU3 IPA is maturing to a modular and extensible system capable of
> handling specific algorithms for the processing blocks on the ImgU.
> 
> Provide a top-level class documentation to provide an overview of the
> IPA, detailing what events are used and what algorithms are currently
> supported, as well as the limitations currently imposed.
> 
> Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois at ideasonboard.com>
> Signed-off-by: Kieran Bingham <kieran.bingham at ideasonboard.com>
> ---
>  src/ipa/ipu3/ipu3.cpp | 64 +++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 64 insertions(+)
> 
> diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp
> index 65d3fd2c..b5438ed1 100644
> --- a/src/ipa/ipu3/ipu3.cpp
> +++ b/src/ipa/ipu3/ipu3.cpp
> @@ -179,6 +179,70 @@ using namespace std::literals::chrono_literals;
>  
>  namespace ipa::ipu3 {
>  
> +/**
> + * \brief The IPU3 IPA implementation
> + *
> + * The IPU3 Pipeline defines an IPU3-specific interface for communication
> + * between the PipelineHandler and the IPA module.
> + *
> + * We extend the IPAIPU3Interface to implement our algorithms and handle events
> + * from the IPU3 PipelineHandler to satisfy requests from the application.
> + *
> + * At initialisation time, a CameraSensorHelper is instantiated to support
> + * camera-specific calculations, while the default controls are computed, and
> + * the algorithms are constructed and placed in an ordered list.
> + *
> + * The IPU3 ImgU operates with a grid layout to divide the overall frame into
> + * rectangular cells of pixels. When the IPA is configured, we determine the
> + * best grid for the statistics based on the pipeline handler Bayer Down Scaler
> + * output size.
> + *
> + * Two main events are then handled to operate the IPU3 ImgU by populating its
> + * parameter buffer, and adapting the settings of the sensor attached to the
> + * IPU3 CIO2 through sensor-specific V4L2 controls.
> + *
> + * When the event \a EventFillParams occurs we populate the ImgU parameter
> + * buffer with settings to configure the device in preparation for handling the
> + * frame queued in the Request.
> + *
> + * When the frame has completed processing, the ImgU will generate a statistics
> + * buffer which is given to the IPA as part of the \a EventStatReady event. At
> + * this event we run the algorithms to parse the statistics and cache any
> + * results for the next \a EventFillParams event.
> + *
> + * The individual algorithms are split into modular components that are called
> + * iteratively to allow them to process statistics from the ImgU in a defined
> + * order.
> + *
> + * The current implementation supports three core algorithms:
> + * - Automatic white balance (AWB)
> + * - Automatic gain and exposure control (AGC)
> + * - Tone mapping (Gamma)

Has this been extended with BLC now? Or is it not worth mentioning that
explicitly?

> + *
> + * AWB is implemented using a Greyworld algorithm, and calculates the red and
> + * blue gains to apply to generate a neutral grey frame overall.
> + *
> + * AGC is handled by calculating a histogram of the green channel to estimate an
> + * analogue gain and shutter time which will provide a well exposed frame. A
> + * low-pass IIR filter is used to smooth the changes to the sensor to reduce
> + * perceivable steps.
> + *
> + * The Tone mapping algorithm provides a gamma correction table to improve the
> + * contrast of the scene.
> + *
> + * The IPU3 ImgU has further processing blocks to support image quality
> + * improvements through bayer and temporal noise reductions, however those are
> + * not supported in the current implementation, and will use default settings as
> + * provided by the kernel driver.
> + *
> + * Demosaicing is operating with the default parameters and could be further
> + * optimised to provide improved sharpening coefficients, checker artifact
> + * removal, and false color correction.
> + *
> + * Additional image enhancements can be made by providing lens and
> + * sensor-specific tuning to adapt for Black Level compensation (BLC), Lens

And perhaps check if BLC needs removing from here. If BLC still needs
more wrok, then perhaps it should stay.

Reviewed-by: Kieran Bingham <kieran.bingham at ideasonboard.com>

> + * shading correction (SHD) and Color correction (CCM).
> + */
>  class IPAIPU3 : public IPAIPU3Interface
>  {
>  public:
> -- 
> 2.32.0
>