[libcamera-devel] [PATCH v3 10/12] ipa: ipu3: awb: Introduce Black Level Correction

[Laurent Pinchart]

Hi Jean-Michel,

Thank you for the patch.

On Wed, Oct 06, 2021 at 04:00:40PM +0200, Jean-Michel Hautbois wrote:
> The pixels output by the camera normally include a black level, because
> sensors do not always report a signal level of '0' for black. Pixels at
> or below this level should be considered black and to achieve that, we
> need to substract an offset to all the pixels. This can be taken into
> account by reading the lowest value of a special region on sensors which
> is not exposed to light. This provides a substracting factor to be
> able to adjust the expected black levels in the resulting images.
> 
> For a camera outputting 10-bit pixel values (in the range 0 to 1023) a
> typical black level might be 64. It is a fixed value, obtained by
> capturing a raw frame with minimum exposure and gain fixed to 1.0 while
> covering the sensor (the darker the better). We consider it good enough
> as a very first approximation, until we measure it during a tuning
> process and include it in a configuration file
> 
> Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois at ideasonboard.com>
> Reviewed-by: Kieran Bingham <kieran.bingham at ideasonboard.com>
> Reviewed-by: Laurent Pinchart <laurent.pinchart at ideasonboard.com>
> ---
>  src/ipa/ipu3/algorithms/blc.cpp     | 66 +++++++++++++++++++++++++++++
>  src/ipa/ipu3/algorithms/blc.h       | 28 ++++++++++++
>  src/ipa/ipu3/algorithms/meson.build |  1 +
>  src/ipa/ipu3/ipu3.cpp               |  2 +
>  4 files changed, 97 insertions(+)
>  create mode 100644 src/ipa/ipu3/algorithms/blc.cpp
>  create mode 100644 src/ipa/ipu3/algorithms/blc.h
> 
> diff --git a/src/ipa/ipu3/algorithms/blc.cpp b/src/ipa/ipu3/algorithms/blc.cpp
> new file mode 100644
> index 00000000..df56e292
> --- /dev/null
> +++ b/src/ipa/ipu3/algorithms/blc.cpp
> @@ -0,0 +1,66 @@
> +/* SPDX-License-Identifier: LGPL-2.1-or-later */
> +/*
> + * Copyright (C) 2021, Google inc.
> + *
> + * blc.cpp - IPU3 Black Level Correction control
> + */
> +
> +#include "blc.h"
> +
> +#include <string.h>
> +
> +/**
> + * \file blc.h

 * \brief IPU3 Black Level Correction control

> + */
> +
> +namespace libcamera {
> +
> +namespace ipa::ipu3::algorithms {
> +
> +/**
> + * \class BlackLevelCorrection
> + * \brief A class to handle optical black correction

s/optical black correction/black level correction/

> + *
> + * The pixels output by the camera normally include a black level, because
> + * sensors do not always report a signal level of '0' for black. Pixels at or
> + * below this level should be considered black. To achieve that, the ImgU BLC
> + * algorithm subtracts a configurable offset from all pixels.
> + *
> + * The black level can be measured at runtime from an optical dark region of the
> + * camera sensor, or measured during the camera tuning process. The first option
> + * isn't currently supported.
> + */
> +
> +BlackLevelCorrection::BlackLevelCorrection()
> +{
> +}
> +
> +/**
> + * \brief Fill in the parameter structure, and enable optical black correction

s/optical black correction/black level correction/

> + * \param context The shared IPA context
> + * \param params The IPU3 parameters
> + *
> + * Populate the IPU3 parameter structure with the correction values for each
> + * channel and enable the corresponding ImgU block processing.
> + */
> +void BlackLevelCorrection::prepare([[maybe_unused]] IPAContext &context,
> +			      ipu3_uapi_params *params)
> +{
> +	/*
> +	 * The Optical Black Level correction values
> +	 * \todo The correction values should come from sensor specific
> +	 * tuning processes. This is a first rough approximation

s/$/./

> +	 */
> +	params->obgrid_param.gr = 64;
> +	params->obgrid_param.r  = 64;
> +	params->obgrid_param.b  = 64;
> +	params->obgrid_param.gb = 64;
> +
> +	/* Enable the custom optical black correction processing */
> +	params->use.obgrid = 1;
> +	params->use.obgrid_param = 1;
> +}
> +
> +} /* namespace ipa::ipu3::algorithms */
> +
> +} /* namespace libcamera */
> diff --git a/src/ipa/ipu3/algorithms/blc.h b/src/ipa/ipu3/algorithms/blc.h
> new file mode 100644
> index 00000000..4b7cb483
> --- /dev/null
> +++ b/src/ipa/ipu3/algorithms/blc.h
> @@ -0,0 +1,28 @@
> +/* SPDX-License-Identifier: LGPL-2.1-or-later */
> +/*
> + * Copyright (C) 2021, Google inc.
> + *
> + * black_correction.h - IPU3 Black Level Correction control
> + */
> +#ifndef __LIBCAMERA_IPU3_ALGORITHMS_BLC_H__
> +#define __LIBCAMERA_IPU3_ALGORITHMS_BLC_H__
> +
> +#include "algorithm.h"
> +
> +namespace libcamera {
> +
> +namespace ipa::ipu3::algorithms {
> +
> +class BlackLevelCorrection : public Algorithm
> +{
> +public:
> +	BlackLevelCorrection();
> +
> +	void prepare(IPAContext &context, ipu3_uapi_params *params) override;
> +};
> +
> +} /* namespace ipa::ipu3::algorithms */
> +
> +} /* namespace libcamera */
> +
> +#endif /* __LIBCAMERA_IPU3_ALGORITHMS_BLC_H__ */
> diff --git a/src/ipa/ipu3/algorithms/meson.build b/src/ipa/ipu3/algorithms/meson.build
> index deae225b..3ec42f72 100644
> --- a/src/ipa/ipu3/algorithms/meson.build
> +++ b/src/ipa/ipu3/algorithms/meson.build
> @@ -4,5 +4,6 @@ ipu3_ipa_algorithms = files([
>      'agc.cpp',
>      'algorithm.cpp',
>      'awb.cpp',
> +    'blc.cpp',
>      'tone_mapping.cpp',
>  ])
> diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp
> index 06f53fbe..6d9bbf39 100644
> --- a/src/ipa/ipu3/ipu3.cpp
> +++ b/src/ipa/ipu3/ipu3.cpp
> @@ -33,6 +33,7 @@
>  #include "algorithms/agc.h"
>  #include "algorithms/algorithm.h"
>  #include "algorithms/awb.h"
> +#include "algorithms/blc.h"
>  #include "algorithms/tone_mapping.h"
>  #include "libipa/camera_sensor_helper.h"
>  
> @@ -282,6 +283,7 @@ int IPAIPU3::init(const IPASettings &settings,
>  	/* Construct our Algorithms */
>  	algorithms_.push_back(std::make_unique<algorithms::Agc>());
>  	algorithms_.push_back(std::make_unique<algorithms::Awb>());
> +	algorithms_.push_back(std::make_unique<algorithms::BlackLevelCorrection>());
>  	algorithms_.push_back(std::make_unique<algorithms::ToneMapping>());
>  
>  	return 0;

-- 
Regards,

Laurent Pinchart