[libcamera-devel] [PATCH v2 2/4] ipa: rkisp1: Introduce Black Level Correction

[Laurent Pinchart]

Hi Jean-Michel,

Thank you for the patch.

On Wed, Feb 23, 2022 at 11:48:22AM +0100, Jean-Michel Hautbois wrote:
> In order to have the proper pixel levels, apply a fixed black level
> correction, based on the imx219 tuning file in RPi. The value is 4096 on
> 16 bits, and the pipeline for RkISP1 is on 12 bits, scale it.
> 
> Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois at ideasonboard.com>
> Tested-by: Peter Griffin <peter.griffin at linaro.org>
> ---
>  src/ipa/rkisp1/algorithms/blc.cpp     | 57 +++++++++++++++++++++++++++
>  src/ipa/rkisp1/algorithms/blc.h       | 30 ++++++++++++++
>  src/ipa/rkisp1/algorithms/meson.build |  1 +
>  src/ipa/rkisp1/rkisp1.cpp             |  2 +
>  4 files changed, 90 insertions(+)
>  create mode 100644 src/ipa/rkisp1/algorithms/blc.cpp
>  create mode 100644 src/ipa/rkisp1/algorithms/blc.h
> 
> diff --git a/src/ipa/rkisp1/algorithms/blc.cpp b/src/ipa/rkisp1/algorithms/blc.cpp
> new file mode 100644
> index 00000000..788953e3
> --- /dev/null
> +++ b/src/ipa/rkisp1/algorithms/blc.cpp
> @@ -0,0 +1,57 @@
> +/* SPDX-License-Identifier: LGPL-2.1-or-later */
> +/*
> + * Copyright (C) 2021, Ideas On Board
> + *
> + * blc.cpp - RkISP1 Black Level Correction control
> + */
> +
> +#include "blc.h"
> +
> +/**
> + * \file blc.h
> + */
> +
> +namespace libcamera {
> +
> +namespace ipa::rkisp1::algorithms {
> +
> +/**
> + * \class BlackLevelCorrection
> + * \brief RkISP1 Black Level Correction control
> + *
> + * The pixels output by the camera normally include a black level, because
> + * sensors do not always report a signal level of '0' for black. Pixels at or
> + * below this level should be considered black. To achieve that, the RkISP BLC
> + * algorithm subtracts a configurable offset from all pixels.
> + *
> + * The black level can be measured at runtime from an optical dark region of the
> + * camera sensor, or measured during the camera tuning process. The first option
> + * isn't currently supported.
> + */
> +
> +/**
> + * \copydoc libcamera::ipa::Algorithm::prepare
> + */
> +void BlackLevelCorrection::prepare(IPAContext &context,
> +				   rkisp1_params_cfg *params)
> +{
	if (context.frameContext.frameId != 0)
		return;

as there's no need to set params->others.bls_config.* either in that
case.

We may want to improve the Algorithm class interface to simplify
one-time settings, but that's OK for now.

There's a potential issue here for the future though. At the moment the
frameId starts from 0 and is sequential, as the pipeline handler doesn't
tie it to the sequence number provided by the driver. That may change
later, in order to inform the IPA about lost frames. If the first frame
is dropped, we'll then never run this. It would be nice to record this
issue somewhere.

There's another issue: this function is called when requests are queued,
while the frameId is stored in the context later, when statistics are
ready. The first few prepare() calls will have frameId uninitialized.

> +	/*
> +	 * Substract fixed values taken from imx219 tuning file.
> +	 * \todo Use a configuration file for it ?
> +	 */
> +	params->others.bls_config.enable_auto = 0;
> +	params->others.bls_config.fixed_val.r = 256;
> +	params->others.bls_config.fixed_val.gr = 256;
> +	params->others.bls_config.fixed_val.gb = 256;
> +	params->others.bls_config.fixed_val.b = 256;
> +
> +	if (context.frameContext.frameId == 0) {
> +		params->module_en_update |= RKISP1_CIF_ISP_MODULE_BLS;
> +		params->module_ens |= RKISP1_CIF_ISP_MODULE_BLS;
> +		params->module_cfg_update |= RKISP1_CIF_ISP_MODULE_BLS;
> +	}
> +}
> +
> +} /* namespace ipa::rkisp1::algorithms */
> +
> +} /* namespace libcamera */
> diff --git a/src/ipa/rkisp1/algorithms/blc.h b/src/ipa/rkisp1/algorithms/blc.h
> new file mode 100644
> index 00000000..331a2209
> --- /dev/null
> +++ b/src/ipa/rkisp1/algorithms/blc.h
> @@ -0,0 +1,30 @@
> +/* SPDX-License-Identifier: LGPL-2.1-or-later */
> +/*
> + * Copyright (C) 2021, Ideas On Board
> + *
> + * blc.h - RkISP1 Black Level Correction control
> + */
> +
> +#pragma once
> +
> +#include <linux/rkisp1-config.h>
> +
> +#include "algorithm.h"
> +
> +namespace libcamera {
> +
> +struct IPACameraSensorInfo;
> +
> +namespace ipa::rkisp1::algorithms {
> +
> +class BlackLevelCorrection : public Algorithm
> +{
> +public:
> +	BlackLevelCorrection() = default;
> +	~BlackLevelCorrection() = default;
> +
> +	void prepare(IPAContext &context, rkisp1_params_cfg *params) override;
> +};
> +
> +} /* namespace ipa::rkisp1::algorithms */
> +} /* namespace libcamera */
> diff --git a/src/ipa/rkisp1/algorithms/meson.build b/src/ipa/rkisp1/algorithms/meson.build
> index a19c1a4f..27c97731 100644
> --- a/src/ipa/rkisp1/algorithms/meson.build
> +++ b/src/ipa/rkisp1/algorithms/meson.build
> @@ -2,4 +2,5 @@
>  
>  rkisp1_ipa_algorithms = files([
>      'agc.cpp',
> +    'blc.cpp',
>  ])
> diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp
> index 732ca2bb..f82b7cb3 100644
> --- a/src/ipa/rkisp1/rkisp1.cpp
> +++ b/src/ipa/rkisp1/rkisp1.cpp
> @@ -27,6 +27,7 @@
>  
>  #include "algorithms/agc.h"
>  #include "algorithms/algorithm.h"
> +#include "algorithms/blc.h"
>  #include "libipa/camera_sensor_helper.h"
>  
>  #include "ipa_context.h"
> @@ -125,6 +126,7 @@ int IPARkISP1::init(const IPASettings &settings, unsigned int hwRevision)
>  
>  	/* Construct our Algorithms */
>  	algorithms_.push_back(std::make_unique<algorithms::Agc>());
> +	algorithms_.push_back(std::make_unique<algorithms::BlackLevelCorrection>());
>  
>  	return 0;
>  }

-- 
Regards,

Laurent Pinchart