[libcamera-devel] [PATCH v3 09/11] ipa: rkisp1: Introduce AGC
Jean-Michel Hautbois
jeanmichel.hautbois at ideasonboard.com
Wed Nov 24 10:58:59 CET 2021
On 24/11/2021 10:57, Jean-Michel Hautbois wrote:
> Hi Kieran
>
> On 23/11/2021 17:08, Kieran Bingham wrote:
>> Hi JM,
>>
>> Some question inline ...
>>
>> Quoting Jean-Michel Hautbois (2021-11-23 15:04:21)
>>> Now that we have IPAContext and Algorithm, we can implement a simple AGC
>>> based on the IPU3 one. It is very similar, except that there is no
>>> histogram used for an inter quantile mean. The RkISP1 is returning a 5x5
>>> array (for V10) of luminance means. Estimating the relative luminance is
>>> thus a simple mean of all the blocks already calculated by the ISP.
>>>
>>> Signed-off-by: Jean-Michel Hautbois
>>> <jeanmichel.hautbois at ideasonboard.com>
>>> ---
>>> src/ipa/rkisp1/algorithms/agc.cpp | 265 ++++++++++++++++++++++++++
>>> src/ipa/rkisp1/algorithms/agc.h | 55 ++++++
>>> src/ipa/rkisp1/algorithms/meson.build | 1 +
>>> src/ipa/rkisp1/ipa_context.cpp | 44 +++++
>>> src/ipa/rkisp1/ipa_context.h | 17 ++
>>> src/ipa/rkisp1/rkisp1.cpp | 72 +++----
>>> 6 files changed, 419 insertions(+), 35 deletions(-)
>>> create mode 100644 src/ipa/rkisp1/algorithms/agc.cpp
>>> create mode 100644 src/ipa/rkisp1/algorithms/agc.h
>>>
>>> diff --git a/src/ipa/rkisp1/algorithms/agc.cpp
>>> b/src/ipa/rkisp1/algorithms/agc.cpp
>>> new file mode 100644
>>> index 00000000..9c6d312e
>>> --- /dev/null
>>> +++ b/src/ipa/rkisp1/algorithms/agc.cpp
>>> @@ -0,0 +1,265 @@
>>> +/* SPDX-License-Identifier: LGPL-2.1-or-later */
>>> +/*
>>> + * Copyright (C) 2021, Ideas On Board
>>> + *
>>> + * agc.cpp - AGC/AEC mean-based control algorithm
>>> + */
>>> +
>>> +#include "agc.h"
>>> +
>>> +#include <algorithm>
>>> +#include <chrono>
>>> +#include <cmath>
>>> +
>>> +#include <libcamera/base/log.h>
>>> +
>>> +#include <libcamera/ipa/core_ipa_interface.h>
>>> +
>>> +/**
>>> + * \file agc.h
>>> + */
>>> +
>>> +namespace libcamera {
>>> +
>>> +using namespace std::literals::chrono_literals;
>>> +
>>> +namespace ipa::rkisp1::algorithms {
>>> +
>>> +/**
>>> + * \class Agc
>>> + * \brief A mean-based auto-exposure algorithm
>>> + */
>>> +
>>> +LOG_DEFINE_CATEGORY(RkISP1Agc)
>>> +
>>> +/* Limits for analogue gain values */
>>> +static constexpr double kMinAnalogueGain = 1.0;
>>> +static constexpr double kMaxAnalogueGain = 8.0;
>>> +
>>> +/* \todo Honour the FrameDurationLimits control instead of
>>> hardcoding a limit */
>>> +static constexpr utils::Duration kMaxShutterSpeed = 60ms;
>>> +
>>> +/* Number of frames to wait before calculating stats on minimum
>>> exposure */
>>> +static constexpr uint32_t kNumStartupFrames = 10;
>>> +
>>> +/*
>>> + * Relative luminance target.
>>> + *
>>> + * It's a number that's chosen so that, when the camera points at a
>>> grey
>>> + * target, the resulting image brightness is considered right.
>>> + */
>>> +static constexpr double kRelativeLuminanceTarget = 0.4;
>>> +
>>> +Agc::Agc()
>>> + : frameCount_(0), lineDuration_(0s), minShutterSpeed_(0s),
>>> + maxShutterSpeed_(0s), filteredExposure_(0s),
>>> currentExposure_(0s)
>>> +{
>>> +}
>>> +
>>> +/**
>>> + * \brief Configure the AGC given a configInfo
>>> + * \param[in] context The shared IPA context
>>> + * \param[in] configInfo The IPA configuration data
>>> + *
>>> + * \return 0
>>> + */
>>> +int Agc::configure(IPAContext &context, const IPACameraSensorInfo
>>> &configInfo)
>>> +{
>>> + /* \todo use the IPAContext to provide the limits */
>>> + lineDuration_ = configInfo.lineLength * 1.0s /
>>> configInfo.pixelRate;
>>> +
>>> + minShutterSpeed_ = context.configuration.agc.minShutterSpeed;
>>> + maxShutterSpeed_ =
>>> std::min(context.configuration.agc.maxShutterSpeed,
>>> + kMaxShutterSpeed);
>>> +
>>> + minAnalogueGain_ =
>>> std::max(context.configuration.agc.minAnalogueGain, kMinAnalogueGain);
>>> + maxAnalogueGain_ =
>>> std::min(context.configuration.agc.maxAnalogueGain, kMaxAnalogueGain);
>>> +
>>> + /* Configure the default exposure and gain. */
>>> + context.frameContext.agc.gain = minAnalogueGain_;
>>> + context.frameContext.agc.exposure = 10ms / lineDuration_;
>>> +
>>> + return 0;
>>> +}
>>> +
>>> +/**
>>> + * \brief Apply a filter on the exposure value to limit the speed of
>>> changes
>>> + */
>>> +void Agc::filterExposure()
>>> +{
>>> + double speed = 0.2;
>>> +
>>> + /* Adapt instantly if we are in startup phase */
>>> + if (frameCount_ < kNumStartupFrames)
>>> + speed = 1.0;
>>> +
>>> + if (filteredExposure_ == 0s) {
>>> + /* DG stands for digital gain.*/
>>
>> I think that comment is now stale.
>> If it's stale in the IPU3 as well, can you remove it there too please?
>>
>>> + filteredExposure_ = currentExposure_;
>>> + } else {
>>> + /*
>>> + * If we are close to the desired result, go faster
>>> to avoid making
>>
>> faster ? or slower?
>>
>> Oh - ok this checks out ;-) I almost got caught out forgetting that the
>> square root of a number less than one - increases towards one ;=)
>>
>>
>>> + * multiple micro-adjustments.
>>> + * \todo Make this customisable?
>>> + */
>>> + if (filteredExposure_ < 1.2 * currentExposure_ &&
>>> + filteredExposure_ > 0.8 * currentExposure_)
>>> + speed = sqrt(speed);
>>> +
>>> + filteredExposure_ = speed * currentExposure_ +
>>> + filteredExposure_ * (1.0 - speed);
>>> + }
>>> +
>>> + LOG(RkISP1Agc, Debug) << "After filtering, total_exposure "
>>> << filteredExposure_;
>>> +}
>>> +
>>> +/**
>>> + * \brief Estimate the new exposure and gain values
>>> + * \param[inout] frameContext The shared IPA frame Context
>>> + * \param[in] yGain The gain calculated on the current brightness level
>>> + */
>>> +void Agc::computeExposure(IPAFrameContext &frameContext, double yGain)
>>> +{
>>> + /* Get the effective exposure and gain applied on the sensor. */
>>> + uint32_t exposure = frameContext.sensor.exposure;
>>> + double analogueGain = frameContext.sensor.gain;
>>> +
>>> + /* Consider within 1% of the target as correctly exposed */
>>> + if (std::abs(yGain - 1.0) < 0.01)
>>> + LOG(RkISP1Agc, Debug) << "We are well exposed (iqMean
>>> = "
>>> + << yGain << ")";
>>> +
>>> + /* extracted from Rpi::Agc::computeTargetExposure */
>>> +
>>> + /* Calculate the shutter time in seconds */
>>> + utils::Duration currentShutter = exposure * lineDuration_;
>>> +
>>> + /*
>>> + * Update the exposure value for the next computation using
>>> the values
>>> + * of exposure and gain really used by the sensor.
>>> + */
>>> + utils::Duration effectiveExposureValue = currentShutter *
>>> analogueGain;
>>> +
>>> + LOG(RkISP1Agc, Debug) << "Actual total exposure " <<
>>> currentShutter * analogueGain
>>> + << " Shutter speed " << currentShutter
>>> + << " Gain " << analogueGain
>>> + << " Needed ev gain " << yGain;
>>> +
>>> + /*
>>> + * Calculate the current exposure value for the scene as the
>>> latest
>>> + * exposure value applied multiplied by the new estimated gain.
>>> + */
>>> + currentExposure_ = effectiveExposureValue * yGain;
>>> +
>>> + /* Clamp the exposure value to the min and max authorized */
>>> + utils::Duration maxTotalExposure = maxShutterSpeed_ *
>>> maxAnalogueGain_;
>>> + currentExposure_ = std::min(currentExposure_, maxTotalExposure);
>>> + LOG(RkISP1Agc, Debug) << "Target total exposure " <<
>>> currentExposure_
>>> + << ", maximum is " << maxTotalExposure;
>>> +
>>> + /* \todo: estimate if we need to desaturate */
>>> + filterExposure();
>>
>> Would returning the filtedExposure value from the function make it
>> clearer or more explicit than it gets used ?
>>
>>> +
>>> + /* Divide the exposure value as new exposure and gain values */
>>> + utils::Duration exposureValue = filteredExposure_;
>>
>> i.e.
>> utils::Duration exposureValue = filteredExposure();
>
> There is only one case which is anoying if we do that: initial value.
> When we start, we need to pass currentExposure_ and not the filtered
> value (this is the 'if (filteredExposure_ == 0s)' test).
Or, we can have:
utils::Duration Agc::filterExposure()
{
utils::Duration filteredExposure = currentExposure_;
...
return filteredExposure;
}
Which works fine.
More information about the libcamera-devel
mailing list