[libcamera-devel] [PATCH v3 09/11] ipa: rkisp1: Introduce AGC
Jean-Michel Hautbois
jeanmichel.hautbois at ideasonboard.com
Wed Nov 24 10:57:30 CET 2021
Hi Kieran
On 23/11/2021 17:08, Kieran Bingham wrote:
> Hi JM,
>
> Some question inline ...
>
> Quoting Jean-Michel Hautbois (2021-11-23 15:04:21)
>> Now that we have IPAContext and Algorithm, we can implement a simple AGC
>> based on the IPU3 one. It is very similar, except that there is no
>> histogram used for an inter quantile mean. The RkISP1 is returning a 5x5
>> array (for V10) of luminance means. Estimating the relative luminance is
>> thus a simple mean of all the blocks already calculated by the ISP.
>>
>> Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois at ideasonboard.com>
>> ---
>> src/ipa/rkisp1/algorithms/agc.cpp | 265 ++++++++++++++++++++++++++
>> src/ipa/rkisp1/algorithms/agc.h | 55 ++++++
>> src/ipa/rkisp1/algorithms/meson.build | 1 +
>> src/ipa/rkisp1/ipa_context.cpp | 44 +++++
>> src/ipa/rkisp1/ipa_context.h | 17 ++
>> src/ipa/rkisp1/rkisp1.cpp | 72 +++----
>> 6 files changed, 419 insertions(+), 35 deletions(-)
>> create mode 100644 src/ipa/rkisp1/algorithms/agc.cpp
>> create mode 100644 src/ipa/rkisp1/algorithms/agc.h
>>
>> diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp
>> new file mode 100644
>> index 00000000..9c6d312e
>> --- /dev/null
>> +++ b/src/ipa/rkisp1/algorithms/agc.cpp
>> @@ -0,0 +1,265 @@
>> +/* SPDX-License-Identifier: LGPL-2.1-or-later */
>> +/*
>> + * Copyright (C) 2021, Ideas On Board
>> + *
>> + * agc.cpp - AGC/AEC mean-based control algorithm
>> + */
>> +
>> +#include "agc.h"
>> +
>> +#include <algorithm>
>> +#include <chrono>
>> +#include <cmath>
>> +
>> +#include <libcamera/base/log.h>
>> +
>> +#include <libcamera/ipa/core_ipa_interface.h>
>> +
>> +/**
>> + * \file agc.h
>> + */
>> +
>> +namespace libcamera {
>> +
>> +using namespace std::literals::chrono_literals;
>> +
>> +namespace ipa::rkisp1::algorithms {
>> +
>> +/**
>> + * \class Agc
>> + * \brief A mean-based auto-exposure algorithm
>> + */
>> +
>> +LOG_DEFINE_CATEGORY(RkISP1Agc)
>> +
>> +/* Limits for analogue gain values */
>> +static constexpr double kMinAnalogueGain = 1.0;
>> +static constexpr double kMaxAnalogueGain = 8.0;
>> +
>> +/* \todo Honour the FrameDurationLimits control instead of hardcoding a limit */
>> +static constexpr utils::Duration kMaxShutterSpeed = 60ms;
>> +
>> +/* Number of frames to wait before calculating stats on minimum exposure */
>> +static constexpr uint32_t kNumStartupFrames = 10;
>> +
>> +/*
>> + * Relative luminance target.
>> + *
>> + * It's a number that's chosen so that, when the camera points at a grey
>> + * target, the resulting image brightness is considered right.
>> + */
>> +static constexpr double kRelativeLuminanceTarget = 0.4;
>> +
>> +Agc::Agc()
>> + : frameCount_(0), lineDuration_(0s), minShutterSpeed_(0s),
>> + maxShutterSpeed_(0s), filteredExposure_(0s), currentExposure_(0s)
>> +{
>> +}
>> +
>> +/**
>> + * \brief Configure the AGC given a configInfo
>> + * \param[in] context The shared IPA context
>> + * \param[in] configInfo The IPA configuration data
>> + *
>> + * \return 0
>> + */
>> +int Agc::configure(IPAContext &context, const IPACameraSensorInfo &configInfo)
>> +{
>> + /* \todo use the IPAContext to provide the limits */
>> + lineDuration_ = configInfo.lineLength * 1.0s / configInfo.pixelRate;
>> +
>> + minShutterSpeed_ = context.configuration.agc.minShutterSpeed;
>> + maxShutterSpeed_ = std::min(context.configuration.agc.maxShutterSpeed,
>> + kMaxShutterSpeed);
>> +
>> + minAnalogueGain_ = std::max(context.configuration.agc.minAnalogueGain, kMinAnalogueGain);
>> + maxAnalogueGain_ = std::min(context.configuration.agc.maxAnalogueGain, kMaxAnalogueGain);
>> +
>> + /* Configure the default exposure and gain. */
>> + context.frameContext.agc.gain = minAnalogueGain_;
>> + context.frameContext.agc.exposure = 10ms / lineDuration_;
>> +
>> + return 0;
>> +}
>> +
>> +/**
>> + * \brief Apply a filter on the exposure value to limit the speed of changes
>> + */
>> +void Agc::filterExposure()
>> +{
>> + double speed = 0.2;
>> +
>> + /* Adapt instantly if we are in startup phase */
>> + if (frameCount_ < kNumStartupFrames)
>> + speed = 1.0;
>> +
>> + if (filteredExposure_ == 0s) {
>> + /* DG stands for digital gain.*/
>
> I think that comment is now stale.
> If it's stale in the IPU3 as well, can you remove it there too please?
>
>> + filteredExposure_ = currentExposure_;
>> + } else {
>> + /*
>> + * If we are close to the desired result, go faster to avoid making
>
> faster ? or slower?
>
> Oh - ok this checks out ;-) I almost got caught out forgetting that the
> square root of a number less than one - increases towards one ;=)
>
>
>> + * multiple micro-adjustments.
>> + * \todo Make this customisable?
>> + */
>> + if (filteredExposure_ < 1.2 * currentExposure_ &&
>> + filteredExposure_ > 0.8 * currentExposure_)
>> + speed = sqrt(speed);
>> +
>> + filteredExposure_ = speed * currentExposure_ +
>> + filteredExposure_ * (1.0 - speed);
>> + }
>> +
>> + LOG(RkISP1Agc, Debug) << "After filtering, total_exposure " << filteredExposure_;
>> +}
>> +
>> +/**
>> + * \brief Estimate the new exposure and gain values
>> + * \param[inout] frameContext The shared IPA frame Context
>> + * \param[in] yGain The gain calculated on the current brightness level
>> + */
>> +void Agc::computeExposure(IPAFrameContext &frameContext, double yGain)
>> +{
>> + /* Get the effective exposure and gain applied on the sensor. */
>> + uint32_t exposure = frameContext.sensor.exposure;
>> + double analogueGain = frameContext.sensor.gain;
>> +
>> + /* Consider within 1% of the target as correctly exposed */
>> + if (std::abs(yGain - 1.0) < 0.01)
>> + LOG(RkISP1Agc, Debug) << "We are well exposed (iqMean = "
>> + << yGain << ")";
>> +
>> + /* extracted from Rpi::Agc::computeTargetExposure */
>> +
>> + /* Calculate the shutter time in seconds */
>> + utils::Duration currentShutter = exposure * lineDuration_;
>> +
>> + /*
>> + * Update the exposure value for the next computation using the values
>> + * of exposure and gain really used by the sensor.
>> + */
>> + utils::Duration effectiveExposureValue = currentShutter * analogueGain;
>> +
>> + LOG(RkISP1Agc, Debug) << "Actual total exposure " << currentShutter * analogueGain
>> + << " Shutter speed " << currentShutter
>> + << " Gain " << analogueGain
>> + << " Needed ev gain " << yGain;
>> +
>> + /*
>> + * Calculate the current exposure value for the scene as the latest
>> + * exposure value applied multiplied by the new estimated gain.
>> + */
>> + currentExposure_ = effectiveExposureValue * yGain;
>> +
>> + /* Clamp the exposure value to the min and max authorized */
>> + utils::Duration maxTotalExposure = maxShutterSpeed_ * maxAnalogueGain_;
>> + currentExposure_ = std::min(currentExposure_, maxTotalExposure);
>> + LOG(RkISP1Agc, Debug) << "Target total exposure " << currentExposure_
>> + << ", maximum is " << maxTotalExposure;
>> +
>> + /* \todo: estimate if we need to desaturate */
>> + filterExposure();
>
> Would returning the filtedExposure value from the function make it
> clearer or more explicit than it gets used ?
>
>> +
>> + /* Divide the exposure value as new exposure and gain values */
>> + utils::Duration exposureValue = filteredExposure_;
>
> i.e.
> utils::Duration exposureValue = filteredExposure();
There is only one case which is anoying if we do that: initial value.
When we start, we need to pass currentExposure_ and not the filtered
value (this is the 'if (filteredExposure_ == 0s)' test).
More information about the libcamera-devel
mailing list