[PATCH v2 3/8] ipa: libipa: Add ExposureModeHelper
Dan Scally
dan.scally at ideasonboard.com
Thu Apr 18 13:48:13 CEST 2024
Hi Paul
On 18/04/2024 12:35, Paul Elder wrote:
> Hi Dan,
>
> On Wed, Apr 17, 2024 at 02:15:31PM +0100, Daniel Scally wrote:
>> From: Paul Elder <paul.elder at ideasonboard.com>
>>
>> Add a helper for managing exposure modes and splitting exposure times
>> into shutter and gain values.
>>
>> Signed-off-by: Paul Elder <paul.elder at ideasonboard.com>
>> Signed-off-by: Daniel Scally <dan.scally at ideasonboard.com>
>> ---
>> Changes in v2:
>>
>> - Expanded the documentation
>> - Dropped the overloads for fixed shutter / gain - the same
>> functionality is instead done by setting min and max shutter and gain
>> to the same value
>> - Changed ::init() to consume a vector of pairs instead of two separate
>> vectors
>> - Reworked splitExposure()
>>
>> src/ipa/libipa/exposure_mode_helper.cpp | 257 ++++++++++++++++++++++++
>> src/ipa/libipa/exposure_mode_helper.h | 53 +++++
>> src/ipa/libipa/meson.build | 2 +
>> 3 files changed, 312 insertions(+)
>> create mode 100644 src/ipa/libipa/exposure_mode_helper.cpp
>> create mode 100644 src/ipa/libipa/exposure_mode_helper.h
>>
>> diff --git a/src/ipa/libipa/exposure_mode_helper.cpp b/src/ipa/libipa/exposure_mode_helper.cpp
>> new file mode 100644
>> index 00000000..6463de18
>> --- /dev/null
>> +++ b/src/ipa/libipa/exposure_mode_helper.cpp
>> @@ -0,0 +1,257 @@
>> +/* SPDX-License-Identifier: LGPL-2.1-or-later */
>> +/*
>> + * Copyright (C) 2024, Paul Elder <paul.elder at ideasonboard.com>
>> + *
>> + * exposure_mode_helper.cpp - Helper class that performs computations relating to exposure
>> + */
>> +#include "exposure_mode_helper.h"
>> +
>> +#include <algorithm>
>> +
>> +#include <libcamera/base/log.h>
>> +
>> +/**
>> + * \file exposure_mode_helper.h
>> + * \brief Helper class that performs computations relating to exposure
>> + *
>> + * AEGC algorithms have a need to split exposure between shutter and gain.
>> + * Multiple implementations do so based on paired sets of shutter and gain
>> + * limits; provide a helper to avoid duplicating the code.
>> + */
>> +
>> +namespace libcamera {
>> +
>> +LOG_DEFINE_CATEGORY(ExposureModeHelper)
>> +
>> +namespace ipa {
>> +
>> +/**
>> + * \class ExposureModeHelper
>> + * \brief Class for splitting exposure into shutter and gain
>> + *
>> + * The ExposureModeHelper class provides a standard interface through which an
>> + * AEGC algorithm can divide exposure between shutter and gain. It is configured
>> + * with a set of shutter and gain pairs and works by initially fixing gain at
>> + * 1.0 and increasing shutter up to the shutter value from the first pair in the
>> + * set in an attempt to meet the required exposure value.
>> + *
>> + * If the required exposure is not achievable by the first shutter value alone
>> + * it ramps gain up to the value from the first pair in the set. If the required
>> + * exposure is still not met it then allows shutter to ramp up to the shutter
>> + * value from the second pair in the set, and continues in this vein until
>> + * either the required exposure value is met, or else the hardware's shutter or
>> + * gain limits are reached.
>> + *
>> + * This method allows users to strike a balance between a well-exposed image and
>> + * an acceptable frame-rate, as opposed to simply maximising shutter followed by
>> + * gain. The same helpers can be used to perform the latter operation if needed
>> + * by passing an empty set of pairs to the initialisation function.
>> + *
>> + * The gain values may exceed a camera sensor's analogue gain limits if either
>> + * it or the IPA is also capable of digital gain. The configure() function must
>> + * be called with the hardware's limits to inform the helper of those
>> + * constraints. Any gain that is needed will be applied as analogue gain first
>> + * until the hardware's limit is reached, following which digital gain will be
>> + * used.
>> + */
>> +
>> +/**
>> + * \brief Construct an ExposureModeHelper instance
>> + */
>> +ExposureModeHelper::ExposureModeHelper()
>> + : minShutter_(0), maxShutter_(0), minGain_(0), maxGain_(0)
>> +{
>> +}
>> +
>> +/**
>> + * \brief Initialize an ExposureModeHelper instance
>> + * \param[in] stages The vector of paired shutter time and gain limits
>> + *
>> + * This function is expected to be called a single time once the algorithm that
>> + * is using these helpers has built the necessary list of shutter and gain pairs
>> + * to use (archetypically by parsing them from a tuning file during the
>> + * algorithm's .init() call).
>> + *
>> + * The input steps are shutter and _total_ gain pairs; the gain encompasses both
>> + * analogue and digital gain.
>> + *
>> + * The vector of stages may be empty. In that case, the helper will simply use
>> + * the runtime limits set through setShutterGainLimits() instead.
>> + */
>> +void ExposureModeHelper::init(const std::vector<std::pair<utils::Duration, double>> stages)
>> +{
>> + /* We only need to check shutters_, as gains_ is filled alongside it */
>> + if (!shutters_.empty()) {
>> + LOG(ExposureModeHelper, Warning)
>> + << "Initialization attempted multiple times";
>> + return;
>> + }
>> +
>> + for (auto stage : stages) {
>> + shutters_.push_back(stage.first);
>> + gains_.push_back(stage.second);
>> + }
>> +}
>> +
>> +/**
>> + * \brief Set the shutter and gain limits
>> + * \param[in] minShutter The minimum shutter time supported
>> + * \param[in] maxShutter The maximum shutter time supported
>> + * \param[in] minGain The minimum analogue gain supported
>> + * \param[in] maxGain The maximum analogue gain supported
>> + *
>> + * This function configures the shutter and analogue gain limits that need to be
>> + * adhered to as the helper divides up exposure. Note that these function *must*
> s/these/this/
>
>> + * be called whenever those limits change and before splitExposure() is used.
>> + *
>> + * If the algorithm using the helpers needs to indicate that either shutter,
>> + * analogue gain or both should be fixed it can do so by setting both the minima
>> + * and maxima to the same value.
>> + */
>> +void ExposureModeHelper::setShutterGainLimits(utils::Duration minShutter,
>> + utils::Duration maxShutter,
>> + double minGain,
>> + double maxGain)
>> +{
>> + minShutter_ = minShutter;
>> + maxShutter_ = maxShutter;
>> + minGain_ = minGain;
>> + maxGain_ = maxGain;
>> +}
>> +
>> +utils::Duration ExposureModeHelper::clampShutter(utils::Duration shutter) const
>> +{
>> + return std::clamp(shutter, minShutter_, maxShutter_);
>> +}
>> +
>> +double ExposureModeHelper::clampGain(double gain) const
>> +{
>> + return std::clamp(gain, minGain_, maxGain_);
>> +}
>> +
>> +/**
>> + * \brief Split exposure time into shutter and gain
>> + * \param[in] exposure Exposure time
>> + *
>> + * This function divides a given exposure value into shutter time, analogue and
>> + * digital gain by iterating through sets of shutter and gain limits. At each
>> + * stage the current stage's shutter limit is multiplied by the previous stage's
>> + * gain limit (or 1.0 initially) to see if the combination of the two can meet
>> + * the required exposure value. If they cannot then splitExpothe current stage's shutter
> typo "splitExpothe"?
Argh, I've actually got a whole section duplicated there!
>
>> + * limit is multiplied by the same stage's gain limit to see if that combination
>> + * can meet the required exposure value. If they cannot then the function moves
>> + * to consider the next stage.
>> + *
>> + * When a combination of shutter and gain _stage_ limits are found that are
>> + * sufficient to meet the required exposure value, the function attempts to
>> + * reduce shutter time as much as possible whilst fixing gain and still meeting
>> + * the exposure value. If a _runtime_ limit prevents shutter time from being
>> + * lowered enough to meet the exposure value with gain fixed at the stage limit,
>> + * gain is also lowered to compensate.
>> + *
>> + * Once the shutter time and gain values are ascertained, gain is assigned as
>> + * analogue gain as much as possible, with digital gain only in use if the
>> + * maximum analogue gain runtime limit is unable to accomodate the exposure
>> + * value.
>> + *
>> + * If no combination of shutter and gain limits is found that meets the required
>> + * exposure value, the helper falls-back to simply maximising the shutter time
>> + * first, followed by analogue gain, followed by digital gain.
>> + *
>> + * @return Tuple of shutter time, analogue gain, and digital gain
>> + */
>> +std::tuple<utils::Duration, double, double>
>> +ExposureModeHelper::splitExposure(utils::Duration exposure) const
>> +{
>> + ASSERT(maxShutter_);
>> + ASSERT(maxGain_);
>> +
>> + bool gainFixed = minGain_ == maxGain_;
>> + bool shutterFixed = minShutter_ == maxShutter_;
>> +
>> + /*
>> + * There's no point entering the loop if we cannot change either gain
>> + * nor shutter anyway.
>> + */
>> + if (shutterFixed && gainFixed)
>> + return { minShutter_, minGain_, exposure / (minShutter_ * minGain_) };
>> +
>> + utils::Duration shutter;
>> + double stageGain;
>> + double gain;
>> +
>> + for (unsigned int stage = 0; stage < gains_.size(); stage++) {
>> + double lastStageGain = stage == 0 ? 1.0 : clampGain(gains_[stage - 1]);
>> + utils::Duration stageShutter = clampShutter(shutters_[stage]);
>> + stageGain = clampGain(gains_[stage]);
>> +
>> + /*
>> + * We perform the clamping on both shutter and gain in case the
>> + * helper has had limits set that prevent those values being
>> + * lowered beyond a certain minimum...this can happen at runtime
>> + * for various reasons and so would not be known when the stage
>> + * limits are initialised.
>> + */
>> +
>> + if (stageShutter * lastStageGain >= exposure) {
>> + shutter = clampShutter(exposure / clampGain(lastStageGain));
>> + gain = clampGain(exposure / shutter);
>> +
>> + return { shutter, gain, exposure / (shutter * gain) };
>> + }
>> +
>> + if (stageShutter * stageGain >= exposure) {
>> + shutter = clampShutter(exposure / clampGain(stageGain));
>> + gain = clampGain(exposure / shutter);
>> +
>> + return { shutter, gain, exposure / (shutter * gain) };
>> + }
> This is really nice and clean but if I'm not mistaken you're ignoring
> fixed gain and fixed shutter...
That should just be accounted for by the fact that clampShutter() and clampGain() will fix the value
at that fixed shutter/gain - at least that's the idea. I tested by comparing the original
implementation to this one, including with fixed shutter/gain/both and the calculated shutter time
and gains matched.
>
> Paul
>
>> + }
>> +
>> + /*
>> + * From here on all we can do is max out the shutter, followed by the
>> + * analogue gain. If we still haven't achieved the target we send the
>> + * rest of the exposure time to digital gain. If we were given no stages
>> + * to use then set stageGain to 1.0 so that shutter is maxed before gain
>> + * touched at all.
>> + */
>> + if (gains_.empty())
>> + stageGain = 1.0;
>> +
>> + shutter = clampShutter(exposure / clampGain(stageGain));
>> + gain = clampGain(exposure / shutter);
>> +
>> + return { shutter, gain, exposure / (shutter * gain) };
>> +}
>> +
>> +/**
>> + * \fn ExposureModeHelper::minShutter()
>> + * \brief Retrieve the configured minimum shutter time limit set through
>> + * setShutterGainLimits()
>> + * \return The minShutter_ value
>> + */
>> +
>> +/**
>> + * \fn ExposureModeHelper::maxShutter()
>> + * \brief Retrieve the configured maximum shutter time set through
>> + * setShutterGainLimits()
>> + * \return The maxShutter_ value
>> + */
>> +
>> +/**
>> + * \fn ExposureModeHelper::minGain()
>> + * \brief Retrieve the configured minimum gain set through
>> + * setShutterGainLimits()
>> + * \return The minGain_ value
>> + */
>> +
>> +/**
>> + * \fn ExposureModeHelper::maxGain()
>> + * \brief Retrieve the configured maximum gain set through
>> + * setShutterGainLimits()
>> + * \return The maxGain_ value
>> + */
>> +
>> +} /* namespace ipa */
>> +
>> +} /* namespace libcamera */
>> diff --git a/src/ipa/libipa/exposure_mode_helper.h b/src/ipa/libipa/exposure_mode_helper.h
>> new file mode 100644
>> index 00000000..1f672135
>> --- /dev/null
>> +++ b/src/ipa/libipa/exposure_mode_helper.h
>> @@ -0,0 +1,53 @@
>> +/* SPDX-License-Identifier: LGPL-2.1-or-later */
>> +/*
>> + * Copyright (C) 2024, Paul Elder <paul.elder at ideasonboard.com>
>> + *
>> + * exposure_mode_helper.h - Helper class that performs computations relating to exposure
>> + */
>> +
>> +#pragma once
>> +
>> +#include <tuple>
>> +#include <vector>
>> +
>> +#include <libcamera/base/utils.h>
>> +
>> +namespace libcamera {
>> +
>> +namespace ipa {
>> +
>> +class ExposureModeHelper
>> +{
>> +public:
>> + ExposureModeHelper();
>> + ~ExposureModeHelper() = default;
>> +
>> + void init(const std::vector<std::pair<utils::Duration, double>> stages);
>> + void setShutterGainLimits(utils::Duration minShutter,
>> + utils::Duration maxShutter,
>> + double minGain, double maxGain);
>> +
>> + std::tuple<utils::Duration, double, double>
>> + splitExposure(utils::Duration exposure) const;
>> +
>> + utils::Duration minShutter() const { return minShutter_; }
>> + utils::Duration maxShutter() const { return maxShutter_; }
>> + double minGain() const { return minGain_; }
>> + double maxGain() const { return maxGain_; }
>> +
>> +private:
>> + utils::Duration clampShutter(utils::Duration shutter) const;
>> + double clampGain(double gain) const;
>> +
>> + std::vector<utils::Duration> shutters_;
>> + std::vector<double> gains_;
>> +
>> + utils::Duration minShutter_;
>> + utils::Duration maxShutter_;
>> + double minGain_;
>> + double maxGain_;
>> +};
>> +
>> +} /* namespace ipa */
>> +
>> +} /* namespace libcamera */
>> diff --git a/src/ipa/libipa/meson.build b/src/ipa/libipa/meson.build
>> index 016b8e0e..37fbd177 100644
>> --- a/src/ipa/libipa/meson.build
>> +++ b/src/ipa/libipa/meson.build
>> @@ -3,6 +3,7 @@
>> libipa_headers = files([
>> 'algorithm.h',
>> 'camera_sensor_helper.h',
>> + 'exposure_mode_helper.h',
>> 'fc_queue.h',
>> 'histogram.h',
>> 'module.h',
>> @@ -11,6 +12,7 @@ libipa_headers = files([
>> libipa_sources = files([
>> 'algorithm.cpp',
>> 'camera_sensor_helper.cpp',
>> + 'exposure_mode_helper.cpp',
>> 'fc_queue.cpp',
>> 'histogram.cpp',
>> 'module.cpp',
>> --
>> 2.34.1
>>
More information about the libcamera-devel
mailing list