[libcamera-devel] [PATCH v4 1/3] controls: Reorganize the AE-related controls
Paul Elder
paul.elder at ideasonboard.com
Wed May 18 15:47:26 CEST 2022
We have multiple goals:
- we need a lock of some sort, to instruct the AEGC to not update output
results
- we need manual modes, to override the values computed by the AEGC
- we need to support seamless transitions from auto -> manual, and do so
without flickering
- we need custom minimum values for the manual controls, that is no
magic values for enabling/disabling auto
- all of these need to be done with AE sub-controls (exposure time,
analogue gain)
To achieve these goals, we introduce mode controls for the AE
sub-controls: ExposureTimeMode and AnalogueGainMode. These have an auto
state, and a disabled state. The disabled state has an internal one-way
state change from locked to manual, triggered by the presence of the
value-controls (ExposureTime and AnalogueGain).
We then remove the AeEnable control, as it is a redundant control in the
face of these two mode controls.
We also remove AeLocked, as it is insufficient for reporting the AE
state, and we promote AeState to non-draft to fill its role. Notably,
the locked state is removed, since this information can be obtained from
the aforementioned mode controls.
Bug: https://bugs.libcamera.org/show_bug.cgi?id=42
Bug: https://bugs.libcamera.org/show_bug.cgi?id=43
Bug: https://bugs.libcamera.org/show_bug.cgi?id=47
Signed-off-by: Paul Elder <paul.elder at ideasonboard.com>
---
Changes in v4:
- remove FlashRequired and Precapture from AeState
- upgrade documentation of all the controls
Changes in v3:
- improve wording of the control descriptions
- make more succinct and clear
- add description of how to do a flickerless transition
Changes in v2:
- No changes, just resubmitting at the head of this series so that it's
together and so that /people will actually see it/
Initial version:
Still RFC as I haven't updated the users of the control yet, and I want
to check that these are the controls and docs that we want.
We've decided that the "master AE control" will be implemented by a
helper... but looking at uvcvideo and the V4L2 controls I'm wondering if
such helper should come earlier than later?
---
src/libcamera/control_ids.yaml | 262 +++++++++++++++++++++++++--------
1 file changed, 200 insertions(+), 62 deletions(-)
diff --git a/src/libcamera/control_ids.yaml b/src/libcamera/control_ids.yaml
index 9d4638ae..9f5ce5e8 100644
--- a/src/libcamera/control_ids.yaml
+++ b/src/libcamera/control_ids.yaml
@@ -7,23 +7,46 @@
# Unless otherwise stated, all controls are bi-directional, i.e. they can be
# set through Request::controls() and returned out through Request::metadata().
controls:
- - AeEnable:
- type: bool
+ - AeState:
+ type: int32_t
description: |
- Enable or disable the AE.
+ Control to report the AE algorithm state associated with the capture
+ result.
- \sa ExposureTime AnalogueGain
+ The state is still reported even if ExposureTimeMode or
+ AnalogueGainMode is set to Disabled.
- - AeLocked:
- type: bool
- description: |
- Report the lock status of a running AE algorithm.
+ \sa AnalogueGain
+ \sa AnalogueGainMode
+ \sa ExposureTime
+ \sa ExposureTimeMode
- If the AE algorithm is locked the value shall be set to true, if it's
- converging it shall be set to false. If the AE algorithm is not
- running the control shall not be present in the metadata control list.
+ enum:
+ - name: AeStateInactive
+ value: 0
+ description: |
+ The AE algorithm is inactive.
- \sa AeEnable
+ This state should be returned if both AnalogueGainMode and
+ ExposureTimeMode are set to disabled (or one, if the camera only
+ supports one of the two controls).
+ - name: AeStateSearching
+ value: 1
+ description: |
+ The AE algorithm has not converged yet.
+
+ This state should be returned if at least one of AnalogueGainMode
+ or ExposureTimeMode is set to auto, and the AE algorithm hasn't
+ converged yet. If the AE algorithm converges, the state shall go to
+ AeStateConverged.
+ - name: AeStateConverged
+ value: 2
+ description: |
+ The AE algorithm has converged.
+
+ This state should be returned if at least one of AnalogueGainMode
+ or ExposureTimeMode is set to auto, and the AE algorithm has
+ converged.
# AeMeteringMode needs further attention:
# - Auto-generate max enum value.
@@ -93,6 +116,13 @@ controls:
how the desired total exposure is divided between the shutter time
and the sensor's analogue gain. The exposure modes are platform
specific, and not all exposure modes may be supported.
+
+ When one of AnalogueGainMode or ExposureTimeMode is set to Disabled,
+ the fixed values will override any choices made by AeExposureMode.
+
+ \sa AnalogueGainMode
+ \sa ExposureTimeMode
+
enum:
- name: ExposureNormal
value: 0
@@ -111,13 +141,15 @@ controls:
type: float
description: |
Specify an Exposure Value (EV) parameter. The EV parameter will only be
- applied if the AE algorithm is currently enabled.
+ applied if the AE algorithm is currently enabled, that is, at least one
+ of AnalogueGainMode and ExposureTimeMode are auto.
By convention EV adjusts the exposure as log2. For example
EV = [-2, -1, 0.5, 0, 0.5, 1, 2] results in an exposure adjustment
of [1/4x, 1/2x, 1/sqrt(2)x, 1x, sqrt(2)x, 2x, 4x].
- \sa AeEnable
+ \sa AnalogueGainMode
+ \sa ExposureTimeMode
- ExposureTime:
type: int32_t
@@ -125,17 +157,85 @@ controls:
Exposure time (shutter speed) for the frame applied in the sensor
device. This value is specified in micro-seconds.
- Setting this value means that it is now fixed and the AE algorithm may
- not change it. Setting it back to zero returns it to the control of the
- AE algorithm.
+ This control will only take effect if ExposureTimeMode is Disabled. If
+ this control is set when ExposureTimeMode is Auto, the value will be
+ ignored and will not be retained.
+
+ When reported in metadata, this control indicates what exposure time
+ was used for the current request, regardless of ExposureTimeMode.
+ ExposureTimeMode will indicate the source of the exposure time value,
+ whether it came from the AE algorithm or not.
+
+ \sa AnalogueGain
+ \sa ExposureTimeMode
+
+ - ExposureTimeMode:
+ type: int32_t
+ description: |
+ Controls the source of the exposure time that is applied to the image
+ sensor. When set to Auto, the AE algorithm computes the exposure time
+ and configures the image sensor accordingly. When set to Disabled,
+ exposure time specified in ExposureTime is applied to the image sensor.
+ If ExposureTime is not set, then the value last computed by the AE
+ algorithm when the mode was Auto will be used.
+
+ If ExposureTime is not set and the mode is ExposureTimeModeDisabled and
+ AE was never Auto (either because the camera started in Disabled mode,
+ or Auto is not supported by the camera), the camera should use a
+ best-effort default value.
+
+ When ExposureTimeMode is set Auto, the value set in ExposureTime is
+ ignored and is not retained. This means that if ExposureTimeMode is set
+ to Disabled and ExposureTime is not also set, the exposure time that
+ was last computed by the AE algorithm while the mode was Auto will be
+ applied to the sensor.
+
+ If ExposureTimeModeDisabled is supported, the ExposureTime control must
+ also be supported.
+
+ The set of ExposureTimeMode modes that are supported by the camera must
+ have an intersection with the supported set of AnalogueGainMode modes.
- \sa AnalogueGain AeEnable
+ As it takes a few frames to apply the exposure time, there is a period of
+ time between submitting a request with ExposureTimeMode set to Disabled
+ and the exposure time component of the AE actually being disabled,
+ during which the AE algorithm can still update the exposure time. If an
+ application is switching from automatic and manual control and wishes
+ to eliminate any flicker during the switch, the following procedure is
+ recommended.
- \todo Document the interactions between AeEnable and setting a fixed
- value for this control. Consider interactions with other AE features,
- such as aperture and aperture/shutter priority mode, and decide if
- control of which features should be automatically adjusted shouldn't
- better be handled through a separate AE mode control.
+ 1. Start with ExposureTimeMode set to Auto
+
+ 2. Set ExposureTimeMode to Disabled
+
+ 3. Wait for the first request to be output that has ExposureTimeMode
+ set to Disabled
+
+ 4. Copy the value reported in ExposureTime into a new request, and
+ submit it
+
+ 5. Proceed to run manual exposure time
+
+ \sa ExposureTime
+ enum:
+ - name: ExposureTimeModeAuto
+ value: 0
+ description: |
+ The exposure time will be calculated automatically and set by the
+ AE algorithm. If ExposureTime is set while this mode is active, it
+ will be ignored, and it will also not be retained.
+ - name: ExposureTimeModeDisabled
+ value: 1
+ description: |
+ The exposure time will not be updated by the AE algorithm. It will
+ come from the last calculated value when the mode was Auto, or from
+ the value specified in ExposureTime.
+
+ When transitioning from Auto to Disabled mode, the last computed
+ exposure value is used until a new value is specified through the
+ ExposureTime control. If an ExposureTime value is specified in the
+ same request where the ExposureTimeMode is changed from Auto to
+ Disabled, the provided ExposureTime is applied.
- AnalogueGain:
type: float
@@ -144,17 +244,85 @@ controls:
The value of the control specifies the gain multiplier applied to all
colour channels. This value cannot be lower than 1.0.
- Setting this value means that it is now fixed and the AE algorithm may
- not change it. Setting it back to zero returns it to the control of the
- AE algorithm.
+ This control will only take effect if AnalogueGainMode is Disabled. If
+ this control is set when AnalogueGainMode is Auto, the value will be
+ ignored and will not be retained.
+
+ When reported in metadata, this control indicates what analogue gain
+ was used for the current request, regardless of AnalogueGainMode.
+ AnalogueGainMode will indicate the source of the analogue gain value,
+ whether it came from the AE algorithm or not.
+
+ \sa ExposureTime
+ \sa AnalogueGainMode
+
+ - AnalogueGainMode:
+ type: int32_t
+ description: |
+ Controls the source of the analogue gain that is applied to the image
+ sensor. When set to Auto, the AE algorithm computes the analogue gain
+ and configures the image sensor accordingly. When set to Disabled,
+ analogue gain specified in AnalogueGain is applied to the image sensor.
+ If AnalogueGain is not set, then the value last computed by the AE
+ algorithm when the mode was Auto will be used.
+
+ If AnalogueGain is not set and the mode is AnalogueGainModeDisabled and
+ AE was never Auto (either because the camera started in Disabled mode,
+ or Auto is not supported by the camera), the camera should use a
+ best-effort default value.
+
+ When AnalogueGainMode is set Auto, the value set in AnalogueGain is
+ ignored and is not retained. This means that if AnalogueGainMode is set
+ to Disabled and AnalogueGain is not also set, the analogue gain that
+ was last computed by the AE algorithm while the mode was Auto will be
+ applied to the sensor.
- \sa ExposureTime AeEnable
+ If AnalogueGainModeDisabled is supported, the AnalogueGain control must
+ also be supported.
+
+ The set of AnalogueGainMode modes that are supported by the camera must
+ have an intersection with the supported set of ExposureTimeMode modes.
+
+ As it takes a few frames to apply the analogue gain, there is a period of
+ time between submitting a request with AnalogueGainMode set to Disabled
+ and the analogue gain component of the AE actually being disabled,
+ during which the AE algorithm can still update the analogue gain. If an
+ application is switching from automatic and manual control and wishes
+ to eliminate any flicker during the switch, the following procedure is
+ recommended.
+
+ 1. Start with AnalogueGainMode set to Auto
+
+ 2. Set AnalogueGainMode to Disabled
+
+ 3. Wait for the first request to be output that has AnalogueGainMode
+ set to Disabled
+
+ 4. Copy the value reported in AnalogueGain into a new request, and
+ submit it
+
+ 5. Proceed to run manual analogue gain
+
+ \sa AnalogueGain
+ enum:
+ - name: AnalogueGainModeAuto
+ value: 0
+ description: |
+ The analogue gain will be calculated automatically and set by the
+ AE algorithm. If AnalogueGain is set while this mode is active, it
+ will be ignored, and it will also not be retained.
+ - name: AnalogueGainModeDisabled
+ value: 1
+ description: |
+ The analogue gain will not be updated by the AE algorithm. It will
+ come from the last calculated value when the mode was Auto, or from
+ the value specified in AnalogueGain.
- \todo Document the interactions between AeEnable and setting a fixed
- value for this control. Consider interactions with other AE features,
- such as aperture and aperture/shutter priority mode, and decide if
- control of which features should be automatically adjusted shouldn't
- better be handled through a separate AE mode control.
+ When transitioning from Auto to Disabled mode the last computed
+ gain value is used until a new value is specified through the
+ AnalogueGain control. If an AnalogueGain value is specified in the
+ same request where the AnalogueGainMode is set to Disabled, the
+ provided AnalogueGain is applied.
- Brightness:
type: float
@@ -477,36 +645,6 @@ controls:
High quality aberration correction which might reduce the frame
rate.
- - AeState:
- type: int32_t
- draft: true
- description: |
- Control to report the current AE algorithm state. Currently identical to
- ANDROID_CONTROL_AE_STATE.
-
- Current state of the AE algorithm.
- enum:
- - name: AeStateInactive
- value: 0
- description: The AE algorithm is inactive.
- - name: AeStateSearching
- value: 1
- description: The AE algorithm has not converged yet.
- - name: AeStateConverged
- value: 2
- description: The AE algorithm has converged.
- - name: AeStateLocked
- value: 3
- description: The AE algorithm is locked.
- - name: AeStateFlashRequired
- value: 4
- description: The AE algorithm would need a flash for good results
- - name: AeStatePrecapture
- value: 5
- description: |
- The AE algorithm has started a pre-capture metering session.
- \sa AePrecaptureTrigger
-
- AfState:
type: int32_t
draft: true
--
2.30.2
More information about the libcamera-devel
mailing list