[libcamera-devel] [PATCH v3 1/8] controls: Reorganize the AE-related controls
Paul Elder
paul.elder at ideasonboard.com
Tue Dec 21 05:36:03 CET 2021
We have multiple goals:
- we need a lock of some sort, to instruct the AEGC to not update output
results
- we need manual modes, to override the values computed by the AEGC
- we need to support seamless transitions from auto -> manual, and do so
without flickering
- we need custom minimum values for the manual controls, that is no
magic values for enabling/disabling auto
- all of these need to be done with AE sub-controls (exposure time,
analogue gain)
To achieve these goals, we introduce mode controls for the AE
sub-controls: ExposureTimeMode and AnalogueGainMode. These have an auto
state, and a disabled state. The disabled state has an internal one-way
state change from locked to manual, triggered by the presence of the
value-controls (ExposureTime and AnalogueGain).
We then remove the AeEnable control, as it is a redundant control in the
face of these two mode controls.
We also remove AeLocked, as it is insufficient for reporting the AE
state, and we promote AeState to non-draft to fill its role. Notably,
the locked state is removed, since this information can be obtained from
the aforementioned mode controls.
Bug: https://bugs.libcamera.org/show_bug.cgi?id=42
Bug: https://bugs.libcamera.org/show_bug.cgi?id=43
Bug: https://bugs.libcamera.org/show_bug.cgi?id=47
Signed-off-by: Paul Elder <paul.elder at ideasonboard.com>
---
Changes in v3:
- improve wording of the control descriptions
- make more succinct and clear
- add description of how to do a flickerless transition
Changes in v2:
- No changes, just resubmitting at the head of this series so that it's
together and so that /people will actually see it/
Initial version:
Still RFC as I haven't updated the users of the control yet, and I want
to check that these are the controls and docs that we want.
We've decided that the "master AE control" will be implemented by a
helper... but looking at uvcvideo and the V4L2 controls I'm wondering if
such helper should come earlier than later?
---
src/libcamera/control_ids.yaml | 239 ++++++++++++++++++++++++---------
1 file changed, 177 insertions(+), 62 deletions(-)
diff --git a/src/libcamera/control_ids.yaml b/src/libcamera/control_ids.yaml
index 9d4638ae..84679317 100644
--- a/src/libcamera/control_ids.yaml
+++ b/src/libcamera/control_ids.yaml
@@ -7,23 +7,63 @@
# Unless otherwise stated, all controls are bi-directional, i.e. they can be
# set through Request::controls() and returned out through Request::metadata().
controls:
- - AeEnable:
- type: bool
+ - AeState:
+ type: int32_t
description: |
- Enable or disable the AE.
+ Control to report the current AE algorithm state. The camera device can do
+ several state transitions between two results, if it is allowed by the
+ state transition table. For example, AeStateInactive may never
+ actually be seen in a result.
- \sa ExposureTime AnalogueGain
+ The state in the result is the state for this request. If AE state
+ becomes AeStateConverged, then the image data associated with the
+ result should be good to use.
- - AeLocked:
- type: bool
- description: |
- Report the lock status of a running AE algorithm.
+ The state is still reported even if ExposureTimeMode or
+ AnalogueGainMode is set to Disabled.
- If the AE algorithm is locked the value shall be set to true, if it's
- converging it shall be set to false. If the AE algorithm is not
- running the control shall not be present in the metadata control list.
+ \sa AnalogueGain
+ \sa AnalogueGainMode
+ \sa ExposureTime
+ \sa ExposureTimeMode
- \sa AeEnable
+ enum:
+ - name: AeStateInactive
+ value: 0
+ description: |
+ The AE algorithm is inactive.
+ If the camera initiates an AE scan, the state shall go to
+ AeStateSearching.
+
+ Unlike the analogous control in Android, enabling or disabling any
+ of the AE-related mode controls (eg. AnalogueGainMode,
+ ExposureTimeMode) does not require the state to be reset to to
+ AeStateInactive.
+ - name: AeStateSearching
+ value: 1
+ description: |
+ The AE algorithm has not converged yet.
+ If the camera finishes an AE scan, the state shall go to
+ AeStateConverged. If the camera finishes an AE scan, but flash is
+ required, the state shall go to AeStateFlashRequired.
+ - name: AeStateConverged
+ value: 2
+ description: |
+ The AE algorithm has converged.
+ If the camera initiates an AE scan, the state shall go to
+ AeStateSearching.
+ - name: AeStateFlashRequired
+ value: 3
+ description: |
+ The AE algorithm would need a flash for good results.
+ If the camera initiates an AE scan, the state shall go to
+ AeStateSearching.
+ - name: AeStatePrecapture
+ value: 4
+ description: |
+ The AE algorithm has started a pre-capture metering session.
+ After the sequence is finished, the state shall go to AeStateConverged if
+ \sa AePrecaptureTrigger
# AeMeteringMode needs further attention:
# - Auto-generate max enum value.
@@ -93,6 +133,13 @@ controls:
how the desired total exposure is divided between the shutter time
and the sensor's analogue gain. The exposure modes are platform
specific, and not all exposure modes may be supported.
+
+ When one of AnalogueGainMode or ExposureTimeMode is set to Disabled,
+ the fixed values will override any choices made by AeExposureMode.
+
+ \sa AnalogueGainMode
+ \sa ExposureTimeMode
+
enum:
- name: ExposureNormal
value: 0
@@ -111,13 +158,15 @@ controls:
type: float
description: |
Specify an Exposure Value (EV) parameter. The EV parameter will only be
- applied if the AE algorithm is currently enabled.
+ applied if the AE algorithm is currently enabled, that is, at least one
+ of AnalogueGainMode and ExposureTimeMode are auto.
By convention EV adjusts the exposure as log2. For example
EV = [-2, -1, 0.5, 0, 0.5, 1, 2] results in an exposure adjustment
of [1/4x, 1/2x, 1/sqrt(2)x, 1x, sqrt(2)x, 2x, 4x].
- \sa AeEnable
+ \sa AnalogueGainMode
+ \sa ExposureTimeMode
- ExposureTime:
type: int32_t
@@ -125,17 +174,65 @@ controls:
Exposure time (shutter speed) for the frame applied in the sensor
device. This value is specified in micro-seconds.
- Setting this value means that it is now fixed and the AE algorithm may
- not change it. Setting it back to zero returns it to the control of the
- AE algorithm.
+ This control will only take effect if ExposureTimeMode is Disabled. Its
+ presence in a request acts as a trigger to switch to the internal
+ manual mode within ExposureTimeModeDisabled.
+
+ When reported in metadata, this control indicates what exposure time
+ was used for the current request, regardless of ExposureTimeMode.
+ ExposureTimeMode will indicate the source of the exposure time value,
+ whether it came from the AE algorithm or not.
+
+ \sa AnalogueGain
+ \sa ExposureTimeMode
- \sa AnalogueGain AeEnable
+ - ExposureTimeMode:
+ type: int32_t
+ description: |
+ Controls how the frame exposure time is computed. When set to Auto, the
+ AE algorithm computes the exposure time of the next frame and
+ configures the image sensor accordingly. When set to Disabled, the AE
+ algorithm stops updating the exposure time.
- \todo Document the interactions between AeEnable and setting a fixed
- value for this control. Consider interactions with other AE features,
- such as aperture and aperture/shutter priority mode, and decide if
- control of which features should be automatically adjusted shouldn't
- better be handled through a separate AE mode control.
+ As the camera device has a pipeline of in-flight requests, there is a
+ period of time between submitting a request with ExposureTimeMode set
+ to Disabled and the exposure time component of the AE actually being
+ disabled, during which the AE algorithm can still update the exposure
+ time. If an application is switching from automatic and manual control
+ and wishes to eliminate any flicker during the switch, the following
+ procedure is recommended.
+
+ 1. Start with ExposureTimeMode set to Auto
+
+ 2. Set ExposureTimeMode to Disabled
+
+ 3. Wait for the first request to be output that has ExposureTimeMode
+ set to Disabled
+
+ 4. Copy the value reported in ExposureTime into a new request, and
+ submit it
+
+ 5. Proceed to run manual exposure time
+
+ \sa ExposureTime
+ enum:
+ - name: ExposureTimeModeAuto
+ value: 0
+ description: |
+ The exposure time will be calculated automatically and set by the
+ AE algorithm.
+ - name: ExposureTimeModeDisabled
+ value: 1
+ description: |
+ The exposure time will not be updated by the AE algorithm. It will
+ come from the last calculated value when the mode was Auto, or from
+ the value specified in ExposureTime.
+
+ When transitioning from Auto to Disabled mode the last computed
+ exposure value is used until a new value is specified through the
+ ExposureTime control. If an ExposureTime value is specified in the
+ same request where the ExposureTimeMode is set to Disabled, the
+ provided ExposureTime is applied immediately.
- AnalogueGain:
type: float
@@ -144,17 +241,65 @@ controls:
The value of the control specifies the gain multiplier applied to all
colour channels. This value cannot be lower than 1.0.
- Setting this value means that it is now fixed and the AE algorithm may
- not change it. Setting it back to zero returns it to the control of the
- AE algorithm.
+ This control will only take effect if ExposureTimeMode is Disabled. Its
+ presence in a request acts as a trigger to switch to the internal
+ manual mode within ExposureTimeModeDisabled.
+
+ When reported in metadata, this control indicates what exposure time
+ was used for the current request, regardless of ExposureTimeMode.
+ ExposureTimeMode will indicate the source of the exposure time value,
+ whether it came from the AE algorithm or not.
+
+ \sa ExposureTime
+ \sa AnalogueGainMode
+
+ - AnalogueGainMode:
+ type: int32_t
+ description: |
+ Controls how the analogue gain is computed. When set to Auto, the
+ AE algorithm computes the analogue gain of the next frame and
+ configures the image sensor accordingly. When set to Disabled, the AE
+ algorithm stops updating the analogue gain.
+
+ As the camera device has a pipeline of in-flight requests, there is a
+ period of time between submitting a request with AnalogueGainMode set
+ to Disabled and the analogue gain component of the AE actually being
+ disabled, during which the AE algorithm can still update the analogue
+ gain. If an application is switching from automatic and manual control
+ and wishes to eliminate any flicker during the switch, the following
+ procedure is recommended.
+
+ 1. Start with AnalogueGainMode set to Auto
- \sa ExposureTime AeEnable
+ 2. Set AnalogueGainMode to Disabled
- \todo Document the interactions between AeEnable and setting a fixed
- value for this control. Consider interactions with other AE features,
- such as aperture and aperture/shutter priority mode, and decide if
- control of which features should be automatically adjusted shouldn't
- better be handled through a separate AE mode control.
+ 3. Wait for the first request to be output that has AnalogueGainMode
+ set to Disabled
+
+ 4. Copy the value reported in AnalogueGain into a new request, and
+ submit it
+
+ 5. Proceed to run manual analogue gain
+
+ \sa AnalogueGain
+ enum:
+ - name: AnalogueGainModeAuto
+ value: 0
+ description: |
+ The analogue gain will be calculated automatically and set by the
+ AE algorithm.
+ - name: AnalogueGainModeDisabled
+ value: 1
+ description: |
+ The analogue gain will not be updated by the AE algorithm. It will
+ come from the last calculated value when the mode was Auto, or from
+ the value specified in AnalogueGain.
+
+ When transitioning from Auto to Disabled mode the last computed
+ gain value is used until a new value is specified through the
+ AnalogueGain control. If an AnalogueGain value is specified in the
+ same request where the AnalogueGainMode is set to Disabled, the
+ provided AnalogueGain is applied immediately.
- Brightness:
type: float
@@ -477,36 +622,6 @@ controls:
High quality aberration correction which might reduce the frame
rate.
- - AeState:
- type: int32_t
- draft: true
- description: |
- Control to report the current AE algorithm state. Currently identical to
- ANDROID_CONTROL_AE_STATE.
-
- Current state of the AE algorithm.
- enum:
- - name: AeStateInactive
- value: 0
- description: The AE algorithm is inactive.
- - name: AeStateSearching
- value: 1
- description: The AE algorithm has not converged yet.
- - name: AeStateConverged
- value: 2
- description: The AE algorithm has converged.
- - name: AeStateLocked
- value: 3
- description: The AE algorithm is locked.
- - name: AeStateFlashRequired
- value: 4
- description: The AE algorithm would need a flash for good results
- - name: AeStatePrecapture
- value: 5
- description: |
- The AE algorithm has started a pre-capture metering session.
- \sa AePrecaptureTrigger
-
- AfState:
type: int32_t
draft: true
--
2.27.0
More information about the libcamera-devel
mailing list