[libcamera-devel] [PATCH v2 1/1] libcamera: controls: Controls for driving AF (autofocus) algorithms
Jean-Michel Hautbois
jeanmichel.hautbois at ideasonboard.com
Fri Apr 15 10:21:57 CEST 2022
Hello David,
Now that I'm starting to play a bit with it, I see a few things we might
want to improve.
On 31/03/2022 17:17, David Plowman wrote:
> This patch describes a series of controls that allow applications to
> drive AF algorithms:
>
> AfMode - manual, auto or continuous
> AfLensPosition - set lens position to macro/hyperfocal/infinity
> AfRange - full, macro or normal
> AfSpeed - fast or slow
> AfWindows - AF window locations
> AfTrigger - start (trigger an AF scan) or cancel
> AfPause - pause continuous AF
> LensPosition - position of lens from lens driver
> AfState - reports the mode, whether scanning/success/failure
> ---
> src/libcamera/control_ids.yaml | 373 +++++++++++++++++++++++++++------
> 1 file changed, 313 insertions(+), 60 deletions(-)
>
> diff --git a/src/libcamera/control_ids.yaml b/src/libcamera/control_ids.yaml
> index 9d4638ae..23607368 100644
> --- a/src/libcamera/control_ids.yaml
> +++ b/src/libcamera/control_ids.yaml
> @@ -406,27 +406,6 @@ controls:
> The camera will cancel any active or completed metering sequence.
> The AE algorithm is reset to its initial state.
>
> - - AfTrigger:
> - type: int32_t
> - draft: true
> - description: |
> - Control for AF trigger. Currently identical to
> - ANDROID_CONTROL_AF_TRIGGER.
> -
> - Whether the camera device will trigger autofocus for this request.
> - enum:
> - - name: AfTriggerIdle
> - value: 0
> - description: The trigger is idle.
> - - name: AfTriggerStart
> - value: 1
> - description: The AF routine is started by the camera.
> - - name: AfTriggerCancel
> - value: 2
> - description: |
> - The camera will cancel any active trigger and the AF routine is
> - reset to its initial state.
> -
> - NoiseReductionMode:
> type: int32_t
> draft: true
> @@ -507,45 +486,6 @@ controls:
> The AE algorithm has started a pre-capture metering session.
> \sa AePrecaptureTrigger
>
> - - AfState:
> - type: int32_t
> - draft: true
> - description: |
> - Control to report the current AF algorithm state. Currently identical to
> - ANDROID_CONTROL_AF_STATE.
> -
> - Current state of the AF algorithm.
> - enum:
> - - name: AfStateInactive
> - value: 0
> - description: The AF algorithm is inactive.
> - - name: AfStatePassiveScan
> - value: 1
> - description: |
> - AF is performing a passive scan of the scene in continuous
> - auto-focus mode.
> - - name: AfStatePassiveFocused
> - value: 2
> - description: |
> - AF believes the scene is in focus, but might restart scanning.
> - - name: AfStateActiveScan
> - value: 3
> - description: |
> - AF is performing a scan triggered by an AF trigger request.
> - \sa AfTrigger
> - - name: AfStateFocusedLock
> - value: 4
> - description: |
> - AF believes has focused correctly and has locked focus.
> - - name: AfStateNotFocusedLock
> - value: 5
> - description: |
> - AF has not been able to focus and has locked.
> - - name: AfStatePassiveUnfocused
> - value: 6
> - description: |
> - AF has completed a passive scan without finding focus.
> -
> - AwbState:
> type: int32_t
> draft: true
> @@ -690,4 +630,317 @@ controls:
> value. All of the custom test patterns will be static (that is the
> raw image must not vary from frame to frame).
>
> + - AfMode:
> + type: int32_t
> + draft: true
> + description: |
> + Control to set the mode of the AF (autofocus) algorithm. Applications
> + are allowed to set a new mode, and to send additional controls for
> + that new mode, in the same request.
> +
> + An implementation may choose not to implement all the modes.
> +
> + enum:
> + - name: AfModeManual
> + value: 0
> + description: |
> + The AF algorithm is in manual mode. In this mode it will never
> + perform any action nor move the lens of its own accord, but an
> + application can set controls to move the lens.
> +
> + In this mode the AfState will always report AfStateManual.
> + - name: AfModeAuto
> + value: 1
> + description: |
> + The AF algorithm is in auto mode. This means that the algorithm
> + will never move the lens or change state unless the AfTrigger
> + control is used. The AfTrigger control can be used to initiate a
> + focus scan, the results of which will also be reported by AfState.
> +
> + If the autofocus algorithm is moved from AfModeAuto to another
> + mode while a scan is in progress, the scan is cancelled
> + immediately, without waiting for the scan to finish.
> +
> + When first entering this mode the AfState will report
> + AfStateAuto. When a trigger control is sent, AfState will
> + report AfStateAutoScanning for a period before spontaneously
> + changing to AfStateAutoFocused or AfStateAutoFailed, depending on
> + the outcome of the scan. It will remain in this state until
> + another scan is initiated by the AfTrigger control. If a scan is
> + cancelled (without changing to another mode), AfState will return
> + to AfStateAuto.
> + - name: AfModeContinuous
> + value: 2
> + description: |
> + The AF algorithm is in continuous mode. This means that the lens
> + can re-start a scan spontaneously at any moment, without any user
> + intervention. The AfState still reports whether the algorithm is
> + currently scanning or not, though the application has no ability
> + to initiate or cancel scans (though it can "pause" them), nor to
> + move the lens for itself.
> +
> + When set to AfModeContinuous, the system will immediately initiate
> + a scan so AfState will report AfStateContinuousScanning, and will
> + settle on one of AfStateContinuousFocused or AfStateContinuousFailed,
> + depending on the scan result.
> +
> + - AfLensPosition:
> + type: int32_t
> + draft: true
> + description: |
> + Control to set the position of the lens to one of the following
> + predefined locations. This control only has any effect when the AfMode
> + is set to AfModeManual.
> +
> + This control is distinct from the LensPosition control, which sets the
> + lens position using the lens driver's units.
> + enum:
> + - name: AfLensPositionMacro
> + value: 0
> + description: The closest focal position of the lens.
> + - name: AfLensPositionHyperfocal
> + value: 1
> + description: Hyperfocal position.
> + - name: AfLensPositionInfinity
> + value: 2
> + description: The furthest focal position (usually infinity).
> +
> + - AfRange:
> + type: int32_t
> + draft: true
> + description: |
> + Control to set the range of focus distances that is scanned. An
> + implementation may choose not to implement all the options here.
> + enum:
> + - name: AfRangeNormal
> + value: 0
> + description: |
> + A wide range of focus distances is scanned, all the way from
> + infinity down to close distances, though depending on the
> + implementation, possibly not including the very closest macro
> + positions.
> + - name: AfRangeMacro
> + value: 1
> + description: Only close distances are scanned.
> + - name: AfRangeFull
> + value: 2
> + description: |
> + The full range of focus distances is scanned just as with
> + AfRangeNormal but this time including the very closest macro
> + positions.
> +
> + - AfSpeed:
> + type: int32_t
> + draft: true
> + description: |
> + Control that determines whether the AF algorithm is to move the lens
> + as quickly as possible or more steadily. For example, during video
> + recording it may be desirable not to move the lens too abruptly, but
> + when in a preview mode (waiting for a still capture) it may be
> + helpful to move the lens as quickly as is reasonably possible.
> + enum:
> + - name: AfSpeedNormal
> + value: 0
> + description: Move the lens at its usual speed.
> + - name: AfSpeedFast
> + value: 1
> + description: Move the lens more quickly.
> +
> + - AfWindows:
> + type: Rectangle
> + draft: true
> + description: |
> + Sets the focus windows used by the AF algorithm. The units used express
> + a proportion of the ScalerCrop control (or if unavailable, of the entire
> + image), as u0.16 format numbers.
I don't mind using ScalerCrop, but I think it would probably make more
sense to use the full pixel array ?
> +
> + In order to be activated, a rectangle must be programmed with non-zero
> + width and height. If no rectangles are programmed in this way, then the
> + system will choose its own single default window in the centre of the
> + image.
> +
> + The details of how the windows are used are platform dependent. We note
> + that when there is more than one AF window, a typical implementation
> + might find the optimal focus position for each one and finally select
> + the window closest to the camera.
> +
> + size: [platform dependent]
And I would like to point you to this one (you probably already now it):
https://developer.android.com/reference/android/hardware/Camera.Parameters#getFocusAreas()
I really like the number of areas variable, and even more the weight per
area. It makes sense, you might want to have more weight in the center,
but still have a larger region.
I don't get why the corrdinates are between [-1000, 1000] though but...
not that important.
What do you think ?
JM
> +
> + - AfTrigger:
> + type: int32_t
> + draft: true
> + description: |
> + This control starts an autofocus scan when AfMode is set to AfModeAuto,
> + and can also be used to terminate a scan early.
> +
> + It is ignored if AfMode is set to AfModeManual or AfModeContinuous.
> +
> + enum:
> + - name: AfTriggerStart
> + value: 0
> + description: Start an AF scan. Ignored if a scan is in progress.
> + - name: AfTriggerCancel
> + value: 1
> + description: Cancel an AF scan. This does not cause the lens to move
> + anywhere else. Ignored if no scan is in progress.
> +
> + - AfPause:
> + type: int32_t
> + draft: true
> + description: |
> + This control has no effect except when in continuous autofocus mode
> + (AfModeContinuous). It can be used to pause any lens movements while
> + (for example) images are captured. The algorithm remains inactive
> + until it is instructed to resume.
> +
> + enum:
> + - name: AfPauseImmediate
> + value: 0
> + description: |
> + Pause the continuous autofocus algorithm immediately, whether or
> + not any kind of scan is underway. If the AfState was previously
> + reporting AfStateContinuousScanning it will now change to
> + AfStateContinuousScanningPaused, and similarly for
> + AfStateContinuousFocused and AfStateContinuousFailed.
> + - name AfPauseDeferred
> + value: 1
> + description: |
> + Pause the continuous autofocus algorithm as soon as it is no longer
> + scanning. If the AfState is currently reporting
> + AfStateContinuousFocused is will change to
> + AfStateContinuousFocusedPaused, and similarly in the case of
> + AfStateContinuousFailed.
> +
> + If AfState reports AfStateContinuousScanning it will change to
> + AfStateContinuousScanningPausing, and then move to one of
> + AfStateContinuousFocusedPaused or AfStateContinuousFailedPaused
> + when the scan completes.
> + - name: AfPauseResume
> + value: 2
> + description: |
> + Resume continous autofocus operation. The algorithm starts again
> + from exactly where it left off, and the AfState will drop the
> + 'Paused' or 'Pausing' part of the state.
> +
> + - LensPosition:
> + type: int32_t
> + draft: true
> + description: |
> + Acts as a control to instruct the lens to move to a particular position
> + and also reports back the position of the lens for each frame.
> +
> + The units are determined by the lens driver.
> +
> + The LensPosition control is ignored unless the AfMode is set to
> + AfModeManual.
> +
> + Note that this control is distinct from AfLensPosition, which allows
> + the lens to be moved to its macro/hyperfocal/infinity position, rather
> + than using lens driver units.
> +
> + - AfState:
> + type: int32_t
> + draft: true
> + description: |
> + Reports the current state of the AF algorithm. The possible state changes
> + are described below, although we add that the following general state
> + changes are also allowed.
> +
> + In any of the AfStateManual or AfStateContinuous states, the AfMode
> + may be set to AfModeAuto and the algorithm will move to the
> + AfStateAuto state. If AfTriggerStart is sent at the same time
> + then the algorithm will move to AfStateAutoScanning state.
> +
> + In any of the AfStateAuto or AfStateManual states, the AfMode may
> + be set to AfModeContinuous and the algorithm will move to
> + AfStateContinuousScanning.
> +
> + In any of the AfStateAuto or AfStateContinuous states, the AfMode may
> + be set to AfModeManual and the algorithm will move to
> + AfStateManual. The lens will not be moved and will be left where
> + it was at that moment.
> +
> + enum:
> + - name: AfStateManual
> + value: 0
> + description: |
> + The AF algorithm is in manual mode (AfModeManual). The LensPosition
> + and AfLensPosition controls can be used directly to move the lens.
> + - name: AfStateAuto
> + value: 1
> + description: |
> + The AF algorithm is in auto mode (AfModeAuto), and has either just
> + been moved into that state, or a scan that was in progress has been
> + cancelled.
> + - name: AfStateAutoScanning
> + value: 2
> + description: |
> + The AF algorithm is in auto mode (AfModeAuto), and a scan has been
> + started using the AfTrigger control. The scan can be cancelled by
> + sending AfTriggerCancel at which point the algorithm will either
> + move back to AfStateAuto or, if the scan actually completes
> + before the cancel request is processed, to one of
> + AfStateAutoFocused or AfStateAutoFailed.
> + - name: AfStateAutoFocused
> + value: 3
> + description: |
> + The AF algorithm is in auto mode (AfModeAuto), and a scan has
> + completed with the result that the algorithm believes the image is
> + now in focus.
> + - name: AfStateAutoFailed
> + value: 4
> + description: |
> + The AF algorithm is in auto mode (AfModeAuto), and a scan has
> + completed with the result that the algorithm did not find a good
> + focus position.
> + - name: AfStateContinuousScanning
> + value: 5
> + description: |
> + The AF algorithm is in continuous mode (AfModeContinuous) and
> + is currently scanning for a good focus position. This occurs when
> + the mode is first set to AfModeContinuous, or may happen
> + spontaneously when the algorithm believes a re-scan is needed.
> + - name: AfStateContinuousScanningPausing
> + value: 6
> + description: |
> + The AF algorithm is in continuous mode (AfModeContinuous) and
> + was scanning when AfPauseDeferred was sent. The scan will complete
> + at which point the algorithm moves to the
> + AfStateContinuousFocusedPaused or AfStateContinuousFailedPaused
> + state.
> + - name: AfStateContinuousScanningPaused
> + value: 7
> + description: |
> + The AF algorithm is in continuous mode (AfModeContinuous) and
> + was scanning when AfPauseImmediate was sent. The scan will not
> + complete and the algorithm will remain in this state. The scan
> + may be resumed by sending AfPauseResume.
> + - name: AfStateContinuousFocused
> + value: 8
> + description: |
> + The AF algorithm is in continuous mode (AfModeContinuous) and
> + a scan has completed with the algorithm believing it has found a
> + good focus position.
> + - name: AfStateContinuousFocusedPaused
> + value: 9
> + description: |
> + The AF algorithm was in the AfStateContinuousFocused state and
> + has been paused (by either AfPauseImmediate or AfPauseDeferred),
> + or it was in the AfStateContinuousScanningPausing state and the
> + scan has completed successfully. The algorithm will now remain
> + in this state, and may be resumed by sending AfPauseResume.
> + - name: AfStateContinuousFailed
> + value: 10
> + description: |
> + The AF algorithm is in continuous mode (AfModeContinuous) and
> + a scan has completed without finding a good focus position.
> + - name: AfStateContinuousFailedPaused
> + value: 11
> + description: |
> + The AF algorithm was in the AfStateContinuousFailed state and
> + has been paused (by either AfPauseImmediate or AfPauseDeferred),
> + or it was in the AfStateContinuousScanningPausing state and the
> + scan has completed unsuccessfully. The algorithm will now remain
> + in this state, and may be resumed by sending AfPauseResume.
> +
> ...
More information about the libcamera-devel
mailing list