[libcamera-devel] [RFC PATCH 1/1] libcamera: controls: Controls for driving AF (autofocus) algorithms
Naushir Patuck
naush at raspberrypi.com
Tue Dec 14 14:44:41 CET 2021
Hi David,
Thank you for this work. I'll kick off with a few questions, but I'm sure
there will be more to come :)
On Mon, 13 Dec 2021 at 15:22, David Plowman <david.plowman at raspberrypi.com>
wrote:
> This patch describes a series of controls that allow applications to
> drive AF algorithms:
>
> AfMode - auto or continuous
> AfRange - full, macro or normal
> AfSpeed - fast or slow
> AfMethod - single or multi-spot
> AfWindow - AF window locations
> AfTrigger - start (trigger an AF scan) or cancel
> LensPosition - position of lens from lens driver
> AfState - reset, scanning, focused or failed
> ---
> src/libcamera/control_ids.yaml | 227 ++++++++++++++++++++++++---------
> 1 file changed, 167 insertions(+), 60 deletions(-)
>
> diff --git a/src/libcamera/control_ids.yaml
> b/src/libcamera/control_ids.yaml
> index 9d4638ae..e579f7b7 100644
> --- a/src/libcamera/control_ids.yaml
> +++ b/src/libcamera/control_ids.yaml
> @@ -406,27 +406,6 @@ controls:
> The camera will cancel any active or completed metering
> sequence.
> The AE algorithm is reset to its initial state.
>
> - - AfTrigger:
> - type: int32_t
> - draft: true
> - description: |
> - Control for AF trigger. Currently identical to
> - ANDROID_CONTROL_AF_TRIGGER.
> -
> - Whether the camera device will trigger autofocus for this request.
> - enum:
> - - name: AfTriggerIdle
> - value: 0
> - description: The trigger is idle.
> - - name: AfTriggerStart
> - value: 1
> - description: The AF routine is started by the camera.
> - - name: AfTriggerCancel
> - value: 2
> - description: |
> - The camera will cancel any active trigger and the AF routine
> is
> - reset to its initial state.
> -
> - NoiseReductionMode:
> type: int32_t
> draft: true
> @@ -507,45 +486,6 @@ controls:
> The AE algorithm has started a pre-capture metering session.
> \sa AePrecaptureTrigger
>
> - - AfState:
> - type: int32_t
> - draft: true
> - description: |
> - Control to report the current AF algorithm state. Currently
> identical to
> - ANDROID_CONTROL_AF_STATE.
> -
> - Current state of the AF algorithm.
> - enum:
> - - name: AfStateInactive
> - value: 0
> - description: The AF algorithm is inactive.
> - - name: AfStatePassiveScan
> - value: 1
> - description: |
> - AF is performing a passive scan of the scene in continuous
> - auto-focus mode.
> - - name: AfStatePassiveFocused
> - value: 2
> - description: |
> - AF believes the scene is in focus, but might restart scanning.
> - - name: AfStateActiveScan
> - value: 3
> - description: |
> - AF is performing a scan triggered by an AF trigger request.
> - \sa AfTrigger
> - - name: AfStateFocusedLock
> - value: 4
> - description: |
> - AF believes has focused correctly and has locked focus.
> - - name: AfStateNotFocusedLock
> - value: 5
> - description: |
> - AF has not been able to focus and has locked.
> - - name: AfStatePassiveUnfocused
> - value: 6
> - description: |
> - AF has completed a passive scan without finding focus.
> -
> - AwbState:
> type: int32_t
> draft: true
> @@ -690,4 +630,171 @@ controls:
> value. All of the custom test patterns will be static (that
> is the
> raw image must not vary from frame to frame).
>
> + - AfMode:
> + type: int32_t
> + draft: true
> + description: |
> + Control to set the mode of the AF (autofocus) algorithm.
> + enum:
> + - name: AfModeAuto
> + value: 0
> + description: |
> + The AF algorithm is in auto mode. This means that the
> algorithm
> + will never move the lens or change state unless the AfTrigger
> + control is used. The AfTrigger control can be used to
> initiate a
> + focus scan, the results of which will be reported by the
> AfState.
> +
> + In this mode, an application can move the lens for itself
> using
> + the LensPosition control. If the algorithm was scanning when
> the
> + lens is moved in this way, then the scan is implicitly
> cancelled.
> + - name: AfModeContinuous
> + value: 1
> + description: |
> + The AF algorithm is in continuous mode. This means that the
> lens
> + can re-start a scan spontaneously at any moment, without any
> user
> + intervention. The AfState still reports whether the algorithm
> is
> + currently scanning or not, though the application has no
> ability
> + to initiate or cancel scans, nor move the lens for itself.
> +
> + The mode can be set to AfModeAuto which has the effect of
> + "pausing" any continuous AF activity, after which it could
> then
> + be moved back to AfModeContinuous to resume operation.
>
I wonder if we need a method to tell the lens to move to a "default"
position?
For example, cancel AF or CAF, then move the lens to hyperfocal. With the
above description, we can stop AF, and use the LensPosition control to move
to a specified position, but how do we know what position corresponds to
hyperfocal? If the hyperfocal position is defined in the tuning file, the
application
would not have access to the value. Perhaps an explicit mode to do this?
> +
> + - AfRange:
> + type: int32_t
> + draft: true
> + description: |
> + Control to set the range of focus distances that is scanned.
> + enum:
> + - name: AfRangeFull
> + value: 0
> + description: The full range of focus distances is scanned.
> + - name: AfRangeMacro
> + value: 1
> + description: Only close distances are scanned.
> + - name: AfRangeNormal
> + value: 2
> + description: |
> + The full range of focus distances is scanned except for some
> of
> + the closest macro positions.
>
Is the "closest macro position" a property of the module, or a user
preference?
> +
> + - AfSpeed:
> + type: int32_t
> + draft: true
> + description: |
> + Control that determines whether the AF algorithm is to move the
> lens
> + as quickly as possible or more steadily. For example, during video
> + recording it may be desirable not to move the lens too abruptly,
> but
> + when in a preview mode (waiting for a still capture) it may be
> + helpful to move the lens as quickly as is reasonably possible.
> + enum:
> + - name: AfSpeedFast
> + value: 0
> + description: Move the lens quickly.
> + - name: AfSpeedSlow
> + value: 0
> + description: Move the lens more steadily.
> +
> + - AfMethod:
> + type: int32_t
> + draft: true
> + description: |
> + Control whether the AF algorithm uses a single window in the
> image to
> + determine the best focus position, or multiple windows
> simultaneously.
> + enum:
> + - name: AfMethodSingle
> + value: 0
> + description: |
> + A single window within the image, defaulting to the centre,
> is used
> + to select the best focus distance.
> + - name: AfMethodMultiSpot
> + value: 0
> + description: |
> + Multiple windows within the image are used to select the best
> focus
> + distance. The best focus distance is found for each one of the
> + windows, and then the distance that is closest to the camera
> is
> + selected.
>
Would AFfMethod be used in continuous focus modes as well? if yes, then
perhaps
s/AF algorithm/focus algorithms/? Ditto for AfWindow.
> +
> + - AfWindow:
> + type: Rectangle
> + draft: true
> + description: |
> + Sets the focus windows used by the AF algorithm. The units used
> express
> + a proportion of the ScalerCrop control (or if unavailable, of
> the entire
> + image), as u0.16 format numbers.
> +
> + In order to be activated, a rectangle must be programmed with
> non-zero
> + width and height. If no rectangles are programmed in this way,
> then the
> + system will choose its own single default window in the centre
> of the
> + image.
> +
> + If AfMethod is set to AfMethodSingle, then only the first
> Rectangle in
> + this list is used (or the system default one if it is
> unprogrammed).
> +
> + If AfMethod is set to AfMethodMultiSpot then all the valid
> Rectangles in
> + this list are used. The size of the control indicates how many
> such
> + windows can be programmed and will vary between different
> platforms.
> +
> + size: [platform dependent]
> +
> + - AfTrigger:
> + type: int32_t
> + draft: true
> + description: |
> + This control starts an autofocus scan when AfMode is set to
> AfModeAuto,
> + and can also be used to terminate a scan early.
> +
> + It is ignored if AfMode is set to AfModeContinuous.
> +
> + enum:
> + - name: AfTriggerStart
> + value: 0
> + description: Start an AF scan. Ignored if a scan is in progress.
> + - name: AfTriggerCancel
> + value: 1
> + description: Cancel an AF scan. Ingored if no scan is in
> progress.
> +
> + - LensPosition:
> + type: int32_t
> + draft: true
> + description: |
> + Acts as a control to instruct the lens to move to a particular
> position
> + and also reports back the position of the lens for each frame.
> +
> + The units are determined by the lens driver.
>
This would mean applications might not see the same/consistent behavior with
different lens modules. Should we define a libcamera range (e.g. 0 ->
Infinity,
1023 -> Macro) and convert to/from lens driver units in the IPA or pipeline
handler?
> +
> + If the LensPosition control is used while an AF scan is in
> progress,
> + the scan is implicitly cancelled and the lens is then moved to
> the
> + desired location. It is ignored if AfMode is set to
> AfModeContinuous.
> +
> + - AfState:
> + type: int32_t
> + draft: true
> + description: |
> + Reports the current state of the AF algorithm.
> + enum:
> + - name: AfStateReset
> + value: 0
> + description: |
> + The AF algorithm reports this state when:
> + * The camera system has just been started.
> + * A scan has been cancelled.
> + * The lens has been moved by the LensPosition control.
> + - name: AfStateScanning
> + value: 1
> + description: |
> + AF is performing a scan. This state can be entered
> spontaneously
> + if AfMode is set to AfModeContinuous, otherwise it requires
> the
> + application to use the AfTrigger control to start the scan.
> + - name: AfStateFocused
> + value: 2
> + description: |
> + An AF scan has been performed and the algorithm believes the
> + scene is in focus.
> + - name: AfStateFailed
> + value: 3
> + description: |
> + An AF scan has been performed but the algorithm has not
> been able
> + to find the best focus position.
> +
>
One last point, we currently have a FocusFoM control that returns a focus
measure
to the application. In light of the new AfWindow measure, should that
perhaps be
redefined?
Regards,
Naush
> ...
> --
> 2.30.2
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.libcamera.org/pipermail/libcamera-devel/attachments/20211214/ffa4725a/attachment-0001.htm>
More information about the libcamera-devel
mailing list