[libcamera-devel] [RFC PATCH 1/1] libcamera: controls: Controls for driving AF (autofocus) algorithms
Hanlin Chen
hanlinchen at chromium.org
Thu Dec 16 12:50:07 CET 2021
On Tue, Dec 14, 2021 at 9:45 PM Naushir Patuck <naush at raspberrypi.com> wrote:
>
> Hi David,
>
> Thank you for this work. I'll kick off with a few questions, but I'm sure there will be more to come :)
>
> On Mon, 13 Dec 2021 at 15:22, David Plowman <david.plowman at raspberrypi.com> wrote:
>>
>> This patch describes a series of controls that allow applications to
>> drive AF algorithms:
>>
>> AfMode - auto or continuous
>> AfRange - full, macro or normal
>> AfSpeed - fast or slow
>> AfMethod - single or multi-spot
>> AfWindow - AF window locations
>> AfTrigger - start (trigger an AF scan) or cancel
>> LensPosition - position of lens from lens driver
>> AfState - reset, scanning, focused or failed
>> ---
>> src/libcamera/control_ids.yaml | 227 ++++++++++++++++++++++++---------
>> 1 file changed, 167 insertions(+), 60 deletions(-)
>>
>> diff --git a/src/libcamera/control_ids.yaml b/src/libcamera/control_ids.yaml
>> index 9d4638ae..e579f7b7 100644
>> --- a/src/libcamera/control_ids.yaml
>> +++ b/src/libcamera/control_ids.yaml
>> @@ -406,27 +406,6 @@ controls:
>> The camera will cancel any active or completed metering sequence.
>> The AE algorithm is reset to its initial state.
>>
>> - - AfTrigger:
>> - type: int32_t
>> - draft: true
>> - description: |
>> - Control for AF trigger. Currently identical to
>> - ANDROID_CONTROL_AF_TRIGGER.
>> -
>> - Whether the camera device will trigger autofocus for this request.
>> - enum:
>> - - name: AfTriggerIdle
>> - value: 0
>> - description: The trigger is idle.
>> - - name: AfTriggerStart
>> - value: 1
>> - description: The AF routine is started by the camera.
>> - - name: AfTriggerCancel
>> - value: 2
>> - description: |
>> - The camera will cancel any active trigger and the AF routine is
>> - reset to its initial state.
>> -
>> - NoiseReductionMode:
>> type: int32_t
>> draft: true
>> @@ -507,45 +486,6 @@ controls:
>> The AE algorithm has started a pre-capture metering session.
>> \sa AePrecaptureTrigger
>>
>> - - AfState:
>> - type: int32_t
>> - draft: true
>> - description: |
>> - Control to report the current AF algorithm state. Currently identical to
>> - ANDROID_CONTROL_AF_STATE.
>> -
>> - Current state of the AF algorithm.
>> - enum:
>> - - name: AfStateInactive
>> - value: 0
>> - description: The AF algorithm is inactive.
>> - - name: AfStatePassiveScan
>> - value: 1
>> - description: |
>> - AF is performing a passive scan of the scene in continuous
>> - auto-focus mode.
>> - - name: AfStatePassiveFocused
>> - value: 2
>> - description: |
>> - AF believes the scene is in focus, but might restart scanning.
>> - - name: AfStateActiveScan
>> - value: 3
>> - description: |
>> - AF is performing a scan triggered by an AF trigger request.
>> - \sa AfTrigger
>> - - name: AfStateFocusedLock
>> - value: 4
>> - description: |
>> - AF believes has focused correctly and has locked focus.
>> - - name: AfStateNotFocusedLock
>> - value: 5
>> - description: |
>> - AF has not been able to focus and has locked.
>> - - name: AfStatePassiveUnfocused
>> - value: 6
>> - description: |
>> - AF has completed a passive scan without finding focus.
>> -
>> - AwbState:
>> type: int32_t
>> draft: true
>> @@ -690,4 +630,171 @@ controls:
>> value. All of the custom test patterns will be static (that is the
>> raw image must not vary from frame to frame).
>>
>> + - AfMode:
>> + type: int32_t
>> + draft: true
>> + description: |
>> + Control to set the mode of the AF (autofocus) algorithm.
>> + enum:
>> + - name: AfModeAuto
>> + value: 0
>> + description: |
>> + The AF algorithm is in auto mode. This means that the algorithm
>> + will never move the lens or change state unless the AfTrigger
>> + control is used. The AfTrigger control can be used to initiate a
>> + focus scan, the results of which will be reported by the AfState.
>> +
>> + In this mode, an application can move the lens for itself using
>> + the LensPosition control. If the algorithm was scanning when the
>> + lens is moved in this way, then the scan is implicitly cancelled.
>> + - name: AfModeContinuous
>> + value: 1
>> + description: |
>> + The AF algorithm is in continuous mode. This means that the lens
>> + can re-start a scan spontaneously at any moment, without any user
>> + intervention. The AfState still reports whether the algorithm is
>> + currently scanning or not, though the application has no ability
>> + to initiate or cancel scans, nor move the lens for itself.
>> +
>> + The mode can be set to AfModeAuto which has the effect of
>> + "pausing" any continuous AF activity, after which it could then
>> + be moved back to AfModeContinuous to resume operation.
>
>
> I wonder if we need a method to tell the lens to move to a "default" position?
> For example, cancel AF or CAF, then move the lens to hyperfocal. With the
> above description, we can stop AF, and use the LensPosition control to move
> to a specified position, but how do we know what position corresponds to
> hyperfocal? If the hyperfocal position is defined in the tuning file, the application
> would not have access to the value. Perhaps an explicit mode to do this?
Could we set the default value (and min/max) for LensPosition as the
hyperfocal? As the definition of Sharpness.
And the user can move to the hyperfocal position manually, or other
position within the range.
>
>>
>> +
>> + - AfRange:
>> + type: int32_t
>> + draft: true
>> + description: |
>> + Control to set the range of focus distances that is scanned.
>> + enum:
>> + - name: AfRangeFull
>> + value: 0
>> + description: The full range of focus distances is scanned.
>> + - name: AfRangeMacro
>> + value: 1
>> + description: Only close distances are scanned.
>> + - name: AfRangeNormal
>> + value: 2
>> + description: |
>> + The full range of focus distances is scanned except for some of
>> + the closest macro positions.
>
>
> Is the "closest macro position" a property of the module, or a user preference?
>
>>
>> +
>> + - AfSpeed:
>> + type: int32_t
>> + draft: true
>> + description: |
>> + Control that determines whether the AF algorithm is to move the lens
>> + as quickly as possible or more steadily. For example, during video
>> + recording it may be desirable not to move the lens too abruptly, but
>> + when in a preview mode (waiting for a still capture) it may be
>> + helpful to move the lens as quickly as is reasonably possible.
>> + enum:
>> + - name: AfSpeedFast
>> + value: 0
>> + description: Move the lens quickly.
>> + - name: AfSpeedSlow
>> + value: 0
>> + description: Move the lens more steadily.
>> +
>> + - AfMethod:
>> + type: int32_t
>> + draft: true
>> + description: |
>> + Control whether the AF algorithm uses a single window in the image to
>> + determine the best focus position, or multiple windows simultaneously.
>> + enum:
>> + - name: AfMethodSingle
>> + value: 0
>> + description: |
>> + A single window within the image, defaulting to the centre, is used
>> + to select the best focus distance.
>> + - name: AfMethodMultiSpot
>> + value: 0
>> + description: |
>> + Multiple windows within the image are used to select the best focus
>> + distance. The best focus distance is found for each one of the
>> + windows, and then the distance that is closest to the camera is
>> + selected.
>
>
> Would AFfMethod be used in continuous focus modes as well? if yes, then perhaps
> s/AF algorithm/focus algorithms/? Ditto for AfWindow.
>
>>
>> +
>> + - AfWindow:
>> + type: Rectangle
>> + draft: true
>> + description: |
>> + Sets the focus windows used by the AF algorithm. The units used express
>> + a proportion of the ScalerCrop control (or if unavailable, of the entire
>> + image), as u0.16 format numbers.
>> +
>> + In order to be activated, a rectangle must be programmed with non-zero
>> + width and height. If no rectangles are programmed in this way, then the
>> + system will choose its own single default window in the centre of the
>> + image.
>> +
>> + If AfMethod is set to AfMethodSingle, then only the first Rectangle in
>> + this list is used (or the system default one if it is unprogrammed).
>> +
>> + If AfMethod is set to AfMethodMultiSpot then all the valid Rectangles in
>> + this list are used. The size of the control indicates how many such
>> + windows can be programmed and will vary between different platforms.
>> +
>> + size: [platform dependent]
>> +
>> + - AfTrigger:
>> + type: int32_t
>> + draft: true
>> + description: |
>> + This control starts an autofocus scan when AfMode is set to AfModeAuto,
>> + and can also be used to terminate a scan early.
>> +
>> + It is ignored if AfMode is set to AfModeContinuous.
>> +
>> + enum:
>> + - name: AfTriggerStart
>> + value: 0
>> + description: Start an AF scan. Ignored if a scan is in progress.
>> + - name: AfTriggerCancel
>> + value: 1
>> + description: Cancel an AF scan. Ingored if no scan is in progress.
>> +
>> + - LensPosition:
>> + type: int32_t
>> + draft: true
>> + description: |
>> + Acts as a control to instruct the lens to move to a particular position
>> + and also reports back the position of the lens for each frame.
>> +
>> + The units are determined by the lens driver.
>
>
> This would mean applications might not see the same/consistent behavior with
> different lens modules. Should we define a libcamera range (e.g. 0 -> Infinity,
> 1023 -> Macro) and convert to/from lens driver units in the IPA or pipeline handler?
>
>>
>> +
>> + If the LensPosition control is used while an AF scan is in progress,
>> + the scan is implicitly cancelled and the lens is then moved to the
>> + desired location. It is ignored if AfMode is set to AfModeContinuous.
>> +
>> + - AfState:
>> + type: int32_t
>> + draft: true
>> + description: |
>> + Reports the current state of the AF algorithm.
>> + enum:
>> + - name: AfStateReset
>> + value: 0
>> + description: |
>> + The AF algorithm reports this state when:
>> + * The camera system has just been started.
>> + * A scan has been cancelled.
>> + * The lens has been moved by the LensPosition control.
>> + - name: AfStateScanning
>> + value: 1
>> + description: |
>> + AF is performing a scan. This state can be entered spontaneously
>> + if AfMode is set to AfModeContinuous, otherwise it requires the
>> + application to use the AfTrigger control to start the scan.
>> + - name: AfStateFocused
>> + value: 2
>> + description: |
>> + An AF scan has been performed and the algorithm believes the
>> + scene is in focus.
>> + - name: AfStateFailed
>> + value: 3
>> + description: |
>> + An AF scan has been performed but the algorithm has not been able
>> + to find the best focus position.
>> +
>
>
> One last point, we currently have a FocusFoM control that returns a focus measure
> to the application. In light of the new AfWindow measure, should that perhaps be
> redefined?
>
> Regards,
> Naush
>
>
>>
>> ...
>> --
>> 2.30.2
>>
More information about the libcamera-devel
mailing list