[libcamera-devel] [RFC PATCH 1/1] libcamera: controls: Controls for driving AF (autofocus) algorithms
David Plowman
david.plowman at raspberrypi.com
Fri Dec 17 09:51:37 CET 2021
Hi again
Thanks for the comments!
On Thu, 16 Dec 2021 at 11:18, Hanlin Chen <hanlinchen at chromium.org> wrote:
>
> Hi David, thanks for this work!
>
> On Mon, Dec 13, 2021 at 11:22 PM David Plowman
> <david.plowman at raspberrypi.com> wrote:
> >
> > This patch describes a series of controls that allow applications to
> > drive AF algorithms:
> >
> > AfMode - auto or continuous
> > AfRange - full, macro or normal
> > AfSpeed - fast or slow
> > AfMethod - single or multi-spot
> > AfWindow - AF window locations
> > AfTrigger - start (trigger an AF scan) or cancel
> > LensPosition - position of lens from lens driver
> > AfState - reset, scanning, focused or failed
> > ---
> > src/libcamera/control_ids.yaml | 227 ++++++++++++++++++++++++---------
> > 1 file changed, 167 insertions(+), 60 deletions(-)
> >
> > diff --git a/src/libcamera/control_ids.yaml b/src/libcamera/control_ids.yaml
> > index 9d4638ae..e579f7b7 100644
> > --- a/src/libcamera/control_ids.yaml
> > +++ b/src/libcamera/control_ids.yaml
> > @@ -406,27 +406,6 @@ controls:
> > The camera will cancel any active or completed metering sequence.
> > The AE algorithm is reset to its initial state.
> >
> > - - AfTrigger:
> > - type: int32_t
> > - draft: true
> > - description: |
> > - Control for AF trigger. Currently identical to
> > - ANDROID_CONTROL_AF_TRIGGER.
> > -
> > - Whether the camera device will trigger autofocus for this request.
> > - enum:
> > - - name: AfTriggerIdle
> > - value: 0
> > - description: The trigger is idle.
> > - - name: AfTriggerStart
> > - value: 1
> > - description: The AF routine is started by the camera.
> > - - name: AfTriggerCancel
> > - value: 2
> > - description: |
> > - The camera will cancel any active trigger and the AF routine is
> > - reset to its initial state.
> > -
> > - NoiseReductionMode:
> > type: int32_t
> > draft: true
> > @@ -507,45 +486,6 @@ controls:
> > The AE algorithm has started a pre-capture metering session.
> > \sa AePrecaptureTrigger
> >
> > - - AfState:
> > - type: int32_t
> > - draft: true
> > - description: |
> > - Control to report the current AF algorithm state. Currently identical to
> > - ANDROID_CONTROL_AF_STATE.
> > -
> > - Current state of the AF algorithm.
> > - enum:
> > - - name: AfStateInactive
> > - value: 0
> > - description: The AF algorithm is inactive.
> > - - name: AfStatePassiveScan
> > - value: 1
> > - description: |
> > - AF is performing a passive scan of the scene in continuous
> > - auto-focus mode.
> > - - name: AfStatePassiveFocused
> > - value: 2
> > - description: |
> > - AF believes the scene is in focus, but might restart scanning.
> > - - name: AfStateActiveScan
> > - value: 3
> > - description: |
> > - AF is performing a scan triggered by an AF trigger request.
> > - \sa AfTrigger
> > - - name: AfStateFocusedLock
> > - value: 4
> > - description: |
> > - AF believes has focused correctly and has locked focus.
> > - - name: AfStateNotFocusedLock
> > - value: 5
> > - description: |
> > - AF has not been able to focus and has locked.
> > - - name: AfStatePassiveUnfocused
> > - value: 6
> > - description: |
> > - AF has completed a passive scan without finding focus.
> > -
> > - AwbState:
> > type: int32_t
> > draft: true
> > @@ -690,4 +630,171 @@ controls:
> > value. All of the custom test patterns will be static (that is the
> > raw image must not vary from frame to frame).
> >
> > + - AfMode:
> > + type: int32_t
> > + draft: true
> > + description: |
> > + Control to set the mode of the AF (autofocus) algorithm.
> > + enum:
> > + - name: AfModeAuto
> > + value: 0
> > + description: |
> > + The AF algorithm is in auto mode. This means that the algorithm
> > + will never move the lens or change state unless the AfTrigger
> > + control is used. The AfTrigger control can be used to initiate a
> > + focus scan, the results of which will be reported by the AfState.
> > +
> > + In this mode, an application can move the lens for itself using
> > + the LensPosition control. If the algorithm was scanning when the
> > + lens is moved in this way, then the scan is implicitly cancelled.
>
> I would suggest splitting the Auto mode into AfModeOnDemand and
> AfModeManual (The naming could be re-considered).
> The reason is that the sub-controls for each AfModes could be distinct.
> 1. AfModeManual:
> LensPosition is valid only for the mode, and AfState is always AfStateReset.
> 2. AfModeOnDemand
> AfTrigger and AfRange are valid only for the mode, and AfState reports
> accordingly. The scan is implicitly canceled when changing from the
> mode.
I'm a little bit nervous about having a mode where the lens position
control doesn't work. In my earlier email I described performing an AF
scan, capturing an image, then moving the lens back to hyperfocal. It
seems inconvenient to have to put the AF algorithm into a separate
mode in order to move the lens, only to put it back in the mode where
a scan is once again allowed.
I don't think anything is lost by doing it this way - if an
application doesn't want to perform an AF scan it should simply not
request one.
I think AfRange applies to CAF as well. I see no reason why you might
not want to restrict the range of searches for just the same reasons
as you might do with "regular" AF.
> 3. AfModeContinuous
> AfSpeed and "pause" are valid only for the mode, and AfState reports
> accordingly. The scan is implicitly canceled when changing from the
> mode.
I'm not sure AfSpeed is *necessarily* only for CAF. I think we've done
AF systems in the past where even regular AF has a "I really want to
take the picture quickly" mode. But it could be a platform choice?
>
> > + - name: AfModeContinuous
> > + value: 1
> > + description: |
> > + The AF algorithm is in continuous mode. This means that the lens
> > + can re-start a scan spontaneously at any moment, without any user
> > + intervention. The AfState still reports whether the algorithm is
> > + currently scanning or not, though the application has no ability
> > + to initiate or cancel scans, nor move the lens for itself.
> > +
> > + The mode can be set to AfModeAuto which has the effect of
> > + "pausing" any continuous AF activity, after which it could then
> > + be moved back to AfModeContinuous to resume operation.
> > +
> > + - AfRange:
> > + type: int32_t
> > + draft: true
> > + description: |
> > + Control to set the range of focus distances that is scanned.
> > + enum:
> > + - name: AfRangeFull
> > + value: 0
> > + description: The full range of focus distances is scanned.
> > + - name: AfRangeMacro
> > + value: 1
> > + description: Only close distances are scanned.
> > + - name: AfRangeNormal
> > + value: 2
> > + description: |
> > + The full range of focus distances is scanned except for some of
> > + the closest macro positions.
> > +
> > + - AfSpeed:
> > + type: int32_t
> > + draft: true
> > + description: |
> > + Control that determines whether the AF algorithm is to move the lens
> > + as quickly as possible or more steadily. For example, during video
> > + recording it may be desirable not to move the lens too abruptly, but
> > + when in a preview mode (waiting for a still capture) it may be
> > + helpful to move the lens as quickly as is reasonably possible.
> > + enum:
> > + - name: AfSpeedFast
> > + value: 0
> > + description: Move the lens quickly.
> > + - name: AfSpeedSlow
> > + value: 0
> > + description: Move the lens more steadily.
> How about
> s/Fast/Aggressively
> s/Slow/Steadily
I agree the choice is tricky here. I think I would regard what I
called "Slow" to be the "usual" choice, so perhaps "AfSpeedNormal" is
better (and maybe make that value 0). I still prefer "AfModeFast" to
"AfModeAggressive" - "aggressive" sounds just a bit too... aggressive?
:)
But thanks for all the discussion. Glad we're making progress on all this!
Best regards
David
> > +
> > + - AfMethod:
> > + type: int32_t
> > + draft: true
> > + description: |
> > + Control whether the AF algorithm uses a single window in the image to
> > + determine the best focus position, or multiple windows simultaneously.
> > + enum:
> > + - name: AfMethodSingle
> > + value: 0
> > + description: |
> > + A single window within the image, defaulting to the centre, is used
> > + to select the best focus distance.
> > + - name: AfMethodMultiSpot
> > + value: 0
> > + description: |
> > + Multiple windows within the image are used to select the best focus
> > + distance. The best focus distance is found for each one of the
> > + windows, and then the distance that is closest to the camera is
> > + selected.
> > +
> > + - AfWindow:
> > + type: Rectangle
> > + draft: true
> > + description: |
> > + Sets the focus windows used by the AF algorithm. The units used express
> > + a proportion of the ScalerCrop control (or if unavailable, of the entire
> > + image), as u0.16 format numbers.
> > +
> > + In order to be activated, a rectangle must be programmed with non-zero
> > + width and height. If no rectangles are programmed in this way, then the
> > + system will choose its own single default window in the centre of the
> > + image.
> > +
> > + If AfMethod is set to AfMethodSingle, then only the first Rectangle in
> > + this list is used (or the system default one if it is unprogrammed).
> > +
> > + If AfMethod is set to AfMethodMultiSpot then all the valid Rectangles in
> > + this list are used. The size of the control indicates how many such
> > + windows can be programmed and will vary between different platforms.
> > +
> > + size: [platform dependent]
> > +
> > + - AfTrigger:
> > + type: int32_t
> > + draft: true
> > + description: |
> > + This control starts an autofocus scan when AfMode is set to AfModeAuto,
> > + and can also be used to terminate a scan early.
> > +
> > + It is ignored if AfMode is set to AfModeContinuous.
> > +
> > + enum:
> > + - name: AfTriggerStart
> > + value: 0
> > + description: Start an AF scan. Ignored if a scan is in progress.
> > + - name: AfTriggerCancel
> > + value: 1
> > + description: Cancel an AF scan. Ingored if no scan is in progress.
> > +
> > + - LensPosition:
> > + type: int32_t
> > + draft: true
> > + description: |
> > + Acts as a control to instruct the lens to move to a particular position
> > + and also reports back the position of the lens for each frame.
> > +
> > + The units are determined by the lens driver.
> > +
> > + If the LensPosition control is used while an AF scan is in progress,
> > + the scan is implicitly cancelled and the lens is then moved to the
> > + desired location. It is ignored if AfMode is set to AfModeContinuous.
> > +
> > + - AfState:
> > + type: int32_t
> > + draft: true
> > + description: |
> > + Reports the current state of the AF algorithm.
> > + enum:
> > + - name: AfStateReset
> > + value: 0
> > + description: |
> > + The AF algorithm reports this state when:
> > + * The camera system has just been started.
> > + * A scan has been cancelled.
> > + * The lens has been moved by the LensPosition control.
> > + - name: AfStateScanning
> > + value: 1
> > + description: |
> > + AF is performing a scan. This state can be entered spontaneously
> > + if AfMode is set to AfModeContinuous, otherwise it requires the
> > + application to use the AfTrigger control to start the scan.
> > + - name: AfStateFocused
> > + value: 2
> > + description: |
> > + An AF scan has been performed and the algorithm believes the
> > + scene is in focus.
> > + - name: AfStateFailed
> > + value: 3
> > + description: |
> > + An AF scan has been performed but the algorithm has not been able
> > + to find the best focus position.
> > +
> > ...
> > --
> > 2.30.2
> >
More information about the libcamera-devel
mailing list