[libcamera-devel] [RFC PATCH 0/1] Autofocus controls

[Hanlin Chen]

Hi David, Thanks for the work!


On Mon, Dec 13, 2021 at 11:22 PM David Plowman
<david.plowman at raspberrypi.com> wrote:
>
> Hi everyone
>
> I thought it was time for a slightly more formal API proposal for AF
> (autofocus).
>
> I've sent a patch listing all the new controls that I think we would
> need. These controls cover both the use-cases that I can imagine we
> (Raspberry Pi) would expect, and I think it covers all the Android
> use-cases too. Nonetheless, I think a few words of additional
> discussion and questions are warranted.
>
>
> 1. AfState
>
> This closely follows what Android does, only Android duplicates them
> all, once for "passive" (aka. continuous) AF and once for "active"
> (user-triggered) AF. I can't really see why duplicating them would be
> necessary, it seems pretty clear to me what mode we're in, but of
> course we could if there is a good reason.
I think it's related to the "pause" control mentioned below.

>
> I wondered a bit about having an "Error" state. Maybe that would get
> reported if the AfWindow rectangles are ill-defined?
>
> 2. AfWindow
>
> Allowing applications to set the AF windows seems required. For
> example, you might be running some kind of object or face detector,
> and then you want to feed those results back to the AF algorithm.
>
> I was unsure what units to use for the Rectangles. It seemed to me
> that you'd want the rectangles automatically to follow whatever
> ScalerCrop you are using. For example, doing digital zoom and leaving
> the focus windows in a part of the image you can't see seems
> particularly unhelpful!
>
> So anyway, I went with the units being "a proportion of the ScalerCrop
> region". Can we think of something better? Would floating point
> Rectangles be less annoying?
>
> 3. AfMode
>
> As you probably already know, I'm not really seeing the need for an
> "off" mode as it seems the same as "auto" to me - so I didn't define
> one.
>
> One thing I wondered about was "pausing" CAF (continuous AF). You
> might want to let it run, then "pause" for a moment to take some
> captures, and then let it re-start from where it was previously. How
> to accomplish this?
>
> We could have a separate "pause" control, just for CAF. Or I've
> suggested that you could set the mode back to "auto" which would have
> the same effect. But then how would you go back to CAF "from where it
> left off" as opposed to letting it re-start as though it wasn't
> running before? Maybe the "pause" control isn't such a bad idea.

I would vote for the "pause" control. I tried to map the combination
of pause and AfState with Android's definition.
It describes the "passive" cases.

PASSIVE_SCAN = CAF + AfStateScanning
PASSIVE_FOCUSED = CAF + AfStateFocused
PASSIVE_UNFOCUSED = CAF + AfStateFailed

ACTIVE_SCAN = CAF + AfStateScanning + pause on
FOCUSED_LOCKED = CAF + AfStateFocused + paused
NOT_FOCUSED_LOCKED = CAF + AfStateFailed + paused

The difference between PASSIVE_FOCUSED and FOCUSED_LOCKED are:
PASSIVE_FOCUSED tells the user "Although it's focused, it may change
anytime due to CAF".
FOCUSED_LOCKED tell the user "It's focused, and won't change unless
the user stop pausing even in CAF mode"

Android use the combination to describe "pause":
CAF + TriggerStart = pause on
CAF + TriggerCancel = pause off

I notice that the TriggerStart is forbidden for CAF in the Patch, but
I think maybe "pause" is a more clear definition for CAF.
And the user doesn't need to do the trick by changing modes.

Another thing I'd like to notice is that:
"CAF + TriggerStart" in Android is not going to abruptly "pause" if
CAF is already scanning, but wait for the scan to finish, and then
pause.

>
> 4. LensPosition
>
> I've proposed using the lens driver's units for controlling the lens
> position. It's easy and naturally mimics what we tend to do for
> exposure/gain.
>
> Another option would be to go a bit more Android-y and insist that
> zero means "infinity". Android seems to categorise lenses into
> "calibrated" and "uncalibrated" types. For the latter type, it looks
> like pretty much the only stipulation is that zero means
> "infinity". For "calibrated" lenses the units are formally defined to
> be dioptres, but I'm thinking that we don't need to go there right
> now.
>
> There's also the question of exactly where we get the lens range from.
> It will be different for each module, even ones using the same lens
> and/or driver - it will in each case be a different subset of the
> range that the driver advertises. But I think that's not a question
> for this control API.
>
>
> Hopefully all that's enough to start taking this forward. There are of
> course lots of points of detail to think about here, but clearly the
> "big picture" is most important in the first instance.
>
> Thanks!
>
> David
>
> David Plowman (1):
>   libcamera: controls: Controls for driving AF (autofocus) algorithms
>
>  src/libcamera/control_ids.yaml | 227 ++++++++++++++++++++++++---------
>  1 file changed, 167 insertions(+), 60 deletions(-)
>
> --
> 2.30.2
>