[libcamera-devel] How to deal with "atomic sequences" of AF controls?

[Nick Hollinghurst]

Hi Umang,

On Fri, 25 Nov 2022 at 10:47, Umang Jain <umang.jain at ideasonboard.com> wrote:
>
> Hi Nick,
>
> On 11/25/22 4:01 PM, Nick Hollinghurst via libcamera-devel wrote:
> > Hi all,
> >
> > I'm having some difficulty with the definitions of Autofocus controls
> > (perhaps the question can be generalized to other groups of controls).
> > Consider these example sequences of control writes:
> >
> >   1. set LensPosition
> >   2. switch to Continuous mode
> >
> >   1. switch to Auto mode
> >   2. trigger a scan
> >
> >   1. switch to Manual mode
> >   2. set LensPosition
> >
> > For each pair, the order is important -- it would not work if
> > reversed. It's entirely reasonable that an application might want to
> > send such a sequence quickly, ideally atomically. And that's a problem
> > because the order of controls in a ControlList is not well defined.
>
> Can you explain a bit more on 'order of controls in a ControlList'?
>
> Does it mean sending controls with sequential queuing to requests? For .e.g
>
> Request 1 queued with - ControlList A
> Request 2 queued with - ControlList B
> Request 3 queued with - ControlList C
>
> So A -> B -> C is the order of Controls sent to the camera?

I'm referring to the case where more than one AF control is set in the
same request (all in Request 1 - ControlList A). In our IPA, controls
are enumerated in raspberrypi.cpp IPARPi::queueRequest(). The order in
which each individual control is visited by the loop is undefined. It
means that the same ControlList could have different effects in
different builds, or change randomly over time. This appears to be
unchanged in the patched version.

> >
> > An implementation could (and probably should) use heuristics to ensure
> > such combinations of controls are interpreted in the "most useful"
> > order. But in the absence of guidance from the specification, it seems
> > an application could never rely on this working!
>
> I was looking at one of the autofocus implementation just yesterday.  If
> I understand correctly, you are askng about AF states?
>
> There is a out-of-tree AF algorithm series from our (IoB) side which
> seems to implement/handle AF states [1].
>
> The state machine can be validated using a capture script as included in
> [10/10]. So I believe that's what are you trying to do i.e. simulating a
> use case where the state / controls queued doesn't behave as expected.
>
> [1] https://patchwork.libcamera.org/project/libcamera/list/?series=3174

I agree that putting each control in a separate request would fix
their order. It would also slow down the application, as operations
(like the examples above) that are intended to be atomic, would have
to be spaced out over multiple frame-intervals. A specified ordering
for multiple controls within the same ControlList, could help here.

> >
> > Is this something that needs addressing either in documentation, or by
> > some kind of change to the API?
>
> My opinion is that, if something is not logical and is not valid by the
> state machine - I think they should be documented. I haven't looked deep
> in AF controls specifically, so let's wait for others to pitch in on this!
> >
> > Regards,
> >
> >   Nick
>