[libcamera-devel] How to deal with "atomic sequences" of AF controls?

[Nick Hollinghurst]

Hi Umang,

On Fri, 25 Nov 2022 at 11:50, Umang Jain <umang.jain at ideasonboard.com> wrote:
>
> Hi Nick,
>
> On 11/25/22 4:47 PM, Nick Hollinghurst wrote:
> > Hi Umang,
> >
> > On Fri, 25 Nov 2022 at 10:47, Umang Jain <umang.jain at ideasonboard.com> wrote:
> >> Hi Nick,
> >>
> >> On 11/25/22 4:01 PM, Nick Hollinghurst via libcamera-devel wrote:
> >>> Hi all,
> >>>
> >>> I'm having some difficulty with the definitions of Autofocus controls
> >>> (perhaps the question can be generalized to other groups of controls).
> >>> Consider these example sequences of control writes:
> >>>
> >>>    1. set LensPosition
> >>>    2. switch to Continuous mode
> >>>
> >>>    1. switch to Auto mode
> >>>    2. trigger a scan
> >>>
> >>>    1. switch to Manual mode
> >>>    2. set LensPosition
> >>>
> >>> For each pair, the order is important -- it would not work if
> >>> reversed. It's entirely reasonable that an application might want to
> >>> send such a sequence quickly, ideally atomically. And that's a problem
> >>> because the order of controls in a ControlList is not well defined.
> >> Can you explain a bit more on 'order of controls in a ControlList'?
> >>
> >> Does it mean sending controls with sequential queuing to requests? For .e.g
> >>
> >> Request 1 queued with - ControlList A
> >> Request 2 queued with - ControlList B
> >> Request 3 queued with - ControlList C
> >>
> >> So A -> B -> C is the order of Controls sent to the camera?
> > I'm referring to the case where more than one AF control is set in the
> > same request (all in Request 1 - ControlList A). In our IPA, controls
>
> Glad I asked for this clarification beforehand.
>
>
> > are enumerated in raspberrypi.cpp IPARPi::queueRequest(). The order in
> > which each individual control is visited by the loop is undefined. It
> > means that the same ControlList could have different effects in
> > different builds, or change randomly over time. This appears to be
> > unchanged in the patched version.
>
> Let's back up a little bit. You are trying to set, 3 different modes for
> the [control::AfMode] in the same control list and figuring out the
> order in which they'll be parsed by ?

Ah, my apologies. I meant them as three separate examples, each one
containing two different-but-dependent controls in a ControlList.

Now I understand the ordering within a ControlList -- In fact it's
only an issue at all if you think of the state machine (as I was
doing) in terms of serialized events, rather than composite ones.

Thanks!

 Nick


>
> I don't see how it can be so without the the control mode being
> overwritten (since single control list). So only the last one that you
> set, shall be reflected in the control list, no?
> >
> >>> An implementation could (and probably should) use heuristics to ensure
> >>> such combinations of controls are interpreted in the "most useful"
> >>> order. But in the absence of guidance from the specification, it seems
> >>> an application could never rely on this working!
> >> I was looking at one of the autofocus implementation just yesterday.  If
> >> I understand correctly, you are askng about AF states?
> >>
> >> There is a out-of-tree AF algorithm series from our (IoB) side which
> >> seems to implement/handle AF states [1].
> >>
> >> The state machine can be validated using a capture script as included in
> >> [10/10]. So I believe that's what are you trying to do i.e. simulating a
> >> use case where the state / controls queued doesn't behave as expected.
> >>
> >> [1] https://patchwork.libcamera.org/project/libcamera/list/?series=3174
> > I agree that putting each control in a separate request would fix
> > their order. It would also slow down the application, as operations
> > (like the examples above) that are intended to be atomic, would have
> > to be spaced out over multiple frame-intervals. A specified ordering
> > for multiple controls within the same ControlList, could help here.
> >
> >>> Is this something that needs addressing either in documentation, or by
> >>> some kind of change to the API?
> >> My opinion is that, if something is not logical and is not valid by the
> >> state machine - I think they should be documented. I haven't looked deep
> >> in AF controls specifically, so let's wait for others to pitch in on this!
> >>> Regards,
> >>>
> >>>    Nick
>