[libcamera-devel] AeLock proposal v4

[David Plowman]

Hi Paul

Thanks for the summary, I think it sounds good. Just one thing I
wanted to add, which was the question of a "global" AE auto/disable
control. We did discuss this briefly, though I think the jury remained
undecided. There are a couple of reasons why this might be useful:

1. If an application wants to "lock everything" it's slightly annoying
to have to lock each lever in turn.

2. As time goes on we might get sensors with new levers (e.g.
aperture, filters). This would provide a "forwards compatible" way of
"locking everything".

Finally one might wonder about the implementation. You could have a
new "super control" that implicitly sets/resets the individual
controls, or perhaps you could have a single bit-field type control
for everything. Obviously there are some details that need further
consideration.

It feels like something that application writers would probably like
to have in some form, but am interested to hear other people's
thoughts.

Thanks!
David

On Thu, 16 Sept 2021 at 09:46, Kieran Bingham
<kieran.bingham at ideasonboard.com> wrote:
>
> On 16/09/2021 04:37, paul.elder at ideasonboard.com wrote:
> > Hi Kieran,
> >
> > On Thu, Sep 16, 2021 at 12:02:45AM +0100, Kieran Bingham wrote:
> >> Hi Paul,
> >>
> >> On 15/09/2021 09:56, paul.elder at ideasonboard.com wrote:
> >>> Hello,
> >>>
> >>> Here's v4 of the aelock proposal, based on what we discussed in the
> >>> meeting yesterday.
> >>>
> >>> Requirements:
> >>> - LOCK: AGC do not update output results
> >>> - MANUAL: override the values computed by AGC
> >>> - SEAMLESS TRANSITIONS: no flickering
> >>> - custom minimum values for the manual controls (no magic value for
> >>>   enabling auto)
> >>>
> >>> The conclusion was that the locked state, as an explicitly selectable
> >>> state for the user, is unnecessary. Its presence causes not-very-nice
> >>> cases (eg. manual -> locked transition, manual with no values, etc), so
> >>> we've decided to turn it into an implicit state combined with manual,
> >>> with the presence of a manual control value to act as a trigger to
> >>> switch from the internal "locked" state to "manual" state. This blocks
> >>> going the other way, but allows going back to auto.
> >>>
> >>> We of course have separate controls for the sub-controls.
> >>>
> >>> In which case the design is so simple that we don't need a diagram...?
> >>>
> >>> Controls:
> >>>
> >>> ExposureTime
> >>> - input: the manual exposure time value to use
> >>> - output: the exposure time value that was used for the request that is
> >>>   returned
> >>>
> >>> ExposureTimeMode:
> >>> - states: Auto, Disabled
> >>> - input: In Auto mode use the exposure time value computed by the AEC.
> >>>   In Disabled mode, don't update the exposure time value until
> >>>   ExposureTime is provided. When it is, start to an continue to use the
> >>>   provided value.
> >>
> >> This implies that the 'last setting from auto' will persist?
> >> So to implement the 'lock' options we've seen in Android - it sounds
> >> like we can just move from Auto ExposureTimeMode to Disabled
> >> ExposureTimeMode?
> >
> > Yeah, that's how it'll work.
> >
> >>
> >>
> >>> - output: the mode that was used for the request that is returned
> >>>
> >>> Same two controls for AnalogGain.
> >>>
> >>>
> >>> The flow for seamless switching from auto to manual:
> >>> 1. Set ExposureTimeMode to Disabled
> >>> 2. Wait for the first request where ExposureTimeMode is Disabled
> >>> 3. Copy the ExposureTime value from that request, and submit it
> >>
> >> Is this from the perspective of the application or an IPA?
> >>
> >> (I assume Application?)
> >
> > Yeah.
> >
> >>
> >> But in that case, I would presume they would be able to / expect to set
> >> ExposureTimeMode to Disabled (This really sounds like 'Manual' mode to
> >> me?) and set the ExposureTime value that they want ...
> >
> > If they do that then they skip the internal locked mode, which is fine
> > if that's what they want.
>
> If they know the expected manual setting to apply then indeed.
>
>
>
>
> >> Why is the 'waiting' necessary? - or indeed the copying of a value from
> >> that request to resubmit it ? Surely resubmitting the value from that
> >> request, is the same value it would stay on anyway?
> >>
> >> Perhaps I haven't grasped the 'requirement' for 'Seamless'.
> >
> > The waiting is necessary so that you're guaranteed that the exposure
> > time value that you copied from the result won't change. If it's not
> > "locked", then the value that you copied could be changed in the
> > requests that are still in-flight that still have auto on.
>
> Ah yes, I see. So the app will know that it only 'settled' when the
> existing requests with 'auto' were flushed through. And I think that is
> how it would be expected to operate from the application side anyway.
>
> They know that they queued requests previously, and those requests were
> on Auto - so those frames are expected to complete as auto.
>
>
> I'd call this a 'queued' control vs an 'immediate' control, as a
> distinction. But I can't think of any controls that would be 'immediate'
> right now anyway.
>
> Is there any control that would be expected to be set, and the effect be
> visible on frames/requests that had already been queued?
>
>
>
>
> >> I guess the process above is to allow the app to 'know' what value the
> >> 'last' automatic setting provided. (I.e. the first time a request comes
> >> back with ExposureTimeMode as Disabled, that request control metadata
> >> tells the app what the exposure was at that point, and then it's up to
> >> the app to make changes from there.
> >
> > Yeah, that and there's the guarantee that the exposure time won't change
> > once it's set to disabled.
>
>
> Ok.
>
>
> >>> If a manual value is provided while the mode is auto, the manual value
> >>> will be ignored.
> >>>
> >>>
> >>> The one thing that deviates from the consistency is that a value-control
> >>> is used as a trigger. Its value is still saved (thus resubmission is
> >>> unnecessary if the value doesn't change), but its presence acts as a
> >>> trigger to switch from the internal locked state to manual state. This
> >>> is seen as fine, since it's not too intrusive, and it enables a cleaner
> >>> API.
> >>
> >> Sounds fairly good to me so far.
> >> Much easier to digest anyway ;-)
> >
> > \o/
> >
> >>
> >> Setting ExposureTimeMode=Disabled(/Manual) without giving a value, I
> >> would expect it to 'stay as it was, whatever the last auto value
> >> calculated was'.
> >>
> >> So indeed that's the same as an internal 'lock'.
> >
> > Yep, that's the idea :D
>
> Sounds good. To me Manual/Disabled is always 'locked' it's either locked
> to the last auto value that was calculated ... or if a manual value is
> provided it's locked to that.
>
> So it fits the requirements that I've understood well.
>
>
>
> > Thanks,
> >
> > Paul
> >