[libcamera-devel] AeLock proposal v4

[paul.elder at ideasonboard.com]

Hi David,

On Thu, Sep 16, 2021 at 11:24:21AM +0100, David Plowman wrote:
> Hi Paul
> 
> Thanks for the summary, I think it sounds good. Just one thing I
> wanted to add, which was the question of a "global" AE auto/disable
> control. We did discuss this briefly, though I think the jury remained
> undecided. There are a couple of reasons why this might be useful:
> 
> 1. If an application wants to "lock everything" it's slightly annoying
> to have to lock each lever in turn.
> 
> 2. As time goes on we might get sensors with new levers (e.g.
> aperture, filters). This would provide a "forwards compatible" way of
> "locking everything".

That's true. I guess there is merit in a global AE switch.

> 
> Finally one might wonder about the implementation. You could have a
> new "super control" that implicitly sets/resets the individual

Changing other controls isn't very nice...

> controls, or perhaps you could have a single bit-field type control

That means that the global control will end up containing all the
sub-levers...

> for everything. Obviously there are some details that need further
> consideration.

In v1 of my proposal I had a tri-state global lever, that had
"full-auto", "full-manual", and "semi-manual". It acts as a mux to the
output of the sub-controls. Now that I think about it it won't quite
work though, unless it's an override. I guess I gotta go brainstorm how
this would work :/

> 
> It feels like something that application writers would probably like
> to have in some form, but am interested to hear other people's
> thoughts.

Yeah I think so too tbh.


Thanks,

Paul

> 
> On Thu, 16 Sept 2021 at 09:46, Kieran Bingham
> <kieran.bingham at ideasonboard.com> wrote:
> >
> > On 16/09/2021 04:37, paul.elder at ideasonboard.com wrote:
> > > Hi Kieran,
> > >
> > > On Thu, Sep 16, 2021 at 12:02:45AM +0100, Kieran Bingham wrote:
> > >> Hi Paul,
> > >>
> > >> On 15/09/2021 09:56, paul.elder at ideasonboard.com wrote:
> > >>> Hello,
> > >>>
> > >>> Here's v4 of the aelock proposal, based on what we discussed in the
> > >>> meeting yesterday.
> > >>>
> > >>> Requirements:
> > >>> - LOCK: AGC do not update output results
> > >>> - MANUAL: override the values computed by AGC
> > >>> - SEAMLESS TRANSITIONS: no flickering
> > >>> - custom minimum values for the manual controls (no magic value for
> > >>>   enabling auto)
> > >>>
> > >>> The conclusion was that the locked state, as an explicitly selectable
> > >>> state for the user, is unnecessary. Its presence causes not-very-nice
> > >>> cases (eg. manual -> locked transition, manual with no values, etc), so
> > >>> we've decided to turn it into an implicit state combined with manual,
> > >>> with the presence of a manual control value to act as a trigger to
> > >>> switch from the internal "locked" state to "manual" state. This blocks
> > >>> going the other way, but allows going back to auto.
> > >>>
> > >>> We of course have separate controls for the sub-controls.
> > >>>
> > >>> In which case the design is so simple that we don't need a diagram...?
> > >>>
> > >>> Controls:
> > >>>
> > >>> ExposureTime
> > >>> - input: the manual exposure time value to use
> > >>> - output: the exposure time value that was used for the request that is
> > >>>   returned
> > >>>
> > >>> ExposureTimeMode:
> > >>> - states: Auto, Disabled
> > >>> - input: In Auto mode use the exposure time value computed by the AEC.
> > >>>   In Disabled mode, don't update the exposure time value until
> > >>>   ExposureTime is provided. When it is, start to an continue to use the
> > >>>   provided value.
> > >>
> > >> This implies that the 'last setting from auto' will persist?
> > >> So to implement the 'lock' options we've seen in Android - it sounds
> > >> like we can just move from Auto ExposureTimeMode to Disabled
> > >> ExposureTimeMode?
> > >
> > > Yeah, that's how it'll work.
> > >
> > >>
> > >>
> > >>> - output: the mode that was used for the request that is returned
> > >>>
> > >>> Same two controls for AnalogGain.
> > >>>
> > >>>
> > >>> The flow for seamless switching from auto to manual:
> > >>> 1. Set ExposureTimeMode to Disabled
> > >>> 2. Wait for the first request where ExposureTimeMode is Disabled
> > >>> 3. Copy the ExposureTime value from that request, and submit it
> > >>
> > >> Is this from the perspective of the application or an IPA?
> > >>
> > >> (I assume Application?)
> > >
> > > Yeah.
> > >
> > >>
> > >> But in that case, I would presume they would be able to / expect to set
> > >> ExposureTimeMode to Disabled (This really sounds like 'Manual' mode to
> > >> me?) and set the ExposureTime value that they want ...
> > >
> > > If they do that then they skip the internal locked mode, which is fine
> > > if that's what they want.
> >
> > If they know the expected manual setting to apply then indeed.
> >
> >
> >
> >
> > >> Why is the 'waiting' necessary? - or indeed the copying of a value from
> > >> that request to resubmit it ? Surely resubmitting the value from that
> > >> request, is the same value it would stay on anyway?
> > >>
> > >> Perhaps I haven't grasped the 'requirement' for 'Seamless'.
> > >
> > > The waiting is necessary so that you're guaranteed that the exposure
> > > time value that you copied from the result won't change. If it's not
> > > "locked", then the value that you copied could be changed in the
> > > requests that are still in-flight that still have auto on.
> >
> > Ah yes, I see. So the app will know that it only 'settled' when the
> > existing requests with 'auto' were flushed through. And I think that is
> > how it would be expected to operate from the application side anyway.
> >
> > They know that they queued requests previously, and those requests were
> > on Auto - so those frames are expected to complete as auto.
> >
> >
> > I'd call this a 'queued' control vs an 'immediate' control, as a
> > distinction. But I can't think of any controls that would be 'immediate'
> > right now anyway.
> >
> > Is there any control that would be expected to be set, and the effect be
> > visible on frames/requests that had already been queued?
> >
> >
> >
> >
> > >> I guess the process above is to allow the app to 'know' what value the
> > >> 'last' automatic setting provided. (I.e. the first time a request comes
> > >> back with ExposureTimeMode as Disabled, that request control metadata
> > >> tells the app what the exposure was at that point, and then it's up to
> > >> the app to make changes from there.
> > >
> > > Yeah, that and there's the guarantee that the exposure time won't change
> > > once it's set to disabled.
> >
> >
> > Ok.
> >
> >
> > >>> If a manual value is provided while the mode is auto, the manual value
> > >>> will be ignored.
> > >>>
> > >>>
> > >>> The one thing that deviates from the consistency is that a value-control
> > >>> is used as a trigger. Its value is still saved (thus resubmission is
> > >>> unnecessary if the value doesn't change), but its presence acts as a
> > >>> trigger to switch from the internal locked state to manual state. This
> > >>> is seen as fine, since it's not too intrusive, and it enables a cleaner
> > >>> API.
> > >>
> > >> Sounds fairly good to me so far.
> > >> Much easier to digest anyway ;-)
> > >
> > > \o/
> > >
> > >>
> > >> Setting ExposureTimeMode=Disabled(/Manual) without giving a value, I
> > >> would expect it to 'stay as it was, whatever the last auto value
> > >> calculated was'.
> > >>
> > >> So indeed that's the same as an internal 'lock'.
> > >
> > > Yep, that's the idea :D
> >
> > Sounds good. To me Manual/Disabled is always 'locked' it's either locked
> > to the last auto value that was calculated ... or if a manual value is
> > provided it's locked to that.
> >
> > So it fits the requirements that I've understood well.
> >
> >
> >
> > > Thanks,
> > >
> > > Paul
> > >