[libcamera-devel] [PATCH v2 1/3] libcamera: controls: Add frame duration control

[Jacopo Mondi]

Hi Naush,

On Wed, Jun 03, 2020 at 10:44:18AM +0100, Naushir Patuck wrote:
> Hi Jacopo,
>
> On Wed, 3 Jun 2020 at 09:42, Jacopo Mondi <jacopo at jmondi.org> wrote:
> >
> > Hi Naush,
> >
> >     I won't go into details here as I would like Laurent to chime in
> > and express his view on this topic, but I would have a question in the
> > meantime.
> >
> > On Tue, Jun 02, 2020 at 10:38:41AM +0100, Naushir Patuck wrote:
> > > Hi Jacopo,
> > >
> > >
> > > On Mon, 1 Jun 2020 at 16:40, Jacopo Mondi <jacopo at jmondi.org> wrote:
> > > >
> > > > Hi Naush,
> > > >
> > > > On Thu, May 28, 2020 at 09:52:51AM +0100, Naushir Patuck wrote:
> >
> > [snip]
> >
> > > > > > > On "FrameDuration" itself: would it make sense to only considered it
> > > > > > > when running with AE off ? In that case it would be applications
> > > > > > > responsibility to opportunely calculate the exposure time and the frame
> > > > > > > duration, but I think it's expected if you run in manual mode. And for
> > > > > > > reference, that's what Android does as well :)
> > > > > > >
> > > > > > > It would also rule out the need to specify how the frameDurationLimits
> > > > > > > would be used when running in manual mode and wanting to configure a
> > > > > > > precise value there, as I guess you would have to give to min and max
> > > > > > > the same value to express that.
> > > > >
> > > > > FrameDurationLimits should only really apply with AE enabled.  AE
> > > > > varies the shutter speed all over the place, that could cause
> > > > > framerate changes that we need to clip.  If AE is disabled and we have
> > > > > a manual shutter speed set, that dictates the frame duration, and any
> > > > > limits will not apply.  Perhaps I should add this in the description?
> > > > >
> > >
> > > I have to correct myself here.  FrameDurationLimits will also apply
> > > with manual shutter speeds when AE is disabled.  We could, say, have a
> > > manual shutter speed request of 25 ms, but still want to run at 30fps.
> > > There is a question on what happens if they clash, e.g. request a
> > > manual shutter speed of 66ms, and a FrameDuration of 30fps?  What gets
> > > prioritised?  Currently, in our implementation, FrameDuration will
> > > always be prioritised.
> >
> > Is there a rationle behind this decision ? I'm asking as I see Android
> > going in the other direction, with the exposure time always overriding
> > the requested frame duration.
>
> This was an entirely arbitrary choice on my part - based on what was
> easiest to do at the time :)
>
> > It is specified in their documentation that the mximum frame duration
> > a camera reports (lower frame rate) shall be at least the maximum
> > allowed exposure time, so that all possible exposure values are known
> > to be in the requested FPS range. I guess this should be enforced even
> > if we give frame duration priority, or should we cap the maximum frame
> > duration to the maximum available exposure time ?
>
> I have not looked at the Android docs in ages, but do they have two
> sets of frame duration limits, one that the sensor will report (sounds
> like this is what you are working on?) and one that the user requests

yes, there's a static property that reports the minFrameDurations for
each supported stream configuration (including the ISP introduced
dealys)
https://jmondi.org/android_metadata_tags/docs.html#static_android.scaler.availableMinFrameDurations

And two controls for the sensor frame durations, one static property
https://jmondi.org/android_metadata_tags/docs.html#static_android.sensor.info.maxFrameDuration

and a control
https://jmondi.org/android_metadata_tags/docs.html#controls_android.sensor.frameDuration

> (of course, bounded by the former)?  This was where my thinking was
> going, and with frame duration priority, exposure time is capped by
> the user request (if given), or the sensor limits.  But as always, if
> the consensus is to behave differently, I am happy to change things
> around.

I don't have preferences, nor reasons that make me think one is better
than the other. I wonder if we should enforce this at all or let pipeline
handler/IPA decide that.

Thanks
  j

>
> Regards,
> Naush
>
>
> >
> > Just trying to get more data points to make an informed decision, as
> > your question on what has to be prioritized is quite sensible...
> >
> > Thanks
> >   j