[libcamera-devel] Initial settings for camera exposure and analogue gain

[Jacopo Mondi]

Hi Naush,

On Thu, Jun 11, 2020 at 08:41:27AM +0100, Naushir Patuck wrote:
> Hi Jacopo,
>
> On Wed, 10 Jun 2020 at 17:06, Jacopo Mondi <jacopo at jmondi.org> wrote:
> >
> > Hi David,
> >
> > On Wed, Jun 10, 2020 at 02:22:59PM +0100, David Plowman wrote:
> > > Hi Jacopo
> > >
> > > Thanks for the feedback.
> > >
> > > On Wed, 10 Jun 2020 at 13:23, Jacopo Mondi <jacopo at jmondi.org> wrote:
> > > >
> > > > Hi David,
> > > >
> > > > On Tue, Jun 09, 2020 at 03:00:49PM +0100, David Plowman wrote:
> > > > > Hi
> > > > >
> > > > > I'd like to discuss the question of being able to set a sensor's initial
> > > > > exposure time and analogue gain before the sensor is started by calling the
> > > > > Camera::start method. Currently I believe this is not possible; they can
> > > > > only be adjusted in the ControlList passed with a Request.
> > > > >
> > > >
> > > > For application-supplied controls I think you're right. IPAs can
> > > > pre-configure sensors I guess. If I'm not mistaken your IPA protocol
> > > > has a RPI_IPA_ACTION_SET_SENSOR_CONFIG event, which is be used to
> > > > pre-configure sensor with known delay values, but that works at
> > > > start-up time only...
> > > >
> > > > > This would be a helpful feature, for example when switching from preview to
> > > > > capture you might want to use different exposure/gain combinations. Or for
> > > > > capturing multiple exposures - you could straight away get the frame you
> > > > > want without waiting a couple of frames for the camera to start streaming
> > > > > with the wrong values and then change.
> > > > >
> > > > > As regards an API, we were thinking in terms of adding a "ControlList const
> > > > > &controls" parameter to the Camera::start method, which would forward the
> > > > > controls to the pipeline handler prior to the camera actually being
> > > > > started. (We're happy to work on the feature once there's sufficient
> > > > > consensus.)
> > > >
> > > > We started planning support for the same feature, with the idea to
> > > > support Android's session parameters
> > > > https://source.android.com/devices/camera/session-parameters
> > > >
> > > > When we (briefly) discussed this, we were considering providing a
> > > > ControlList at Camera:;configure() time, probably adding it to the
> > > > CameraConfiguration parameter. Thinking out loud, this would allow,
> > > > in example, to return a set of pre-configured control sets from
> > > > Camera::generateConfiguration() depending on the requested stream role
> > > > and yet-to-be-designed capture profile.
> > > >
> > > > Why would you prefer to do at start() time ? That's certainly possible
> > > > if there's good reasons to do so...
> > > >
> > >
> > > As you say, having a ControlList in the configure() method would allow
> > > control of these parameters whenever the camera mode has to be
> > > changed, but not when it is started. I think multi-exposure/image-fusion is
> > > a use-case where this would be helpful.
> > >
> > > For example, suppose we've been running preview and have decided
> > > that we're going to use 2 exposures to capture an HDR image. The
> > > sequence of events in an application might then be:
> > >
> > > 1. Stop the preview and teardown the preview configuration
> > > 2. Configure for still capture mode.
> > > 3. Write the short exposure and start the camera.
> > > 4. Grab the first frame that comes back and immediately stop the camera.
> > > 5. Write the long exposure and start the camera.
> > > 6. Grab the first frame again, and immediately stop the camera.
> > > 7. Now images can be fused and HDR processing performed...
> > >
> > > This would require a ControlList in the start() method (or equivalent).
> > > I appreciate there are other ways to do this, but it seems a particularly
> > > straightforward template that applications might wish to adopt.
> >
> > Surely not my strongest expertise, but if you asked me I would have
> >
> > 1. Stop the preview and teardown the preview configuration
> > 2. Configure the camera for still capture with an initial short exposure time
> > 3. Start the camera
> > 4. Queue a request with the new long exposure value
> > 5. Grab the first frame that comes back which will have the initially
> > programmed short exposure time
> > 6. Continue capturing frames until the desired long exposure time is
> > not locked
> > 7. Use the two images for HDR/fusion whatever.
> >
> > I know I'm probably missing something, so consider this exercise
> > useful for my own education :)
> >
>
> The above steps could indeed be used for HDR captures.  It's a toss-up
> as to which is more efficient - starting/stopping the camera over
> multiple exposures or letting it run on but having to wait (generally
> 2 frames) before the sensor will consume the next exposure values.
>
> There may be other reasons (I cannot think of anything specific apart
> from HDR right now) where an application would want to sequence
> start(), stop(), start() and have the last start() take in some new
> ControlList parameters.  If the ControlList was passed into
> configure(), the sequence would have to run like start(), stop(),
> configure(), start(), which might be less efficient.  Hence having the
> ControlList passed into start() would cover all use cases.

Indeed not having to go through configure() is more efficient, and my
reasoning and above example is not actually accurate, as if you're
running a (viewfinder+still capture), at configure() time you would
have instructed the camera to run with an exposure time ideal for
viewfinder, not the 'short exposure' I mentioned in step 2.

I would not rule out the option to pass parameters to configure(), but
I see your point and to me it's something worth exploring. What do
others think ?

Thanks
  j

>
> Regards,
> Naush