[libcamera-devel] reporting of gain and analogue gain

[David Plowman]

Hi Laurent

On Sat, 10 Oct 2020 at 02:43, Laurent Pinchart
<laurent.pinchart at ideasonboard.com> wrote:
>
> Hi David,
>
> On Wed, Oct 07, 2020 at 05:15:15PM +0100, David Plowman wrote:
> > On Wed, 7 Oct 2020 at 15:58, Kieran Bingham wrote:
> > > On 07/10/2020 15:41, David Plowman wrote:
> > > > Hi everyone
> > > >
> > > > Whilst doing some maintenance on the Raspberry Pi AGC I noticed a bit
> > > > of an oversight - we have no way to report the total gain being
> > > > applied to an image. We have controls to report:
> > > >
> > > > * the exposure time of the sensor
> > > > * the analogue gain applied in the sensor
> > > > * red and blue colour gains applied by the AWB
> > > >
> > > > but nothing for any (global) digital gain demanded by the AGC. The
> > > > consequence is that when I capture a jpeg I always report an ISO value
> > > > based purely on the analogue gain which may be incorrect (when a
> > > > higher gain is required than the sensor allows then we make up the
> > > > difference with digital gain).
>
> Just to make sure there's no misunderstanding, that's the digital gain
> applied in the sensor, right ?

Sorry, I wasn't clear about that. No, I was referring to the gain
applied by the ISP.

>
> > > > Any thoughts on this? Obviously there could be a digital gain control...
> > >
> > > I assume the IPA would be then responsible for setting this? Is it
> > > read-only?
> >
> > Certainly in the Raspberry Pi implementation it would be a read-only
> > control which the IPA sets for you. You could imagine some
> > implementations would allow you to set it (the old proprietary stack
> > of ours does, which I think is a source of confusion) but we're trying
> > to keep the behaviour simple.
> >
> > > Would we expect an application to be able to control both the analogue
> > > gain and the digital gain manually? (When AGC is disabled I guess?)
> > >
> > > Or would they only see an overall 'gain' control?
> >
> > The AGC algorithm we've implemented actually treats "gain" as a single
> > thing. If the analogue gain goes high enough that's what you'll get.
> > If you want to go higher, the analogue gain will max out and the
> > remainder will be digital gain.
> >
> > > I'm suspecting that indeed, it would be exposed as a separate control,
> > > because it will expose a feature that the ISP is capable of delivering.
> >
> > Yes, probably. I think maybe I feel another patch set coming on.
> > Perhaps I'll get the other AGC ones out of the way first while we wait
> > to see if anyone else has any opinions...
>
> I can see two options, either reporting the analog and digital gains as
> separate controls, or as a combined gain control. The latter would allow
> abstracting different sensor architectures more easily, at the expense
> of not exposing the analog/digital split.
>
> Are there use cases on the application side that would benefit from
> knowing the analog/digital gain split ? And are there use cases for
> applying digital gain without maxing the analog gain out first ?

The discussion probably changes a bit since I was referring to the
digital gain in the ISP. Dealing with that one first, I think you
could expect that an application would want to distinguish between the
two. For example, when recording the "ISO" for a raw file you'd use
the analogue gain, but when recording it for a jpeg you'd have to take
both into account.

I can imagine that some AGC implementations would let you set the
digital gain in the ISP explicitly (indeed the existing Broadcom
camera stack does!) but I think that's often been a source of
confusion for us and our users and so, as part of my obsessive mission
to make the IPAs simpler, I've combined the two together.

(There'd be nothing to stop us limiting the analogue gain if we wanted
to or - here's an entertaining idea - splitting the two gains up in
the exposure profile so you could ramp them up separately, but
overall, I'm disinclined to add this unless there's a requirement.)

As you say, digital gain in the sensor is a thing too, though I can't
say that we've ever used it much (maybe ever?) ourselves. It's
certainly more reasonable to sweep that into the analogue gain value,
I'd have thought, as it would apply to raw files too. Exposing that to
application control is perfectly reasonable, I guess, though I think
in Pi-world we would want to "opt out" of extra complexity like that!

Thanks
David

>
> --
> Regards,
>
> Laurent Pinchart