[libcamera-devel] How to activate AGC control in libcamera, or set the GAIN/EXPOSURE parameter by v4l2 command.

[David Plowman]

Hi Nomura-san

Thank you for your questions.

Perhaps you can explain more precisely how you can tell the AGC is not
running - is there perhaps some parameter you are measuring, or is it
a visual assessment of the image?

On the Raspberry Pi AEC and AGC run together, you can't really have
one without the other. Exposure time and analogue gain are divided
according to an "exposure profile". Please read here
https://www.raspberrypi.org/documentation/linux/software/libcamera/rpi_SOFT_libcamera_1p0.pdf
(section 5.8) for more information.

I don't recall without checking whether the jpegs produced by qcam
record the exposure time or gain, but if you capture DNG files you
should certainly see both exposure time and ISO changing according to
the scene brightness.

I hope that helps.

Best regards
David

On Mon, 28 Sep 2020 at 14:46, Kieran Bingham
<kieran.bingham at ideasonboard.com> wrote:
>
> Hi Nomura-san,
>
> I've added David on CC as he might have some insights,
> (I hope you don't mind David)
>
>
> On 28/09/2020 12:55, tetsuya.nomura at soho-enterprise.com wrote:
> > Dear Kieran-san
> >
> > Thank you very much for your prompt reply.
> >
> > I'm sorry I wrote wrong, though this is the most important point.
> > I can see "Auto Exposure Control" is working, and I can see the exposure
> > parameter is reaching to the maximum value.
> > But after that, I don't see the noise level never increase, this means the
> > "Auto Gain Control" does not work in qcam.
> > I understand the qcam at this moment does not support AGC.
>
> Ok, so that sounds slightly different. The AGC is handled as part of the
> Raspberry Pi pipeline handler, and that is a lower layer than QCam ...
> Qcam is just displaying the results.
>
> It sounds like you are saying that once the exposure has reached its'
> maximum, you would expect the gain control to start increasing... (and
> you have observed that it is not).
>
>
>
> > I write the comment below.
> >
> > Best Regards,
> >
> > NOMURA
> > -----Original Message-----
> > From: Kieran Bingham <kieran.bingham at ideasonboard.com>
> > Sent: Monday, September 28, 2020 8:00 PM
> > To: tetsuya.nomura at soho-enterprise.com; libcamera-devel at lists.libcamera.org
> > Subject: Re: [libcamera-devel] How to activate AGC control in libcamera, or
> > set the GAIN/EXPOSURE parameter by v4l2 command.
> >
> > Hello Nomura-san,
> >
> > On 28/09/2020 10:50, tetsuya.nomura at soho-enterprise.com wrote:
> >> Dir Sirs and Madams
> >>
> >> It might be already discussed in the other thread but It would be
> >> great if you help us to develop the vision processing application with
> >> libcamera on RaspberryPi4.
> >>
> >> I’m evaluating libcamera and qcam by IMX219 image sensor on RaspberryPi4.
> >
> > Please note that qcam is a test application and under development, and as
> > such does not expose all features available from libcamera.
> > -> (nomura)I understood.
> >
> >> I can see the image of qcam and gstreamer, I can switch the image size
> >> from full to VGA listed in imx219.c.
> >>
> >> I can see AGC and AWB are working correctly, however, I am guessing
> >> AGC does not work.
> >
> >
> > This statement is confusing,
> >   - You can see that AGC are working correctly
> >   - Yet you guess AGC does not work.
> >
> > Could you expand upon your interpretations there please?
> >
> > -> (nomura)AEC is working, AGC seems not working.
> >
>
> Thank you, the issue at hand sounds clearer now.
>
> The AGC algorithm appears to be handled at :
>
>         src/ipa/raspberrypi/controller/rpi/agc.cpp
>
> You might want to add some debug prints in there to investigate.
>
> I think you can enable logging by adding this to the top of the agc.cpp
> file:
>
> #define RPI_LOGGING_ENABLE 1
>
> >> I neither cannot set the ANALOG GAIN nor DIGITAL GAIN by v4l2 control
> >> commands.
> >
> > When using libcamera, only libcamera controls are relevant. Setting v4l2
> > controls directly is not applicable. (They would be overridden by / conflist
> > with the libcamera pipelines)
> > -> (nomura) I understood.
> >
> >> I’d like to know how to activate the AGC function to handle the darker
> >> condition.
> >>
> >> Also it would be nice to know how to disable the AEC/AGC and set the
> >> desired value by v4l2 commands from the application SW.
> >
> > When operating with libcamera, these would have to be set through libcamera
> > controls.
> >
> > If you are using a gstreamer pipeline, that would probably need to be
> > implemented there still, as I don't think our gstreamer element lets us set
> > per-frame (or any?) controls yet.
> >
> > You could also write your own libcamera native application and set those
> > controls directly,
> >
> > -> (nomura) Do you mean to write the native application code like qcam? I
> > feel it will be easier to do so.
> > It would be great if you could tell me where I can find the reference code?
>
>
> src/cam and src/qcam in the repository are our reference applications.
>
> We also have
>
>         https://github.com/libcamera-org/simple-cam
>
> As a shorter example to get started.
>
>
> >> I am an expert to handle IMX219 image sensor, then I can provide wide
> >> variety of FoV, high-speed binning mode enabling x2 speed than normal
> >> binning mode, etc.
> >>
> >> After knowing the information above, I can provide more versatile
> >> image streaming application for vision processing with wide angle
> >> camera modules up to 187 degree.
> >
> > It sounds like you will be utilising different camera modules which may
> > potentially require retuning (the Raspberry Pi IPA provides the tuning
> > tool). This could be a reason for your perceived darker images, as the
> > existing tuning is based upon the Raspberry Pi v2 camera, so perhaps your
> > lens changes have had an effect.
> >
> > Alternatively, if it's just that the images are coming through too dark,
> > perhaps a control is needed to allow some configuration here too. It may
> > require some further research, specific to your use-case.
> >
> > -> (nomura) Actually I am using the fish-eye lens but even with Raspi
> > camera, FoV=76degree, the AGC will not work.
> > I understand I need re-tuning for the other lens, but basically LSC tuning
> > will be necessary.
>
> I don't know what effect the fish-eye would have on the metering points
> for the AGC, I'd hope it wouldn't affect things too negatively, but I
> don't know for sure.
>
> David is certainly the expert there :-)
>
> --
> Regards
> --
> Kieran