[libcamera-devel] How to activate AGC control in libcamera, or set the GAIN/EXPOSURE parameter by v4l2 command.

Tue Sep 29 01:24:33 CEST 2020

Dear David-san

Thank you for your comments.
As you mentioned, I also think AEC and AGC is the inseparable functions,
and if AGC is active in dark condition I see the noise is amplified to reach to AGC target.
However, I don't see the noise increasing, so I estimated the AGC is not working.

I'll do the experiment with original Raspi camera and report to you.

And I'll try to capture the JPEG image.
(I wonder qcam does not support jpeg capture at this moment, right?)

Best Regards,

NOMURA

-----Original Message-----
From: David Plowman <david.plowman at raspberrypi.com> 
Sent: Monday, September 28, 2020 10:54 PM
To: Kieran Bingham <kieran.bingham at ideasonboard.com>
Cc: tetsuya.nomura at soho-enterprise.com; libcamera-devel at lists.libcamera.org
Subject: Re: [libcamera-devel] How to activate AGC control in libcamera, or set the GAIN/EXPOSURE parameter by v4l2 command.

Hi Nomura-san

Thank you for your questions.

Perhaps you can explain more precisely how you can tell the AGC is not running - is there perhaps some parameter you are measuring, or is it a visual assessment of the image?

On the Raspberry Pi AEC and AGC run together, you can't really have one without the other. Exposure time and analogue gain are divided according to an "exposure profile". Please read here https://www.raspberrypi.org/documentation/linux/software/libcamera/rpi_SOFT_libcamera_1p0.pdf
(section 5.8) for more information.

I don't recall without checking whether the jpegs produced by qcam record the exposure time or gain, but if you capture DNG files you should certainly see both exposure time and ISO changing according to the scene brightness.

I hope that helps.

Best regards
David

On Mon, 28 Sep 2020 at 14:46, Kieran Bingham <kieran.bingham at ideasonboard.com> wrote:
>
> Hi Nomura-san,
>
> I've added David on CC as he might have some insights, (I hope you 
> don't mind David)
>
>
> On 28/09/2020 12:55, tetsuya.nomura at soho-enterprise.com wrote:
> > Dear Kieran-san
> >
> > Thank you very much for your prompt reply.
> >
> > I'm sorry I wrote wrong, though this is the most important point.
> > I can see "Auto Exposure Control" is working, and I can see the 
> > exposure parameter is reaching to the maximum value.
> > But after that, I don't see the noise level never increase, this 
> > means the "Auto Gain Control" does not work in qcam.
> > I understand the qcam at this moment does not support AGC.
>
> Ok, so that sounds slightly different. The AGC is handled as part of 
> the Raspberry Pi pipeline handler, and that is a lower layer than QCam ...
> Qcam is just displaying the results.
>
> It sounds like you are saying that once the exposure has reached its'
> maximum, you would expect the gain control to start increasing... (and 
> you have observed that it is not).
>
>
>
> > I write the comment below.
> >
> > Best Regards,
> >
> > NOMURA
> > -----Original Message-----
> > From: Kieran Bingham <kieran.bingham at ideasonboard.com>
> > Sent: Monday, September 28, 2020 8:00 PM
> > To: tetsuya.nomura at soho-enterprise.com; 
> > libcamera-devel at lists.libcamera.org
> > Subject: Re: [libcamera-devel] How to activate AGC control in 
> > libcamera, or set the GAIN/EXPOSURE parameter by v4l2 command.
> >
> > Hello Nomura-san,
> >
> > On 28/09/2020 10:50, tetsuya.nomura at soho-enterprise.com wrote:
> >> Dir Sirs and Madams
> >>
> >> It might be already discussed in the other thread but It would be 
> >> great if you help us to develop the vision processing application 
> >> with libcamera on RaspberryPi4.
> >>
> >> I’m evaluating libcamera and qcam by IMX219 image sensor on RaspberryPi4.
> >
> > Please note that qcam is a test application and under development, 
> > and as such does not expose all features available from libcamera.
> > -> (nomura)I understood.
> >
> >> I can see the image of qcam and gstreamer, I can switch the image 
> >> size from full to VGA listed in imx219.c.
> >>
> >> I can see AGC and AWB are working correctly, however, I am guessing 
> >> AGC does not work.
> >
> >
> > This statement is confusing,
> >   - You can see that AGC are working correctly
> >   - Yet you guess AGC does not work.
> >
> > Could you expand upon your interpretations there please?
> >
> > -> (nomura)AEC is working, AGC seems not working.
> >
>
> Thank you, the issue at hand sounds clearer now.
>
> The AGC algorithm appears to be handled at :
>
>         src/ipa/raspberrypi/controller/rpi/agc.cpp
>
> You might want to add some debug prints in there to investigate.
>
> I think you can enable logging by adding this to the top of the 
> agc.cpp
> file:
>
> #define RPI_LOGGING_ENABLE 1
>
> >> I neither cannot set the ANALOG GAIN nor DIGITAL GAIN by v4l2 
> >> control commands.
> >
> > When using libcamera, only libcamera controls are relevant. Setting 
> > v4l2 controls directly is not applicable. (They would be overridden 
> > by / conflist with the libcamera pipelines)
> > -> (nomura) I understood.
> >
> >> I’d like to know how to activate the AGC function to handle the 
> >> darker condition.
> >>
> >> Also it would be nice to know how to disable the AEC/AGC and set 
> >> the desired value by v4l2 commands from the application SW.
> >
> > When operating with libcamera, these would have to be set through 
> > libcamera controls.
> >
> > If you are using a gstreamer pipeline, that would probably need to 
> > be implemented there still, as I don't think our gstreamer element 
> > lets us set per-frame (or any?) controls yet.
> >
> > You could also write your own libcamera native application and set 
> > those controls directly,
> >
> > -> (nomura) Do you mean to write the native application code like 
> > -> qcam? I
> > feel it will be easier to do so.
> > It would be great if you could tell me where I can find the reference code?
>
>
> src/cam and src/qcam in the repository are our reference applications.
>
> We also have
>
>         https://github.com/libcamera-org/simple-cam
>
> As a shorter example to get started.
>
>
> >> I am an expert to handle IMX219 image sensor, then I can provide 
> >> wide variety of FoV, high-speed binning mode enabling x2 speed than 
> >> normal binning mode, etc.
> >>
> >> After knowing the information above, I can provide more versatile 
> >> image streaming application for vision processing with wide angle 
> >> camera modules up to 187 degree.
> >
> > It sounds like you will be utilising different camera modules which 
> > may potentially require retuning (the Raspberry Pi IPA provides the 
> > tuning tool). This could be a reason for your perceived darker 
> > images, as the existing tuning is based upon the Raspberry Pi v2 
> > camera, so perhaps your lens changes have had an effect.
> >
> > Alternatively, if it's just that the images are coming through too 
> > dark, perhaps a control is needed to allow some configuration here 
> > too. It may require some further research, specific to your use-case.
> >
> > -> (nomura) Actually I am using the fish-eye lens but even with 
> > -> Raspi
> > camera, FoV=76degree, the AGC will not work.
> > I understand I need re-tuning for the other lens, but basically LSC 
> > tuning will be necessary.
>
> I don't know what effect the fish-eye would have on the metering 
> points for the AGC, I'd hope it wouldn't affect things too negatively, 
> but I don't know for sure.
>
> David is certainly the expert there :-)
>
> --
> Regards
> --
> Kieran