[libcamera-devel] Suboptimal resolution for digital zoom (controls::ScalerCrop)

[David Plowman]

Hi

It should also be possible to request a raw stream at the full sensor
resolution, but set the output size to the crop that you really want.
Initially you'll obviously find that the full image is being
downscaled to your output resolution. But then set the ScalerCrop to
match the output size that you asked for, and you should be able to
get a 1:1 crop from the full resolution image. Does that help?

David

On Tue, 25 Apr 2023 at 07:34, Laurent Pinchart via libcamera-devel
<libcamera-devel at lists.libcamera.org> wrote:
>
> Hi Elias,
>
> On Mon, Apr 24, 2023 at 08:04:22PM -0600, Elias Naur via libcamera-devel wrote:
> > Hi,
> >
> > I'd like to squeeze out as much resolution as possible from my Raspberry Pi camera v1
> > for scanning QR codes. The camera  has a fixed focus at ~50cm from the lens, so the
> > resolution gain from moving the QR codes closer is lost to blur from the loss of focus.
> >
> > My current strategy is to scan codes at a distance, but use the considerable resolution
> > of the sensor (~2500x1900) to gain enough resolution for successful scans.
> >
> > However, I fail to find any way to achieve that with libcamera. I tried three options, all
> > suboptimal:
> >
> > 1. Configure the camera for full resolution and crop to a smaller section in my program.
> > That works surprisingly well, except that the white-balance etc. algorithms run on the full
> > image and not just the cropped part.
> > 2. Configure the camera for full resolution and set controls::ScalerCrop to the area
> > of interest. This seems to fix the white-balance algorithms, but results in the cropped
> > area being *upscaled* to full resolution when I receive them. Scanning for QR codes
> > at full resolution is way too slow on a RPi Zero.
> > 3. Configure the camera for the same resolution of the area of interest, and set
> > controls::ScalerCrop. This is efficient, but the sensor is configured for the low
> > resolution and thus ruins the resolution gain.
> >
> > Questions:
> > 1. Is there another way?
>
> How about capturing full resolution frames, without cropping in the
> capture pipeline, and scanning for the QR code in the area of interest
> only ?
>
> > 2. The following hack works for me,
> >
> > diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp
> > b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp
> > index 00600441..6ed81dc1 100644
> > --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp
> > +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp
> > @@ -842,7 +842,7 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config)
> >          * request, and apply it to the sensor with the cached transform, if
> >          * any.
> >          */
> > -       V4L2SubdeviceFormat sensorFormat = findBestFormat(data->sensorFormats_, rawStream ? sensorSize : maxSize, bitDepth);
> > +       V4L2SubdeviceFormat sensorFormat = findBestFormat(data->sensorFormats_, Size{2592,1944}, bitDepth);
> >         const RPiCameraConfiguration *rpiConfig = static_cast<const RPiCameraConfiguration *>(config);
> >         ret = data->sensor_->setFormat(&sensorFormat, rpiConfig->combinedTransform_);
> >         if (ret)
> >
> > Would a similar change that *always* uses the full sensor resolution be acceptable?
> > I can't discern any performance issues between full resolution and lower resolution.
>
> You can't hardcode a specific resolution, as this wouldn't work properly
> with other sensors. Always selecting the sensor full resolution is also
> not acceptable, as that will limit the frame rate on high resolution
> sensors (e.g. the Raspberry Pi HQ camera will be limited to 15fps).
>
> > 3. If the above fix is unacceptable, what is the right way to fix this
> > issue in libcamera?
>
> The current way to select the sensor resolution is a bit of a hack. You
> need to add a raw stream to your camera configuration, set to the full
> sensor resolution. You don't need to submit buffers for that stream when
> queuing requests, so you can just capture from the processed YUV stream
> as you do now.
>
> We will likely implement in the future an easier API to select the
> sensor resolution.
>
> --
> Regards,
>
> Laurent Pinchart