[PATCH v1 0/7] Raspberry Pi: Add controls::rpi::ScalerCrops

[Naushir Patuck]

Hi all,

Gentle ping for any feedback on this please?

Thanks,
Naush

On Thu, 8 Aug 2024 at 11:23, Naushir Patuck <naush at raspberrypi.com> wrote:
>
> Hi,
>
> This patch series adds preliminary support for setting individual crop
> rectangles for each ISP output stream. The current ScalerCrop control takes a
> single rectangle and applies it to all output streams. In this series, we define
> a new rpi::ScalerCrops vendor control that takes a span of rectangles, one for
> each output stream. The behaviour of the existing ScalerCrop control remains
> unchanged.
>
> One example of the use of this feature is when generating low resolution images
> for AI inferencing. Typically, vision neural network models are trained on
> square dimension image with 1:1 aspect ratio (e.g. 640x640 or 320x320). If an
> application requests, say, a 1920x1080 (16:9) output stream + 640x640 (1:1)
> inference stream, then the latter images end up squished and distorted. This
> affects the performance of the neural network.  By allowing separate crops on
> each stream, the inference stream can preserve the pixel aspect ratio for any
> requested resolution.
>
> Note that VC4 based platforms cannot apply different crops to each output,
> whereas the PiSP/Pi5 platform can. However, all the infrastructure for this
> change lives in the core pipeline handler shared by both platforms.
>
> Thanks,
> Naush
>
> Naushir Patuck (7):
>   controls: rpi: Add a vendor rpi::ScalerCrops control
>   pipeline: rpi: Remove CameraData::scalerCrop_
>   pipeline: rpi: Pass crop rectangle as a parameter to
>     platformSetIspCrop()
>   pipeline: rpi: Introduce CameraData::CropParams
>   pipeline: rpi: Track which ISP output is configured for a stream
>   pipeline: rpi: Pass ISP output index into platformSetIspCrop()
>   pipeline: rpi: Handler controls::rpi::ScalerCrops
>
>  src/ipa/rpi/common/ipa_base.cpp               |  2 +
>  src/libcamera/control_ids_rpi.yaml            | 15 ++++
>  .../pipeline/rpi/common/pipeline_base.cpp     | 84 +++++++++++++------
>  .../pipeline/rpi/common/pipeline_base.h       | 26 +++++-
>  src/libcamera/pipeline/rpi/vc4/vc4.cpp        | 18 ++--
>  5 files changed, 112 insertions(+), 33 deletions(-)
>
> --
> 2.34.1
>