[PATCH v1 1/7] controls: rpi: Add a vendor rpi::ScalerCrops control

[Naushir Patuck]

Hi Laurent,

On Wed, 28 Aug 2024 at 17:17, Laurent Pinchart
<laurent.pinchart at ideasonboard.com> wrote:
>
> Hello,
>
> On Wed, Aug 28, 2024 at 10:58:55AM +0100, Naushir Patuck wrote:
> > On Wed, 28 Aug 2024 at 10:10, Jacopo Mondi wrote:
> > > On Thu, Aug 08, 2024 at 11:23:40AM GMT, Naushir Patuck wrote:
> > > > Add a vendor control rpi::ScalerCrops that is analogous to the current
> > > > core::ScalerCrop, but can apply a different crop to each configured
> > > > stream.
> > > >
> > > > This control takes a span of Rectangle structures - the order of
> > > > rectangles must match the order of streams configured by the application.
> > > >
> > > > Signed-off-by: Naushir Patuck <naush at raspberrypi.com>
> > > > ---
> > > >  src/ipa/rpi/common/ipa_base.cpp    |  2 ++
> > > >  src/libcamera/control_ids_rpi.yaml | 15 +++++++++++++++
> > > >  2 files changed, 17 insertions(+)
> > > >
> > > > diff --git a/src/ipa/rpi/common/ipa_base.cpp b/src/ipa/rpi/common/ipa_base.cpp
> > > > index ee3848b54f21..463f6d384c9e 100644
> > > > --- a/src/ipa/rpi/common/ipa_base.cpp
> > > > +++ b/src/ipa/rpi/common/ipa_base.cpp
> > > > @@ -71,6 +71,7 @@ const ControlInfoMap::Map ipaControls{
> > > >       { &controls::HdrMode, ControlInfo(controls::HdrModeValues) },
> > > >       { &controls::Sharpness, ControlInfo(0.0f, 16.0f, 1.0f) },
> > > >       { &controls::ScalerCrop, ControlInfo(Rectangle{}, Rectangle(65535, 65535, 65535, 65535), Rectangle{}) },
> > > > +     { &controls::rpi::ScalerCrops, ControlInfo(Rectangle{}, Rectangle(65535, 65535, 65535, 65535), Rectangle{}) },
> > >
> > > Can this be registered only in the pisp IPA ? In this way you won't
> > > see the control on vc4 platforms, and you can safely keep the code
> > > handling it in pipeline_base, but it will only be exercized on pi5
> >
> > Yes this is possible.  I can rework it for v2.
>
> I would prefer that too.

Ack

>
> > > >       { &controls::FrameDurationLimits, ControlInfo(INT64_C(33333), INT64_C(120000)) },
> > > >       { &controls::draft::NoiseReductionMode, ControlInfo(controls::draft::NoiseReductionModeValues) },
> > > >       { &controls::rpi::StatsOutputEnable, ControlInfo(false, true, false) },
> > > > @@ -1070,6 +1071,7 @@ void IpaBase::applyControls(const ControlList &controls)
> > > >                       break;
> > > >               }
> > > >
> > > > +             case controls::rpi::SCALER_CROPS:
> > > >               case controls::SCALER_CROP: {
> > > >                       /* We do nothing with this, but should avoid the warning below. */
> > > >                       break;
> > > > diff --git a/src/libcamera/control_ids_rpi.yaml b/src/libcamera/control_ids_rpi.yaml
> > > > index cb097f887e16..3d4a305dd8f5 100644
> > > > --- a/src/libcamera/control_ids_rpi.yaml
> > > > +++ b/src/libcamera/control_ids_rpi.yaml
> > > > @@ -26,4 +26,19 @@ controls:
> > > >
> > > >          \sa StatsOutputEnable
> > > >
> > > > +  - ScalerCrops:
> > > > +      type: Rectangle
> > > > +      size: [n]
> > > > +      description: |
> > > > +        An array of rectangles, where each singular value has identical functionality
> > > > +        to the ScalerCrop control. This control allows the Raspberry Pi pipeline
> > > > +        handler to control individual scaler crops per output stream.
> > > > +
> > > > +        The order of rectangles passed into the control must match the order of
> > > > +        streams configured by the application.
>
> How about the number of rectangles ? Can you specify less rectangles
> than the number of streams ? Or more ? What happens if you do, is the
> control completely ignored, or do part of the rectangles get applied ?
>
> Is there also a need to allow applications to change the scaler crop for
> a subset of the streams, without affecting other ones ?

Sadly this mechanism is not flexible at all.  So the application must
specify the correct number of scalar crop rectangles based on the
number of configured streams unconditionally.  A new control API will
definitely need such flexibility.


>
> Something that isn't explained in this, and barely mentioned in the
> cover letter (it just says "preliminary" without elaborating) is that
> this vendor control is a work around for the lack of a per-stream
> controls API. Do we agree that this control will be replaced by
> per-stream controls in the future, and that it will then be dropped and
> not kept for backward compatibility ? If so, could you please record
> this here, with a \todo note ?
>
> Can I also volunteer you for a per-stream control RFC ? :-)

This change is really a stop-gap until we have an official API for
per-stream controls.  I'll add a todo to note this.  I will also make
a RFC proposal for the changes and perhaps we can spend some time
talking about it in the Vienna F2F.

Regards,
Naush

>
> > > > +
> > > > +        Note that using different crop rectangles for each output stream is only
> > > > +        applicable on the Pi5/PiSP platform.
> > > > +
> > > > +        \sa ScalerCrop
> > > >  ...
>
> --
> Regards,
>
> Laurent Pinchart