[PATCH v1 1/7] controls: rpi: Add a vendor rpi::ScalerCrops control

[Laurent Pinchart]

On Fri, Aug 30, 2024 at 11:05:28AM +0100, Naushir Patuck wrote:
> On Wed, 28 Aug 2024 at 17:17, Laurent Pinchart wrote:
> > On Wed, Aug 28, 2024 at 10:58:55AM +0100, Naushir Patuck wrote:
> > > On Wed, 28 Aug 2024 at 10:10, Jacopo Mondi wrote:
> > > > On Thu, Aug 08, 2024 at 11:23:40AM GMT, Naushir Patuck wrote:
> > > > > Add a vendor control rpi::ScalerCrops that is analogous to the current
> > > > > core::ScalerCrop, but can apply a different crop to each configured
> > > > > stream.
> > > > >
> > > > > This control takes a span of Rectangle structures - the order of
> > > > > rectangles must match the order of streams configured by the application.
> > > > >
> > > > > Signed-off-by: Naushir Patuck <naush at raspberrypi.com>
> > > > > ---
> > > > >  src/ipa/rpi/common/ipa_base.cpp    |  2 ++
> > > > >  src/libcamera/control_ids_rpi.yaml | 15 +++++++++++++++
> > > > >  2 files changed, 17 insertions(+)
> > > > >
> > > > > diff --git a/src/ipa/rpi/common/ipa_base.cpp b/src/ipa/rpi/common/ipa_base.cpp
> > > > > index ee3848b54f21..463f6d384c9e 100644
> > > > > --- a/src/ipa/rpi/common/ipa_base.cpp
> > > > > +++ b/src/ipa/rpi/common/ipa_base.cpp
> > > > > @@ -71,6 +71,7 @@ const ControlInfoMap::Map ipaControls{
> > > > >       { &controls::HdrMode, ControlInfo(controls::HdrModeValues) },
> > > > >       { &controls::Sharpness, ControlInfo(0.0f, 16.0f, 1.0f) },
> > > > >       { &controls::ScalerCrop, ControlInfo(Rectangle{}, Rectangle(65535, 65535, 65535, 65535), Rectangle{}) },
> > > > > +     { &controls::rpi::ScalerCrops, ControlInfo(Rectangle{}, Rectangle(65535, 65535, 65535, 65535), Rectangle{}) },
> > > >
> > > > Can this be registered only in the pisp IPA ? In this way you won't
> > > > see the control on vc4 platforms, and you can safely keep the code
> > > > handling it in pipeline_base, but it will only be exercized on pi5
> > >
> > > Yes this is possible.  I can rework it for v2.
> >
> > I would prefer that too.
> 
> Ack
> 
> > > > >       { &controls::FrameDurationLimits, ControlInfo(INT64_C(33333), INT64_C(120000)) },
> > > > >       { &controls::draft::NoiseReductionMode, ControlInfo(controls::draft::NoiseReductionModeValues) },
> > > > >       { &controls::rpi::StatsOutputEnable, ControlInfo(false, true, false) },
> > > > > @@ -1070,6 +1071,7 @@ void IpaBase::applyControls(const ControlList &controls)
> > > > >                       break;
> > > > >               }
> > > > >
> > > > > +             case controls::rpi::SCALER_CROPS:
> > > > >               case controls::SCALER_CROP: {
> > > > >                       /* We do nothing with this, but should avoid the warning below. */
> > > > >                       break;
> > > > > diff --git a/src/libcamera/control_ids_rpi.yaml b/src/libcamera/control_ids_rpi.yaml
> > > > > index cb097f887e16..3d4a305dd8f5 100644
> > > > > --- a/src/libcamera/control_ids_rpi.yaml
> > > > > +++ b/src/libcamera/control_ids_rpi.yaml
> > > > > @@ -26,4 +26,19 @@ controls:
> > > > >
> > > > >          \sa StatsOutputEnable
> > > > >
> > > > > +  - ScalerCrops:
> > > > > +      type: Rectangle
> > > > > +      size: [n]
> > > > > +      description: |
> > > > > +        An array of rectangles, where each singular value has identical functionality
> > > > > +        to the ScalerCrop control. This control allows the Raspberry Pi pipeline
> > > > > +        handler to control individual scaler crops per output stream.
> > > > > +
> > > > > +        The order of rectangles passed into the control must match the order of
> > > > > +        streams configured by the application.
> >
> > How about the number of rectangles ? Can you specify less rectangles
> > than the number of streams ? Or more ? What happens if you do, is the
> > control completely ignored, or do part of the rectangles get applied ?
> >
> > Is there also a need to allow applications to change the scaler crop for
> > a subset of the streams, without affecting other ones ?
> 
> Sadly this mechanism is not flexible at all.  So the application must
> specify the correct number of scalar crop rectangles based on the
> number of configured streams unconditionally.  A new control API will
> definitely need such flexibility.

Fine with me. Could you please document the requirement here ?

> > Something that isn't explained in this, and barely mentioned in the
> > cover letter (it just says "preliminary" without elaborating) is that
> > this vendor control is a work around for the lack of a per-stream
> > controls API. Do we agree that this control will be replaced by
> > per-stream controls in the future, and that it will then be dropped and
> > not kept for backward compatibility ? If so, could you please record
> > this here, with a \todo note ?
> >
> > Can I also volunteer you for a per-stream control RFC ? :-)
> 
> This change is really a stop-gap until we have an official API for
> per-stream controls.  I'll add a todo to note this.  I will also make
> a RFC proposal for the changes and perhaps we can spend some time
> talking about it in the Vienna F2F.

Great, thank you !

> > > > > +
> > > > > +        Note that using different crop rectangles for each output stream is only
> > > > > +        applicable on the Pi5/PiSP platform.
> > > > > +
> > > > > +        \sa ScalerCrop
> > > > >  ...

-- 
Regards,

Laurent Pinchart