[PATCH 0/1] Add StreamRole into StreamConfiguration

[Jacopo Mondi]

Hi Harvey

On Wed, Sep 18, 2024 at 04:33:27PM GMT, Cheng-Hao Yang wrote:
> Sorry Jacopo, my bad to miss the first message:
>
> On Mon, Sep 16, 2024 at 8:45 PM Jacopo Mondi <jacopo.mondi at ideasonboard.com>
> wrote:
>
> > Hi Harvey
> >
> > On Mon, Sep 16, 2024 at 03:51:00PM GMT, Cheng-Hao Yang wrote:
> > > Hi Jacopo,
> > >
> > > On Mon, Sep 16, 2024 at 3:35 PM Jacopo Mondi <
> > jacopo.mondi at ideasonboard.com>
> > > wrote:
> > >
> > > > Hi Harvey
> > > >
> > > > On Mon, Sep 16, 2024 at 04:51:54AM GMT, Harvey Yang wrote:
> > > > > Hi folks,
> > > > >
> > > > > Currently applications set resolutions, pixelFormat, bufferCount,
> > etc,
> > > > > into StreamConfigurations, and Pipeline Handler decides which streams
> > > > > they're assigned to. However, it doesn't allow application to assign
> > > > > streams that cannot be distinguished by those arguments into
> > > > > VideoRecording or StillCapture (say YUV/NV12 format), which is
> > needed in
> > > > > mtkisp7.
> > > >
> > > > Could you explain in a bit more detail why this "is needed" and how
> > > > you plan to use StreamRole as part of the stream configuration ?
> > > >
> >
> > Could you explain in a bit more detail why this "is needed" and how
> > you plan to use StreamRole as part of the stream configuration ?
> >
> >
> >
> In mtkisp7 pipeline handler, both preview (or video) and still capture
> streams support the same format (NV12) and bufferCount, and the
> maximum resolutions are also the same. Therefore, when calling
> `validate()` and `configure()`, mtkisp7 pipeline handler doesn't know
> if applications/Android adapter wants the stream to go through the
> still captur pipeline or not.

I still have an hard time understanding why validate() and configure()
needs to know the "role". Is this to assign "pipes" to Streams ? Does
the mtkisp7 pipes have different capabilities when it comes to
supported formats and resolutions ?

>
> Does that make sense :P?

I guess so, but I wonder if the "role" is the main criteria that
should be taken into account when assiging pipes to streams.

How would this work for applications that operates on platforms where
the pipe selection depends on other criteria like the supported image
formats and stream resolutions ? In example, for rkisp1
the "self" pipelines can do RGB while the "main" can't and that's how
we assign pipes during validate().

To be honest this has not proven to be optimal and I'm not opposed to
add Roles to the pipe selection criteria, but they shouldn't be made the
only criteria on which pipes are assigned (unless we support this on
all pipelines).

Also I would not based any future-proof design on Roles, they will
likely be heavily reworked or removed going forward.

As your main use case is Android, I think it would be doable for you
to
1) Request a generateConfiguration() and keep track of the
   StreamConfiguration order.

   generateConfiguration(Viewfinder, Still, Raw)

   will give you StreamConfiguration for the above rols in order as
   you have requested them, if supported.

2) Code validate() so that you try to assing pipes based on the
   formats/sizes and if formats/sizes are the same define an ordering.
   In example the first YUV/WxH stream goes to Viewfinder if an
   identical YUV/WxH stream is requested for Still Capture.

3) As validate() assigns Steams * to StreamConfiguration (with
  ::setStream) it should be easy to keep track to which pipe a
  StreamConfiguration has been assigned to at configure() time.

Could you list what are the platform's pipes capabilities ?

As said, I'm not opposed to use Roles for assigning pipes, but I don't
think we should based any new development on Roles as there's an high
chance they can be reworked.

Cc-Laurent for opinions.

>
> BR,
> Harvey