[libcamera-devel] [PATCH v5 2/6] pipeline: raspberrypi: Set the ISP Output1 to 1/4 resolution if unused
Naushir Patuck
naush at raspberrypi.com
Fri Feb 5 14:38:41 CET 2021
Hi Laurent,
Thank you for your review feedback.
On Thu, 4 Feb 2021 at 21:32, Laurent Pinchart <
laurent.pinchart at ideasonboard.com> wrote:
> Hi Naush,
>
> Thank you for the patch.
>
> On Fri, Jan 29, 2021 at 03:11:50PM +0000, Naushir Patuck wrote:
> > In preparation for fast colour denoise, set the low resolution ISP
> > output stream (Output1) to a 1/4 resolution of the application requested
> > stream (Output0). This only happens if the application has not requested
> > an additional YUV or RGB stream.
> >
> > We also constrain this 1/4 resolution to at most 1200 pixels in the
> > largest dimension to avoid being too taxing on memory usage and system
> > bandwidth.
> >
> > Also switch the default StreamRole::VideoRecording to YUV420 to allow
> > fast colour denoise to run.
> >
> > Signed-off-by: Naushir Patuck <naush at raspberrypi.com>
> > Reviewed-by: David Plowman <david.plowman at raspberrypi.com>
> > Reviewed-by: Kieran Bingham <kieran.bingham at ideasonboard.com>
> > ---
> > .../pipeline/raspberrypi/raspberrypi.cpp | 35 ++++++++++++++++++-
> > 1 file changed, 34 insertions(+), 1 deletion(-)
> >
> > diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp
> b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp
> > index d9895c779725..fe4c75f09925 100644
> > --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp
> > +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp
> > @@ -496,7 +496,7 @@ CameraConfiguration
> *PipelineHandlerRPi::generateConfiguration(Camera *camera,
> >
> > case StreamRole::VideoRecording:
> > fmts = data->isp_[Isp::Output0].dev()->formats();
> > - pixelFormat = formats::NV12;
> > + pixelFormat = formats::YUV420;
>
> Does this mean that the colour denoise can only run when outputting
> YUV420 ? Will it be silently disabled otherwise ? You set below the same
> pixel format for outputs 0 and 1, if colour denoise can only run when
> the format is YUV420, should we skip enabling output 1 if a different
> pixel format has been selected, as that would waste resources ?
>
The colour denoise will only run if both image formats are YUV420.
Otherwise it will silently fail. There is an optimisation to be had where
we could disable the output 1 stream if the format is not YUV420 and the
application has not requested a second stream, however see below....
>
> Actually, now that I think about it, and unless I'm mistaken, the
> pipeline handler doesn't allocate and queue buffers for output 1 when
> only output 0 is used by the application, so maybe it doesn't waste
> resources and it's fine as is ?
>
Sadly, this channel is currently always enabled - even if never used. I
need to do more rigorous testing of our drivers to ensure there is no
problem with disabling only output 1. Given that it would be default
behavior, I would like some confidence on the change before submitting.
This is also somewhat related to some work that I recently touched upon for
disable Unicam embedded streams when a sensor does not use time. I will
add a \todo here to remind me to get to it.
>
> > size = { 1920, 1080 };
> > bufferCount = 4;
> > outCount++;
> > @@ -608,6 +608,7 @@ int PipelineHandlerRPi::configure(Camera *camera,
> CameraConfiguration *config)
> > * StreamConfiguration appropriately.
> > */
> > V4L2DeviceFormat format;
> > + bool output1Set = false;
> > for (unsigned i = 0; i < config->size(); i++) {
> > StreamConfiguration &cfg = config->at(i);
> >
> > @@ -632,6 +633,9 @@ int PipelineHandlerRPi::configure(Camera *camera,
> CameraConfiguration *config)
> > format.size = cfg.size;
> > format.fourcc = fourcc;
> >
> > + LOG(RPI, Info) << "Setting " << stream->name() << " to a
> resolution of "
> > + << format.toString();
>
> Should this be a debug message ? I'd also write
>
> LOG(RPI, Debug) << "Setting " << stream->name() << " to "
> << format.toString();
>
> as format.toString() will output both the resolution and pixel format.
>
Ack.
>
> > +
> > ret = stream->dev()->setFormat(&format);
> > if (ret)
> > return -EINVAL;
> > @@ -645,6 +649,35 @@ int PipelineHandlerRPi::configure(Camera *camera,
> CameraConfiguration *config)
> >
> > cfg.setStream(stream);
> > stream->setExternal(true);
> > +
> > + if (i != maxIndex)
> > + output1Set = true;
> > + }
> > +
> > + /*
> > + * If ISP::Output1 stream has not been requested by the
> application, we
> > + * set it up for internal use now. This second stream will be used
> for
> > + * fast colour denoise, and must be a quarter resolution of the
> ISP::Output0
> > + * stream. However, also limit the maximum size to 1200 pixels in
> the
> > + * larger dimension, just to avoid being wasteful with buffer
> allocations
> > + * and memory bandwidth.
> > + */
> > + if (!output1Set) {
> > + V4L2DeviceFormat output1Format = format;
> > + constexpr unsigned int maxDimensions = 1200;
> > + const Size limit = Size(maxDimensions,
> maxDimensions).boundedToAspectRatio(format.size);
>
> You could also write
>
> constexpr Size maxDimensions(1200, 1200);
> const Size limit =
> maxDimensions.boundedToAspectRatio(format.size);
>
> Up to you.
>
Ack.
>
> > +
> > + output1Format.size = (format.size /
> 2).boundedTo(limit).alignedDownTo(2, 2);
> > +
> > + LOG(RPI, Info) << "Setting ISP Output1 (internal) to a
> resolution of "
> > + << output1Format.toString();
>
> Same comment as above.
>
Ack.
Regards,
Naush
>
> > +
> > + ret =
> data->isp_[Isp::Output1].dev()->setFormat(&output1Format);
> > + if (ret) {
> > + LOG(RPI, Error) << "Failed to set format on ISP
> Output1: "
> > + << ret;
> > + return -EINVAL;
> > + }
> > }
> >
> > /* ISP statistics output format. */
>
> --
> Regards,
>
> Laurent Pinchart
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.libcamera.org/pipermail/libcamera-devel/attachments/20210205/30f88ec2/attachment.htm>
More information about the libcamera-devel
mailing list