<div dir="ltr"><div dir="ltr">Hi Laurent,<div><br></div><div>Thank you for your review feedback.</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, 4 Feb 2021 at 21:32, Laurent Pinchart <<a href="mailto:laurent.pinchart@ideasonboard.com" target="_blank">laurent.pinchart@ideasonboard.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi Naush,<br>
<br>
Thank you for the patch.<br>
<br>
On Fri, Jan 29, 2021 at 03:11:50PM +0000, Naushir Patuck wrote:<br>
> In preparation for fast colour denoise, set the low resolution ISP<br>
> output stream (Output1) to a 1/4 resolution of the application requested<br>
> stream (Output0). This only happens if the application has not requested<br>
> an additional YUV or RGB stream.<br>
> <br>
> We also constrain this 1/4 resolution to at most 1200 pixels in the<br>
> largest dimension to avoid being too taxing on memory usage and system<br>
> bandwidth.<br>
> <br>
> Also switch the default StreamRole::VideoRecording to YUV420 to allow<br>
> fast colour denoise to run.<br>
> <br>
> Signed-off-by: Naushir Patuck <<a href="mailto:naush@raspberrypi.com" target="_blank">naush@raspberrypi.com</a>><br>
> Reviewed-by: David Plowman <<a href="mailto:david.plowman@raspberrypi.com" target="_blank">david.plowman@raspberrypi.com</a>><br>
> Reviewed-by: Kieran Bingham <<a href="mailto:kieran.bingham@ideasonboard.com" target="_blank">kieran.bingham@ideasonboard.com</a>><br>
> ---<br>
> .../pipeline/raspberrypi/raspberrypi.cpp | 35 ++++++++++++++++++-<br>
> 1 file changed, 34 insertions(+), 1 deletion(-)<br>
> <br>
> diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp<br>
> index d9895c779725..fe4c75f09925 100644<br>
> --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp<br>
> +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp<br>
> @@ -496,7 +496,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera,<br>
> <br>
> case StreamRole::VideoRecording:<br>
> fmts = data->isp_[Isp::Output0].dev()->formats();<br>
> - pixelFormat = formats::NV12;<br>
> + pixelFormat = formats::YUV420;<br>
<br>
Does this mean that the colour denoise can only run when outputting<br>
YUV420 ? Will it be silently disabled otherwise ? You set below the same<br>
pixel format for outputs 0 and 1, if colour denoise can only run when<br>
the format is YUV420, should we skip enabling output 1 if a different<br>
pixel format has been selected, as that would waste resources ?<br></blockquote><div><br></div><div>The colour denoise will only run if both image formats are YUV420. Otherwise it will silently fail. There is an optimisation to be had where we could disable the output 1 stream if the format is not YUV420 and the application has not requested a second stream, however see below....</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
Actually, now that I think about it, and unless I'm mistaken, the<br>
pipeline handler doesn't allocate and queue buffers for output 1 when<br>
only output 0 is used by the application, so maybe it doesn't waste<br>
resources and it's fine as is ?<br></blockquote><div><br></div><div>Sadly, this channel is currently always enabled - even if never used. I need to do more rigorous testing of our drivers to ensure there is no problem with disabling only output 1. Given that it would be default behavior, I would like some confidence on the change before submitting. This is also somewhat related to some work that I recently touched upon for disable Unicam embedded streams when a sensor does not use time. I will add a \todo here to remind me to get to it.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
> size = { 1920, 1080 };<br>
> bufferCount = 4;<br>
> outCount++;<br>
> @@ -608,6 +608,7 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config)<br>
> * StreamConfiguration appropriately.<br>
> */<br>
> V4L2DeviceFormat format;<br>
> + bool output1Set = false;<br>
> for (unsigned i = 0; i < config->size(); i++) {<br>
> StreamConfiguration &cfg = config->at(i);<br>
> <br>
> @@ -632,6 +633,9 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config)<br>
> format.size = cfg.size;<br>
> format.fourcc = fourcc;<br>
> <br>
> + LOG(RPI, Info) << "Setting " << stream->name() << " to a resolution of "<br>
> + << format.toString();<br>
<br>
Should this be a debug message ? I'd also write<br>
<br>
LOG(RPI, Debug) << "Setting " << stream->name() << " to "<br>
<< format.toString();<br>
<br>
as format.toString() will output both the resolution and pixel format.<br></blockquote><div><br></div><div>Ack.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
> +<br>
> ret = stream->dev()->setFormat(&format);<br>
> if (ret)<br>
> return -EINVAL;<br>
> @@ -645,6 +649,35 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config)<br>
> <br>
> cfg.setStream(stream);<br>
> stream->setExternal(true);<br>
> +<br>
> + if (i != maxIndex)<br>
> + output1Set = true;<br>
> + }<br>
> +<br>
> + /*<br>
> + * If ISP::Output1 stream has not been requested by the application, we<br>
> + * set it up for internal use now. This second stream will be used for<br>
> + * fast colour denoise, and must be a quarter resolution of the ISP::Output0<br>
> + * stream. However, also limit the maximum size to 1200 pixels in the<br>
> + * larger dimension, just to avoid being wasteful with buffer allocations<br>
> + * and memory bandwidth.<br>
> + */<br>
> + if (!output1Set) {<br>
> + V4L2DeviceFormat output1Format = format;<br>
> + constexpr unsigned int maxDimensions = 1200;<br>
> + const Size limit = Size(maxDimensions, maxDimensions).boundedToAspectRatio(format.size);<br>
<br>
You could also write<br>
<br>
constexpr Size maxDimensions(1200, 1200);<br>
const Size limit = maxDimensions.boundedToAspectRatio(format.size);<br>
<br>
Up to you.<br></blockquote><div><br></div><div>Ack.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
> +<br>
> + output1Format.size = (format.size / 2).boundedTo(limit).alignedDownTo(2, 2);<br>
> +<br>
> + LOG(RPI, Info) << "Setting ISP Output1 (internal) to a resolution of "<br>
> + << output1Format.toString();<br>
<br>
Same comment as above.<br></blockquote><div><br></div><div>Ack.</div><div><br></div><div>Regards,</div><div>Naush</div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
> +<br>
> + ret = data->isp_[Isp::Output1].dev()->setFormat(&output1Format);<br>
> + if (ret) {<br>
> + LOG(RPI, Error) << "Failed to set format on ISP Output1: "<br>
> + << ret;<br>
> + return -EINVAL;<br>
> + }<br>
> }<br>
> <br>
> /* ISP statistics output format. */<br>
<br>
-- <br>
Regards,<br>
<br>
Laurent Pinchart<br>
</blockquote></div></div>