[libcamera-devel] Application requesting statistics buffers

[Laurent Pinchart]

Hi Naush,

On Fri, Jun 26, 2020 at 12:56:21PM +0100, Naushir Patuck wrote:
> Hi,
> 
> As part of the work (just submitted for review) on providing a focus
> FoM to the application through libcamera controls, one of the options
> I explored was to allow the application to explicitly request the ISP
> statistics buffer along with the image frame through the usual Request
> mechanism.  The Raspberry Pi pipeline handler is written with enough
> flexibility to allow this to happen.  However, one thing missing is a
> way for the application to request (and provide) the statistics buffer
> as an explicit format.

You know you ask very good and very annoying questions, right ? :-)

> In the v4l2 world, we have a specific format
> (V4L2_META_FMT_BCM2835_ISP_STATS) for the ISP statistics buffer
> defined.  In order to create a libcamera::format (from formats.yaml),
> it seems I must have an equivalent DRM format defined.  However, from
> what I can tell, DRM formats are not defined for metadata types, only
> image data, is that correct?  Would it be reasonable for me to define
> an equivalent DRM_FORMAT_BCM2835_ISP_STATS for this purpose?

I wonder if we shouldn't use an entirely different mechanism here. The
PixelFormat class, as its name implies, describes pixel formats, and is
ill-fitted for this purpose. Same for StreamConfiguration. I would
rather create a property that reports the statistics format (as it is
immutable), and extending the Request class to add a statistics buffer
explicitly. We'll have to figure out how to handle that in
FrameBufferAllocator and most probably in other parts of the stack.

Maybe this would create too much complexity and interfacing with the
existing streams mechanism would be best, but I think we need to
evaluate all options before deciding which one to pick.

> This may seem a somewhat isolated use case, but I can see that some
> specific applications could want to do their own analysis of the
> statistics data for their purposes.  In this example, a user wanted to
> create a microscope application...

Would the user also have a need for embedded data ?

> Right now I will not worry about the other problem with this where the
> application will have to provide a stats buffer on every Request,
> similar to the problem with RAW captures - but a zero-copy mechanism
> is in the works from what I can tell :)
> 
> Let me know your thoughts on how we can achieve this.

-- 
Regards,

Laurent Pinchart