[libcamera-devel] [PATCH 1/1] libcamera: controls: Add StartupFrame control

[Naushir Patuck]

Hi all,

On a semi-related topic, we talked offline about improving the drop frame
support by queuing a request buffer multiple times to avoid the need for
allocating internal buffers.  I've tried this out and here are my findings.

Firstly, to handle queuing a single buffer multiple times, I need to increase
the number of cache slots in V4L2BufferCache().  Perhaps
V4L2VideoDevice::importBuffers()
should be updated to not take in a count parameter and we just allocate slots
for the maximum buffer count possible in V4L2 (32 I think)?  There has been a
long-standing \todo in the RPi code to choose an appropriate value, and the
maximum number is really the only appropriate value I can think of.

Once I got this working, unfortunately I realised this method would never
actually work correctly in the common scenario where the application configures
and uses a RAW stream.  In such cases we would queue the RAW buffer into Unicam
N times for N dropped frames.  However this buffer is also imported into the ISP
for processing and stats generation, all while it is also being filled by Unicam
for the next sensor frame.  This makes the stats entirely unusable.

So in the end we either have to allocate additional buffers for drop frames
(like we do right now), or we implement something like this series where the
application is responsible for dropping/ignoring these frames.

Regards,
Naush





On Wed, 31 May 2023 at 13:50, David Plowman via libcamera-devel
<libcamera-devel at lists.libcamera.org> wrote:
>
> This control is passed back in a frame as metadata to indicate whether
> the camera system is still in a startup phase, and the application is
> advised to avoid using the frame.
>
> Signed-off-by: David Plowman <david.plowman at raspberrypi.com>
> ---
>  src/libcamera/control_ids.yaml | 15 +++++++++++++++
>  1 file changed, 15 insertions(+)
>
> diff --git a/src/libcamera/control_ids.yaml b/src/libcamera/control_ids.yaml
> index adea5f90..4742d907 100644
> --- a/src/libcamera/control_ids.yaml
> +++ b/src/libcamera/control_ids.yaml
> @@ -694,6 +694,21 @@ controls:
>              Continuous AF is paused. No further state changes or lens movements
>              will occur until the AfPauseResume control is sent.
>
> +  - StartupFrame:
> +      type: bool
> +      description: |
> +        The value true indicates that the camera system is still in a startup
> +        phase where the output images may not be reliable, or that certain of
> +        the control algorithms (such as AEC/AGC, AWB, and possibly others) may
> +        still be changing quite rapidly.
> +
> +        Applications are advised to avoid using these frames. Mostly, they will
> +        occur when the camera system starts for the first time, although,
> +        depending on the sensor and the implementation, they could occur at
> +        other times.
> +
> +        The value false indicates that this is a normal frame.
> +
>    # ----------------------------------------------------------------------------
>    # Draft controls section
>
> --
> 2.30.2
>