[libcamera-devel] [PATCH 1/1] libcamera: controls: Add StartupFrame control

[Laurent Pinchart]

Hello,

On Wed, May 31, 2023 at 01:57:16PM +0100, Naushir Patuck via libcamera-devel wrote:
> Hi David,
> 
> Thank you for this patch.  Indeed removing the drop frame logic the pipeline
> handler would simplify things!

That's a change I would welcome :-)

> On Wed, 31 May 2023 at 13:50, David Plowman via libcamera-devel wrote:
> >
> > This control is passed back in a frame as metadata to indicate whether
> > the camera system is still in a startup phase, and the application is
> > advised to avoid using the frame.
> >
> > Signed-off-by: David Plowman <david.plowman at raspberrypi.com>
> > ---
> >  src/libcamera/control_ids.yaml | 15 +++++++++++++++
> >  1 file changed, 15 insertions(+)
> >
> > diff --git a/src/libcamera/control_ids.yaml b/src/libcamera/control_ids.yaml
> > index adea5f90..4742d907 100644
> > --- a/src/libcamera/control_ids.yaml
> > +++ b/src/libcamera/control_ids.yaml
> > @@ -694,6 +694,21 @@ controls:
> >              Continuous AF is paused. No further state changes or lens movements
> >              will occur until the AfPauseResume control is sent.
> >
> > +  - StartupFrame:
> > +      type: bool
> > +      description: |
> > +        The value true indicates that the camera system is still in a startup
> > +        phase where the output images may not be reliable, or that certain of
> > +        the control algorithms (such as AEC/AGC, AWB, and possibly others) may
> > +        still be changing quite rapidly.

I think we need to decide with a bit more details what constitute a
"startup frame" and what doesn't, or we will have all kinds of
inconsistent behaviour.

I read it that we have multiple criteria on which we base the decision:

- output images not being "reliable"
- 3A algorithms convergence
- "possibly others"

The second criteria is fairly clear, but I'm thinking we should possibly
exclude it from the startup frames and report convergence of the 3A
algorithms instead.

The first and third criteria are quite vague. If I recall correctly, the
first one includes bad frames from the sensor (as in completely
corrupted frames, such as frames that are all black or made of random
data). Those are completely unusable for applications, is there a value
in still making them available instead of dropping them correctly ?

What are the other reasons a frame would be a "startup frame" ?

> > +
> > +        Applications are advised to avoid using these frames. Mostly, they will
> > +        occur when the camera system starts for the first time, although,
> > +        depending on the sensor and the implementation, they could occur at
> > +        other times.
> > +
> > +        The value false indicates that this is a normal frame.
> 
> Just throwing it out there, but would it be useful if this control was an
> integer with the count of startup frames left to handle? A value of 0, or the
> absence of the control would indicate this is a "valid" frame.
> 
> > +
> >    # ----------------------------------------------------------------------------
> >    # Draft controls section
> >

-- 
Regards,

Laurent Pinchart