[PATCH 2/3] gstreamer: Generate controls from control_ids_*.yaml files

[Nicolas Dufresne]

Le jeudi 08 août 2024 à 10:50 +0100, Kieran Bingham a écrit :
> Hi Nicolas, Jaslo,
> 
> 
[...]

> > >   
> > >   af-trigger          : This control starts an autofocus scan when AfMode is set to AfModeAuto, and can also be used to terminate a scan early. It is ignored if AfMode is set to AfModeManual or AfModeContinuous. 
> > >                         flags: readable, writable, controllable
> > >                         Enum "AfTrigger" Default: 0, "start"
> > >                            (0): start            - Start an AF scan. Ignored if a scan is in progress. 
> > >                            (1): cancel           - Cancel an AF scan. This does not cause the lens to move anywhere else. Ignored if no scan is in progress. 
> > 
> > This one is missing something to make sense, at least from GStreamer point of view. When you are not starting or cancelling, what value this control should change to ? That question might be relevant for libcamera globally.
> > 
> > Without this third state, I would say this one should not be there, and replace with action signals.
> 
> 
> In libcamera - controls only 'exist' in a single request. What is an
> 'action signal' in Gstreamer?

Its more or less a method call. The main difference is that you don't need a .h
and a static link or dlopen to call it, and bindings let you call them without
the need for per-element code. This allow GStreamer plugins to offer advance
API. You can check multiudpsink as an example. It has 4 actions, "add",
"remove", "clear", this is related to the transmission address, and "get-stats",
which let you get the statistics but for a specific destination rather then the
combination (which is offered as property). It also adds 2 signals, "client-
added" and "client-remove", which is pretty common to allow a second component
monitor the sate while application adds/remove clients.

https://gstreamer.freedesktop.org/documentation/udp/multiudpsink.html?gi-language=c#action-signals

This discussion kind of reveals that libcamera controls are many different
mechanisms under a single umbrella.

You have the controls that has a known state, that you can write in one request,
and read back on the response of that request. And that you don't expect the
state to change unless requested. These are 1:1 match with properties
(properties can be asynchronous, and we have a notifiy signal to let application
know when applied). The difference is that the yaml that serves as a spec is not
abstract the ranges and default values. This is manageable, libcamera way of
specifying control will just pass through the GStreamer element, and the
libcamera based GStreamer applications will implement per camera glue to make
things work, just like they would have to do by using the libcamera API
directly. In reality, folks will hard code their app to the Raspberry Pi way,
and it will break on other platform, but that is not my battle ;-P

There is metadata (the states in their documentation), which are not expected to
be set in a request, but only read. This is also good fit for read-only
properties. Though, I kind of suspect that the specification leave it open in
regard to these being per-stream (which which case they should be pad
properties), or for the entire camera (libcamerasrc property).

> 
> 
> So - AfTrigger for start and cancel are, in this case 'events'
> 
>  - "Start an AFscan at this request"
>  - "Cancel any active scan"

And finally, there is triggers. This is the same concept as V4L2 buttons, except
that you group multiple actions in the same button by passing an argument. That
is not a property. If we introduce a classification for settables, metadata and
triggers, that will be enough for the python generator to avoid this. My
recommandation Jaslo, would be to try and reduce the set of properties we expose
in a first pass, or make them opt-in.

> 
> 
> There are no 'events' in between. Controls are only 'set' in Requests.
> Any 'response' is only available in the metadata from a completed
> Request.
> 
> You can not 'read' the state of a control in the same sense of a V4L2
> control.
> 
> (Maybe this should change for some use cases, or for an abilty to set
> controls directly on a camera, which I believe Stefan has explored, and
> not through a request, but it's the current model).

I haven't read the implementation yet, but indeed, this requires a bit of glue.
There will be a config that is what has been set since the last request (this is
mostly likely the case already). As soon as we have a response, we can save
everything, and use that to reply to the get() calls. It always good though to
filter unsupported controls before making a request, otherwise the warning
message will repeat every request, and that will be very spammy.

> 
> > >   analogue-gain       : Analogue gain value applied in the sensor device. The value of the control specifies the gain multiplier applied to all colour channels. This value cannot be lower than 1.0. Setting this value means that it is now fixed and the AE algorithm may not change it. Setting it back to zero returns it to the control of the AE algorithm. See also: exposure-time, ae-enable. Todo: Document the interactions between AeEnable and setting a fixed value for this control. Consider interactions with other AE features, such as aperture and aperture/shutter priority mode, and decide if control of which features should be automatically adjusted shouldn't better be handled through a separate AE mode control. 
> > >                         flags: readable, writable, controllable
> > >                         Float. Range:   -3.402823e+38 -    3.402823e+38 Default:               0 
> > 
> > I'm not confident there is no range, would be nice if we can align to something sensible (might need yaml changes of course).
> 
> Different cameras will have different ranges ... only determinable at
> runtime. We can't specify this in YAML. Though - it should always be
> positive at least! (and as you've mentioned elsewhere, if it was

Well, that would already be an improvement, signed / unsigned. Will it fit 32bit
? 64bit, it is float or double ? etc.

> machine readable to say always above 1.0 would be beneficial too).

Notice that with proper types, there is always -INF and INF defined, and worse
case, we can define that.

> 

[...]

> 
> > I think my comments begins to be repetitive, and if we solve the earlier issues
> > we will solve them all. The main thing is that the yaml should explicitly define
> > more stuff rather then relying to text. And using that, we can easily adjust
> 
> Agreed, I think at least the yaml files could specify if a control can
> only be accessed as metadata which would simplify all of the 'read-only'
> issues.

Ack, we should propose some improvement to the yaml.

> 
> > GStreamer properties. A final note, not all of these will be supported. I was
> > wondering if we should add an extra property that tells use which one are
> > actually supported ? We don't need anything fency, a "/" seperate list in a
> > string or similar could do. We can also go for GstStructure.
> 
> I don't understand this part fully. Do you mean a new control/property
> that gstreamer libcamerasrc will read at runtime from libcamera? It
> already has a Camera->controls() that it can read that from.
> 
> But you mention GstStructure - do you mean that the libcamerasrc will
> generate this ? (Perhaps from the Camera->controls()?)

I think you see core libcamera changes in every comments (must feel discouraging
:-D), This is in regard to how we glue the two worlds. I'd like to see
"property" that reports the list of supported controls, or the map of supported
controls (with the run-time limits).

> 
> > Nicolas
> 
> Kieran