[libcamera-devel] Expected Request queuing behavior by apps

[Umang Jain]

Hi everyone,

Goal:
I am kicking-off a thread to discuss the Request queuing behaviour that 
can be expected by libcamera  from application's side. In the recent 
discussions, we have found that we don't yet advertise or document this 
aspect explicitly. Hence, it would be helpful to do so, especially it 
helps with internal plumbing of requests routing through 
pipeline-handlers(s), IPA and various requests holding placeholders for 
state tracking (queues, arrays, FCQueue etc.)  or even post-processing.

Assumptions up till this point:
Atleast what _I_ have assumed up until this point is that applications 
queue a certain number of requests (let's say 4) and then continuing 
queuing the request from the `requestCompleted` callback handler (slot). 
I see this behaviour implemented in our `libcamera/src/apps`. This 
assumption inherently suggests that these applications, are not trying 
to over-queue to the hardware hence, respecting a certain depth of the 
hardware pipeline. They makes sure new requests get queued when previous 
ones gets completed. Other argument can be that, reuse of Request 
objects are cheaper than constructing a new one for each queueRequest() 
and queuing all at once.

Defining the explicit behaviour by applications:
Note, in the last segment - "Assumption". We do not explicitly say that 
the applications should queue the requests in a certain manner.

Hence, it is time to defining our expectations on what applications are 
allowed or not allowed to do, with respect of queuing requests. Below 
are a few documentation headers to move the discussion forwards.

- Exposing the hardware capabilities
Should libcamera expose the underlying hardware capabilities/constraints 
to applications and expect them to rate-limit the queuing of requests based?

- No limit
Advertise explicitly that there are no constraints in queuing Request to 
libcamera *whatsoever*, from the applications side?

- Number of requests to be queued by the application, to get X number of 
frames

         X   =    1 Frame
         X <=    frames less than processing blocks on a platform
         X   >    frames less than processing blocks on a platform

Depending on the request queuing behaviour libcamera expects, I see that 
documenting the aforementioned might be useful for applications 
developers as well (maybe this goes in per-platform information?). 
lc-compliance can incorporate checks for these as well.

These are just on top off my head, new discussions points are welcomed 
as well.

Thanks,
Umang