[libcamera-devel] A proposal about implementation rules and return values for CameraConfiguration::validate()

[Hirokazu Honda]

Hi,

I look at validate() implementations of CameraConfigurations (e.g.
IPU3CameraConfiguration) thinking of mapping the accepted
configurations to ones requested by Android HAL client.
CameraConfiguration::validate() returns Valid, Adjusted and Invalid.
>From the implementations, each of them is returned
[Valid]
when all configurations are accepted by a native camera. In other
words, the native camera will produce the same streams as requested.
[Adjusted]
when some configurations can be accepted by native camera, while some
of them are modified like a dimension is aligned and a format is
changed.
[Invalid]
when there is a configuration that cannot be accepted by a native camera.

Although these may not match the definitions, the implementations look
to follow definitions like this.
However, the difference between Adjusted and Invalid is very ambiguous.
I find two problems.

First, which value is returned Adjusted or Invalid, is dependent on
the order of configurations and CameraConfiguration implementation,
because present CameraConfiguration classes
1.) drop given configurations and regard as Adjusted e.g. [1], and
2.) returns Invalid if there is a configuration that cannot be
accepted by a native camera. e.g. [2, 3]
For example, if the configuration is {acceptable, non-acceptable,
non-acceptable}, a CameraConfiguration returns Adjusted if it drops
the last two ones thanks to 1.).
On the other hand, if the configuration is {non-acceptable,
acceptable, no-acceptable}, a CameraConfiguration returns Invalid
because the first one is configured to a native camera thanks to 2.).

Let's consider the second problem with another example, Android HAL
requests NV12 360p and MJPEG 360p, and a native camera can output only
a single stream, not both of them, at the same time.
Libcamera Android HAL adaptation layer executes
Configuration::validate() with them.
The validate() returns Invalid because both of them cannot be produced
simultaneously.
However, then Android HAL adaptation layer may want to reproduce MJPEG
360p from NV12 360p.
We would like to know "best-effort" accepted configurations and manage
to produce all the requested streams from the accepted configurations.

My proposal is to
1.) Change the Invalid definition to "no configuration is accepted or
other critical fatals like config is empty".
2.) [Filter-rule] When CameraConfiguration drops given configurations,
it drops ones given later. In other words, earlier the configuration
is put in cong lists, higher priority the config has.
3.) [Best-effort config] When CameraConfiguration find a native camera
cannot accept all the configurations, it tries to try the given
configurations from begin to end increasing one by one.
4.) (Optional) Add a new return value between Adjusted and Invalid
like Dropped/Filtered, so that we can know not all the streams are
accepted. Though we can detect this in a caller by seeing the returned
config.

Any comments are appreciated.
Thanks in advance!

Best Regards,
-Hiro
---------
[1] https://git.linuxtv.org/libcamera.git/tree/src/libcamera/pipeline/ipu3/ipu3.cpp#n154
[2] https://git.linuxtv.org/libcamera.git/tree/src/libcamera/pipeline/ipu3/ipu3.cpp#n175
[3] https://git.linuxtv.org/libcamera.git/tree/src/libcamera/pipeline/ipu3/ipu3.cpp#n306