[libcamera-devel] [PATCH v2 00/17] IPU3 control info update and HAL frame durations

[Jacopo Mondi]

As the patch subject suggests, this series is composed of two main parts, which
concur in correctly handling the frame durations reported to the Android HAL.

The series starts by allowing the IPU3 pipeline handler to select the optimal
sensor size to feed the ImgU with. Umang's series is a requirement for this
patch, otherwise the ImgU configuration fails.

Patches 2 and 3 update the Camera's ControlInfoMap in response to a
Camera::configure() call, by updating the controls limits in the ph and the IPA.

>From patch 4 on the Android HAL parts begins by collecting per-stream frame
durations. As the frame durations are updated as part of the Camera's
ControlInfoMap limits update it is necessary to configure the camera during the
HAL startup. A few cleanup patches follows, including the correct handling of
STALL durations

Patch 10 introduces a filtering criteria for YUV streams: only streams that can
produce 30 FPS can be registered for preview. I was not able to find this
clearly specified in documentation but the requirement has been confirmed by the
cros camera team and in the Intel HAL implementation.

After two more cleanup patches the way
ANDROID_CONTROL_AE_AVAILABLE_TARGET_FPS_RANGES gets populated is changed to
comply with the documentation. Note that this does not align with the Intel HAL
implementation which registers 8 values instead of 4.

The last part of the series is there mostly for comments.
The idea is that streams capable of running at more than 30 FPS should be
limited to such frame rate, as it has proven to be the optimal one for the
platform, and including more frame rates (in example ov5670 is capable of 60FPS
at smaller resolutions) would require a different handling of
ANDROID_CONTROL_AE_AVAILABLE_TARGET_FPS_RANGES which I've not been able to fully
clarify yet.

As a confirmation, the Intel HAL limits all its streams to 30 FPS as well for
IQ optimization reasons.

The series contains one patch to limit the reported duration to 30 FPS and
please CTS without actually limiting the frame rate.
The patch is then reverted and the same mechanism is implemented in the IPA,
this time actually setting the right vblank to maintain the 30 FPS frame rate
(confirmed inspecting timestamps reported by 'cam').

With both solutions applied I got several 0 failures CTS runs. With the HAL-only
solution the flaky recording tests failed once, with the IPA solution I got not
failures in 3 CTS runs.

v1->v2:
- add a patch to centralize the ImgU sizes definition
- small grammar/spelling fixes here and there
- Collect the actual minFPS when populating AE_AVAILABLE_FPS_RANGE instead of
  limiting it to 15, as suggested by Paul.

Thanks
   j

Jacopo Mondi (17):
  libcamera: ipu3: Use the optimal sensor size
  libcamera: ipu3: Centralize ImgU sizes definition
  libcamera: ipu3: Split controls init/update
  ipa: ipu3: Update camera controls in configure()
  android: capabilities: Collect per-stream frame durations
  android: capabilities: Initialize camera state when building
    properties
  android: capabilties: Assume controls::FrameDurationLimits is
    supported
  android: capabilities: Use per-configuration durations
  android: capabilties: Correctly populate STALL durations
  android: capabilities: Collect absolute max frame durations
  android: Filter preview streams on FPS
  android: capabilities: Print output stream list
  android: Populate streams and duration in the same loop
  android: capabilties: Fix
    ANDROID_CONTROL_AE_AVAILABLE_TARGET_FPS_RANGES
  android: capabilities: Cap frame rate to 30 FPS
  Revert "android: capabilities: Cap frame rate to 30 FPS"
  ipa: ipu3: Cap frame duration to 30 FPS

 include/libcamera/ipa/ipu3.mojom     |   3 +-
 src/android/camera_capabilities.cpp  | 205 +++++++++++++++++----------
 src/android/camera_capabilities.h    |   3 +
 src/ipa/ipu3/ipu3.cpp                | 119 ++++++++++++----
 src/libcamera/pipeline/ipu3/imgu.cpp |  86 +++++------
 src/libcamera/pipeline/ipu3/imgu.h   |  27 ++++
 src/libcamera/pipeline/ipu3/ipu3.cpp | 119 ++++++++++------
 7 files changed, 357 insertions(+), 205 deletions(-)

--
2.32.0