[libcamera-devel] [PATCH RFC 0/7] android: add YUYV->NV12 conversion via libyuv

[Mattijs Korpershoek]

On some platforms, it's possible that the gralloc implementation
and the CSI receiver cannot agree on a pixel format.
When that happens, there is usually a m2m converter in the pipeline
which handles pixel format conversion.

On platforms without pixel format converters, such as the AM62x,
we need to do software conversion.

The AM62x platform:
* uses a CSI receiver (j721e-csi2rx), that only supports
  packed YUV422 formats such as YUYV, YVYU, UYVY and VYUY.
* Has a gralloc implementation that only supports of semi-planar
  YUV420 formats such as NV12.

This series add support for software conversion using libyuv.
Right now, only YUYV->NV12 is supported.

This has been send as an RFC because:

1. I don't have much media knowledge, nor C++ knowledge so this is
   probably not good enough for libcamera's high quality codebase.

2. I'm not sure that transforming the "main Type::Direct" stream into
   an Internal stream has no side effects.

3. I'm unsure what to do with the gralloc size in case of conversion.
   Right now, we always allocate HAL_PIXEL_FORMAT_YCBCR_420_888 in
   CameraStream::getBuffer(). This should be adapted both on:
   - the gralloc capabilities of the platform
   - the conversion (if any)

This has been tested on a AM62x SK EVM board with an Alinx
AN5641 (ov5640) sensor. In Android 13.
It might work on BeaglePlay with the same sensor, but I have not
tested that.

Both preview and still capture are functional with the
default com.android.camera2 app.
Note that video recording is not supported, because that would require
a similar conversion from YUYV->MJPEG which I did not investigate yet.

I would appreciate some feedback. Thanks a lot in advance if you
have a look!

Depends on:
- https://patchwork.libcamera.org/cover/18995/

Signed-off-by: Mattijs Korpershoek <mkorpershoek at baylibre.com>
---
Mattijs Korpershoek (7):
      android: yuv: separate source destination in length check
      android: yuv: loop over each plane for size check
      android: yuv: prepare support for other pixel formats
      android: camera_device: support when no Direct stream is found
      android: camera_stream: add add explicit input configuration
      android: yuv: add YUYV -> NV12 conversion
      WIP: android: add YUYV->NV12 format conversion via libyuv

 src/android/camera_capabilities.cpp    |  90 ++++++++++++++++++-
 src/android/camera_capabilities.h      |   4 +
 src/android/camera_device.cpp          |   9 +-
 src/android/camera_stream.cpp          |  57 +++++++++++-
 src/android/camera_stream.h            |   5 ++
 src/android/yuv/post_processor_yuv.cpp | 156 ++++++++++++++++++++++-----------
 src/android/yuv/post_processor_yuv.h   |  12 ++-
 7 files changed, 270 insertions(+), 63 deletions(-)
---
base-commit: 58e501c71c47e57f02afde1bd296a037038cd6d5
change-id: 20230914-libyuv-convert-fec7082343e4

Cheers,
-- 
Mattijs