[libcamera-devel] [PATCH RFC 0/7] android: add YUYV->NV12 conversion via libyuv

[Laurent Pinchart]

Hi Mattijs,

On Fri, Sep 15, 2023 at 09:57:24AM +0200, Mattijs Korpershoek via libcamera-devel wrote:
> On some platforms, it's possible that the gralloc implementation
> and the CSI receiver cannot agree on a pixel format.
> When that happens, there is usually a m2m converter in the pipeline
> which handles pixel format conversion.
> 
> On platforms without pixel format converters, such as the AM62x,
> we need to do software conversion.
> 
> The AM62x platform:
> * uses a CSI receiver (j721e-csi2rx), that only supports
>   packed YUV422 formats such as YUYV, YVYU, UYVY and VYUY.
> * Has a gralloc implementation that only supports of semi-planar
>   YUV420 formats such as NV12.

:-(

Is this because the display hardware doesn't support YUYV, or because
it's not implement in TI's gralloc ? A quick look at the TRM shows that
the DSS supports packed YUV inputs.

> This series add support for software conversion using libyuv.
> Right now, only YUYV->NV12 is supported.
> 
> This has been send as an RFC because:
> 
> 1. I don't have much media knowledge, nor C++ knowledge so this is
>    probably not good enough for libcamera's high quality codebase.

I'll take that as a compliment, but there's no need to denigrate your
skills like that :-)

> 2. I'm not sure that transforming the "main Type::Direct" stream into
>    an Internal stream has no side effects.
> 
> 3. I'm unsure what to do with the gralloc size in case of conversion.
>    Right now, we always allocate HAL_PIXEL_FORMAT_YCBCR_420_888 in
>    CameraStream::getBuffer(). This should be adapted both on:
>    - the gralloc capabilities of the platform
>    - the conversion (if any)
> 
> This has been tested on a AM62x SK EVM board with an Alinx
> AN5641 (ov5640) sensor.

That sensor should have disappeared a looooong time ago...

> In Android 13.
> It might work on BeaglePlay with the same sensor, but I have not
> tested that.
> 
> Both preview and still capture are functional with the
> default com.android.camera2 app.
> Note that video recording is not supported, because that would require
> a similar conversion from YUYV->MJPEG which I did not investigate yet.
> 
> I would appreciate some feedback. Thanks a lot in advance if you
> have a look!
> 
> Depends on:
> - https://patchwork.libcamera.org/cover/18995/
> 
> Signed-off-by: Mattijs Korpershoek <mkorpershoek at baylibre.com>
> ---
> Mattijs Korpershoek (7):
>       android: yuv: separate source destination in length check
>       android: yuv: loop over each plane for size check
>       android: yuv: prepare support for other pixel formats
>       android: camera_device: support when no Direct stream is found
>       android: camera_stream: add add explicit input configuration
>       android: yuv: add YUYV -> NV12 conversion
>       WIP: android: add YUYV->NV12 format conversion via libyuv
> 
>  src/android/camera_capabilities.cpp    |  90 ++++++++++++++++++-
>  src/android/camera_capabilities.h      |   4 +
>  src/android/camera_device.cpp          |   9 +-
>  src/android/camera_stream.cpp          |  57 +++++++++++-
>  src/android/camera_stream.h            |   5 ++
>  src/android/yuv/post_processor_yuv.cpp | 156 ++++++++++++++++++++++-----------
>  src/android/yuv/post_processor_yuv.h   |  12 ++-
>  7 files changed, 270 insertions(+), 63 deletions(-)
> ---
> base-commit: 58e501c71c47e57f02afde1bd296a037038cd6d5
> change-id: 20230914-libyuv-convert-fec7082343e4

-- 
Regards,

Laurent Pinchart