[libcamera-devel] [PATCH 00/13] libcamera: pipeline: simple: Support the NXP i.MX8 ISI

[Laurent Pinchart]

Hello,

This patch series adds support for the NXP i.MX8 ISI to the simple
pipeline handler.

The ISI (which stands for Image Sensor Interface) is an IP core found in
multiple SoCs from the NXP i.MX8 family. It combines a crossbar switch
with processing pipelines that each include a scaler, a colorspace
converter and a DMA engine to capture frames from multiple inputs to
memory. As data flows linearly from the source to the DMA engine, the
ISI is a good candidate for the simple pipeline handler (while the ISI
can be used with Bayer sensors by bypassing the processing pipeline to
capture raw data, most use cases will use RGB or YUV sources). 

The ISI driver has been posted to the linux-media mailing list,
currently as a v2 ([1]). It depends on the V4L2 routing API to control
the crossbar switch, and as the API is still under development ([2]),
this blocks merging the driver for the time being. However, the routing
API is now at v12, and the development effort isn't slowing down, we may
consider merging support for the API in libcamera before it hits
upstream. I would appreciate feedback on this.

The series starts with three patches that update the Linux kernel
header, with 01/13 moving to headers from kernel v5.19, and patches
02/13 and 03/13 adding new pixel formats that have been merged in
subsystem trees and will be present in v5.20. Patch 04/13 then adds
support for those formats to libcamera. If we decide that the routing
API shouldn't be merged in libcamera yet, these first four patches are
still candidates for upstreaming without delay.

Patch 05/13 then adds the V4L2 routing API to the kernel headers, and
patches 06/13 to 08/13 extend the V4L2Subdevice class to support the
API. Patches 09/13 to 12/13 extend the simple pipeline handler to take
routes into account during pipeline discovery and configuration.
Finally, patch 13/13 enables support for the ISI in the simple pipeline
handler.

[1] https://lore.kernel.org/linux-media/20220712000251.13607-1-laurent.pinchart@ideasonboard.com/
[2] https://lore.kernel.org/linux-media/20220301161156.1119557-1-tomi.valkeinen@ideasonboard.com/

Jacopo Mondi (4):
  libcamera: v4l2_subdevice: Change V4L2Subdevice::Whence
  libcamera: v4l2_subdevice: Collect subdev capabilities
  libcamera: v4l2_subdevice: Add support for the V4L2 subdev routing API
  libcamera: pipeline: simple: Reset routing table of subdevs

Laurent Pinchart (7):
  include: linux: Update kernel headers to version v5.19
  include: drm_fourcc: Add AVUY and XVUY 4:4:4 packet formats
  include: linux: Add V4L2 YUVA32 and YUVX32 pixel formats
  libcamera: formats: Add AVUY8888 and XVUY8888 formats
  include: linux: Add V4L2 subdev internal routing API
  libcamera: pipeline: simple: Setup links in the context of sink
    entities
  libcamera: pipeline: simple: Add support for NXP ISI

Phi-Bang Nguyen (2):
  libcamera: pipeline: simple: Walk pipeline using subdev internal
    routing
  libcamera: pipeline: simple: Don't disable links carrying other
    streams

 include/libcamera/internal/v4l2_subdevice.h |  28 +-
 include/linux/README                        |   2 +-
 include/linux/dma-buf.h                     |   4 +-
 include/linux/drm_fourcc.h                  |  88 +++++-
 include/linux/intel-ipu3.h                  |  35 ++-
 include/linux/v4l2-controls.h               | 301 +++++++++++++++++++-
 include/linux/v4l2-subdev.h                 |  88 +++++-
 include/linux/videodev2.h                   |  15 +-
 src/libcamera/formats.cpp                   |  26 ++
 src/libcamera/formats.yaml                  |   4 +
 src/libcamera/pipeline/simple/simple.cpp    | 173 ++++++++++-
 src/libcamera/v4l2_pixelformat.cpp          |   4 +
 src/libcamera/v4l2_subdevice.cpp            | 161 ++++++++++-
 13 files changed, 877 insertions(+), 52 deletions(-)


base-commit: 69ae75b0cc211f82665b3e92fb3de64a9852b403
-- 
Regards,

Laurent Pinchart