[libcamera-devel] [PATCH v2 0/8] libcamera: Add DRM/KMS viewfinder display to cam

[Laurent Pinchart]

Hello,

This patch series extends the cam application to implement live
viewfinder display using a DRM/KMS device. Supported display features
are currently fairly limited (scaling and overlays are not supported for
instance), but should nonetheless be a good first step.

Patches 1/8 extends to EventLoop class to support file descriptor
events, needed by KMS. Patches 2/8 to 4/8 then refactor the cam
application and the BufferWriter class to introduce an abstract
FrameSink class. Patch 5/8 adds a set of classes to wrap the DRM/KMS API
(using libdrm). Patches 6/8 and 7/8 add a DRM/KMS frame sink
implementation, and patch 8/8 wires it up in the cam application.

I've kept patch 7/8 separate from 6/8 from now as I'm not entirely sure
what the best course of action is. Enabling the display is a costly
operation, so doing it at start() time would be best. However, we would
need a frame buffer to be displayed at that point, and none is available
yet from the camera. Patch 7/8 thus delays enabling the display to the
first frame. The operation isn't too costly from a userspace point of
view as the atomic request is committed asynchronously (although it
still requires two ioctl calls, one to create the mode blob - but this
could be moved to start() - and one to commit the request). Another
option would be to allocate an extra dumb buffer and fill it with static
data for the first frame.

I've tested this with the simple pipeline handler on an R-Car H3
Salvator-XS board, running

cam -c 'platform/vimc.0 Sensor B' \
	-s pixelformat=BGR888,width=1440,height=900 -C -D HDMI-A-1

with vimc as a source. As the R-Car DU requires DMA contiguous memory by
default (IOMMU support is enabled), a patch is required to use the vb2
dma-contig allocator in vimc. It can be found at [1].

The size of the stream has to match the size of the display, as scaling
isn't supported yet. As vimc requires the output size to be a multiple
of 3, this restricts the possible resolutions (we should *really* fix
this in the vimc driver).

Compared to v1, the frame sink implementation is now based on requests
instead of buffers, which should address Niklas' concerns.

[1] https://lore.kernel.org/linux-media/20210730001939.30769-1-laurent.pinchart+renesas@ideasonboard.com/T/#u

Laurent Pinchart (8):
  cam: event_loop: Add support for file descriptor events
  cam: Add FrameSink base class
  cam: Turn BufferWriter into a FrameSink
  cam: Rename BufferWriter to FileSink
  cam: Add DRM helper classes
  cam: Add KMS sink class
  cam: kms_sink: Enable display on first frame
  cam: Add support for viewfinder through DRM/KMS

 src/cam/buffer_writer.h                      |  31 -
 src/cam/camera_session.cpp                   |  92 ++-
 src/cam/camera_session.h                     |   6 +-
 src/cam/drm.cpp                              | 663 +++++++++++++++++++
 src/cam/drm.h                                | 331 +++++++++
 src/cam/event_loop.cpp                       |  44 ++
 src/cam/event_loop.h                         |  20 +
 src/cam/{buffer_writer.cpp => file_sink.cpp} |  43 +-
 src/cam/file_sink.h                          |  35 +
 src/cam/frame_sink.cpp                       |  31 +
 src/cam/frame_sink.h                         |  34 +
 src/cam/kms_sink.cpp                         | 289 ++++++++
 src/cam/kms_sink.h                           |  74 +++
 src/cam/main.cpp                             |   6 +
 src/cam/main.h                               |   1 +
 src/cam/meson.build                          |  17 +-
 16 files changed, 1664 insertions(+), 53 deletions(-)
 delete mode 100644 src/cam/buffer_writer.h
 create mode 100644 src/cam/drm.cpp
 create mode 100644 src/cam/drm.h
 rename src/cam/{buffer_writer.cpp => file_sink.cpp} (68%)
 create mode 100644 src/cam/file_sink.h
 create mode 100644 src/cam/frame_sink.cpp
 create mode 100644 src/cam/frame_sink.h
 create mode 100644 src/cam/kms_sink.cpp
 create mode 100644 src/cam/kms_sink.h

-- 
Regards,

Laurent Pinchart