[libcamera-devel] [PATCH 00/20] libcamera: Multi-stream support for the simple pipeline handler

[Laurent Pinchart]

Hello,

This patch series implements multi-stream support in the simple pipeline
handler.

Multi-stream support in the capture device requires hardware processing
capabilities (at least format conversion and scaling), which would
likely not qualify the platform for support in the simple pipeline
handler. However, for platforms that include a converter, the V4L2 M2M
API allows opening multiple instances of the converter to create
multiple output streams with different formats and resolution (as long
as the hardware has enough bandwidth to run all the processing passes).
This approach is generic and doesn't require any extra hardware feature
compared to what the simple pipeline handler already uses.

The patches starts with refactoring of the converter class to prepare
for multiple streams support, and then refactors the simple pipeline
handler to achieve the same goal. The last two patches enable
multi-streams support

The code has been tested on the MediaTek Pumpkin i500 board. Its
support isn't available in the simple pipeline handler yet due to the
drivers not being upstream, which is an issue that is being addressed.
Enabling it, after importing the kernel headers that define the
MediaTek-specific pixel formats and enabling them in libcamera, only
requires the following change in the simple pipeline handler.

diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp
index 55a5528611c8..6b0170d1aba8 100644
--- a/src/libcamera/pipeline/simple/simple.cpp
+++ b/src/libcamera/pipeline/simple/simple.cpp
@@ -132,6 +132,7 @@ namespace {

 static const SimplePipelineInfo supportedDevices[] = {
 	{ "imx7-csi", "pxp", 1 },
+	{ "mtk-camsv-p1", "mtk-mdp3", 3 },
 	{ "qcom-camss", nullptr, 1 },
 	{ "sun6i-csi", nullptr, 1 },
 };

Laurent Pinchart (20):
  libcamera: pipeline: simple: Manage converter with std::unique_ptr<>
  libcamera: pipeline: simple: Don't override stride at configure time
  libcamera: pipeline: simple: converter: Group query functions together
  libcamera: pipeline: simple: converter: Use StreamConfiguration for
    input configuration
  libcamera: pipeline: simple: converter: Configure input stride
  libcamera: pipeline: simple: converter: Differentiate input and output
    buffers count
  libcamera: pipeline: simple: converter: Replace open() with isValid()
  libcamera: pipeline: simple: converter: Decouple input and output
    completion
  libcamera: pipeline: simple: converter: Add multi-stream support
  libcamera: pipeline: simple: Store streams in a vector
  libcamera: pipeline: simple: Drop unused members of configuration
  libcamera: pipeline: simple: Document the pipeline handler design
  libcamera: pipeline: simple: Rename Configuration::pixelFormat
  libcamera: pipeline: simple: Cache pipeline config in
    SimpleCameraConfiguration
  libcamera: pipeline: simple: Add output formats to Configuration
  libcamera: pipeline: simple: Move converter data to camera data
  libcamera: pipeline: simple: Hardcode the number of internal buffers
  libcamera: pipeline: simple: Support configuration of multiple streams
  libcamera: pipeline: simple: Support usage of multiple streams
  libcamera: pipeline: simple: Enable multiple streams for compatible
    devices

 src/libcamera/pipeline/simple/converter.cpp | 398 ++++++++------
 src/libcamera/pipeline/simple/converter.h   |  60 ++-
 src/libcamera/pipeline/simple/simple.cpp    | 542 +++++++++++++-------
 3 files changed, 650 insertions(+), 350 deletions(-)

-- 
Regards,

Laurent Pinchart