[libcamera-devel] [PATCH] Documentation: Vivid Pipeline
Kieran Bingham
kieran.bingham at ideasonboard.com
Mon Aug 10 15:01:10 CEST 2020
Provide the patches for the Vivid pipeline within the Documentation tree.
These patches are not destined for direct integration, but serve as a reference
for implementing a new pipeline handler.
Keep the patches within the tree to allow tracking updates, and making them
accessible to developers.
Signed-off-by: Kieran Bingham <kieran.bingham at ideasonboard.com>
---
.../0000-cover-letter.patch | 56 ++++++
...ne-Introduce-skeleton-Vivid-Pipeline.patch | 139 +++++++++++++++
...ibcamera-pipeline-vivid-Match-device.patch | 54 ++++++
...amera-pipeline-vivid-Create-a-Camera.patch | 110 ++++++++++++
...ne-vivid-Generate-and-validate-Strea.patch | 160 ++++++++++++++++++
...-pipeline-vivid-Configure-the-device.patch | 71 ++++++++
...ne-vivid-Buffer-handling-and-stream-.patch | 102 +++++++++++
...camera-pipeline-vivid-Queue-requests.patch | 45 +++++
...peline-vivid-Initialise-key-controls.patch | 70 ++++++++
...amera-pipeline-vivid-Handle-controls.patch | 133 +++++++++++++++
10 files changed, 940 insertions(+)
create mode 100644 Documentation/vivid-pipeline-handler/0000-cover-letter.patch
create mode 100644 Documentation/vivid-pipeline-handler/0001-libcamera-pipeline-Introduce-skeleton-Vivid-Pipeline.patch
create mode 100644 Documentation/vivid-pipeline-handler/0002-libcamera-pipeline-vivid-Match-device.patch
create mode 100644 Documentation/vivid-pipeline-handler/0003-libcamera-pipeline-vivid-Create-a-Camera.patch
create mode 100644 Documentation/vivid-pipeline-handler/0004-libcamera-pipeline-vivid-Generate-and-validate-Strea.patch
create mode 100644 Documentation/vivid-pipeline-handler/0005-libcamera-pipeline-vivid-Configure-the-device.patch
create mode 100644 Documentation/vivid-pipeline-handler/0006-libcamera-pipeline-vivid-Buffer-handling-and-stream-.patch
create mode 100644 Documentation/vivid-pipeline-handler/0007-libcamera-pipeline-vivid-Queue-requests.patch
create mode 100644 Documentation/vivid-pipeline-handler/0008-libcamera-pipeline-vivid-Initialise-key-controls.patch
create mode 100644 Documentation/vivid-pipeline-handler/0009-libcamera-pipeline-vivid-Handle-controls.patch
diff --git a/Documentation/vivid-pipeline-handler/0000-cover-letter.patch b/Documentation/vivid-pipeline-handler/0000-cover-letter.patch
new file mode 100644
index 000000000000..25e61d4313ff
--- /dev/null
+++ b/Documentation/vivid-pipeline-handler/0000-cover-letter.patch
@@ -0,0 +1,56 @@
+From 03bbac3265554b73e6ddcdebaf95090062e3669f Mon Sep 17 00:00:00 2001
+From: Kieran Bingham <kieran.bingham at ideasonboard.com>
+Date: Mon, 10 Aug 2020 13:28:23 +0100
+Subject: [PATCH 0/9] libcamera: pipeline: Add a VIVID Pipeline Handler
+
+Introduce a new pipeline handler to support the virtual video test
+driver provided by V4L2.
+
+These patches are intended to serve as a code-guide for writing a new
+pipeline handler, and can be referenced along side the Pipeline Handler
+Writers Guide documentation.
+
+To test these patches, apply locally to your libcamera sources by
+executing the following:
+
+ git am Documentation/vivid-pipeline-handler/*
+
+Then test by ensuring you have the vivid module loaded:
+
+ sudo modprobe vivid
+
+To enable the new pipeline handler the build must be re-configured. A
+limitation in the meson build system means that this requires a full
+reconfiguration. The easiest solution of which is to remove any existing
+build and commence with a fresh build tree.
+
+Configure, build and test:
+
+ rm -r build
+ meson build
+ cd build
+ ninja
+
+
+
+Kieran Bingham (9):
+ libcamera: pipeline: Introduce skeleton Vivid Pipeline
+ libcamera: pipeline: vivid: Match device
+ libcamera: pipeline: vivid: Create a Camera
+ libcamera: pipeline: vivid: Generate and validate StreamConfigurations
+ libcamera: pipeline: vivid: Configure the device
+ libcamera: pipeline: vivid: Buffer handling and stream control
+ libcamera: pipeline: vivid: Queue requests
+ libcamera: pipeline: vivid: Initialise key controls
+ libcamera: pipeline: vivid: Handle controls
+
+ meson_options.txt | 2 +-
+ src/libcamera/pipeline/vivid/meson.build | 5 +
+ src/libcamera/pipeline/vivid/vivid.cpp | 382 +++++++++++++++++++++++
+ 3 files changed, 388 insertions(+), 1 deletion(-)
+ create mode 100644 src/libcamera/pipeline/vivid/meson.build
+ create mode 100644 src/libcamera/pipeline/vivid/vivid.cpp
+
+--
+2.25.1
+
diff --git a/Documentation/vivid-pipeline-handler/0001-libcamera-pipeline-Introduce-skeleton-Vivid-Pipeline.patch b/Documentation/vivid-pipeline-handler/0001-libcamera-pipeline-Introduce-skeleton-Vivid-Pipeline.patch
new file mode 100644
index 000000000000..ca1dc48e4fb9
--- /dev/null
+++ b/Documentation/vivid-pipeline-handler/0001-libcamera-pipeline-Introduce-skeleton-Vivid-Pipeline.patch
@@ -0,0 +1,139 @@
+From 2480348e61bfcbc8a113a03e6090308f9b904acb Mon Sep 17 00:00:00 2001
+From: Kieran Bingham <kieran.bingham at ideasonboard.com>
+Date: Fri, 10 Jul 2020 15:13:05 +0100
+Subject: [PATCH 1/9] libcamera: pipeline: Introduce skeleton Vivid Pipeline
+
+Provide all of the skeleton stubs to succesfully compile
+and register a new Pipeline Handler for the Vivid test device.
+
+Meson must be reconfigured to ensure that this pipeline handler is
+included in the selected pipelines configuration, and after building, we
+can test that the PipelineHandler is successfully registered by listing
+the cameras on the system with LIBCAMERA_LOG_LEVELS enabled:
+
+"""
+LIBCAMERA_LOG_LEVELS=Pipeline:0 ./build-vivid/src/cam/cam -l
+[230:30:03.624102821] [2867886] DEBUG Pipeline pipeline_handler.cpp:680 Registered pipeline handler "PipelineHandlerVivid"
+Available cameras:
+"""
+
+Signed-off-by: Kieran Bingham <kieran.bingham at ideasonboard.com>
+---
+ meson_options.txt | 2 +-
+ src/libcamera/pipeline/vivid/meson.build | 5 ++
+ src/libcamera/pipeline/vivid/vivid.cpp | 78 ++++++++++++++++++++++++
+ 3 files changed, 84 insertions(+), 1 deletion(-)
+ create mode 100644 src/libcamera/pipeline/vivid/meson.build
+ create mode 100644 src/libcamera/pipeline/vivid/vivid.cpp
+
+diff --git a/meson_options.txt b/meson_options.txt
+index e9e815fde366..ea38f1553240 100644
+--- a/meson_options.txt
++++ b/meson_options.txt
+@@ -16,7 +16,7 @@ option('gstreamer',
+
+ option('pipelines',
+ type : 'array',
+- choices : ['ipu3', 'raspberrypi', 'rkisp1', 'simple', 'uvcvideo', 'vimc'],
++ choices : ['ipu3', 'raspberrypi', 'rkisp1', 'simple', 'uvcvideo', 'vimc', 'vivid'],
+ description : 'Select which pipeline handlers to include')
+
+ option('qcam',
+diff --git a/src/libcamera/pipeline/vivid/meson.build b/src/libcamera/pipeline/vivid/meson.build
+new file mode 100644
+index 000000000000..086bb825387c
+--- /dev/null
++++ b/src/libcamera/pipeline/vivid/meson.build
+@@ -0,0 +1,5 @@
++# SPDX-License-Identifier: CC0-1.0
++
++libcamera_sources += files([
++ 'vivid.cpp',
++])
+diff --git a/src/libcamera/pipeline/vivid/vivid.cpp b/src/libcamera/pipeline/vivid/vivid.cpp
+new file mode 100644
+index 000000000000..4418f616fe84
+--- /dev/null
++++ b/src/libcamera/pipeline/vivid/vivid.cpp
+@@ -0,0 +1,78 @@
++/* SPDX-License-Identifier: LGPL-2.1-or-later */
++/*
++ * Copyright (C) 2020, Google Inc.
++ *
++ * vivid.cpp - Pipeline handler for the vivid capture device
++ */
++
++#include "libcamera/internal/log.h"
++#include "libcamera/internal/pipeline_handler.h"
++
++namespace libcamera {
++
++LOG_DEFINE_CATEGORY(VIVID)
++
++class PipelineHandlerVivid : public PipelineHandler
++{
++public:
++ PipelineHandlerVivid(CameraManager *manager);
++
++ CameraConfiguration *generateConfiguration(Camera *camera,
++ const StreamRoles &roles) override;
++ int configure(Camera *camera, CameraConfiguration *config) override;
++
++ int exportFrameBuffers(Camera *camera, Stream *stream,
++ std::vector<std::unique_ptr<FrameBuffer>> *buffers) override;
++
++ int start(Camera *camera) override;
++ void stop(Camera *camera) override;
++
++ int queueRequestDevice(Camera *camera, Request *request) override;
++
++ bool match(DeviceEnumerator *enumerator) override;
++};
++
++PipelineHandlerVivid::PipelineHandlerVivid(CameraManager *manager)
++ : PipelineHandler(manager)
++{
++}
++
++CameraConfiguration *PipelineHandlerVivid::generateConfiguration(Camera *camera,
++ const StreamRoles &roles)
++{
++ return nullptr;
++}
++
++int PipelineHandlerVivid::configure(Camera *camera, CameraConfiguration *config)
++{
++ return -1;
++}
++
++int PipelineHandlerVivid::exportFrameBuffers(Camera *camera, Stream *stream,
++ std::vector<std::unique_ptr<FrameBuffer>> *buffers)
++{
++ return -1;
++}
++
++int PipelineHandlerVivid::start(Camera *camera)
++{
++ return -1;
++}
++
++void PipelineHandlerVivid::stop(Camera *camera)
++{
++}
++
++int PipelineHandlerVivid::queueRequestDevice(Camera *camera, Request *request)
++{
++ return -1;
++}
++
++bool PipelineHandlerVivid::match(DeviceEnumerator *enumerator)
++{
++ return false;
++}
++
++REGISTER_PIPELINE_HANDLER(PipelineHandlerVivid);
++
++} /* namespace libcamera */
+--
+2.25.1
+
diff --git a/Documentation/vivid-pipeline-handler/0002-libcamera-pipeline-vivid-Match-device.patch b/Documentation/vivid-pipeline-handler/0002-libcamera-pipeline-vivid-Match-device.patch
new file mode 100644
index 000000000000..a48eeeac1fe9
--- /dev/null
+++ b/Documentation/vivid-pipeline-handler/0002-libcamera-pipeline-vivid-Match-device.patch
@@ -0,0 +1,54 @@
+From d676cff10779a5a8f286cf0553f52c5f747436f0 Mon Sep 17 00:00:00 2001
+From: Kieran Bingham <kieran.bingham at ideasonboard.com>
+Date: Fri, 10 Jul 2020 15:37:55 +0100
+Subject: [PATCH 2/9] libcamera: pipeline: vivid: Match device
+
+Verify that we can match on our expected device(s).
+
+Use a temporary debug print to check that the pipeline finds
+our device:
+
+"""
+LIBCAMERA_LOG_LEVELS=Pipeline,VIVID:0 ./src/cam/cam -l
+<snipped>
+[230:51:10.670503423] [2872877] DEBUG VIVID vivid.cpp:81 Obtained Vivid Device
+"""
+
+Signed-off-by: Kieran Bingham <kieran.bingham at ideasonboard.com>
+---
+ src/libcamera/pipeline/vivid/vivid.cpp | 12 +++++++++++-
+ 1 file changed, 11 insertions(+), 1 deletion(-)
+
+diff --git a/src/libcamera/pipeline/vivid/vivid.cpp b/src/libcamera/pipeline/vivid/vivid.cpp
+index 4418f616fe84..9811f6ef5095 100644
+--- a/src/libcamera/pipeline/vivid/vivid.cpp
++++ b/src/libcamera/pipeline/vivid/vivid.cpp
+@@ -5,6 +5,7 @@
+ * vivid.cpp - Pipeline handler for the vivid capture device
+ */
+
++#include "libcamera/internal/device_enumerator.h"
+ #include "libcamera/internal/log.h"
+ #include "libcamera/internal/pipeline_handler.h"
+
+@@ -70,7 +71,16 @@ int PipelineHandlerVivid::queueRequestDevice(Camera *camera, Request *request)
+
+ bool PipelineHandlerVivid::match(DeviceEnumerator *enumerator)
+ {
+- return false;
++ DeviceMatch dm("vivid");
++ dm.add("vivid-000-vid-cap");
++
++ MediaDevice *media = acquireMediaDevice(enumerator, dm);
++ if (!media)
++ return false;
++
++ LOG(VIVID, Debug) << "Obtained Vivid Device";
++
++ return false; // Prevent infinite loops for now
+ }
+
+ REGISTER_PIPELINE_HANDLER(PipelineHandlerVivid);
+--
+2.25.1
+
diff --git a/Documentation/vivid-pipeline-handler/0003-libcamera-pipeline-vivid-Create-a-Camera.patch b/Documentation/vivid-pipeline-handler/0003-libcamera-pipeline-vivid-Create-a-Camera.patch
new file mode 100644
index 000000000000..1d8552752047
--- /dev/null
+++ b/Documentation/vivid-pipeline-handler/0003-libcamera-pipeline-vivid-Create-a-Camera.patch
@@ -0,0 +1,110 @@
+From f80f723a010501c6d159ae9601c2787d5d02906f Mon Sep 17 00:00:00 2001
+From: Kieran Bingham <kieran.bingham at ideasonboard.com>
+Date: Fri, 10 Jul 2020 16:28:36 +0100
+Subject: [PATCH 3/9] libcamera: pipeline: vivid: Create a Camera
+
+Create a VividCameraData inheriting from the CameraData to handle camera
+specific data, and use it to create and register the camera with the
+CameraManager.
+
+This can now be tested to see that the camera becomes available to
+applications:
+
+"""
+LIBCAMERA_LOG_LEVELS=Pipeline,VIVID:0 ./src/cam/cam -l
+[231:44:49.325333712] [2880028] INFO IPAManager ipa_manager.cpp:136 libcamera is not installed. Adding '/home/libcamera/build-vivid/src/ipa' to the IPA search path
+[231:44:49.325428449] [2880028] WARN IPAManager ipa_manager.cpp:147 No IPA found in '/usr/local/lib/x86_64-linux-gnu/libcamera'
+[231:44:49.325446253] [2880028] INFO Camera camera_manager.cpp:283 libcamera v0.0.11+713-d175334d-dirty
+Available cameras:
+1: vivid
+"""
+
+Signed-off-by: Kieran Bingham <kieran.bingham at ideasonboard.com>
+---
+ src/libcamera/pipeline/vivid/vivid.cpp | 54 +++++++++++++++++++++++++-
+ 1 file changed, 52 insertions(+), 2 deletions(-)
+
+diff --git a/src/libcamera/pipeline/vivid/vivid.cpp b/src/libcamera/pipeline/vivid/vivid.cpp
+index 9811f6ef5095..a8922ce70ed4 100644
+--- a/src/libcamera/pipeline/vivid/vivid.cpp
++++ b/src/libcamera/pipeline/vivid/vivid.cpp
+@@ -5,14 +5,46 @@
+ * vivid.cpp - Pipeline handler for the vivid capture device
+ */
+
++#include <libcamera/camera.h>
++
+ #include "libcamera/internal/device_enumerator.h"
+ #include "libcamera/internal/log.h"
++#include "libcamera/internal/media_device.h"
+ #include "libcamera/internal/pipeline_handler.h"
++#include "libcamera/internal/v4l2_videodevice.h"
+
+ namespace libcamera {
+
+ LOG_DEFINE_CATEGORY(VIVID)
+
++class VividCameraData : public CameraData
++{
++public:
++ VividCameraData(PipelineHandler *pipe, MediaDevice *media)
++ : CameraData(pipe), media_(media), video_(nullptr)
++ {
++ }
++
++ ~VividCameraData()
++ {
++ delete video_;
++ }
++
++ int init();
++
++ MediaDevice *media_;
++ V4L2VideoDevice *video_;
++ Stream stream_;
++};
++
++class VividCameraConfiguration : public CameraConfiguration
++{
++public:
++ VividCameraConfiguration();
++
++ Status validate() override;
++};
++
+ class PipelineHandlerVivid : public PipelineHandler
+ {
+ public:
+@@ -78,9 +110,27 @@ bool PipelineHandlerVivid::match(DeviceEnumerator *enumerator)
+ if (!media)
+ return false;
+
+- LOG(VIVID, Debug) << "Obtained Vivid Device";
++ std::unique_ptr<VividCameraData> data = std::make_unique<VividCameraData>(this, media);
++
++ /* Locate and open the capture video node. */
++ if (data->init())
++ return false;
++
++ /* Create and register the camera. */
++ std::set<Stream *> streams{ &data->stream_ };
++ std::shared_ptr<Camera> camera = Camera::create(this, data->video_->deviceName(), streams);
++ registerCamera(std::move(camera), std::move(data));
++
++ return true;
++}
++
++int VividCameraData::init()
++{
++ video_ = new V4L2VideoDevice(media_->getEntityByName("vivid-000-vid-cap"));
++ if (video_->open())
++ return -ENODEV;
+
+- return false; // Prevent infinite loops for now
++ return 0;
+ }
+
+ REGISTER_PIPELINE_HANDLER(PipelineHandlerVivid);
+--
+2.25.1
+
diff --git a/Documentation/vivid-pipeline-handler/0004-libcamera-pipeline-vivid-Generate-and-validate-Strea.patch b/Documentation/vivid-pipeline-handler/0004-libcamera-pipeline-vivid-Generate-and-validate-Strea.patch
new file mode 100644
index 000000000000..43a203cffa7b
--- /dev/null
+++ b/Documentation/vivid-pipeline-handler/0004-libcamera-pipeline-vivid-Generate-and-validate-Strea.patch
@@ -0,0 +1,160 @@
+From 928ab8ae5728677c21bd31309b782d0856441aa2 Mon Sep 17 00:00:00 2001
+From: Kieran Bingham <kieran.bingham at ideasonboard.com>
+Date: Fri, 10 Jul 2020 16:44:01 +0100
+Subject: [PATCH 4/9] libcamera: pipeline: vivid: Generate and validate
+ StreamConfigurations
+
+Implement the support for Generating and Validating the streams the
+Camera can provide.
+
+Vivid is a simple case with only a single stream.
+
+Test the configurations can be generated and reported with cam -I:
+
+"""
+LIBCAMERA_LOG_LEVELS=Pipeline,VIVID:0 ./src/cam/cam -c 1 -I
+[232:02:09.633067174] [2882911] INFO IPAManager ipa_manager.cpp:136 libcamera is not installed. Adding '/home//libcamera/build-vivid/src/ipa' to the IPA search path
+[232:02:09.633332451] [2882911] WARN IPAManager ipa_manager.cpp:147 No IPA found in '/usr/local/lib/x86_64-linux-gnu/libcamera'
+[232:02:09.633373414] [2882911] INFO Camera camera_manager.cpp:283 libcamera v0.0.11+714-d1ebd889-dirty
+Using camera vivid
+0: 1280x720-BGR888
+ * Pixelformat: NV21 (320x180)-(3840x2160)/(+0,+0)
+ - 320x180
+ - 640x360
+ - 640x480
+ - 1280x720
+ - 1920x1080
+ - 3840x2160
+ * Pixelformat: NV12 (320x180)-(3840x2160)/(+0,+0)
+ - 320x180
+ - 640x360
+ - 640x480
+ - 1280x720
+ - 1920x1080
+ - 3840x2160
+ * Pixelformat: BGRA8888 (320x180)-(3840x2160)/(+0,+0)
+ - 320x180
+ - 640x360
+ - 640x480
+ - 1280x720
+ - 1920x1080
+ - 3840x2160
+ * Pixelformat: RGBA8888 (320x180)-(3840x2160)/(+0,+0)
+ - 320x180
+ - 640x360
+ - 640x480
+ - 1280x720
+ - 1920x1080
+ - 3840x2160
+
+"""
+
+Signed-off-by: Kieran Bingham <kieran.bingham at ideasonboard.com>
+---
+ src/libcamera/pipeline/vivid/vivid.cpp | 74 +++++++++++++++++++++++++-
+ 1 file changed, 73 insertions(+), 1 deletion(-)
+
+diff --git a/src/libcamera/pipeline/vivid/vivid.cpp b/src/libcamera/pipeline/vivid/vivid.cpp
+index a8922ce70ed4..9e95bae8bc30 100644
+--- a/src/libcamera/pipeline/vivid/vivid.cpp
++++ b/src/libcamera/pipeline/vivid/vivid.cpp
+@@ -6,6 +6,7 @@
+ */
+
+ #include <libcamera/camera.h>
++#include <libcamera/formats.h>
+
+ #include "libcamera/internal/device_enumerator.h"
+ #include "libcamera/internal/log.h"
+@@ -63,8 +64,50 @@ public:
+ int queueRequestDevice(Camera *camera, Request *request) override;
+
+ bool match(DeviceEnumerator *enumerator) override;
++
++private:
++ int processControls(VividCameraData *data, Request *request);
++
++ VividCameraData *cameraData(const Camera *camera)
++ {
++ return static_cast<VividCameraData *>(
++ PipelineHandler::cameraData(camera));
++ }
+ };
+
++VividCameraConfiguration::VividCameraConfiguration()
++ : CameraConfiguration()
++{
++}
++
++CameraConfiguration::Status VividCameraConfiguration::validate()
++{
++ Status status = Valid;
++
++ if (config_.empty())
++ return Invalid;
++
++ /* Cap the number of entries to the available streams. */
++ if (config_.size() > 1) {
++ config_.resize(1);
++ status = Adjusted;
++ }
++
++ StreamConfiguration &cfg = config_[0];
++
++ /* Adjust the pixel format. */
++ const std::vector<libcamera::PixelFormat> formats = cfg.formats().pixelformats();
++ if (std::find(formats.begin(), formats.end(), cfg.pixelFormat) == formats.end()) {
++ cfg.pixelFormat = cfg.formats().pixelformats()[0];
++ LOG(VIVID, Debug) << "Adjusting format to " << cfg.pixelFormat.toString();
++ status = Adjusted;
++ }
++
++ cfg.bufferCount = 4;
++
++ return status;
++}
++
+ PipelineHandlerVivid::PipelineHandlerVivid(CameraManager *manager)
+ : PipelineHandler(manager)
+ {
+@@ -73,7 +116,36 @@ PipelineHandlerVivid::PipelineHandlerVivid(CameraManager *manager)
+ CameraConfiguration *PipelineHandlerVivid::generateConfiguration(Camera *camera,
+ const StreamRoles &roles)
+ {
+- return nullptr;
++ CameraConfiguration *config = new VividCameraConfiguration();
++ VividCameraData *data = cameraData(camera);
++
++ if (roles.empty())
++ return config;
++
++ std::map<V4L2PixelFormat, std::vector<SizeRange>> v4l2Formats =
++ data->video_->formats();
++ std::map<PixelFormat, std::vector<SizeRange>> deviceFormats;
++ std::transform(v4l2Formats.begin(), v4l2Formats.end(),
++ std::inserter(deviceFormats, deviceFormats.begin()),
++ [&](const decltype(v4l2Formats)::value_type &format) {
++ return decltype(deviceFormats)::value_type{
++ format.first.toPixelFormat(),
++ format.second
++ };
++ });
++
++ StreamFormats formats(deviceFormats);
++ StreamConfiguration cfg(formats);
++
++ cfg.pixelFormat = formats::BGR888;
++ cfg.size = { 1280, 720 };
++ cfg.bufferCount = 4;
++
++ config->addConfiguration(cfg);
++
++ config->validate();
++
++ return config;
+ }
+
+ int PipelineHandlerVivid::configure(Camera *camera, CameraConfiguration *config)
+--
+2.25.1
+
diff --git a/Documentation/vivid-pipeline-handler/0005-libcamera-pipeline-vivid-Configure-the-device.patch b/Documentation/vivid-pipeline-handler/0005-libcamera-pipeline-vivid-Configure-the-device.patch
new file mode 100644
index 000000000000..c750ce4e80d4
--- /dev/null
+++ b/Documentation/vivid-pipeline-handler/0005-libcamera-pipeline-vivid-Configure-the-device.patch
@@ -0,0 +1,71 @@
+From 08df6b40552e67c3607aa1d6802f47beec76e151 Mon Sep 17 00:00:00 2001
+From: Kieran Bingham <kieran.bingham at ideasonboard.com>
+Date: Mon, 13 Jul 2020 09:51:51 +0100
+Subject: [PATCH 5/9] libcamera: pipeline: vivid: Configure the device
+
+When the configurations have been generated and validated, they can be
+applied to a device.
+
+Vivid supports only a single stream, so it directly obtains the first
+StreamConfiguration from the CameraConfiguration.
+
+The VIVID catpure device is a V4L2Video device, so we generate a
+V4L2DeviceFormat to apply directly to the capture device node.
+
+Note that we explicitly convert the libcamera Format stored in
+cfg.pixelFormat to a V4L2PixelFormat using the helpers provided by the
+V4L2VideoDevice to ensure that any multiplanar formats are handled
+correctly and accordingly.
+
+Following the call to set the format using the Kernel API, if the format
+has been adjusted in any way by the kernel driver, then we have failed
+to correctly handle the validation stages, and thus the configure
+operation is idendified has having failed.
+
+Finally stream specific data can be directly stored and set as
+reflecting the state of the stream.
+
+[NOTE: the cfg.setStream() call here associates the stream to the
+StreamConfiguration however that should quite likely be done as part of
+the validation process. TBD]
+
+Signed-off-by: Kieran Bingham <kieran.bingham at ideasonboard.com>
+---
+ src/libcamera/pipeline/vivid/vivid.cpp | 21 ++++++++++++++++++++-
+ 1 file changed, 20 insertions(+), 1 deletion(-)
+
+diff --git a/src/libcamera/pipeline/vivid/vivid.cpp b/src/libcamera/pipeline/vivid/vivid.cpp
+index 9e95bae8bc30..dbc19424e75a 100644
+--- a/src/libcamera/pipeline/vivid/vivid.cpp
++++ b/src/libcamera/pipeline/vivid/vivid.cpp
+@@ -150,7 +150,26 @@ CameraConfiguration *PipelineHandlerVivid::generateConfiguration(Camera *camera,
+
+ int PipelineHandlerVivid::configure(Camera *camera, CameraConfiguration *config)
+ {
+- return -1;
++ VividCameraData *data = cameraData(camera);
++ StreamConfiguration &cfg = config->at(0);
++ int ret;
++
++ V4L2DeviceFormat format = {};
++ format.fourcc = data->video_->toV4L2PixelFormat(cfg.pixelFormat);
++ format.size = cfg.size;
++
++ ret = data->video_->setFormat(&format);
++ if (ret)
++ return ret;
++
++ if (format.size != cfg.size ||
++ format.fourcc != data->video_->toV4L2PixelFormat(cfg.pixelFormat))
++ return -EINVAL;
++
++ cfg.setStream(&data->stream_);
++ cfg.stride = format.planes[0].bpl;
++
++ return 0;
+ }
+
+ int PipelineHandlerVivid::exportFrameBuffers(Camera *camera, Stream *stream,
+--
+2.25.1
+
diff --git a/Documentation/vivid-pipeline-handler/0006-libcamera-pipeline-vivid-Buffer-handling-and-stream-.patch b/Documentation/vivid-pipeline-handler/0006-libcamera-pipeline-vivid-Buffer-handling-and-stream-.patch
new file mode 100644
index 000000000000..b04ef3e10473
--- /dev/null
+++ b/Documentation/vivid-pipeline-handler/0006-libcamera-pipeline-vivid-Buffer-handling-and-stream-.patch
@@ -0,0 +1,102 @@
+From 8b6d4200c60513bdb9c435c94d512efb4f59441c Mon Sep 17 00:00:00 2001
+From: Kieran Bingham <kieran.bingham at ideasonboard.com>
+Date: Mon, 13 Jul 2020 10:46:26 +0100
+Subject: [PATCH 6/9] libcamera: pipeline: vivid: Buffer handling and stream
+ control
+
+We can now add buffer management, and connect up our bufferReady signal
+to a callback.
+
+Note that we provide the ability to export buffers from our capture
+device (data->video_) using the exportBuffers() functionality from the
+V4L2VideoDevice which allows a FrameBufferAllocater to obtain buffers
+from this device.
+
+When buffers are obtained through the exportFrameBuffers API, they are
+orphaned and left unassociated with the device, and must be reimported
+at start() time anyway. This allows the same interface to be used
+whether internal buffers, or external buffers are used for the stream.
+
+When a buffer completes, we call the buffer completion handler on the
+pipeline handler, and because we have only a single stream, we can also
+immediately complete the request.
+
+Signed-off-by: Kieran Bingham <kieran.bingham at ideasonboard.com>
+---
+ src/libcamera/pipeline/vivid/vivid.cpp | 35 ++++++++++++++++++++++++--
+ 1 file changed, 33 insertions(+), 2 deletions(-)
+
+diff --git a/src/libcamera/pipeline/vivid/vivid.cpp b/src/libcamera/pipeline/vivid/vivid.cpp
+index dbc19424e75a..1a945a744055 100644
+--- a/src/libcamera/pipeline/vivid/vivid.cpp
++++ b/src/libcamera/pipeline/vivid/vivid.cpp
+@@ -32,6 +32,7 @@ public:
+ }
+
+ int init();
++ void bufferReady(FrameBuffer *buffer);
+
+ MediaDevice *media_;
+ V4L2VideoDevice *video_;
+@@ -175,16 +176,36 @@ int PipelineHandlerVivid::configure(Camera *camera, CameraConfiguration *config)
+ int PipelineHandlerVivid::exportFrameBuffers(Camera *camera, Stream *stream,
+ std::vector<std::unique_ptr<FrameBuffer>> *buffers)
+ {
+- return -1;
++ VividCameraData *data = cameraData(camera);
++ unsigned int count = stream->configuration().bufferCount;
++
++ return data->video_->exportBuffers(count, buffers);
+ }
+
+ int PipelineHandlerVivid::start(Camera *camera)
+ {
+- return -1;
++ VividCameraData *data = cameraData(camera);
++ unsigned int count = data->stream_.configuration().bufferCount;
++
++ int ret = data->video_->importBuffers(count);
++ if (ret < 0)
++ return ret;
++
++ ret = data->video_->streamOn();
++ if (ret < 0) {
++ data->ipa_->stop();
++ data->video_->releaseBuffers();
++ return ret;
++ }
++
++ return 0;
+ }
+
+ void PipelineHandlerVivid::stop(Camera *camera)
+ {
++ VividCameraData *data = cameraData(camera);
++ data->video_->streamOff();
++ data->video_->releaseBuffers();
+ }
+
+ int PipelineHandlerVivid::queueRequestDevice(Camera *camera, Request *request)
+@@ -221,9 +242,19 @@ int VividCameraData::init()
+ if (video_->open())
+ return -ENODEV;
+
++ video_->bufferReady.connect(this, &VividCameraData::bufferReady);
++
+ return 0;
+ }
+
++void VividCameraData::bufferReady(FrameBuffer *buffer)
++{
++ Request *request = buffer->request();
++
++ pipe_->completeBuffer(camera_, request, buffer);
++ pipe_->completeRequest(camera_, request);
++}
++
+ REGISTER_PIPELINE_HANDLER(PipelineHandlerVivid);
+
+ } /* namespace libcamera */
+--
+2.25.1
+
diff --git a/Documentation/vivid-pipeline-handler/0007-libcamera-pipeline-vivid-Queue-requests.patch b/Documentation/vivid-pipeline-handler/0007-libcamera-pipeline-vivid-Queue-requests.patch
new file mode 100644
index 000000000000..a896e6d12e37
--- /dev/null
+++ b/Documentation/vivid-pipeline-handler/0007-libcamera-pipeline-vivid-Queue-requests.patch
@@ -0,0 +1,45 @@
+From 22e1f2c2736cd860a63086695f1dded0b2b2aa0e Mon Sep 17 00:00:00 2001
+From: Kieran Bingham <kieran.bingham at ideasonboard.com>
+Date: Mon, 13 Jul 2020 11:35:11 +0100
+Subject: [PATCH 7/9] libcamera: pipeline: vivid: Queue requests
+
+When a request is given to a pipeline handler, it must parse the request
+and identify what actions the pipeline handler should take to enact on hardware.
+
+In the case of the VIVID pipeline handler, we identify the buffer from the only
+supported stream, and queue it to the video capture device.
+
+Signed-off-by: Kieran Bingham <kieran.bingham at ideasonboard.com>
+---
+ src/libcamera/pipeline/vivid/vivid.cpp | 15 ++++++++++++++-
+ 1 file changed, 14 insertions(+), 1 deletion(-)
+
+diff --git a/src/libcamera/pipeline/vivid/vivid.cpp b/src/libcamera/pipeline/vivid/vivid.cpp
+index 1a945a744055..4362e73f49a5 100644
+--- a/src/libcamera/pipeline/vivid/vivid.cpp
++++ b/src/libcamera/pipeline/vivid/vivid.cpp
+@@ -210,7 +210,20 @@ void PipelineHandlerVivid::stop(Camera *camera)
+
+ int PipelineHandlerVivid::queueRequestDevice(Camera *camera, Request *request)
+ {
+- return -1;
++ VividCameraData *data = cameraData(camera);
++ FrameBuffer *buffer = request->findBuffer(&data->stream_);
++ if (!buffer) {
++ LOG(VIVID, Error)
++ << "Attempt to queue request with invalid stream";
++
++ return -ENOENT;
++ }
++
++ int ret = data->video_->queueBuffer(buffer);
++ if (ret < 0)
++ return ret;
++
++ return 0;
+ }
+
+ bool PipelineHandlerVivid::match(DeviceEnumerator *enumerator)
+--
+2.25.1
+
diff --git a/Documentation/vivid-pipeline-handler/0008-libcamera-pipeline-vivid-Initialise-key-controls.patch b/Documentation/vivid-pipeline-handler/0008-libcamera-pipeline-vivid-Initialise-key-controls.patch
new file mode 100644
index 000000000000..8e0fca45fa04
--- /dev/null
+++ b/Documentation/vivid-pipeline-handler/0008-libcamera-pipeline-vivid-Initialise-key-controls.patch
@@ -0,0 +1,70 @@
+From 703b1d24259934781c9f9ff80774e5845876c47f Mon Sep 17 00:00:00 2001
+From: Kieran Bingham <kieran.bingham at ideasonboard.com>
+Date: Mon, 13 Jul 2020 12:48:37 +0100
+Subject: [PATCH 8/9] libcamera: pipeline: vivid: Initialise key controls
+
+The VIVID pipeline handler retains state globally of it's controls.
+Ensure that when we configure this specific pipeline we set initial
+parameters on the device that suit our (specific) needs.
+
+This introduces how controls can be set directly on a device, however
+under normal circumstances controls should usually be set from libcamera
+controls as part of a request. These are VIVID specific only.
+
+Signed-off-by: Kieran Bingham <kieran.bingham at ideasonboard.com>
+---
+ src/libcamera/pipeline/vivid/vivid.cpp | 31 ++++++++++++++++++++++++++
+ 1 file changed, 31 insertions(+)
+
+diff --git a/src/libcamera/pipeline/vivid/vivid.cpp b/src/libcamera/pipeline/vivid/vivid.cpp
+index 4362e73f49a5..1744d78f2f28 100644
+--- a/src/libcamera/pipeline/vivid/vivid.cpp
++++ b/src/libcamera/pipeline/vivid/vivid.cpp
+@@ -14,6 +14,18 @@
+ #include "libcamera/internal/pipeline_handler.h"
+ #include "libcamera/internal/v4l2_videodevice.h"
+
++#define VIVID_CID_VIVID_BASE (0x00f00000 | 0xf000)
++#define VIVID_CID_VIVID_CLASS (0x00f00000 | 1)
++#define VIVID_CID_TEST_PATTERN (VIVID_CID_VIVID_BASE + 0)
++#define VIVID_CID_OSD_TEXT_MODE (VIVID_CID_VIVID_BASE + 1)
++#define VIVID_CID_HOR_MOVEMENT (VIVID_CID_VIVID_BASE + 2)
++#define VIVID_CID_VERT_MOVEMENT (VIVID_CID_VIVID_BASE + 3)
++#define VIVID_CID_SHOW_BORDER (VIVID_CID_VIVID_BASE + 4)
++#define VIVID_CID_SHOW_SQUARE (VIVID_CID_VIVID_BASE + 5)
++#define VIVID_CID_INSERT_SAV (VIVID_CID_VIVID_BASE + 6)
++#define VIVID_CID_INSERT_EAV (VIVID_CID_VIVID_BASE + 7)
++#define VIVID_CID_VBI_CAP_INTERLACED (VIVID_CID_VIVID_BASE + 8)
++
+ namespace libcamera {
+
+ LOG_DEFINE_CATEGORY(VIVID)
+@@ -167,6 +179,25 @@ int PipelineHandlerVivid::configure(Camera *camera, CameraConfiguration *config)
+ format.fourcc != data->video_->toV4L2PixelFormat(cfg.pixelFormat))
+ return -EINVAL;
+
++ /* Set initial controls specific to VIVID */
++ ControlList controls(data->video_->controls());
++ controls.set(VIVID_CID_TEST_PATTERN, 0); /* Vertical Colour Bars */
++ controls.set(VIVID_CID_OSD_TEXT_MODE, 0); /* Display all OSD */
++
++ /* Ensure clear colours configured. */
++ controls.set(V4L2_CID_BRIGHTNESS, 128);
++ controls.set(V4L2_CID_CONTRAST, 128);
++ controls.set(V4L2_CID_SATURATION, 128);
++
++ /* Enable movement to visualise buffer updates. */
++ controls.set(VIVID_CID_HOR_MOVEMENT, 5);
++
++ ret = data->video_->setControls(&controls);
++ if (ret) {
++ LOG(VIVID, Error) << "Failed to set controls: " << ret;
++ return ret < 0 ? ret : -EINVAL;
++ }
++
+ cfg.setStream(&data->stream_);
+ cfg.stride = format.planes[0].bpl;
+
+--
+2.25.1
+
diff --git a/Documentation/vivid-pipeline-handler/0009-libcamera-pipeline-vivid-Handle-controls.patch b/Documentation/vivid-pipeline-handler/0009-libcamera-pipeline-vivid-Handle-controls.patch
new file mode 100644
index 000000000000..ea88ef44c102
--- /dev/null
+++ b/Documentation/vivid-pipeline-handler/0009-libcamera-pipeline-vivid-Handle-controls.patch
@@ -0,0 +1,133 @@
+From 03bbac3265554b73e6ddcdebaf95090062e3669f Mon Sep 17 00:00:00 2001
+From: Kieran Bingham <kieran.bingham at ideasonboard.com>
+Date: Mon, 13 Jul 2020 12:56:11 +0100
+Subject: [PATCH 9/9] libcamera: pipeline: vivid: Handle controls
+
+When constructing the camera, we parse the available controls on the
+video capture device, and map supported controls to libcamera controls,
+and initialise the defaults.
+
+The controls are handled during queueRequestDevice for each request and
+applied to the device through the capture node.
+
+Signed-off-by: Kieran Bingham <kieran.bingham at ideasonboard.com>
+---
+ src/libcamera/pipeline/vivid/vivid.cpp | 80 +++++++++++++++++++++++++-
+ 1 file changed, 79 insertions(+), 1 deletion(-)
+
+diff --git a/src/libcamera/pipeline/vivid/vivid.cpp b/src/libcamera/pipeline/vivid/vivid.cpp
+index 1744d78f2f28..b8b8e3ae0287 100644
+--- a/src/libcamera/pipeline/vivid/vivid.cpp
++++ b/src/libcamera/pipeline/vivid/vivid.cpp
+@@ -5,7 +5,11 @@
+ * vivid.cpp - Pipeline handler for the vivid capture device
+ */
+
++#include <math.h>
++
+ #include <libcamera/camera.h>
++#include <libcamera/control_ids.h>
++#include <libcamera/controls.h>
+ #include <libcamera/formats.h>
+
+ #include "libcamera/internal/device_enumerator.h"
+@@ -239,6 +243,46 @@ void PipelineHandlerVivid::stop(Camera *camera)
+ data->video_->releaseBuffers();
+ }
+
++int PipelineHandlerVivid::processControls(VividCameraData *data, Request *request)
++{
++ ControlList controls(data->video_->controls());
++
++ for (auto it : request->controls()) {
++ unsigned int id = it.first;
++ unsigned int offset;
++ uint32_t cid;
++
++ if (id == controls::Brightness) {
++ cid = V4L2_CID_BRIGHTNESS;
++ offset = 128;
++ } else if (id == controls::Contrast) {
++ cid = V4L2_CID_CONTRAST;
++ offset = 0;
++ } else if (id == controls::Saturation) {
++ cid = V4L2_CID_SATURATION;
++ offset = 0;
++ } else {
++ continue;
++ }
++
++ int32_t value = lroundf(it.second.get<float>() * 128 + offset);
++ controls.set(cid, utils::clamp(value, 0, 255));
++ }
++
++ for (const auto &ctrl : controls)
++ LOG(VIVID, Debug)
++ << "Setting control " << utils::hex(ctrl.first)
++ << " to " << ctrl.second.toString();
++
++ int ret = data->video_->setControls(&controls);
++ if (ret) {
++ LOG(VIVID, Error) << "Failed to set controls: " << ret;
++ return ret < 0 ? ret : -EINVAL;
++ }
++
++ return ret;
++}
++
+ int PipelineHandlerVivid::queueRequestDevice(Camera *camera, Request *request)
+ {
+ VividCameraData *data = cameraData(camera);
+@@ -250,7 +294,11 @@ int PipelineHandlerVivid::queueRequestDevice(Camera *camera, Request *request)
+ return -ENOENT;
+ }
+
+- int ret = data->video_->queueBuffer(buffer);
++ int ret = processControls(data, request);
++ if (ret < 0)
++ return ret;
++
++ ret = data->video_->queueBuffer(buffer);
+ if (ret < 0)
+ return ret;
+
+@@ -288,6 +336,36 @@ int VividCameraData::init()
+
+ video_->bufferReady.connect(this, &VividCameraData::bufferReady);
+
++ /* Initialise the supported controls. */
++ const ControlInfoMap &controls = video_->controls();
++ ControlInfoMap::Map ctrls;
++
++ for (const auto &ctrl : controls) {
++ const ControlId *id;
++ ControlInfo info;
++
++ switch (ctrl.first->id()) {
++ case V4L2_CID_BRIGHTNESS:
++ id = &controls::Brightness;
++ info = ControlInfo{ { -1.0f }, { 1.0f }, { 0.0f } };
++ break;
++ case V4L2_CID_CONTRAST:
++ id = &controls::Contrast;
++ info = ControlInfo{ { 0.0f }, { 2.0f }, { 1.0f } };
++ break;
++ case V4L2_CID_SATURATION:
++ id = &controls::Saturation;
++ info = ControlInfo{ { 0.0f }, { 2.0f }, { 1.0f } };
++ break;
++ default:
++ continue;
++ }
++
++ ctrls.emplace(id, info);
++ }
++
++ controlInfo_ = std::move(ctrls);
++
+ return 0;
+ }
+
+--
+2.25.1
+
--
2.25.1
More information about the libcamera-devel
mailing list