[libcamera-devel] [PATCH] Documentation: Vivid Pipeline
Laurent Pinchart
laurent.pinchart at ideasonboard.com
Mon Aug 10 15:33:06 CEST 2020
Hi Kieran,
Thank you for the patch.
On Mon, Aug 10, 2020 at 02:01:10PM +0100, Kieran Bingham wrote:
> Provide the patches for the Vivid pipeline within the Documentation tree.
> These patches are not destined for direct integration, but serve as a reference
> for implementing a new pipeline handler.
>
> Keep the patches within the tree to allow tracking updates, and making them
> accessible to developers.
>
> Signed-off-by: Kieran Bingham <kieran.bingham at ideasonboard.com>
I'll have to think more about this, but needless to say, I'm not
entirely thrilled :-) Patch review will be horrible. I think I'd rather
keep a separate repository that we can point to from the documentation.
It can live at an official URL, on libcamera.org for instance.
> ---
> .../0000-cover-letter.patch | 56 ++++++
> ...ne-Introduce-skeleton-Vivid-Pipeline.patch | 139 +++++++++++++++
> ...ibcamera-pipeline-vivid-Match-device.patch | 54 ++++++
> ...amera-pipeline-vivid-Create-a-Camera.patch | 110 ++++++++++++
> ...ne-vivid-Generate-and-validate-Strea.patch | 160 ++++++++++++++++++
> ...-pipeline-vivid-Configure-the-device.patch | 71 ++++++++
> ...ne-vivid-Buffer-handling-and-stream-.patch | 102 +++++++++++
> ...camera-pipeline-vivid-Queue-requests.patch | 45 +++++
> ...peline-vivid-Initialise-key-controls.patch | 70 ++++++++
> ...amera-pipeline-vivid-Handle-controls.patch | 133 +++++++++++++++
> 10 files changed, 940 insertions(+)
> create mode 100644 Documentation/vivid-pipeline-handler/0000-cover-letter.patch
> create mode 100644 Documentation/vivid-pipeline-handler/0001-libcamera-pipeline-Introduce-skeleton-Vivid-Pipeline.patch
> create mode 100644 Documentation/vivid-pipeline-handler/0002-libcamera-pipeline-vivid-Match-device.patch
> create mode 100644 Documentation/vivid-pipeline-handler/0003-libcamera-pipeline-vivid-Create-a-Camera.patch
> create mode 100644 Documentation/vivid-pipeline-handler/0004-libcamera-pipeline-vivid-Generate-and-validate-Strea.patch
> create mode 100644 Documentation/vivid-pipeline-handler/0005-libcamera-pipeline-vivid-Configure-the-device.patch
> create mode 100644 Documentation/vivid-pipeline-handler/0006-libcamera-pipeline-vivid-Buffer-handling-and-stream-.patch
> create mode 100644 Documentation/vivid-pipeline-handler/0007-libcamera-pipeline-vivid-Queue-requests.patch
> create mode 100644 Documentation/vivid-pipeline-handler/0008-libcamera-pipeline-vivid-Initialise-key-controls.patch
> create mode 100644 Documentation/vivid-pipeline-handler/0009-libcamera-pipeline-vivid-Handle-controls.patch
>
> diff --git a/Documentation/vivid-pipeline-handler/0000-cover-letter.patch b/Documentation/vivid-pipeline-handler/0000-cover-letter.patch
> new file mode 100644
> index 000000000000..25e61d4313ff
> --- /dev/null
> +++ b/Documentation/vivid-pipeline-handler/0000-cover-letter.patch
> @@ -0,0 +1,56 @@
> +From 03bbac3265554b73e6ddcdebaf95090062e3669f Mon Sep 17 00:00:00 2001
> +From: Kieran Bingham <kieran.bingham at ideasonboard.com>
> +Date: Mon, 10 Aug 2020 13:28:23 +0100
> +Subject: [PATCH 0/9] libcamera: pipeline: Add a VIVID Pipeline Handler
> +
> +Introduce a new pipeline handler to support the virtual video test
> +driver provided by V4L2.
> +
> +These patches are intended to serve as a code-guide for writing a new
> +pipeline handler, and can be referenced along side the Pipeline Handler
> +Writers Guide documentation.
> +
> +To test these patches, apply locally to your libcamera sources by
> +executing the following:
> +
> + git am Documentation/vivid-pipeline-handler/*
> +
> +Then test by ensuring you have the vivid module loaded:
> +
> + sudo modprobe vivid
> +
> +To enable the new pipeline handler the build must be re-configured. A
> +limitation in the meson build system means that this requires a full
> +reconfiguration. The easiest solution of which is to remove any existing
> +build and commence with a fresh build tree.
> +
> +Configure, build and test:
> +
> + rm -r build
> + meson build
> + cd build
> + ninja
> +
> +
> +
> +Kieran Bingham (9):
> + libcamera: pipeline: Introduce skeleton Vivid Pipeline
> + libcamera: pipeline: vivid: Match device
> + libcamera: pipeline: vivid: Create a Camera
> + libcamera: pipeline: vivid: Generate and validate StreamConfigurations
> + libcamera: pipeline: vivid: Configure the device
> + libcamera: pipeline: vivid: Buffer handling and stream control
> + libcamera: pipeline: vivid: Queue requests
> + libcamera: pipeline: vivid: Initialise key controls
> + libcamera: pipeline: vivid: Handle controls
> +
> + meson_options.txt | 2 +-
> + src/libcamera/pipeline/vivid/meson.build | 5 +
> + src/libcamera/pipeline/vivid/vivid.cpp | 382 +++++++++++++++++++++++
> + 3 files changed, 388 insertions(+), 1 deletion(-)
> + create mode 100644 src/libcamera/pipeline/vivid/meson.build
> + create mode 100644 src/libcamera/pipeline/vivid/vivid.cpp
> +
> +--
> +2.25.1
> +
> diff --git a/Documentation/vivid-pipeline-handler/0001-libcamera-pipeline-Introduce-skeleton-Vivid-Pipeline.patch b/Documentation/vivid-pipeline-handler/0001-libcamera-pipeline-Introduce-skeleton-Vivid-Pipeline.patch
> new file mode 100644
> index 000000000000..ca1dc48e4fb9
> --- /dev/null
> +++ b/Documentation/vivid-pipeline-handler/0001-libcamera-pipeline-Introduce-skeleton-Vivid-Pipeline.patch
> @@ -0,0 +1,139 @@
> +From 2480348e61bfcbc8a113a03e6090308f9b904acb Mon Sep 17 00:00:00 2001
> +From: Kieran Bingham <kieran.bingham at ideasonboard.com>
> +Date: Fri, 10 Jul 2020 15:13:05 +0100
> +Subject: [PATCH 1/9] libcamera: pipeline: Introduce skeleton Vivid Pipeline
> +
> +Provide all of the skeleton stubs to succesfully compile
> +and register a new Pipeline Handler for the Vivid test device.
> +
> +Meson must be reconfigured to ensure that this pipeline handler is
> +included in the selected pipelines configuration, and after building, we
> +can test that the PipelineHandler is successfully registered by listing
> +the cameras on the system with LIBCAMERA_LOG_LEVELS enabled:
> +
> +"""
> +LIBCAMERA_LOG_LEVELS=Pipeline:0 ./build-vivid/src/cam/cam -l
> +[230:30:03.624102821] [2867886] DEBUG Pipeline pipeline_handler.cpp:680 Registered pipeline handler "PipelineHandlerVivid"
> +Available cameras:
> +"""
> +
> +Signed-off-by: Kieran Bingham <kieran.bingham at ideasonboard.com>
> +---
> + meson_options.txt | 2 +-
> + src/libcamera/pipeline/vivid/meson.build | 5 ++
> + src/libcamera/pipeline/vivid/vivid.cpp | 78 ++++++++++++++++++++++++
> + 3 files changed, 84 insertions(+), 1 deletion(-)
> + create mode 100644 src/libcamera/pipeline/vivid/meson.build
> + create mode 100644 src/libcamera/pipeline/vivid/vivid.cpp
> +
> +diff --git a/meson_options.txt b/meson_options.txt
> +index e9e815fde366..ea38f1553240 100644
> +--- a/meson_options.txt
> ++++ b/meson_options.txt
> +@@ -16,7 +16,7 @@ option('gstreamer',
> +
> + option('pipelines',
> + type : 'array',
> +- choices : ['ipu3', 'raspberrypi', 'rkisp1', 'simple', 'uvcvideo', 'vimc'],
> ++ choices : ['ipu3', 'raspberrypi', 'rkisp1', 'simple', 'uvcvideo', 'vimc', 'vivid'],
> + description : 'Select which pipeline handlers to include')
> +
> + option('qcam',
> +diff --git a/src/libcamera/pipeline/vivid/meson.build b/src/libcamera/pipeline/vivid/meson.build
> +new file mode 100644
> +index 000000000000..086bb825387c
> +--- /dev/null
> ++++ b/src/libcamera/pipeline/vivid/meson.build
> +@@ -0,0 +1,5 @@
> ++# SPDX-License-Identifier: CC0-1.0
> ++
> ++libcamera_sources += files([
> ++ 'vivid.cpp',
> ++])
> +diff --git a/src/libcamera/pipeline/vivid/vivid.cpp b/src/libcamera/pipeline/vivid/vivid.cpp
> +new file mode 100644
> +index 000000000000..4418f616fe84
> +--- /dev/null
> ++++ b/src/libcamera/pipeline/vivid/vivid.cpp
> +@@ -0,0 +1,78 @@
> ++/* SPDX-License-Identifier: LGPL-2.1-or-later */
> ++/*
> ++ * Copyright (C) 2020, Google Inc.
> ++ *
> ++ * vivid.cpp - Pipeline handler for the vivid capture device
> ++ */
> ++
> ++#include "libcamera/internal/log.h"
> ++#include "libcamera/internal/pipeline_handler.h"
> ++
> ++namespace libcamera {
> ++
> ++LOG_DEFINE_CATEGORY(VIVID)
> ++
> ++class PipelineHandlerVivid : public PipelineHandler
> ++{
> ++public:
> ++ PipelineHandlerVivid(CameraManager *manager);
> ++
> ++ CameraConfiguration *generateConfiguration(Camera *camera,
> ++ const StreamRoles &roles) override;
> ++ int configure(Camera *camera, CameraConfiguration *config) override;
> ++
> ++ int exportFrameBuffers(Camera *camera, Stream *stream,
> ++ std::vector<std::unique_ptr<FrameBuffer>> *buffers) override;
> ++
> ++ int start(Camera *camera) override;
> ++ void stop(Camera *camera) override;
> ++
> ++ int queueRequestDevice(Camera *camera, Request *request) override;
> ++
> ++ bool match(DeviceEnumerator *enumerator) override;
> ++};
> ++
> ++PipelineHandlerVivid::PipelineHandlerVivid(CameraManager *manager)
> ++ : PipelineHandler(manager)
> ++{
> ++}
> ++
> ++CameraConfiguration *PipelineHandlerVivid::generateConfiguration(Camera *camera,
> ++ const StreamRoles &roles)
> ++{
> ++ return nullptr;
> ++}
> ++
> ++int PipelineHandlerVivid::configure(Camera *camera, CameraConfiguration *config)
> ++{
> ++ return -1;
> ++}
> ++
> ++int PipelineHandlerVivid::exportFrameBuffers(Camera *camera, Stream *stream,
> ++ std::vector<std::unique_ptr<FrameBuffer>> *buffers)
> ++{
> ++ return -1;
> ++}
> ++
> ++int PipelineHandlerVivid::start(Camera *camera)
> ++{
> ++ return -1;
> ++}
> ++
> ++void PipelineHandlerVivid::stop(Camera *camera)
> ++{
> ++}
> ++
> ++int PipelineHandlerVivid::queueRequestDevice(Camera *camera, Request *request)
> ++{
> ++ return -1;
> ++}
> ++
> ++bool PipelineHandlerVivid::match(DeviceEnumerator *enumerator)
> ++{
> ++ return false;
> ++}
> ++
> ++REGISTER_PIPELINE_HANDLER(PipelineHandlerVivid);
> ++
> ++} /* namespace libcamera */
> +--
> +2.25.1
> +
> diff --git a/Documentation/vivid-pipeline-handler/0002-libcamera-pipeline-vivid-Match-device.patch b/Documentation/vivid-pipeline-handler/0002-libcamera-pipeline-vivid-Match-device.patch
> new file mode 100644
> index 000000000000..a48eeeac1fe9
> --- /dev/null
> +++ b/Documentation/vivid-pipeline-handler/0002-libcamera-pipeline-vivid-Match-device.patch
> @@ -0,0 +1,54 @@
> +From d676cff10779a5a8f286cf0553f52c5f747436f0 Mon Sep 17 00:00:00 2001
> +From: Kieran Bingham <kieran.bingham at ideasonboard.com>
> +Date: Fri, 10 Jul 2020 15:37:55 +0100
> +Subject: [PATCH 2/9] libcamera: pipeline: vivid: Match device
> +
> +Verify that we can match on our expected device(s).
> +
> +Use a temporary debug print to check that the pipeline finds
> +our device:
> +
> +"""
> +LIBCAMERA_LOG_LEVELS=Pipeline,VIVID:0 ./src/cam/cam -l
> +<snipped>
> +[230:51:10.670503423] [2872877] DEBUG VIVID vivid.cpp:81 Obtained Vivid Device
> +"""
> +
> +Signed-off-by: Kieran Bingham <kieran.bingham at ideasonboard.com>
> +---
> + src/libcamera/pipeline/vivid/vivid.cpp | 12 +++++++++++-
> + 1 file changed, 11 insertions(+), 1 deletion(-)
> +
> +diff --git a/src/libcamera/pipeline/vivid/vivid.cpp b/src/libcamera/pipeline/vivid/vivid.cpp
> +index 4418f616fe84..9811f6ef5095 100644
> +--- a/src/libcamera/pipeline/vivid/vivid.cpp
> ++++ b/src/libcamera/pipeline/vivid/vivid.cpp
> +@@ -5,6 +5,7 @@
> + * vivid.cpp - Pipeline handler for the vivid capture device
> + */
> +
> ++#include "libcamera/internal/device_enumerator.h"
> + #include "libcamera/internal/log.h"
> + #include "libcamera/internal/pipeline_handler.h"
> +
> +@@ -70,7 +71,16 @@ int PipelineHandlerVivid::queueRequestDevice(Camera *camera, Request *request)
> +
> + bool PipelineHandlerVivid::match(DeviceEnumerator *enumerator)
> + {
> +- return false;
> ++ DeviceMatch dm("vivid");
> ++ dm.add("vivid-000-vid-cap");
> ++
> ++ MediaDevice *media = acquireMediaDevice(enumerator, dm);
> ++ if (!media)
> ++ return false;
> ++
> ++ LOG(VIVID, Debug) << "Obtained Vivid Device";
> ++
> ++ return false; // Prevent infinite loops for now
> + }
> +
> + REGISTER_PIPELINE_HANDLER(PipelineHandlerVivid);
> +--
> +2.25.1
> +
> diff --git a/Documentation/vivid-pipeline-handler/0003-libcamera-pipeline-vivid-Create-a-Camera.patch b/Documentation/vivid-pipeline-handler/0003-libcamera-pipeline-vivid-Create-a-Camera.patch
> new file mode 100644
> index 000000000000..1d8552752047
> --- /dev/null
> +++ b/Documentation/vivid-pipeline-handler/0003-libcamera-pipeline-vivid-Create-a-Camera.patch
> @@ -0,0 +1,110 @@
> +From f80f723a010501c6d159ae9601c2787d5d02906f Mon Sep 17 00:00:00 2001
> +From: Kieran Bingham <kieran.bingham at ideasonboard.com>
> +Date: Fri, 10 Jul 2020 16:28:36 +0100
> +Subject: [PATCH 3/9] libcamera: pipeline: vivid: Create a Camera
> +
> +Create a VividCameraData inheriting from the CameraData to handle camera
> +specific data, and use it to create and register the camera with the
> +CameraManager.
> +
> +This can now be tested to see that the camera becomes available to
> +applications:
> +
> +"""
> +LIBCAMERA_LOG_LEVELS=Pipeline,VIVID:0 ./src/cam/cam -l
> +[231:44:49.325333712] [2880028] INFO IPAManager ipa_manager.cpp:136 libcamera is not installed. Adding '/home/libcamera/build-vivid/src/ipa' to the IPA search path
> +[231:44:49.325428449] [2880028] WARN IPAManager ipa_manager.cpp:147 No IPA found in '/usr/local/lib/x86_64-linux-gnu/libcamera'
> +[231:44:49.325446253] [2880028] INFO Camera camera_manager.cpp:283 libcamera v0.0.11+713-d175334d-dirty
> +Available cameras:
> +1: vivid
> +"""
> +
> +Signed-off-by: Kieran Bingham <kieran.bingham at ideasonboard.com>
> +---
> + src/libcamera/pipeline/vivid/vivid.cpp | 54 +++++++++++++++++++++++++-
> + 1 file changed, 52 insertions(+), 2 deletions(-)
> +
> +diff --git a/src/libcamera/pipeline/vivid/vivid.cpp b/src/libcamera/pipeline/vivid/vivid.cpp
> +index 9811f6ef5095..a8922ce70ed4 100644
> +--- a/src/libcamera/pipeline/vivid/vivid.cpp
> ++++ b/src/libcamera/pipeline/vivid/vivid.cpp
> +@@ -5,14 +5,46 @@
> + * vivid.cpp - Pipeline handler for the vivid capture device
> + */
> +
> ++#include <libcamera/camera.h>
> ++
> + #include "libcamera/internal/device_enumerator.h"
> + #include "libcamera/internal/log.h"
> ++#include "libcamera/internal/media_device.h"
> + #include "libcamera/internal/pipeline_handler.h"
> ++#include "libcamera/internal/v4l2_videodevice.h"
> +
> + namespace libcamera {
> +
> + LOG_DEFINE_CATEGORY(VIVID)
> +
> ++class VividCameraData : public CameraData
> ++{
> ++public:
> ++ VividCameraData(PipelineHandler *pipe, MediaDevice *media)
> ++ : CameraData(pipe), media_(media), video_(nullptr)
> ++ {
> ++ }
> ++
> ++ ~VividCameraData()
> ++ {
> ++ delete video_;
> ++ }
> ++
> ++ int init();
> ++
> ++ MediaDevice *media_;
> ++ V4L2VideoDevice *video_;
> ++ Stream stream_;
> ++};
> ++
> ++class VividCameraConfiguration : public CameraConfiguration
> ++{
> ++public:
> ++ VividCameraConfiguration();
> ++
> ++ Status validate() override;
> ++};
> ++
> + class PipelineHandlerVivid : public PipelineHandler
> + {
> + public:
> +@@ -78,9 +110,27 @@ bool PipelineHandlerVivid::match(DeviceEnumerator *enumerator)
> + if (!media)
> + return false;
> +
> +- LOG(VIVID, Debug) << "Obtained Vivid Device";
> ++ std::unique_ptr<VividCameraData> data = std::make_unique<VividCameraData>(this, media);
> ++
> ++ /* Locate and open the capture video node. */
> ++ if (data->init())
> ++ return false;
> ++
> ++ /* Create and register the camera. */
> ++ std::set<Stream *> streams{ &data->stream_ };
> ++ std::shared_ptr<Camera> camera = Camera::create(this, data->video_->deviceName(), streams);
> ++ registerCamera(std::move(camera), std::move(data));
> ++
> ++ return true;
> ++}
> ++
> ++int VividCameraData::init()
> ++{
> ++ video_ = new V4L2VideoDevice(media_->getEntityByName("vivid-000-vid-cap"));
> ++ if (video_->open())
> ++ return -ENODEV;
> +
> +- return false; // Prevent infinite loops for now
> ++ return 0;
> + }
> +
> + REGISTER_PIPELINE_HANDLER(PipelineHandlerVivid);
> +--
> +2.25.1
> +
> diff --git a/Documentation/vivid-pipeline-handler/0004-libcamera-pipeline-vivid-Generate-and-validate-Strea.patch b/Documentation/vivid-pipeline-handler/0004-libcamera-pipeline-vivid-Generate-and-validate-Strea.patch
> new file mode 100644
> index 000000000000..43a203cffa7b
> --- /dev/null
> +++ b/Documentation/vivid-pipeline-handler/0004-libcamera-pipeline-vivid-Generate-and-validate-Strea.patch
> @@ -0,0 +1,160 @@
> +From 928ab8ae5728677c21bd31309b782d0856441aa2 Mon Sep 17 00:00:00 2001
> +From: Kieran Bingham <kieran.bingham at ideasonboard.com>
> +Date: Fri, 10 Jul 2020 16:44:01 +0100
> +Subject: [PATCH 4/9] libcamera: pipeline: vivid: Generate and validate
> + StreamConfigurations
> +
> +Implement the support for Generating and Validating the streams the
> +Camera can provide.
> +
> +Vivid is a simple case with only a single stream.
> +
> +Test the configurations can be generated and reported with cam -I:
> +
> +"""
> +LIBCAMERA_LOG_LEVELS=Pipeline,VIVID:0 ./src/cam/cam -c 1 -I
> +[232:02:09.633067174] [2882911] INFO IPAManager ipa_manager.cpp:136 libcamera is not installed. Adding '/home//libcamera/build-vivid/src/ipa' to the IPA search path
> +[232:02:09.633332451] [2882911] WARN IPAManager ipa_manager.cpp:147 No IPA found in '/usr/local/lib/x86_64-linux-gnu/libcamera'
> +[232:02:09.633373414] [2882911] INFO Camera camera_manager.cpp:283 libcamera v0.0.11+714-d1ebd889-dirty
> +Using camera vivid
> +0: 1280x720-BGR888
> + * Pixelformat: NV21 (320x180)-(3840x2160)/(+0,+0)
> + - 320x180
> + - 640x360
> + - 640x480
> + - 1280x720
> + - 1920x1080
> + - 3840x2160
> + * Pixelformat: NV12 (320x180)-(3840x2160)/(+0,+0)
> + - 320x180
> + - 640x360
> + - 640x480
> + - 1280x720
> + - 1920x1080
> + - 3840x2160
> + * Pixelformat: BGRA8888 (320x180)-(3840x2160)/(+0,+0)
> + - 320x180
> + - 640x360
> + - 640x480
> + - 1280x720
> + - 1920x1080
> + - 3840x2160
> + * Pixelformat: RGBA8888 (320x180)-(3840x2160)/(+0,+0)
> + - 320x180
> + - 640x360
> + - 640x480
> + - 1280x720
> + - 1920x1080
> + - 3840x2160
> +
> +"""
> +
> +Signed-off-by: Kieran Bingham <kieran.bingham at ideasonboard.com>
> +---
> + src/libcamera/pipeline/vivid/vivid.cpp | 74 +++++++++++++++++++++++++-
> + 1 file changed, 73 insertions(+), 1 deletion(-)
> +
> +diff --git a/src/libcamera/pipeline/vivid/vivid.cpp b/src/libcamera/pipeline/vivid/vivid.cpp
> +index a8922ce70ed4..9e95bae8bc30 100644
> +--- a/src/libcamera/pipeline/vivid/vivid.cpp
> ++++ b/src/libcamera/pipeline/vivid/vivid.cpp
> +@@ -6,6 +6,7 @@
> + */
> +
> + #include <libcamera/camera.h>
> ++#include <libcamera/formats.h>
> +
> + #include "libcamera/internal/device_enumerator.h"
> + #include "libcamera/internal/log.h"
> +@@ -63,8 +64,50 @@ public:
> + int queueRequestDevice(Camera *camera, Request *request) override;
> +
> + bool match(DeviceEnumerator *enumerator) override;
> ++
> ++private:
> ++ int processControls(VividCameraData *data, Request *request);
> ++
> ++ VividCameraData *cameraData(const Camera *camera)
> ++ {
> ++ return static_cast<VividCameraData *>(
> ++ PipelineHandler::cameraData(camera));
> ++ }
> + };
> +
> ++VividCameraConfiguration::VividCameraConfiguration()
> ++ : CameraConfiguration()
> ++{
> ++}
> ++
> ++CameraConfiguration::Status VividCameraConfiguration::validate()
> ++{
> ++ Status status = Valid;
> ++
> ++ if (config_.empty())
> ++ return Invalid;
> ++
> ++ /* Cap the number of entries to the available streams. */
> ++ if (config_.size() > 1) {
> ++ config_.resize(1);
> ++ status = Adjusted;
> ++ }
> ++
> ++ StreamConfiguration &cfg = config_[0];
> ++
> ++ /* Adjust the pixel format. */
> ++ const std::vector<libcamera::PixelFormat> formats = cfg.formats().pixelformats();
> ++ if (std::find(formats.begin(), formats.end(), cfg.pixelFormat) == formats.end()) {
> ++ cfg.pixelFormat = cfg.formats().pixelformats()[0];
> ++ LOG(VIVID, Debug) << "Adjusting format to " << cfg.pixelFormat.toString();
> ++ status = Adjusted;
> ++ }
> ++
> ++ cfg.bufferCount = 4;
> ++
> ++ return status;
> ++}
> ++
> + PipelineHandlerVivid::PipelineHandlerVivid(CameraManager *manager)
> + : PipelineHandler(manager)
> + {
> +@@ -73,7 +116,36 @@ PipelineHandlerVivid::PipelineHandlerVivid(CameraManager *manager)
> + CameraConfiguration *PipelineHandlerVivid::generateConfiguration(Camera *camera,
> + const StreamRoles &roles)
> + {
> +- return nullptr;
> ++ CameraConfiguration *config = new VividCameraConfiguration();
> ++ VividCameraData *data = cameraData(camera);
> ++
> ++ if (roles.empty())
> ++ return config;
> ++
> ++ std::map<V4L2PixelFormat, std::vector<SizeRange>> v4l2Formats =
> ++ data->video_->formats();
> ++ std::map<PixelFormat, std::vector<SizeRange>> deviceFormats;
> ++ std::transform(v4l2Formats.begin(), v4l2Formats.end(),
> ++ std::inserter(deviceFormats, deviceFormats.begin()),
> ++ [&](const decltype(v4l2Formats)::value_type &format) {
> ++ return decltype(deviceFormats)::value_type{
> ++ format.first.toPixelFormat(),
> ++ format.second
> ++ };
> ++ });
> ++
> ++ StreamFormats formats(deviceFormats);
> ++ StreamConfiguration cfg(formats);
> ++
> ++ cfg.pixelFormat = formats::BGR888;
> ++ cfg.size = { 1280, 720 };
> ++ cfg.bufferCount = 4;
> ++
> ++ config->addConfiguration(cfg);
> ++
> ++ config->validate();
> ++
> ++ return config;
> + }
> +
> + int PipelineHandlerVivid::configure(Camera *camera, CameraConfiguration *config)
> +--
> +2.25.1
> +
> diff --git a/Documentation/vivid-pipeline-handler/0005-libcamera-pipeline-vivid-Configure-the-device.patch b/Documentation/vivid-pipeline-handler/0005-libcamera-pipeline-vivid-Configure-the-device.patch
> new file mode 100644
> index 000000000000..c750ce4e80d4
> --- /dev/null
> +++ b/Documentation/vivid-pipeline-handler/0005-libcamera-pipeline-vivid-Configure-the-device.patch
> @@ -0,0 +1,71 @@
> +From 08df6b40552e67c3607aa1d6802f47beec76e151 Mon Sep 17 00:00:00 2001
> +From: Kieran Bingham <kieran.bingham at ideasonboard.com>
> +Date: Mon, 13 Jul 2020 09:51:51 +0100
> +Subject: [PATCH 5/9] libcamera: pipeline: vivid: Configure the device
> +
> +When the configurations have been generated and validated, they can be
> +applied to a device.
> +
> +Vivid supports only a single stream, so it directly obtains the first
> +StreamConfiguration from the CameraConfiguration.
> +
> +The VIVID catpure device is a V4L2Video device, so we generate a
> +V4L2DeviceFormat to apply directly to the capture device node.
> +
> +Note that we explicitly convert the libcamera Format stored in
> +cfg.pixelFormat to a V4L2PixelFormat using the helpers provided by the
> +V4L2VideoDevice to ensure that any multiplanar formats are handled
> +correctly and accordingly.
> +
> +Following the call to set the format using the Kernel API, if the format
> +has been adjusted in any way by the kernel driver, then we have failed
> +to correctly handle the validation stages, and thus the configure
> +operation is idendified has having failed.
> +
> +Finally stream specific data can be directly stored and set as
> +reflecting the state of the stream.
> +
> +[NOTE: the cfg.setStream() call here associates the stream to the
> +StreamConfiguration however that should quite likely be done as part of
> +the validation process. TBD]
> +
> +Signed-off-by: Kieran Bingham <kieran.bingham at ideasonboard.com>
> +---
> + src/libcamera/pipeline/vivid/vivid.cpp | 21 ++++++++++++++++++++-
> + 1 file changed, 20 insertions(+), 1 deletion(-)
> +
> +diff --git a/src/libcamera/pipeline/vivid/vivid.cpp b/src/libcamera/pipeline/vivid/vivid.cpp
> +index 9e95bae8bc30..dbc19424e75a 100644
> +--- a/src/libcamera/pipeline/vivid/vivid.cpp
> ++++ b/src/libcamera/pipeline/vivid/vivid.cpp
> +@@ -150,7 +150,26 @@ CameraConfiguration *PipelineHandlerVivid::generateConfiguration(Camera *camera,
> +
> + int PipelineHandlerVivid::configure(Camera *camera, CameraConfiguration *config)
> + {
> +- return -1;
> ++ VividCameraData *data = cameraData(camera);
> ++ StreamConfiguration &cfg = config->at(0);
> ++ int ret;
> ++
> ++ V4L2DeviceFormat format = {};
> ++ format.fourcc = data->video_->toV4L2PixelFormat(cfg.pixelFormat);
> ++ format.size = cfg.size;
> ++
> ++ ret = data->video_->setFormat(&format);
> ++ if (ret)
> ++ return ret;
> ++
> ++ if (format.size != cfg.size ||
> ++ format.fourcc != data->video_->toV4L2PixelFormat(cfg.pixelFormat))
> ++ return -EINVAL;
> ++
> ++ cfg.setStream(&data->stream_);
> ++ cfg.stride = format.planes[0].bpl;
> ++
> ++ return 0;
> + }
> +
> + int PipelineHandlerVivid::exportFrameBuffers(Camera *camera, Stream *stream,
> +--
> +2.25.1
> +
> diff --git a/Documentation/vivid-pipeline-handler/0006-libcamera-pipeline-vivid-Buffer-handling-and-stream-.patch b/Documentation/vivid-pipeline-handler/0006-libcamera-pipeline-vivid-Buffer-handling-and-stream-.patch
> new file mode 100644
> index 000000000000..b04ef3e10473
> --- /dev/null
> +++ b/Documentation/vivid-pipeline-handler/0006-libcamera-pipeline-vivid-Buffer-handling-and-stream-.patch
> @@ -0,0 +1,102 @@
> +From 8b6d4200c60513bdb9c435c94d512efb4f59441c Mon Sep 17 00:00:00 2001
> +From: Kieran Bingham <kieran.bingham at ideasonboard.com>
> +Date: Mon, 13 Jul 2020 10:46:26 +0100
> +Subject: [PATCH 6/9] libcamera: pipeline: vivid: Buffer handling and stream
> + control
> +
> +We can now add buffer management, and connect up our bufferReady signal
> +to a callback.
> +
> +Note that we provide the ability to export buffers from our capture
> +device (data->video_) using the exportBuffers() functionality from the
> +V4L2VideoDevice which allows a FrameBufferAllocater to obtain buffers
> +from this device.
> +
> +When buffers are obtained through the exportFrameBuffers API, they are
> +orphaned and left unassociated with the device, and must be reimported
> +at start() time anyway. This allows the same interface to be used
> +whether internal buffers, or external buffers are used for the stream.
> +
> +When a buffer completes, we call the buffer completion handler on the
> +pipeline handler, and because we have only a single stream, we can also
> +immediately complete the request.
> +
> +Signed-off-by: Kieran Bingham <kieran.bingham at ideasonboard.com>
> +---
> + src/libcamera/pipeline/vivid/vivid.cpp | 35 ++++++++++++++++++++++++--
> + 1 file changed, 33 insertions(+), 2 deletions(-)
> +
> +diff --git a/src/libcamera/pipeline/vivid/vivid.cpp b/src/libcamera/pipeline/vivid/vivid.cpp
> +index dbc19424e75a..1a945a744055 100644
> +--- a/src/libcamera/pipeline/vivid/vivid.cpp
> ++++ b/src/libcamera/pipeline/vivid/vivid.cpp
> +@@ -32,6 +32,7 @@ public:
> + }
> +
> + int init();
> ++ void bufferReady(FrameBuffer *buffer);
> +
> + MediaDevice *media_;
> + V4L2VideoDevice *video_;
> +@@ -175,16 +176,36 @@ int PipelineHandlerVivid::configure(Camera *camera, CameraConfiguration *config)
> + int PipelineHandlerVivid::exportFrameBuffers(Camera *camera, Stream *stream,
> + std::vector<std::unique_ptr<FrameBuffer>> *buffers)
> + {
> +- return -1;
> ++ VividCameraData *data = cameraData(camera);
> ++ unsigned int count = stream->configuration().bufferCount;
> ++
> ++ return data->video_->exportBuffers(count, buffers);
> + }
> +
> + int PipelineHandlerVivid::start(Camera *camera)
> + {
> +- return -1;
> ++ VividCameraData *data = cameraData(camera);
> ++ unsigned int count = data->stream_.configuration().bufferCount;
> ++
> ++ int ret = data->video_->importBuffers(count);
> ++ if (ret < 0)
> ++ return ret;
> ++
> ++ ret = data->video_->streamOn();
> ++ if (ret < 0) {
> ++ data->ipa_->stop();
> ++ data->video_->releaseBuffers();
> ++ return ret;
> ++ }
> ++
> ++ return 0;
> + }
> +
> + void PipelineHandlerVivid::stop(Camera *camera)
> + {
> ++ VividCameraData *data = cameraData(camera);
> ++ data->video_->streamOff();
> ++ data->video_->releaseBuffers();
> + }
> +
> + int PipelineHandlerVivid::queueRequestDevice(Camera *camera, Request *request)
> +@@ -221,9 +242,19 @@ int VividCameraData::init()
> + if (video_->open())
> + return -ENODEV;
> +
> ++ video_->bufferReady.connect(this, &VividCameraData::bufferReady);
> ++
> + return 0;
> + }
> +
> ++void VividCameraData::bufferReady(FrameBuffer *buffer)
> ++{
> ++ Request *request = buffer->request();
> ++
> ++ pipe_->completeBuffer(camera_, request, buffer);
> ++ pipe_->completeRequest(camera_, request);
> ++}
> ++
> + REGISTER_PIPELINE_HANDLER(PipelineHandlerVivid);
> +
> + } /* namespace libcamera */
> +--
> +2.25.1
> +
> diff --git a/Documentation/vivid-pipeline-handler/0007-libcamera-pipeline-vivid-Queue-requests.patch b/Documentation/vivid-pipeline-handler/0007-libcamera-pipeline-vivid-Queue-requests.patch
> new file mode 100644
> index 000000000000..a896e6d12e37
> --- /dev/null
> +++ b/Documentation/vivid-pipeline-handler/0007-libcamera-pipeline-vivid-Queue-requests.patch
> @@ -0,0 +1,45 @@
> +From 22e1f2c2736cd860a63086695f1dded0b2b2aa0e Mon Sep 17 00:00:00 2001
> +From: Kieran Bingham <kieran.bingham at ideasonboard.com>
> +Date: Mon, 13 Jul 2020 11:35:11 +0100
> +Subject: [PATCH 7/9] libcamera: pipeline: vivid: Queue requests
> +
> +When a request is given to a pipeline handler, it must parse the request
> +and identify what actions the pipeline handler should take to enact on hardware.
> +
> +In the case of the VIVID pipeline handler, we identify the buffer from the only
> +supported stream, and queue it to the video capture device.
> +
> +Signed-off-by: Kieran Bingham <kieran.bingham at ideasonboard.com>
> +---
> + src/libcamera/pipeline/vivid/vivid.cpp | 15 ++++++++++++++-
> + 1 file changed, 14 insertions(+), 1 deletion(-)
> +
> +diff --git a/src/libcamera/pipeline/vivid/vivid.cpp b/src/libcamera/pipeline/vivid/vivid.cpp
> +index 1a945a744055..4362e73f49a5 100644
> +--- a/src/libcamera/pipeline/vivid/vivid.cpp
> ++++ b/src/libcamera/pipeline/vivid/vivid.cpp
> +@@ -210,7 +210,20 @@ void PipelineHandlerVivid::stop(Camera *camera)
> +
> + int PipelineHandlerVivid::queueRequestDevice(Camera *camera, Request *request)
> + {
> +- return -1;
> ++ VividCameraData *data = cameraData(camera);
> ++ FrameBuffer *buffer = request->findBuffer(&data->stream_);
> ++ if (!buffer) {
> ++ LOG(VIVID, Error)
> ++ << "Attempt to queue request with invalid stream";
> ++
> ++ return -ENOENT;
> ++ }
> ++
> ++ int ret = data->video_->queueBuffer(buffer);
> ++ if (ret < 0)
> ++ return ret;
> ++
> ++ return 0;
> + }
> +
> + bool PipelineHandlerVivid::match(DeviceEnumerator *enumerator)
> +--
> +2.25.1
> +
> diff --git a/Documentation/vivid-pipeline-handler/0008-libcamera-pipeline-vivid-Initialise-key-controls.patch b/Documentation/vivid-pipeline-handler/0008-libcamera-pipeline-vivid-Initialise-key-controls.patch
> new file mode 100644
> index 000000000000..8e0fca45fa04
> --- /dev/null
> +++ b/Documentation/vivid-pipeline-handler/0008-libcamera-pipeline-vivid-Initialise-key-controls.patch
> @@ -0,0 +1,70 @@
> +From 703b1d24259934781c9f9ff80774e5845876c47f Mon Sep 17 00:00:00 2001
> +From: Kieran Bingham <kieran.bingham at ideasonboard.com>
> +Date: Mon, 13 Jul 2020 12:48:37 +0100
> +Subject: [PATCH 8/9] libcamera: pipeline: vivid: Initialise key controls
> +
> +The VIVID pipeline handler retains state globally of it's controls.
> +Ensure that when we configure this specific pipeline we set initial
> +parameters on the device that suit our (specific) needs.
> +
> +This introduces how controls can be set directly on a device, however
> +under normal circumstances controls should usually be set from libcamera
> +controls as part of a request. These are VIVID specific only.
> +
> +Signed-off-by: Kieran Bingham <kieran.bingham at ideasonboard.com>
> +---
> + src/libcamera/pipeline/vivid/vivid.cpp | 31 ++++++++++++++++++++++++++
> + 1 file changed, 31 insertions(+)
> +
> +diff --git a/src/libcamera/pipeline/vivid/vivid.cpp b/src/libcamera/pipeline/vivid/vivid.cpp
> +index 4362e73f49a5..1744d78f2f28 100644
> +--- a/src/libcamera/pipeline/vivid/vivid.cpp
> ++++ b/src/libcamera/pipeline/vivid/vivid.cpp
> +@@ -14,6 +14,18 @@
> + #include "libcamera/internal/pipeline_handler.h"
> + #include "libcamera/internal/v4l2_videodevice.h"
> +
> ++#define VIVID_CID_VIVID_BASE (0x00f00000 | 0xf000)
> ++#define VIVID_CID_VIVID_CLASS (0x00f00000 | 1)
> ++#define VIVID_CID_TEST_PATTERN (VIVID_CID_VIVID_BASE + 0)
> ++#define VIVID_CID_OSD_TEXT_MODE (VIVID_CID_VIVID_BASE + 1)
> ++#define VIVID_CID_HOR_MOVEMENT (VIVID_CID_VIVID_BASE + 2)
> ++#define VIVID_CID_VERT_MOVEMENT (VIVID_CID_VIVID_BASE + 3)
> ++#define VIVID_CID_SHOW_BORDER (VIVID_CID_VIVID_BASE + 4)
> ++#define VIVID_CID_SHOW_SQUARE (VIVID_CID_VIVID_BASE + 5)
> ++#define VIVID_CID_INSERT_SAV (VIVID_CID_VIVID_BASE + 6)
> ++#define VIVID_CID_INSERT_EAV (VIVID_CID_VIVID_BASE + 7)
> ++#define VIVID_CID_VBI_CAP_INTERLACED (VIVID_CID_VIVID_BASE + 8)
> ++
> + namespace libcamera {
> +
> + LOG_DEFINE_CATEGORY(VIVID)
> +@@ -167,6 +179,25 @@ int PipelineHandlerVivid::configure(Camera *camera, CameraConfiguration *config)
> + format.fourcc != data->video_->toV4L2PixelFormat(cfg.pixelFormat))
> + return -EINVAL;
> +
> ++ /* Set initial controls specific to VIVID */
> ++ ControlList controls(data->video_->controls());
> ++ controls.set(VIVID_CID_TEST_PATTERN, 0); /* Vertical Colour Bars */
> ++ controls.set(VIVID_CID_OSD_TEXT_MODE, 0); /* Display all OSD */
> ++
> ++ /* Ensure clear colours configured. */
> ++ controls.set(V4L2_CID_BRIGHTNESS, 128);
> ++ controls.set(V4L2_CID_CONTRAST, 128);
> ++ controls.set(V4L2_CID_SATURATION, 128);
> ++
> ++ /* Enable movement to visualise buffer updates. */
> ++ controls.set(VIVID_CID_HOR_MOVEMENT, 5);
> ++
> ++ ret = data->video_->setControls(&controls);
> ++ if (ret) {
> ++ LOG(VIVID, Error) << "Failed to set controls: " << ret;
> ++ return ret < 0 ? ret : -EINVAL;
> ++ }
> ++
> + cfg.setStream(&data->stream_);
> + cfg.stride = format.planes[0].bpl;
> +
> +--
> +2.25.1
> +
> diff --git a/Documentation/vivid-pipeline-handler/0009-libcamera-pipeline-vivid-Handle-controls.patch b/Documentation/vivid-pipeline-handler/0009-libcamera-pipeline-vivid-Handle-controls.patch
> new file mode 100644
> index 000000000000..ea88ef44c102
> --- /dev/null
> +++ b/Documentation/vivid-pipeline-handler/0009-libcamera-pipeline-vivid-Handle-controls.patch
> @@ -0,0 +1,133 @@
> +From 03bbac3265554b73e6ddcdebaf95090062e3669f Mon Sep 17 00:00:00 2001
> +From: Kieran Bingham <kieran.bingham at ideasonboard.com>
> +Date: Mon, 13 Jul 2020 12:56:11 +0100
> +Subject: [PATCH 9/9] libcamera: pipeline: vivid: Handle controls
> +
> +When constructing the camera, we parse the available controls on the
> +video capture device, and map supported controls to libcamera controls,
> +and initialise the defaults.
> +
> +The controls are handled during queueRequestDevice for each request and
> +applied to the device through the capture node.
> +
> +Signed-off-by: Kieran Bingham <kieran.bingham at ideasonboard.com>
> +---
> + src/libcamera/pipeline/vivid/vivid.cpp | 80 +++++++++++++++++++++++++-
> + 1 file changed, 79 insertions(+), 1 deletion(-)
> +
> +diff --git a/src/libcamera/pipeline/vivid/vivid.cpp b/src/libcamera/pipeline/vivid/vivid.cpp
> +index 1744d78f2f28..b8b8e3ae0287 100644
> +--- a/src/libcamera/pipeline/vivid/vivid.cpp
> ++++ b/src/libcamera/pipeline/vivid/vivid.cpp
> +@@ -5,7 +5,11 @@
> + * vivid.cpp - Pipeline handler for the vivid capture device
> + */
> +
> ++#include <math.h>
> ++
> + #include <libcamera/camera.h>
> ++#include <libcamera/control_ids.h>
> ++#include <libcamera/controls.h>
> + #include <libcamera/formats.h>
> +
> + #include "libcamera/internal/device_enumerator.h"
> +@@ -239,6 +243,46 @@ void PipelineHandlerVivid::stop(Camera *camera)
> + data->video_->releaseBuffers();
> + }
> +
> ++int PipelineHandlerVivid::processControls(VividCameraData *data, Request *request)
> ++{
> ++ ControlList controls(data->video_->controls());
> ++
> ++ for (auto it : request->controls()) {
> ++ unsigned int id = it.first;
> ++ unsigned int offset;
> ++ uint32_t cid;
> ++
> ++ if (id == controls::Brightness) {
> ++ cid = V4L2_CID_BRIGHTNESS;
> ++ offset = 128;
> ++ } else if (id == controls::Contrast) {
> ++ cid = V4L2_CID_CONTRAST;
> ++ offset = 0;
> ++ } else if (id == controls::Saturation) {
> ++ cid = V4L2_CID_SATURATION;
> ++ offset = 0;
> ++ } else {
> ++ continue;
> ++ }
> ++
> ++ int32_t value = lroundf(it.second.get<float>() * 128 + offset);
> ++ controls.set(cid, utils::clamp(value, 0, 255));
> ++ }
> ++
> ++ for (const auto &ctrl : controls)
> ++ LOG(VIVID, Debug)
> ++ << "Setting control " << utils::hex(ctrl.first)
> ++ << " to " << ctrl.second.toString();
> ++
> ++ int ret = data->video_->setControls(&controls);
> ++ if (ret) {
> ++ LOG(VIVID, Error) << "Failed to set controls: " << ret;
> ++ return ret < 0 ? ret : -EINVAL;
> ++ }
> ++
> ++ return ret;
> ++}
> ++
> + int PipelineHandlerVivid::queueRequestDevice(Camera *camera, Request *request)
> + {
> + VividCameraData *data = cameraData(camera);
> +@@ -250,7 +294,11 @@ int PipelineHandlerVivid::queueRequestDevice(Camera *camera, Request *request)
> + return -ENOENT;
> + }
> +
> +- int ret = data->video_->queueBuffer(buffer);
> ++ int ret = processControls(data, request);
> ++ if (ret < 0)
> ++ return ret;
> ++
> ++ ret = data->video_->queueBuffer(buffer);
> + if (ret < 0)
> + return ret;
> +
> +@@ -288,6 +336,36 @@ int VividCameraData::init()
> +
> + video_->bufferReady.connect(this, &VividCameraData::bufferReady);
> +
> ++ /* Initialise the supported controls. */
> ++ const ControlInfoMap &controls = video_->controls();
> ++ ControlInfoMap::Map ctrls;
> ++
> ++ for (const auto &ctrl : controls) {
> ++ const ControlId *id;
> ++ ControlInfo info;
> ++
> ++ switch (ctrl.first->id()) {
> ++ case V4L2_CID_BRIGHTNESS:
> ++ id = &controls::Brightness;
> ++ info = ControlInfo{ { -1.0f }, { 1.0f }, { 0.0f } };
> ++ break;
> ++ case V4L2_CID_CONTRAST:
> ++ id = &controls::Contrast;
> ++ info = ControlInfo{ { 0.0f }, { 2.0f }, { 1.0f } };
> ++ break;
> ++ case V4L2_CID_SATURATION:
> ++ id = &controls::Saturation;
> ++ info = ControlInfo{ { 0.0f }, { 2.0f }, { 1.0f } };
> ++ break;
> ++ default:
> ++ continue;
> ++ }
> ++
> ++ ctrls.emplace(id, info);
> ++ }
> ++
> ++ controlInfo_ = std::move(ctrls);
> ++
> + return 0;
> + }
> +
> +--
> +2.25.1
> +
--
Regards,
Laurent Pinchart
More information about the libcamera-devel
mailing list