[libcamera-devel] [PATCH 1/3] libcamera: pipeline: Pass libcamera controls into pipeline_handler::start()
Naushir Patuck
naush at raspberrypi.com
Wed Nov 18 14:26:26 CET 2020
Hi Umang,
On Wed, 18 Nov 2020 at 13:16, Umang Jain <email at uajain.com> wrote:
> Hi Naush,
>
> On 11/12/20 2:29 PM, Naushir Patuck wrote:
> > Applications now have the ability to pass in controls that need to be
> > applied on startup, rather than doing it through Request where there
> might
> > be some frames of delay in getting the controls applied.
> >
> > This commit adds the ability to pass in a set of libcamera controls into
> > the pipeline handlers through the pipeline_handler::start() method. These
> > controls are provided by the application through the camera::start()
> > public API.
> Is there a possiblity that the controls are passed in through
> Camera::configure() that will subsequently call
> PipeHandler::configure()? Looking at the way Requests use ControlList,
> it seems there is also a 'validator' that can(/should?) be used with
> this API. *IF* we take that into account, then it feels like
> Camera::configure() is a good place for passing in ControlsList, rather
> than Camera::start()
>
> This is just a quick comment / line of thought. I haven't actually dug
> deep into the Request/ControlsList code at this very moment.
>
Indeed this was an option that we considered. However, there are some
usage issues with passing in controls on configure(). Below is an extract
from an earlier conversion with Kieran on this topic:
On Mon, 19 Oct 2020 at 15:01, Kieran Bingham <
kieran.bingham at ideasonboard.com> wrote:
> Then when a pipeline is called with configure(), it needs to apply any
> controls in the list at that point. And it looks like the
> CameraConfiguration is already passed through to:
>
> int RPiCameraData::configureIPA(const CameraConfiguration *config)
>
> So, then there is the information required at that phase too?
>
> I'm wondering if I've missed something obvious that would have prevented
> you from already using this approach?
>
Indeed, this was one of the approaches we discussed some time back.
However, as you mentioned, CameraConfiguration gets set in the
camera->configure() phase, and this may not necessarily be what we want.
An application may want to set startup parameters without doing a
camera->configure() in a run. With this change, we would do a sequence
like start(), stop() start(new config parameters). If we were to do start(),
stop() configure(new config parameters), start(), it may incur additional
overheads in calling configure() only to setup some new startup params,
where the configure() might be going to the kernel and calling the sensors
to set itself up again, even though it is not needed. Hope that makes
sense?
Regards,
Naush
>
> Thanks!
> >
> > Signed-off-by: Naushir Patuck <naush at raspberrypi.com>
> > ---
> > Documentation/guides/pipeline-handler.rst | 4 ++--
> > include/libcamera/camera.h | 2 +-
> > include/libcamera/internal/pipeline_handler.h | 2 +-
> > src/libcamera/camera.cpp | 11 ++++++-----
> > src/libcamera/pipeline/ipu3/ipu3.cpp | 4 ++--
> > src/libcamera/pipeline/raspberrypi/raspberrypi.cpp | 4 ++--
> > src/libcamera/pipeline/rkisp1/rkisp1.cpp | 4 ++--
> > src/libcamera/pipeline/simple/simple.cpp | 4 ++--
> > src/libcamera/pipeline/uvcvideo/uvcvideo.cpp | 4 ++--
> > src/libcamera/pipeline/vimc/vimc.cpp | 4 ++--
> > src/libcamera/pipeline_handler.cpp | 1 +
> > 11 files changed, 23 insertions(+), 21 deletions(-)
> >
> > diff --git a/Documentation/guides/pipeline-handler.rst
> b/Documentation/guides/pipeline-handler.rst
> > index 57aee455..63275a12 100644
> > --- a/Documentation/guides/pipeline-handler.rst
> > +++ b/Documentation/guides/pipeline-handler.rst
> > @@ -209,7 +209,7 @@ methods for the overridden class members.
> > int exportFrameBuffers(Camera *camera, Stream *stream,
> > std::vector<std::unique_ptr<FrameBuffer>> *buffers) override;
> >
> > - int start(Camera *camera) override;
> > + int start(Camera *camera, ControlList *controls) override;
> > void stop(Camera *camera) override;
> >
> > int queueRequestDevice(Camera *camera, Request *request)
> override;
> > @@ -239,7 +239,7 @@ methods for the overridden class members.
> > return -1;
> > }
> >
> > - int PipelineHandlerVivid::start(Camera *camera)
> > + int PipelineHandlerVivid::start(Camera *camera, ControlList
> *controls)
> > {
> > return -1;
> > }
> > diff --git a/include/libcamera/camera.h b/include/libcamera/camera.h
> > index 5c5f1a05..f94f8599 100644
> > --- a/include/libcamera/camera.h
> > +++ b/include/libcamera/camera.h
> > @@ -103,7 +103,7 @@ public:
> > std::unique_ptr<Request> createRequest(uint64_t cookie = 0);
> > int queueRequest(Request *request);
> >
> > - int start();
> > + int start(ControlList *controls = nullptr);
> > int stop();
> >
> > private:
> > diff --git a/include/libcamera/internal/pipeline_handler.h
> b/include/libcamera/internal/pipeline_handler.h
> > index c12c8904..bd3c4a81 100644
> > --- a/include/libcamera/internal/pipeline_handler.h
> > +++ b/include/libcamera/internal/pipeline_handler.h
> > @@ -78,7 +78,7 @@ public:
> > virtual int exportFrameBuffers(Camera *camera, Stream *stream,
> >
> std::vector<std::unique_ptr<FrameBuffer>> *buffers) = 0;
> >
> > - virtual int start(Camera *camera) = 0;
> > + virtual int start(Camera *camera, ControlList *controls) = 0;
> > virtual void stop(Camera *camera) = 0;
> >
> > int queueRequest(Camera *camera, Request *request);
> > diff --git a/src/libcamera/camera.cpp b/src/libcamera/camera.cpp
> > index dffbd6bd..de9c6c86 100644
> > --- a/src/libcamera/camera.cpp
> > +++ b/src/libcamera/camera.cpp
> > @@ -943,9 +943,10 @@ int Camera::queueRequest(Request *request)
> > /**
> > * \brief Start capture from camera
> > *
> > - * Start the camera capture session. Once the camera is started the
> application
> > - * can queue requests to the camera to process and return to the
> application
> > - * until the capture session is terminated with \a stop().
> > + * Start the camera capture session, optionally providing a list of
> controls to
> > + * action before starting. Once the camera is started the application
> can queue
> > + * requests to the camera to process and return to the application
> until the
> > + * capture session is terminated with \a stop().
> > *
> > * \context This function may only be called when the camera is in the
> > * Configured state as defined in \ref camera_operation, and shall be
> > @@ -956,7 +957,7 @@ int Camera::queueRequest(Request *request)
> > * \retval -ENODEV The camera has been disconnected from the system
> > * \retval -EACCES The camera is not in a state where it can be started
> > */
> > -int Camera::start()
> > +int Camera::start(ControlList *controls)
> > {
> > Private *const d = LIBCAMERA_D_PTR();
> >
> > @@ -967,7 +968,7 @@ int Camera::start()
> > LOG(Camera, Debug) << "Starting capture";
> >
> > ret = d->pipe_->invokeMethod(&PipelineHandler::start,
> > - ConnectionTypeBlocking, this);
> > + ConnectionTypeBlocking, this,
> controls);
> > if (ret)
> > return ret;
> >
> > diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp
> b/src/libcamera/pipeline/ipu3/ipu3.cpp
> > index 4cedb32b..8a1918d5 100644
> > --- a/src/libcamera/pipeline/ipu3/ipu3.cpp
> > +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp
> > @@ -105,7 +105,7 @@ public:
> > int exportFrameBuffers(Camera *camera, Stream *stream,
> > std::vector<std::unique_ptr<FrameBuffer>>
> *buffers) override;
> >
> > - int start(Camera *camera) override;
> > + int start(Camera *camera, ControlList *controls) override;
> > void stop(Camera *camera) override;
> >
> > int queueRequestDevice(Camera *camera, Request *request) override;
> > @@ -596,7 +596,7 @@ int PipelineHandlerIPU3::freeBuffers(Camera *camera)
> > return 0;
> > }
> >
> > -int PipelineHandlerIPU3::start(Camera *camera)
> > +int PipelineHandlerIPU3::start(Camera *camera, [[maybe_unused]]
> ControlList *controls)
> > {
> > IPU3CameraData *data = cameraData(camera);
> > CIO2Device *cio2 = &data->cio2_;
> > diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp
> b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp
> > index 7ad66f21..ddb30e49 100644
> > --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp
> > +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp
> > @@ -237,7 +237,7 @@ public:
> > int exportFrameBuffers(Camera *camera, Stream *stream,
> > std::vector<std::unique_ptr<FrameBuffer>>
> *buffers) override;
> >
> > - int start(Camera *camera) override;
> > + int start(Camera *camera, ControlList *controls) override;
> > void stop(Camera *camera) override;
> >
> > int queueRequestDevice(Camera *camera, Request *request) override;
> > @@ -726,7 +726,7 @@ int
> PipelineHandlerRPi::exportFrameBuffers([[maybe_unused]] Camera *camera, Stre
> > return ret;
> > }
> >
> > -int PipelineHandlerRPi::start(Camera *camera)
> > +int PipelineHandlerRPi::start(Camera *camera, [[maybe_unused]]
> ControlList *controls)
> > {
> > RPiCameraData *data = cameraData(camera);
> > int ret;
> > diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp
> b/src/libcamera/pipeline/rkisp1/rkisp1.cpp
> > index 1b1922a9..2e8d2930 100644
> > --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp
> > +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp
> > @@ -187,7 +187,7 @@ public:
> > int exportFrameBuffers(Camera *camera, Stream *stream,
> > std::vector<std::unique_ptr<FrameBuffer>>
> *buffers) override;
> >
> > - int start(Camera *camera) override;
> > + int start(Camera *camera, ControlList *controls) override;
> > void stop(Camera *camera) override;
> >
> > int queueRequestDevice(Camera *camera, Request *request) override;
> > @@ -822,7 +822,7 @@ int PipelineHandlerRkISP1::freeBuffers(Camera
> *camera)
> > return 0;
> > }
> >
> > -int PipelineHandlerRkISP1::start(Camera *camera)
> > +int PipelineHandlerRkISP1::start(Camera *camera, [[maybe_unused]]
> ControlList *controls)
> > {
> > RkISP1CameraData *data = cameraData(camera);
> > int ret;
> > diff --git a/src/libcamera/pipeline/simple/simple.cpp
> b/src/libcamera/pipeline/simple/simple.cpp
> > index 0d3078f7..b047aeb9 100644
> > --- a/src/libcamera/pipeline/simple/simple.cpp
> > +++ b/src/libcamera/pipeline/simple/simple.cpp
> > @@ -126,7 +126,7 @@ public:
> > int exportFrameBuffers(Camera *camera, Stream *stream,
> > std::vector<std::unique_ptr<FrameBuffer>>
> *buffers) override;
> >
> > - int start(Camera *camera) override;
> > + int start(Camera *camera, ControlList *controls) override;
> > void stop(Camera *camera) override;
> >
> > bool match(DeviceEnumerator *enumerator) override;
> > @@ -646,7 +646,7 @@ int SimplePipelineHandler::exportFrameBuffers(Camera
> *camera, Stream *stream,
> > return data->video_->exportBuffers(count, buffers);
> > }
> >
> > -int SimplePipelineHandler::start(Camera *camera)
> > +int SimplePipelineHandler::start(Camera *camera, [[maybe_unused]]
> ControlList *controls)
> > {
> > SimpleCameraData *data = cameraData(camera);
> > V4L2VideoDevice *video = data->video_;
> > diff --git a/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp
> b/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp
> > index 0f3241cc..87b0f03d 100644
> > --- a/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp
> > +++ b/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp
> > @@ -76,7 +76,7 @@ public:
> > int exportFrameBuffers(Camera *camera, Stream *stream,
> > std::vector<std::unique_ptr<FrameBuffer>>
> *buffers) override;
> >
> > - int start(Camera *camera) override;
> > + int start(Camera *camera, ControlList *controls) override;
> > void stop(Camera *camera) override;
> >
> > int queueRequestDevice(Camera *camera, Request *request) override;
> > @@ -236,7 +236,7 @@ int PipelineHandlerUVC::exportFrameBuffers(Camera
> *camera, Stream *stream,
> > return data->video_->exportBuffers(count, buffers);
> > }
> >
> > -int PipelineHandlerUVC::start(Camera *camera)
> > +int PipelineHandlerUVC::start(Camera *camera, [[maybe_unused]]
> ControlList *controls)
> > {
> > UVCCameraData *data = cameraData(camera);
> > unsigned int count = data->stream_.configuration().bufferCount;
> > diff --git a/src/libcamera/pipeline/vimc/vimc.cpp
> b/src/libcamera/pipeline/vimc/vimc.cpp
> > index 914b6b54..d81b8598 100644
> > --- a/src/libcamera/pipeline/vimc/vimc.cpp
> > +++ b/src/libcamera/pipeline/vimc/vimc.cpp
> > @@ -92,7 +92,7 @@ public:
> > int exportFrameBuffers(Camera *camera, Stream *stream,
> > std::vector<std::unique_ptr<FrameBuffer>>
> *buffers) override;
> >
> > - int start(Camera *camera) override;
> > + int start(Camera *camera, ControlList *controls) override;
> > void stop(Camera *camera) override;
> >
> > int queueRequestDevice(Camera *camera, Request *request) override;
> > @@ -313,7 +313,7 @@ int PipelineHandlerVimc::exportFrameBuffers(Camera
> *camera, Stream *stream,
> > return data->video_->exportBuffers(count, buffers);
> > }
> >
> > -int PipelineHandlerVimc::start(Camera *camera)
> > +int PipelineHandlerVimc::start(Camera *camera, [[maybe_unused]]
> ControlList *controls)
> > {
> > VimcCameraData *data = cameraData(camera);
> > unsigned int count = data->stream_.configuration().bufferCount;
> > diff --git a/src/libcamera/pipeline_handler.cpp
> b/src/libcamera/pipeline_handler.cpp
> > index 894200ee..bafcf21b 100644
> > --- a/src/libcamera/pipeline_handler.cpp
> > +++ b/src/libcamera/pipeline_handler.cpp
> > @@ -351,6 +351,7 @@ const ControlList &PipelineHandler::properties(const
> Camera *camera) const
> > * \fn PipelineHandler::start()
> > * \brief Start capturing from a group of streams
> > * \param[in] camera The camera to start
> > + * \param[in] controls Controls for the IPA to action before starting
> the camera
> > *
> > * Start the group of streams that have been configured for capture by
> > * \a configure(). The intended caller of this method is the Camera
> class which
>
> _______________________________________________
> libcamera-devel mailing list
> libcamera-devel at lists.libcamera.org
> https://lists.libcamera.org/listinfo/libcamera-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.libcamera.org/pipermail/libcamera-devel/attachments/20201118/dff5e60e/attachment-0001.htm>
More information about the libcamera-devel
mailing list