[PATCH v3 4/7] pipeline: rpi:: Add wallclock timestamp support

[Kieran Bingham]

Quoting David Plowman (2025-01-09 14:32:08)
> A ClockRecovery object is added for derived classes to use, and
> wallclock timetamps are copied into the request metadata for
> applications.
> 
> In the derived class, wallclock timestamps are derived corresponding
> to the sensor timestamp, and made available to the base pipeline
> handler class.

Thanks, this looks like a clear and easy way to use this and to
facilitate implementing the same in all pipeline handlers.

But we'll deal with other pipeline handlers on top.


Reviewed-by: Kieran Bingham <kieran.bingham at ideasonboard.com>

> 
> Signed-off-by: David Plowman <david.plowman at raspberrypi.com>
> ---
>  src/libcamera/pipeline/rpi/common/pipeline_base.cpp |  5 +++++
>  src/libcamera/pipeline/rpi/common/pipeline_base.h   |  3 +++
>  src/libcamera/pipeline/rpi/vc4/vc4.cpp              | 10 ++++++++--
>  3 files changed, 16 insertions(+), 2 deletions(-)
> 
> diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp
> index 4b147fdb..56f3c4df 100644
> --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp
> +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp
> @@ -686,6 +686,9 @@ int PipelineHandlerBase::start(Camera *camera, const ControlList *controls)
>                 return ret;
>         }
>  
> +       /* A good moment to add an initial clock sample. */
> +       data->wallClockRecovery_.addSample();
> +
>         /*
>          * Reset the delayed controls with the gain and exposure values set by
>          * the IPA.
> @@ -1510,6 +1513,8 @@ void CameraData::fillRequestMetadata(const ControlList &bufferControls, Request
>  {
>         request->metadata().set(controls::SensorTimestamp,
>                                 bufferControls.get(controls::SensorTimestamp).value_or(0));
> +       request->metadata().set(controls::FrameWallClock,
> +                               bufferControls.get(controls::FrameWallClock).value_or(0));
>  
>         if (cropParams_.size()) {
>                 std::vector<Rectangle> crops;
> diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h
> index aae0c2f3..ca706bf3 100644
> --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h
> +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h
> @@ -20,6 +20,7 @@
>  #include "libcamera/internal/bayer_format.h"
>  #include "libcamera/internal/camera.h"
>  #include "libcamera/internal/camera_sensor.h"
> +#include "libcamera/internal/clock_recovery.h"
>  #include "libcamera/internal/framebuffer.h"
>  #include "libcamera/internal/media_device.h"
>  #include "libcamera/internal/media_object.h"
> @@ -177,6 +178,8 @@ public:
>  
>         Config config_;
>  
> +       ClockRecovery wallClockRecovery_;
> +
>  protected:
>         void fillRequestMetadata(const ControlList &bufferControls,
>                                  Request *request);
> diff --git a/src/libcamera/pipeline/rpi/vc4/vc4.cpp b/src/libcamera/pipeline/rpi/vc4/vc4.cpp
> index fd8d84b1..3d8c65c6 100644
> --- a/src/libcamera/pipeline/rpi/vc4/vc4.cpp
> +++ b/src/libcamera/pipeline/rpi/vc4/vc4.cpp
> @@ -781,9 +781,15 @@ void Vc4CameraData::unicamBufferDequeue(FrameBuffer *buffer)
>                 auto [ctrl, delayContext] = delayedCtrls_->get(buffer->metadata().sequence);
>                 /*
>                  * Add the frame timestamp to the ControlList for the IPA to use
> -                * as it does not receive the FrameBuffer object.
> +                * as it does not receive the FrameBuffer object. Also derive a
> +                * corresponding wallclock value.
>                  */
> -               ctrl.set(controls::SensorTimestamp, buffer->metadata().timestamp);
> +               wallClockRecovery_.addSample();
> +               uint64_t sensorTimestamp = buffer->metadata().timestamp;
> +               uint64_t wallClockTimestamp = wallClockRecovery_.getOutput(sensorTimestamp / 1000);
> +
> +               ctrl.set(controls::SensorTimestamp, sensorTimestamp);
> +               ctrl.set(controls::FrameWallClock, wallClockTimestamp);
>                 bayerQueue_.push({ buffer, std::move(ctrl), delayContext });
>         } else {
>                 embeddedQueue_.push(buffer);
> -- 
> 2.39.5
>