[libcamera-devel] [PATCH 3/3] libcamera: v4l2: Detect frame drops on V4L2VideoDevice

[Umang Jain]

Hi Kieran,

Thank you for the patch

On 4/30/22 01:04, Kieran Bingham via libcamera-devel wrote:
> Pipelines with ISPs will report the output sequence number of the
> V4L2VideoDevice of the ISP as the sequence number of the request buffer
> completion. These have been seen to provide their own monotonic sequence
> number, which means that any frame drop from the CSI2 receiver is
> silently ignored and can easily go unnoticed. (except for frame rate
> changes).
>
> Add an internal tracking of the sequence number on V4L2VideoDevice
> instances to identify if there is ever a break in the stream.
>
> Signed-off-by: Kieran Bingham <kieran.bingham at ideasonboard.com>


Reviewed-by: Umang Jain <umang.jain at ideasonboard.com>

> ---
>   include/libcamera/internal/v4l2_videodevice.h |  2 ++
>   src/libcamera/v4l2_videodevice.cpp            | 10 ++++++++++
>   2 files changed, 12 insertions(+)
>
> diff --git a/include/libcamera/internal/v4l2_videodevice.h b/include/libcamera/internal/v4l2_videodevice.h
> index 9c9493cc16ed..ab4c4c56436d 100644
> --- a/include/libcamera/internal/v4l2_videodevice.h
> +++ b/include/libcamera/internal/v4l2_videodevice.h
> @@ -28,6 +28,7 @@
>   #include <libcamera/framebuffer.h>
>   #include <libcamera/geometry.h>
>   #include <libcamera/pixel_format.h>
> +#include <libcamera/sequence.h>
>   
>   #include "libcamera/internal/formats.h"
>   #include "libcamera/internal/v4l2_device.h"
> @@ -273,6 +274,7 @@ private:
>   	EventNotifier *fdBufferNotifier_;
>   
>   	State state_;
> +	Sequence monotonicObserver_;
>   
>   	Timer watchdog_;
>   	utils::Duration watchdogDuration_;
> diff --git a/src/libcamera/v4l2_videodevice.cpp b/src/libcamera/v4l2_videodevice.cpp
> index 05d3273ebb58..4cef5d40e0c6 100644
> --- a/src/libcamera/v4l2_videodevice.cpp
> +++ b/src/libcamera/v4l2_videodevice.cpp
> @@ -1749,6 +1749,14 @@ FrameBuffer *V4L2VideoDevice::dequeueBuffer()
>   	if (V4L2_TYPE_IS_OUTPUT(buf.type))
>   		return buffer;
>   
> +	/*
> +	 * Observe the buffer sequence number on capture devices to check for
> +	 * frame drops.
> +	 */
> +	int drops = monotonicObserver_.update(buf.sequence);
> +	if (drops)
> +		LOG(V4L2, Warning) << "Dropped " << drops << " frames";
> +
>   	unsigned int numV4l2Planes = multiPlanar ? buf.length : 1;
>   	FrameMetadata &metadata = buffer->metadata_;
>   
> @@ -1832,6 +1840,8 @@ int V4L2VideoDevice::streamOn()
>   		return ret;
>   	}
>   
> +	monotonicObserver_.reset();
> +
>   	state_ = State::Streaming;
>   	if (watchdogDuration_.count())
>   		watchdog_.start(std::chrono::duration_cast<std::chrono::milliseconds>(watchdogDuration_));