[libcamera-devel] [PATCH 3/3] libcamera: v4l2: Detect frame drops on V4L2VideoDevice

[Kieran Bingham]

Pipelines with ISPs will report the output sequence number of the
V4L2VideoDevice of the ISP as the sequence number of the request buffer
completion. These have been seen to provide their own monotonic sequence
number, which means that any frame drop from the CSI2 receiver is
silently ignored and can easily go unnoticed. (except for frame rate
changes).

Add an internal tracking of the sequence number on V4L2VideoDevice
instances to identify if there is ever a break in the stream.

Signed-off-by: Kieran Bingham <kieran.bingham at ideasonboard.com>
---
 include/libcamera/internal/v4l2_videodevice.h |  2 ++
 src/libcamera/v4l2_videodevice.cpp            | 10 ++++++++++
 2 files changed, 12 insertions(+)

diff --git a/include/libcamera/internal/v4l2_videodevice.h b/include/libcamera/internal/v4l2_videodevice.h
index 9c9493cc16ed..ab4c4c56436d 100644
--- a/include/libcamera/internal/v4l2_videodevice.h
+++ b/include/libcamera/internal/v4l2_videodevice.h
@@ -28,6 +28,7 @@
 #include <libcamera/framebuffer.h>
 #include <libcamera/geometry.h>
 #include <libcamera/pixel_format.h>
+#include <libcamera/sequence.h>
 
 #include "libcamera/internal/formats.h"
 #include "libcamera/internal/v4l2_device.h"
@@ -273,6 +274,7 @@ private:
 	EventNotifier *fdBufferNotifier_;
 
 	State state_;
+	Sequence monotonicObserver_;
 
 	Timer watchdog_;
 	utils::Duration watchdogDuration_;
diff --git a/src/libcamera/v4l2_videodevice.cpp b/src/libcamera/v4l2_videodevice.cpp
index 05d3273ebb58..4cef5d40e0c6 100644
--- a/src/libcamera/v4l2_videodevice.cpp
+++ b/src/libcamera/v4l2_videodevice.cpp
@@ -1749,6 +1749,14 @@ FrameBuffer *V4L2VideoDevice::dequeueBuffer()
 	if (V4L2_TYPE_IS_OUTPUT(buf.type))
 		return buffer;
 
+	/*
+	 * Observe the buffer sequence number on capture devices to check for
+	 * frame drops.
+	 */
+	int drops = monotonicObserver_.update(buf.sequence);
+	if (drops)
+		LOG(V4L2, Warning) << "Dropped " << drops << " frames";
+
 	unsigned int numV4l2Planes = multiPlanar ? buf.length : 1;
 	FrameMetadata &metadata = buffer->metadata_;
 
@@ -1832,6 +1840,8 @@ int V4L2VideoDevice::streamOn()
 		return ret;
 	}
 
+	monotonicObserver_.reset();
+
 	state_ = State::Streaming;
 	if (watchdogDuration_.count())
 		watchdog_.start(std::chrono::duration_cast<std::chrono::milliseconds>(watchdogDuration_));
-- 
2.25.1