[libcamera-devel] [PATCH 4/8] cam: Use Sensor sequence numbers and detect frame drop

[Umang Jain]

Hi Kieran,

On 12/7/21 5:09 AM, Kieran Bingham wrote:
> The stream buffer sequence numbers might produce sequential
> monotonic sequence numbers from an ISP producing a frame for every
> input.
>
> This however, doesn't capture pipeline stalls that cause us to miss or
> drop frames from the sensor.
>
> Use the SensorSequence metadata to report sequence information, and add
> a warning to the summary line if a frame drop is detected.
>
> Signed-off-by: Kieran Bingham <kieran.bingham at ideasonboard.com>
> ---
>   src/cam/camera_session.cpp | 17 +++++++++++++++--
>   src/cam/camera_session.h   |  1 +
>   2 files changed, 16 insertions(+), 2 deletions(-)
>
> diff --git a/src/cam/camera_session.cpp b/src/cam/camera_session.cpp
> index 50170723c30f..f9495f7f7e42 100644
> --- a/src/cam/camera_session.cpp
> +++ b/src/cam/camera_session.cpp
> @@ -29,8 +29,8 @@ CameraSession::CameraSession(CameraManager *cm,
>   			     unsigned int cameraIndex,
>   			     const OptionsParser::Options &options)
>   	: options_(options), cameraIndex_(cameraIndex), last_(0),
> -	  queueCount_(0), captureCount_(0), captureLimit_(0),
> -	  printMetadata_(false)
> +	  sequence_(0), queueCount_(0), captureCount_(0),
> +	  captureLimit_(0), printMetadata_(false)
>   {
>   	char *endptr;
>   	unsigned long index = strtoul(cameraId.c_str(), &endptr, 10);
> @@ -375,6 +375,11 @@ void CameraSession::processRequest(Request *request)
>   	     << std::setw(6) << std::setfill('0') << ts / 1000 % 1000000
>   	     << " (" << std::fixed << std::setprecision(2) << fps << " fps)";
>   
> +	uint64_t sequence = request->metadata().get(controls::SensorSequence);
> +	// Report capture sequence number


no checkstyle complain here?

Reviewed-by: Umang Jain <umang.jain at ideasonboard.com>

> +	info << " [" << std::setw(6) << std::setfill('0')
> +	     << sequence << "]";
> +
>   	for (auto it = buffers.begin(); it != buffers.end(); ++it) {
>   		const Stream *stream = it->first;
>   		FrameBuffer *buffer = it->second;
> @@ -398,6 +403,14 @@ void CameraSession::processRequest(Request *request)
>   			requeue = false;
>   	}
>   
> +	/* Handle basic frame drop detection and reporting. */
> +	if (sequence_ == 0)
> +		sequence_ = sequence - 1;
> +	unsigned int drops = sequence - sequence_ - 1;
> +	if (drops)
> +		info << " *" << drops << " frame drops detected* ";
> +	sequence_ = sequence;
> +
>   	std::cout << info.str() << std::endl;
>   
>   	if (printMetadata_) {
> diff --git a/src/cam/camera_session.h b/src/cam/camera_session.h
> index bf966bd15ab0..14dc1421da4d 100644
> --- a/src/cam/camera_session.h
> +++ b/src/cam/camera_session.h
> @@ -65,6 +65,7 @@ private:
>   	unsigned int cameraIndex_;
>   
>   	uint64_t last_;
> +	uint64_t sequence_;
>   
>   	unsigned int queueCount_;
>   	unsigned int captureCount_;