[libcamera-devel] [RFC PATCH v3 1/3] pipeline: ipu3: Store requests in the case a buffer shortage

[Laurent Pinchart]

Hi Hiro,

Thank you for the patch.

On Thu, Apr 08, 2021 at 05:50:59PM +0900, Hirokazu Honda wrote:
> PipelineHandlerIPU3 returns -ENOBUFS and -ENOMEM on queueing a
> request when there are not sufficient buffers for the request.
> Since the request will be successful if it is queued later when
> enough buffers are available. The requests failed due to a buffer
> shortage should be stored and retried later in the FIFO order.
> This introduces the queue in IPU3CameraData to do that.
> 
> Signed-off-by: Hirokazu Honda <hiroh at chromium.org>
> ---
>  src/libcamera/pipeline/ipu3/ipu3.cpp | 58 ++++++++++++++++++----------
>  1 file changed, 37 insertions(+), 21 deletions(-)
> 
> diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp
> index 519cad4f..c73e4f7c 100644
> --- a/src/libcamera/pipeline/ipu3/ipu3.cpp
> +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp
> @@ -66,6 +66,7 @@ public:
>  	void cio2BufferReady(FrameBuffer *buffer);
>  	void paramBufferReady(FrameBuffer *buffer);
>  	void statBufferReady(FrameBuffer *buffer);
> +	int queuePendingRequests();
>  
>  	CIO2Device cio2_;
>  	ImgUDevice *imgu_;
> @@ -84,6 +85,7 @@ public:
>  
>  	std::unique_ptr<ipa::ipu3::IPAProxyIPU3> ipa_;
>  
> +	std::queue<Request *> pendingRequests_;

A blank line would be nice here.

>  private:
>  	void queueFrameAction(unsigned int id,
>  			      const ipa::ipu3::IPU3Action &action);
> @@ -764,6 +766,8 @@ void PipelineHandlerIPU3::stop(Camera *camera)
>  	IPU3CameraData *data = cameraData(camera);
>  	int ret = 0;
>  
> +	data->pendingRequests_ = {};

This leaks any pending request, as it will never be completed. Patch 3/3
in this series fixes the issue, but there's a bisection breakage, which
would be nice to avoid. One option would be to squash the three patches
together.

> +
>  	data->ipa_->stop();
>  
>  	ret |= data->imgu_->stop();
> @@ -774,36 +778,48 @@ void PipelineHandlerIPU3::stop(Camera *camera)
>  	freeBuffers(camera);
>  }
>  
> -int PipelineHandlerIPU3::queueRequestDevice(Camera *camera, Request *request)
> +int IPU3CameraData::queuePendingRequests()
>  {
> -	IPU3CameraData *data = cameraData(camera);
> +	while (!pendingRequests_.empty()) {
> +		Request *request = pendingRequests_.front();
>  
> -	IPU3Frames::Info *info = data->frameInfos_.create(request);
> -	if (!info)
> -		return -ENOBUFS;
> +		IPU3Frames::Info *info = frameInfos_.create(request);
> +		if (!info)
> +			break;
>  
> -	/*
> -	 * Queue a buffer on the CIO2, using the raw stream buffer provided in
> -	 * the request, if any, or a CIO2 internal buffer otherwise.
> -	 */
> -	FrameBuffer *reqRawBuffer = request->findBuffer(&data->rawStream_);
> -	FrameBuffer *rawBuffer = data->cio2_.queueBuffer(request, reqRawBuffer);
> -	if (!rawBuffer) {
> -		data->frameInfos_.remove(info);
> -		return -ENOMEM;
> -	}
> +		/*
> +		 * Queue a buffer on the CIO2, using the raw stream buffer
> +		 * provided in the request, if any, or a CIO2 internal buffer
> +		 * otherwise.
> +		 */
> +		FrameBuffer *reqRawBuffer = request->findBuffer(&rawStream_);
> +		FrameBuffer *rawBuffer = cio2_.queueBuffer(request, reqRawBuffer);
> +		if (!rawBuffer) {
> +			frameInfos_.remove(info);
> +			break;
> +		}
>  
> -	info->rawBuffer = rawBuffer;
> +		info->rawBuffer = rawBuffer;
>  
> -	ipa::ipu3::IPU3Event ev;
> -	ev.op = ipa::ipu3::EventProcessControls;
> -	ev.frame = info->id;
> -	ev.controls = request->controls();
> -	data->ipa_->processEvent(ev);
> +		ipa::ipu3::IPU3Event ev;
> +		ev.op = ipa::ipu3::EventProcessControls;
> +		ev.frame = info->id;
> +		ev.controls = request->controls();
> +		ipa_->processEvent(ev);

I was going to say that the event should likely be sent to the IPA right
away, to make sure that the IPA get's enough data aobut future frames to
make the best decision, but it will likely not need to peek ahead in the
queue for more requests than we have internal buffers. It should thus be
fine, and we can address this later if it becomes a problem.

> +
> +		pendingRequests_.pop();
> +	}
>  
>  	return 0;

This function now always returns 0, let's make it a void function.

>  }
>  
> +int PipelineHandlerIPU3::queueRequestDevice(Camera *camera, Request *request)
> +{
> +	IPU3CameraData *data = cameraData(camera);
> +	data->pendingRequests_.emplace(request);

It's a pointer, you can call

	data->pendingRequests_.push(request);

> +	return data->queuePendingRequests();
> +}
> +
>  bool PipelineHandlerIPU3::match(DeviceEnumerator *enumerator)
>  {
>  	int ret;

-- 
Regards,

Laurent Pinchart