[libcamera-devel] [PATCH 16/16] ipa: ipu3: Cap frame duration to 30 FPS

[Jacopo Mondi]

Hi Paul,

On Wed, Sep 01, 2021 at 05:33:57PM +0900, paul.elder at ideasonboard.com wrote:
> Hi Jacopo,
>
> On Fri, Aug 27, 2021 at 02:07:57PM +0200, Jacopo Mondi wrote:
> > Limit the IPU3 frame rate to 30 FPS.
> >
> > The reason to do is to bring the IPU3 IPA in par with the Intel
> > HAL implementation on IPU3 platform, where 30FPS is the frame rate used
> > to perform quality tuning in the closed-source IPA module and has been
> > validated as the most efficient rate for the power/performace budget.
> >
> > Compute the vertical blanking to maintain such frame rate and configure
> > the sensor with that.
> >
> > Signed-off-by: Jacopo Mondi <jacopo at jmondi.org>
> > ---
> >  src/ipa/ipu3/ipu3.cpp | 37 ++++++++++++++++++++++++++++++++-----
> >  1 file changed, 32 insertions(+), 5 deletions(-)
> >
> > diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp
> > index fc5f69ed5ddc..0e5d5e479e20 100644
> > --- a/src/ipa/ipu3/ipu3.cpp
> > +++ b/src/ipa/ipu3/ipu3.cpp
> > @@ -183,7 +183,7 @@ private:
> >  	IPACameraSensorInfo sensorInfo_;
> >
> >  	/* Camera sensor controls. */
> > -	uint32_t defVBlank_;
> > +	uint32_t vBlank_;
> >  	uint32_t exposure_;
> >  	uint32_t minExposure_;
> >  	uint32_t maxExposure_;
> > @@ -257,10 +257,39 @@ void IPAIPU3::updateControls(const IPACameraSensorInfo &sensorInfo,
> >  		frameDurations[i] = frameSize / (sensorInfo.pixelRate / 1000000U);
> >  	}
> >
> > +	/*
> > +	 * Cap minimum frame duration to 30FPS.
> > +	 *
> > +	 * 30 FPS has been validated in the closed source Intel 3A module as the
> > +	 * most opportune frame rate for quality tuning, and power
> > +	 * vs performances budget on Intel IPU3.
> > +	 *
> > +	 * Reduce the minimum achievable frame rate to 30 FPS and compute the
>
> Since it's a "frame rate" should this be the "maximum achievable frame
> rate"?

Indeed! Thanks for spotting, I'll fix!

>
> Reviewed-by: Paul Elder <paul.elder at ideasonboard.com>
>
> > +	 * vertical blanking to maintain that rate.
> > +	 */
> > +	int64_t *minFrameDuration = &frameDurations[0];
> > +	if (*minFrameDuration < 1e6 / 30.0)
> > +		*minFrameDuration = 1e6 / 30.0;
> > +
> >  	controls[&controls::FrameDurationLimits] = ControlInfo(frameDurations[0],
> >  							       frameDurations[1],
> >  							       frameDurations[2]);
> >
> > +	/*
> > +	 * Adjust the vertical blanking to obtain the desired frame duration.
> > +	 *
> > +	 * Assume a fixed line length as horizontal blanking is seldom
> > +	 * controllable.
> > +	 *
> > +	 * \todo Support making this overridable by the application through
> > +	 * controls::FrameDuration.
> > +	 *
> > +	 * \todo Clamp exposure to frame duration.
> > +	 */
> > +	vBlank_ = *minFrameDuration * (sensorInfo.pixelRate / 1000000U);
> > +	vBlank_ /= lineLength;
> > +	vBlank_ -= sensorInfo.outputSize.height;
> > +
> >  	*ipaControls = ControlInfoMap(std::move(controls), controls::controls);
> >  }
> >
> > @@ -399,8 +428,6 @@ int IPAIPU3::configure(const IPAConfigInfo &configInfo,
> >  	maxGain_ = itGain->second.max().get<int32_t>();
> >  	gain_ = minGain_;
> >
> > -	defVBlank_ = itVBlank->second.def().get<int32_t>();
> > -
> >  	/* Clean context at configuration */
> >  	context_ = {};
> >
> > @@ -511,8 +538,7 @@ void IPAIPU3::parseStatistics(unsigned int frame,
> >
> >  	setControls(frame);
> >
> > -	/* \todo Use VBlank value calculated from each frame exposure. */
> > -	int64_t frameDuration = sensorInfo_.lineLength * (defVBlank_ + sensorInfo_.outputSize.height) /
> > +	int64_t frameDuration = sensorInfo_.lineLength * (vBlank_ + sensorInfo_.outputSize.height) /
> >  				(sensorInfo_.pixelRate / 1e6);
> >  	ctrls.set(controls::FrameDuration, frameDuration);
> >
> > @@ -534,6 +560,7 @@ void IPAIPU3::setControls(unsigned int frame)
> >  	ControlList ctrls(ctrls_);
> >  	ctrls.set(V4L2_CID_EXPOSURE, static_cast<int32_t>(exposure_));
> >  	ctrls.set(V4L2_CID_ANALOGUE_GAIN, static_cast<int32_t>(gain_));
> > +	ctrls.set(V4L2_CID_VBLANK, static_cast<int32_t>(vBlank_));
> >  	op.controls = ctrls;
> >
> >  	queueFrameAction.emit(frame, op);
> > --
> > 2.32.0
> >