[libcamera-devel] [PATCH 13/16] android: capabilties: Fix ANDROID_CONTROL_AE_AVAILABLE_TARGET_FPS_RANGES

[Jacopo Mondi]

Hi Paul,

On Wed, Sep 01, 2021 at 06:31:42PM +0900, paul.elder at ideasonboard.com wrote:
> Hi Jacopo,
>
> On Fri, Aug 27, 2021 at 02:07:54PM +0200, Jacopo Mondi wrote:
> > As reported by the CONTROL_AE_AVAILABLE_TARGET_FPS_RANGES documentation
> > in the Android developer reference:
> >
> > "For devices advertising any color filter arrangement other than NIR, or
> > devices not advertising color filter arrangement, this list will always
> > include (min, max) and (max, max) where min <= 15 and max = the maximum
> > output frame rate of the maximum YUV_420_888 output size."
> >
> > Collect the higher FPS of the larger YUV stream and use 15 FPS as the
> > minimum value, if the camera can go slower than that. Populate the
>
> If min is allowed to be less than 15, then why do you cap the minimum at
> 15?

Good question! I would say 'because the Intel HAL does so' but I know
it's a very fragile reasoning.

I'll try to use the max frame duration, after all our definition of
the FrameDurationLimits control matches

        The maximum frame duration provides the absolute limit to the shutter
        speed computed by the AE algorithm and it overrides any exposure mode
        setting specified with controls::AeExposureMode.

So I think it's fair to report it as the AE algorithm FPS limit to
Android.

-If- Android needs something different for reasons to be clarified,
we'll have to adjust it in the HAL I guess.

Thanks
   j

>
>
> Paul
>
> > ANDROID_CONTROL_AE_AVAILABLE_TARGET_FPS_RANGES static metadata with the
> > newly computed values.
> >
> > Signed-off-by: Jacopo Mondi <jacopo at jmondi.org>
> > ---
> >  src/android/camera_capabilities.cpp | 75 ++++++++++-------------------
> >  1 file changed, 26 insertions(+), 49 deletions(-)
> >
> > diff --git a/src/android/camera_capabilities.cpp b/src/android/camera_capabilities.cpp
> > index 8bde06e824ef..01cdbcf1395d 100644
> > --- a/src/android/camera_capabilities.cpp
> > +++ b/src/android/camera_capabilities.cpp
> > @@ -862,55 +862,6 @@ int CameraCapabilities::initializeStaticMetadata()
> >  	staticMetadata_->addEntry(ANDROID_CONTROL_AE_AVAILABLE_MODES,
> >  				  aeAvailableModes);
> >
> > -	/* Initialize the AE frame duration limits. */
> > -	int64_t minFrameDurationNsec = -1;
> > -	int64_t maxFrameDurationNsec = -1;
> > -	const auto frameDurationsInfo = controlsInfo.find(&controls::FrameDurationLimits);
> > -	minFrameDurationNsec = frameDurationsInfo->second.min().get<int64_t>() * 1000;
> > -	maxFrameDurationNsec = frameDurationsInfo->second.max().get<int64_t>() * 1000;
> > -
> > -	/*
> > -	 * Adjust the minimum frame duration to comply with Android
> > -	 * requirements. The camera service mandates all preview/record
> > -	 * streams to have a minimum frame duration < 33,366 milliseconds
> > -	 * (see MAX_PREVIEW_RECORD_DURATION_NS in the camera service
> > -	 * implementation).
> > -	 *
> > -	 * If we're close enough (+ 500 useconds) to that value, round
> > -	 * the minimum frame duration of the camera to an accepted
> > -	 * value.
> > -	 */
> > -	static constexpr int64_t MAX_PREVIEW_RECORD_DURATION_NS = 1e9 / 29.97;
> > -	if (minFrameDurationNsec > MAX_PREVIEW_RECORD_DURATION_NS &&
> > -	    minFrameDurationNsec < MAX_PREVIEW_RECORD_DURATION_NS + 500000)
> > -		minFrameDurationNsec = MAX_PREVIEW_RECORD_DURATION_NS - 1000;
> > -
> > -	/*
> > -	 * The AE routine frame rate limits are computed using the frame
> > -	 * duration limits, as libcamera clips the AE routine to the
> > -	 * frame durations.
> > -	 */
> > -	int32_t maxFps = std::round(1e9 / minFrameDurationNsec);
> > -	int32_t minFps = std::round(1e9 / maxFrameDurationNsec);
> > -	minFps = std::max(1, minFps);
> > -
> > -	/*
> > -	 * Force rounding errors so that we have the proper frame
> > -	 * durations for when we reuse these variables later
> > -	 */
> > -	minFrameDurationNsec = 1e9 / maxFps;
> > -	maxFrameDurationNsec = 1e9 / minFps;
> > -
> > -	/*
> > -	 * Register to the camera service {min, max} and {max, max}
> > -	 * intervals as requested by the metadata documentation.
> > -	 */
> > -	int32_t availableAeFpsTarget[] = {
> > -		minFps, maxFps, maxFps, maxFps
> > -	};
> > -	staticMetadata_->addEntry(ANDROID_CONTROL_AE_AVAILABLE_TARGET_FPS_RANGES,
> > -				  availableAeFpsTarget);
> > -
> >  	std::vector<int32_t> aeCompensationRange = {
> >  		0, 0,
> >  	};
> > @@ -1256,8 +1207,12 @@ int CameraCapabilities::initializeStaticMetadata()
> >
> >  	std::vector<uint32_t> availableStreamConfigurations;
> >  	std::vector<int64_t> minFrameDurations;
> > +	int maxYUVFps = 0;
> > +	Size maxYUVSize;
> > +
> >  	availableStreamConfigurations.reserve(streamConfigurations_.size() * 4);
> >  	minFrameDurations.reserve(streamConfigurations_.size() * 4);
> > +
> >  	for (const auto &entry : streamConfigurations_) {
> >  		/*
> >  		 * Filter out YUV streams not capable of running at 30 FPS.
> > @@ -1274,6 +1229,16 @@ int CameraCapabilities::initializeStaticMetadata()
> >  		if (entry.androidFormat != HAL_PIXEL_FORMAT_BLOB && fps < 30)
> >  			continue;
> >
> > +		/*
> > +		 * Collect the FPS of the maximum YUV output size to populate
> > +		 * AE_AVAILABLE_TARGET_FPS_RANGE
> > +		 */
> > +		if (entry.androidFormat == HAL_PIXEL_FORMAT_YCbCr_420_888 &&
> > +		    entry.resolution > maxYUVSize) {
> > +			maxYUVSize = entry.resolution;
> > +			maxYUVFps = fps;
> > +		}
> > +
> >  		/* Stream configuration map. */
> >  		availableStreamConfigurations.push_back(entry.androidFormat);
> >  		availableStreamConfigurations.push_back(entry.resolution.width);
> > @@ -1301,6 +1266,18 @@ int CameraCapabilities::initializeStaticMetadata()
> >  	staticMetadata_->addEntry(ANDROID_SCALER_AVAILABLE_MIN_FRAME_DURATIONS,
> >  				  minFrameDurations);
> >
> > +	/*
> > +	 * Register to the camera service {min, max} and {max, max} with
> > +	 * max being the larger YUV stream maximum frame rate and min being
> > +	 * 15 if the camera can do less than that.
> > +	 */
> > +	int32_t minFps = std::max(1e9 / maxFrameDuration_, 15.0);
> > +	int32_t availableAeFpsTarget[] = {
> > +		minFps, maxYUVFps, maxYUVFps, maxYUVFps,
> > +	};
> > +	staticMetadata_->addEntry(ANDROID_CONTROL_AE_AVAILABLE_TARGET_FPS_RANGES,
> > +				  availableAeFpsTarget);
> > +
> >  	std::vector<int64_t> availableStallDurations;
> >  	for (const auto &entry : streamConfigurations_) {
> >  		if (entry.androidFormat != HAL_PIXEL_FORMAT_BLOB)
> > --
> > 2.32.0
> >