[libcamera-devel] [PATCH v2 3/6] android: camera_device: Compute frame durations

[Laurent Pinchart]

Hi Jacopo,

Thank you for the patch.

On Tue, Jan 26, 2021 at 06:30:05PM +0100, Jacopo Mondi wrote:
> Use the FrameDuration control reported by pipeline handlers to register
> the Auto-Exposure routine FPS range, the minimum stream frame durations
> and the sensor maximum frame duration.
> 
> Signed-off-by: Jacopo Mondi <jacopo at jmondi.org>
> ---
>  src/android/camera_device.cpp | 72 +++++++++++++++++++++++++----------
>  1 file changed, 52 insertions(+), 20 deletions(-)
> 
> diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp
> index 1a384460974c..e3d43bea4700 100644
> --- a/src/android/camera_device.cpp
> +++ b/src/android/camera_device.cpp
> @@ -9,6 +9,7 @@
>  #include "camera_ops.h"
>  #include "post_processor.h"
>  
> +#include <math.h>
>  #include <sys/mman.h>
>  #include <tuple>
>  #include <vector>
> @@ -682,10 +683,10 @@ std::tuple<uint32_t, uint32_t> CameraDevice::calculateStaticMetadataSize()
>  {
>  	/*
>  	 * \todo Keep this in sync with the actual number of entries.
> -	 * Currently: 53 entries, 782 bytes of static metadata
> +	 * Currently: 54 entries, 802 bytes of static metadata
>  	 */
> -	uint32_t numEntries = 53;
> -	uint32_t byteSize = 782;
> +	uint32_t numEntries = 54;
> +	uint32_t byteSize = 802;
>  
>  	/*
>  	 * Calculate space occupation in bytes for dynamically built metadata
> @@ -760,12 +761,34 @@ const camera_metadata_t *CameraDevice::getStaticMetadata()
>  				  aeAvailableModes.data(),
>  				  aeAvailableModes.size());
>  
> -	std::vector<int32_t> availableAeFpsTarget = {
> -		15, 30,
> -	};
> -	staticMetadata_->addEntry(ANDROID_CONTROL_AE_AVAILABLE_TARGET_FPS_RANGES,
> -				  availableAeFpsTarget.data(),
> -				  availableAeFpsTarget.size());
> +	int64_t minFrameDurationUsec = -1;
> +	int64_t maxFrameDurationUsec = -1;

As Android reports frame durations in nanoseconds, how about storing
them in nanoseconds in these two variables as well (and naming them
minFrameDurationNsec and maxFrameDurationNsec) ?

> +	const auto frameDurationsInfo = controlsInfo.find(&controls::FrameDurations);
> +	if (frameDurationsInfo != controlsInfo.end()) {
> +		minFrameDurationUsec = frameDurationsInfo->second.min().get<int64_t>();
> +		maxFrameDurationUsec = frameDurationsInfo->second.max().get<int64_t>();
> +
> +		/*
> +		 * The AE routine frame rate limits are computed using the frame
> +		 * duration limits, as libcamera clips the AE routine to the
> +		 * frame durations.

Do we ? :-) It's fine for now, we'll likely rework this later in any
case.

> +		 */
> +		int32_t minFps = static_cast<int32_t>(::round(1000000.0f /

I'd use std::round(), for the reason explained in the coding style
document. Do you need the cast, isn't it implicit ? Maybe 1e6 would be
more readable ? Up to you.

> +							      maxFrameDurationUsec));
> +		minFps = std::max(1, minFps);
> +		int maxFps = static_cast<int32_t>(::round(1000000.0f /
> +							  minFrameDurationUsec));

Same here.

Maybe int32_t, as minFps is an int32_t too ?

> +
> +		/*
> +		 * Register to the camera service {min, max} and {max, max}
> +		 * intervals as requested by the metadata documentation.
> +		 */

This may not be enough, there's a comment for camcorder profiles in the
documentation, but we can care about this later.

> +		int32_t availableAeFpsTarget[] = {
> +			minFps, maxFps, maxFps, maxFps
> +		};
> +		staticMetadata_->addEntry(ANDROID_CONTROL_AE_AVAILABLE_TARGET_FPS_RANGES,
> +					  availableAeFpsTarget, 4);
> +	}
>  
>  	std::vector<int32_t> aeCompensationRange = {
>  		0, 0,
> @@ -929,6 +952,12 @@ const camera_metadata_t *CameraDevice::getStaticMetadata()
>  	staticMetadata_->addEntry(ANDROID_SENSOR_INFO_TIMESTAMP_SOURCE,
>  				  &timestampSource, 1);
>  
> +	if (maxFrameDurationUsec > 0) {
> +		int64_t maxFrameDuration = maxFrameDurationUsec * 1000;

You'll be able to drop this variable if you turn maxFrameDurationUsec
into maxFrameDurationNsec.

> +		staticMetadata_->addEntry(ANDROID_SENSOR_INFO_MAX_FRAME_DURATION,
> +					  &maxFrameDuration, 1);
> +	}
> +
>  	/* Statistics static metadata. */
>  	uint8_t faceDetectMode = ANDROID_STATISTICS_FACE_DETECT_MODE_OFF;
>  	staticMetadata_->addEntry(ANDROID_STATISTICS_INFO_AVAILABLE_FACE_DETECT_MODES,
> @@ -1063,18 +1092,20 @@ const camera_metadata_t *CameraDevice::getStaticMetadata()
>  				  availableStallDurations.data(),
>  				  availableStallDurations.size());
>  
> -	/* \todo Collect the minimum frame duration from the camera. */
> -	std::vector<int64_t> minFrameDurations;
> -	minFrameDurations.reserve(streamConfigurations_.size() * 4);
> -	for (const auto &entry : streamConfigurations_) {
> -		minFrameDurations.push_back(entry.androidFormat);
> -		minFrameDurations.push_back(entry.resolution.width);
> -		minFrameDurations.push_back(entry.resolution.height);
> -		minFrameDurations.push_back(33333333);
> +	/* Use the minimum frame duration for all the YUV/RGB formats. */
> +	if (minFrameDurationUsec > 0) {
> +		std::vector<int64_t> minFrameDurations;
> +		minFrameDurations.reserve(streamConfigurations_.size() * 4);
> +		for (const auto &entry : streamConfigurations_) {
> +			minFrameDurations.push_back(entry.androidFormat);
> +			minFrameDurations.push_back(entry.resolution.width);
> +			minFrameDurations.push_back(entry.resolution.height);
> +			minFrameDurations.push_back(minFrameDurationUsec * 1000);
> +		}
> +		staticMetadata_->addEntry(ANDROID_SCALER_AVAILABLE_MIN_FRAME_DURATIONS,
> +					  minFrameDurations.data(),
> +					  minFrameDurations.size());
>  	}
> -	staticMetadata_->addEntry(ANDROID_SCALER_AVAILABLE_MIN_FRAME_DURATIONS,
> -				  minFrameDurations.data(),
> -				  minFrameDurations.size());
>  
>  	uint8_t croppingType = ANDROID_SCALER_CROPPING_TYPE_CENTER_ONLY;
>  	staticMetadata_->addEntry(ANDROID_SCALER_CROPPING_TYPE, &croppingType, 1);
> @@ -1156,6 +1187,7 @@ const camera_metadata_t *CameraDevice::getStaticMetadata()
>  		ANDROID_SENSOR_INFO_SENSITIVITY_RANGE,
>  		ANDROID_SENSOR_INFO_COLOR_FILTER_ARRANGEMENT,
>  		ANDROID_SENSOR_INFO_EXPOSURE_TIME_RANGE,
> +		ANDROID_SENSOR_INFO_MAX_FRAME_DURATION,
>  		ANDROID_SENSOR_ORIENTATION,
>  		ANDROID_SENSOR_AVAILABLE_TEST_PATTERN_MODES,
>  		ANDROID_SENSOR_INFO_PHYSICAL_SIZE,

-- 
Regards,

Laurent Pinchart