[libcamera-devel] [RFC PATCH 3/3] ipu3: cio2: Customize sensor format selection logic
Umang Jain
umang.jain at ideasonboard.com
Fri Jul 30 10:28:28 CEST 2021
Adapt the sensor format selection logic such that it address
platform constraints on Soraka and Nautilus. This is best-effort
basis hence, new platforms can bring new constraints in future
and we will need to adapt accordingly.
Currently we prioritise sensor format resolutions which closest to the
FoV of desired output. However, Intel seems to prioritize the selection
based on the closest FoV with respect to sensor's maximum resolution.
For e.g. for a desired output of 1080p, Soraka will select 2112x1568
since it's a better FoV match to sensor's maximum resolution
(4224x3136). It will not match the provided resolution of 2112x1188,
even if that matches exactly to the desired ratio of 1080p.
On the other hand, on Nautilus, currently the sensor's maximum
resolution(4208x3118) is the only selection for any 16:9 desired
formats, whether it is 640x360 or 1080p or 3840x2160. That means
the sensor will run at full bandwidth even for ridiculously smaller
resolutions so, we probably don't want that either.
This patch how the sensor format resolution is selected:
- It prioritizes FoV with respect to sensor's maximum resolution size
- It shall never return sensor's maximum resolution provided if there
are other lower resolutions available.
/* DNI: Preliminary testing of getSensorFormat():
* cam -c1 -swidth=640,height=360,role=raw
* cam -c1 -swidth=1280,height=720,role=raw
* cam -c1 -swidth=1920,height=1080,role=raw
* cam -c1 -swidth=3840,height=2160,role=raw
*/
Signed-off-by: Umang Jain <umang.jain at ideasonboard.com>
---
Output of preliminary testing:
($) cam -c1 -swidth=640,height=360,role=raw
INFO IPU3 cio2.cpp:326 Desired Size: 640x360, Found SGRBG10_IPU3 with Size: 2104x1560
($) cam -c1 -swidth=1280,height=720,role=raw
INFO IPU3 cio2.cpp:326 Desired Size: 1280x720, Found SGRBG10_IPU3 with Size: 2104x1560
($) cam -c1 -swidth=1920,height=1080,role=raw
INFO IPU3 cio2.cpp:326 Desired Size: 1920x1080, Found SGRBG10_IPU3 with Size: 2104x1560
($) cam -c1 -swidth=3840,height=2160,role=raw
INFO IPU3 cio2.cpp:326 Desired Size: 3840x2160, Found SGRBG10_IPU3 with Size: 4208x3118
---
src/libcamera/pipeline/ipu3/cio2.cpp | 62 +++++++++++++++++-----------
1 file changed, 38 insertions(+), 24 deletions(-)
diff --git a/src/libcamera/pipeline/ipu3/cio2.cpp b/src/libcamera/pipeline/ipu3/cio2.cpp
index aef88afd..4fd57ef7 100644
--- a/src/libcamera/pipeline/ipu3/cio2.cpp
+++ b/src/libcamera/pipeline/ipu3/cio2.cpp
@@ -245,20 +245,16 @@ StreamConfiguration CIO2Device::generateConfiguration(Size size) const
*
* - The desired \a size shall fit in the sensor output size to avoid the need
* to up-scale.
- * - The sensor output size shall match the desired aspect ratio to avoid the
- * need to crop the field of view.
- * - The sensor output size shall be as small as possible to lower the required
- * bandwidth.
+ * - The sensor output size will be set to maximum resolution only when there
+ * are no other available lower sensor resolutions from any of \a mbusCodes.
+ * - The sensor output size shall have the closest FoV with respect
+ * to the sensor's maximum resolution.
* - The desired \a size shall be supported by one of the media bus code listed
* in \a mbusCodes.
*
* When multiple media bus codes can produce the same size, the code at the
* lowest position in \a mbusCodes is selected.
*
- * The use of this method is optional, as the above criteria may not match the
- * needs of all pipeline handlers. Pipeline handlers may implement custom
- * sensor format selection when needed.
- *
* The returned sensor output format is guaranteed to be acceptable by the
* setFormat() method without any modification.
*
@@ -268,14 +264,15 @@ StreamConfiguration CIO2Device::generateConfiguration(Size size) const
V4L2SubdeviceFormat CIO2Device::getSensorFormat(const std::vector<unsigned int> &mbusCodes,
const Size &size) const
{
- unsigned int desiredArea = size.width * size.height;
- unsigned int bestArea = UINT_MAX;
- float desiredRatio = static_cast<float>(size.width) / size.height;
- float bestRatio = FLT_MAX;
- const Size *bestSize = nullptr;
+ std::map<Size, float> selectedSizes;
+ Size bestSize;
+ float maxResRatio = FLT_MAX;
+ float bestRatioDiff = -1.0;
uint32_t bestCode = 0;
for (unsigned int code : mbusCodes) {
+ Size maxSensorResolution;
+
const auto formats = formats_.find(code);
if (formats == formats_.end())
continue;
@@ -286,33 +283,50 @@ V4L2SubdeviceFormat CIO2Device::getSensorFormat(const std::vector<unsigned int>
if (sz.width < size.width || sz.height < size.height)
continue;
+ if (sz > maxSensorResolution)
+ maxSensorResolution = sz;
+
float ratio = static_cast<float>(sz.width) / sz.height;
- float ratioDiff = fabsf(ratio - desiredRatio);
- unsigned int area = sz.width * sz.height;
- unsigned int areaDiff = area - desiredArea;
+ selectedSizes.emplace(sz, ratio);
+ }
- if (ratioDiff > bestRatio)
- continue;
+ if (!selectedSizes.empty()) {
+ maxResRatio = selectedSizes[maxSensorResolution];
- if (ratioDiff < bestRatio || areaDiff < bestArea) {
- bestRatio = ratioDiff;
- bestArea = areaDiff;
- bestSize = &sz;
+ selectedSizes.erase(maxSensorResolution);
+ if (selectedSizes.empty()) {
bestCode = code;
+ bestSize = maxSensorResolution;
+ } else { /* Find the best FoV to sensor's max resolution */
+ for (auto iter : selectedSizes) {
+ float ratioDiff = fabsf(maxResRatio - iter.second);
+
+ if (bestRatioDiff < 0 || (ratioDiff < bestRatioDiff)) {
+ bestRatioDiff = ratioDiff;
+ bestCode = code;
+ bestSize = iter.first;
+ }
+ }
}
}
+ selectedSizes.clear();
}
- if (!bestSize) {
+ if (bestSize.isNull()) {
LOG(IPU3, Debug) << "No supported format or size found";
return {};
}
V4L2SubdeviceFormat format{
.mbus_code = bestCode,
- .size = *bestSize,
+ .size = bestSize,
};
+ LOG(IPU3, Info)
+ << "Desired Size: " << size.toString()
+ << ", Found " << mbusCodesToPixelFormat.at(format.mbus_code).toString()
+ << " with Size: " << format.size.toString();
+
return format;
}
--
2.31.0
More information about the libcamera-devel
mailing list