[libcamera-devel] [PATCH v2 1/1] pipeline: rkisp1: Fix sensor-ISP format mismatch

[Sebastian Fricke]

Make sure to use a sensor format resolution that is smaller or equal to
the maximum allowed resolution for the RkISP1. The maximum resolution
is defined within the `rkisp1-common.h` file as:
define RKISP1_ISP_MAX_WIDTH		4032
define RKISP1_ISP_MAX_HEIGHT		3024

Change the order of setting the formats, in order to first check if the
requested resolution exceeds the maximum and search for the next smaller
available sensor resolution if that is the case.
Fail if no viable sensor format was located.

This means that some camera sensors can never operate with their maximum
resolution, for example on my OV13850 camera sensor, there are two
possible resolutions: 4224x3136 & 2112x1568, the first of those two will
never be picked as it surpasses the maximum of the ISP.

Signed-off-by: Sebastian Fricke <sebastian.fricke.linux at gmail.com>
---
 src/libcamera/pipeline/rkisp1/rkisp1.cpp | 46 ++++++++++++++++++++----
 1 file changed, 40 insertions(+), 6 deletions(-)

diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp
index 1b1922a..3ef8acd 100644
--- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp
+++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp
@@ -677,22 +677,56 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c)
 		return ret;
 
 	/*
-	 * Configure the format on the sensor output and propagate it through
-	 * the pipeline.
+	 * Configure the format at the ISP input and pass it on through
+	 * the pipeline after checking that the maximum resolution allowed
+	 * for the ISP is not exceeded.
 	 */
 	V4L2SubdeviceFormat format = config->sensorFormat();
-	LOG(RkISP1, Debug) << "Configuring sensor with " << format.toString();
+	LOG(RkISP1, Debug) << "Configuring ISP with " << format.toString();
+	/*
+	 * format is changed in setFormat, keep the resolution for comparison
+	 */
+	Size originalFormatSize = format.size;
 
-	ret = sensor->setFormat(&format);
+	ret = isp_->setFormat(0, &format);
 	if (ret < 0)
 		return ret;
+	LOG(RkISP1, Debug) << "ISP configured with " << format.toString();
+
+	if (originalFormatSize != format.size) {
+		Size maxSize = Size(0, 0);
+		LOG(RkISP1, Info) << "Configured resolution is greater than "
+				     "the maximum resolution for the ISP, "
+				     "trying to re-configure to a smaller "
+				     "valid sensor format";
+
+		for (const Size &size : sensor->sizes()) {
+			if (size.width > format.size.width ||
+			    size.height > format.size.height)
+				continue;
+			maxSize = std::max(maxSize, size);
+		}
+		if (maxSize == Size(0, 0)) {
+			LOG(RkISP1, Error) << "No available sensor resolution"
+					      "that is smaller or equal to "
+					   << format.toString();
+			return -1;
+		}
+		format = sensor->getFormat(sensor->mbusCodes(), maxSize);
 
-	LOG(RkISP1, Debug) << "Sensor configured with " << format.toString();
+		ret = isp_->setFormat(0, &format);
+		if (ret < 0)
+			return ret;
+		LOG(RkISP1, Debug) << "ISP re-configured with "
+				   << format.toString();
+	}
 
-	ret = isp_->setFormat(0, &format);
+	ret = sensor->setFormat(&format);
 	if (ret < 0)
 		return ret;
 
+	LOG(RkISP1, Debug) << "Sensor configured with " << format.toString();
+
 	Rectangle rect(0, 0, format.size);
 	ret = isp_->setSelection(0, V4L2_SEL_TGT_CROP, &rect);
 	if (ret < 0)
-- 
2.20.1