[PATCH v3 08/16] libcamera: software_isp: Add DebayerCpu class

[Milan Zamazal]

Hans de Goede <hdegoede at redhat.com> writes:

> On 2/15/24 16:33, Milan Zamazal wrote:
>> Hans de Goede <hdegoede at redhat.com> writes:
>> 
>>> +SizeRange DebayerCpu::sizes(PixelFormat inputFormat, const Size &inputSize)
>>> +{
>>> +	Size pattern_size = patternSize(inputFormat);
>>> +	unsigned int border_height = pattern_size.height;
>>> +
>>> +	if (pattern_size.isNull())
>>> +		return {};
>>> +
>>> +	/* No need for top/bottom border with a pattern height of 2 */
>>> +	if (pattern_size.height == 2)
>>> +		border_height = 0;
>>> +
>>> +	/*
>>> +	 * For debayer interpolation a border is kept around the entire image
>>> +	 * and the minimum output size is pattern-height x pattern-width.
>>> +	 */
>> 
>> What if the output size is larger?  The border is quite impractical because it
>> forces (or not?) the output size to be non-standard, assuming the camera
>> provides common resolutions.  Consider e.g. full-HD camera resolution not being
>> able to be output 1:1 to a full-HD display.
>
> Hardware ISPs also need a similar border, because including special hardware
> to deal with the edges would be quite expensive both in silicon area as
> well as in power consumption. So sensors typically have a slightly bigger
> resolution then the standard resolutions. E.g. the ov2680 sensor has a
> pixelarray of 1616x1216 pixels and the ov2740 used in ThinkPads has
> 1932x1092 pixels.

And can those extra pixels be obtained for softisp?  I guess so as my camera can
provide its full 3280x2464 resolution, which is a bit more than QUXGA resolution
(3200x2400).  But when playing with different resolutions in the current branch,
I get confused.

When I request 1920x1080 resolution, 1920x1080 hardware resolution is selected
and I get 1916x1080 output resolution, with distorted image due to y field of
view being non-proportionally larger then x field of view (the image looks
stretched horizontally or squeezed vertically).

When I request 1924x1080, 3280x2464 hardware resolution is selected, the output
resolution is 1924x1080, the frame rate is a bit lower and the image looks all
right.

It looks like in the 1920x1080 case, this is what the hardware provides and the
softisp has to cut off 4 horizontal pixels for the borders.  As for the
distortion, it looks like the image gets cropped from 3280 to 1924 horizontally
while cropped from 2464 to something like 2160 and then compressed, rather than
cropped, to 1080 vertically (a bug in a driver?).

In the 1924x1080 case, it looks like there are plenty of extra pixels, the
(right) cropping happens elsewhere and there is no need to cut off anything in
softisp from the requested resolution.

I'm not sure my understanding is correct but anyway, I apparently cannot get
1920x1080 output resolution (and probably other standard resolutions if
available directly from v4l2).  At least in theory, it should be possible to use
3280x2464 for that but for the price of reduced performance.  Well, I think we
can live with this glitch in this version, but I'm still curious what's going on
and whether things could be reasonably improved in future.

Regards,
Milan