[libcamera-devel] [PATCH 2/2] qcam: dng_writer: Add support for IPU3 Bayer formats

[Laurent Pinchart]

Hi Niklas,

Thank you for the patch.

On Sat, Jun 06, 2020 at 05:50:39PM +0200, Niklas Söderlund wrote:
> Add support for the Bayer formats produced on the IPU3. The format uses
> a memory layout that is hard to repack and keep the 10-bit sample size,
> therefore scale the samples to 16-bit when creating the scanlines.
> 
> Signed-off-by: Niklas Söderlund <niklas.soderlund at ragnatech.se>
> ---
>  src/qcam/dng_writer.cpp | 77 +++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 77 insertions(+)
> 
> diff --git a/src/qcam/dng_writer.cpp b/src/qcam/dng_writer.cpp
> index 9435eeec5438f158..8c128041c37f26fe 100644
> --- a/src/qcam/dng_writer.cpp
> +++ b/src/qcam/dng_writer.cpp
> @@ -83,6 +83,55 @@ void thumbScanlineSBGGRxxP(const FormatInfo &info, void *output,
>  	}
>  }
>  
> +void packScanlineIPU3(void *output, const void *input, unsigned int width)
> +{
> +	const uint8_t *in = static_cast<const uint8_t *>(input);
> +	uint16_t *out = static_cast<uint16_t *>(output);
> +
> +	/*
> +	 * Upsacle the 10-bit format to 16-bit as it's none trivial to pack it

s/Upsacle/Upscale/
s/none/not/

> +	 * it as 10-bits without gaps.

s/it //
s/10-bits/10-bit/

> +	 *
> +	 * \todo Improve packig to keep the 10-bit sample size.

s/packig/packing/

> +	 */

As much as I'd like to improve that, I agree it's a nasty one. The only
option I can think of is a two-pass operation, unpack + pack.

> +	for (unsigned int x = 0; x < width; x += 25) {

What happens if the width isn't a multiple of 25 pixels ? Will you read
beyond the end of the input buffer and/or write beyond the end of the
output buffer ?

> +		for (unsigned int i = 0; i < 6; i++) {
> +			*out++ = ((in[1] & 0x02) << 8 | in[0]) << 6;

s/0x02/0x03/ ?

> +			*out++ = ((in[2] & 0x0f) << 6 | in[1] >> 2) << 6;
> +			*out++ = ((in[3] & 0x3f) << 4 | in[2] >> 4) << 6;
> +			*out++ = ((in[4] & 0xff) << 2 | in[3] >> 6) << 6;

I would have written this

			*out++ = (in[1] & 0x03) << 14 | (in[0] & 0xff) << 6;
			*out++ = (in[2] & 0x0f) << 12 | (in[1] & 0xfc) << 4;
			*out++ = (in[3] & 0x3f) << 10 | (in[2] & 0xf0) << 2;
			*out++ = (in[4] & 0xff) <<  8 | (in[3] & 0xc0) << 0;

Up to you.

> +			in += 5;
> +		}
> +
> +		*out++ = ((in[1] & 0x02) << 8 | in[0]) << 6;

s/0x02/0x03/ here too. Same in thumbScanlineIPU3().

> +		in += 2;
> +	}
> +}
> +
> +void thumbScanlineIPU3(const FormatInfo &info, void *output,
> +		       const void *input, unsigned int width,
> +		       unsigned int stride)
> +{
> +	const uint8_t *in = static_cast<const uint8_t *>(input);
> +	uint8_t *out = static_cast<uint8_t *>(output);
> +
> +	/* Number of bytes corresponding to 25 pixels. */
> +	unsigned int skip = 32;
> +
> +	for (unsigned int x = 0; x < width; x++) {
> +		uint8_t val1 = ((in[1] & 0x02) << 8 | in[0]) << 6;
> +		uint8_t val2 = ((in[2] & 0x0f) << 6 | in[1] >> 2) << 6;
> +		uint8_t val3 = ((in[stride + 1] & 0x02) << 8 | in[stride + 0]) << 6;
> +		uint8_t val4 = ((in[stride + 2] & 0x0f) << 6 | in[stride + 1] >> 2) << 6;

Have you checked the contents of the thumbnail ? val[1-4] are all
uint8_t and you store 16-bit values in them.

> +		uint8_t value = (val1 + val2 + val3 + val4) >> 2;
> +
> +		*out++ = value;
> +		*out++ = value;
> +		*out++ = value;
> +		in += skip;
> +	}
> +}
> +
>  static const std::map<PixelFormat, FormatInfo> formatInfo = {
>  	{ PixelFormat(DRM_FORMAT_SBGGR10, MIPI_FORMAT_MOD_CSI2_PACKED), {
>  		.bitsPerSample = 10,
> @@ -140,6 +189,34 @@ static const std::map<PixelFormat, FormatInfo> formatInfo = {
>  		.thumbDownscaleFactor = 16,
>  		.thumbScanline = thumbScanlineSBGGRxxP,
>  	} },
> +	{ PixelFormat(DRM_FORMAT_SBGGR10, IPU3_FORMAT_MOD_PACKED), {
> +		.bitsPerSample = 16,
> +		.pattern = { CFAPatternBlue, CFAPatternGreen, CFAPatternGreen, CFAPatternRed },
> +		.packScanline = packScanlineIPU3,
> +		.thumbDownscaleFactor = 25,

Not a very nice downscaling factor :-( I think it would be possible for
thumbScanlineIPU3() to handle a downscaling of 16, and you've nearly
nerd-sniped me into doing so :-) Would it be difficult to implement if
we calculated, for each output pixel, the byte and bit offset of val1
and val2 ? It could be a simple arithmetical calculation.

> +		.thumbScanline = thumbScanlineIPU3,
> +	} },
> +	{ PixelFormat(DRM_FORMAT_SGBRG10, IPU3_FORMAT_MOD_PACKED), {
> +		.bitsPerSample = 16,
> +		.pattern = { CFAPatternGreen, CFAPatternBlue, CFAPatternRed, CFAPatternGreen },
> +		.packScanline = packScanlineIPU3,
> +		.thumbDownscaleFactor = 25,
> +		.thumbScanline = thumbScanlineIPU3,
> +	} },
> +	{ PixelFormat(DRM_FORMAT_SGRBG10, IPU3_FORMAT_MOD_PACKED), {
> +		.bitsPerSample = 16,
> +		.pattern = { CFAPatternGreen, CFAPatternRed, CFAPatternBlue, CFAPatternGreen },
> +		.packScanline = packScanlineIPU3,
> +		.thumbDownscaleFactor = 25,
> +		.thumbScanline = thumbScanlineIPU3,
> +	} },
> +	{ PixelFormat(DRM_FORMAT_SRGGB10, IPU3_FORMAT_MOD_PACKED), {
> +		.bitsPerSample = 16,
> +		.pattern = { CFAPatternRed, CFAPatternGreen, CFAPatternGreen, CFAPatternBlue },
> +		.packScanline = packScanlineIPU3,
> +		.thumbDownscaleFactor = 25,
> +		.thumbScanline = thumbScanlineIPU3,
> +	} },
>  };
>  
>  int DNGWriter::write(const char *filename, const Camera *camera,

-- 
Regards,

Laurent Pinchart