[libcamera-devel] [RFC 3/6] [DNI] include: drm_fourcc: Add Bayer FourCC and modifiers
Jacopo Mondi
jacopo at jmondi.org
Wed Mar 25 11:44:35 CET 2020
HI Niklas,
On Mon, Mar 16, 2020 at 03:41:43AM +0100, Niklas Söderlund wrote:
> Add Bayer format and modifiers for patch submitted for upstream
> inclusion. The formats have not been accepted upstream yet so should not
> yet be merged i libcamera.
>
> The formats is however needed to demonstrate RAW capture on the IPU3
> pipeline.
>
> Signed-off-by: Niklas Söderlund <niklas.soderlund at ragnatech.se>
> ---
> include/linux/drm_fourcc.h | 94 ++++++++++++++++++++++++++++++++++++++
> 1 file changed, 94 insertions(+)
>
> diff --git a/include/linux/drm_fourcc.h b/include/linux/drm_fourcc.h
> index 4bb1bfe93d5f4390..69240dfe1df0df28 100644
> --- a/include/linux/drm_fourcc.h
> +++ b/include/linux/drm_fourcc.h
> @@ -288,6 +288,62 @@ extern "C" {
> /* Compressed formats */
> #define DRM_FORMAT_MJPEG fourcc_code('M', 'J', 'P', 'G') /* Motion-JPEG */
>
> +/*
> + * Bayer formats
> + *
> + * Bayer formats contain green, red and blue components, with alternating lines
> + * of red and green, and blue and green pixels in different orders. For each
"alternating lines of red and green samples and lines of blue and green samples"
I would leave "different orders" out
> + * block of 2x2 pixels there is one pixel with a red filter, two with a green
> + * filter, and one with a blue filter. The filters can be arranged in different
> + * patterns.
> + *
> + * For example, RGGB:
> + * row0: RGRGRGRG...
> + * row1: GBGBGBGB...
> + * row3: RGRGRGRG...
> + * row4: GBGBGBGB...
> + * ...
> + *
> + * Vendors have different methods to pack the sampling formats to increase data
s/sampling formats/pixels samples/
> + * density. For this reason the fourcc only describes pixel sample size and the
> + * filter pattern for each block of 2x2 pixels. A modifier is needed to
> + * describe the memory layout.
> + *
> + * In addition to vendor modifiers for memory layout DRM_FORMAT_MOD_LINEAR may
> + * be used to describe a layout where all samples are placed consecutively in
> + * memory. If the sample does not fit inside a single byte, the sample storage
> + * is extended to the minimum number of (little endian) bytes that can hold the
> + * sample and any unused most-significant bits are defined as padding.
> + *
> + * For example, SRGGB10:
> + * Each 10-bit sample is contained in 2 consecutive little endian bytes, where
> + * the 6 most-significant bits are unused.
Repeating the same question, is it necessary to specify the bit order ?
+ - - - - - - - - + - - - - - - - - +
| b0 b1 b2 b3 b4 b5 b6 b7 | b8 b9 b10 x x x x x |
+ - - - - - - - - + - - - - - - - - +
vs
+ - - - - - - - - + - - - - - - - - +
| b7 b6 b5 b4 b3 b2 b1 b0 | x x x b10 b9 b8 b7 b6 |
+ - - - - - - - - + - - - - - - - - +
vs
+ - - - - - - - - + - - - - - - - - +
| b0 b1 b2 b3 b4 b5 b6 b7 | b8 b9 b10 x x x x x |
+ - - - - - - - - + - - - - - - - - +
etc
> + */
> +
> +/* 8-bit Bayer formats */
nit: 8-bits ?
> +#define DRM_FORMAT_SRGGB8 fourcc_code('R', 'G', 'G', 'B')
> +#define DRM_FORMAT_SGRBG8 fourcc_code('G', 'R', 'B', 'G')
> +#define DRM_FORMAT_SGBRG8 fourcc_code('G', 'B', 'R', 'G')
> +#define DRM_FORMAT_SBGGR8 fourcc_code('B', 'A', '8', '1')
> +
> +/* 10-bit Bayer formats */
> +#define DRM_FORMAT_SRGGB10 fourcc_code('R', 'G', '1', '0')
> +#define DRM_FORMAT_SGRBG10 fourcc_code('B', 'A', '1', '0')
> +#define DRM_FORMAT_SGBRG10 fourcc_code('G', 'B', '1', '0')
> +#define DRM_FORMAT_SBGGR10 fourcc_code('B', 'G', '1', '0')
> +
> +/* 12-bit Bayer formats */
> +#define DRM_FORMAT_SRGGB12 fourcc_code('R', 'G', '1', '2')
> +#define DRM_FORMAT_SGRBG12 fourcc_code('B', 'A', '1', '2')
> +#define DRM_FORMAT_SGBRG12 fourcc_code('G', 'B', '1', '2')
> +#define DRM_FORMAT_SBGGR12 fourcc_code('B', 'G', '1', '2')
> +
> +/* 14-bit Bayer formats */
> +#define DRM_FORMAT_SRGGB14 fourcc_code('R', 'G', '1', '4')
> +#define DRM_FORMAT_SGRBG14 fourcc_code('B', 'A', '1', '4')
> +#define DRM_FORMAT_SGBRG14 fourcc_code('G', 'B', '1', '4')
> +#define DRM_FORMAT_SBGGR14 fourcc_code('B', 'G', '1', '4')
> +
> /*
> * Format Modifiers:
> *
> @@ -311,6 +367,7 @@ extern "C" {
> #define DRM_FORMAT_MOD_VENDOR_BROADCOM 0x07
> #define DRM_FORMAT_MOD_VENDOR_ARM 0x08
> #define DRM_FORMAT_MOD_VENDOR_ALLWINNER 0x09
> +#define DRM_FORMAT_MOD_VENDOR_MIPI 0x0a
>
> /* add more to the end as needed */
>
> @@ -412,6 +469,16 @@ extern "C" {
> #define I915_FORMAT_MOD_Y_TILED_CCS fourcc_mod_code(INTEL, 4)
> #define I915_FORMAT_MOD_Yf_TILED_CCS fourcc_mod_code(INTEL, 5)
>
> +/*
> + * IPU3 Bayer packing layout
> + *
> + * The IPU3 raw Bayer formats use a custom packing layout where there are no
> + * gaps between each 10-bit sample. It packs 25 pixels into 32 bytes leaving
> + * the 6 most significant bits in the last byte unused. The format is little
> + * endian.
Here I think it is necessary to provide a visual description of the
packing method as it's all but trivial how color component samples are
arranged, or at least refer the readed to the V4L2 documentation of the
V4L2_PIX_FMT_IPU3_SBGGR10 format.
If I got it right it looks like :
0 31
+ - - - - - - - - + - - - - - - - - + - - - - - - - - + - - - - - - - - +
| b0 b1 b2 b3 b4 b5 b6 b7 | g0 g1 g2 g3 g4 g5 b8 b9 | b0 b1 b2 b3 g6 g7 g8 g9 | g0 g1 b4 b5 b6 b7 b8 b9 |
+ - - - - - - - - + - - - - - - - - + - - - - - - - - + - - - - - - - - +
32 63
+ - - - - - - - - + - - - - - - - - + - - - - - - - - + - - - - - - - - +
| g2 g3 g4 g5 g6 g7 g8 g9 | b0 b1 b2 b3 b4 b5 b6 b7 | g0 g1 g2 g3 g4 g5 b8 b9 | b0 b1 b2 b3 g6 b7 g8 g9 |
+ - - - - - - - - + - - - - - - - - + - - - - - - - - + - - - - - - - - +
Which expressed in bytes looks like
| B0L[7:0] | G0L[7:2] B0H[1:0] | B1L[7:4] G0H[3:0] | G1L[7:5] B1H[4:0] |
| G1H[7:0] | B2L[7:0] | G2L[7:2] B2H[1:0] | B3L[7:4] B3H[4:0] G2H[3:0] |
Or, with a simpler visualization:
| B0L[7] | G0L[5] B0H[2] | B1L[3] G0H[4] | G1L[2] B1H[5] |
| G1H[7] | B2L[7] | G2L[5] B2H[2] | B3L[3] G2H[4] |
I would like this comment to be recorded and at least discussed with
upstream DRM if you don't agree it is necessary to provide a visual
representation.
> + */
> +#define IPU3_FORMAT_MOD_PACKED fourcc_mod_code(INTEL, 8)
> +
> /*
> * Tiled, NV12MT, grouped in 64 (pixels) x 32 (lines) -sized macroblocks
> *
> @@ -758,6 +825,33 @@ extern "C" {
> */
> #define DRM_FORMAT_MOD_ALLWINNER_TILED fourcc_mod_code(ALLWINNER, 1)
>
> +/* Mobile Industry Processor Interface (MIPI) modifiers */
> +
> +/*
> + * MIPI CSI-2 packing layout
> + *
> + * The CSI-2 RAW formats (for example Bayer) use a different packing layout
> + * depenindg on the sample size.
> + *
> + * - 10-bits per sample
> + * Every four consecutive samples are packed into 5 bytes. Each of the first 4
> + * bytes contain the 8 high order bits of the pixels, and the 5th byte
> + * contains the 2 least-significant bits of each pixel, in the same order.
> + *
> + * - 12-bits per sample
> + * Every two consecutive samples are packed into three bytes. Each of the
> + * first two bytes contain the 8 high order bits of the pixels, and the third
> + * byte contains the four least-significant bits of each pixel, in the same
> + * order.
> + *
> + * - 14-bits per sample
> + * Every four consecutive samples are packed into seven bytes. Each of the
> + * first four bytes contain the eight high order bits of the pixels, and the
> + * three following bytes contains the six least-significant bits of each
> + * pixel, in the same order.
This is clear enough without visualizations, and match what I can read
out of the CSI-2 specs, where anyway the visualization really helps.
If you feel like to, as an ascii-art addicted, I would provide
diagrams here as well, otherwise people have to turn to the CSI-2
specs.
Thanks
j
> + */
> +#define MIPI_FORMAT_MOD_CSI2_PACKED fourcc_mod_code(MIPI, 1)
> +
> #if defined(__cplusplus)
> }
> #endif
> --
> 2.25.1
>
> _______________________________________________
> libcamera-devel mailing list
> libcamera-devel at lists.libcamera.org
> https://lists.libcamera.org/listinfo/libcamera-devel
More information about the libcamera-devel
mailing list