[libcamera-devel] JPEG support for libcamera

[Jacopo Mondi]

Hi Pavel,
   I discussed it with Rafael (see message id
<20220701075644.mcjbcwthzsofmssx at uno.localdomain>) and my take was
slightly different: the driver should be fixed to report MJPEG:

Let me re-paste it here, mostly to validate my understanding as I
might have got this very wrong.

-------------------------------------------------------------------------------

Libcamera only supports:
#define V4L2_PIX_FMT_MJPEG    v4l2_fourcc('M', 'J', 'P', 'G') /* Motion-JPEG   */

I admit I know nothing about JPEG related standards but looking at the
media drivers, it seems FMT_JPEG is used more often than MJPEG.

However libcamera only seems to support MJPEG. You might have noticed
libcamera uses the DRM pixelformat identifiers, and it seems that in
the drm_fourcc.h header we ship there's no JPEG format at all, but
only MJPEG. If my understanding is correct this makes sense, as the
MJPEG standard defines a video container format for streaming
JPEG-compressed frames while JFIF JPEG defines a file-encoding scheme
for exchanging single images. Being DRM about video output devices
only MJPEG applies there, so there's no "JIF/JFIF JPEG" format in
there.

Now, I would not be surprised if the multiple users of FMT_JPEG in
drivers/media/ are actually sending MJPEG but driver developers pick
FMT_JPEG because of some form of cargo-cult.

Laurent do you know more about this maybe ?

When it comes to libcamera, for the reasons explained above, as a
streaming format it seems only MJPEG would make sense, while JFIF JPEG
could rather be used as the container when saving still images,
something for which we use DNG in qcam, in example.

To remove the above error I would change your driver to report
FMT_MJPEG for the time being until the above doesn't get clarified.
-------------------------------------------------------------------------------

What do you think ?

On Mon, Jul 04, 2022 at 08:07:20PM +0200, Pavel Machek via libcamera-devel wrote:
> JPEG is very similar to MJPEG format (AFAICT, it may contain extra
> EXIF headers). Add support for it. Tested with PinePhone and command
> line cam utility.
>
> Signed-off-by: Pavel Machek <pavel at ucw.cz>
>
> diff --git a/include/linux/drm_fourcc.h b/include/linux/drm_fourcc.h
> index ea11dcb4..b30a705d 100644
> --- a/include/linux/drm_fourcc.h
> +++ b/include/linux/drm_fourcc.h
> @@ -352,6 +352,7 @@ extern "C" {
>
>  /* Compressed formats */
>  #define DRM_FORMAT_MJPEG	fourcc_code('M', 'J', 'P', 'G') /* Motion-JPEG */
> +#define DRM_FORMAT_JPEG	fourcc_code('J', 'P', 'E', 'G') /* JFIF JPEG */
>
>  /*
>   * Bayer formats
> diff --git a/src/cam/sdl_sink.cpp b/src/cam/sdl_sink.cpp
> index f8e3e95d..673bd642 100644
> --- a/src/cam/sdl_sink.cpp
> +++ b/src/cam/sdl_sink.cpp
> @@ -63,6 +63,7 @@ int SDLSink::configure(const libcamera::CameraConfiguration &config)
>
>  	switch (cfg.pixelFormat) {
>  #ifdef HAVE_SDL_IMAGE
> +	case libcamera::formats::JPEG:
>  	case libcamera::formats::MJPEG:
>  		texture_ = std::make_unique<SDLTextureMJPG>(rect_);
>  		break;
> diff --git a/src/gstreamer/gstlibcamera-utils.cpp b/src/gstreamer/gstlibcamera-utils.cpp
> index 3f242286..fd5689d9 100644
> --- a/src/gstreamer/gstlibcamera-utils.cpp
> +++ b/src/gstreamer/gstlibcamera-utils.cpp
> @@ -80,6 +80,7 @@ bare_structure_from_format(const PixelFormat &format)
>  					 gst_video_format_to_string(gst_format), nullptr);
>
>  	switch (format) {
> +	case formats::JPEG:
>  	case formats::MJPEG:
>  		return gst_structure_new_empty("image/jpeg");
>  	default:
> diff --git a/src/libcamera/formats.cpp b/src/libcamera/formats.cpp
> index 283ecb3d..c5e97198 100644
> --- a/src/libcamera/formats.cpp
> +++ b/src/libcamera/formats.cpp
> @@ -944,6 +944,19 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  	} },
>
>  	/* Compressed formats. */
> +	{ formats::JPEG, {
> +		.name = "JPEG",
> +		.format = formats::JPEG,
> +		.v4l2Formats = {
> +			.single = V4L2PixelFormat(V4L2_PIX_FMT_JPEG),
> +			.multi = V4L2PixelFormat(),
> +		},
> +		.bitsPerPixel = 0,
> +		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
> +		.packed = false,
> +		.pixelsPerGroup = 1,
> +		.planes = {{ { 1, 1 }, { 0, 0 }, { 0, 0 } }},
> +	} },
>  	{ formats::MJPEG, {
>  		.name = "MJPEG",
>  		.format = formats::MJPEG,
> diff --git a/src/libcamera/formats.yaml b/src/libcamera/formats.yaml
> index 7dda0132..6b931c34 100644
> --- a/src/libcamera/formats.yaml
> +++ b/src/libcamera/formats.yaml
> @@ -76,6 +76,8 @@ formats:
>    - YVU444:
>        fourcc: DRM_FORMAT_YVU444
>
> +  - JPEG:
> +      fourcc: DRM_FORMAT_JPEG
>    - MJPEG:
>        fourcc: DRM_FORMAT_MJPEG
>
> diff --git a/src/libcamera/v4l2_pixelformat.cpp b/src/libcamera/v4l2_pixelformat.cpp
> index 58fc4e9d..82dfbd1e 100644
> --- a/src/libcamera/v4l2_pixelformat.cpp
> +++ b/src/libcamera/v4l2_pixelformat.cpp
> @@ -183,6 +183,8 @@ const std::map<V4L2PixelFormat, V4L2PixelFormat::Info> vpf2pf{
>  	/* Compressed formats. */
>  	{ V4L2PixelFormat(V4L2_PIX_FMT_MJPEG),
>  		{ formats::MJPEG, "Motion-JPEG" } },
> +	{ V4L2PixelFormat(V4L2_PIX_FMT_JPEG),
> +		{ formats::JPEG, "JPEG" } },
>  };
>
>  } /* namespace */
>
> --
> People of Russia, stop Putin before his war on Ukraine escalates.


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://lists.libcamera.org/pipermail/libcamera-devel/attachments/20220705/793093a3/attachment.sig>