[libcamera-devel] using libcamerasrc gst element

[Laurent Pinchart]

Hi Martin,

(CC'ing Nicolas)

On Fri, Sep 03, 2021 at 09:28:10AM +0200, Martin Kepplinger wrote:
> Am Donnerstag, dem 02.09.2021 um 17:48 +0300 schrieb Laurent Pinchart:
> > On Thu, Sep 02, 2021 at 10:56:41AM +0200, Martin Kepplinger wrote:
> > > hi all,
> > > 
> > > I'm starting to use libcamera with the hi846 camera on imx8mq. (The
> > > following all works on my laptop btw, where there's only 1 camera.)
> > > 
> > > I test using the "libaperture" gstreamer app with the libcamera gst src element
> > > (https://gitlab.gnome.org/jwestman/libaperture/-/issues/14 ).
> > > But it doesn't work on imx8mq where I have 2 cameras (and 1 is working
> > > with libcamera currently, but that shouldn't matter I think).
> > > 
> > > Here's the end of the logs. earlier logs seem to be the same as the
> > > successful case with qcam, see below.
> > > 
> > > 
> > > [1:57:12.541751844] [3187] DEBUG Camera camera_manager.cpp:150
> > > Found registered pipeline handler 'PipelineHandlerUVC'
> > > [1:57:12.541817127] [3187] DEBUG Camera camera_manager.cpp:150
> > > Found registered pipeline handler 'PipelineHandlerVimc'
> > > [1:57:12.542646484] [3180] DEBUG Camera camera.cpp:878 streams
> > > configuration: (0) 1632x1224-SGBRG16
> > > [1:57:12.543146546] [3187] DEBUG V4L2 v4l2_videodevice.cpp:1423
> > > /dev/video0[32:cap]: Releasing buffers
> > > [1:57:12.543212069] [3187] DEBUG V4L2 v4l2_videodevice.cpp:1160
> > > /dev/video0[32:cap]: 0 buffers requested.
> 
> [snip]
> 
> > > what does libcamerasrc do wrong? Why isn't it configuring the stream?
> > > What config options do I have for the gstreamer element to play with?
> > 
> > Have you tried using libcamerasrc with a simple gst-launch pipeline
> > first ? That could give us additional feedback from GStreamer itself.
> 
> thanks. first, verified that the gstreamer test src works:
> 
> LANG=C LIBCAMERA_LOG_FILE=/home/purism/lc.log LIBCAMERA_LOG_LEVELS=0
> GST_PLUGIN_PATH=/usr/local/lib/aarch64-linux-gnu/gstreamer-1.0/
> gst-launch-1.0 videotestsrc ! glimagesink
> 
> one appearent problem: gstreamer and libcamerasrc don't know about the
> SGBRG16 format I'm receiving. Tried to workaround that like so (?):
> 
> --- a/src/gstreamer/gstlibcamera-utils.cpp
> +++ b/src/gstreamer/gstlibcamera-utils.cpp
> @@ -30,6 +30,8 @@ static struct {
>         { GST_VIDEO_FORMAT_YUY2, formats::YUYV },
>         { GST_VIDEO_FORMAT_YVYU, formats::YVYU },
>         /* \todo NV42 is used in libcamera but is not mapped in
> GStreamer yet. */
> +       { GST_VIDEO_FORMAT_GRAY16_BE, formats::SGBRG16 },

I think this should be GST_VIDEO_FORMAT_GRAY16_LE.

We should also add support for the other Bayer patterns, but it won't be
as trivial as adding the SBGGR, SGRBG and SRGGB entries here, as
gst_format_to_pixel_format() won't necessarily pick the right format in
that case.

This reminds me that I think we should decouple the CFA pattern from the
format in libcamera, but that's for later. Nicolas, do I understand
correctly that GStreamer uses x-bayer instead of x-raw for Bayer formats
? I can't see a bit depth being specified by x-bayer, does it mean it
supports 8bpp only ?

> +       /* 16bit bayer not supported by gstreamer yet (?) */
>  };
>  
> 
> but that results in:
> 
> purism at pureos:~$ GST_DEBUG=3 LANG=C
> LIBCAMERA_LOG_FILE=/home/purism/lc.log LIBCAMERA_LOG_LEVELS=0
> GST_PLUGIN_PATH=/usr/local/lib/aarch64-linux-gnu/gstreamer-1.0/
> gst-launch-1.0 libcamerasrc ! videoconvert ! glimagesink
> Setting pipeline to PAUSED ...
> Pipeline is live and does not need PREROLL ...
> 0:00:01.793370564  3386 0xaaaaddf97760 FIXME                default gstutils.c:4025:gst_pad_create_stream_id_internal:<libcamerasrc0:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
> Got context from element 'sink': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayWayland\)\gldisplaywayland0";
> Pipeline is PREROLLED ...
> Setting pipeline to PLAYING ...
> New clock: GstSystemClock
> 
> (gst-launch-1.0:3386): GStreamer-CRITICAL **: 09:20:50.114: gst_value_set_int_range_step: assertion 'start < end' failed
> 
> (gst-launch-1.0:3386): GStreamer-CRITICAL **: 09:20:50.116: gst_value_set_int_range_step: assertion 'start < end' failed
> ^Chandling interrupt.
> Interrupt: Stopping pipeline ...
> Execution ended after 0:00:05.128408250
> Setting pipeline to NULL ...
> Freeing pipeline ...
> 
> 
> the clock is *not* running. it just shows how long it ran after killing
> it.
> 
> not sure what my specific question is but in case you've seen this
> before and have an idea, just say so :)

I haven't tried to capture raw data using libcamerasrc yet, so I'm not
sure what to advise :-S

-- 
Regards,

Laurent Pinchart