[libcamera-devel] raspberrypi pipeline gstreamer bug

[Naushir Patuck]

Hi Xishan and Kieran,

On Wed, 10 Jun 2020 at 11:26, Kieran Bingham
<kieran.bingham at ideasonboard.com> wrote:
>
> Hi Xishan, (Adding Naush)
>
> On 10/06/2020 10:57, Xishan Sun wrote:
> >
> > Trying to use gstreamer plugin on raspberrypi platform.
> >
> >
> > For very simple test started with
> >
> > gst-launch-1.0 libcamerasrc ! fakesink
> >
> > will get error:
> >
> > Setting pipeline to PAUSED ...
> > [5:14:54.902707301] [6325]  INFO Camera camera_manager.cpp:277 libcamera
> > v0.0.0+1423-78150899-dirty
> > Pipeline is live and does not need PREROLL ...
> > Setting pipeline to PLAYING ...
> > ERROR: from element
> > /GstPipeline:pipeline0/GstLibcameraSrc:libcamerasrc0: Internal data
> > stream error.
> > Additional debug info:
> > ../src/gstreamer/gstlibcamerasrc.cpp(461): gst_libcamera_src_task_enter
> > (): /GstPipeline:pipeline0/GstLibcameraSrc:libcamerasrc0:
> > streaming stopped, reason not-negotiated (-4)
> > Execution ended after 0:00:00.000989404
> > Setting pipeline to PAUSED ...
> > Setting pipeline to READY ...
> > Setting pipeline to NULL ...
> > Freeing pipeline ...
> >
> > debug tracking all the way down to
> > file src/gstreamer/gstlibcamerasrc.cpp line 377, the function
> >
> > g_autoptr(GstCaps) filter =
> > gst_libcamera_stream_formats_to_caps(stream_cfg.formats());
> >
> > will always return empty filter list, since stream_cfg.formats() return
> > empty.
> >
> > further down to the source:
> > src/libcamera/pipeline/raspberrypi/raspberrypi.cpp line 515, in the
> > function
> >
> > CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera
> > *camera, const StreamRoles &roles)
> >
> > there is no configuration for the output format sets. In order to
> > generate the format sets, I copied the code from uvc pipeline. add it
> > into line 522
> >
> > ....
> >         if (roles.empty())
> >                  return config;
> >
> >         std::map<V4L2PixelFormat, std::vector<SizeRange>> v4l2Formats =
> >                  data->isp_[Isp::Output0].dev()->formats();
> >         std::map<PixelFormat, std::vector<SizeRange>> deviceFormats;
> >         std::transform(v4l2Formats.begin(), v4l2Formats.end(),
> >                         std::inserter(deviceFormats, deviceFormats.begin()),
> >                         [&](const decltype(v4l2Formats)::value_type
> > &format) {
> >                                 return decltype(deviceFormats)::value_type{
> >                                         format.first.toPixelFormat(),
> >                                         format.second
> >                                 };
> >                          });
>
> I wish all that code was friendlier to read (from the original) ... But
> I think that's going to be cleaned up by Jacopo's ImageFormats anyway?
>
> >          StreamFormats formats(deviceFormats);
> >
> >          for (const StreamRole role : roles) {
> >                  //StreamConfiguration cfg{};
> >                  StreamConfiguration cfg(formats);
> > ....
> >
> > it can solve the issue but obviously it is not the best way to do it.
> > Could you please revise the code here and advice me a better solution?
>
> Actually it might be quite close to being the right solution,
>
> The raspberrypi StreamConfiguration isn't currently being populated with
> the 'capabilities' of the stream, - only a single option based on the
> role given, so applications have no way to know what they 'could' ask
> for other than the specific choice given by the requested role.
>

Sorry, was not aware this was expected :)

>
> Preloading the StreamConfiguration with the formats supported by the ISP
> is probably the right thing to do indeed, as long as it only exposes
> things which are supported by the stream.
>
> There may be cases (depending on capabilities of the hardware) where
> this list might have to be filtered depending on the number (or type) of
> streams requested, but at least initially - populating the
> StreamConfiguration with the output capabilities of the ISP should get
> things going again.
>
> Naush - Is this something you can look at please?

I think for now the above change by Xishan would be suitable as-is.
Things do get a bit more restrictive when we are processing two
streams simultaneously, but the pipeline::validate() will account for
this and adjust the stream format appropriately.

I will prepare and test a patch.

Regards,
Naush


>
> --
> Regards
>
> Kieran
>
>
> >
> > Thanks
> >
> > --
> > Xishan Sun
> >
> > _______________________________________________
> > libcamera-devel mailing list
> > libcamera-devel at lists.libcamera.org
> > https://lists.libcamera.org/listinfo/libcamera-devel
> >
>
> --
> Regards
> --
> Kieran