[libcamera-devel] raspberrypi pipeline gstreamer bug
Jacopo Mondi
jacopo at jmondi.org
Wed Jun 10 14:10:19 CEST 2020
Hello,
On Wed, Jun 10, 2020 at 12:25:27PM +0100, Naushir Patuck wrote:
> Hi,
>
> On Wed, 10 Jun 2020 at 12:22, Naushir Patuck <naush at raspberrypi.com> wrote:
> >
> > Hi Xishan and Kieran,
> >
> > On Wed, 10 Jun 2020 at 11:26, Kieran Bingham
> > <kieran.bingham at ideasonboard.com> wrote:
> > >
> > > Hi Xishan, (Adding Naush)
> > >
> > > On 10/06/2020 10:57, Xishan Sun wrote:
> > > >
> > > > Trying to use gstreamer plugin on raspberrypi platform.
> > > >
> > > >
> > > > For very simple test started with
> > > >
> > > > gst-launch-1.0 libcamerasrc ! fakesink
> > > >
> > > > will get error:
> > > >
> > > > Setting pipeline to PAUSED ...
> > > > [5:14:54.902707301] [6325] INFO Camera camera_manager.cpp:277 libcamera
> > > > v0.0.0+1423-78150899-dirty
> > > > Pipeline is live and does not need PREROLL ...
> > > > Setting pipeline to PLAYING ...
> > > > ERROR: from element
> > > > /GstPipeline:pipeline0/GstLibcameraSrc:libcamerasrc0: Internal data
> > > > stream error.
> > > > Additional debug info:
> > > > ../src/gstreamer/gstlibcamerasrc.cpp(461): gst_libcamera_src_task_enter
> > > > (): /GstPipeline:pipeline0/GstLibcameraSrc:libcamerasrc0:
> > > > streaming stopped, reason not-negotiated (-4)
> > > > Execution ended after 0:00:00.000989404
> > > > Setting pipeline to PAUSED ...
> > > > Setting pipeline to READY ...
> > > > Setting pipeline to NULL ...
> > > > Freeing pipeline ...
> > > >
> > > > debug tracking all the way down to
> > > > file src/gstreamer/gstlibcamerasrc.cpp line 377, the function
> > > >
> > > > g_autoptr(GstCaps) filter =
> > > > gst_libcamera_stream_formats_to_caps(stream_cfg.formats());
> > > >
> > > > will always return empty filter list, since stream_cfg.formats() return
> > > > empty.
> > > >
> > > > further down to the source:
> > > > src/libcamera/pipeline/raspberrypi/raspberrypi.cpp line 515, in the
> > > > function
> > > >
> > > > CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera
> > > > *camera, const StreamRoles &roles)
> > > >
> > > > there is no configuration for the output format sets. In order to
> > > > generate the format sets, I copied the code from uvc pipeline. add it
> > > > into line 522
> > > >
> > > > ....
> > > > if (roles.empty())
> > > > return config;
> > > >
> > > > std::map<V4L2PixelFormat, std::vector<SizeRange>> v4l2Formats =
> > > > data->isp_[Isp::Output0].dev()->formats();
> > > > std::map<PixelFormat, std::vector<SizeRange>> deviceFormats;
> > > > std::transform(v4l2Formats.begin(), v4l2Formats.end(),
> > > > std::inserter(deviceFormats, deviceFormats.begin()),
> > > > [&](const decltype(v4l2Formats)::value_type
> > > > &format) {
> > > > return decltype(deviceFormats)::value_type{
> > > > format.first.toPixelFormat(),
> > > > format.second
> > > > };
> > > > });
> > >
> > > I wish all that code was friendlier to read (from the original) ... But
> > > I think that's going to be cleaned up by Jacopo's ImageFormats anyway?
> > >
Depending how well is received the ImageFormat rework series :) (care
to review? :)
In this case the std::map<V4L2PixelFormat, std::vector<SizeRange>>
type could be replaced, and I feel we could help creating
StreamFormats from ImageFormats<V4L2PixelFormat> with an helper...
> > > > StreamFormats formats(deviceFormats);
> > > >
> > > > for (const StreamRole role : roles) {
> > > > //StreamConfiguration cfg{};
> > > > StreamConfiguration cfg(formats);
> > > > ....
> > > >
> > > > it can solve the issue but obviously it is not the best way to do it.
> > > > Could you please revise the code here and advice me a better solution?
> > >
> > > Actually it might be quite close to being the right solution,
> > >
> > > The raspberrypi StreamConfiguration isn't currently being populated with
> > > the 'capabilities' of the stream, - only a single option based on the
> > > role given, so applications have no way to know what they 'could' ask
> > > for other than the specific choice given by the requested role.
> > >
> >
> > Sorry, was not aware this was expected :)
> >
Not your fault, as you've noticed already, neither IPU3 nor RkISP1 populates
their StreamFormats. That was somehow not enforced, it might as well
be that StreamFormats has been introduced earlier and never made
mandatory. If this becomes problematic for other components, like the
gstreamer element, we should look into enforcing this in all pipeline
handlers.
> > >
> > > Preloading the StreamConfiguration with the formats supported by the ISP
> > > is probably the right thing to do indeed, as long as it only exposes
> > > things which are supported by the stream.
> > >
Yes, the above patch if I'm not mistaken does not make any such
distinction, associating in example, non-raw formats to raw streams.
Am I mistaken ?
> > > There may be cases (depending on capabilities of the hardware) where
> > > this list might have to be filtered depending on the number (or type) of
> > > streams requested, but at least initially - populating the
> > > StreamConfiguration with the output capabilities of the ISP should get
> > > things going again.
> > >
> > > Naush - Is this something you can look at please?
> >
> > I think for now the above change by Xishan would be suitable as-is.
> > Things do get a bit more restrictive when we are processing two
> > streams simultaneously, but the pipeline::validate() will account for
> > this and adjust the stream format appropriately.
> >
> > I will prepare and test a patch.
> >
> > Regards,
> > Naush
> >
>
> Sorry forgot to mention, it does not seem that rkisp1 and ipu3
> pipeline handlers do this either, so somebody may have to do something
> similar there as well.
I think so, and probably enforce that to make sure StreamFormats are
populated. There's some work in flux on ImageFormats, which might be a
good occasion to tidy up all of this up.
Thanks
j
>
> Naush
> _______________________________________________
> libcamera-devel mailing list
> libcamera-devel at lists.libcamera.org
> https://lists.libcamera.org/listinfo/libcamera-devel
More information about the libcamera-devel
mailing list