<div dir="auto">Hi Kieran,<div dir="auto">Sounds like a good idea. Once I figure how to use GstDeviceProvider, I'll try to do this.</div><div dir="auto"><br></div><div dir="auto">Regards,</div><div dir="auto">Vedant Paranjape</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, 16 Aug, 2021, 19:52 Kieran Bingham, <<a href="mailto:kieran.bingham@ideasonboard.com">kieran.bingham@ideasonboard.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On 16/08/2021 15:14, Vedant Paranjape wrote:<br>
> Hi Laurent,<br>
> Yes, what's the camera-name with which vimc camera can be selected ?<br>
> I could simply set the object property to use it. I think it would be<br>
> nice to have two separate tests, one which runs on real camera and other<br>
> one on vimc camera.<br>
<br>
<br>
Or runs on all available cameras?<br>
<br>
--<br>
Kieran<br>
<br>
<br>
> Regards,<br>
> /*Vedant Paranjape*/<br>
> <br>
> On Mon, Aug 16, 2021 at 7:29 PM Laurent Pinchart<br>
> <<a href="mailto:laurent.pinchart@ideasonboard.com" target="_blank" rel="noreferrer">laurent.pinchart@ideasonboard.com</a><br>
> <mailto:<a href="mailto:laurent.pinchart@ideasonboard.com" target="_blank" rel="noreferrer">laurent.pinchart@ideasonboard.com</a>>> wrote:<br>
> <br>
> Hi Vedant,<br>
> <br>
> On Mon, Aug 16, 2021 at 07:04:00PM +0530, Vedant Paranjape wrote:<br>
> > Hi Umang,<br>
> > Thanks for the Review. I feel the changes are appropriate. I will<br>
> send a<br>
> > patch with the changes.<br>
> <br>
> While at it, I'm wondering if it would make sense to select the vimc<br>
> camera explicitly from the libcamerasrc element instead of using<br>
> whatever camera happens to be the first. It would make tests more<br>
> reproducible.<br>
> <br>
> > On Mon, Aug 16, 2021 at 11:17 AM Umang Jain wrote:<br>
> ><br>
> > > Hi Vedant and Laurent,<br>
> > ><br>
> > > Thanks for the patch. Overall looks good to me,<br>
> > ><br>
> > > a few comments below for cleanup paths :)<br>
> > > On 8/14/21 5:16 AM, Laurent Pinchart wrote:<br>
> > ><br>
> > > From: Vedant Paranjape <<a href="mailto:vedantparanjape160201@gmail.com" target="_blank" rel="noreferrer">vedantparanjape160201@gmail.com</a><br>
> <mailto:<a href="mailto:vedantparanjape160201@gmail.com" target="_blank" rel="noreferrer">vedantparanjape160201@gmail.com</a>>><br>
> <<a href="mailto:vedantparanjape160201@gmail.com" target="_blank" rel="noreferrer">vedantparanjape160201@gmail.com</a><br>
> <mailto:<a href="mailto:vedantparanjape160201@gmail.com" target="_blank" rel="noreferrer">vedantparanjape160201@gmail.com</a>>><br>
> > ><br>
> > > This patch adds a test to test if single stream using libcamera's<br>
> > > gstreamer element works.<br>
> > ><br>
> > > We need to work around two distinct issues with ASan when<br>
> enabled in the<br>
> > > build:<br>
> > ><br>
> > > - glib has a known leak at initialization time. This is covered<br>
> by the<br>
> > > suppression file shipped with glib, but it's not clear how to<br>
> use it<br>
> > > automatically. For now, disable leak detection to avoid test<br>
> failures.<br>
> > ><br>
> > > - GStreamer spawns a child process to scan plugins. If GStreamer is<br>
> > > compiled without ASan (which is likely) but libcamera is,<br>
> dlopen()ing<br>
> > > the libcamera plugin will cause an ASan link order verification<br>
> > > failure. Disable the verification child processes to work<br>
> around the<br>
> > > problem. This requires gcc 8 or newer.<br>
> > ><br>
> > > Signed-off-by: Vedant Paranjape <<a href="mailto:vedantparanjape160201@gmail.com" target="_blank" rel="noreferrer">vedantparanjape160201@gmail.com</a><br>
> <mailto:<a href="mailto:vedantparanjape160201@gmail.com" target="_blank" rel="noreferrer">vedantparanjape160201@gmail.com</a>>><br>
> <<a href="mailto:vedantparanjape160201@gmail.com" target="_blank" rel="noreferrer">vedantparanjape160201@gmail.com</a><br>
> <mailto:<a href="mailto:vedantparanjape160201@gmail.com" target="_blank" rel="noreferrer">vedantparanjape160201@gmail.com</a>>><br>
> > > Reviewed-by: Paul Elder <<a href="mailto:paul.elder@ideasonboard.com" target="_blank" rel="noreferrer">paul.elder@ideasonboard.com</a><br>
> <mailto:<a href="mailto:paul.elder@ideasonboard.com" target="_blank" rel="noreferrer">paul.elder@ideasonboard.com</a>>> <<a href="mailto:paul.elder@ideasonboard.com" target="_blank" rel="noreferrer">paul.elder@ideasonboard.com</a><br>
> <mailto:<a href="mailto:paul.elder@ideasonboard.com" target="_blank" rel="noreferrer">paul.elder@ideasonboard.com</a>>><br>
> > > Reviewed-by: Kieran Bingham <<a href="mailto:kieran.bingham@ideasonboard.com" target="_blank" rel="noreferrer">kieran.bingham@ideasonboard.com</a><br>
> <mailto:<a href="mailto:kieran.bingham@ideasonboard.com" target="_blank" rel="noreferrer">kieran.bingham@ideasonboard.com</a>>><br>
> <<a href="mailto:kieran.bingham@ideasonboard.com" target="_blank" rel="noreferrer">kieran.bingham@ideasonboard.com</a><br>
> <mailto:<a href="mailto:kieran.bingham@ideasonboard.com" target="_blank" rel="noreferrer">kieran.bingham@ideasonboard.com</a>>><br>
> > > Tested-by: Kieran Bingham <kieran.bingham@@<a href="http://ideasonboard.com" rel="noreferrer noreferrer" target="_blank">ideasonboard.com</a><br>
> <<a href="http://ideasonboard.com" rel="noreferrer noreferrer" target="_blank">http://ideasonboard.com</a>>> <kieran.bingham@@<a href="http://ideasonboard.com" rel="noreferrer noreferrer" target="_blank">ideasonboard.com</a><br>
> <<a href="http://ideasonboard.com" rel="noreferrer noreferrer" target="_blank">http://ideasonboard.com</a>>><br>
> > > Reviewed-by: Laurent Pinchart <<a href="mailto:laurent.pinchart@ideasonboard.com" target="_blank" rel="noreferrer">laurent.pinchart@ideasonboard.com</a><br>
> <mailto:<a href="mailto:laurent.pinchart@ideasonboard.com" target="_blank" rel="noreferrer">laurent.pinchart@ideasonboard.com</a>>><br>
> <<a href="mailto:laurent.pinchart@ideasonboard.com" target="_blank" rel="noreferrer">laurent.pinchart@ideasonboard.com</a><br>
> <mailto:<a href="mailto:laurent.pinchart@ideasonboard.com" target="_blank" rel="noreferrer">laurent.pinchart@ideasonboard.com</a>>><br>
> > > Signed-off-by: Laurent Pinchart<br>
> <<a href="mailto:laurent.pinchart@ideasonboard.com" target="_blank" rel="noreferrer">laurent.pinchart@ideasonboard.com</a><br>
> <mailto:<a href="mailto:laurent.pinchart@ideasonboard.com" target="_blank" rel="noreferrer">laurent.pinchart@ideasonboard.com</a>>><br>
> <<a href="mailto:laurent.pinchart@ideasonboard.com" target="_blank" rel="noreferrer">laurent.pinchart@ideasonboard.com</a><br>
> <mailto:<a href="mailto:laurent.pinchart@ideasonboard.com" target="_blank" rel="noreferrer">laurent.pinchart@ideasonboard.com</a>>><br>
> > > ---<br>
> > > This version incorporates changes coming from my review of v10, and<br>
> > > fixes for ASan issues. ASan is a bit of a pain with GStreamer,<br>
> for two<br>
> > > independent reasons as explained in the commit message. I'd like<br>
> to find<br>
> > > a way to use leak suppression files to handle the glib<br>
> initialization<br>
> > > leak, but that's a big rabbit hole. The workaround for the<br>
> second issue<br>
> > > is acceptable in my opinion.<br>
> > ><br>
> > > The biggest trouble with these workarounds is that they don't<br>
> work with<br>
> > > gcc version older than 8. As the stable version of the most<br>
> common Linux<br>
> > > distributions ship gcc 8 or newer, skipping this test for gcc 7<br>
> (which<br>
> > > is the oldest version that libcamera supports) is also<br>
> acceptable in my<br>
> > > opinion.<br>
> > ><br>
> > > Changes since v10:<br>
> > ><br>
> > > - Disable ASan leak detection<br>
> > > - Disable ASan link order verification for child processes<br>
> > > - Include source_path.h<br>
> > > - Remove unneeded explicit std::string construction<br>
> > > - Declare variables at usage site<br>
> > > - Make constants constexpr<br>
> > > - Add a variable for the message type<br>
> > > - Blank space fixes<br>
> > > ---<br>
> > > .../gstreamer_single_stream_test.cpp | 188<br>
> ++++++++++++++++++<br>
> > > test/gstreamer/meson.build | 19 ++<br>
> > > test/meson.build | 1 +<br>
> > > 3 files changed, 208 insertions(+)<br>
> > > create mode 100644 test/gstreamer/gstreamer_single_stream_test.cpp<br>
> > > create mode 100644 test/gstreamer/meson.build<br>
> > ><br>
> > > diff --git a/test/gstreamer/gstreamer_single_stream_test.cpp<br>
> b/test/gstreamer/gstreamer_single_stream_test.cpp<br>
> > > new file mode 100644<br>
> > > index 000000000000..e26673b3471a<br>
> > > --- /dev/null<br>
> > > +++ b/test/gstreamer/gstreamer_single_stream_test.cpp<br>
> > > @@ -0,0 +1,188 @@<br>
> > > +/* SPDX-License-Identifier: GPL-2.0-or-later */<br>
> > > +/*<br>
> > > + * Copyright (C) 2021, Vedant Paranjape<br>
> > > + *<br>
> > > + * ipa_interface_test.cpp - Test the IPA interface<br>
> > > + */<br>
> > > +<br>
> > > +#include <iostream><br>
> > > +#include <unistd.h><br>
> > > +<br>
> > > +#include <libcamera/base/utils.h><br>
> > > +<br>
> > > +#include "libcamera/internal/source_paths.h"<br>
> > > +<br>
> > > +#include <gst/gst.h><br>
> > > +<br>
> > > +#include "test.h"<br>
> > > +<br>
> > > +using namespace std;<br>
> > > +<br>
> > > +extern "C" {<br>
> > > +const char *__asan_default_options()<br>
> > > +{<br>
> > > + /*<br>
> > > + * Disable leak detection due to a known global variable<br>
> initialization<br>
> > > + * leak in glib's g_quark_init(). This should ideally be<br>
> handled by<br>
> > > + * using a suppression file instead of disabling leak detection.<br>
> > > + */<br>
> > > + return "detect_leaks=false";<br>
> > > +}<br>
> > > +}<br>
> > > +<br>
> > > +class GstreamerSingleStreamTest : public Test<br>
> > > +{<br>
> > > +protected:<br>
> > > + int init() override<br>
> > > + {<br>
> > > + /*<br>
> > > + * GStreamer spawns a process to run the<br>
> gst-plugin-scanner<br>
> > > + * helper. If libcamera is compiled with ASan<br>
> enabled, and as<br>
> > > + * GStreamer is most likely not, this will cause the<br>
> ASan link<br>
> > > + * order check to fail when gst-plugin-scanner<br>
> dlopen()s the<br>
> > > + * plugin as many libraries will have already been<br>
> loaded by<br>
> > > + * then. Work around this issue by disabling the<br>
> link order<br>
> > > + * check. This will only affect child processes, as<br>
> ASan is<br>
> > > + * already loaded for this process by the time this<br>
> code is<br>
> > > + * executed, and should thus hopefully be safe.<br>
> > > + *<br>
> > > + * This option is not available in gcc older than 8,<br>
> the only<br>
> > > + * option in that case is to skip the test.<br>
> > > + */<br>
> > > +#if defined(__SANITIZE_ADDRESS__) && !defined(__clang__) &&<br>
> __GNUC__ < 8<br>
> > > + return TestSkip;<br>
> > > +#endif<br>
> > > + setenv("ASAN_OPTIONS", "verify_asan_link_order=0", 1);<br>
> > > +<br>
> > > + /* Initialize GStreamer */<br>
> > > + GError *errInit = nullptr;<br>
> > ><br>
> > > One can use:<br>
> > ><br>
> > > g_autoptr(GError) errInit = NULL;<br>
> > ><br>
> > > ...<br>
> > ><br>
> > > + if (!gst_init_check(nullptr, nullptr, &errInit)) {<br>
> > > + g_printerr("Could not initialize GStreamer:<br>
> %s\n",<br>
> > > + errInit ? errInit->message :<br>
> "unknown error");<br>
> > > + if (errInit)<br>
> > > + g_error_free(errInit);<br>
> > ><br>
> > > And remove manual free here.<br>
> > ><br>
> > > +<br>
> > > + return TestFail;<br>
> > > + }<br>
> > > +<br>
> > > + /*<br>
> > > + * Remove the system libcamera plugin, if any, and<br>
> add the<br>
> > > + * plugin from the build directory.<br>
> > > + */<br>
> > > + GstRegistry *registry = gst_registry_get();<br>
> > > + GstPlugin *plugin = gst_registry_lookup(registry,<br>
> "libgstlibcamera.so");<br>
> > > + if (plugin) {<br>
> > > + gst_registry_remove_plugin(registry, plugin);<br>
> > > + gst_object_unref(plugin);<br>
> > > + }<br>
> > > +<br>
> > > + std::string path =<br>
> libcamera::utils::libcameraBuildPath()<br>
> > > + + "src/gstreamer";<br>
> > > + if (!gst_registry_scan_path(registry, path.c_str())) {<br>
> > > + g_printerr("Failed to add plugin to<br>
> registry\n");<br>
> > > + gst_deinit();<br>
> > > + return TestFail;<br>
> > > + }<br>
> > > +<br>
> > > + /* Create the elements */<br>
> > > + libcameraSrc_ =<br>
> gst_element_factory_make("libcamerasrc", "libcamera");<br>
> > > + convert0_ = gst_element_factory_make("videoconvert",<br>
> "convert0");<br>
> > > + sink0_ = gst_element_factory_make("fakesink", "sink0");<br>
> > > +<br>
> > > + /* Create the empty pipeline_ */<br>
> > > + pipeline_ = gst_pipeline_new("test-pipeline");<br>
> > > +<br>
> > > + if (!pipeline_ || !convert0_ || !sink0_ ||<br>
> !libcameraSrc_) {<br>
> > > + g_printerr("Not all elements could be<br>
> created. %p.%p.%p.%p\n",<br>
> > > + pipeline_, convert0_, sink0_,<br>
> libcameraSrc_);<br>
> > > + if (pipeline_)<br>
> > > + gst_object_unref(pipeline_);<br>
> > > + if (convert0_)<br>
> > > + gst_object_unref(convert0_);<br>
> > > + if (sink0_)<br>
> > > + gst_object_unref(sink0_);<br>
> > > + if (libcameraSrc_)<br>
> > > + gst_object_unref(libcameraSrc_);<br>
> > > + gst_deinit();<br>
> > > +<br>
> > > + return TestFail;<br>
> > > + }<br>
> > > +<br>
> > > + return TestPass;<br>
> > > + }<br>
> > > +<br>
> > > + void cleanup() override<br>
> > > + {<br>
> > > + gst_object_unref(pipeline_);<br>
> > > + gst_deinit();<br>
> > > + }<br>
> > > +<br>
> > > + int run() override<br>
> > > + {<br>
> > > + GstStateChangeReturn ret;<br>
> > > +<br>
> > > + /* Build the pipeline */<br>
> > > + gst_bin_add_many(GST_BIN(pipeline_), libcameraSrc_,<br>
> convert0_, sink0_, NULL);<br>
> > > + if (gst_element_link_many(libcameraSrc_, convert0_,<br>
> sink0_, NULL) != TRUE) {<br>
> > > + g_printerr("Elements could not be linked.\n");<br>
> > > + return TestFail;<br>
> > > + }<br>
> > > +<br>
> > > + /* Start playing */<br>
> > > + ret = gst_element_set_state(pipeline_,<br>
> GST_STATE_PLAYING);<br>
> > > + if (ret == GST_STATE_CHANGE_FAILURE) {<br>
> > > + g_printerr("Unable to set the pipeline to<br>
> the playing state.\n");<br>
> > > + return TestFail;<br>
> > > + }<br>
> > > +<br>
> > > + /* Wait until error or EOS or timeout after 2 seconds */<br>
> > > + constexpr GstMessageType msgType =<br>
> > > + <br>
> static_cast<GstMessageType>(GST_MESSAGE_ERROR | GST_MESSAGE_EOS);<br>
> > > + constexpr GstClockTime timeout = 2000000000;<br>
> > > +<br>
> > > + g_autoptr(GstBus) bus = gst_element_get_bus(pipeline_);<br>
> > > + g_autoptr(GstMessage) msg =<br>
> gst_bus_timed_pop_filtered(bus, timeout, msgType);<br>
> > > +<br>
> > > + gst_element_set_state(pipeline_, GST_STATE_NULL);<br>
> > > +<br>
> > > + /* Parse error message */<br>
> > > + if (msg == NULL)<br>
> > > + return TestPass;<br>
> > > +<br>
> > > + switch (GST_MESSAGE_TYPE(msg)) {<br>
> > > + case GST_MESSAGE_ERROR:<br>
> > > + gstreamer_print_error(msg);<br>
> > > + break;<br>
> > > + case GST_MESSAGE_EOS:<br>
> > > + g_print("End-Of-Stream reached.\n");<br>
> > ><br>
> > > EOS doesn't sound like a error/Test-fail condition to me. If<br>
> there is a<br>
> > > possibility that EOS will be reached before timeout, what's<br>
> ensures(in the<br>
> > > first place) that the pipeline will be in playing state for<br>
> entire timeout,<br>
> > > before we request masked EOS/Error messages from the bus? What<br>
> am I missing?<br>
> ><br>
> > I am not a gstreamer expert, to the best of my knowledge, EOS<br>
> won't be ever<br>
> > sent by libcamerasrc, as it is the element which drives the<br>
> pipeline. I<br>
> > think this event can be dropped, I just want to confirm it with<br>
> Nicolas<br>
> > once he's back.<br>
> ><br>
> > > It is important to note that *only elements driving the pipeline<br>
> should<br>
> > ever send an EOS event*. If your element is chain-based, it is not<br>
> driving<br>
> > the pipeline. Chain-based elements should just return GST_FLOW_EOS<br>
> from<br>
> > their chain function at the end of the stream (or the configured<br>
> segment)<br>
> ><br>
> ><br>
> <a href="https://gstreamer.freedesktop.org/documentation/plugin-development/advanced/events.html?gi-language=c#end-of-stream-eos" rel="noreferrer noreferrer" target="_blank">https://gstreamer.freedesktop.org/documentation/plugin-development/advanced/events.html?gi-language=c#end-of-stream-eos</a><br>
> <<a href="https://gstreamer.freedesktop.org/documentation/plugin-development/advanced/events.html?gi-language=c#end-of-stream-eos" rel="noreferrer noreferrer" target="_blank">https://gstreamer.freedesktop.org/documentation/plugin-development/advanced/events.html?gi-language=c#end-of-stream-eos</a>><br>
> ><br>
> > + break;<br>
> > > + default:<br>
> > > + g_printerr("Unexpected message received.\n");<br>
> > > + break;<br>
> > > + }<br>
> > > +<br>
> > > + return TestFail;<br>
> > > + }<br>
> > > +<br>
> > > +private:<br>
> > > + void gstreamer_print_error(GstMessage *msg)<br>
> > > + {<br>
> > > + GError *err;<br>
> > ><br>
> > > This can use:<br>
> > ><br>
> > > g_autoptr(GError) err = NULL;<br>
> > ><br>
> > > + gchar *debug_info;<br>
> > ><br>
> > > and<br>
> > ><br>
> > > g_autofree char *debug_info = NULL;<br>
> > ><br>
> > > +<br>
> > > + gst_message_parse_error(msg, &err, &debug_info);<br>
> > > + g_printerr("Error received from element %s: %s\n",<br>
> > > + GST_OBJECT_NAME(msg->src), err->message);<br>
> > > + g_printerr("Debugging information: %s\n",<br>
> > > + debug_info ? debug_info : "none");<br>
> > > + g_clear_error(&err);<br>
> > > + g_free(debug_info);<br>
> > ><br>
> > > So, that will lead us to dropping manual free of err and debug info<br>
> > ><br>
> > ><br>
> > > Minor things, so:<br>
> > ><br>
> > > Reviewed-by: Umang Jain <<a href="mailto:umang.jain@ideasonboard.com" target="_blank" rel="noreferrer">umang.jain@ideasonboard.com</a><br>
> <mailto:<a href="mailto:umang.jain@ideasonboard.com" target="_blank" rel="noreferrer">umang.jain@ideasonboard.com</a>>><br>
> > ><br>
> > > + }<br>
> > > +<br>
> > > + GstElement *pipeline_;<br>
> > > + GstElement *libcameraSrc_;<br>
> > > + GstElement *convert0_;<br>
> > > + GstElement *sink0_;<br>
> > > +};<br>
> > > +<br>
> > > +TEST_REGISTER(GstreamerSingleStreamTest)<br>
> > > diff --git a/test/gstreamer/meson.build b/test/gstreamer/meson.build<br>
> > > new file mode 100644<br>
> > > index 000000000000..b99aa0da0ba3<br>
> > > --- /dev/null<br>
> > > +++ b/test/gstreamer/meson.build<br>
> > > @@ -0,0 +1,19 @@<br>
> > > +# SPDX-License-Identifier: CC0-1.0<br>
> > > +<br>
> > > +if not gst_enabled<br>
> > > + subdir_done()<br>
> > > +endif<br>
> > > +<br>
> > > +gstreamer_tests = [<br>
> > > + ['single_stream_test', 'gstreamer_single_stream_test.cpp'],<br>
> > > +]<br>
> > > +gstreamer_dep = dependency('gstreamer-1.0', required: true)<br>
> > > +<br>
> > > +foreach t : gstreamer_tests<br>
> > > + exe = executable(t[0], t[1],<br>
> > > + dependencies : [libcamera_private,<br>
> gstreamer_dep],<br>
> > > + link_with : test_libraries,<br>
> > > + include_directories : test_includes_internal)<br>
> > > +<br>
> > > + test(t[0], exe, suite : 'gstreamer', is_parallel : false)<br>
> > > +endforeach<br>
> > > diff --git a/test/meson.build b/test/meson.build<br>
> > > index 3bceb5df586f..d0466f17d7b6 100644<br>
> > > --- a/test/meson.build<br>
> > > +++ b/test/meson.build<br>
> > > @@ -11,6 +11,7 @@ subdir('libtest')<br>
> > ><br>
> > > subdir('camera')<br>
> > > subdir('controls')<br>
> > > +subdir('gstreamer')<br>
> > > subdir('ipa')<br>
> > > subdir('ipc')<br>
> > > subdir('log')<br>
> <br>
> -- <br>
> Regards,<br>
> <br>
> Laurent Pinchart<br>
> <br>
</blockquote></div>