<div dir="ltr"><div dir="ltr">Hi Laurent,<br></div>Thanks for the review.<br><div><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sun, Aug 29, 2021 at 6:39 AM Laurent Pinchart <<a href="mailto:laurent.pinchart@ideasonboard.com">laurent.pinchart@ideasonboard.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi Vedant,<br>
<br>
Thank you for the patch.<br>
<br>
On Sat, Aug 28, 2021 at 11:07:53PM +0530, Vedant Paranjape wrote:<br>
> This patch adds a test to test if multi stream using libcamera's<br>
> gstreamer element works.<br>
> <br>
> We need to work around one issue with ASan when enabled in the<br>
> build:<br>
> <br>
> - glib has a known leak at initialization time. This is covered by the<br>
> suppression file shipped with glib, but it's not clear how to use it<br>
> automatically. For now, disable leak detection to avoid test failures.<br>
> <br>
> Signed-off-by: Vedant Paranjape <<a href="mailto:vedantparanjape160201@gmail.com" target="_blank">vedantparanjape160201@gmail.com</a>><br>
> ---<br>
> .../gstreamer/gstreamer_multi_stream_test.cpp | 225 ++++++++++++++++++<br>
> test/gstreamer/meson.build | 1 +<br>
> 2 files changed, 226 insertions(+)<br>
> create mode 100644 test/gstreamer/gstreamer_multi_stream_test.cpp<br>
> <br>
> diff --git a/test/gstreamer/gstreamer_multi_stream_test.cpp b/test/gstreamer/gstreamer_multi_stream_test.cpp<br>
> new file mode 100644<br>
> index 00000000..e32b07a4<br>
> --- /dev/null<br>
> +++ b/test/gstreamer/gstreamer_multi_stream_test.cpp<br>
> @@ -0,0 +1,225 @@<br>
> +/* SPDX-License-Identifier: GPL-2.0-or-later */<br>
> +/*<br>
> + * Copyright (C) 2021, Vedant Paranjape<br>
> + *<br>
> + * gstreamer_single_stream_test.cpp - GStreamer single stream capture test<br>
> + */<br>
> +<br>
> +#include <iostream><br>
> +#include <unistd.h><br>
> +<br>
> +#include <libcamera/base/utils.h><br>
> +<br>
> +#include <libcamera/libcamera.h><br>
> +<br>
> +#include "libcamera/internal/source_paths.h"<br>
> +<br>
> +#include <gst/gst.h><br>
> +<br>
> +#include "test.h"<br>
> +<br>
> +using namespace std;<br>
> +<br>
> +extern "C" {<br>
> +const char *__asan_default_options()<br>
> +{<br>
> + /*<br>
> + * Disable leak detection due to a known global variable initialization<br>
> + * leak in glib's g_quark_init(). This should ideally be handled by<br>
> + * using a suppression file instead of disabling leak detection.<br>
> + */<br>
> + return "detect_leaks=false";<br>
> +}<br>
> +}<br>
> +<br>
> +class GstreamerSingleStreamTest : public Test<br>
> +{<br>
> +protected:<br>
> + int init() override<br>
> + {<br>
> + /*<br>
> + * GStreamer by default spawns a process to run the<br>
> + * gst-plugin-scanner helper. If libcamera is compiled with ASan<br>
> + * enabled, and as GStreamer is most likely not, this causes the<br>
> + * ASan link order check to fail when gst-plugin-scanner<br>
> + * dlopen()s the plugin as many libraries will have already been<br>
> + * loaded by then. Fix this issue by disabling spawning of a<br>
> + * child helper process when scanning the build directory for<br>
> + * plugins.<br>
> + */<br>
> + gst_registry_fork_set_enabled(false);<br>
> +<br>
> + /* Initialize GStreamer */<br>
> + g_autoptr(GError) errInit = NULL;<br>
> + if (!gst_init_check(nullptr, nullptr, &errInit)) {<br>
> + g_printerr("Could not initialize GStreamer: %s\n",<br>
> + errInit ? errInit->message : "unknown error");<br>
> +<br>
> + return TestFail;<br>
> + }<br>
> +<br>
> + /*<br>
> + * Remove the system libcamera plugin, if any, and add the<br>
> + * plugin from the build directory.<br>
> + */<br>
> + GstRegistry *registry = gst_registry_get();<br>
> + GstPlugin *plugin = gst_registry_lookup(registry, "libgstlibcamera.so");<br>
> + if (plugin) {<br>
> + gst_registry_remove_plugin(registry, plugin);<br>
> + gst_object_unref(plugin);<br>
> + }<br>
> +<br>
> + std::string path = libcamera::utils::libcameraBuildPath()<br>
> + + "src/gstreamer";<br>
> + if (!gst_registry_scan_path(registry, path.c_str())) {<br>
> + g_printerr("Failed to add plugin to registry\n");<br>
> + gst_deinit();<br>
> + return TestFail;<br>
> + }<br>
<br>
There's quite a bit of duplicated code between this and the single<br>
stream test. Could you refactor the single stream test to create a base<br>
class that can be shared with this one ?<br></blockquote><div><br></div><div>Yes, I am going to do this, but can we go about it as follows, merge this patch with duplicated code, <br></div><div>then I think over it a bit and then submit new patch which refactors a lot of the duplicated code, <br></div><div>also I don't think going ahead with making a base class is a good idea as, there's much functional difference between two codes,</div><div>they rather have isolated code that repeats, but no so much as to roll it into a class.</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
> +<br>
> + /* Check if platform support multistream output */<br>
> + libcamera::CameraManager cm;<br>
> + cm.start();<br>
> + if (cm.cameras()[0]->streams().size() <= 1) {<br>
> + cm.stop();<br>
> + return TestSkip;<br>
> + }<br>
> + cm.stop();<br>
<br>
Is there a guarantee that libcamerasrc will pick the same camera ? It<br></blockquote><div> </div><div>Yes, it is guaranteed if we don't pass it a camera-name which we aren't see here: <a href="https://git.linuxtv.org/libcamera.git/tree/src/gstreamer/gstlibcamerasrc.cpp#n237">https://git.linuxtv.org/libcamera.git/tree/src/gstreamer/gstlibcamerasrc.cpp#n237</a> <br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">seems quite fragile to me. Isn't there a way to instead query the<br>
libcamerasrc element for the number of pads it supports ? </blockquote><div><br></div><div>I think this won't be possible unless we add code to the gstreamer-element, <a class="gmail_plusreply" id="plusReplyChip-0" href="mailto:nicolas@ndufresne.ca" tabindex="-1">@Nicolas Dufresne</a> can</div><div>you answer this please, I am not sure :)</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">More than<br>
that, as not all cameras support multiple streams, it would be nice if<br>
the test could iterate over all cameras to find one that does.<br></blockquote><div><br></div><div>This sounds like a nice idea, I could store the camera-name of the specific camera and set the camera-name property</div><div>of the libcamera element and start on it, but now I don't a multicamera setup.<br></div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
> +<br>
> + /* Create the elements */<br>
> + libcameraSrc_ = gst_element_factory_make("libcamerasrc", "libcamera");<br>
> + convert0_ = gst_element_factory_make("videoconvert", "convert0");<br>
> + convert1_ = gst_element_factory_make("videoconvert", "convert1");<br>
> + sink0_ = gst_element_factory_make("fakesink", "sink0");<br>
> + sink1_ = gst_element_factory_make("fakesink", "sink1");<br>
> + queue0_ = gst_element_factory_make("queue", "camera_queue0");<br>
> + queue1_ = gst_element_factory_make("queue", "camera_queue1");<br>
<br>
Looks like there's room to make the code more generic, posibly storing<br>
the elements in containers.<br></blockquote><div><br></div><div>Okay something like a vector to hold the elements and then a map to hold <element name, unique name>.</div><div>But this is a test code, is this fancy stuff really needed ? !!<br></div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
> +<br>
> + /* Create the empty pipeline_ */<br>
> + pipeline_ = gst_pipeline_new("test-pipeline");<br>
> +<br>
> + if (!pipeline_ || !convert0_ || !convert1_ || !sink0_ ||<br>
> + !sink1_ || !queue0_ || !queue1_ || !libcameraSrc_) {<br>
> + g_printerr("Not all elements could be created. %p.%p.%p.%p.%p.%p.%p.%p\n",<br>
> + pipeline_, convert0_, convert1_, sink0_,<br>
> + sink1_, queue0_, queue1_, libcameraSrc_);<br>
> + if (pipeline_)<br>
> + gst_object_unref(pipeline_);<br>
> + if (convert0_)<br>
> + gst_object_unref(convert0_);<br>
> + if (convert1_)<br>
> + gst_object_unref(convert1_);<br>
> + if (sink0_)<br>
> + gst_object_unref(sink0_);<br>
> + if (sink1_)<br>
> + gst_object_unref(sink1_);<br>
> + if (queue0_)<br>
> + gst_object_unref(queue0_);<br>
> + if (queue1_)<br>
> + gst_object_unref(queue1_);<br>
> + if (libcameraSrc_)<br>
> + gst_object_unref(libcameraSrc_);<br>
> + gst_deinit();<br>
> +<br>
> + return TestFail;<br>
> + }<br>
> +<br>
> + return TestPass;<br>
> + }<br>
> +<br>
> + void cleanup() override<br>
> + {<br>
> + gst_object_unref(pipeline_);<br>
> + gst_deinit();<br>
> + }<br>
> +<br>
> + int run() override<br>
> + {<br>
> + GstStateChangeReturn ret;<br>
> +<br>
> + /* Build the pipeline */<br>
> + gst_bin_add_many(GST_BIN(pipeline_), libcameraSrc_, queue0_, queue1_,<br>
> + convert0_, convert1_, sink0_, sink1_, NULL);<br>
> + if (gst_element_link_many(queue0_, convert0_, sink0_, NULL) != TRUE<br>
> + || gst_element_link_many(queue1_, convert1_, sink1_, NULL) != TRUE) {<br>
> + g_printerr("Elements could not be linked.\n");<br>
> + return TestFail;<br>
> + }<br>
> +<br>
> + g_autoptr(GstPad) src_pad = gst_element_get_static_pad(libcameraSrc_, "src");<br>
> + g_autoptr(GstPad) request_pad = gst_element_get_request_pad(libcameraSrc_, "src_%u");<br>
> + GstPad *queue0_sink_pad = gst_element_get_static_pad(queue0_, "sink");<br>
> + GstPad *queue1_sink_pad = gst_element_get_static_pad(queue1_, "sink");<br>
> +<br>
> + if (gst_pad_link(src_pad, queue0_sink_pad) != GST_PAD_LINK_OK<br>
> + || gst_pad_link(request_pad, queue1_sink_pad) != GST_PAD_LINK_OK) {<br>
> + if (queue0_sink_pad)<br>
> + gst_object_unref(queue0_sink_pad);<br>
> + if (queue1_sink_pad)<br>
> + gst_object_unref(queue1_sink_pad);<br>
> + g_printerr("Pads could not be linked.\n");<br>
> + return TestFail;<br>
> + }<br>
> + gst_object_unref(queue0_sink_pad);<br>
> + gst_object_unref(queue1_sink_pad);<br>
> +<br>
> + /* Start playing */<br>
> + ret = gst_element_set_state(pipeline_, GST_STATE_PLAYING);<br>
> + if (ret == GST_STATE_CHANGE_FAILURE) {<br>
> + g_printerr("Unable to set the pipeline to the playing state.\n");<br>
> + return TestFail;<br>
> + }<br>
> +<br>
> + /* Wait until error or EOS or timeout after 2 seconds */<br>
> + constexpr GstMessageType msgType =<br>
> + static_cast<GstMessageType>(GST_MESSAGE_ERROR | GST_MESSAGE_EOS);<br>
> + constexpr GstClockTime timeout = 2 * GST_SECOND;<br>
> +<br>
> + g_autoptr(GstBus) bus = gst_element_get_bus(pipeline_);<br>
> + g_autoptr(GstMessage) msg = gst_bus_timed_pop_filtered(bus, timeout, msgType);<br>
> +<br>
> + gst_element_set_state(pipeline_, GST_STATE_NULL);<br>
> +<br>
> + /* Parse error message */<br>
> + if (msg == NULL)<br>
> + return TestPass;<br>
> +<br>
> + switch (GST_MESSAGE_TYPE(msg)) {<br>
> + case GST_MESSAGE_ERROR:<br>
> + gstreamer_print_error(msg);<br>
> + break;<br>
> + case GST_MESSAGE_EOS:<br>
> + g_print("End-Of-Stream reached.\n");<br>
> + break;<br>
> + default:<br>
> + g_printerr("Unexpected message received.\n");<br>
> + break;<br>
> + }<br>
> +<br>
> + return TestFail;<br>
> + }<br>
> +<br>
> +private:<br>
> + void gstreamer_print_error(GstMessage *msg)<br>
> + {<br>
> + g_autoptr(GError) err = NULL;<br>
> + g_autofree gchar *debug_info = NULL;<br>
> +<br>
> + gst_message_parse_error(msg, &err, &debug_info);<br>
> + g_printerr("Error received from element %s: %s\n",<br>
> + GST_OBJECT_NAME(msg->src), err->message);<br>
> + g_printerr("Debugging information: %s\n",<br>
> + debug_info ? debug_info : "none");<br>
> + }<br>
> +<br>
> + GstElement *pipeline_;<br>
> + GstElement *libcameraSrc_;<br>
> + GstElement *queue0_;<br>
> + GstElement *queue1_;<br>
> + GstElement *convert0_;<br>
> + GstElement *convert1_;<br>
> + GstElement *sink0_;<br>
> + GstElement *sink1_;<br>
> +};<br>
> +<br>
> +TEST_REGISTER(GstreamerSingleStreamTest)<br>
> diff --git a/test/gstreamer/meson.build b/test/gstreamer/meson.build<br>
> index b99aa0da..e28225b2 100644<br>
> --- a/test/gstreamer/meson.build<br>
> +++ b/test/gstreamer/meson.build<br>
> @@ -6,6 +6,7 @@ endif<br>
> <br>
> gstreamer_tests = [<br>
> ['single_stream_test', 'gstreamer_single_stream_test.cpp'],<br>
> + ['multi_stream_test', 'gstreamer_multi_stream_test.cpp'],<br>
> ]<br>
> gstreamer_dep = dependency('gstreamer-1.0', required: true)<br>
> <br>
<br>
-- <br>
Regards,<br>
<br>
Laurent Pinchart<br></blockquote><div><br></div><div>Regards</div><div><i><b>Vedant Paranjape</b></i></div></div></div></div>