<div dir="ltr"><div dir="ltr"><div>Hi Umang,</div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Sep 14, 2021 at 12:53 PM Umang Jain <<a href="mailto:umang.jain@ideasonboard.com">umang.jain@ideasonboard.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi Vedant,<br>
<br>
On 9/14/21 12:04 PM, Vedant Paranjape wrote:<br>
> This patch adds a test to test if multi stream using libcamera's<br>
> gstreamer element works.<br>
<br>
<br>
Can you please document how and when this test is run?<br>
<br>
Currently, my system (with one UVC and vimc) has been able to run <br>
single_stream_test but multi_stream_test is run:<br>
<br>
<br>
11/65 libcamera:gstreamer / single_stream_test OK 2.53s<br>
12/65 libcamera:gstreamer / multi_stream_test SKIP 0.07s<br>
<br>
What's your test setup where this tests passes. I assume I have to <br>
satisfy the following condition from the test, in order to run:<br>
<br>
if (camera->streams().size() > 1) {<br>
<br>
cameraFound = true;<br>
<br>
....<br>
<br>
}<br>
<br>
But I don't know how, looking at the patch. So can you explain (not just <br>
here but in the patch / commit message as well) ? It shall help others <br>
as well.<br></blockquote><div><br></div><div>Multistream will run on rpi and IPU pipelines only. Will add it to commit message.<br></div><div> <br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
><br>
> Signed-off-by: Vedant Paranjape <<a href="mailto:vedantparanjape160201@gmail.com" target="_blank">vedantparanjape160201@gmail.com</a>><br>
> Reviewed-by: Paul Elder <<a href="mailto:paul.elder@ideasonboard.com" target="_blank">paul.elder@ideasonboard.com</a>><br>
> Tested-by: Paul Elder <<a href="mailto:paul.elder@ideasonboard.com" target="_blank">paul.elder@ideasonboard.com</a>><br>
> ---<br>
> .../gstreamer/gstreamer_multi_stream_test.cpp | 138 ++++++++++++++++++<br>
> test/gstreamer/meson.build | 1 +<br>
> 2 files changed, 139 insertions(+)<br>
> create mode 100644 test/gstreamer/gstreamer_multi_stream_test.cpp<br>
<br>
<br>
Right off the bat, there is a checkstyle failure: Can you look into it, <br>
please?<br>
<br>
[uajain@fedora libcamera]$ ./utils/checkstyle.py HEAD<br>
-----------------------------------------------------------------------------------------------<br>
9d0efed89b03d57790db4eebefa909bf169ce789 test: gstreamer: Add a test for <br>
gstreamer multi stream<br>
-----------------------------------------------------------------------------------------------<br>
--- test/gstreamer/gstreamer_multi_stream_test.cpp<br>
+++ test/gstreamer/gstreamer_multi_stream_test.cpp<br>
@@ -104,8 +104,7 @@<br>
GstPad *queue0_sink_pad = gst_element_get_static_pad(queue0_, <br>
"sink");<br>
GstPad *queue1_sink_pad = gst_element_get_static_pad(queue1_, <br>
"sink");<br>
<br>
- if (gst_pad_link(src_pad, queue0_sink_pad) != GST_PAD_LINK_OK<br>
- || gst_pad_link(request_pad, queue1_sink_pad) != <br>
GST_PAD_LINK_OK) {<br>
+ if (gst_pad_link(src_pad, queue0_sink_pad) != GST_PAD_LINK_OK <br>
|| gst_pad_link(request_pad, queue1_sink_pad) != GST_PAD_LINK_OK) {<br>
if (queue0_sink_pad)<br>
gst_object_unref(queue0_sink_pad);<br>
if (queue1_sink_pad)<br>
---<br>
1 potential issue detected, please review<br></blockquote><div><br></div><div>Please see Paul's review !<br></div><div> <br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
><br>
> diff --git a/test/gstreamer/gstreamer_multi_stream_test.cpp b/test/gstreamer/gstreamer_multi_stream_test.cpp<br>
> new file mode 100644<br>
> index 000000000000..aba86a3f8e27<br>
> --- /dev/null<br>
> +++ b/test/gstreamer/gstreamer_multi_stream_test.cpp<br>
> @@ -0,0 +1,138 @@<br>
> +/* SPDX-License-Identifier: GPL-2.0-or-later */<br>
> +/*<br>
> + * Copyright (C) 2021, Vedant Paranjape<br>
> + *<br>
> + * gstreamer_multi_stream_test.cpp - GStreamer multi stream capture test<br>
> + */<br>
> +<br>
> +#include <iostream><br>
> +#include <unistd.h><br>
> +<br>
> +#include <libcamera/base/utils.h><br>
> +<br>
> +#include <libcamera/libcamera.h><br>
> +<br>
> +#include "libcamera/internal/source_paths.h"<br>
> +<br>
> +#include <gst/gst.h><br>
> +<br>
> +#include "gstreamer_test.h"<br>
> +#include "test.h"<br>
> +<br>
> +using namespace std;<br>
> +<br>
> +class GstreamerMultiStreamTest : public GstreamerTest, public Test<br>
> +{<br>
> +public:<br>
> + GstreamerMultiStreamTest()<br>
> + : GstreamerTest()<br>
> + {<br>
> + }<br>
> +<br>
> +protected:<br>
> + int init() override<br>
> + {<br>
> + if (status_ != TestPass)<br>
> + return status_;<br>
> +<br>
> + /* Check if platform support multistream output */<br>
> + libcamera::CameraManager cm;<br>
> + cm.start();<br>
> + bool cameraFound = false;<br>
> + for (auto &camera : cm.cameras()) {<br>
> + if (camera->streams().size() > 1) {<br>
> + cameraName = camera->id();<br>
> + cameraFound = true;<br>
> + cm.stop();<br>
> + break;<br>
> + }<br>
> + }<br>
> +<br>
> + if (!cameraFound) {<br>
> + cm.stop();<br>
> + return TestSkip;<br>
> + }<br>
> +<br>
> + g_autoptr(GstElement) convert0 = gst_element_factory_make("videoconvert", "convert0");<br>
> + g_autoptr(GstElement) convert1 = gst_element_factory_make("videoconvert", "convert1");<br>
> + g_autoptr(GstElement) sink0 = gst_element_factory_make("fakesink", "sink0");<br>
> + g_autoptr(GstElement) sink1 = gst_element_factory_make("fakesink", "sink1");<br>
> + g_autoptr(GstElement) queue0 = gst_element_factory_make("queue", "queue0");<br>
> + g_autoptr(GstElement) queue1 = gst_element_factory_make("queue", "queue1");<br>
> + g_object_ref_sink(convert0);<br>
> + g_object_ref_sink(convert1);<br>
> + g_object_ref_sink(sink0);<br>
> + g_object_ref_sink(sink1);<br>
> + g_object_ref_sink(queue0);<br>
> + g_object_ref_sink(queue1);<br>
<br>
<br>
What's the idea here of first taking ownership by locally scope <br>
g_autoptr() pointers and then setting to the private members below? Is <br>
it to validate/null-check them? I would directly set to private members <br>
and null-check them directly instead, and if it fails, so be it and <br>
return TestFail. Would you be tempted to take this route?<br></blockquote><div><br></div><div>I was doing it before, and it let to a lot of repetitive code and buggy error handling. Also, Laurent and Nicolas were kinda reserved against it.<br></div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
> +<br>
> + if (!convert0 || !convert1 || !sink0 || !sink1 || !queue0 || !queue1) {<br>
> + g_printerr("Not all elements could be created. %p.%p.%p.%p.%p.%p\n",<br>
> + convert0, convert1, sink0, sink1, queue0, queue1);<br>
> +<br>
> + return TestFail;<br>
> + }<br>
> +<br>
> + convert0_ = reinterpret_cast<GstElement *>(g_steal_pointer(&convert0));<br>
> + convert1_ = reinterpret_cast<GstElement *>(g_steal_pointer(&convert1));<br>
> + sink0_ = reinterpret_cast<GstElement *>(g_steal_pointer(&sink0));<br>
> + sink1_ = reinterpret_cast<GstElement *>(g_steal_pointer(&sink1));<br>
> + queue0_ = reinterpret_cast<GstElement *>(g_steal_pointer(&queue0));<br>
> + queue1_ = reinterpret_cast<GstElement *>(g_steal_pointer(&queue1));<br>
> +<br>
> + if (createPipeline() != TestPass)<br>
> + return TestFail;<br>
<br>
<br>
Should this `if` block be present before you set the elements above?<br></blockquote><div><br></div><div>This was a good catch, all those elements won't be unrefed if createPipeline fails. I will make the fix !</div><div> <br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
> +<br>
> + return TestPass;<br>
> + }<br>
> +<br>
> + int run() override<br>
> + {<br>
> + g_object_set(libcameraSrc_, "camera-name", cameraName.c_str(), NULL);<br>
> +<br>
> + /* Build the pipeline */<br>
> + gst_bin_add_many(GST_BIN(pipeline_), libcameraSrc_, queue0_, queue1_,<br>
> + convert0_, convert1_, sink0_, sink1_, NULL);<br>
> + if (gst_element_link_many(queue0_, convert0_, sink0_, NULL) != TRUE ||<br>
> + gst_element_link_many(queue1_, convert1_, sink1_, NULL) != TRUE) {<br>
> + g_printerr("Elements could not be linked.\n");<br>
> + return TestFail;<br>
> + }<br>
> +<br>
> + g_autoptr(GstPad) src_pad = gst_element_get_static_pad(libcameraSrc_, "src");<br>
> + g_autoptr(GstPad) request_pad = gst_element_get_request_pad(libcameraSrc_, "src_%u");<br>
> + GstPad *queue0_sink_pad = gst_element_get_static_pad(queue0_, "sink");<br>
> + GstPad *queue1_sink_pad = gst_element_get_static_pad(queue1_, "sink");<br>
<br>
<br>
Why can't you use g_autoptr() for queue0_sink_pad and queue1_sink_pad <br>
too? gst_element_get_static_pad() will return a [Full] transfer-able <br>
value, so it should be un-reffed after usage (like you do below). If you <br>
use g_autoptr() the manual _unref shall be taken care of by itself no? <br>
Am I missing something? It'll help cleanup error paths.<br></blockquote><div><br></div><div>The pad must be unrefed immediately after use, that is after the pads are linked, and if I use an autoptr it won't unref right after the if.</div><div>I had thought to put the code where pads are created and linked into an inner scope like this, but then it will be over kill. <br></div><div><br></div><div>func <br></div><div>{</div><div> { // code // }</div><div>}<br></div><div><br></div><div><div>I assume this is a nitpick and not a blocker. I don't see much improvement by doing so, It'd be a different scenario if I had, say 10 pads.</div><div><br></div></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
> +<br>
> + if (gst_pad_link(src_pad, queue0_sink_pad) != GST_PAD_LINK_OK<br>
> + || gst_pad_link(request_pad, queue1_sink_pad) != GST_PAD_LINK_OK) {<br>
> + if (queue0_sink_pad)<br>
> + gst_object_unref(queue0_sink_pad);<br>
> + if (queue1_sink_pad)<br>
> + gst_object_unref(queue1_sink_pad);<br>
> + g_printerr("Pads could not be linked.\n");<br>
> + return TestFail;<br>
> + }<br>
> + gst_object_unref(queue0_sink_pad);<br>
> + gst_object_unref(queue1_sink_pad);<br>
> +<br>
> + if (startPipeline() != TestPass)<br>
> + return TestFail;<br>
> +<br>
> + if (processEvent() != TestPass)<br>
> + return TestFail;<br>
> +<br>
> + return TestPass;<br>
> + }<br>
> +<br>
> +private:<br>
> + std::string cameraName;<br>
> + GstElement *convert0_;<br>
> + GstElement *convert1_;<br>
> + GstElement *sink0_;<br>
> + GstElement *sink1_;<br>
> + GstElement *queue0_;<br>
> + GstElement *queue1_;<br>
> +};<br>
> +<br>
> +TEST_REGISTER(GstreamerMultiStreamTest)<br>
> diff --git a/test/gstreamer/meson.build b/test/gstreamer/meson.build<br>
> index aca53b920365..13652e87d05c 100644<br>
> --- a/test/gstreamer/meson.build<br>
> +++ b/test/gstreamer/meson.build<br>
> @@ -6,6 +6,7 @@ endif<br>
> <br>
> gstreamer_tests = [<br>
> ['single_stream_test', 'gstreamer_single_stream_test.cpp'],<br>
> + ['multi_stream_test', 'gstreamer_multi_stream_test.cpp'],<br>
> ]<br>
> gstreamer_dep = dependency('gstreamer-1.0', required: true)<br>
> <br></blockquote><div><br></div><div>Regards,</div><div>Vedant Paranjape<br></div></div></div>