Gstreamer assertion failed after a short period when running two cameras simultaneously

[Nicolas Dufresne]

Hi,

Le vendredi 11 octobre 2024 à 03:10 +0000, 康 心奕 a écrit :
> Hello, everyone
> 
> This is a re-post of the issue at the Raspberry PI branch:
> https://github.com/raspberrypi/libcamera/issues/186?notification_referrer_id=NT_kwDOAH7EvLMxMjc0NDgwODE0NTo4MzA3OTAw
> . The admin redirected me here. I'd really appreciate it if anyone could point
> me some directions on how to resolve or further debug the issue. Thanks .
> SymptomI'm experiencing the following assertion fail error when running two
> RPi
> cameras simultaneously.
> void GstLibcameraSrcState::requestCompleted(libcamera::Request*): assertion
> 'wrap->request_.get() == request' failed 
This assertion is what needs to be debugged. Preferably, it would be nice to
take this as an opportunity to turn this assertion into a crash safe format (see
g_return_if* glib assertions). The best way to debug GLib assertion is to ask
GLib to abort on it. This can be done using environment variable:

  G_DEBUG=fatal_criticals

>From there, you will be able to check the backtrace of the current and other
threads running, and also inspect local variables state. This information will
be key in understand what is going on, and what could be the fix. Please try and
provide similar information. Note thats libcamera project track bufs like this
one at https://bugs.libcamera.org/

> Some additional tests:
> * Setting both cameras down to 5 FPS can avoid the problem.
> * No problem at any FPS if we run one camera only
> Spec/ Platform
> * Hardware: Raspberry PI-5 4GB/ 8GB
> * Camera: Camera module V2

If someone from RPi could provide me a second module, I would then be able to
reproduce.

cheers,
Nicolas

> * OS: Bookworm (Linux gmpi6 6.6.51+rpt-rpi-2712 #1 SMP PREEMPT Debian 1:6.6.51-
> 1+rpt2 (2024-10-01) aarch64 GNU/Linux)
> * Version: libcamera0.3/stable,now 0.3.2+rpt20240927-1 arm64
> * Camera debug file with camera-bug-report: bug-report.txt [3]
> DetailI have two IMX219 (Pi camera V2) connected to two Raspberry Pi-5. I'm
> launching the two cameras with the gstreamer element libcamerasrc:
> gst-launch-1.0 -v libcamerasrc camera-
> name="/base/axi/pcie at 120000/rp1/i2c at 80000/imx219 at 10" ! video/x-
> raw,colorimetry=bt709,format=NV12,width=1664,height=1232,framerate=10/1 !
> x264enc tune=zerolatency speed-preset=fast pass=qual quantizer=23
> bitrate=16384 ! video/x-h264,profile=high,stream-format=avc ! rtph264pay pt=96
> ! udpsink host=$IP port=8600 gst-launch-1.0 -v libcamerasrc camera-
> name="/base/axi/pcie at 120000/rp1/i2c at 88000/imx219 at 10" ! video/x-
> raw,colorimetry=bt709,format=NV12,width=1664,height=1232,framerate=10/1 !
> x264enc tune=zerolatency speed-preset=fast pass=qual quantizer=23
> bitrate=16384 ! video/x-h264,profile=high,stream-format=avc ! rtph264pay pt=96
> ! udpsink host=$IP port=8601
> For a short period, I can receive streaming from both cameras on the client
> side. However, the first launched camera will always hangs after a few 10's of
> seconds with following error:
> 0:01:37.9 / 99:99:99. ** (gst-launch-1.0:4162): CRITICAL **: 15:26:46.151:
> void GstLibcameraSrcState::requestCompleted(libcamera::Request*): assertion
> 'wrap->request_.get() == request' failed Caught SIGSEGV #0 0x00007fffbc08ba58
> in gst_pad_query_default at plt () from /lib/aarch64-linux-gnu/gstreamer-
> 1.0/libgstrtp.so #1 0x00007fffbf11bb9c in gst_pad_query () from /lib/aarch64-
> linux-gnu/libgstreamer-1.0.so.0 #2 0x00007fffbf11c36c in gst_pad_peer_query ()
> from /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0 #3 0x00007fffbe248d8c in ??
> () from /lib/aarch64-linux-gnu/libgstbase-1.0.so.0 #4 0x00007fffbf0f0b98 in
> gst_element_query () from /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0 #5
> 0x00007fffbf0c2ee4 in ?? () from /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
> #6 0x00007fffbf1050f8 in gst_iterator_fold () from /lib/aarch64-linux-
> gnu/libgstreamer-1.0.so.0 #7 0x00007fffbf0c2fc0 in ?? () from /lib/aarch64-
> linux-gnu/libgstreamer-1.0.so.0 #8 0x00007fffbf0c9140 in ?? () from
> /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0 #9 0x00007fffbf0f0b98 in
> gst_element_query () from /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0 #10
> 0x00007fffbf15d350 in gst_element_query_duration () from /lib/aarch64-linux-
> gnu/libgstreamer-1.0.so.0 #11 0x00005555922c3aa4 in ?? () #12
> 0x00007fffbef782ac in ?? () from /lib/aarch64-linux-gnu/libglib-2.0.so.0 #13
> 0x00007fffbef77614 in g_main_context_dispatch () from /lib/aarch64-linux-
> gnu/libglib-2.0.so.0 #14 0x00007fffbef779e0 in ?? () from /lib/aarch64-linux-
> gnu/libglib-2.0.so.0 #15 0x00007fffbef77d04 in g_main_loop_run () from
> /lib/aarch64-linux-gnu/libglib-2.0.so.0 #16 0x00005555922c60d4 in ?? () #17
> 0x00007fffbed07740 in __libc_start_call_main (main=main at entry=0x5555922c38c0,
> argc=argc at entry=22, argv=argv at entry=0x7fffe8059bc8) at
> ../sysdeps/nptl/libc_start_call_main.h:58 #18 0x00007fffbed07818 in
> __libc_start_main_impl (main=0x5555922c38c0, argc=22, argv=0x7fffe8059bc8,
> init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>,
> stack_end=<optimized out>) at ../csu/libc-start.c:360 #19 0x00005555922c3930
> in ?? () Spinning. Please run 'gdb gst-launch-1.0 4162' to continue debugging,
> Ctrl-C to quit, or Ctrl-\ to dump core. ^\Quit8.3 / 99:99:99.
> Additional background info:
> * I intended to have the resolution of 1640x1232, which is the default binning
> mode for the IMX219. However, due to some presumably alignment issue [1], I
> can only set it to 1664x1232
> * I cannot use the rpicam-apps since at such low FPS, the 8-frame delay [2]
> become a huge delay
> * I did enlarge the UDP memroy size beforehand by: sudo sysctl -w
> net.core.wmem_max=134217728
> 
> 
> Best regards,
> Ewing Kang
> 2024.10.11
> 


[1] presumably alignment issue
    https://forums.raspberrypi.com/viewtopic.php?t=377359
[2] 8-frame delay https://forums.raspberrypi.com/viewtopic.php?t=377323
[3] bug-report.txt
    https://github.com/user-attachments/files/17275031/113.10.7.pi6.bug-report.txt
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.libcamera.org/pipermail/libcamera-devel/attachments/20241017/7ee20e9f/attachment.htm>