Gstreamer assertion failed after a short period when running two cameras simultaneously

[康 心奕]

Hello, everyone
This is a re-post of the issue at the Raspberry PI branch: https://github.com/raspberrypi/libcamera/issues/186?notification_referrer_id=NT_kwDOAH7EvLMxMjc0NDgwODE0NTo4MzA3OTAw. The admin redirected me here. I'd really appreciate it if anyone could point me some directions on how to resolve or further debug the issue. Thanks .

Symptom
I'm experiencing the following assertion fail error when running two RPi cameras simultaneously.

void GstLibcameraSrcState::requestCompleted(libcamera::Request*): assertion 'wrap->request_.get() == request' failed 

Some additional tests:
   
   - Setting both cameras down to 5 FPS can avoid the problem.
   - No problem at any FPS if we run one camera only

Spec/ Platform

   
   - Hardware: Raspberry PI-5 4GB/ 8GB
   - Camera: Camera module V2
   - OS: Bookworm (Linux gmpi6 6.6.51+rpt-rpi-2712 #1 SMP PREEMPT Debian 1:6.6.51-1+rpt2 (2024-10-01) aarch64 GNU/Linux)
   - Version: libcamera0.3/stable,now 0.3.2+rpt20240927-1 arm64
   - Camera debug file with camera-bug-report: bug-report.txt

Detail

I have two IMX219 (Pi camera V2) connected to two Raspberry Pi-5. I'm launching the two cameras with the gstreamer element libcamerasrc:
gst-launch-1.0 -v libcamerasrc camera-name="/base/axi/pcie at 120000/rp1/i2c at 80000/imx219 at 10" ! video/x-raw,colorimetry=bt709,format=NV12,width=1664,height=1232,framerate=10/1 ! x264enc tune=zerolatency speed-preset=fast pass=qual quantizer=23 bitrate=16384 ! video/x-h264,profile=high,stream-format=avc ! rtph264pay pt=96 ! udpsink host=$IP port=8600
gst-launch-1.0 -v libcamerasrc camera-name="/base/axi/pcie at 120000/rp1/i2c at 88000/imx219 at 10" ! video/x-raw,colorimetry=bt709,format=NV12,width=1664,height=1232,framerate=10/1 ! x264enc tune=zerolatency speed-preset=fast pass=qual quantizer=23 bitrate=16384 ! video/x-h264,profile=high,stream-format=avc ! rtph264pay pt=96 ! udpsink host=$IP port=8601 
For a short period, I can receive streaming from both cameras on the client side. However, the first launched camera will always hangs after a few 10's of seconds with following error:
0:01:37.9 / 99:99:99.
** (gst-launch-1.0:4162): CRITICAL **: 15:26:46.151: void GstLibcameraSrcState::requestCompleted(libcamera::Request*): assertion 'wrap->request_.get() == request' failed
Caught SIGSEGV
#0  0x00007fffbc08ba58 in gst_pad_query_default at plt () from /lib/aarch64-linux-gnu/gstreamer-1.0/libgstrtp.so
#1  0x00007fffbf11bb9c in gst_pad_query () from /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#2  0x00007fffbf11c36c in gst_pad_peer_query () from /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#3  0x00007fffbe248d8c in ?? () from /lib/aarch64-linux-gnu/libgstbase-1.0.so.0
#4  0x00007fffbf0f0b98 in gst_element_query () from /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#5  0x00007fffbf0c2ee4 in ?? () from /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#6  0x00007fffbf1050f8 in gst_iterator_fold () from /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#7  0x00007fffbf0c2fc0 in ?? () from /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#8  0x00007fffbf0c9140 in ?? () from /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#9  0x00007fffbf0f0b98 in gst_element_query () from /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#10 0x00007fffbf15d350 in gst_element_query_duration () from /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#11 0x00005555922c3aa4 in ?? ()
#12 0x00007fffbef782ac in ?? () from /lib/aarch64-linux-gnu/libglib-2.0.so.0
#13 0x00007fffbef77614 in g_main_context_dispatch () from /lib/aarch64-linux-gnu/libglib-2.0.so.0
#14 0x00007fffbef779e0 in ?? () from /lib/aarch64-linux-gnu/libglib-2.0.so.0
#15 0x00007fffbef77d04 in g_main_loop_run () from /lib/aarch64-linux-gnu/libglib-2.0.so.0
#16 0x00005555922c60d4 in ?? ()
#17 0x00007fffbed07740 in __libc_start_call_main (main=main at entry=0x5555922c38c0, argc=argc at entry=22, argv=argv at entry=0x7fffe8059bc8) at ../sysdeps/nptl/libc_start_call_main.h:58
#18 0x00007fffbed07818 in __libc_start_main_impl (main=0x5555922c38c0, argc=22, argv=0x7fffe8059bc8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=<optimized out>) at ../csu/libc-start.c:360
#19 0x00005555922c3930 in ?? ()
Spinning.  Please run 'gdb gst-launch-1.0 4162' to continue debugging, Ctrl-C to quit, or Ctrl-\ to dump core.
^\Quit8.3 / 99:99:99.
 
Additional background info:
   
   - I intended to have the resolution of 1640x1232, which is the default binning mode for the IMX219. However, due to some presumably alignment issue, I can only set it to 1664x1232
   - I cannot use the rpicam-apps since at such low FPS, the 8-frame delay become a huge delay
   - I did enlarge the UDP memroy size beforehand by: sudo sysctl -w net.core.wmem_max=134217728


Best regards,Ewing Kang2024.10.11

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.libcamera.org/pipermail/libcamera-devel/attachments/20241011/cae9bfce/attachment.htm>