OpenCV and libcamera [WAS Issue Running Raspberry Pi Camera Module 3 with libcamera on Raspberry Pi 5 + Ubuntu 24.04]

Jacopo Mondi jacopo.mondi at ideasonboard.com
Thu Nov 7 15:29:48 CET 2024 

Hi Meryem

  the mailing list got dropped, let me re-add it back so that we can
keep this conversation archived

On Thu, Nov 07, 2024 at 04:36:26PM +0300, Meryem Koc wrote:
> Thank you for the information and the link! I managed to solve the issue by
> using a node from this repository.
>
> https://github.com/christianrauch/camera_ros

Christian is active on this list. I'll cc him so that he knows about
one more user about his work.

Thanks
   j

>
>  It allowed me to capture images from the camera and successfully feed them
> into OpenCV.
>
>  Best regards,
>  Meryem
>
>
> 7 Kas 2024 Per, saat 15:54 tarihinde Jacopo Mondi <
> jacopo.mondi at ideasonboard.com> şunu yazdı:
>
> > Hi Meryem
> >
> > On Thu, Nov 07, 2024 at 03:18:35PM +0300, Meryem Koc wrote:
> > > Thank you for the detailed response and clarifications. It seems I may
> > have
> > > misunderstood the purpose of libpisp in this context. My primary goal is
> > to
> > > capture images from the Raspberry Pi camera using OpenCV within ROS 2 for
> > > image processing tasks.
> >
> > Ok, so it's about using OpenCV.
> >
> > Unfortunately I'm not an expert here, but I've asked around and it
> > seems currently OpenCV doesn't support capturing from a libcamera
> > source.
> >
> > The best course of actions is to capture from the camera and feed
> > OpenCV with the result. I'm not knowledgeable enough about the
> > details, but I can share an old experiment Tomi made to test the
> > Python bindings with OpenCV
> >
> > https://github.com/tomba/libcamera.git opencv
> >
> > I've also see online there are a few experimental OpenCV plugins to
> > interface with libcamera.
> >
> > >
> > > Here is the current situation:
> > >
> > >    - *Qcam*: It works fine and confirms that the compiled version of
> > >    libcamera is functioning as expected.
> > >    - *OpenCV*: I tried to use OpenCV to capture images directly from the
> > >    camera, but the camera opens briefly and then closes without
> > displaying an
> > >    image.
> >
> > No surprise, afaict the OpenCV camera backend only supports raw V4L2
> > interfacing (iow what would work with a USB camera) but no libcamera
> > (which is needed for complex camera pipelines involving an ISP).
> >
> > >    - *libcamera with OpenCV*: When attempting to use libcamera with
> > OpenCV,
> > >    it seems OpenCV doesn’t recognize libcamera, resulting in no output.
> > >
> > > What I have tried so far:
> > >
> > >    1. Reinstalling the libraries (libcamera and libpisp) from the
> > official
> > >    Raspberry Pi sources.
> > >    2. Setting PKG_CONFIG_PATH and LD_LIBRARY_PATH to ensure the paths are
> > >    correctly configured for library discovery.
> > >
> > > Ideally, I’d like to use OpenCV in combination with libcamera to capture
> > > and process images directly within ROS 2. Are there any additional
> > > configurations or libraries that would enable OpenCV to work seamlessly
> > > with libcamera? Alternatively, would you recommend a different approach,
> > > such as using gstreamer or another tool, to bridge OpenCV and libcamera?
> >
> > Bridging libcamera and OpenCV is probably the best solution yes.
> > I'm not sure I can actually suggest how to better do that though.
> >
> > Thanks
> >   j
> >
> > >
> > > Thank you in advance for any suggestions!
> > >
> > > Jacopo Mondi <jacopo.mondi at ideasonboard.com>, 7 Kas 2024 Per, 14:34
> > > tarihinde şunu yazdı:
> > >
> > > > Hi Meryem
> > > >
> > > > On Thu, Nov 07, 2024 at 02:22:07PM +0300, Meryem Koc wrote:
> > > > > Hello Jacopo,
> > > > >
> > > > > As you suggested, I removed the libraries and reinstalled them from
> > the
> > > > > official Raspberry Pi sources (libcamera and libpisp). The
> > installation
> > > > and
> > > > > compilation processes were successfully completed, but I am still
> > unable
> > > > to
> > > > > integrate the libpisp library with ROS 2 in my system. Apart from
> > Qcam, I
> > > >
> > > > mmm, why libpisp ? libpisp is the low-level library (lower level than
> > > > libcamera) that translates the ISP commands from libcamera to the
> > > > driver-consumable interface. I'm not sure in what use cases you would
> > > > want to interface with it directly.
> > > >
> > > > > am struggling to obtain camera images on any other platform.
> > > >
> > > > Ok, so if Qcam works it means the version of libcamera (which depends
> > > > and uses libpisp) you have compiled and installed works.
> > > >
> > > > I'm not sure what are the "other platforms" you are referring to
> > > > though :)
> > > >
> > > > >
> > > > > I reviewed the installation steps again and tried setting paths like
> > > > > PKG_CONFIG_PATH and LD_LIBRARY_PATH, but the issue persists. Do you
> > have
> > > > > any additional suggestions for using the Raspberry Pi 5 and V3 camera
> > > > > module with ROS 2? I would greatly appreciate any assistance you can
> > > > > provide.
> > > >
> > > > I'm sorry but to know what is going wrong I should know what you're
> > > > trying to achieve here. The libcamera version you installed seems to
> > > > work correctly (according to the fact that qcam works).
> > > >
> > > > Now to use libcamera you either go through the rpicam applications
> > > > (set of C++ and Python apps that RaspberryPi developed for their
> > > > platforms), you write your own app using the libcamera API, or you use
> > > > it through an higher level framework like gstreamer or pipewire (that
> > > > provides interfacing to the camera from system applications like the
> > > > browser).
> > > >
> > > > I don't know much about ROS2 and their camera stack, if any, so
> > > > I'm not sure I can help with questions related to that specific OS.
> > > >
> > > > >
> > > > > Thank you,
> > > > > Meryem
> > > > >
> > > > > Jacopo Mondi <jacopo.mondi at ideasonboard.com>, 4 Kas 2024 Pzt, 13:06
> > > > > tarihinde şunu yazdı:
> > > > >
> > > > > > Hi Meryem
> > > > > >
> > > > > > On Sat, Nov 02, 2024 at 06:56:28PM +0300, Meryem Koc wrote:
> > > > > > > Hello libcamera Development Team,
> > > > > > >
> > > > > > > I'm encountering significant challenges trying to get the
> > Raspberry
> > > > Pi
> > > > > > > Camera Module 3 working with libcamera on a Raspberry Pi 5
> > running
> > > > Ubuntu
> > > > > > > 24.04. Despite numerous troubleshooting attempts, I have not been
> > > > able to
> > > > > > > capture any output from the camera, and the issue seems to be
> > > > worsening.
> > > > > > >
> > > > > > > *System Information:*
> > > > > > >
> > > > > > >    - *Device:* Raspberry Pi 5
> > > > > > >    - *Operating System:* Ubuntu 24.04
> > > > > > >    - *Camera Module:* Raspberry Pi Camera Module 3
> > > > > > >
> > > > > > > *Steps I’ve Tried:*
> > > > > > >
> > > > > > >    1. I attempted to install the *libcamera-dev* and
> > *libcamera-apps*
> > > > > > packages,
> > > > > > >    but I found they were unavailable on Ubuntu 24.04.
> > > > > > >    2. I modified the config.txt file by adding dtoverlay=imx708
> > and
> > > > > > >    disabling auto-detection with camera_auto_detect=0.
> > > > > > >    3. Running the libcamera-hello command in the terminal
> > resulted
> > > > in the
> > > > > > >    camera not being detected.
> > > > > > >    4. Initially, Qcam was able to detect the camera, but now no
> > > > > > application
> > > > > > >    can detect it at all.
> > > > > > >    5. I have tried various reboots and checked connections, but
> > none
> > > > have
> > > > > > >    yielded a positive outcome.
> > > > > > >
> > > > > > > When I tested the camera on Raspberry Pi OS, it worked without
> > any
> > > > > > issues,
> > > > > > > so it seems specific to Ubuntu 24.04 and libcamera.
> > > > > > >
> > > > > > > *Questions:*
> > > > > > >
> > > > > > >    - Could there be compatibility issues with this combination of
> > > > > > Raspberry
> > > > > > >    Pi 5, Ubuntu 24.04, and the Raspberry Pi Camera Module 3?
> > > > > > >    - Is there a specific libcamera configuration or alternative
> > > > method
> > > > > > that
> > > > > > >    you recommend for this setup?
> > > > > > >    - Do you have any suggestions for alternative drivers or
> > support
> > > > > > >    packages?
> > > > > > >
> > > > > > > I would greatly appreciate any guidance you can provide. Thank
> > you in
> > > > > > > advance for your assistance.
> > > > > >
> > > > > > You should check which version of libcamera ubuntu pacakges.
> > > > > > Specifically on your use case, the Pi5 support is not merged in
> > > > > > mainline libcamera yet, so the tree from raspberry pi
> > > > > > https://github.com/raspberrypi/libcamera
> > > > > > should be used. I doubt Ubuntu packages that version but I can't
> > tell.
> > > > > >
> > > > > > You should compile libcamera (and libpisp
> > > > > > https://github.com/raspberrypi/libpisp) manually to get support
> > for
> > > > > > Pi5 (and probably you'll have to re-compile rpicam-apps as well
> > once
> > > > > > you have updated libcamera)
> > > > > >
> > > > > > I suggest you go through:
> > > > > >
> > > > > >
> > > >
> > https://www.raspberrypi.com/documentation/computers/camera_software.html#building-libcamera
> > > > > >
> > > > > > >
> > > > > > > Best regards,
> > > > > > > Meryem
> > > > > >
> > > >
> >

More information about the libcamera-devel mailing list