Rpi 4 & 5 Libcamera, preview streaming and image capture

[jan at radicalsystems.co.za]

Hi Laurent & Kieran

I still have not solved my issue and I'm trying to get some insight into the process of streaming to preview and then capturing an image at the same time.
I'm very unfamiliar with Licamera and I'm trying to understand how the whole process is put together.

So, what is working is the preview streaming in Android and running Google MLKIT analytics.
But as soon as the image capture instruction is sent, it falls over.
And this is the trace of what goes wrong.

1. Android initiate createCaptureRequest
2. generic_frame_buffer_allocator.cpp => PlatformFrameBufferAllocator::Private::allocate
	2.1 gralloc_open
	2.2 hardwareModule_->methods->open
3. camera3_hal.cpp => hal_dev_open
	3.1 CameraHalManager::instance()->open
4. camera_device.cpp => Camera::open
	4.1 camera_->acquire
5. camera.cpp => Camera::qcquire
	5.1 isAccessAllowed(Private::CameraAvailabe) is where it fails, because it assumes the camera is not open I guess

This is the stack trace:

06-04 13:46:54.783  3213  3296 D Camera2CaptureRequestBuilder: createCaptureRequest
06-04 13:46:54.795  3324  3534 E libcamera: ERROR Camera camera.cpp:675 Camera in Running state trying acquire() requiring state Available
06-04 13:46:54.795  3324  3534 E libcamera: ERROR HAL camera_device.cpp:390 '/base/axi/pcie at 120000/rp1/i2c at 88000/imx708 at 1a': Failed to acquire the camera
06-04 13:46:54.795  3324  3534 E libcamera: ERROR HAL camera3_hal.cpp:81 Failed to open camera module '0'
06-04 13:46:54.796  3324  3534 E libcamera: FATAL HAL generic_frame_buffer_allocator.cpp:106 gralloc_open() failed: -87
06-04 13:46:54.796  3324  3534 D libcamera: Backtrace:

Can you guys maybe enlighten me how this process is supposed to work when previewing a streaming and then capturing the image?
Because this seems to be the hiccup with this.
Are there two devices that must be opened or is the image capture done by the streaming instance?
I see /dev/media0 to 3 gets created for the camera.

I would really appreciate it if you can give me some pointers.
We need the get this hardware solution going.

Cheers
Jan


-----Original Message-----
From: Laurent Pinchart <laurent.pinchart at ideasonboard.com> 
Sent: Tuesday, June 4, 2024 10:14 PM
To: jan at radicalsystems.co.za
Cc: libcamera-devel at lists.libcamera.org
Subject: Re: Rpi 4 & 5 Libcamera & Mmap permission issue on Android 14

Hello Jan,

On Tue, Jun 04, 2024 at 12:26:20PM +0200, jan at radicalsystems.co.za wrote:
> Hi Libcamera Fundis!
> 
> Having a problem capturing an image on Android 14 (& 13).
> 
> Tombstone looks like this:
> 
> 06-03 15:13:03.880 308 578 I DMABUFHEAPS: Using : Non-legacy ION heaps
> 06-03 15:13:03.880 308 578 I [minigbm:gbm_mesa_internals.cpp(233)]: 
> Found GPU v3d
> 06-03 15:13:04.052 308 359 E libcamera: ERROR Buffer 
> mapped_framebuffer.cpp:230 Failed to mmap plane: Permission denied
> 
> .... lots of Permission denied!
> 
> 06-03 15:13:11.282 308 359 E libcamera: ERROR Buffer 
> mapped_framebuffer.cpp:230 Failed to mmap plane: Permission denied
> 06-03 15:13:12.008 308 579 E libcamera: ERROR Camera camera.cpp:675 
> Camera in Running state trying acquire() requiring state Available
> 06-03 15:13:12.008 308 579 E libcamera: ERROR HAL 
> camera_device.cpp:390 '/base/axi/pcie at 120000/rp1/i2c at 88000/imx708 at 1a': 
> Failed to acquire the camera
> 06-03 15:13:12.008 308 579 E libcamera: ERROR HAL camera3_hal.cpp:81 Failed to open camera module '0'
> 06-03 15:13:12.008 308 579 E libcamera: FATAL HAL 
> generic_frame_buffer_allocator.cpp:106 gralloc_open() failed: -87
> 
> MLKIT Streaming & analyzing works until trying to capture an image the 
> above happens.
> 
> The various Android versions for Rpi5 I could find gave the same issue.
> 
> I tried Emteria, AOSP & LineageOS 13 & 14
> 
> Android built from Raspberry Vanilla and even replacing with later 
> libcamera had the same issue.
> 
> I’m not sure if this is purely a configuration problem, wrt to 
> policies or permissions or specifically to the RPi’s architecture.
>
> Please help, I can rebuild and test.
> 
> I have tried the latest code pulled into RaspVanilla, but had the same issue.

This doesn't ring a bell, but let's see if we can debug it.

The first step is to figure out if the mmap() parameters are right. In MappedFrameBuffer::MappedFrameBuffer(), at the location of the failure, could you print the value of fd, info.mapLength, and mmapFlags ?
Something like the following should do:

diff --git a/src/libcamera/mapped_framebuffer.cpp b/src/libcamera/mapped_framebuffer.cpp
index b3104e0504d4..baa0587b4fcf 100644
--- a/src/libcamera/mapped_framebuffer.cpp
+++ b/src/libcamera/mapped_framebuffer.cpp
@@ -223,6 +223,9 @@ MappedFrameBuffer::MappedFrameBuffer(const FrameBuffer *buffer, MapFlags flags)
 		const int fd = plane.fd.get();
 		auto &info = mappedBuffers[fd];
 		if (!info.address) {
+			LOG(Buffer, Info)
+				<< "Mapping fd: " << fd << ", length: "
+				<< info.mapLength << ", flags: " << mmapFlags;
 			void *address = mmap(nullptr, info.mapLength, mmapFlags,
 					     MAP_SHARED, fd, 0);
 			if (address == MAP_FAILED) {

(I haven't tested it though)

--
Regards,

Laurent Pinchart