Rpi 4 & 5 Libcamera, preview streaming and image capture

[Mattijs Korpershoek]

Hi Jan,

On ven., juin 07, 2024 at 15:54, <jan at radicalsystems.co.za> wrote:

> Hi Laurent & Kieran
>
> I still have not solved my issue and I'm trying to get some insight into the process of streaming to preview and then capturing an image at the same time.
> I'm very unfamiliar with Licamera and I'm trying to understand how the whole process is put together.
>
> So, what is working is the preview streaming in Android and running Google MLKIT analytics.
> But as soon as the image capture instruction is sent, it falls over.
> And this is the trace of what goes wrong.
>
> 1. Android initiate createCaptureRequest
> 2. generic_frame_buffer_allocator.cpp => PlatformFrameBufferAllocator::Private::allocate
> 	2.1 gralloc_open
> 	2.2 hardwareModule_->methods->open
> 3. camera3_hal.cpp => hal_dev_open
> 	3.1 CameraHalManager::instance()->open
> 4. camera_device.cpp => Camera::open
> 	4.1 camera_->acquire
> 5. camera.cpp => Camera::qcquire
> 	5.1 isAccessAllowed(Private::CameraAvailabe) is where it fails, because it assumes the camera is not open I guess
>
> This is the stack trace:
>
> 06-04 13:46:54.783  3213  3296 D Camera2CaptureRequestBuilder: createCaptureRequest
> 06-04 13:46:54.795  3324  3534 E libcamera: ERROR Camera camera.cpp:675 Camera in Running state trying acquire() requiring state Available
> 06-04 13:46:54.795  3324  3534 E libcamera: ERROR HAL camera_device.cpp:390 '/base/axi/pcie at 120000/rp1/i2c at 88000/imx708 at 1a': Failed to acquire the camera
> 06-04 13:46:54.795  3324  3534 E libcamera: ERROR HAL camera3_hal.cpp:81 Failed to open camera module '0'
> 06-04 13:46:54.796  3324  3534 E libcamera: FATAL HAL generic_frame_buffer_allocator.cpp:106 gralloc_open() failed: -87

This reminds of a bug I solved long ago:

Here, libcamera tries to open the camera HAL instead of the gralloc HAL.

I send a patch for this here:
https://patchwork.libcamera.org/patch/18655/

Can you confirm the above patch is in your tree?

It should be, since it's been merged a year ago.


> 06-04 13:46:54.796  3324  3534 D libcamera: Backtrace:
>
> Can you guys maybe enlighten me how this process is supposed to work when previewing a streaming and then capturing the image?
> Because this seems to be the hiccup with this.
> Are there two devices that must be opened or is the image capture done by the streaming instance?
> I see /dev/media0 to 3 gets created for the camera.
>
> I would really appreciate it if you can give me some pointers.
> We need the get this hardware solution going.
>
> Cheers
> Jan
>
>
> -----Original Message-----
> From: Laurent Pinchart <laurent.pinchart at ideasonboard.com> 
> Sent: Tuesday, June 4, 2024 10:14 PM
> To: jan at radicalsystems.co.za
> Cc: libcamera-devel at lists.libcamera.org
> Subject: Re: Rpi 4 & 5 Libcamera & Mmap permission issue on Android 14
>
> Hello Jan,
>
> On Tue, Jun 04, 2024 at 12:26:20PM +0200, jan at radicalsystems.co.za wrote:
>> Hi Libcamera Fundis!
>> 
>> Having a problem capturing an image on Android 14 (& 13).
>> 
>> Tombstone looks like this:
>> 
>> 06-03 15:13:03.880 308 578 I DMABUFHEAPS: Using : Non-legacy ION heaps
>> 06-03 15:13:03.880 308 578 I [minigbm:gbm_mesa_internals.cpp(233)]: 
>> Found GPU v3d
>> 06-03 15:13:04.052 308 359 E libcamera: ERROR Buffer 
>> mapped_framebuffer.cpp:230 Failed to mmap plane: Permission denied
>> 
>> .... lots of Permission denied!
>> 
>> 06-03 15:13:11.282 308 359 E libcamera: ERROR Buffer 
>> mapped_framebuffer.cpp:230 Failed to mmap plane: Permission denied
>> 06-03 15:13:12.008 308 579 E libcamera: ERROR Camera camera.cpp:675 
>> Camera in Running state trying acquire() requiring state Available
>> 06-03 15:13:12.008 308 579 E libcamera: ERROR HAL 
>> camera_device.cpp:390 '/base/axi/pcie at 120000/rp1/i2c at 88000/imx708 at 1a': 
>> Failed to acquire the camera
>> 06-03 15:13:12.008 308 579 E libcamera: ERROR HAL camera3_hal.cpp:81 Failed to open camera module '0'
>> 06-03 15:13:12.008 308 579 E libcamera: FATAL HAL 
>> generic_frame_buffer_allocator.cpp:106 gralloc_open() failed: -87
>> 
>> MLKIT Streaming & analyzing works until trying to capture an image the 
>> above happens.
>> 
>> The various Android versions for Rpi5 I could find gave the same issue.
>> 
>> I tried Emteria, AOSP & LineageOS 13 & 14
>> 
>> Android built from Raspberry Vanilla and even replacing with later 
>> libcamera had the same issue.
>> 
>> I’m not sure if this is purely a configuration problem, wrt to 
>> policies or permissions or specifically to the RPi’s architecture.
>>
>> Please help, I can rebuild and test.
>> 
>> I have tried the latest code pulled into RaspVanilla, but had the same issue.
>
> This doesn't ring a bell, but let's see if we can debug it.
>
> The first step is to figure out if the mmap() parameters are right. In MappedFrameBuffer::MappedFrameBuffer(), at the location of the failure, could you print the value of fd, info.mapLength, and mmapFlags ?
> Something like the following should do:
>
> diff --git a/src/libcamera/mapped_framebuffer.cpp b/src/libcamera/mapped_framebuffer.cpp
> index b3104e0504d4..baa0587b4fcf 100644
> --- a/src/libcamera/mapped_framebuffer.cpp
> +++ b/src/libcamera/mapped_framebuffer.cpp
> @@ -223,6 +223,9 @@ MappedFrameBuffer::MappedFrameBuffer(const FrameBuffer *buffer, MapFlags flags)
>  		const int fd = plane.fd.get();
>  		auto &info = mappedBuffers[fd];
>  		if (!info.address) {
> +			LOG(Buffer, Info)
> +				<< "Mapping fd: " << fd << ", length: "
> +				<< info.mapLength << ", flags: " << mmapFlags;
>  			void *address = mmap(nullptr, info.mapLength, mmapFlags,
>  					     MAP_SHARED, fd, 0);
>  			if (address == MAP_FAILED) {
>
> (I haven't tested it though)
>
> --
> Regards,
>
> Laurent Pinchart