[libcamera-devel] Issue allocating many frame buffers on RasPi

[Jacopo Mondi]

Hi Alan,

On Sat, Jul 02, 2022 at 07:48:48AM -0400, Alan W Szlosek Jr via libcamera-devel wrote:
> Hi libcamera, I'm creating a security camera app for RasPis and I'm
> having trouble allocating 20 frame buffers (would like to alloc even
> more). Do you know why? Do you have suggestions? I'm currently testing
> on a Raspberry Pi 3B+.
>

Can I ask why you need to allocate tham many buffers in the video
device ?

> This is the output I'm getting. The return value from allocate() seems
> to imply that everything is fine ("Allocated 20 buffers for stream")
> when it's not fine behind the scenes.
>
> [1:23:50.594602178] [1217]  INFO Camera camera_manager.cpp:293
> libcamera v0.0.0+3544-22656360
> [1:23:50.657034054] [1218]  WARN RPI raspberrypi.cpp:1241 Mismatch
> between Unicam and CamHelper for embedded data usage!
> [1:23:50.659149325] [1218]  INFO RPI raspberrypi.cpp:1356 Registered
> camera /base/soc/i2c0mux/i2c at 1/imx219 at 10 to Unicam device /dev/media3
> and ISP device /dev/media0
> [1:23:50.660510009] [1217]  INFO Camera camera.cpp:1029 configuring
> streams: (0) 1640x922-YUV420
> [1:23:50.661246471] [1218]  INFO RPI raspberrypi.cpp:760 Sensor:
> /base/soc/i2c0mux/i2c at 1/imx219 at 10 - Selected sensor format:
> 1920x1080-SBGGR10_1X10 - Selected unicam format: 1920x1080-pBAA
> Allocated 20 buffers for stream
> [1:23:50.733980221] [1218] ERROR V4L2 v4l2_videodevice.cpp:1218
> /dev/video14[14:cap]: Not enough buffers provided by V4L2VideoDevice
> [1:23:50.734467203] [1218] ERROR RPI raspberrypi.cpp:1008 Failed to
> allocate buffers

This seems to happen when the pipeline starts and tries to allocate
buffers for its internal usage. Might it be you simply run out of
available memory ?

Is there anything on your dmesg output that might suggest that, like a
message from your CMA allocator ?

Can you try with allocating an increasing number of buffers until you
don't get to the failure limit ?

> [1:23:50.739962387] [1217] ERROR Camera camera.cpp:528 Camera in
> Configured state trying queueRequest() requiring state Running
> [1:23:50.740078898] [1217] ERROR Camera camera.cpp:528 Camera in
> Configured state trying queueRequest() requiring state Running
>
> Here's how I'm compiling:
>
> clang++ -g -std=c++17 -o scaffold \
>     -I /usr/include/libcamera \
>     -L /usr/lib/aarch64-linux-gnu \
>     -l camera -l camera-base \
>     scaffold.cpp
>
> And here's the code I'm using. Thank you!
>
> #include <iomanip>
> #include <iostream>
> #include <memory>
> #include <thread>
>
> #include <libcamera/libcamera.h>
>
> using namespace libcamera;
>
> static std::shared_ptr<Camera> camera;
>
> time_t previousSeconds = 0;
> int frames = 0;
> static void requestComplete(Request *request)
> {
>     std::unique_ptr<Request> request2;
>     if (request->status() == Request::RequestCancelled)
>         return;
>     const std::map<const Stream *, FrameBuffer *> &buffers = request->buffers();
>
>     request->reuse(Request::ReuseBuffers);
>     camera->queueRequest(request);
>
>     struct timespec delta;
>     clock_gettime(CLOCK_REALTIME, &delta);
>     if (previousSeconds == delta.tv_sec) {
>         frames++;
>     } else {
>         fprintf(stdout, "Frames: %d\n", frames);
>         frames = 1;
>         previousSeconds = delta.tv_sec;
>     }
> }
>
> int main()
> {
>     std::unique_ptr<CameraManager> cm = std::make_unique<CameraManager>();
>     cm->start();
>
>     if (cm->cameras().empty()) {
>        std::cout << "No cameras were identified on the system."
>                  << std::endl;
>        cm->stop();
>        return EXIT_FAILURE;
>     }
>
>     std::string cameraId = cm->cameras()[0]->id();
>     camera = cm->get(cameraId);
>
>     camera->acquire();
>
>     // VideoRecording
>     std::unique_ptr<CameraConfiguration> config =
> camera->generateConfiguration( { StreamRole::VideoRecording } );
>     StreamConfiguration &streamConfig = config->at(0);
>     streamConfig.size.width = 1640; //640;
>     streamConfig.size.height = 922; //480;
>     // This seems to default to 4, but we want to queue buffers for post
>     // processing, so we need to raise it.
>     // 10 works ... oddly, but 20 fails behind the scenes. doesn't apear
>     // to be an error we can catch
>     streamConfig.bufferCount = 20;
>
>     // TODO: check return value of this
>     CameraConfiguration::Status status = config->validate();
>     if (status == CameraConfiguration::Invalid) {
>         fprintf(stderr, "Camera Configuration is invalid\n");
>     } else if (status == CameraConfiguration::Adjusted) {
>         fprintf(stderr, "Camera Configuration was invalid and has been
> adjusted\n");
>     }
>
>     camera->configure(config.get());
>
>     FrameBufferAllocator *allocator = new FrameBufferAllocator(camera);
>
>     for (StreamConfiguration &cfg : *config) {
>         // TODO: it's possible we'll need our own allocator for raspi,
>         // so we can enqueue many frames for processing
>         int ret = allocator->allocate(cfg.stream());
>         // This error handling doesn't catch a failure to allocate 20 buffers
>         if (ret < 0) {
>             std::cerr << "Can't allocate buffers" << std::endl;
>             return -ENOMEM;
>         }
>
>         size_t allocated = allocator->buffers(cfg.stream()).size();
>         std::cout << "Allocated " << allocated << " buffers for
> stream" << std::endl;
>     }
>
>
>     Stream *stream = streamConfig.stream();
>     const std::vector<std::unique_ptr<FrameBuffer>> &buffers =
> allocator->buffers(stream);
>     std::vector<std::unique_ptr<Request>> requests;
>
>     for (unsigned int i = 0; i < buffers.size(); ++i) {
>         std::unique_ptr<Request> request = camera->createRequest();
>         if (!request)
>         {
>             std::cerr << "Can't create request" << std::endl;
>             return -ENOMEM;
>         }
>
>         const std::unique_ptr<FrameBuffer> &buffer = buffers[i];
>         int ret = request->addBuffer(stream, buffer.get());
>         if (ret < 0)
>         {
>             std::cerr << "Can't set buffer for request"
>                     << std::endl;
>             return ret;
>         }
>
>         requests.push_back(std::move(request));
>     }
>
>     camera->requestCompleted.connect(requestComplete);
>
>     // sets fps (via frame duration limts)
>     // TODO: create ControlList and move to global var
>     // TODO: is there a raspi-specific implementation of this?
>     libcamera::ControlList controls(libcamera::controls::controls);
>     int framerate = 30;
>     int64_t frame_time = 1000000 / framerate; // in microseconds
>     controls.set(libcamera::controls::FrameDurationLimits, {
> frame_time, frame_time });
>
>     camera->start(&controls);
>     for (auto &request : requests)
>        camera->queueRequest(request.get());
>
>     //60 * 60 * 24 * 7; // days
>     int duration = 10;
>
>     for (int i = 0; i < duration; i++) {
>         std::cout << "Sleeping" << std::endl;
>         std::this_thread::sleep_for(std::chrono::milliseconds(1000));
>     }
>
>
>     return 0;
> }
>
> --
> Alan Szlosek