<div dir="ltr"><div dir="ltr">Hi Alan,<div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, 4 Jul 2022 at 20:16, Alan W Szlosek Jr <<a href="mailto:alan.szlosek@gmail.com">alan.szlosek@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Thanks Naush and Jacopo,<br>
<br>
I'm trying to allocate lots of buffers to avoid unnecessary data<br>
copying. And I'm hoping to support high frame rates of 30 fps as well<br>
so I'll ultimately need to allocate around 60 frame buffers. Basically<br>
I plan to have a circular buffer of frame buffers, spanning at least a<br>
second or two. On slower machines (like Pi Zeros) I'll likely run a<br>
motion detection algorithm every second. On RasPis with more cores<br>
I'll probably run it 3 times a second. When motion is detected I'll<br>
transcode the frames to h264 using the hardware encoder and save to a<br>
file. Hopefully that explains why I want to allocate so many buffers<br>
ahead of time.<br></blockquote><div><br></div><div>I think you might be better off encoding the frames and writing the output</div><div>to a circular buffer instead of the other way round. That way you can keep</div><div>the frame buffer count down, and increase the size of the circular buffer</div><div>(allocated in user memory) to a size that matches your required record time.</div><div><br></div><div>Conveniently, we do have an example of exactly this in libcamera-vid.</div><div>You can find more details here:<br></div><div><br></div><div> <a href="https://www.raspberrypi.com/documentation/accessories/camera.html#video-command-line-options">https://www.raspberrypi.com/documentation/accessories/camera.html#video-command-line-options</a></div><div><br></div><div>Additionally, we do also have a motion detect software post-processing stage</div><div>in libcamera-vid that you might want to look at. It might not do exactly what</div><div>you want, but would be a good template for your custom application.</div><div><br></div><div>Thanks,</div><div>Naush</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
Jacopo, to answer your question about dmesg output .... No, there's<br>
nothing in dmesg, see the following:<br>
<br>
[ 17.904193] NET: Registered PF_BLUETOOTH protocol family<br>
[ 17.904202] Bluetooth: HCI device and connection manager initialized<br>
[ 17.904227] Bluetooth: HCI socket layer initialized<br>
[ 17.904240] Bluetooth: L2CAP socket layer initialized<br>
[ 17.904263] Bluetooth: SCO socket layer initialized<br>
[ 17.923327] Bluetooth: HCI UART driver ver 2.3<br>
[ 17.923355] Bluetooth: HCI UART protocol H4 registered<br>
[ 17.923440] Bluetooth: HCI UART protocol Three-wire (H5) registered<br>
[ 17.923694] Bluetooth: HCI UART protocol Broadcom registered<br>
[ 18.541155] Bluetooth: BNEP (Ethernet Emulation) ver 1.3<br>
[ 18.541179] Bluetooth: BNEP filters: protocol multicast<br>
[ 18.541200] Bluetooth: BNEP socket layer initialized<br>
[ 18.561456] NET: Registered PF_ALG protocol family<br>
[ 20.449306] vc4-drm soc:gpu: [drm] Cannot find any crtc or sizes<br>
[ 33.761159] cam-dummy-reg: disabling<br>
<br>
And Naush, when you say CMA do you mean the Contiguous Memory<br>
Allocator? Does this mean that when I ask for 20 buffers it's trying<br>
to allocate 1 contiguous block of memory behind the scenes, resulting<br>
in 1 dmabuf file descriptor? If so, it sounds like I should somehow<br>
ask for smaller, separate dmabuf blocks to cover what I need. What do<br>
you think? Is that easily doable?<br>
<br>
Thanks to you both for your help.<br>
<br>
On Mon, Jul 4, 2022 at 4:40 AM Naushir Patuck <<a href="mailto:naush@raspberrypi.com" target="_blank">naush@raspberrypi.com</a>> wrote:<br>
><br>
> Hi Alan,<br>
><br>
> On Mon, 4 Jul 2022 at 09:31, Jacopo Mondi via libcamera-devel <<a href="mailto:libcamera-devel@lists.libcamera.org" target="_blank">libcamera-devel@lists.libcamera.org</a>> wrote:<br>
>><br>
>> Hi Alan,<br>
>><br>
>> On Sat, Jul 02, 2022 at 07:48:48AM -0400, Alan W Szlosek Jr via libcamera-devel wrote:<br>
>> > Hi libcamera, I'm creating a security camera app for RasPis and I'm<br>
>> > having trouble allocating 20 frame buffers (would like to alloc even<br>
>> > more). Do you know why? Do you have suggestions? I'm currently testing<br>
>> > on a Raspberry Pi 3B+.<br>
>> ><br>
>><br>
>> Can I ask why you need to allocate tham many buffers in the video<br>
>> device ?<br>
><br>
><br>
> Snap. I was going to ask the same question. All frame buffers are allocated out<br>
> of CMA space. 20 x 2MP YUV420 buffers is approx 60 MBytes only for a single<br>
> set of buffers. Typically, you ought to get aways with < 10 buffers for most video<br>
> use cases.<br>
><br>
> Naush<br>
><br>
><br>
>><br>
>><br>
>> > This is the output I'm getting. The return value from allocate() seems<br>
>> > to imply that everything is fine ("Allocated 20 buffers for stream")<br>
>> > when it's not fine behind the scenes.<br>
>> ><br>
>> > [1:23:50.594602178] [1217] INFO Camera camera_manager.cpp:293<br>
>> > libcamera v0.0.0+3544-22656360<br>
>> > [1:23:50.657034054] [1218] WARN RPI raspberrypi.cpp:1241 Mismatch<br>
>> > between Unicam and CamHelper for embedded data usage!<br>
>> > [1:23:50.659149325] [1218] INFO RPI raspberrypi.cpp:1356 Registered<br>
>> > camera /base/soc/i2c0mux/i2c@1/imx219@10 to Unicam device /dev/media3<br>
>> > and ISP device /dev/media0<br>
>> > [1:23:50.660510009] [1217] INFO Camera camera.cpp:1029 configuring<br>
>> > streams: (0) 1640x922-YUV420<br>
>> > [1:23:50.661246471] [1218] INFO RPI raspberrypi.cpp:760 Sensor:<br>
>> > /base/soc/i2c0mux/i2c@1/imx219@10 - Selected sensor format:<br>
>> > 1920x1080-SBGGR10_1X10 - Selected unicam format: 1920x1080-pBAA<br>
>> > Allocated 20 buffers for stream<br>
>> > [1:23:50.733980221] [1218] ERROR V4L2 v4l2_videodevice.cpp:1218<br>
>> > /dev/video14[14:cap]: Not enough buffers provided by V4L2VideoDevice<br>
>> > [1:23:50.734467203] [1218] ERROR RPI raspberrypi.cpp:1008 Failed to<br>
>> > allocate buffers<br>
>><br>
>> This seems to happen when the pipeline starts and tries to allocate<br>
>> buffers for its internal usage. Might it be you simply run out of<br>
>> available memory ?<br>
>><br>
>> Is there anything on your dmesg output that might suggest that, like a<br>
>> message from your CMA allocator ?<br>
>><br>
>> Can you try with allocating an increasing number of buffers until you<br>
>> don't get to the failure limit ?<br>
>><br>
>> > [1:23:50.739962387] [1217] ERROR Camera camera.cpp:528 Camera in<br>
>> > Configured state trying queueRequest() requiring state Running<br>
>> > [1:23:50.740078898] [1217] ERROR Camera camera.cpp:528 Camera in<br>
>> > Configured state trying queueRequest() requiring state Running<br>
>> ><br>
>> > Here's how I'm compiling:<br>
>> ><br>
>> > clang++ -g -std=c++17 -o scaffold \<br>
>> > -I /usr/include/libcamera \<br>
>> > -L /usr/lib/aarch64-linux-gnu \<br>
>> > -l camera -l camera-base \<br>
>> > scaffold.cpp<br>
>> ><br>
>> > And here's the code I'm using. Thank you!<br>
>> ><br>
>> > #include <iomanip><br>
>> > #include <iostream><br>
>> > #include <memory><br>
>> > #include <thread><br>
>> ><br>
>> > #include <libcamera/libcamera.h><br>
>> ><br>
>> > using namespace libcamera;<br>
>> ><br>
>> > static std::shared_ptr<Camera> camera;<br>
>> ><br>
>> > time_t previousSeconds = 0;<br>
>> > int frames = 0;<br>
>> > static void requestComplete(Request *request)<br>
>> > {<br>
>> > std::unique_ptr<Request> request2;<br>
>> > if (request->status() == Request::RequestCancelled)<br>
>> > return;<br>
>> > const std::map<const Stream *, FrameBuffer *> &buffers = request->buffers();<br>
>> ><br>
>> > request->reuse(Request::ReuseBuffers);<br>
>> > camera->queueRequest(request);<br>
>> ><br>
>> > struct timespec delta;<br>
>> > clock_gettime(CLOCK_REALTIME, &delta);<br>
>> > if (previousSeconds == delta.tv_sec) {<br>
>> > frames++;<br>
>> > } else {<br>
>> > fprintf(stdout, "Frames: %d\n", frames);<br>
>> > frames = 1;<br>
>> > previousSeconds = delta.tv_sec;<br>
>> > }<br>
>> > }<br>
>> ><br>
>> > int main()<br>
>> > {<br>
>> > std::unique_ptr<CameraManager> cm = std::make_unique<CameraManager>();<br>
>> > cm->start();<br>
>> ><br>
>> > if (cm->cameras().empty()) {<br>
>> > std::cout << "No cameras were identified on the system."<br>
>> > << std::endl;<br>
>> > cm->stop();<br>
>> > return EXIT_FAILURE;<br>
>> > }<br>
>> ><br>
>> > std::string cameraId = cm->cameras()[0]->id();<br>
>> > camera = cm->get(cameraId);<br>
>> ><br>
>> > camera->acquire();<br>
>> ><br>
>> > // VideoRecording<br>
>> > std::unique_ptr<CameraConfiguration> config =<br>
>> > camera->generateConfiguration( { StreamRole::VideoRecording } );<br>
>> > StreamConfiguration &streamConfig = config->at(0);<br>
>> > streamConfig.size.width = 1640; //640;<br>
>> > streamConfig.size.height = 922; //480;<br>
>> > // This seems to default to 4, but we want to queue buffers for post<br>
>> > // processing, so we need to raise it.<br>
>> > // 10 works ... oddly, but 20 fails behind the scenes. doesn't apear<br>
>> > // to be an error we can catch<br>
>> > streamConfig.bufferCount = 20;<br>
>> ><br>
>> > // TODO: check return value of this<br>
>> > CameraConfiguration::Status status = config->validate();<br>
>> > if (status == CameraConfiguration::Invalid) {<br>
>> > fprintf(stderr, "Camera Configuration is invalid\n");<br>
>> > } else if (status == CameraConfiguration::Adjusted) {<br>
>> > fprintf(stderr, "Camera Configuration was invalid and has been<br>
>> > adjusted\n");<br>
>> > }<br>
>> ><br>
>> > camera->configure(config.get());<br>
>> ><br>
>> > FrameBufferAllocator *allocator = new FrameBufferAllocator(camera);<br>
>> ><br>
>> > for (StreamConfiguration &cfg : *config) {<br>
>> > // TODO: it's possible we'll need our own allocator for raspi,<br>
>> > // so we can enqueue many frames for processing<br>
>> > int ret = allocator->allocate(cfg.stream());<br>
>> > // This error handling doesn't catch a failure to allocate 20 buffers<br>
>> > if (ret < 0) {<br>
>> > std::cerr << "Can't allocate buffers" << std::endl;<br>
>> > return -ENOMEM;<br>
>> > }<br>
>> ><br>
>> > size_t allocated = allocator->buffers(cfg.stream()).size();<br>
>> > std::cout << "Allocated " << allocated << " buffers for<br>
>> > stream" << std::endl;<br>
>> > }<br>
>> ><br>
>> ><br>
>> > Stream *stream = streamConfig.stream();<br>
>> > const std::vector<std::unique_ptr<FrameBuffer>> &buffers =<br>
>> > allocator->buffers(stream);<br>
>> > std::vector<std::unique_ptr<Request>> requests;<br>
>> ><br>
>> > for (unsigned int i = 0; i < buffers.size(); ++i) {<br>
>> > std::unique_ptr<Request> request = camera->createRequest();<br>
>> > if (!request)<br>
>> > {<br>
>> > std::cerr << "Can't create request" << std::endl;<br>
>> > return -ENOMEM;<br>
>> > }<br>
>> ><br>
>> > const std::unique_ptr<FrameBuffer> &buffer = buffers[i];<br>
>> > int ret = request->addBuffer(stream, buffer.get());<br>
>> > if (ret < 0)<br>
>> > {<br>
>> > std::cerr << "Can't set buffer for request"<br>
>> > << std::endl;<br>
>> > return ret;<br>
>> > }<br>
>> ><br>
>> > requests.push_back(std::move(request));<br>
>> > }<br>
>> ><br>
>> > camera->requestCompleted.connect(requestComplete);<br>
>> ><br>
>> > // sets fps (via frame duration limts)<br>
>> > // TODO: create ControlList and move to global var<br>
>> > // TODO: is there a raspi-specific implementation of this?<br>
>> > libcamera::ControlList controls(libcamera::controls::controls);<br>
>> > int framerate = 30;<br>
>> > int64_t frame_time = 1000000 / framerate; // in microseconds<br>
>> > controls.set(libcamera::controls::FrameDurationLimits, {<br>
>> > frame_time, frame_time });<br>
>> ><br>
>> > camera->start(&controls);<br>
>> > for (auto &request : requests)<br>
>> > camera->queueRequest(request.get());<br>
>> ><br>
>> > //60 * 60 * 24 * 7; // days<br>
>> > int duration = 10;<br>
>> ><br>
>> > for (int i = 0; i < duration; i++) {<br>
>> > std::cout << "Sleeping" << std::endl;<br>
>> > std::this_thread::sleep_for(std::chrono::milliseconds(1000));<br>
>> > }<br>
>> ><br>
>> ><br>
>> > return 0;<br>
>> > }<br>
>> ><br>
>> > --<br>
>> > Alan Szlosek<br>
<br>
<br>
<br>
-- <br>
Alan Szlosek<br>
</blockquote></div></div>