<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, 17 Oct 2022 at 10:00, Kieran Bingham <<a href="mailto:kieran.bingham@ideasonboard.com">kieran.bingham@ideasonboard.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Quoting Naushir Patuck via libcamera-devel (2022-10-14 15:55:46)<br>
> On Fri, 14 Oct 2022 at 14:18, Naushir Patuck <<a href="mailto:naush@raspberrypi.com" target="_blank">naush@raspberrypi.com</a>> wrote:<br>
> <br>
> > Hi,<br>
> ><br>
> > One big issue that is plaguing libcamera users on Raspberry Pi is memory<br>
> > usage.<br>
> > All our frame buffers have to be allocated in CMA space because of hardware<br>
> > restrictions. Unfortunately CMA space is very limited (particularly on the<br>
> > Raspberry Pi Zero), and is prone to frequent fragmentation as<br>
> > display/rendering<br>
> > buffers also use this pool. Not to mention that the kernel is also allowed<br>
> > to<br>
> > allocate in the CMA pool for its own needs. As a result, some of our<br>
> > libcamera<br>
> > applications intermittently fail to run because of buffer allocation<br>
> > failure,<br>
> > and some use-cases (e.g. > 20MPix captures) will simply not be able to run<br>
> > on<br>
> > some platforms at all.<br>
> ><br>
> > This patch series attempts to try and fix this by easing the memory usage<br>
> > of<br>
> > the Raspberry Pi pipeline handler. It does this by using a platform<br>
> > configuration<br>
> > file which (amongst other things) instructs the pipeline handler on how<br>
> > many<br>
> > internal buffers it should allocate. There may be a performance penality<br>
> > (e.g<br>
> > dropped frames, or longer time taken for initial AE/AWB convergence) that<br>
> > have<br>
> > been documented. But if the application can guarantee certain behavior,<br>
> > we can<br>
> > ensure the pipeline can still process requests with the same performance<br>
> > as we<br>
> > have today. The Raspberry Pi libcamera-apps and picamera2 framework<br>
> > already use<br>
> > this behavior, so the change should be transparent to applications built on<br>
> > these, apart from using significantly less memory.<br>
> ><br>
> > Patches 1 to 4 and the framework for using a configuration file to setup<br>
> > the<br>
> > buffer allocation parameters<br>
> ><br>
> > Patch 5 to 7 improve the buffer handling dealing with dropped frames,<br>
> > allowing<br>
> > us to use fewer internal buffers.<br>
> ><br>
> <br>
> I should add that this particular change requires a firmware update to work<br>
> correctly.<br>
> If you want to test this out, it's best to run "rpi-update" to get the<br>
> latest firmware.<br>
<br>
Mostly from curiosity - is there a 'standard' way for the Raspberry Pi<br>
pipeline handler to interogate or query the rpi firmware to do a check<br>
in case it is not the correct version?<br></blockquote><div><br></div><div>Unfortunately not in an easy way.</div><div>We could run a shell command "vcgencmd version" to get a sha</div><div>hash, but I would find that really ugly to do in the pipeline hanalder.</div><div><br></div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
--<br>
Kieran<br>
<br>
> Naush<br>
> <br>
> <br>
> ><br>
> > Patch 8 adds a parameter to modify drop frame behavior - not related to<br>
> > this<br>
> > problem, but a nice to have.<br>
> ><br>
> > Patches 9 and 10 add configuration files for minimal memory configuration,<br>
> > and<br>
> > another for optimal memory usage for libcamera-apps and picamera2.<br>
> ><br>
> > I've considered how these various performance tuning parameters could be<br>
> > made<br>
> > part of the core StreamConfiguration structure or even as a Control.<br>
> > However,<br>
> > they are entirely platform specific, and I am struggling to come up with a<br>
> > clean<br>
> > way to put this into libcamera core with polluting it with various<br>
> > platform specific<br>
> > parameters. If others have any ideas or suggestions on using alternate<br>
> > means<br>
> > of passing platform specific parameters into the pipeline handler, please<br>
> > do<br>
> > comment on this review.<br>
> ><br>
> > Thanks,<br>
> > Naush<br>
> ><br>
> > Naushir Patuck (10):<br>
> > libcamera: pipeline: Add a platform configuration file helper<br>
> > pipeline: raspberrypi: Add a pipeline config structure<br>
> > pipeline: raspberrypi: Split out ISP Output0 buffer allocation<br>
> > pipeline: raspberrypi: Read config parameters from a file<br>
> > pipeline: raspberrypi: Disable StreamOn for ISP Output0/1 when<br>
> > dropping frames<br>
> > pipeline: raspberrypi: Reorder startup drop frame initialisation<br>
> > pipeline: raspberrypi: Add a parameter to disable startup drop frames<br>
> > pipeline: raspberrypi: Allow pipeline handler to always use the newest<br>
> > frame<br>
> > pipeline: raspberrypi: Add minimal memory usage config file<br>
> > pipeline: raspberrypi: Add libcamera-apps and picamera2 config file<br>
> ><br>
> > include/libcamera/internal/pipeline_handler.h | 2 +<br>
> > .../pipeline/raspberrypi/data/default.json | 28 +++<br>
> > .../pipeline/raspberrypi/data/meson.build | 10 +<br>
> > .../raspberrypi/data/minimal_mem.json | 28 +++<br>
> > .../pipeline/raspberrypi/data/rpi_apps.json | 28 +++<br>
> > .../pipeline/raspberrypi/meson.build | 2 +<br>
> > .../pipeline/raspberrypi/raspberrypi.cpp | 215 +++++++++++++++---<br>
> > src/libcamera/pipeline_handler.cpp | 57 +++++<br>
> > 8 files changed, 339 insertions(+), 31 deletions(-)<br>
> > create mode 100644 src/libcamera/pipeline/raspberrypi/data/default.json<br>
> > create mode 100644 src/libcamera/pipeline/raspberrypi/data/meson.build<br>
> > create mode 100644<br>
> > src/libcamera/pipeline/raspberrypi/data/minimal_mem.json<br>
> > create mode 100644 src/libcamera/pipeline/raspberrypi/data/rpi_apps.json<br>
> ><br>
> > --<br>
> > 2.25.1<br>
> ><br>
> ><br>
</blockquote></div></div>