<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, 17 Oct 2022 at 10:30, Laurent Pinchart <<a href="mailto:laurent.pinchart@ideasonboard.com">laurent.pinchart@ideasonboard.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi Naush,<br>
<br>
On Mon, Oct 17, 2022 at 10:10:43AM +0100, Naushir Patuck via libcamera-devel wrote:<br>
> On Mon, 17 Oct 2022 at 10:00, Kieran Bingham wrote:<br>
> > Quoting Naushir Patuck via libcamera-devel (2022-10-14 15:55:46)<br>
> > > On Fri, 14 Oct 2022 at 14:18, Naushir Patuck wrote:<br>
> > > > Hi,<br>
> > > ><br>
> > > > One big issue that is plaguing libcamera users on Raspberry Pi is memory usage.<br>
> > > > All our frame buffers have to be allocated in CMA space because of hardware<br>
> > > > restrictions. Unfortunately CMA space is very limited (particularly on the<br>
> > > > Raspberry Pi Zero), and is prone to frequent fragmentation as display/rendering<br>
> > > > buffers also use this pool. Not to mention that the kernel is also allowed to<br>
> > > > allocate in the CMA pool for its own needs. As a result, some of our libcamera<br>
> > > > applications intermittently fail to run because of buffer allocation failure,<br>
> > > > and some use-cases (e.g. > 20MPix captures) will simply not be able to run on<br>
> > > > some platforms at all.<br>
> > > ><br>
> > > > This patch series attempts to try and fix this by easing the memory usage of<br>
> > > > the Raspberry Pi pipeline handler. It does this by using a platform configuration<br>
> > > > file which (amongst other things) instructs the pipeline handler on how many<br>
> > > > internal buffers it should allocate. There may be a performance penality (e.g<br>
> > > > dropped frames, or longer time taken for initial AE/AWB convergence) that have<br>
> > > > been documented. But if the application can guarantee certain behavior, we can<br>
> > > > ensure the pipeline can still process requests with the same performance as we<br>
> > > > have today. The Raspberry Pi libcamera-apps and picamera2 framework already use<br>
> > > > this behavior, so the change should be transparent to applications built on<br>
> > > > these, apart from using significantly less memory.<br>
> > > ><br>
> > > > Patches 1 to 4 and the framework for using a configuration file to setup the<br>
> > > > buffer allocation parameters<br>
> > > ><br>
> > > > Patch 5 to 7 improve the buffer handling dealing with dropped frames, allowing<br>
> > > > us to use fewer internal buffers.<br>
> > ><br>
> > > I should add that this particular change requires a firmware update to work correctly.<br>
> > > If you want to test this out, it's best to run "rpi-update" to get the<br>
> > > latest firmware.<br>
> ><br>
> > Mostly from curiosity - is there a 'standard' way for the Raspberry Pi<br>
> > pipeline handler to interogate or query the rpi firmware to do a check<br>
> > in case it is not the correct version?<br>
> <br>
> Unfortunately not in an easy way.<br>
> We could run a shell command "vcgencmd version" to get a sha<br>
> hash, but I would find that really ugly to do in the pipeline hanalder.<br>
<br>
Not great indeed.<br>
<br>
Is this something we can improve going forward ? I'm thinking of a<br>
message to query the vc4 firmware version at runtime, that the ISP<br>
driver would use to retrieve the version. We could then expose it to<br>
userspace through the ISP video nodes.<br>
<br>
Is there a "traditional" version number for the firmware, or just a git<br>
sha1 ?<br></blockquote><div><br></div><div>We don't have a traditional version number available, only the sha1 hash.</div><div>I doubt this is something that folks over here will have an appetite to change.</div><div><br></div><div>However, things are not really that bad for the regular user. Our apt dependency</div><div>setup will ensure that if people get the latest libcamera which requires a firmware</div><div>change, the firmware blob will also be updated. This only really hits folks who</div><div>do manual builds of libcamera.</div><div><br></div><div>Naush</div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
> > > > Patch 8 adds a parameter to modify drop frame behavior - not related to this<br>
> > > > problem, but a nice to have.<br>
> > > ><br>
> > > > Patches 9 and 10 add configuration files for minimal memory configuration, and<br>
> > > > another for optimal memory usage for libcamera-apps and picamera2.<br>
> > > ><br>
> > > > I've considered how these various performance tuning parameters could be made<br>
> > > > part of the core StreamConfiguration structure or even as a Control. However,<br>
> > > > they are entirely platform specific, and I am struggling to come up with a clean<br>
> > > > way to put this into libcamera core with polluting it with various platform specific<br>
> > > > parameters. If others have any ideas or suggestions on using alternate means<br>
> > > > of passing platform specific parameters into the pipeline handler, please do<br>
> > > > comment on this review.<br>
> > > ><br>
> > > > Thanks,<br>
> > > > Naush<br>
> > > ><br>
> > > > Naushir Patuck (10):<br>
> > > > libcamera: pipeline: Add a platform configuration file helper<br>
> > > > pipeline: raspberrypi: Add a pipeline config structure<br>
> > > > pipeline: raspberrypi: Split out ISP Output0 buffer allocation<br>
> > > > pipeline: raspberrypi: Read config parameters from a file<br>
> > > > pipeline: raspberrypi: Disable StreamOn for ISP Output0/1 when<br>
> > > > dropping frames<br>
> > > > pipeline: raspberrypi: Reorder startup drop frame initialisation<br>
> > > > pipeline: raspberrypi: Add a parameter to disable startup drop frames<br>
> > > > pipeline: raspberrypi: Allow pipeline handler to always use the newest<br>
> > > > frame<br>
> > > > pipeline: raspberrypi: Add minimal memory usage config file<br>
> > > > pipeline: raspberrypi: Add libcamera-apps and picamera2 config file<br>
> > > ><br>
> > > > include/libcamera/internal/pipeline_handler.h | 2 +<br>
> > > > .../pipeline/raspberrypi/data/default.json | 28 +++<br>
> > > > .../pipeline/raspberrypi/data/meson.build | 10 +<br>
> > > > .../raspberrypi/data/minimal_mem.json | 28 +++<br>
> > > > .../pipeline/raspberrypi/data/rpi_apps.json | 28 +++<br>
> > > > .../pipeline/raspberrypi/meson.build | 2 +<br>
> > > > .../pipeline/raspberrypi/raspberrypi.cpp | 215 +++++++++++++++---<br>
> > > > src/libcamera/pipeline_handler.cpp | 57 +++++<br>
> > > > 8 files changed, 339 insertions(+), 31 deletions(-)<br>
> > > > create mode 100644 src/libcamera/pipeline/raspberrypi/data/default.json<br>
> > > > create mode 100644 src/libcamera/pipeline/raspberrypi/data/meson.build<br>
> > > > create mode 100644 src/libcamera/pipeline/raspberrypi/data/minimal_mem.json<br>
> > > > create mode 100644 src/libcamera/pipeline/raspberrypi/data/rpi_apps.json<br>
<br>
-- <br>
Regards,<br>
<br>
Laurent Pinchart<br>
</blockquote></div></div>