[PATCH v2 00/18] libcamera: introduce Software ISP and Software IPA

[Milan Zamazal]

Hans de Goede <hdegoede at redhat.com> writes:

> Hi All,
>
> So during our last video conference about the Software ISP Laurent
> requested some docs on how the performance of the Software ISP has
> been measured so far.
>
> I have written the following patch (to be included in v3) for this:
> https://github.com/jwrdegoede/libcamera/commit/94377ed500d8bb031d312c0f88cfa5d0905fd222
>
> And since reading documentation in patch format sucks I have also
> just put the full restructured-text below.
>
> If you have time please take a look and provide feedback before
> v3 eventually gets posted with this included.

Thank you for writing it down, a couple of comments below.

> Regards,
>
> Hans
>
>
> .. SPDX-License-Identifier: CC-BY-SA-4.0
>
> .. _software-isp-benchmarking:
>
> Software ISP benchmarking
> =========================
>
> The Software ISP is paricular sensitive to performance regressions
                      ^^^^^^^^^
> therefor it is a good idea to always benchmark the Software ISP
  ^^^^^^^^

It would be useful to run a spell checker on the document.

> before and after making changes to it and ensure that there are
> no performance regressions.
>
> DebayerCpu class builtin benchmark
> ----------------------------------
>
> The DebayerCpu class has a builtin benchmark. This benchmark
> measures the time spend on processing (collecting statistics
> and debayering) only, it does not measure the time spend on
                                                     ^^^^^

spent

> capturing or outputting the frames.
>
> The builtin benchmark always runs. So this can be used by simply
> running "cam" or "qcam" with a pipeline using the Software ISP.
>
> When it runs it will skip measuring the first 30 frames to
> allow the caches and the CPU temperature (turbo-ing) to warm-up
> and then it measures 30 fps and shows the total and per frame
> processing time using an info level log message:
>
> .. code-block::
>
>    INFO Debayer debayer_cpu.cpp:907 Processed 30 frames in 244317us, 8143 us/frame
>
> To get stable measurements it is advised to disable any other processes which
> may cause significant CPU usage (e.g. disable wifi, bluetooth and browsers).
> When possible it is also advisable to disable CPU turbo-ing and
> frequency-scaling.
>
> For example when benchmarking on a Lenovo ThinkPad X1 Yoga Gen 8, with
> the charger plugged in, the CPU can be fixed to run at 2 GHz using:
>
> .. code-block::
>
>    sudo x86_energy_perf_policy --turbo-enable 0
>    sudo cpupower frequency-set -d 2GHz -u 2GHz
>
> with these settings the builtin bench reports a processing time of ~7.8ms/frame
> on this laptop for FHD SGRBG10 (unpacked) bayer data.
>
> Measuring power consumption
> ---------------------------
>
> Since the Software ISP is often used on mobile devices it is also
> important to measure power consumption and ensure that that does
> not regress.
>
> For example to measure power consumption on a Lenovo ThinkPad X1 Yoga Gen 8
> it needs to be running on battery and it should be configured with its
> platform-profile (/sys/firmware/acpi/platform_profile) set to balanced and
> with its default turbo and frequency-scaling behavior to match real world usage.
>
> Then start qcam to capture a FHD picture at 30 fps and position the qcam window
> so that it is fully visible. After this run the following command to monitor
> the power consumption:
>
> .. code-block::
>
>    watch -n 10 cat /sys/class/power_supply/BAT0/power_now /sys/class/hwmon/hwmon6/fan?_input
>
> Note this not only measures the power consumption in ųW it also monitors
> the speed of this laptop's 2 fans. This is important because depending on
> the ambient temperature the 2 fans may spin up while testing and this
> will cause an additional power consumption of approx. 0.5W messing up
> the measurement.
>
> After starting qcam + the watch command let the laptop sit without using
> it for 2 minutes for the readings to stabilize. Then check that the fans
> have not turned on and manually take a couple of consecutive power readings
> and avarage these.

I think this kind of test can be influenced by many circumstances (besides those
already mentioned the current battery state and its interpretation by the
battery firmware).  Which means it's important to measure before and after
changes alternately several times and to see whether the measurements are
stable.  Maybe it was meant to be done this way but it's not clear from the
text.

Is using a watt meter an option or another measurable external source?  At least
such high values as those cited below should be reasonably measurable and it
would allow to make measurements on devices without battery (if it is any useful
there).  Of course, the same constraints and advice apply and it may not be
measurable automatically this way.

And doesn't modern hardware provide means to get such values?

(IIRC it has been discussed at one of the meetings but I don't remember what has
been said there; I'm sure others can provide better informed comments.)

> On the example Lenovo ThinkPad X1 Yoga Gen 8 laptop this results in
> a measured power consumption of approx. 13W while running qcam versus
> approx. 4-5W while setting idle with its OLED panel on.