[libcamera-devel] softISP for libcamera

[Andrey Konovalov]

Hi Pavel!

On 05.12.2023 16:46, Pavel Machek wrote:
> Hi!
> 
>> I forgot to add the README.soft-isp.txt, in particular the command to setup the build:
>>
>> $ meson setup -Dprefix=$INSTALL_DIR -Dpipelines=simple/linaro -Dipas=simple/linaro $BUILD_DIR
>> $ ninja -C $BUILD_DIR
>> $ ninja -C $BUILD_DIR install
>>
>> With "-Dpipelines=simple/linaro -Dipas=simple/linaro" the libcamera/ipa/soft_ipa_interface.h
>> will be generated OK.
> 
> Thank you, that got me something working.
> 
> I believe modifying the code to support bayer8 was fairly easy (but I
> did not look at the results yet).
> 
> Unfortunately processing is quite slow:
> 
> 58747.865896 (0.57 fps) cam0-stream0 seq: 000016 bytesused: 2752812
> 58750.104232 (0.45 fps) cam0-stream0 seq: 000035 bytesused: 2752812
> 58752.342576 (0.45 fps) cam0-stream0 seq: 000054 bytesused: 2752812
> 58754.580913 (0.45 fps) cam0-stream0 seq: 000073 bytesused: 2752812
> 58756.819244 (0.45 fps) cam0-stream0 seq: 000092 bytesused: 2752812
> 58759.057586 (0.45 fps) cam0-stream0 seq: 000111 bytesused: 2752812

On a sm8250 board (pre-release version of RB5) with imx219 camera I am getting:

1) <snip>cam -c1 -C --stream pixelformat=RGB888,width=3278,height=2462

548.839035 (5.00 fps) cam0-stream0 seq: 000003 bytesused: 24211308
549.039002 (5.00 fps) cam0-stream0 seq: 000004 bytesused: 24211308
549.238953 (5.00 fps) cam0-stream0 seq: 000005 bytesused: 24211308
549.505582 (3.75 fps) cam0-stream0 seq: 000006 bytesused: 24211308
549.705541 (5.00 fps) cam0-stream0 seq: 000007 bytesused: 24211308
549.905508 (5.00 fps) cam0-stream0 seq: 000008 bytesused: 24211308
550.105495 (5.00 fps) cam0-stream0 seq: 000009 bytesused: 24211308

2) <snip>cam -c1 -C --stream pixelformat=RGB888,width=1918,height=1078

516.629323 (30.02 fps) cam0-stream0 seq: 000007 bytesused: 6202812
516.696066 (14.98 fps) cam0-stream0 seq: 000008 bytesused: 6202812
516.762734 (15.00 fps) cam0-stream0 seq: 000009 bytesused: 6202812
516.795796 (30.25 fps) cam0-stream0 seq: 000010 bytesused: 6202812
516.862703 (14.95 fps) cam0-stream0 seq: 000011 bytesused: 6202812
516.929254 (15.03 fps) cam0-stream0 seq: 000012 bytesused: 6202812
516.962579 (30.01 fps) cam0-stream0 seq: 000013 bytesused: 6202812
517.029238 (15.00 fps) cam0-stream0 seq: 000014 bytesused: 6202812

3) <snip>cam -c1 -C --stream pixelformat=RGB888,width=638,height=478

428.020526 (29.34 fps) cam0-stream0 seq: 000004 bytesused: 914892
428.053850 (30.01 fps) cam0-stream0 seq: 000005 bytesused: 914892
428.086536 (30.59 fps) cam0-stream0 seq: 000006 bytesused: 914892
428.119836 (30.03 fps) cam0-stream0 seq: 000007 bytesused: 914892
428.153190 (29.98 fps) cam0-stream0 seq: 000008 bytesused: 914892
428.187158 (29.44 fps) cam0-stream0 seq: 000009 bytesused: 914892
428.219844 (30.59 fps) cam0-stream0 seq: 000010 bytesused: 914892

That said, debayering and statistics calculation on CPU aren't very fast.
But your numbers still look too low for me...

The statistics calculations for AWB aren't optimized in the current
patch set (use all the pixels instead of a subset of them), but even
dropping the stats calculations completely could improve the fps by
a factor of two or so.

> Am I right that you only use top 8 bits from the bayer10 data?

Yes, this is correct.

Thanks,
Andrey

> Best regards,
> 								Pavel