[libcamera-devel] [RFC PATCH 0/7] libcamera: introduce Software ISP and Software IPA

[Andrey Konovalov]

Here is a draft implementation of Software ISP and Software IPA
which provide debayering and implementation of image processing
algorithms for systems without a hardware ISP, or in cases when
there are no public drivers for the hardware ISP present in the
system.

The implementation of the Software ISP is a reference one.
A naive AWB alorithm is implemented as part of a function which
does debayering and statistics calculations - the algorithm part
is to be moved to the IPA in the next version of the patch set.
And for debayering itself there is already a more efficient
implementation by Hans de Goede. This patch set is currently using
the earlier debayering implementation as it is less lines of code
and is OK for the initial discussion.
Only RAW10P format from the sensor is currently supported, but
other debayering functions can be easily added (which of them to
call is decided based on the input format).

The Software IPA has only auto exposure and AGC. For the AGC
the analogue gain control of the camera sensor is used (if
available). The algorithm is very much simplified, and is
mostly included as a reference code.

The 6th patch renames some variables in the simple pipeline
handler for the Software ISP to use the same buffer handling
code as the Converter currently does. This lets one to
avoid adding extra code to the pipeline handler, but also
makes the Software ISP and the Converter mutually exclusive.

The Software ISP / IPA are intended to be used with the simple
pipeline handler. The proper integration isn't done yet, and
the last patch in this series unconditionally creates the
Software ISP and Software IPA instances if the pipeline is
not using a Converter. In the future the Software ISP / IPA
would be created if the media pipeline can only produce raw
Bayer frames. And a possibility to use different implementations
of Software ISP and IPA will be added.

This patch set uses SharedMemObject class used by the RPi pipeline
handler - the second patch in the series moves the header file
to a common directory.

This patch set has been tested on Qualcomm RB5 board with
a mezzanine board equipped with RPi camera v2 (not the
standard RB5 camera mezzanine).

Andrey Konovalov (7):
  libcamera: introduce SoftwareIsp class
  libcamera: internal: Move SharedMemObject class to a common directory
  libcamera: software_isp: add SwIspLinaro implementation of SoftwareIsp
  libcamera: ipa: add Soft IPA
  libcamera: software_isp: introduce getStatsFD()
  libcamera: pipeline: simple: rename converterBuffers_ and related vars
  [DNI] libcamera: pipeline: simple: enable use of SoftwareISP and Soft
    IPA

 include/libcamera/internal/meson.build        |   3 +
 .../libcamera/internal}/shared_mem_object.h   |   4 -
 include/libcamera/internal/software_isp.h     |  71 +++
 .../internal/software_isp/meson.build         |   6 +
 .../internal/software_isp/statistics-linaro.h |  17 +
 .../internal/software_isp/swisp_linaro.h      | 111 ++++
 include/libcamera/ipa/meson.build             |   1 +
 include/libcamera/ipa/soft.mojom              |  27 +
 meson_options.txt                             |   3 +-
 src/ipa/simple/common/meson.build             |  17 +
 src/ipa/simple/common/soft_base.cpp           |  66 +++
 src/ipa/simple/common/soft_base.h             |  47 ++
 src/ipa/simple/linaro/data/meson.build        |   8 +
 src/ipa/simple/linaro/data/soft.conf          |   3 +
 src/ipa/simple/linaro/meson.build             |  26 +
 src/ipa/simple/linaro/soft_linaro.cpp         | 210 +++++++
 src/ipa/simple/meson.build                    |  12 +
 src/libcamera/meson.build                     |   2 +
 src/libcamera/pipeline/simple/simple.cpp      | 193 +++++--
 src/libcamera/software_isp.cpp                |  25 +
 src/libcamera/software_isp/meson.build        |   5 +
 src/libcamera/software_isp/swisp_linaro.cpp   | 539 ++++++++++++++++++
 22 files changed, 1339 insertions(+), 57 deletions(-)
 rename {src/libcamera/pipeline/rpi/common => include/libcamera/internal}/shared_mem_object.h (98%)
 create mode 100644 include/libcamera/internal/software_isp.h
 create mode 100644 include/libcamera/internal/software_isp/meson.build
 create mode 100644 include/libcamera/internal/software_isp/statistics-linaro.h
 create mode 100644 include/libcamera/internal/software_isp/swisp_linaro.h
 create mode 100644 include/libcamera/ipa/soft.mojom
 create mode 100644 src/ipa/simple/common/meson.build
 create mode 100644 src/ipa/simple/common/soft_base.cpp
 create mode 100644 src/ipa/simple/common/soft_base.h
 create mode 100644 src/ipa/simple/linaro/data/meson.build
 create mode 100644 src/ipa/simple/linaro/data/soft.conf
 create mode 100644 src/ipa/simple/linaro/meson.build
 create mode 100644 src/ipa/simple/linaro/soft_linaro.cpp
 create mode 100644 src/ipa/simple/meson.build
 create mode 100644 src/libcamera/software_isp.cpp
 create mode 100644 src/libcamera/software_isp/meson.build
 create mode 100644 src/libcamera/software_isp/swisp_linaro.cpp

-- 
2.34.1