[PATCH 0/9] Software ISP support for CCM

[Milan Zamazal]

This series implements application of color correction matrices in software ISP.

Support for color temperature is added to auto white balance algorithm,
this is needed to obtain the right CCM.  A new Ccm algorithm is added to
determine the matrix.  Both roughly follow what the hardware pipelines
do.

Lut algorithm is modified to incorporate the CCM and perform some
precomputations in the form of 10 lookup tables (one per each matrix
element and one for gamma).

A tricky part is to support both CCM and the original (no-CCM)
transformations.  CCM obviously adds a significant overhead; per-frame
time increases by ~45% on Debix Model A and ~85% on TI AM69 SK.  This
means CCM must be optional, which is determined by presence of Ccm
algorithm in the tuning file.

The performance critical debayering code must not include extra
conditionals and must work efficiently for both the cases.  The macros
in debayering code are refactored and restructured for this.  An added
bonus is that this also removes some annoying code duplication.  And some more, partially already present, templating is used to select between CCM and non-CCM processing.

Performance is much influenced by the data structures used in
debayering.  I tried to define something reasonable and performed only
basic performance testing.  Suggestions for possible improvements are
welcome.

See the commit messages for more details.

Milan Zamazal (9):
  libcamera: software_isp: Determine color temperature
  libcamera: software_isp: Store color temperature to metadata
  libcamera: software_isp: lut: Remove maybe_unused on a used argument
  libcamera: software_isp: Use common code to store debayered pixels
  libcamera: software_isp: Use a macro to assign debayering methods
  libcamera: software_isp: Add CCM algorithm
  libcamera: software_isp: Add an example CCM to uncalibrated.yaml
  libcamera: software_isp: Track whether CCM is enabled
  libcamera: software_isp: Apply CCM in debayering

 .../internal/software_isp/debayer_params.h    |  20 ++-
 include/libcamera/ipa/soft.mojom              |   2 +-
 src/ipa/simple/algorithms/awb.cpp             |  18 ++-
 src/ipa/simple/algorithms/ccm.cpp             |  86 ++++++++++++
 src/ipa/simple/algorithms/ccm.h               |  45 ++++++
 src/ipa/simple/algorithms/lut.cpp             |  62 +++++---
 src/ipa/simple/algorithms/lut.h               |   1 +
 src/ipa/simple/algorithms/meson.build         |   1 +
 src/ipa/simple/data/uncalibrated.yaml         |   7 +
 src/ipa/simple/ipa_context.h                  |  22 ++-
 src/ipa/simple/soft_simple.cpp                |   8 +-
 src/libcamera/software_isp/debayer.cpp        |  53 ++++++-
 src/libcamera/software_isp/debayer.h          |   3 +-
 src/libcamera/software_isp/debayer_cpu.cpp    | 132 ++++++++++--------
 src/libcamera/software_isp/debayer_cpu.h      |  31 ++--
 src/libcamera/software_isp/software_isp.cpp   |  11 +-
 16 files changed, 393 insertions(+), 109 deletions(-)
 create mode 100644 src/ipa/simple/algorithms/ccm.cpp
 create mode 100644 src/ipa/simple/algorithms/ccm.h

-- 
2.44.2