[libcamera-devel] [PATCH 2/2] utils: rkisp1: gen-csc-table: Perform simple rounding for inverted CSC

[Laurent Pinchart]

The CSC coefficients rounding keeps the sum of all coefficients in the
line identical. This was implemented for the RGB to YUV matrix
calculation, where the sum of luma coefficients is 1.0 and the sum of
chroma coefficients 0.0 (in full range), in order to avoid introducing
biases and overflow. This however doesn't make much sense for the YUV to
RGB components. Use simple rounding in that case.

Signed-off-by: Laurent Pinchart <laurent.pinchart at ideasonboard.com>
---
 utils/rkisp1/gen-csc-table.py | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/utils/rkisp1/gen-csc-table.py b/utils/rkisp1/gen-csc-table.py
index 934ace377e17..aca21ded2c8e 100755
--- a/utils/rkisp1/gen-csc-table.py
+++ b/utils/rkisp1/gen-csc-table.py
@@ -180,7 +180,10 @@ def main(argv):
     rounded_coeffs = []
     for line in scaled_coeffs:
         line = [coeff * (1 << precision.fractional) for coeff in line]
-        line = round_array(line)
+        if args.invert:
+            line = [round(coeff) for coeff in line]
+        else:
+            line = round_array(line)
 
         # Convert coefficients to the number of bits selected by the precision.
         # Negative values will be turned into positive integers using 2's
-- 
Regards,

Laurent Pinchart