[libcamera-devel] [PATCH] utils: rkisp1: gen-csc-table: Specify coefficients with full precision

[Umang Jain]

Hi Laurent,

Thank you for the patch

On 8/29/22 2:17 PM, Laurent Pinchart via libcamera-devel wrote:
> The Rec01, Rec709, Rec2020 and SMTPE 240M standards specify the Cb and
> Cr coefficients of the RGB to Y'CbCr conversion matrix using a quotient
> of two values. Use the exact same formulas instead of hardcoding the
> division results with a lower precision.
>
> Signed-off-by: Laurent Pinchart <laurent.pinchart at ideasonboard.com>

Reviewed-by: Umang Jain <umang.jain at ideasonboard.com>
> ---
>   utils/rkisp1/gen-csc-table.py | 24 ++++++++++++------------
>   1 file changed, 12 insertions(+), 12 deletions(-)
>
> diff --git a/utils/rkisp1/gen-csc-table.py b/utils/rkisp1/gen-csc-table.py
> index ca0187ec24c6..2fb702746421 100755
> --- a/utils/rkisp1/gen-csc-table.py
> +++ b/utils/rkisp1/gen-csc-table.py
> @@ -12,24 +12,24 @@ import sys
>   
>   encodings = {
>       'rec601': [
> -        [  0.2990,  0.5870,  0.1140 ],
> -        [ -0.1687, -0.3313,  0.5    ],
> -        [  0.5,    -0.4187, -0.0813 ]
> +        [  0.299,          0.587,          0.114         ],
> +        [ -0.299 / 1.772, -0.587 / 1.772,  0.886 / 1.772 ],
> +        [  0.701 / 1.402, -0.587 / 1.402, -0.114 / 1.402 ]
>       ],
>       'rec709': [
> -        [  0.2126,  0.7152,  0.0722 ],
> -        [ -0.1146, -0.3854,  0.5    ],
> -        [  0.5,    -0.4542, -0.0458 ]
> +        [  0.2126,           0.7152,           0.0722          ],
> +        [ -0.2126 / 1.8556, -0.7152 / 1.8556,  0.9278 / 1.8556 ],
> +        [  0.7874 / 1.5748, -0.7152 / 1.5748, -0.0722 / 1.5748 ]
>       ],
>       'rec2020': [
> -        [  0.2627,  0.6780,  0.0593 ],
> -        [ -0.1396, -0.3604,  0.5    ],
> -        [  0.5,    -0.4598, -0.0402 ]
> +        [  0.2627,           0.6780,           0.0593          ],
> +        [ -0.2627 / 1.8814, -0.6780 / 1.8814,  0.9407 / 1.8814 ],
> +        [  0.7373 / 1.4746, -0.6780 / 1.4746, -0.0593 / 1.4746 ],
>       ],
>       'smpte240m': [
> -        [  0.2122,  0.7013,  0.0865 ],
> -        [ -0.1161, -0.3839,  0.5    ],
> -        [  0.5,    -0.4451, -0.0549 ]
> +        [  0.2122,           0.7013,           0.0865          ],
> +        [ -0.2122 / 1.8270, -0.7013 / 1.8270,  0.9135 / 1.8270 ],
> +        [  0.7878 / 1.5756, -0.7013 / 1.5756, -0.0865 / 1.5756 ],
>       ],
>   }
>   
>
> base-commit: 3fad116f89e0d3497567043cbf6d8c49f1c102db