[libcamera-devel] [PATCH 1/3] utils: rkisp1: Add script to extract LSC tables from Android
Laurent Pinchart
laurent.pinchart at ideasonboard.com
Tue Mar 7 15:15:29 CET 2023
Hi Jacopo,
On Tue, Mar 07, 2023 at 02:49:37PM +0100, Jacopo Mondi wrote:
> On Tue, Mar 07, 2023 at 11:49:23AM +0200, Laurent Pinchart via libcamera-devel wrote:
> > On Mon, Mar 06, 2023 at 06:24:38PM +0100, Jacopo Mondi via libcamera-devel wrote:
> > > Android ship a per-sensor configuration file in .xml format.
> >
> > s/ship/ships/
> >
> > > The .xml file contains a main <matfile> node and a <sensor> sub-node
> > > which contains an <LSC> entry. The LSC tables there contained can be
> > > re-used for libcamera, by parsing them opportunely.
> > >
> > > Add a script to utils/rkisp1/ to extract the LSC tables from Android
> > > configuration file for the RkISP1 platform, modeled after the
> > > requirements of the Rockchip closed source IQ tuning module.
> > >
> > > Compared to the Rockchip IQ LSC module the one implemented in libcamera
> > > is slightly simpler, and the parsing of the LSC table takes that into
> > > account by:
> > > - Only outputting tables for the larger found sensor resolution
> > > - Only outputting tables for "vignetting" value == 70 (ignoring the ones
> > > for vignetting values of 100)
> > >
> > > The script outputs to stdout a "LensShadingCorrection" section that
> > > can be directly pasted in a libcamera sensor configuration file.
> >
> > Could we generate a full tuning file instead ? I would imagine that
> > other tuning data could be (later) extracted, it would be nice to
> > prepare for that.
>
> I'm not sure about this.. how should the other algorithms already
> described in a config file be handled ? are we going to overwrite
> everything ?
I'd say the tool should output a full tuning file, skipping algorithms
that it doesn't handle, and then a human can pick individual algorithms
from the output file and copy&paste them to the target tuning file.
> > > Signed-off-by: Jacopo Mondi <jacopo.mondi at ideasonboard.com>
> > > ---
> > > utils/rkisp1/lsc_parse_android_config.py | 187 +++++++++++++++++++++++
> >
> > And we could already name the script in a more generic way, to hint that
> > it converts a Rockchip tuning file to a libcamera tuning file.
> >
> > Do you know if the XML format is Android-specific ?
>
> I presume is specific to what I think is the Rockchip 3A closed source library.
> The IPU3 in example, uses a different (binary) format for the sensor
> tuning paramters. So I guess is the closed source 3A module that
> defines the format.
That's my guess too.
https://gitlab.com/firefly-linux/external/camera_engine_rkaiq/-/tree/rk3588/firefly
may provide some interesting information.
> So maybe yes, "android" should not be part of the name
>
> lsc_IQ_to_libcamera.py ?
I'd go for rkisp1_aiq_to_libcamera.py (or with dashes instead of
underscores).
> > > 1 file changed, 187 insertions(+)
> > > create mode 100755 utils/rkisp1/lsc_parse_android_config.py
> > >
> > > diff --git a/utils/rkisp1/lsc_parse_android_config.py b/utils/rkisp1/lsc_parse_android_config.py
> > > new file mode 100755
> > > index 000000000000..a7c2c160319d
> > > --- /dev/null
> > > +++ b/utils/rkisp1/lsc_parse_android_config.py
> > > @@ -0,0 +1,187 @@
> > > +#!/usr/bin/env python
> > > +
> > > +# SPDX-License-Identifier: GPL-2.0-or-later
> > > +# Copyright (C) 2023, Jacopo Mondi - Ideas on Board Oy
> > > +#
> > > +# Parse Android .xml configuration file and extract the LSC tables.
> > > +#
> > > +# Print to standard output a "LensShadingCorrection" section, understandable by
> > > +# libcamera LSC algorithm, that can be pasted to the sensor configuration file.
> > > +
> > > +import argparse
> > > +import string
> > > +import sys
> > > +import re
> >
> > Alphabetical order please.
> >
> > > +import xml.etree.ElementTree as et
> > > +
> > > +
> > > +def sanitize(name):
> > > + return re.sub(r"[\n\t\s]*", "", name)
> >
> > We use single quotes for strings when there's no specific reason to do
> > otherwise.
> >
> > > +
> > > +
> > > +def split_table(table):
> > > + values = ""
> > > + for v in table.text.strip(' ').split():
> > > + values += v.strip('[').strip(']') + ", "
> > > + return values
> > > +
> > > +
> > > +def print_cell(cell):
> > > + lsc_template = string.Template(''' #${name} - ${illuminant}
> > > + - ct: ${ct}
> > > + resolution: ${res}
> > > + r: [${red}]
> > > + gr: [${greenr}]
> > > + gb: [${greenb}]
> > > + b: [${blue}]''')
> > > +
> > > + illuminant = cell.find("illumination")
> > > + ct = illuminant_to_ct(illuminant)
> > > +
> > > + template_dict = {
> > > + 'name': sanitize(cell.find("name").text),
> > > + 'illuminant': sanitize(illuminant.text),
> > > + 'ct': ct,
> > > + 'res': sanitize(cell.find("resolution").text)
> > > + }
> > > +
> > > + red_table = cell.find("LSC_SAMPLES_red")
> > > + greenr_table = cell.find("LSC_SAMPLES_greenR")
> > > + greenb_table = cell.find("LSC_SAMPLES_greenB")
> > > + blue_table = cell.find("LSC_SAMPLES_blue")
> > > +
> > > + if red_table is None or greenr_table is None or greenb_table is None or blue_table is None:
> > > + return
> > > +
> > > + template_dict['red'] = split_table(red_table)
> > > + template_dict['greenr'] = split_table(greenr_table)
> > > + template_dict['greenb'] = split_table(greenb_table)
> > > + template_dict['blue'] = split_table(blue_table)
> > > +
> > > + return lsc_template.substitute(template_dict)
> > > +
> > > +
> > > +def illuminant_to_ct(illuminant):
> > > + # Standard CIE Illiminants to Color Temperature in Kelvin
> > > + # https://en.wikipedia.org/wiki/Standard_illuminant
> > > + #
> > > + # Not included (and then ignored when parsing the configuration file):
> > > + # - "Horizon" == D50 == 5003
> > > + # - "BW" == ?
> > > + # - "PREFLASH" == ?
> > > + illuminants_dict = {
> > > + 'A': 2856,
> > > + 'D50': 5003,
> > > + 'D65': 6504,
> > > + 'D75': 7504,
> > > + 'F11_TL84': 4000,
> > > + 'F2_CWF': 4230,
> > > + }
> > > +
> > > + ill_key = sanitize(illuminant.text)
> > > + try:
> > > + ct = illuminants_dict[ill_key]
> > > + except KeyError:
> > > + return None
> > > +
> > > + return ct
> > > +
> > > +
> > > +# Make sure the cell is well formed and return it
> > > +def filter_cells(cell, res, lsc_cells):
> > > + name = cell.find("name")
> > > + resolution = cell.find("resolution")
> > > + illumination = cell.find("illumination")
> > > + vignetting = cell.find("vignetting")
> > > +
> > > + if name is None or resolution is None or \
> > > + illumination is None or vignetting is None:
> > > + return
> > > +
> > > + # Skip tables for smaller sensor resolutions
> > > + if res != sanitize(resolution.text):
> > > + return
> > > +
> > > + # Skip tables for which we don't know how to translate the illuminant value
> > > + ct = illuminant_to_ct(illumination)
> > > + if ct is None:
> > > + return
> > > +
> > > + # Only pick tables with vignetting == 70
> > > + if sanitize(vignetting.text) != "[70]":
> > > + return
> > > +
> > > + lsc_cells.append(cell)
> > > +
> > > +
> > > +# Get the "LSC" node
> > > +def find_lsc_table(root):
> > > + sensor = root.find('sensor')
> > > + if sensor is None:
> > > + print("Failed to find \"sensor\" node in config file")
> > > + raise Exception
> >
> > raise RuntimeError('Failed to find "sensor" node in config file')
> >
> > and print the message in the caller. Same below.
> >
> > > +
> > > + lsc = sensor.find('LSC')
> > > + if lsc is None:
> > > + print("Filed to find \"LSC\" node in config file")
> > > + raise Exception
> > > +
> > > + return lsc
> > > +
> > > +# libcamera LSC algorithm only operates on a single resolution.
> > > +# Find the largest sensor mode among the ones reported in the LSC tables
> > > +
> > > +
> > > +def parse_max_res(cells):
> > > + max_res = ""
> > > + max_size = 0
> > > +
> > > + for cell in cells:
> > > + resolution = sanitize(cell.find("resolution").text)
> > > + [w, h] = resolution.split('x')
> > > +
> > > + area = int(w) * int(h)
> > > + if area > max_size:
> > > + max_res = resolution
> > > +
> > > + return max_res
> > > +
> > > +
> > > +def main(argv):
> > > + # Parse command line arguments.
> > > + parser = argparse.ArgumentParser(
> > > + description='Parse Android camera configuration file to extract LSC tables')
> > > + parser.add_argument('--file', '-f', required=True,
> > > + help='Path to the Android .xml configuration file')
> >
> > It's quite traditional in command line tools to pass the input file as a
> > positional argument instead of a named argument. Up to you.
> >
> > > + args = parser.parse_args(argv[1:])
> > > +
> > > + root = et.parse(args.file).getroot()
> >
> > It would be nice to catch parse errors and print a human-readable
> > message.
> >
> > > + try:
> > > + lsc_node = find_lsc_table(root)
> > > + except Exception:
> > > + return 1
> > > +
> > > + cells = lsc_node.findall("cell")
> > > +
> > > + max_res = parse_max_res(cells)
> > > + if max_res == "":
> > > + return
> > > +
> > > + lsc_cells = []
> > > + for cell in cells:
> > > + filter_cells(cell, max_res, lsc_cells)
> > > +
> > > + lsc_section = ''' - LensShadingCorrection:
> > > + x-size: [ 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625 ]
> > > + y-size: [ 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625 ]
> > > + sets:
> > > +'''
> >
> > Given that you're generating a rkisp1 tuning file, using the YamlOutput
> > class from libtuning would make sense.
>
> If we get to add more algorithms yes, it might make sense
>
> > > +
> > > + for cell in lsc_cells:
> > > + lsc_section += print_cell(cell) + "\n"
> > > +
> > > + print(lsc_section)
> > > +
> > > +
> > > +if __name__ == '__main__':
> > > + sys.exit(main(sys.argv))
--
Regards,
Laurent Pinchart
More information about the libcamera-devel
mailing list