[libcamera-devel] [PATCH] ipa: raspberrypi: Add a "scientific" tuning for the IMX477
cpixip at pixelcircus.com
cpixip at pixelcircus.com
Tue Dec 20 13:09:23 CET 2022
Hello - just a short comment below ( I hope I am doing this right - not
really familiar with this email/list-based system)
Am 19.12.2022 um 23:53 schrieb Laurent Pinchart:
> Hello,
>
> On Fri, Dec 16, 2022 at 03:09:55PM +0000, Naushir Patuck wrote:
>> On Fri, 16 Dec 2022 at 14:25, Kieran Bingham wrote:
>>> Quoting Naushir Patuck via libcamera-devel (2022-12-16 13:59:03)
>>>> Add a tuning file for the IMX477 more suited to scientific applications.
>>>> The key differences from the original tuning file are:
>>>>
>>>> - Disable ALSC block completely
>>>> - Pure rec709 gamma curve, and no contrast enhance
>>>> - New CT curve and CCMs based on the illumination spectrum of a black body
>>>> radiator up to about 3600 K and the CIE illuminant D for higher color
>>>> temperatures.
>>>>
>>>> Further details on the changes can be found at:
>>>> https://forums.raspberrypi.com/viewtopic.php?t=343449
>>>>
>>>> All credit for these changes go to Dr. Rolf Henkel.
>>> Some very interesting investigations in that thread.
>>>
>>> I'm curious if we'll end up with more configurations like this for each
>>> sensor!
>> I think that's entirely possible - and all part of the design with json config
>> files!
>>
>>> But I really like the graphs to present what the tuning files
>>> are doing comparitively. Much easier to grasp than a file of json
>>> data...
>>>
>>> No objections to this here but apart from admiring the graphs, I won't
>>> judge the data so:
>>>
>>> Acked-by: Kieran Bingham <kieran.bingham at ideasonboard.com>
>>>
>>>
>>> My only concern/comment would be how and where would we document the
>>> differences between these tuning files? Would users just be expected to
>>> try them to see ? Or would it be clear that a 'scientific' tuning file
>>> has a specific set of properties ?
>> At the end of the day, I think this is a choice for informed users. We provide
>> a sensible default config for visually pleasing images, but if users have a
>> specific application that requires something custom, we leave it to them to
>> switch to the appropriate tuning file that they know works for them.
> Overall that makes sense to me. In this particular case, I think it
> would however make sense to give applications a way to control the tone
> mapping curve at runtime (including disabling contrast enhancement)
> instead of having different curves specified in tuning files.
Indeed, it would be great to be able to control certain parameters at
runtime. There are a number of automatic algorithms working in libcamera
which presently can only be deactivated in akward ways - mostly by
editing the tuning file. That is, you need to stop and restart the
camera for such changes to take place. It would be great (for people who
know what they are doing) to be able to deactivate automatic processing
while the camera is running, for example.
> This isn't something that needs to be implemented urgently, and I'm fine
> with this patch, but if we have tuning files that differed only by their
> tone mapping curve, I'd get more annoyed.
Note that the tone mapping curve is not the most important deviation of
this new tuning file from the standard tuning file. It does help to get
slightly less trendy colors.
However, more important is probably the ct-curve, combined with new CCM
matrices. These new values vary much smoother than the original values
over the range of CCTs.
Another point: as soon as libcamera is starting to support sensors with
interchangable lenses (and the Raspberry Pi HQ camera is one of these),
the inclusion of the ALSC section in the tuning file makes it as well
necessary to have a dedicated tuning file for every lens which you are
planning to use. So maybe the addition of an info-section in the tuning
file would be a thing to consider. Such an info-section could also
include details on how the tuning file was created - something like that
is currently missing. For example, the IMX477 tuning file for the HQ
camera features ALSC-tables - but there is no information whatsoever
which specific lens was used in the calibration.
LSC is in a sense rather independent from other sections in the
processing pipeline. Maybe it even makes sense to split the monolithic
json-file into independent parts?
>>> But any / all of that could be addressed later if needed I think.
>>>
>>>> Signed-off-by: Naushir Patuck <naush at raspberrypi.com>
>>>> Signed-off-by: Rolf Henkel <cpixip at pixelcircus.com>
>>>> ---
>>>> .../raspberrypi/data/imx477_scientific.json | 482 ++++++++++++++++++
>>>> src/ipa/raspberrypi/data/meson.build | 1 +
>>>> 2 files changed, 483 insertions(+)
>>>> create mode 100755 src/ipa/raspberrypi/data/imx477_scientific.json
>>>>
>>>> diff --git a/src/ipa/raspberrypi/data/imx477_scientific.json b/src/ipa/raspberrypi/data/imx477_scientific.json
>>>> new file mode 100755
>>>> index 000000000000..17c4ed0a5e74
>>>> --- /dev/null
>>>> +++ b/src/ipa/raspberrypi/data/imx477_scientific.json
>>>> @@ -0,0 +1,482 @@
>>>> +{
>>>> + "version": 2.0,
>>>> + "target": "bcm2835",
>>>> + "algorithms": [
>>>> + {
>>>> + "rpi.black_level":
>>>> + {
>>>> + "black_level": 4096
>>>> + }
>>>> + },
>>>> + {
>>>> + "rpi.dpc": { }
>>>> + },
>>>> + {
>>>> + "rpi.lux":
>>>> + {
>>>> + "reference_shutter_speed": 27242,
>>>> + "reference_gain": 1.0,
>>>> + "reference_aperture": 1.0,
>>>> + "reference_lux": 830,
>>>> + "reference_Y": 17755
>>>> + }
>>>> + },
>>>> + {
>>>> + "rpi.noise":
>>>> + {
>>>> + "reference_constant": 0,
>>>> + "reference_slope": 2.767
>>>> + }
>>>> + },
>>>> + {
>>>> + "rpi.geq":
>>>> + {
>>>> + "offset": 204,
>>>> + "slope": 0.01078
>>>> + }
>>>> + },
>>>> + {
>>>> + "rpi.sdn": { }
>>>> + },
>>>> + {
>>>> + "rpi.awb":
>>>> + {
>>>> + "priors": [
>>>> + {
>>>> + "lux": 0,
>>>> + "prior":
>>>> + [
>>>> + 2000, 1.0,
>>>> + 3000, 0.0,
>>>> + 13000, 0.0
>>>> + ]
>>>> + },
>>>> + {
>>>> + "lux": 800,
>>>> + "prior":
>>>> + [
>>>> + 2000, 0.0,
>>>> + 6000, 2.0,
>>>> + 13000, 2.0
>>>> + ]
>>>> + },
>>>> + {
>>>> + "lux": 1500,
>>>> + "prior":
>>>> + [
>>>> + 2000, 0.0,
>>>> + 4000, 1.0,
>>>> + 6000, 6.0,
>>>> + 6500, 7.0,
>>>> + 7000, 1.0,
>>>> + 13000, 1.0
>>>> + ]
>>>> + }
>>>> + ],
>>>> + "modes":
>>>> + {
>>>> + "auto":
>>>> + {
>>>> + "lo": 2500,
>>>> + "hi": 8000
>>>> + },
>>>> + "incandescent":
>>>> + {
>>>> + "lo": 2500,
>>>> + "hi": 3000
>>>> + },
>>>> + "tungsten":
>>>> + {
>>>> + "lo": 3000,
>>>> + "hi": 3500
>>>> + },
>>>> + "fluorescent":
>>>> + {
>>>> + "lo": 4000,
>>>> + "hi": 4700
>>>> + },
>>>> + "indoor":
>>>> + {
>>>> + "lo": 3000,
>>>> + "hi": 5000
>>>> + },
>>>> + "daylight":
>>>> + {
>>>> + "lo": 5500,
>>>> + "hi": 6500
>>>> + },
>>>> + "cloudy":
>>>> + {
>>>> + "lo": 7000,
>>>> + "hi": 8600
>>>> + }
>>>> + },
>>>> + "bayes": 1,
>>>> + "ct_curve":
>>>> + [
>>>> + 2000.0, 0.6331025775790707, 0.27424225990946915,
>>>> + 2200.0, 0.5696117366212947, 0.3116091368689487,
>>>> + 2400.0, 0.5204264653110015, 0.34892179554105873,
>>>> + 2600.0, 0.48148675531667223, 0.38565229719076793,
>>>> + 2800.0, 0.450085403501908, 0.42145684622485047,
>>>> + 3000.0, 0.42436130159169017, 0.45611835670028816,
>>>> + 3200.0, 0.40300023695527337, 0.48950766215198593,
>>>> + 3400.0, 0.3850520052612984, 0.5215567075837261,
>>>> + 3600.0, 0.36981508088230314, 0.5522397906415475,
>>>> + 4100.0, 0.333468007836758, 0.5909770465167908,
>>>> + 4600.0, 0.31196097364221376, 0.6515706327327178,
>>>> + 5100.0, 0.2961860409294588, 0.7068178946570284,
>>>> + 5600.0, 0.2842607232745885, 0.7564837749584288,
>>>> + 6100.0, 0.2750265787051251, 0.8006183524920533,
>>>> + 6600.0, 0.2677057225584924, 0.8398879225373039,
>>>> + 7100.0, 0.2617955199757274, 0.8746456080032436,
>>>> + 7600.0, 0.25693714288250125, 0.905569559506562,
>>>> + 8100.0, 0.25287531441063316, 0.9331696750390895,
>>>> + 8600.0, 0.24946601483331993, 0.9576820904825795
>>>> + ],
>>>> + "sensitivity_r": 1.05,
>>>> + "sensitivity_b": 1.05,
>>>> + "transverse_pos": 0.0238,
>>>> + "transverse_neg": 0.04429,
>>>> + "coarse_step": 0.1
>>>> + }
>>>> + },
>>>> + {
>>>> + "rpi.agc":
>>>> + {
>>>> + "metering_modes":
>>>> + {
>>>> + "centre-weighted":
>>>> + {
>>>> + "weights": [ 3, 3, 3, 2, 2, 2, 2, 1, 1, 1, 1, 0, 0, 0, 0 ]
>>>> + },
>>>> + "spot":
>>>> + {
>>>> + "weights": [ 2, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ]
>>>> + },
>>>> + "matrix":
>>>> + {
>>>> + "weights": [ 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 ]
>>>> + }
>>>> + },
>>>> + "exposure_modes":
>>>> + {
>>>> + "normal":
>>>> + {
>>>> + "shutter": [ 100, 10000, 30000, 60000, 66666 ],
>>>> + "gain": [ 1.0, 2.0, 4.0, 6.0, 8.0 ]
>>>> + },
>>>> + "short":
>>>> + {
>>>> + "shutter": [ 100, 5000, 10000, 20000, 33333 ],
>>>> + "gain": [ 1.0, 2.0, 4.0, 6.0, 8.0 ]
>>>> + },
>>>> + "long":
>>>> + {
>>>> + "shutter": [ 100, 10000, 30000, 60000, 120000 ],
>>>> + "gain": [ 1.0, 2.0, 4.0, 6.0, 12.0 ]
>>>> + }
>>>> + },
>>>> + "constraint_modes":
>>>> + {
>>>> + "normal": [
>>>> + {
>>>> + "bound": "LOWER",
>>>> + "q_lo": 0.98,
>>>> + "q_hi": 1.0,
>>>> + "y_target":
>>>> + [
>>>> + 0, 0.3,
>>>> + 1000, 0.3
>>>> + ]
>>>> + }
>>>> + ],
>>>> + "highlight": [
>>>> + {
>>>> + "bound": "LOWER",
>>>> + "q_lo": 0.98,
>>>> + "q_hi": 1.0,
>>>> + "y_target":
>>>> + [
>>>> + 0, 0.3,
>>>> + 1000, 0.3
>>>> + ]
>>>> + },
>>>> + {
>>>> + "bound": "UPPER",
>>>> + "q_lo": 0.98,
>>>> + "q_hi": 1.0,
>>>> + "y_target":
>>>> + [
>>>> + 0, 0.8,
>>>> + 1000, 0.8
>>>> + ]
>>>> + }
>>>> + ],
>>>> + "shadows": [
>>>> + {
>>>> + "bound": "LOWER",
>>>> + "q_lo": 0.0,
>>>> + "q_hi": 0.5,
>>>> + "y_target":
>>>> + [
>>>> + 0, 0.17,
>>>> + 1000, 0.17
>>>> + ]
>>>> + }
>>>> + ]
>>>> + },
>>>> + "y_target":
>>>> + [
>>>> + 0, 0.16,
>>>> + 1000, 0.165,
>>>> + 10000, 0.17
>>>> + ]
>>>> + }
>>>> + },
>>>> + {
>>>> + "rpi.contrast":
>>>> + {
>>>> + "ce_enable": 0,
>>>> + "gamma_curve":
>>>> + [
>>>> + 0, 0,
>>>> + 512, 2304,
>>>> + 1024, 4608,
>>>> + 1536, 6573,
>>>> + 2048, 8401,
>>>> + 2560, 9992,
>>>> + 3072, 11418,
>>>> + 3584, 12719,
>>>> + 4096, 13922,
>>>> + 4608, 15045,
>>>> + 5120, 16103,
>>>> + 5632, 17104,
>>>> + 6144, 18056,
>>>> + 6656, 18967,
>>>> + 7168, 19839,
>>>> + 7680, 20679,
>>>> + 8192, 21488,
>>>> + 9216, 23028,
>>>> + 10240, 24477,
>>>> + 11264, 25849,
>>>> + 12288, 27154,
>>>> + 13312, 28401,
>>>> + 14336, 29597,
>>>> + 15360, 30747,
>>>> + 16384, 31856,
>>>> + 17408, 32928,
>>>> + 18432, 33966,
>>>> + 19456, 34973,
>>>> + 20480, 35952,
>>>> + 22528, 37832,
>>>> + 24576, 39621,
>>>> + 26624, 41330,
>>>> + 28672, 42969,
>>>> + 30720, 44545,
>>>> + 32768, 46065,
>>>> + 34816, 47534,
>>>> + 36864, 48956,
>>>> + 38912, 50336,
>>>> + 40960, 51677,
>>>> + 43008, 52982,
>>>> + 45056, 54253,
>>>> + 47104, 55493,
>>>> + 49152, 56704,
>>>> + 51200, 57888,
>>>> + 53248, 59046,
>>>> + 55296, 60181,
>>>> + 57344, 61292,
>>>> + 59392, 62382,
>>>> + 61440, 63452,
>>>> + 63488, 64503,
>>>> + 65535, 65535
>>>> + ]
>>>> + }
>>>> + },
>>>> + {
>>>> + "rpi.ccm":
>>>> + {
>>>> + "ccms": [
>>>> + {
>>>> + "ct": 2000,
>>>> + "ccm":
>>>> + [
>>>> + 1.5813882365848004, -0.35293683714581114, -0.27378771561617715,
>>>> + -0.4347297185453639, 1.5792631087746074, -0.12102601986382337,
>>>> + 0.2322290578987574, -1.4382672640468128, 2.1386425781770755
>>>> + ]
>>>> + },
>>>> + {
>>>> + "ct": 2200,
>>>> + "ccm":
>>>> + [
>>>> + 1.6322048484088305, -0.45932286857238486, -0.21373542690252198,
>>>> + -0.3970719209901105, 1.5877868651467202, -0.17249380832122455,
>>>> + 0.20753774825903412, -1.2660673594740142, 2.005654261091916
>>>> + ]
>>>> + },
>>>> + {
>>>> + "ct": 2400,
>>>> + "ccm":
>>>> + [
>>>> + 1.6766610071470398, -0.5447101051688111, -0.16838641107407676,
>>>> + -0.3659845183388154, 1.592223692670396, -0.2127091997471162,
>>>> + 0.1833964516767549, -1.1339155942419321, 1.9089342978542396
>>>> + ]
>>>> + },
>>>> + {
>>>> + "ct": 2600,
>>>> + "ccm":
>>>> + [
>>>> + 1.7161984340622154, -0.6152585785678794, -0.1331100845092582,
>>>> + -0.33972082628066275, 1.5944888273736966, -0.2453979465898787,
>>>> + 0.1615577497676328, -1.0298684958833109, 1.8357854177422053
>>>> + ]
>>>> + },
>>>> + {
>>>> + "ct": 2800,
>>>> + "ccm":
>>>> + [
>>>> + 1.7519307259815728, -0.6748682080165339, -0.10515169074540848,
>>>> + -0.3171703484479931, 1.5955820297498486, -0.2727395854813966,
>>>> + 0.14230870739974305, -0.9460976023551511, 1.778709391659538
>>>> + ]
>>>> + },
>>>> + {
>>>> + "ct": 3000,
>>>> + "ccm":
>>>> + [
>>>> + 1.7846716625128374, -0.7261240476375332, -0.08274697420358428,
>>>> + -0.2975654035173307, 1.5960425637021738, -0.2961043416505157,
>>>> + 0.12546426281675097, -0.8773434727076518, 1.7330356805246685
>>>> + ]
>>>> + },
>>>> + {
>>>> + "ct": 3200,
>>>> + "ccm":
>>>> + [
>>>> + 1.8150085872943436, -0.7708109672515514, -0.06469468211419174,
>>>> + -0.2803468940646277, 1.596168842967451, -0.3164044170681625,
>>>> + 0.11071494533513807, -0.8199772290209191, 1.69572135046367
>>>> + ]
>>>> + },
>>>> + {
>>>> + "ct": 3400,
>>>> + "ccm":
>>>> + [
>>>> + 1.8433668304932087, -0.8102060605062592, -0.05013485852801454,
>>>> + -0.2650934036324084, 1.5961288492969294, -0.33427554893845535,
>>>> + 0.0977478941863518, -0.7714303112098978, 1.6647070820146963
>>>> + ]
>>>> + },
>>>> + {
>>>> + "ct": 3600,
>>>> + "ccm":
>>>> + [
>>>> + 1.8700575831917468, -0.8452518300291346, -0.03842644337477299,
>>>> + -0.2514794528347016, 1.5960178299141876, -0.3501774949366156,
>>>> + 0.08628520830733245, -0.729841503339915, 1.638553343939267
>>>> + ]
>>>> + },
>>>> + {
>>>> + "ct": 4100,
>>>> + "ccm":
>>>> + [
>>>> + 1.8988700903560716, -0.8911278803351247, -0.018848644425650693,
>>>> + -0.21487101487384094, 1.599236541382614, -0.39405450457918206,
>>>> + 0.08251488056482173, -0.7178919368326191, 1.6267009056502704
>>>> + ]
>>>> + },
>>>> + {
>>>> + "ct": 4600,
>>>> + "ccm":
>>>> + [
>>>> + 1.960355191764125, -0.9624344812121991, -0.0017122408632169205,
>>>> + -0.19444620905212898, 1.5978493736948447, -0.416727638296156,
>>>> + 0.06310261513271084, -0.6483790952487849, 1.5834605477213093
>>>> + ]
>>>> + },
>>>> + {
>>>> + "ct": 5100,
>>>> + "ccm":
>>>> + [
>>>> + 2.014680536961399, -1.0195930302148566, 0.007728256612638915,
>>>> + -0.17751999660735496, 1.5977081555831, -0.4366085498741474,
>>>> + 0.04741267583041334, -0.5950327902073489, 1.5512919847321853
>>>> + ]
>>>> + },
>>>> + {
>>>> + "ct": 5600,
>>>> + "ccm":
>>>> + [
>>>> + 2.062652337917251, -1.0658386679125478, 0.011886354256281267,
>>>> + -0.16319197721451495, 1.598363237584736, -0.45422061523742235,
>>>> + 0.03465810928795378, -0.5535454108047286, 1.5269025836946852
>>>> + ]
>>>> + },
>>>> + {
>>>> + "ct": 6100,
>>>> + "ccm":
>>>> + [
>>>> + 2.104985902038069, -1.103597868736314, 0.012503517136539277,
>>>> + -0.15090797064906178, 1.5994703078166095, -0.4698414300864995,
>>>> + 0.02421766063474242, -0.5208922818196823, 1.5081270847783788
>>>> + ]
>>>> + },
>>>> + {
>>>> + "ct": 6600,
>>>> + "ccm":
>>>> + [
>>>> + 2.1424988751299714, -1.134760232367728, 0.010730356010435522,
>>>> + -0.14021846798466234, 1.600822462230719, -0.48379204794526487,
>>>> + 0.015521315410496622, -0.49463630325832275, 1.4933313534840327
>>>> + ]
>>>> + },
>>>> + {
>>>> + "ct": 7100,
>>>> + "ccm":
>>>> + [
>>>> + 2.1758034100130925, -1.1607558481037359, 0.007452724895469076,
>>>> + -0.13085694672641826, 1.6022648614493245, -0.4962330524084075,
>>>> + 0.008226943206113427, -0.4733077192319791, 1.4815336120437468
>>>> + ]
>>>> + },
>>>> + {
>>>> + "ct": 7600,
>>>> + "ccm":
>>>> + [
>>>> + 2.205529206931895, -1.1826662383072108, 0.0032019529917605167,
>>>> + -0.122572009780486, 1.6037258133595753, -0.5073973734282445,
>>>> + 0.0020132587619863425, -0.4556590236414181, 1.471939788496745
>>>> + ]
>>>> + },
>>>> + {
>>>> + "ct": 8100,
>>>> + "ccm":
>>>> + [
>>>> + 2.232224969223067, -1.2013672897252885, -0.0016234598095482985,
>>>> + -0.11518026734442414, 1.6051544769439803, -0.5174558699422255,
>>>> + -0.0033378143542219835, -0.4408590373867774, 1.4640252230667452
>>>> + ]
>>>> + },
>>>> + {
>>>> + "ct": 8600,
>>>> + "ccm":
>>>> + [
>>>> + 2.256082295891265, -1.2173210549996634, -0.0067231350481711675,
>>>> + -0.10860272839843167, 1.6065150139140594, -0.5264728573611493,
>>>> + -0.007952618707984149, -0.4284003574050791, 1.4574646927117558
>>>> + ]
>>>> + }
>>>> + ]
>>>> + }
>>>> + },
>>>> + {
>>>> + "rpi.sharpen": { }
>>>> + },
>>>> + {
>>>> + "rpi.focus": { }
>>>> + }
>>>> + ]
>>>> +}
>>>> \ No newline at end of file
>>>> diff --git a/src/ipa/raspberrypi/data/meson.build b/src/ipa/raspberrypi/data/meson.build
>>>> index 84c1dabbec9c..5565508cf602 100644
>>>> --- a/src/ipa/raspberrypi/data/meson.build
>>>> +++ b/src/ipa/raspberrypi/data/meson.build
>>>> @@ -9,6 +9,7 @@ conf_files = files([
>>>> 'imx378.json',
>>>> 'imx477.json',
>>>> 'imx477_noir.json',
>>>> + 'imx477_scientific.json',
>>>> 'imx519.json',
>>>> 'ov5647.json',
>>>> 'ov5647_noir.json',
More information about the libcamera-devel
mailing list