[libcamera-devel] [PATCH v2 2/2] ipa: rpi: vc4: data: Update tuning files for HDR
David Plowman
david.plowman at raspberrypi.com
Mon Sep 11 17:35:13 CEST 2023
Hi Jacopo
On Wed, 30 Aug 2023 at 10:40, Jacopo Mondi
<jacopo.mondi at ideasonboard.com> wrote:
>
> Hi David
>
> On Wed, Aug 23, 2023 at 03:49:25PM +0100, David Plowman via libcamera-devel wrote:
> > All the Raspberry Pi official camera tuning files are updated for
> > HDR. As stated previously, there is no mechanism in the hardware for
> > combining images so all this does is enable multi-channel AGC to
> > produce short and long exposure frames. It will be up to the
> > application to deal with them.
> >
> > The changes are identical in every tuning file.
> >
> > 1. The existing AGC tuning is duplicated twice so that we have 3 AGC
> > channels.
> >
> > 2. The first is left alone (the default AGC channel), the second is
> > tweaked to under-expose significantly (ev -3) and the final one is
> > tweaked to over-exposure slightly (ev +0.5)
> >
> > 3. Control parameters are provided to the "rpi.hdr" algorithm to
> > associate these AGC channels correctly with the HDR modes.
> >
> > Signed-off-by: David Plowman <david.plowman at raspberrypi.com>
>
> Before reviewing the algorithm implementation itself let me ask a few
> question on the configuration file
Of course...!
>
> > ---
> > src/ipa/rpi/vc4/data/imx219.json | 384 +++++++++++++++-----
> > src/ipa/rpi/vc4/data/imx219_noir.json | 384 +++++++++++++++-----
> > src/ipa/rpi/vc4/data/imx477.json | 376 +++++++++++++++-----
> > src/ipa/rpi/vc4/data/imx477_noir.json | 384 +++++++++++++++-----
> > src/ipa/rpi/vc4/data/imx708.json | 346 +++++++++++++-----
> > src/ipa/rpi/vc4/data/imx708_noir.json | 346 +++++++++++++-----
> > src/ipa/rpi/vc4/data/imx708_wide.json | 346 +++++++++++++-----
> > src/ipa/rpi/vc4/data/imx708_wide_noir.json | 346 +++++++++++++-----
> > src/ipa/rpi/vc4/data/ov5647.json | 388 ++++++++++++++++-----
> > 9 files changed, 2524 insertions(+), 776 deletions(-)
> >
> > diff --git a/src/ipa/rpi/vc4/data/imx219.json b/src/ipa/rpi/vc4/data/imx219.json
> > index e8fce164..54defc0b 100644
> > --- a/src/ipa/rpi/vc4/data/imx219.json
> > +++ b/src/ipa/rpi/vc4/data/imx219.json
> > @@ -131,95 +131,282 @@
> > {
> > "rpi.agc":
> > {
>
> [snip]
>
> > + "channels":
> > + [
> > + {
>
> an index won't hurt to easly connect it to entries in the hdr section
I wasn't quite sure what you meant here. Are you suggesting including
an index number here that identifies it as libcamera's "long" (or
"short") channel? That's certainly possible, though I've tried to keep
HDR and AGC channels as separate concepts, the idea being that AGC
channels know nothing about HDR, but are merely something HDR will
use. Of course, in reality they are quite closely connected... though
I wouldn't want to predict exactly how in all cases. (Or maybe I've
misunderstood the question, in which case apologies!)
>
> > + "metering_modes":
> > + {
> > + "centre-weighted":
> > + {
> > + "weights": [ 3, 3, 3, 2, 2, 2, 2, 1, 1, 1, 1, 0, 0, 0, 0 ]
> > + },
> > + "spot":
> > + {
> > + "weights": [ 2, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ]
> > + },
> > + "matrix":
> > + {
> > + "weights": [ 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 ]
> > + }
> > + },
> > + "exposure_modes":
> > + {
> > + "normal":
> > + {
> > + "shutter": [ 100, 10000, 30000, 60000, 66666 ],
> > + "gain": [ 1.0, 2.0, 4.0, 6.0, 8.0 ]
> > + },
> > + "short":
> > + {
> > + "shutter": [ 100, 5000, 10000, 20000, 33333 ],
> > + "gain": [ 1.0, 2.0, 4.0, 6.0, 8.0 ]
> > + },
> > + "long":
> > + {
> > + "shutter": [ 100, 10000, 30000, 60000, 120000 ],
> > + "gain": [ 1.0, 2.0, 4.0, 6.0, 12.0 ]
> > + }
> > + },
> > + "constraint_modes":
> > + {
> > + "normal": [
> > + {
> > + "bound": "LOWER",
> > + "q_lo": 0.98,
> > + "q_hi": 1.0,
> > + "y_target":
> > + [
> > + 0, 0.5,
> > + 1000, 0.5
> > + ]
> > + }
> > + ],
> > + "highlight": [
> > + {
> > + "bound": "LOWER",
> > + "q_lo": 0.98,
> > + "q_hi": 1.0,
> > + "y_target":
> > + [
> > + 0, 0.5,
> > + 1000, 0.5
> > + ]
> > + },
> > + {
> > + "bound": "UPPER",
> > + "q_lo": 0.98,
> > + "q_hi": 1.0,
> > + "y_target":
> > + [
> > + 0, 0.8,
> > + 1000, 0.8
> > + ]
> > + }
> > + ],
> > + "shadows": [
> > + {
> > + "bound": "LOWER",
> > + "q_lo": 0.0,
> > + "q_hi": 0.5,
> > + "y_target":
> > + [
> > + 0, 0.17,
> > + 1000, 0.17
> > + ]
> > + }
> > + ]
> > + },
> > + "y_target":
> > + [
> > + 0, 0.16,
> > + 1000, 0.165,
> > + 10000, 0.17
> > + ]
> > + },
> > + {
> > + "base_ev": 0.125,
>
> I guess this is what makes the channel "long exposure"
Correct. (Actually "short exposure" in this case as it will be running
at 1/8 of normal exposure!)
>
> > + "metering_modes":
> > + {
> > + "centre-weighted":
> > + {
> > + "weights": [ 3, 3, 3, 2, 2, 2, 2, 1, 1, 1, 1, 0, 0, 0, 0 ]
> > + },
> > + "spot":
> > + {
> > + "weights": [ 2, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ]
> > + },
> > + "matrix":
> > + {
> > + "weights": [ 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 ]
> > + }
> > + },
> > + "exposure_modes":
> > + {
> > + "normal":
> > + {
> > + "shutter": [ 100, 10000, 30000, 60000, 66666 ],
> > + "gain": [ 1.0, 2.0, 4.0, 6.0, 8.0 ]
> > + },
> > + "short":
> > + {
> > + "shutter": [ 100, 5000, 10000, 20000, 33333 ],
> > + "gain": [ 1.0, 2.0, 4.0, 6.0, 8.0 ]
> > + },
> > + "long":
> > + {
> > + "shutter": [ 100, 10000, 30000, 60000, 120000 ],
> > + "gain": [ 1.0, 2.0, 4.0, 6.0, 12.0 ]
> > + }
> > + },
> > + "constraint_modes":
> > + {
> > + "normal": [
> > + {
> > + "bound": "LOWER",
> > + "q_lo": 0.98,
> > + "q_hi": 1.0,
> > + "y_target":
> > + [
> > + 0, 0.5,
> > + 1000, 0.5
> > + ]
> > + }
> > + ],
> > + "highlight": [
> > + {
> > + "bound": "LOWER",
> > + "q_lo": 0.98,
> > + "q_hi": 1.0,
> > + "y_target":
> > + [
> > + 0, 0.5,
> > + 1000, 0.5
> > + ]
> > + },
> > + {
> > + "bound": "UPPER",
> > + "q_lo": 0.98,
> > + "q_hi": 1.0,
> > + "y_target":
> > + [
> > + 0, 0.8,
> > + 1000, 0.8
> > + ]
> > + }
> > + ],
> > + "shadows": [
> > + {
> > + "bound": "LOWER",
> > + "q_lo": 0.0,
> > + "q_hi": 0.5,
> > + "y_target":
> > + [
> > + 0, 0.17,
> > + 1000, 0.17
> > + ]
> > + }
> > + ]
> > + },
> > + "y_target":
> > + [
> > + 0, 0.16,
> > + 1000, 0.165,
> > + 10000, 0.17
> > + ]
> > + },
> > + {
> > + "base_ev": 1.5,
> > + "metering_modes":
> > + {
> > + "centre-weighted":
> > + {
> > + "weights": [ 3, 3, 3, 2, 2, 2, 2, 1, 1, 1, 1, 0, 0, 0, 0 ]
> > + },
> > + "spot":
> > + {
> > + "weights": [ 2, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ]
> > + },
> > + "matrix":
> > + {
> > + "weights": [ 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 ]
> > + }
> > + },
> > + "exposure_modes":
> > + {
> > + "normal":
> > + {
> > + "shutter": [ 100, 10000, 30000, 60000, 66666 ],
> > + "gain": [ 1.0, 2.0, 4.0, 6.0, 8.0 ]
> > + },
> > + "short":
> > + {
> > + "shutter": [ 100, 5000, 10000, 20000, 33333 ],
> > + "gain": [ 1.0, 2.0, 4.0, 6.0, 8.0 ]
> > + },
> > + "long":
> > + {
> > + "shutter": [ 100, 10000, 30000, 60000, 120000 ],
> > + "gain": [ 1.0, 2.0, 4.0, 6.0, 12.0 ]
> > + }
> > + },
> > + "constraint_modes":
> > + {
> > + "normal": [
> > + {
> > + "bound": "LOWER",
> > + "q_lo": 0.98,
> > + "q_hi": 1.0,
> > + "y_target":
> > + [
> > + 0, 0.5,
> > + 1000, 0.5
> > + ]
> > + }
> > + ],
> > + "highlight": [
> > + {
> > + "bound": "LOWER",
> > + "q_lo": 0.98,
> > + "q_hi": 1.0,
> > + "y_target":
> > + [
> > + 0, 0.5,
> > + 1000, 0.5
> > + ]
> > + },
> > + {
> > + "bound": "UPPER",
> > + "q_lo": 0.98,
> > + "q_hi": 1.0,
> > + "y_target":
> > + [
> > + 0, 0.8,
> > + 1000, 0.8
> > + ]
> > + }
> > + ],
> > + "shadows": [
> > + {
> > + "bound": "LOWER",
> > + "q_lo": 0.0,
> > + "q_hi": 0.5,
> > + "y_target":
> > + [
> > + 0, 0.17,
> > + 1000, 0.17
> > + ]
> > + }
> > + ]
> > + },
> > + "y_target":
> > + [
> > + 0, 0.16,
> > + 1000, 0.165,
> > + 10000, 0.17
> > + ]
> > + }
> > + ]
> > + }
> > },
> > {
> > "rpi.alsc":
> > @@ -463,6 +650,21 @@
> > },
> > {
> > "rpi.sharpen": { }
> > - }
> > + },
> > + {
> > + "rpi.hdr":
> > + {
>
>
> > + "MultiExposure":
> > + {
> > + "cadence": [ 1, 2 ],
> > + "channel_map": { "short": 1, "long": 2 }
> > + },
> > + "SingleExposure":
> > + {
> > + "cadence": [ 1 ],
> > + "channel_map": { "short": 1 }
> > + }
>
> Isn't the single exposure mode comparable to the non-hdr mode ? Do we
> need to specify a channel to use in this case, can't the AEGC
> algorithm be driven as usual ?
Not necessarily. I could imagine a platform where the "single
exposure" method uses under-exposed images and is then followed by
some magic image accumulation and tone-mapping. But obviously we can't
do that in hardware on a Pi, you'd have to use some kind of software
post-processing.
Thanks!
David
>
> [snip]
>
More information about the libcamera-devel
mailing list