[libcamera-devel] Allowing the user to specify a colour temperature for WB

[Nick Hollinghurst]

Hi Kieran,

On Wed, 1 Feb 2023 at 11:44, Kieran Bingham
<kieran.bingham at ideasonboard.com> wrote:
>
> Quoting Nick Hollinghurst via libcamera-devel (2023-01-31 15:00:48)
> > Hi all,
> >
> > A user has asked if they can set a fixed illuminant colour
> > temperature, for white balance (rather than having to set fixed R and
> > B gains, which are sensor-specific). It sounds like a reasonable
> > request. I'd like to suggest two possible API changes for it:
>
> This sounds reasonable to me. Do conversions already exist on a
> per-sensor basis in the Raspberry Pi IPA? Or would this need any
> extension to the tuning files to support this?

In the Raspberry Pi IPA, the tuning file already has a sensor-specific
mapping from colour temperature to R and B gains.


> > (a) Allow ColourTemperature control to be set in requests, as well as
> > read from metadata. Its behaviour is somwhat analogous to LensPosition
> > and AF: it can be set only when AWB is disabled (though the AWB
> > algorithm might still need to be invoked, to handle the conversion
> > from colour temperature to gains).
>
> Yes, I think like the other settings, this would simply be a 'manual'
> mode.

Advantages of option (a) are that it's simple, requires no API type
changes (only a spec change?), and (as an added bonus) is what the
user actually requested.


> > (b) Add a new control to set lower and upper bounds on colour
> > temperature, to be applied when AWB mode is AwbCustom. The lower and
> > upper bounds may be equal. The new control is supported only where
> > AwbCustom is supported. When AWB is disabled or the mode is not
> > AwbCustom, the settings would be retained.
> >
> > Does either of those sound useful (or could there be a better approach)?
>
> I'm not sure of any other approach, (but haven't really looked yet,) but
> this sounds reasonable.
>
> From what I know of the AWB algorithms at the moment, they try to look
> at the stats from the colours of the image and determine what gains to
> apply to make those 'average' out to be correct.
>
> Is there a defined way to map a colour temperature to those gains?
>
> This is where perhaps I expect there might be the tuning data calibrated
> for each sensor? - but would that then require tuning to be done on a
> larger sample of input colour temperatures? (That's not necesarily a
> bad thing...?)

Yes. For black-body radiation there's a definite mapping from colour
temperature to R,B gains for "best" colour reproduction, depending on
the sensor CFA and any IR filter. In practice, lighting can depart
from the curve along a "green/purple" axis. In our tuning we
interpolated between a selection of lamps which approximate black body
or sunlight (and we've been taken to task about the "lumpy" shape of
our curves). There's scope to improve this and extend the range, but
it's not a prerequisite for adding the feature.

The Raspberry Pi algorithm tries to estimate both colour temperature
and a green/purple parameter, keeping close to the CT curve (David may
correct me if this is wrong). Each AWB "mode" limits the search in
terms of a lower and upper bound on colour temperature.

That was the motivation for option (b): to make the "Custom" mode
customizable, while generalizing the original request for fixed CT.
((b) would not quite be equivalent to (a) unless we do something to
limit the green/purple parameter as well.)


>
> --
> Kieran
>
>
> >
> > Regards,
> >
> >  Nick