[libcamera-devel] Allowing the user to specify a colour temperature for WB

[David Plowman]

Hi Nick, Kieran

On Wed, 1 Feb 2023 at 13:06, Nick Hollinghurst via libcamera-devel
<libcamera-devel at lists.libcamera.org> wrote:
>
> Hi Kieran,
>
> On Wed, 1 Feb 2023 at 11:44, Kieran Bingham
> <kieran.bingham at ideasonboard.com> wrote:
> >
> > Quoting Nick Hollinghurst via libcamera-devel (2023-01-31 15:00:48)
> > > Hi all,
> > >
> > > A user has asked if they can set a fixed illuminant colour
> > > temperature, for white balance (rather than having to set fixed R and
> > > B gains, which are sensor-specific). It sounds like a reasonable
> > > request. I'd like to suggest two possible API changes for it:
> >
> > This sounds reasonable to me. Do conversions already exist on a
> > per-sensor basis in the Raspberry Pi IPA? Or would this need any
> > extension to the tuning files to support this?
>
> In the Raspberry Pi IPA, the tuning file already has a sensor-specific
> mapping from colour temperature to R and B gains.
>
>
> > > (a) Allow ColourTemperature control to be set in requests, as well as
> > > read from metadata. Its behaviour is somwhat analogous to LensPosition
> > > and AF: it can be set only when AWB is disabled (though the AWB
> > > algorithm might still need to be invoked, to handle the conversion
> > > from colour temperature to gains).
> >
> > Yes, I think like the other settings, this would simply be a 'manual'
> > mode.
>
> Advantages of option (a) are that it's simple, requires no API type
> changes (only a spec change?), and (as an added bonus) is what the
> user actually requested.

I was a bit bothered about using the colour temperature control for
setting the value, because it sometimes "works" and sometimes is
"ignored". But tying it to the AWB being disabled feels OK, and as you
say is like some other controls.

>
>
> > > (b) Add a new control to set lower and upper bounds on colour
> > > temperature, to be applied when AWB mode is AwbCustom. The lower and
> > > upper bounds may be equal. The new control is supported only where
> > > AwbCustom is supported. When AWB is disabled or the mode is not
> > > AwbCustom, the settings would be retained.
> > >
> > > Does either of those sound useful (or could there be a better approach)?
> >
> > I'm not sure of any other approach, (but haven't really looked yet,) but
> > this sounds reasonable.
> >
> > From what I know of the AWB algorithms at the moment, they try to look
> > at the stats from the colours of the image and determine what gains to
> > apply to make those 'average' out to be correct.
> >
> > Is there a defined way to map a colour temperature to those gains?
> >
> > This is where perhaps I expect there might be the tuning data calibrated
> > for each sensor? - but would that then require tuning to be done on a
> > larger sample of input colour temperatures? (That's not necesarily a
> > bad thing...?)
>
> Yes. For black-body radiation there's a definite mapping from colour
> temperature to R,B gains for "best" colour reproduction, depending on
> the sensor CFA and any IR filter. In practice, lighting can depart
> from the curve along a "green/purple" axis. In our tuning we
> interpolated between a selection of lamps which approximate black body
> or sunlight (and we've been taken to task about the "lumpy" shape of
> our curves). There's scope to improve this and extend the range, but
> it's not a prerequisite for adding the feature.
>
> The Raspberry Pi algorithm tries to estimate both colour temperature
> and a green/purple parameter, keeping close to the CT curve (David may
> correct me if this is wrong). Each AWB "mode" limits the search in
> terms of a lower and upper bound on colour temperature.
>
> That was the motivation for option (b): to make the "Custom" mode
> customizable, while generalizing the original request for fixed CT.
> ((b) would not quite be equivalent to (a) unless we do something to
> limit the green/purple parameter as well.)

Specifying the custom range sounds useful, and makes the feature much
handier than if you have to edit tuning files (and other platforms can
use it too).

So in summary I have a feeling that we should do.... both.

Thoughts?
David

>
>
> >
> > --
> > Kieran
> >
> >
> > >
> > > Regards,
> > >
> > >  Nick