[libcamera-devel] Colour Matrix control proposal

[Laurent Pinchart]

Hi David,

On Fri, Jul 03, 2020 at 10:28:22AM +0100, David Plowman wrote:
> On Thu, 2 Jul 2020 at 23:23, Laurent Pinchart wrote:
> > On Thu, Jul 02, 2020 at 04:19:20PM +0100, David Plowman wrote:
> > > Hi everyone
> > >
> > > One thing we've overlooked is the ability to return the colour matrix
> > > used by the pipeline. This is particularly important for people
> > > wanting to process raw files themselves, as we can then arrange for it
> > > to be written into DNG files. (You might allow the matrix to be
> > > written as well as returned; I think both are fine.)
> > >
> > > I'd therefore like to propose a new control called ColourMatrix:
> >
> > Bikeshedding on the name, this is usually referred to as CCM, Colour
> > Correction Matrix. Would ColourCorrectionMatrix make sense a a control
> > name, or do you think it's unnecessary long ?
> 
> Good point. Not sure. I hate naming stuff. We usually use the term
> "CCM" too, of course. On the other hand, DNG files call this field
> "ColorMatrix" so there's precedent there also. Though that's not quite
> the same matrix so maybe ColourCorrectionMatrix is more precise in our
> case...?

I'll let you decide what you think is best. I'm sure we'll go through
all controls for a large cleanup pass before we stabilize the API
anyway, based on all the knowledge we will acquire until then.

> > > * It would hold an array of 9 floats representing a 3x3 matrix, listed
> > > in normal "reading order".
> > >
> > > * It is the cameraRGB -> sRGB matrix used by the ISP (you can find
> > > them in Raspberry Pi camera tuning files under "rpi.ccm"). To be extra
> > > clear, this is the matrix normally applied *after* white balancing,
> > > and *before* the gamma transform.
> >
> > By white balancing, I assume you mean the colour gains, as reported by
> > ColourGains ?
> 
> Correct. White balance is applied according to those "colour gains",
> then this matrix.
> 
> > Looking at the IPA implementation, a CCM is picked from the
> > configuration file based on the colour temperature (with interpolation),
> > and the saturation is then applied to it. Do you think there would be a
> > use case for letting the application set the CCM directly ? Or maybe
> > provide a matrix to replace the Y2RGB * S * RGB2Y term in
> > apply_saturation(), keeping the base CCM interpolation in the IPA ?
> 
> Yes, I think it's plausible that an application might want to set the
> CCM, though a bit like the focus question, I worry that people will
> start to drive the CCM from the application when the IPA should be
> doing it. But maybe there are specific circumstances when it's
> legitimate (e.g. "I'm in some really unusual environment and I know I
> want a particular colour matrix").

I'm thinking about machine vision applications in environment that
provide full control of the illuminants.

> I can't really see cases where you might want to set the matrix that
> is applied to the algorithm's choice of CCM, except in specific
> understandable ways (such as saturation), so direct manual setting of
> the CCM seems more plausible to me.
> 
> From our point of view that would mean adding a notion of enabling or
> disabling the automatic calculation, which is not a problem, but
> something I might be inclined to consider as a later patch. I
> certainly expect that returning the CCM to the application is a much
> more "core" use case.

Sounds good to me. Let's start by just reporting the matrix for now.

> > > I'm hoping this is relatively uncontroversial (??) so I'm happy to
> > > create a patch. If people have other ideas, I'm of course delighted to
> > > discuss and re-think.
> >
> > I think it's a good idea.

-- 
Regards,

Laurent Pinchart