[libcamera-devel] Colour Matrix control proposal

[David Plowman]

Hi Laurent

Thanks for prompting some discussion!

On Thu, 2 Jul 2020 at 23:23, Laurent Pinchart
<laurent.pinchart at ideasonboard.com> wrote:
>
> Hi Dave,
>
> On Thu, Jul 02, 2020 at 04:19:20PM +0100, David Plowman wrote:
> > Hi everyone
> >
> > One thing we've overlooked is the ability to return the colour matrix
> > used by the pipeline. This is particularly important for people
> > wanting to process raw files themselves, as we can then arrange for it
> > to be written into DNG files. (You might allow the matrix to be
> > written as well as returned; I think both are fine.)
> >
> > I'd therefore like to propose a new control called ColourMatrix:
>
> Bikeshedding on the name, this is usually referred to as CCM, Colour
> Correction Matrix. Would ColourCorrectionMatrix make sense a a control
> name, or do you think it's unnecessary long ?

Good point. Not sure. I hate naming stuff. We usually use the term
"CCM" too, of course. On the other hand, DNG files call this field
"ColorMatrix" so there's precedent there also. Though that's not quite
the same matrix so maybe ColourCorrectionMatrix is more precise in our
case...?

>
> > * It would hold an array of 9 floats representing a 3x3 matrix, listed
> > in normal "reading order".
> >
> > * It is the cameraRGB -> sRGB matrix used by the ISP (you can find
> > them in Raspberry Pi camera tuning files under "rpi.ccm"). To be extra
> > clear, this is the matrix normally applied *after* white balancing,
> > and *before* the gamma transform.
>
> By white balancing, I assume you mean the colour gains, as reported by
> ColourGains ?

Correct. White balance is applied according to those "colour gains",
then this matrix.

>
> Looking at the IPA implementation, a CCM is picked from the
> configuration file based on the colour temperature (with interpolation),
> and the saturation is then applied to it. Do you think there would be a
> use case for letting the application set the CCM directly ? Or maybe
> provide a matrix to replace the Y2RGB * S * RGB2Y term in
> apply_saturation(), keeping the base CCM interpolation in the IPA ?

Yes, I think it's plausible that an application might want to set the
CCM, though a bit like the focus question, I worry that people will
start to drive the CCM from the application when the IPA should be
doing it. But maybe there are specific circumstances when it's
legitimate (e.g. "I'm in some really unusual environment and I know I
want a particular colour matrix").

I can't really see cases where you might want to set the matrix that
is applied to the algorithm's choice of CCM, except in specific
understandable ways (such as saturation), so direct manual setting of
the CCM seems more plausible to me.

>From our point of view that would mean adding a notion of enabling or
disabling the automatic calculation, which is not a problem, but
something I might be inclined to consider as a later patch. I
certainly expect that returning the CCM to the application is a much
more "core" use case.

Best regards
David

>
> > I'm hoping this is relatively uncontroversial (??) so I'm happy to
> > create a patch. If people have other ideas, I'm of course delighted to
> > discuss and re-think.
>
> I think it's a good idea.
>
> --
> Regards,
>
> Laurent Pinchart