[libcamera-devel] Colour spaces (again!)

[David Plowman]

Hi Kieran, Jacopo and everyone

Thanks for the recent reviews of the colour space patches. I wanted to
leave aside the "coding details" for the moment and discuss the more
fundamental ideas that were raised.


One ColorSpace field or two?

Kieran highlighted this, and it's a good question. I went with two
ColorSpace fields because I was uncomfortable with the idea that
driver layers (we only have V4L2 at the moment) might be able to use
colour spaces that we haven't seen before.

Again to pick up Kieran's question, colour spaces are not like pixel
formats because there could be an infinite number of variations,
though in practice it's doubtful we'll ever see more than a handful. I
think we have a couple of options.

1. Allow validate() to return an "undefined" ColorSpace. This simply
means the driver has chosen a colour space for which the ColorSpace
class no entry. At the same time I wanted to disallow applications
from requesting this "undefined" colour space because the idea is
meaningless, so it is banned.

This led me to an implementation where the StreamConfiguration has two
fields, one the "colour space you want" (where "undefined" is not
allowed), and one being "the colour space you'll get", where getting
back something you don't understand is possible.

2. Or I think we could do away with the "undefined" return value and
insist on a representation for every colour space we might ever
encounter. For the moment, covering the V4L2 ones would do. If we ever
got to a stage where alternative non-V4L2 driver layers use other
colour spaces, then would have to add them to the ColorSpace class.


Should colour spaces be optional?

Jacopo raised this, prompted at least in part by an earlier version of
my patches where I allowed this, using std::optional.

I've actually moved fairly strongly away from this. The more time I've
spent with this issue the more convinced I am that letting
users/applications "not care" about colour spaces is a bad idea. People
tend to start thinking that "colour spaces are optional", but this is false -
you get some colour space or other whether you choose one or not.

Actually I think it contributes to the general confusion about colour
spaces, and letting folks accept "whatever the driver chooses" is a
recipe for getting the wrong colour space much of the time. This is
especially harmful because it's a problem you simply don't notice. So
I think that:

1. You must say what colour space you want. Letting
generateConfiguration() choose JPEG for stills and Rec.709 for video
is a reasonable start, and trivial to do.

2. Colour space problems need to be made obvious, so that you know
there is "something you need to investigate". Failing completely seems
harsh, but allowing people simply to ignore problems and store them up
for later feels counterproductive.

Finally, I want to stress that colour spaces have to be correct, even
though we get lulled into a false sense of security by the fact that
you can generally ignore them and things "seem OK" - but they are not.


So thanks to everyone for all the debate. I'd very much like to get
these points settled - as you can tell, I think support for colour
spaces is long overdue now!

Thanks and best regards

David