[libcamera-devel] [PATCH] ipa: raspberrypi: Add a "scientific" tuning for the IMX477

[Kieran Bingham]

Quoting cpixip--- via libcamera-devel (2022-12-20 12:09:23)
> Hello - just a short comment below ( I hope I am doing this right - not 
> really familiar with this email/list-based system)
> 

Hi - yes you're doing fine ;-)


> Am 19.12.2022 um 23:53 schrieb Laurent Pinchart:
> > Hello,
> >
> > On Fri, Dec 16, 2022 at 03:09:55PM +0000, Naushir Patuck wrote:
> >> On Fri, 16 Dec 2022 at 14:25, Kieran Bingham wrote:
> >>> Quoting Naushir Patuck via libcamera-devel (2022-12-16 13:59:03)
> >>>> Add a tuning file for the IMX477 more suited to scientific applications.
> >>>> The key differences from the original tuning file are:
> >>>>
> >>>> - Disable ALSC block completely
> >>>> - Pure rec709 gamma curve, and no contrast enhance
> >>>> - New CT curve and CCMs based on the illumination spectrum of a black body
> >>>> radiator up to about 3600 K and the CIE illuminant D for higher color
> >>>> temperatures.
> >>>>
> >>>> Further details on the changes can be found at:
> >>>> https://forums.raspberrypi.com/viewtopic.php?t=343449
> >>>>
> >>>> All credit for these changes go to Dr. Rolf Henkel.
> >>> Some very interesting investigations in that thread.
> >>>
> >>> I'm curious if we'll end up with more configurations like this for each
> >>> sensor!
> >> I think that's entirely possible - and all part of the design with json config
> >> files!
> >>
> >>> But I really like the graphs to present what the tuning files
> >>> are doing comparitively. Much easier to grasp than a file of json
> >>> data...
> >>>
> >>> No objections to this here but apart from admiring the graphs, I won't
> >>> judge the data so:
> >>>
> >>> Acked-by: Kieran Bingham <kieran.bingham at ideasonboard.com>
> >>>
> >>>
> >>> My only concern/comment would be how and where would we document the
> >>> differences between these tuning files? Would users just be expected to
> >>> try them to see ? Or would it be clear that a 'scientific' tuning file
> >>> has a specific set of properties ?
> >> At the end of the day, I think this is a choice for informed users. We provide
> >> a sensible default config for visually pleasing images, but if users have a
> >> specific application that requires something custom, we leave it to them to
> >> switch to the appropriate tuning file that they know works for them.
> > Overall that makes sense to me. In this particular case, I think it
> > would however make sense to give applications a way to control the tone
> > mapping curve at runtime (including disabling contrast enhancement)
> > instead of having different curves specified in tuning files.
> 
> Indeed, it would be great to be able to control certain parameters at 
> runtime. There are a number of automatic algorithms working in libcamera 
> which presently can only be deactivated in akward ways - mostly by 
> editing the tuning file. That is, you need to stop and restart the 
> camera for such changes to take place. It would be great (for people who 
> know what they are doing) to be able to deactivate automatic processing 
> while the camera is running, for example.

I would expect this is entirely possible by providing the controls to
enable/disable automatic features, just like we do with manual
gain/manual exposure.

 
> > This isn't something that needs to be implemented urgently, and I'm fine
> > with this patch, but if we have tuning files that differed only by their
> > tone mapping curve, I'd get more annoyed.
> 
> Note that the tone mapping curve is not the most important deviation of 
> this new tuning file from the standard tuning file. It does help to get 
> slightly less trendy colors.
> 
> However, more important is probably the ct-curve, combined with new CCM 
> matrices. These new values vary much smoother than the original values 
> over the range of CCTs.
> 
> Another point: as soon as libcamera is starting to support sensors with 
> interchangable lenses (and the Raspberry Pi HQ camera is one of these), 
> the inclusion of the ALSC section in the tuning file makes it as well 
> necessary to have a dedicated tuning file for every lens which you are 
> planning to use. So maybe the addition of an info-section in the tuning 
> file would be a thing to consider. Such an info-section could also 
> include details on how the tuning file was created - something like that 
> is currently missing. For example, the IMX477 tuning file for the HQ 
> camera features ALSC-tables - but there is no information whatsoever 
> which specific lens was used in the calibration.
> 
> LSC is in a sense rather independent from other sections in the 
> processing pipeline. Maybe it even makes sense to split the monolithic 
> json-file into independent parts?

Splitting could make sense so there's a default. But either way, we need
to figure out how to make more 'module' specific tuning files.

Ultimately, I would expect a different lens to have impacts across
the whole tuning data?

--
Kieran