[libcamera-devel] Questions about tonemapping

[Naushir Patuck]

Hi Paul,

Here are some initial thoughts from me.  I'm sure David will have some
views on this as well.

On Wed, 16 Jun 2021 at 09:47, <paul.elder at ideasonboard.com> wrote:

> Hello Naush and David,
>
> We are planning to add controls for tonemapping [1], and we were
> thinking to use the same controls that the android HAL uses.
>
> Something along the lines of:
>
> - TonemapMode
>     type: int32_t
>     draft: true
>     description: |
>      High-level global contrast/gamma/tonemapping control. Currently
>      identical to ANDROID_TONEMAP_MODE.
>     enum:
>       - name: TonemapModeContrastCurve
>         value: 0
>         description: |
>          Use the tonemapping curve specified in TonemapCurve* entries.
>          All color enhancement and tonemapping must be disabled, except
>          for applying the tonemapping curve specified by TonemapCurve*.
>       - name: TonemapModeFast
>         value: 1
>         description: |
>          Advanced gamma mapping and color enhancement may be applied,
>          without reducing frame rate compared to raw sensor output.
>       - name: TonemapModeHighQuality
>         value: 2
>         description: |
>          High-quality gamma mapping and color enhancement will be
>          applied, at the cost of possibly reduced frame rate compared to
>          raw sensor output.
>

I always wondered why Android had a Fast and HQ mode for tone mapping -
assuming this
is talking about global contrast enhancement, and not local which the
description implies.
All it is doing is an in -> out mapping, so I don't get why we need
different quality settings.
This may be related, but colour enhancement seems a bit vague, do you have
any more
details on that?


>       - name: TonemapModeGammaValue
>         value: 3
>         description: |
>          Use the gamma value specified in TonemapGamma to perform
>          tonemapping. All color enhancement and tonemapping must be
>          disabled, except for applying the tonemapping curve specified
>          by TonemapGamma.
>       - name: TonemapModePresetCurve
>         value: 4
>         description: |
>          Use the present tonemapping curve specified in
>          TonemapPresetCurve to perform tonemapping. All color
>          enhancement and tonemapping must be disabled, except for
>          applying the tonemapping curve specified by TonemapPresetCurve.
>
> - TonemapGamma
>     type: float
>     draft: true
>     description: |
>      Tonemapping curve to use when TonemapMode is TonemapModeGammaValue.
>      The tonemap curve will be defined by the following formula: * OUT =
>      pow(IN, 1.0 / gamma) where IN and OUT is the input pixel value
>      scaled to range [0.0, 1.0], pow is the power function and gamma is
>      the gamma value specified by this key. The same curve will be
>      applied to all color channels. The camera device may clip the input
>      gamma value to its supported range. The actual applied value will
>      be returned in result metadata. The valid range of gamma value
>      varies on different devices, but values within [1.0, 5.0] are
>      guaranteed not to be clipped. Currently identical to
>      ANDROID_TONEMAP_GAMMA.
>
> - TonemapPresetCurve
>     type: int32_t
>     draft: true
>     description: |
>      Tonemapping curve to use when TonemapMode is
>      TonemapModePresetCurve.
>     enum:
>       - name: TonemapPresetCurveSRGB
>         value: 0
>         description: [2]
>       - name: TonemapPresetCurveREC709
>         value: 1
>         description: [3]
>

This bit is very related to the ColourSpace work that David has submitted
for
review. My feeling is that these presets do not make much sense in
isolation without
specifying things like colour primaries and ranges.


>
> - TonemapCurve{Red,Green,Blue}
>     type: float
>     draft: true
>     description: |
>      An array of [ [in, out], [in, out], ... ] format to describe a
>      tonemapping curve to be applied when TonemapMode is set to
>      TonemapModeContrastCurve.
>     size: [64] (we need some way to specify the size)
>

Generally speaking, do you expect different curves for R/G/B?  There may be
a hardware limitation that only allows  a single curve, so perhaps the
description
could be updated to reflect this?


> What feedback do you have on these controls? How might you implement
> these? Is it something that could be done? Do you forsee any issues?
>

Overall, I do find it quite strange that a user can take over the gamma/ce
control
from the AGC.  I think the biggest challenge is how does this control play
well with
AGC?  Perhaps there needs to be a Auto tone-mapping mode that allows the AGC
to do what it needs to do?    Otherwise, this seems to be an easy way to
mess up the
quality of the AGC output, even though the intention may be the opposite
:-)

Another one is how this plays with Colourspace setup, where the tone
mapping is just one
part of the equation.

Regards,
Naush


>
>
> Thanks,
>
> Paul
>
> [1] https://bugs.libcamera.org/show_bug.cgi?id=56
> [2]
> https://developer.android.com/reference/images/camera2/metadata/android.tonemap.curveRed/srgb_tonemap.png
> [3]
> https://developer.android.com/reference/images/camera2/metadata/android.tonemap.curveRed/rec709_tonemap.png
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.libcamera.org/pipermail/libcamera-devel/attachments/20210616/b8d6a9fa/attachment.htm>