[PATCH v6 17/18] libcamera: software_isp: Apply black level compensation
Milan Zamazal
mzamazal at redhat.com
Mon Apr 22 13:24:56 CEST 2024
Hi,
thank you all for your comments and clarifications. Let me summarize the
discussion, with my notes merged in, before I get lost in it:
- Since our current black level and color gain operations are linear, we average
only the same color pixels and the lookup table is applied after averaging the
pixels, the current debayering implementation is correct. It doesn't matter
whether we do "average pixels -> subtract black -> multiply by color gain ->
apply gamma" or "subtract black -> multiply by color gain -> average pixels ->
apply gamma".
- This may no longer hold if we change the operations, for example:
- The optimization suggested by Hans (preprocessing while copying lines from the
input buffer). It may or may not have a significant impact on the performance
and may or may not have a significant impact on the image quality. I think we
need some experiments here. Performance on the CPU is generally challenging,
so if we can demonstrate a performance benefit, we should consider including
it (most likely configurable), if it doesn't turn the implementation into a
complete mess.
- Or if we add support for non-linear gains, as mentioned by Pavel.
- Laurent suggested adding a CCM matrix between debayering and gamma to achieve
a reasonable image quality. According to Hans, this is a heavy operation on
CPUs. Laurent still sees CPU processing as useful for experiments. I think
we can make this simply configurable (if it gets implemented).
- If we split the processing to pre-bayer and post-bayer parts, we should
probably work with uint16_t or float's, which may have impact on performance.
- Pavel couldn't get a better performance by using SIMD CPU instructions for
debayering. Applying a CCM matrix may be a different matter. Anyway, SIMD on
CPU is hard to use and may differ on architectures, so the question is whether
it's worth to invest into it.
- GPU processing should make many of these things easier and more things
possible.
- Do we already know how to map the buffers to GPU efficiently?
My conclusions are:
- The current CPU debayering computation itself is fine at the moment and we
shouldn't change it without a good reason. It can be replaced in future if
needed, once we have a better understanding of what and how we can
realistically achieve. General cleanups, like moving table computations
elsewhere, would be still useful already now.
- We can experiment with whatever mentioned above to get the better
understanding, but:
- GPU processing is clearly a priority.
- We have also other items in the TODO file! (I already work on some of them.)
- We should probably change the e-mail thread subject.
Thanks,
Milan
More information about the libcamera-devel
mailing list