[libcamera-devel] [RESEND] Summary meeting 2021-06-08 - libcamera IPA discussion

[Sebastian Fricke]

Hello everyone,

I just wanted to quickly gather, the points we talked about and
summarize which points we were not able to talk about.
Additionally, I have added some initial thoughts about how to go further
with some of these.

As reference, my original proposal for general statistics data can be
found here:
https://lists.libcamera.org/pipermail/libcamera-devel/2021-May/020774.html

It became quite clear that a proper test setup for automated testing is
a major requirement for this whole endeavor. So, I will switch my focus
to that topic for the following time. From my understanding this means
that we are searching for a platform to perform the following routine:
- Gather patches from the mailing list
- Apply patches to different test systems
- Test deployment
- Take test pictures in a controlled environment and store the results
   for later review
- Run unit tests/control algorithm tests
- Generate summary data and post results to the mailing list

Is this correct or am I missing something important?

I understood that this system, should be able to handle multiple test
stations (For example 1 @ Kieran, 1 @ my place, etc.).

David also mentioned that we should set up some basic testing guidelines
for the different platforms.

When we are settled with the software part of this task, we will tend
towards setting up the correct hardware in order to test all of the
current platforms under controlled conditions. Which means:
- Raspberry Pi's with imx219, imx290, imx477, ov5647, se327m12 sensors
- A few RkISP1 machines:
      + RockPi4 Model C with imx219, ov5647 sensors
      + NanoPc-T4 with ov13850, ov4689 sensors
- A few IPU3 machines (I would be happy if someone could complete this
    list as I wasn't able to identify easily, which device correspond to
    the different codenames (soraka, etc.)

@Kieran do you have most of this ready, at your test station?

Then we will need some controlled environments, my current view on this
topic contains the following items:
- Test stands
- uniform lighting equipment (lightbox)
- Boxes that enclose each setup
- A multitude of test charts (Macbeth, flatfield, texture charts,
    dynamic range charts, etc.)
- All required accessories (power, construction, etc.)

I found a few vendors for this kind of stuff:
- https://www.imatest.com/products/equipment/
- https://www.image-engineering.de/products/equipment/illumination-devices/382-modular-lightstudio

We will probably have to worry about how to finance all of this, I am
not sure how much the involved companies would like to help out, so
maybe we can think about something like crowdfunding?

-----

Besides the testing part, I briefly explained my idea and we talked
about a few difficult topics:
   - Generalizing histograms
     (What could be a common ground between the abundance of different
      variations?)

   - Generalized handling of regions in the luminance measurement for AEC/AGC
     (Especially how to handle different measuring regions and
      geometries, how to generalize metering methods etc.)

   - Generalizing the resulting ALSC matrix
     (ISPs expect different matrix sizes how to find a general matrix,
      that can be converted later on to the correct matrix for the ISP)

   - Where to store the tuning files?
     (Should it be stored in a sub-folder for each platform or should it be
      stored at a more general location)

   - Algorithm combinations
     (Can we swap out algorithms on demand or are these algorithms tuned
      for a combination with specific other algorithms)

   - Should tuning files be general enough to be used by multiple
     platforms?

   - Generalizing the camera tuning tool

A few topics were not mentioned or only briefly addressed:
   - Platforms with inline-ISP + offline-ISP or other combinations, which
     are currently not present in the supported platforms, how do we
     prepare for those systems?

   - Ownership: how to enable multiple vendors to work with the codebase
     without interfering with each other.

If I forgot something please add it to this mail. Thank you :)

It was a pleasure to have this meeting and I would very much enjoy if we
can continue this talk in the future.

Greetings,
Sebastian