[libcamera-devel] [RFC PATCH v2 0/4] Enabling AF algorithm to get the VCM attributes from the device driver

[Kate Hsuan]

Hi David,

On Thu, Apr 21, 2022 at 4:50 PM David Plowman
<david.plowman at raspberrypi.com> wrote:
>
> Hi Kate
>
> Thanks for doing all this work on autofocus! There are still some
> questions I'm struggling with so I was wondering if I might ask you
> whether you had any opinions.
>
> On Thu, 21 Apr 2022 at 08:29, Kate Hsuan via libcamera-devel
> <libcamera-devel at lists.libcamera.org> wrote:
> >
> > Some of the algorithm, such as autofocus needs the information from the
> > device to configure the range of the search or the maximum values of the
> > configurations. Currently, this information, such as VCM attributes was
> > hardcoded in the code so it can only be used for specific hardware. If the
> > device attributes can be got through the codes, the algorithm can be
> > dynamically configured based on the device attributes got from the kernel.
>
> Yes, retrieving values from the lens driver is clearly much better
> than hard-coding them!
>
> One problem I have is that the appropriate range for the lens is
> determined not only by the lens driver but by the physical
> characteristics of the module. For example, I had a module where the
> driver advertises a range of 0 to 1023, but the module's physical
> characteristics meant that anything outside about 500 to 800 was not
> useful (either you couldn't move the lens that far, or it was beyond
> infinity etc.). Also things like the hyperfocal position would depend
> on the module too.

You are right, the current AF range is determined by the VCM spec
itself but not by a datasheet of the whole image sensor module (sensor
+ VCM).

>
> I'm not clear how we find the correct lens information. I suppose one
> might consider manipulating the device tree to change those numbers,
> but I can't escape the feeling that we need to supply some kind of
> configuration information (even just the module name) when libcamera
> starts. I imagine that this information would allow us to load the
> correct "lens helper" that would give us all this information. It
> would also convert between "application lens positions" and VCM driver
> units (applications are not supposed to know about the driver units,
> AIUI).

I think the "helper" could be a spec provider which provides a
detailed spec of the sensor+VCM module. Based on the information, the
lens information could be set to the AF algorithm. Therefore, AF
algorithm could move the lens from the minimum to the maximum step.
For example, the lens was moved from step 500 to step 800. It is
easier than translating logical lens position to physical position.

Thank you :)

>
> Would this sort of scheme work for you? Does anyone else have any
> thoughts on this?
>
> Thanks again!
>
> David
>
> >
> > This work proposed an implementation that can be used to get the device
> > attribute from the kernel and allows the algorithm has a method to get the
> > device attribute without a specific call. Therefore, for example, the AF
> > algorithm can get the necessary maximum VCM steps value to configure itself.
> >
> > During the AF algorithm configuration, the prefetched VCM steps will be
> > set to the configInfo and based on this the corresponding variable will
> > also be initiated.
> >
> > Kate Hsuan (4):
> >   ipa: ipu3: Variables to hold the maximum VCM steps
> >   libcamera: camera_lens: Get maximum VCM steps
> >   libcamera: pipeline: ipu3: Get and set VCM information
> >   ipa: ipu3: af: Remove hardcoded maximum VCM steps
> >
> >  include/libcamera/internal/camera_lens.h |  1 +
> >  include/libcamera/ipa/core.mojom         |  7 +++++++
> >  src/ipa/ipu3/algorithms/af.cpp           | 13 ++++---------
> >  src/ipa/ipu3/algorithms/af.h             |  2 ++
> >  src/libcamera/camera_lens.cpp            | 13 +++++++++++++
> >  src/libcamera/pipeline/ipu3/ipu3.cpp     | 15 +++++++++++++++
> >  6 files changed, 42 insertions(+), 9 deletions(-)
> >
> > --
> > 2.35.1
> >
>


--
BR,
Kate