[libcamera-devel] libcamera on Librem 5, phase detection auto focus

[Dorota Czaplejewicz]

Hi,

On Mon, 21 Nov 2022 14:55:49 +0200
Laurent Pinchart <laurent.pinchart at ideasonboard.com> wrote:

> Hi Pavel,
> 
> (CC'ing Dorota)
> 
> On Mon, Nov 21, 2022 at 12:32:30PM +0100, Pavel Machek via libcamera-devel wrote:
> > Hi!
> > 
> > I'm playing with camera on Librem 5. So far I added af/ae/awb support
> > to millipixels done completely in software, but I guess libcamera
> > would be more suitable place for it. If you have some hints/example
> > where to look at, it would be nice.  
> 
> The target for the Librem 5 is to use the GPU to do the heavy-lifting,
> and in particular to perform CFA interpolation (a.k.a. debayering) and
> to apply the colour gains. Dorota is working on this. The, for the AE
> and AWB statistics, we should make use of the CSI bridge stats engine if
> possible. This will require some work on the driver side (this feature
> isn't implemented at the moment), and integrate that in libcamera.
> 
> We could also do all this purely in software, but I'm not sure it's
> worth it here. libcamera doesn't have the necessary infrastructure at
> the moment, as all the platforms we've worked on have an ISP or use YUV
> sensors. It could be done, but I think the effort would be better
> invested, on the Librem 5, in making use of the hardware features.
> 
Right now I'm aiming at doing the statistics on the GPU. Barring any extra snags, this will start getting integrated into my experimental branch before the end of the year.

> > So far I'm working with contrast detection, but samsung chip should
> > support phase detection auto focus, too. So far I hacked
> > drivers/media/i2c/s5k3l6xx.ko not to filter out those bits and took
> > photos (and looked at them in darktable), but I don't see any obvious
> > "defects" indicating presense of focus data. Do you have any pointers
> > to more information by chance?
> > 
> > I took deliberately defocused photo to make "defects" easier to see. I
> > guess I should take another one with less gain to keep other noise
> > down. Any ideas how to locate those bits?  
> 
> The PDAF pixels will be located at fixed positions in the pixel array,
> documented by the sensor vendor. Some sensors will process the PDAF
> pixels and provide the processed information in the CSI-2 embedded data.
> I don't have access to the sensor datasheet, so I can't help much I'm
> afraid :-(
> 
The focus pixels will appear sometimes - I haven't quite figured out when - when the resolution is full and when denoising is off. That's not described in the datasheet very well. I did prepare a bunch of scripts to mess with the sensors registers in this repo (beware, here be dragons) https://source.puri.sm/dorota.czaplejewicz/nani

Otherwise, I tried to describe the registers in the s5k3l6xx driver, inline. That driver isn't mainline btw, so only the folks at Purism have any experience with it.

The problem on the L5 is that the full resolution is too much for the CPU to handle, so the debayering/scaling must at least be done on the GPU, or the frame rate will be horrible and the entire exercise of phase-shift focusing useless.

--Dorota
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <https://lists.libcamera.org/pipermail/libcamera-devel/attachments/20221121/8c6fd45f/attachment.sig>