[libcamera-devel] libcamera on Librem 5, phase detection auto focus

[Pavel Machek]

Hi!

> > I'm playing with camera on Librem 5. So far I added af/ae/awb support
> > to millipixels done completely in software, but I guess libcamera
> > would be more suitable place for it. If you have some hints/example
> > where to look at, it would be nice.
> 
> The target for the Librem 5 is to use the GPU to do the heavy-lifting,
> and in particular to perform CFA interpolation (a.k.a. debayering) and
> to apply the colour gains. Dorota is working on this. The, for the AE
> and AWB statistics, we should make use of the CSI bridge stats engine if
> possible. This will require some work on the driver side (this feature
> isn't implemented at the moment), and integrate that in libcamera.
> 
> We could also do all this purely in software, but I'm not sure it's
> worth it here. libcamera doesn't have the necessary infrastructure at
> the moment, as all the platforms we've worked on have an ISP or use YUV
> sensors. It could be done, but I think the effort would be better
> invested, on the Librem 5, in making use of the hardware features.

I believe it is worth the effort; CPU has enough power to do it, and
missing drivers are commonly going to be a problem. In fact, from
three devices with good cameras I have here, I don't think _any_ has
good enough drivers at the moment.

I did a quick run, millipixels takes 250% CPU, with doing preview with
debayer and scale, af/ae/awb. That's simple gtk application without
much optimalizations really. That means I still have 150% CPU to play
with.

Especially statistics are trivial to compute with tiny ammount of CPU,
you can just take samples.

Ammount of C code to implement it is not huge, either, it may be 200
LoC at the moment. Sure, tuning it for "great" results will be some
work.

I can see that GPU can do parts of job, probably with less power,
but... GPUs are just hard to work with.

> > So far I'm working with contrast detection, but samsung chip should
> > support phase detection auto focus, too. So far I hacked
> > drivers/media/i2c/s5k3l6xx.ko not to filter out those bits and took
> > photos (and looked at them in darktable), but I don't see any obvious
> > "defects" indicating presense of focus data. Do you have any pointers
> > to more information by chance?
> > 
> > I took deliberately defocused photo to make "defects" easier to see. I
> > guess I should take another one with less gain to keep other noise
> > down. Any ideas how to locate those bits?
> 
> The PDAF pixels will be located at fixed positions in the pixel array,
> documented by the sensor vendor. Some sensors will process the PDAF
> pixels and provide the processed information in the CSI-2 embedded data.
> I don't have access to the sensor datasheet, so I can't help much I'm
> afraid :-(

Yep, if someone could get datasheet from Samsung, or if someone from
Samsung is here and could provide it to me, that would be nice :-).

Best regards,
								Pavel
-- 
People of Russia, stop Putin before his war on Ukraine escalates.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
URL: <https://lists.libcamera.org/pipermail/libcamera-devel/attachments/20221121/f15a47a7/attachment.sig>