[libcamera-devel] Potential integration of thermal cameras into v4l

[Sophie Tyalie]

Hi Pekka and Jacopo,

I'll not quote the whole exchange now, please tell me if this is an
anti-pattern.

On 10/01/2023 12:45, Jacopo Mondi wrote:
> On Tue, Jan 10, 2023 at 10:46:26AM +0200, Pekka Paalanen wrote:
>> Hi,
>>
>> since no-one else replied to you yet, I thought to mention my 2c
>> (I don't really do camera stuff myself so far):
>>
>> Perhaps the best place is to reach out to the libcamera community:
>> https://libcamera.org/
>>
>
> cc-ed the libcamera list

Thank you. Maybe libcamera is the better place to work on this.

> I agree it would be interesting to better understand what you mean by
> driver here.
>
> The camera seems to be a UVC camera, does it deliver frames with the
> current UVC driver or do you need kernel patches to support it ?

I'm not fully familiar with the UVC driver, but I very much doubt that
the camera speaks anything resembling the UVC protocol.

Guide build an RPC interface over USB, which they use to probe the
camera and retrieve data that is needed later on (manufacturing
calibration files, temperature curves, available modis, …).
Afterwards they tell the camera to send the raw frames on the
same channel (by sending `StartX=1`).

> I would also be interested why it needs to go through v4l2loopback..

I've currently used v4l2loopback in order to get the processed frames
from Python into the v4l2 system, so that I can access it further on.
There are probably better solutions, but it was one that I know and is
easy to work with.

An actual driver developed for v4l2 or libcamera would make use of the
standard ways of course.


>> It sounds to me like you want to do some hardware-specific
>> processing in userspace, and it might not be great to try to come
>> up with a generic thermal camera API at the kernel UAPI level.
>> That's where libcamera fits in.

This is probably true. I'm not sure if a generic thermal camera API is
even feasible, as every manufacturer does their own thing fully outside
of standards.

Together with the Guide MobIR Air, we could also implement the Flir ONE
USB camera which also already has an user space driver¹. But these two
cameras work completely differently in many regards (even conversion of
raw to thermal values).


>> As for the pixel type, maybe use a floating-point type to avoid
>> range/precision problems? E.g. DRM_FORMAT_R32F for application
>> facing API. That format does not seem to exist yet, but it's
>> trivial to add into kernel's drm_fourcc.h and should be well
>> accepted IMO.
>>
>> Thanks,
>> pq

Yeah, I would have preferred a floating point type too, but
v4l2loopback doesn't support one (yet).


>>> [1]:
>>> https://www.guideir.com/products/mobileaccessories/mobirair/data_300.html
>>> [2]: https://github.com/tyalie/pyMobirAir-v4l2/
>
> This link is broken :)

Yeah sorry. I forgot to set the Github repo to public. It should be
fixed now.

Thank you all,
- Sophie

[1]: https://github.com/fnoop/flirone-v4l2