[libcamera-devel] [RFC 4/4] libcamera python bindings

[Laurent Pinchart]

On Sun, Oct 04, 2020 at 10:55:39PM +0300, Tomi Valkeinen wrote:
> On 04/10/2020 22:46, Laurent Pinchart wrote:
> > Hi Tomi,
> > 
> > On Sun, Oct 04, 2020 at 10:42:33PM +0300, Tomi Valkeinen wrote:
> >> On 04/10/2020 21:49, Laurent Pinchart wrote:
> >>> On Fri, Sep 18, 2020 at 06:20:19PM +0300, Tomi Valkeinen wrote:
> >>>> Main issues:
> >>>>
> >>>> - Memory management in general. Who owns the object, how to pass
> >>>>     ownership, etc.
> >>>>
> >>>> - Specifically, Request is currently broken. We can't, afaik, pass
> >>>>     ownership around. So currently Python never frees a Request, and if
> >>>>     the Request is not given to Camera::queueRequest, it will leak.
> >>>>
> >>>> - The forced threading causes some headache. Need to take care to use
> >>>>     gil_scoped_release when C++ context can invoke callbacks, and
> >>>>     gil_scoped_acquire at the invoke wrapper.
> >>>
> >>> I've given this a thought, and I wonder if we could solve it by using an
> >>> event queue and an eventfd to move everything to the Python thread.
> >>> Signals would be connected to internal functions that would push a new
> >>> instance of a message object that binds the signal name and its
> >>> arguments. An eventfd would be used to wake up the python side, which
> >>> would call a process() function that would then dispatch the messages to
> >>> Python code.
> >>
> >> Yeah, at least it would be an interesting comparison case. Curiously,
> >> things seem to work very well. I guess CPython frees the lock quite
> >> often without me specifically doing anything. To be honest, I don't know
> >> why it works.
> >>
> >> That said, I had had deadlock issues previously, and I recently got more
> >> of them when I tried things with a python thread (doing background work
> >> while running interactive python shell).
> >>
> >> I think it may not be worth spending time with eventfd until it's clear
> >> it will give some benefit, and at the moment it's not quite clear to me.
> >>
> >> I can always move the execution from libcamera's thread to my thread in
> >> one way or another. The main question is, will the GIL be freed often
> >> enough to allow the callbacks to be called. Maybe it is freed often, as
> >> I guess that's what python's own threads need also.
> > 
> > Blocking the callbacks will not only delay their delivery to the
> > application, but also potentially break the realtime requirements of
> > pipeline handlers. I think we should really not be doing so.
> 
> Realtime requirement sounds scary. Can you elaborate? Wouldn't a slow 
> application side normally just cause the capture side to re-use the 
> current buffer and overwrite it?

It's not just about image capture for applications, but all the
algorithms that run behind the scenes that consume statistics and
produce parameters for the sensor and ISP. Control loops don't like
delays.

> If it's a requirement that the callbacks may not block for any 
> noticeable time, then I agree, this needs some redesign. But I could 
> probably just add the eventfd on the pybind11 level, without need to 
> change it in the libcamera c++ side? Or maybe that's what you meant =).

That's exactly what I meant :-)

-- 
Regards,

Laurent Pinchart