[libcamera-devel] [PATCH] py: cam: Network renderer

[Laurent Pinchart]

Hi Tomi,

On Sun, Mar 19, 2023 at 05:49:46PM +0200, Tomi Valkeinen wrote:
> On 19/03/2023 15:05, Laurent Pinchart wrote:
> > On Sun, Mar 19, 2023 at 01:30:13PM +0200, Tomi Valkeinen via libcamera-devel wrote:
> >> Here's something I have found useful a few times.
> >>
> >> This adds a "tx" renderer to cam.py, which sends the frames over the
> >> network to a receiver.
> >>
> >> It also adds a "cam-rx" tool (non-libcamera based) which receives the
> >> frames and uses PyQt to show them on the screen, usually ran on a PC.
> >>
> >> This is obviously not super efficient, but on the PC side it doesn't
> >> matter. On the TX side, at least RPi4 seemed to work without noticeable
> >> lag, but my old 32-bit TI DRA76, when sending three camera streams, the
> >> performance dropped to ~5fps. Still, I find that more than enough for
> >> most development work.
> >>
> >> This could be extended to also transmit the metadata.
> > 
> > What's the advantage of this approach compared to using GStreamer for
> > network streaming ? It feels to me that we're reinventing the wheel a
> > bit here.
> 
> Well, these may not matter to other people, but for me:
> 
> - This doesn't need gstreamer

That's an argument I can't disagree with :-)

> - This works, whereas I have a lot of trouble getting gstreamer working. 
> I did manage the get a few formats rendering locally, but I couldn't get 
> anything over a tcp sink.

Then we need to improve GStreamer support, fixing bugs if any, and
providing documentation with sample pipelines for both the TX and RX
sides.

> - The code is short and trivial, and I have the same TX code working on 
> my v4l2 python test app.
> - With this, I have a trivial way to get the raw frames and metadata (to 
> be implemented =) on my PC and process and study them with python & numpy.
> - This could be used for the "py: cam.py: Provide live graph of request 
> metadata"
> 
> I should have emphasized that this is a development/testing helper, not 
> "streaming support".
> 
> Also, doesn't your point apply to any rendering done by cam?

This leads to the real question: where do we draw the line ? A trivial
network streaming implementation is, well, trivial, but it will fail in
various ways in various cases. I don't want to end up with a custom
implementation of RTSP in cam.py, so where will we stop ?

I would also argue that it would be good to keep the feature set of cam
and cam.py as close as possible to each other.

-- 
Regards,

Laurent Pinchart