[libcamera-devel] [PATCH v4] py: cam.py: Provide live graph of request metadata

[Kieran Bingham]

Quoting Tomi Valkeinen (2023-03-06 15:02:04)
> On 06/03/2023 16:59, Kieran Bingham wrote:
> > Quoting Tomi Valkeinen (2023-03-06 14:26:37)
> >> Hi,
> >>
> >> On 02/02/2023 17:03, Daniel Oakley wrote:
> >>> Metadata is very useful when improving specific camera configurations.
> >>> Currently, there is an argument to display the metadata in text form,
> >>> however this can be hard to visualise and spot changes or patterns over
> >>> time. Therefore this proposed patch adds an argument to display this
> >>> metadata in graph form.
> >>>
> >>> The metadata graph has 3 optional parameters:
> >>>    - refresh, number of times a second to update the graph
> >>>    - buffer, amount of historic/previous data to show
> >>>    - graphs, number of graphs to split the metadata between
> >>>    - autoscale, whether or not to autoscale the axis so all the data fits
> >>>
> >>> Displaying the graph does have some performance penalty, however this
> >>> has been mostly mitigated through the refresh parameter. Despite this,
> >>> graphing might not the best of ideas when using the camera to record or
> >>> save data. This is mainly for debugging purposes.
> >>>
> >>> Suggested-by: Kieran Bingham <kieran.bingham at ideasonboard.com>
> >>> Signed-off-by: Daniel Oakley <daniel.oakley at ideasonboard.com>
> >>> ---
> >>> This is the 4th version of my graph-metadata patch.
> >>> v3: hopefully addressing the issue of there being a lot of additional
> >>> code in the main cam.py which is optional. Most of the code has been
> >>> moved into a cam_metadata_graph.py file, which is imported at the start
> >>> of cam.py (as opposed to importing it twice throughout the code to make
> >>> the GraphDrawer and process the arguments). I have slightly tweaked the
> >>> error handling with the metadata-graph's arguments to catch a division
> >>> by zero error for refresh=0 and added a comment explanation for the
> >>> metadata-graph argument processor.
> >>> v4: I forgot to add the copywrite information to the new file, hopefully
> >>> I did it roughly correct.
> >>> There should, again, be no functional change between V1 and V2 and v3
> >>>
> >>>    src/py/cam/cam.py                |  22 +++
> >>>    src/py/cam/cam_metadata_graph.py | 236 +++++++++++++++++++++++++++++++
> >>>    2 files changed, 258 insertions(+)
> >>>    create mode 100644 src/py/cam/cam_metadata_graph.py
> >>
> >> What HW and OS did you use to test this?
> >>
> >> I tried with RPi OS, and after installing lots of stuff and finally getting libcamera compiling on the device, I got:
> >>
> >> Traceback (most recent call last):
> >>     File "/home/tomba/nfs/libcamera/./src/py/cam/cam.py", line 253, in event_handler
> >>       self.__request_handler(ctx, req)
> >>     File "/home/tomba/nfs/libcamera/./src/py/cam/cam.py", line 284, in __request_handler
> >>       ctx.graph_drawer.update_graph(ts, req.metadata)
> >>     File "/home/tomba/nfs/libcamera/src/py/cam/cam_metadata_graph.py", line 81, in update_graph
> >>       self.__animate()
> >>     File "/home/tomba/nfs/libcamera/src/py/cam/cam_metadata_graph.py", line 169, in __animate
> >>       self.blit_manager.update()
> >>     File "/home/tomba/nfs/libcamera/src/py/cam/cam_metadata_graph.py", line 206, in update
> >>       self.on_draw(None)
> >>     File "/home/tomba/nfs/libcamera/src/py/cam/cam_metadata_graph.py", line 189, in on_draw
> >>       self._draw_animated()
> >>     File "/home/tomba/nfs/libcamera/src/py/cam/cam_metadata_graph.py", line 200, in _draw_animated
> >>       fig.draw_artist(a)
> >>     File "/usr/lib/python3/dist-packages/matplotlib/figure.py", line 1879, in draw_artist
> >>       raise AttributeError("draw_artist can only be used after an "
> >> AttributeError: draw_artist can only be used after an initial draw which caches the renderer
> >>
> >> I'm still trying to get pyqt and matplotlib working on my buildroot...
> >>
> >> While I don't yet quite know how this behaves and looks like, I do wonder if this
> >> is something that would be better done with a separate script. We already have
> >> --save-frames option, we could add --save-metadata. And use matplotlib to show that
> >> data, possibly on a separate device (I'm thinking of showing it on my PC, as the dirs
> >> are shared via NFS).
> > 
> > But that doesn't work 'live'. I've tested this (or a previous version) -
> > and it allows me to get an instant and direct view of the repsonses in
> > the IPA algorithms as I adjust for lighting conditions etc.
> 
> Oh, I was thinking of "streaming" from the file. Although I'm not sure 
> how bit lag there is for that with NFS. You could, of course, also run 
> that locally, which I think would remove the lag fully.

If we're able to display 'live' from a file, then that would mean we
could have a single viewer that would work from stored / captured data,
cam, cam.py, and other applications. So I'm not opposed to that, I just
don't know how well it would run.

It also requires multi-step setup to view which is a bit more of a pain,
but anything we can do to make the implementation reusable is good.

--
Kieran


> 
>   Tomi
>