[libcamera-devel] [PATCH v4] py: cam.py: Provide live graph of request metadata

[Tomi Valkeinen]

On 06/03/2023 16:59, Kieran Bingham wrote:
> Quoting Tomi Valkeinen (2023-03-06 14:26:37)
>> Hi,
>>
>> On 02/02/2023 17:03, Daniel Oakley wrote:
>>> Metadata is very useful when improving specific camera configurations.
>>> Currently, there is an argument to display the metadata in text form,
>>> however this can be hard to visualise and spot changes or patterns over
>>> time. Therefore this proposed patch adds an argument to display this
>>> metadata in graph form.
>>>
>>> The metadata graph has 3 optional parameters:
>>>    - refresh, number of times a second to update the graph
>>>    - buffer, amount of historic/previous data to show
>>>    - graphs, number of graphs to split the metadata between
>>>    - autoscale, whether or not to autoscale the axis so all the data fits
>>>
>>> Displaying the graph does have some performance penalty, however this
>>> has been mostly mitigated through the refresh parameter. Despite this,
>>> graphing might not the best of ideas when using the camera to record or
>>> save data. This is mainly for debugging purposes.
>>>
>>> Suggested-by: Kieran Bingham <kieran.bingham at ideasonboard.com>
>>> Signed-off-by: Daniel Oakley <daniel.oakley at ideasonboard.com>
>>> ---
>>> This is the 4th version of my graph-metadata patch.
>>> v3: hopefully addressing the issue of there being a lot of additional
>>> code in the main cam.py which is optional. Most of the code has been
>>> moved into a cam_metadata_graph.py file, which is imported at the start
>>> of cam.py (as opposed to importing it twice throughout the code to make
>>> the GraphDrawer and process the arguments). I have slightly tweaked the
>>> error handling with the metadata-graph's arguments to catch a division
>>> by zero error for refresh=0 and added a comment explanation for the
>>> metadata-graph argument processor.
>>> v4: I forgot to add the copywrite information to the new file, hopefully
>>> I did it roughly correct.
>>> There should, again, be no functional change between V1 and V2 and v3
>>>
>>>    src/py/cam/cam.py                |  22 +++
>>>    src/py/cam/cam_metadata_graph.py | 236 +++++++++++++++++++++++++++++++
>>>    2 files changed, 258 insertions(+)
>>>    create mode 100644 src/py/cam/cam_metadata_graph.py
>>
>> What HW and OS did you use to test this?
>>
>> I tried with RPi OS, and after installing lots of stuff and finally getting libcamera compiling on the device, I got:
>>
>> Traceback (most recent call last):
>>     File "/home/tomba/nfs/libcamera/./src/py/cam/cam.py", line 253, in event_handler
>>       self.__request_handler(ctx, req)
>>     File "/home/tomba/nfs/libcamera/./src/py/cam/cam.py", line 284, in __request_handler
>>       ctx.graph_drawer.update_graph(ts, req.metadata)
>>     File "/home/tomba/nfs/libcamera/src/py/cam/cam_metadata_graph.py", line 81, in update_graph
>>       self.__animate()
>>     File "/home/tomba/nfs/libcamera/src/py/cam/cam_metadata_graph.py", line 169, in __animate
>>       self.blit_manager.update()
>>     File "/home/tomba/nfs/libcamera/src/py/cam/cam_metadata_graph.py", line 206, in update
>>       self.on_draw(None)
>>     File "/home/tomba/nfs/libcamera/src/py/cam/cam_metadata_graph.py", line 189, in on_draw
>>       self._draw_animated()
>>     File "/home/tomba/nfs/libcamera/src/py/cam/cam_metadata_graph.py", line 200, in _draw_animated
>>       fig.draw_artist(a)
>>     File "/usr/lib/python3/dist-packages/matplotlib/figure.py", line 1879, in draw_artist
>>       raise AttributeError("draw_artist can only be used after an "
>> AttributeError: draw_artist can only be used after an initial draw which caches the renderer
>>
>> I'm still trying to get pyqt and matplotlib working on my buildroot...
>>
>> While I don't yet quite know how this behaves and looks like, I do wonder if this
>> is something that would be better done with a separate script. We already have
>> --save-frames option, we could add --save-metadata. And use matplotlib to show that
>> data, possibly on a separate device (I'm thinking of showing it on my PC, as the dirs
>> are shared via NFS).
> 
> But that doesn't work 'live'. I've tested this (or a previous version) -
> and it allows me to get an instant and direct view of the repsonses in
> the IPA algorithms as I adjust for lighting conditions etc.

Oh, I was thinking of "streaming" from the file. Although I'm not sure 
how bit lag there is for that with NFS. You could, of course, also run 
that locally, which I think would remove the lag fully.

  Tomi