[libcamera-devel] [PATCH v5 00/14] libcamera: ipu3: Multiple streams support

[Jacopo Mondi]

Hello,
   this series implements multiple stream support for IPU3 and contains
what I've recently sent as
"[PATCH v7 0/8] libcamera: Framework changes to prepare for multiple streams support"

Patches from 0 to 8 were part of that series, I'm resending them here in the
hope all the series could get in together.
Most of these patches have been reviewed already, I just fixed:
"libcamera: camera: Validate Request before queueing it"
which was buggy, and I just noticed while testing multiple stream support
on top of it.
If you still have to look at v7 of the above mentioned series, you should look
at this one instead.

Patches from 9 to 14 extend the IPU3 pipeline handler to support operations
for multiple streams, up to two per cameras.

I have included comments on v4 received from Laurent and Niklas and now the
patches look much better, thanks you both for the comments.

The most relevant change is related to the long reply I gave to Niklas on v4's
"libcamera: ipu3: Queue requests for multiple streams"

There I detailed how it was necessary to queue buffers to the 'output' node
to have images processed by the ImgU and how I badly worked it around. We
concluded a kernel fix was needed to use dummy buffers in case no buffer has
been made available from userspace. Well, I re-tested now, and it seems the
driver usese dummy buffers already, if the output node is "enabled" (aka the
media link between it and the ImgU subdevice is active). With this current
implementation the 'output' node is active (as it was in v4, I cannot tell what
changed) and the driver uses dummy buffer, as I've verified enabling the
ipu3-imgu module debug output. For this reason the patch is greatly simplified
and just simply queue buffers for the nodes part of the request.

I have tested several stream combinations:
- only viewfinder
- only still capture
- viewfinder + still capture
- viewfinder + (viewfinder + still capture) every 10 frames (see patch 14/14)

using the cam application from the most recent master branch.

Patches available at jmondi/imgu-multi-v5

Hopefully, we should be close to have this merged!

Thanks
  j

Jacopo Mondi (14):
  libcamera: stream: Make Stream inheritable
  libcamera: camera: Pass the stream set to allocate/freeBuffers()
  libcamera: camera: Don't call freeBuffer() on allocateBuffer() error
  libcamera: v4l2_device: Propagate releaseBuffers() error
  libcamera: request: Add hasPendingBuffers() method
  libcamera: camera: Validate Request before queueing it
  libcamera: request: Expose the Stream to Buffers map
  libcamera: buffer: Store Request reference in Buffer
  libcamera: ipu3: Create camera with 2 streams
  libcamera: ipu3: Add multiple stream memory management
  libcamera: ipu3: Queue requests for multiple streams
  libcamera: ipu3: Connect viewfinder's BufferReady signal
  libcamera: ipu3: Use roles in stream configuration
  [HACK] still capture every 10 frames

 include/libcamera/buffer.h               |   6 +
 include/libcamera/request.h              |   3 +
 include/libcamera/stream.h               |   4 +-
 src/cam/main.cpp                         |  46 ++-
 src/libcamera/buffer.cpp                 |  25 +-
 src/libcamera/camera.cpp                 |  25 +-
 src/libcamera/include/pipeline_handler.h |   6 +-
 src/libcamera/pipeline/ipu3/ipu3.cpp     | 410 +++++++++++++++++------
 src/libcamera/pipeline/uvcvideo.cpp      |  13 +-
 src/libcamera/pipeline/vimc.cpp          |  13 +-
 src/libcamera/pipeline_handler.cpp       |  11 +-
 src/libcamera/request.cpp                |  37 +-
 src/libcamera/v4l2_device.cpp            |   3 +-
 13 files changed, 453 insertions(+), 149 deletions(-)

--
2.21.0