[libcamera-devel] [PATCH v2 00/25] libcamera: Rework buffer API

[Niklas Söderlund]

Hi,

This series reworks the buffer API across the whole library. The two
main reasons for the rework is

- The current buffer API is cumbersome to work with as the variations
  between internally allocated buffers (from a V4L2 video device) and
  externally (from other source in the system or other V4l2 video
  device) is slightly different.

- V4L2 concepts such as buffer index have "leaked" into the application
  facing interface which makes the API less intuitive to work with as
  one needs to know more about V4L2 and its limitations to use.

As changing the buffer API touches most parts of the library this series
is unfortunately quiet large and some patches are also quiet large. I
have really tried to break things apart as best I could.

The series starts by adding the new FrameBuffer interface building
blocks and then slowly proceeds to replace the existing API with the new
one. The series is tested on all upstream pipelines and IPAs without any
regressions.

One note on 12/25, there is a temporary hack which is introduced in this 
patch that allows both the Buffer and FrameBuffer API to function side 
by side. That is i the V4L2VideoDevice is setup using the Buffer API 
slots related to Buffer will be emitted while if it is setup with the
FrameBuffer API slots related to FrameBuffers are emitted. This hack is 
removed later in 21/25 where the Buffer API is completely removed.

The series is quiet heavily rewritten since v1 and as I processed the 
review comments things got moved around. I hope I have not missed 
anything when processing the review comments and tracking changes in 
each patch individual changelong. I'm sure I did anyhow and if so I 
would like to apologize in advance as I know v1 was painful to review.

The series depends on '[PATCH v2 0/3] libcamera: Add FileDescriptor to 
help pass numerical fds around'.

Niklas Söderlund (25):
  libcamera: buffer: Add FrameMetadata container for metadata
    information
  libcamera: buffer: Add FrameBuffer interface
  ipa: Switch to FrameBuffer interface
  libcamera: buffer: Switch from Plane to FrameBuffer::Plane
  libcamera: buffers: Remove Plane class
  libcamera: buffer: Drop private function setRequest()
  libcamera: v4l2_videodevice: Align which type variable is used in
    queueBuffer()
  libcamera: v4l2_videodevice: Extract exportDmabufFd()
  libcamera: request: In addBuffer() do not fetch stream from Buffer
  libcamera: buffer: Move captured metadata to FrameMetadata
  libcamera: v4l2_videodevice: Add V4L2BufferCache to deal with index
    mapping
  libcamera: v4l2_videodevice: Add FrameBuffer interface
  test: v4l2_videodevice: Switch to FrameBuffer interface
  test: camera: buffer_import: Update to FrameBuffer restrictions
  libcamera: pipeline: rkisp1: Destroy frame information before
    completing request
  libcamera: pipeline: rkisp1: Switch to FrameBuffer interface for stat
    and param
  libcamera: pipeline: ipu3: Switch to FrameBuffer interface for cio2
    and stat
  libcamera: pipelines: Add FrameBuffer handlers
  libcamera: allocator: Add FrameBufferAllocator to help applications
    allocate buffers
  libcamera: Switch to FrameBuffer interface
  libcamera: v4l2_videodevice: Remove Buffer interface
  libcamera: Remove dead code after switch to FrameBuffer
  qcam: Cache buffer memory mapping
  libcamera: pipeline: Remove explicit buffer handling
  libcamera: camera: Remove the prepared state

 include/ipa/ipa_interface.h                   |   2 +-
 include/libcamera/buffer.h                    | 109 ++---
 include/libcamera/camera.h                    |  13 +-
 include/libcamera/framebuffer_allocator.h     |  47 ++
 include/libcamera/meson.build                 |   1 +
 include/libcamera/request.h                   |  14 +-
 include/libcamera/stream.h                    |  23 -
 src/android/camera_device.cpp                 |  41 +-
 src/cam/buffer_writer.cpp                     |  15 +-
 src/cam/buffer_writer.h                       |   3 +-
 src/cam/capture.cpp                           |  61 +--
 src/cam/capture.h                             |   5 +-
 src/ipa/libipa/ipa_interface_wrapper.cpp      |   9 +-
 src/ipa/rkisp1/rkisp1.cpp                     |  30 +-
 src/libcamera/buffer.cpp                      | 403 +++++-------------
 src/libcamera/camera.cpp                      | 170 +++-----
 src/libcamera/framebuffer_allocator.cpp       | 213 +++++++++
 src/libcamera/include/pipeline_handler.h      |  15 +-
 src/libcamera/include/v4l2_videodevice.h      |  78 +++-
 src/libcamera/ipa_context_wrapper.cpp         |   8 +-
 src/libcamera/ipa_interface.cpp               |   7 +-
 src/libcamera/meson.build                     |   1 +
 src/libcamera/pipeline/ipu3/ipu3.cpp          | 275 ++++++------
 src/libcamera/pipeline/rkisp1/rkisp1.cpp      | 178 ++++----
 src/libcamera/pipeline/uvcvideo.cpp           |  42 +-
 src/libcamera/pipeline/vimc.cpp               |  42 +-
 src/libcamera/pipeline_handler.cpp            |  65 ++-
 src/libcamera/request.cpp                     |  37 +-
 src/libcamera/stream.cpp                      | 248 +----------
 src/libcamera/v4l2_videodevice.cpp            | 381 ++++++++++-------
 src/qcam/main_window.cpp                      |  86 ++--
 src/qcam/main_window.h                        |   7 +-
 test/camera/buffer_import.cpp                 | 383 +++++------------
 test/camera/capture.cpp                       |  48 +--
 test/camera/statemachine.cpp                  |  89 +---
 test/ipa/ipa_wrappers_test.cpp                |  41 +-
 test/v4l2_videodevice/buffer_sharing.cpp      |  34 +-
 test/v4l2_videodevice/capture_async.cpp       |  18 +-
 test/v4l2_videodevice/request_buffers.cpp     |  11 +-
 test/v4l2_videodevice/stream_on_off.cpp       |   6 +-
 test/v4l2_videodevice/v4l2_m2mdevice.cpp      |  40 +-
 test/v4l2_videodevice/v4l2_videodevice_test.h |   2 +-
 42 files changed, 1465 insertions(+), 1836 deletions(-)
 create mode 100644 include/libcamera/framebuffer_allocator.h
 create mode 100644 src/libcamera/framebuffer_allocator.cpp

-- 
2.24.1