[libcamera-devel] [PATCH 00/19] Initial libcamera threading model
Laurent Pinchart
laurent.pinchart at ideasonboard.com
Mon Jan 20 01:24:18 CET 2020
Hello,
This series defines and implements an initial version of the threading
model.
libcamera currently relies on the application providing an event loop.
This cumbersome when the direct user of libcamera is not an application
but a framework, as no event loop is available in that case. The Android
camera HAL and the V4L2 adaptation layers had to create an internal
thread to make an event loop available to libcamera, and handle
synchronization between calls from the upper framework and the internal
thread. A GStreamer element would require a similar implementation,
resulting in lots of code duplication.
Furthermore, relying on an application event loop may block libcamera
from processing events if the application performs long blocking
operations. This is not compatible with the real-time constraints of
pipeline handlers and IPAs.
For this reason, creating and managing threads internally to libcamera
is a better option. This requires defining a threading model, as all of
a sudden multiple contexts of execution will be available when only one
context existed before.
The first 7 patches are preparatory work in the form of small
miscellaneous fixes or cleanups (01/19 and 05/19 to 07/19) and
enhancements (making ConnectionTypeBlocking more useful in 02/19, and
better log support for multithreading contexts in 03/19 and 04/19).
Patch 08/19 then adds a test case to the signal test to catch an
existing bug, and patch 09/19 fixes the bug while reworking the Signal
implementation.
The next two patches consist solely of documentation. Patch 10/19
defines the threading model and patch 11/19 documents it in core
classes. Patch 12/19 then makes signal connection and disconnection
thread-safe to support multi-threaded contexts.
Patch 13/19 is the most disruptive change, as it creates an thread for
the CameraManager. At the moment this is the only thread created by
libcamera, and all pipeline handlers and camera are bound to it. This
model is likely too simple and will need to be extended with more
threads in the future to avoid low-priority blocking operations
interfering with real-time camera constraints. The change is however a
step in the right direction, as it brings consideration of multiple
threads to all of libcamera.
Patches 14/19 to 17/19 then document and implement the threading model
in the Camera and PipelineHandler classes. As a side effect, along with
13/19, they move private data members of the CameraManager and Camera
classes to private classes, helping with ABI breakage avoidance in
future work.
Note that patch 13/19 breaks operation of libcamera, and patch 16/19
fixes that. This is caused by the threading model implementation being
split in two patches. I have decided to organize the changes this way to
keep changes reviewable, but I'm open to alternatives.
Patches 18/19 and 19/19 then simplify the V4L2 compatibility and Android
camera HAL layers by removing the internal threads, as libcamera doesn't
require an externally-provided event loop anymore. I have tested 18/19
with yavta, but 19/19 is currently only compile-tested.
Laurent Pinchart (19):
test: buffer_import: Propagate status code from buffer allocation
libcamera: bound_method: Avoid deadlock with ConnectionTypeBlocking
libcamera: thread: Add a method to return the ID of the current thread
libcamera: log: Print the thread ID in the log
libcamera: Replace ARRAY_SIZE with std::array
libcamera: bound_method: Use std::index_sequence
libcamera: Declare static local variables as const where applicable
test: signal: Add additional disconnection tests for Object
libcamera: signal: Make slots list private
libcamera: Define the threading model
libcamera: Document thread-safety attributes of core classes
libcamera: signal: Make connection and disconnection thread-safe
libcamera: camera_manager: Run the camera manager in a thread
libcamera: camera: Move private data members to private implementation
libcamera: camera: Centralize state checks in Private class
libcamera: camera: Implement the threading model
libcamera: pipeline_handler: Implement the threading model
v4l2: Remove internal thread
android: Remove internal thread
Documentation/Doxyfile.in | 5 +-
include/libcamera/bound_method.h | 38 +--
include/libcamera/camera.h | 28 +-
include/libcamera/camera_manager.h | 13 +-
include/libcamera/object.h | 4 +-
include/libcamera/signal.h | 88 +++---
src/android/camera3_hal.cpp | 8 +-
src/android/camera_device.cpp | 48 +--
src/android/camera_device.h | 17 +-
src/android/camera_hal_manager.cpp | 78 ++---
src/android/camera_hal_manager.h | 16 +-
src/android/camera_ops.cpp | 96 ++++++
src/android/camera_ops.h | 15 +
src/android/camera_proxy.cpp | 180 -----------
src/android/camera_proxy.h | 42 ---
src/android/meson.build | 2 +-
src/libcamera/bound_method.cpp | 22 +-
src/libcamera/camera.cpp | 364 +++++++++++++++--------
src/libcamera/camera_manager.cpp | 295 +++++++++++++-----
src/libcamera/controls.cpp | 2 +-
src/libcamera/device_enumerator.cpp | 2 +
src/libcamera/event_notifier.cpp | 2 +
src/libcamera/formats.cpp | 2 +-
src/libcamera/framebuffer_allocator.cpp | 49 +--
src/libcamera/include/pipeline_handler.h | 4 +-
src/libcamera/include/thread.h | 2 +
src/libcamera/include/utils.h | 2 -
src/libcamera/ipa_module.cpp | 8 +-
src/libcamera/ipc_unixsocket.cpp | 2 +
src/libcamera/log.cpp | 21 +-
src/libcamera/object.cpp | 17 +-
src/libcamera/pipeline_handler.cpp | 41 ++-
src/libcamera/signal.cpp | 69 +++++
src/libcamera/thread.cpp | 106 +++++++
src/libcamera/timer.cpp | 20 +-
src/libcamera/utils.cpp | 5 -
src/libcamera/v4l2_videodevice.cpp | 9 +-
src/v4l2/v4l2_camera.h | 2 +-
src/v4l2/v4l2_camera_proxy.cpp | 43 +--
src/v4l2/v4l2_compat_manager.cpp | 48 +--
src/v4l2/v4l2_compat_manager.h | 13 +-
test/camera/buffer_import.cpp | 18 +-
test/ipc/unixsocket.cpp | 8 +-
test/object-invoke.cpp | 20 ++
test/signal.cpp | 24 ++
45 files changed, 1112 insertions(+), 786 deletions(-)
create mode 100644 src/android/camera_ops.cpp
create mode 100644 src/android/camera_ops.h
delete mode 100644 src/android/camera_proxy.cpp
delete mode 100644 src/android/camera_proxy.h
--
Regards,
Laurent Pinchart
More information about the libcamera-devel
mailing list