[PATCH/RFC 00/32] libcamera: Support the upstream Unicam driver
Laurent Pinchart
laurent.pinchart at ideasonboard.com
Fri Mar 1 22:20:49 CET 2024
Hello everybody,
This large(-ish) patch series is the first complete implementation of
support for the Unicam upstream kernel driver for libcamera.
Or, it would be, if the Unicam driver was upstream. While I have also
completed a version of the Unicam driver for the upstream kernel,
there's a bit of a chicken-and-egg issue here as I would like to
cross-reference the cover letters. To break the loop, I'm posting this
series first, and will reply with a link to the Unicam patches ([1]).
Support for Raspberry Pi 4 in libcamera currently relies on downstream
drivers that live in the Raspberry Pi kernel tree, for Unicam (the CSI-2
receiver) and the ISP. The Unicam driver, in particular, includes a few
workarounds to implement support for sensor embedded data, to compensate
for features that are or were until recently missing in the V4L2
in-kernel and userspace APIs. The new Unicam submission for the kernel
reworks the driver extensively to use the recently merged V4L2 streams
API, as well as the under development V4L2 generic metadata and internal
pads APIs ([2]).
The series starts with nine patches that clean up, rework and improve
the V4L2Subdevice class (01/32 to 09/32). They have been posted to the
list before, and mostly reviewed. Patches 10/32 is where the fun starts,
as it pulls the APIs from [2] in the kernel headers. The next three
patches, 11/32 to 13/32, then update the V4L2Subdevice and
V4L2VideoDevice classes to support those APIs.
The next ten patches rework the CameraSensor class to support sensor
drivers that use the new APIs. Patches 14/32 to to 21/32 contain various
reworks, including moving the camera sensor support to the new
src/libcamera/sensor/ directory, and turning CameraSensor into an
abstract base class with multiple implementations, using a factory
pattern to pick the right match. Patches 22/32 introduces a new
CameraSensor subclass for the new APIs, and patch 23/32 then extends the
CameraSensor interface to support embedded data.
With all the building blocks in place, the next seven patches focus on
the Raspberry Pi VC4 pipeline handler to use the new APIs, when
interacting with the sensor (24/32), or with the mainline Unicam driver
(27/32 to 30/32). Once done, patch 31/32 drops the
V4L2_META_FMT_SENSOR_DATA format from the videodev2.h kernel header, as
the format is not used anymore and has never been present in the
upstream kernel.
The last patch, 32/32, is a hack that unconditionally enables embedded
data for the IMX219 sensor, as that is the platform I've been using to
develop and test this series.
Worthy of a note, this series breaks compatibility with the downstream
Raspberry Pi kernel. I have decided not to preserve backward
compatibiliy to ease development, and to make the new code easier to
review. We still need to discuss how to integrate the various components
(libcamera and kernel space) in a way that will not generate lots of
anger among users. This may lead to some temporary changes to the code
to help with the transition.
For convenience, the patches can be found at [3].
[1] TBD due to lack of a time machine
[2] https://lore.kernel.org/linux-media/20231106122539.1268265-1-sakari.ailus@linux.intel.com
[3] https://git.libcamera.org/libcamera/pinchartl/libcamera.git/log/?h=rpi/streams/next
Jacopo Mondi (1):
libcamera: camera_sensor: Introduce CameraSensorFactory
Laurent Pinchart (31):
libcamera: v4l2_subdevice: Rename V4L2SubdeviceFormatInfo
libcamera: v4l2_subdevice: Add code member to MediaBusFormatInfo
libcamera: v4l2_subdevice: Expose media bus format info as internal
API
libcamera: v4l2_subdevice: Extend MediaBusFormatInfo with metadata
formats
libcamera: v4l2_subdevice: Drop V4L2SubdeviceFormat::bitsPerPixel()
libcamera: v4l2_subdevice: Rename V4L2SubdeviceFormat::mbus_code to
code
libcamera: v4l2_subdevice: Add stream support to get/set functions
libcamera: v4l2_subdevice: Replace Routing::toString() with
operator<<()
libcamera: v4l2_subdevice: Add V4L2Subdevice::Route structure
[DNI] include: linux: Update kernel headers to metadata API
libcamera: v4l2_subdevice: Update to the new kernel routing API
libcamera: v4l2_subdevice: Add new metadata formats
libcamera: v4l2_videodevice: Update to the new kernel metadata API
libcamera: camera_sensor: Move related classes to subdirectory
libcamera: camera_sensor: Drop updateControlInfo() function
libcamera: camera_sensor: Reorder functions
libcamera: camera_sensor: Test for read-only HBLANK with READ_ONLY
flag
libcamera: camera_sensor: Expose the Bayer order
libcamera: camera_sensor: Create abstract base class
libcamera: camera_sensor: Sort factories by priority
libcamera: Add CameraSensor implementation for raw V4L2 sensors
libcamera: camera_sensor: Add support for embedded data
pipeline: raspberrypi: common: Configure sensor embedded data
pipeline: raspberrypi: vc4: Use operator<<(V4L2VideoFormat)
pipeline: raspberrypi: vc4: Reorganize platformConfigure()
pipeline: raspberrypi: vc4: Use the CameraSensor embedded data API
pipeline: raspberrypi: vc4: Unconditionally create embedded data
stream
pipeline: raspberrypi: vc4: Configure format on Unicam subdev
pipeline: raspberrypi: vc4: Fix configuration of the embedded data
node
include: linux: Drop V4L2_META_FMT_SENSOR_DATA
[HACK]: ipa: rpi: cam_helper_imx219: Enable embedded data
Documentation/Doxyfile.in | 2 +
include/libcamera/internal/camera_sensor.h | 162 +--
include/libcamera/internal/v4l2_subdevice.h | 103 +-
include/linux/README | 2 +-
include/linux/media-bus-format.h | 13 +
include/linux/media.h | 1 +
include/linux/v4l2-controls.h | 16 +-
include/linux/v4l2-mediabus.h | 18 +-
include/linux/v4l2-subdev.h | 23 +-
include/linux/videodev2.h | 30 +-
src/ipa/rpi/cam_helper/cam_helper_imx219.cpp | 2 +-
src/libcamera/meson.build | 3 +-
src/libcamera/pipeline/imx8-isi/imx8-isi.cpp | 19 +-
src/libcamera/pipeline/ipu3/cio2.cpp | 15 +-
src/libcamera/pipeline/ipu3/imgu.cpp | 4 +-
src/libcamera/pipeline/rkisp1/rkisp1.cpp | 9 +-
src/libcamera/pipeline/rkisp1/rkisp1_path.cpp | 2 +-
.../pipeline/rpi/common/pipeline_base.cpp | 85 +-
.../pipeline/rpi/common/pipeline_base.h | 6 +-
src/libcamera/pipeline/rpi/vc4/vc4.cpp | 174 ++-
src/libcamera/pipeline/simple/simple.cpp | 29 +-
src/libcamera/pipeline/vimc/vimc.cpp | 11 +-
src/libcamera/sensor/camera_sensor.cpp | 557 ++++++++
.../camera_sensor_legacy.cpp} | 717 ++++------
.../{ => sensor}/camera_sensor_properties.cpp | 0
src/libcamera/sensor/camera_sensor_raw.cpp | 1135 +++++++++++++++
src/libcamera/sensor/meson.build | 8 +
src/libcamera/v4l2_subdevice.cpp | 1239 ++++++++++++++---
src/libcamera/v4l2_videodevice.cpp | 31 +-
test/camera-sensor.cpp | 9 +-
.../v4l2_videodevice_test.cpp | 7 +-
test/v4l2_videodevice/v4l2_videodevice_test.h | 2 +-
32 files changed, 3471 insertions(+), 963 deletions(-)
create mode 100644 src/libcamera/sensor/camera_sensor.cpp
rename src/libcamera/{camera_sensor.cpp => sensor/camera_sensor_legacy.cpp} (60%)
rename src/libcamera/{ => sensor}/camera_sensor_properties.cpp (100%)
create mode 100644 src/libcamera/sensor/camera_sensor_raw.cpp
create mode 100644 src/libcamera/sensor/meson.build
base-commit: c64446c226d4e629884d2f5b148a01969e8ee84a
--
Regards,
Laurent Pinchart
More information about the libcamera-devel
mailing list