[libcamera-devel] [PATCH v2 2/2] pipeline: raspberrypi: Allow registration of multiple cameras
Laurent Pinchart
laurent.pinchart at ideasonboard.com
Mon Nov 22 14:00:40 CET 2021
Hi Naush,
(CC'ing Sakari)
Thank you for the patch.
On Mon, Nov 22, 2021 at 12:34:27PM +0000, Naushir Patuck wrote:
> Expand the pipeline handler camera registration to correctly handle multiple
> cameras attached to the platform. For example, Raspberry Pi Compute Module
> platforms have two camera connectors, and this change would allow the user to
> select either of the two cameras to run.
>
> There are associated kernel driver changes for both Unicam and the ISP needed
> to correctly advertise multiple media devices and nodes for multi-camera usage:
>
> https://github.com/raspberrypi/linux/pull/4140
> https://github.com/raspberrypi/linux/pull/4709
>
> However, this change is backward compatible with kernel builds that do not have
> these changes for standard single camera usage.
>
> Signed-off-by: Naushir Patuck <naush at raspberrypi.com>
> ---
> .../pipeline/raspberrypi/raspberrypi.cpp | 113 ++++++++++++------
> 1 file changed, 74 insertions(+), 39 deletions(-)
>
> diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp
> index 9aa7e9eef5e7..3f9e15514ed9 100644
> --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp
> +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp
> @@ -311,14 +311,11 @@ private:
> return static_cast<RPiCameraData *>(camera->_d());
> }
>
> - bool registerCameras();
> + int registerCameras(MediaDevice *unicam, MediaDevice *isp, const std::string &deviceId);
Given that a pipeline handler instance doesn't register multiple cameras
anymore in v2, should this be called registerCamera() ?
> int queueAllBuffers(Camera *camera);
> int prepareBuffers(Camera *camera);
> void freeBuffers(Camera *camera);
> void mapBuffers(Camera *camera, const RPi::BufferMap &buffers, unsigned int mask);
> -
> - MediaDevice *unicam_;
> - MediaDevice *isp_;
> };
>
> RPiCameraConfiguration::RPiCameraConfiguration(const RPiCameraData *data)
> @@ -509,7 +506,7 @@ CameraConfiguration::Status RPiCameraConfiguration::validate()
> }
>
> PipelineHandlerRPi::PipelineHandlerRPi(CameraManager *manager)
> - : PipelineHandler(manager), unicam_(nullptr), isp_(nullptr)
> + : PipelineHandler(manager)
> {
> }
>
> @@ -993,49 +990,85 @@ int PipelineHandlerRPi::queueRequestDevice(Camera *camera, Request *request)
>
> bool PipelineHandlerRPi::match(DeviceEnumerator *enumerator)
> {
> - DeviceMatch unicam("unicam");
> - DeviceMatch isp("bcm2835-isp");
> + MediaDevice *unicamDevice, *ispDevice;
> + std::string deviceId;
>
> - unicam.add("unicam-image");
> + /*
> + * String of indexes to append to the entity names when searching for
> + * the Unican media devices. The first string is empty (un-indexed) to
> + * to maintain backward compatability with old versions of the Unicam
> + * kernel driver that did not advertise instance indexes.
> + */
> + for (const std::string &id : { "", "0", "1" }) {
> + DeviceMatch unicam("unicam");
> + unicam.add("unicam" + id + "-image");
> + unicamDevice = acquireMediaDevice(enumerator, unicam);
>
> - isp.add("bcm2835-isp0-output0"); /* Input */
> - isp.add("bcm2835-isp0-capture1"); /* Output 0 */
> - isp.add("bcm2835-isp0-capture2"); /* Output 1 */
> - isp.add("bcm2835-isp0-capture3"); /* Stats */
> + if (unicamDevice) {
> + deviceId = id == "1" ? "1" : "0";
> + break;
> + }
> + }
Wouldn't it be better to extend DeviceMatch to accept regexps ?
https://en.cppreference.com/w/cpp/header/regex should make that easy.
>
> - unicam_ = acquireMediaDevice(enumerator, unicam);
> - if (!unicam_)
> + if (!unicamDevice) {
> + LOG(RPI, Debug) << "Unable to acquire a Unicam instance";
> return false;
> + }
>
> - isp_ = acquireMediaDevice(enumerator, isp);
> - if (!isp_)
> + DeviceMatch isp("bcm2835-isp");
> + isp.add("bcm2835-isp" + deviceId + "-output0"); /* Input */
> + isp.add("bcm2835-isp" + deviceId + "-capture1"); /* Output 0 */
> + isp.add("bcm2835-isp" + deviceId + "-capture2"); /* Output 1 */
> + isp.add("bcm2835-isp" + deviceId + "-capture3"); /* Stats */
Unless you need to match the unicam and ISP instances, you could here
drop the isp.add() calls as matching on the device name should be
enough. Unless there are bcm2835-isp instances that don't have the
needed nodes ?
> + ispDevice = acquireMediaDevice(enumerator, isp);
> +
> + if (!ispDevice) {
> + LOG(RPI, Error) << "Unable to acquire ISP instance " << deviceId;
> return false;
> + }
> +
> + int ret = registerCameras(unicamDevice, ispDevice, deviceId);
> + if (ret) {
> + LOG(RPI, Error) << "Failed to register camera: " << ret;
> + return false;
> + }
>
> - return registerCameras();
> + return true;
> }
>
> -bool PipelineHandlerRPi::registerCameras()
> +int PipelineHandlerRPi::registerCameras(MediaDevice *unicam, MediaDevice *isp,
> + const std::string &deviceId)
> {
> std::unique_ptr<RPiCameraData> data = std::make_unique<RPiCameraData>(this);
> +
> if (!data->dmaHeap_.isValid())
> - return false;
> + return -ENOMEM;
> +
> + MediaEntity *unicamImage = unicam->getEntityByName("unicam" + deviceId + "-image");
> + MediaEntity *ispOutput0 = isp->getEntityByName("bcm2835-isp" + deviceId + "-output0");
> + MediaEntity *ispCapture1 = isp->getEntityByName("bcm2835-isp" + deviceId + "-capture1");
> + MediaEntity *ispCapture2 = isp->getEntityByName("bcm2835-isp" + deviceId + "-capture2");
> + MediaEntity *ispCapture3 = isp->getEntityByName("bcm2835-isp" + deviceId + "-capture3");
I'm wondering if wildcards would be nice here too, but maybe a simpler
option would be to not include the device ID in the entity names in the
driver ? We have a bus info field that the kernel reports for the media
device, to differentiate between multiple instances of the same device.
I suppose we are missing guidelines on the kernel side regarding entity
naming. Sakari, any opinion on this ?
> +
> + if (!unicamImage || !ispOutput0 || !ispCapture1 || !ispCapture2 || !ispCapture3)
> + return -ENOENT;
>
> /* Locate and open the unicam video streams. */
> - data->unicam_[Unicam::Image] = RPi::Stream("Unicam Image", unicam_->getEntityByName("unicam-image"));
> + data->unicam_[Unicam::Image] = RPi::Stream("Unicam Image", unicamImage);
>
> /* An embedded data node will not be present if the sensor does not support it. */
> - MediaEntity *embeddedEntity = unicam_->getEntityByName("unicam-embedded");
> - if (embeddedEntity) {
> - data->unicam_[Unicam::Embedded] = RPi::Stream("Unicam Embedded", embeddedEntity);
> + MediaEntity *unicamEmbedded = unicam->getEntityByName("unicam" + deviceId + "-embedded");
> + if (unicamEmbedded) {
> + data->unicam_[Unicam::Embedded] = RPi::Stream("Unicam Embedded", unicamEmbedded);
> data->unicam_[Unicam::Embedded].dev()->bufferReady.connect(data.get(),
> &RPiCameraData::unicamBufferDequeue);
> }
>
> /* Tag the ISP input stream as an import stream. */
> - data->isp_[Isp::Input] = RPi::Stream("ISP Input", isp_->getEntityByName("bcm2835-isp0-output0"), true);
> - data->isp_[Isp::Output0] = RPi::Stream("ISP Output0", isp_->getEntityByName("bcm2835-isp0-capture1"));
> - data->isp_[Isp::Output1] = RPi::Stream("ISP Output1", isp_->getEntityByName("bcm2835-isp0-capture2"));
> - data->isp_[Isp::Stats] = RPi::Stream("ISP Stats", isp_->getEntityByName("bcm2835-isp0-capture3"));
> + data->isp_[Isp::Input] = RPi::Stream("ISP Input", ispOutput0, true);
> + data->isp_[Isp::Output0] = RPi::Stream("ISP Output0", ispCapture1);
> + data->isp_[Isp::Output1] = RPi::Stream("ISP Output1", ispCapture2);
> + data->isp_[Isp::Stats] = RPi::Stream("ISP Stats", ispCapture3);
>
> /* Wire up all the buffer connections. */
> data->unicam_[Unicam::Image].dev()->frameStart.connect(data.get(), &RPiCameraData::frameStarted);
> @@ -1046,7 +1079,7 @@ bool PipelineHandlerRPi::registerCameras()
> data->isp_[Isp::Stats].dev()->bufferReady.connect(data.get(), &RPiCameraData::ispOutputDequeue);
>
> /* Identify the sensor. */
> - for (MediaEntity *entity : unicam_->entities()) {
> + for (MediaEntity *entity : unicam->entities()) {
> if (entity->function() == MEDIA_ENT_F_CAM_SENSOR) {
> data->sensor_ = std::make_unique<CameraSensor>(entity);
> break;
> @@ -1054,23 +1087,23 @@ bool PipelineHandlerRPi::registerCameras()
> }
>
> if (!data->sensor_)
> - return false;
> + return -EINVAL;
>
> if (data->sensor_->init())
> - return false;
> + return -EINVAL;
>
> data->sensorFormats_ = populateSensorFormats(data->sensor_);
>
> ipa::RPi::SensorConfig sensorConfig;
> if (data->loadIPA(&sensorConfig)) {
> LOG(RPI, Error) << "Failed to load a suitable IPA library";
> - return false;
> + return -EINVAL;
> }
>
> - if (sensorConfig.sensorMetadata ^ !!embeddedEntity) {
> + if (sensorConfig.sensorMetadata ^ !!unicamEmbedded) {
> LOG(RPI, Warning) << "Mismatch between Unicam and CamHelper for embedded data usage!";
> sensorConfig.sensorMetadata = false;
> - if (embeddedEntity)
> + if (unicamEmbedded)
> data->unicam_[Unicam::Embedded].dev()->bufferReady.disconnect();
> }
>
> @@ -1091,12 +1124,12 @@ bool PipelineHandlerRPi::registerCameras()
>
> for (auto stream : data->streams_) {
> if (stream->dev()->open())
> - return false;
> + continue;
> }
>
> if (!data->unicam_[Unicam::Image].dev()->caps().hasMediaController()) {
> LOG(RPI, Error) << "Unicam driver does not use the MediaController, please update your kernel!";
> - return false;
> + return -EINVAL;
> }
>
> /*
> @@ -1158,7 +1191,7 @@ bool PipelineHandlerRPi::registerCameras()
>
> if (!bayerFormat.isValid()) {
> LOG(RPI, Error) << "No Bayer format found";
> - return false;
> + return -EINVAL;
> }
> data->nativeBayerOrder_ = bayerFormat.order;
>
> @@ -1173,12 +1206,14 @@ bool PipelineHandlerRPi::registerCameras()
> streams.insert(&data->isp_[Isp::Output1]);
>
> /* Create and register the camera. */
> - const std::string &id = data->sensor_->id();
> + const std::string &cameraId = data->sensor_->id();
> std::shared_ptr<Camera> camera =
> - Camera::create(std::move(data), id, streams);
> + Camera::create(std::move(data), cameraId, streams);
> registerCamera(std::move(camera));
>
> - return true;
> + LOG(RPI, Info) << "Registered camera " << cameraId
> + << " to instance \"" << deviceId << "\"";
> + return 0;
> }
>
> int PipelineHandlerRPi::queueAllBuffers(Camera *camera)
--
Regards,
Laurent Pinchart
More information about the libcamera-devel
mailing list