<div dir="ltr"><div dir="ltr">Hi David,<div><br></div><div>Thank you for your feedback.</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, 21 Dec 2021 at 10:53, David Plowman <<a href="mailto:david.plowman@raspberrypi.com">david.plowman@raspberrypi.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi Naush<br>
<br>
Thanks for this patch!<br>
<br>
On Tue, 14 Dec 2021 at 14:00, Naushir Patuck <<a href="mailto:naush@raspberrypi.com" target="_blank">naush@raspberrypi.com</a>> wrote:<br>
><br>
> This change will allow the pipeline handler to enumerate and control Video<br>
> Mux or Bridge devices that may be attached between sensors and a particular<br>
> Unicam instance. Cascaded mux or bridge devices are also handled.<br>
><br>
> A new member function RPiCameraData::enumerateVideoDevices(), called from<br>
> PipelineHandlerRPi::registerCamera(), is used to identify and open all mux and<br>
> bridge subdevices present in the sensor -> Unicam link.<br>
><br>
> Relevent links are enabled/disabled and pad formats correctly set in<br>
<br>
Maybe s/Relevent/Relevant/ ? But I could be wrong. It is also the most<br>
extreme hair-splitting!<br></blockquote><div><br></div><div>Ack.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
> PipelineHandlerRPi::configure() before the camera is started.<br>
><br>
> Signed-off-by: Naushir Patuck <<a href="mailto:naush@raspberrypi.com" target="_blank">naush@raspberrypi.com</a>><br>
> Reviewed-by: Kieran Bingham <<a href="mailto:kieran.bingham@ideasonboard.com" target="_blank">kieran.bingham@ideasonboard.com</a>><br>
> ---<br>
> .../pipeline/raspberrypi/raspberrypi.cpp | 139 ++++++++++++++++++<br>
> 1 file changed, 139 insertions(+)<br>
><br>
> diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp<br>
> index 2a2fb5273eb8..4ec646d3c7a3 100644<br>
> --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp<br>
> +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp<br>
> @@ -12,6 +12,7 @@<br>
> #include <mutex><br>
> #include <queue><br>
> #include <unordered_set><br>
> +#include <utility><br>
><br>
> #include <libcamera/base/shared_fd.h><br>
> #include <libcamera/base/utils.h><br>
> @@ -191,6 +192,8 @@ public:<br>
> int loadIPA(ipa::RPi::SensorConfig *sensorConfig);<br>
> int configureIPA(const CameraConfiguration *config);<br>
><br>
> + void enumerateVideoDevices(MediaLink *link);<br>
> +<br>
> void statsMetadataComplete(uint32_t bufferId, const ControlList &controls);<br>
> void runIsp(uint32_t bufferId);<br>
> void embeddedComplete(uint32_t bufferId);<br>
> @@ -220,6 +223,11 @@ public:<br>
> std::vector<RPi::Stream *> streams_;<br>
> /* Stores the ids of the buffers mapped in the IPA. */<br>
> std::unordered_set<unsigned int> ipaBuffers_;<br>
> + /*<br>
> + * Stores a cascade of Video Mux or Bridge devices between the sensor and<br>
> + * Unicam together with media link across the entities.<br>
> + */<br>
> + std::vector<std::pair<std::unique_ptr<V4L2Subdevice>, MediaLink *>> bridgeDevices_;<br>
><br>
> /* DMAHEAP allocation helper. */<br>
> RPi::DmaHeap dmaHeap_;<br>
> @@ -868,6 +876,38 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config)<br>
> */<br>
> data->properties_.set(properties::ScalerCropMaximum, data->sensorInfo_.analogCrop);<br>
><br>
> + /* Setup the Video Mux/Bridge entities. */<br>
> + for (auto &[device, link] : data->bridgeDevices_) {<br>
> + /*<br>
> + * Start by disabling all the sink pad links on the devices in the<br>
> + * cascade, with the exception of the link connecting the device.<br>
> + */<br>
> + for (const MediaPad *p : device->entity()->pads()) {<br>
> + if (!(p->flags() & MEDIA_PAD_FL_SINK))<br>
> + continue;<br>
> +<br>
> + for (MediaLink *l : p->links()) {<br>
> + if (l != link)<br>
> + l->setEnabled(false);<br>
> + }<br>
> + }<br>
> +<br>
> + /* Next, enable the entity -> entity links, and setup the pad format. */<br>
> + link->setEnabled(true);<br>
> + const MediaPad *srcPad = link->sink();<br>
> + ret = device->setFormat(srcPad->index(), &sensorFormat);<br>
> + if (ret) {<br>
> + LOG(RPI, Error) << "Failed to set format on " << device->entity()->name()<br>
> + << " pad " << srcPad->index()<br>
> + << " with format " << format.toString()<br>
> + << ": " << ret;<br>
> + return ret;<br>
> + }<br>
> +<br>
> + LOG(RPI, Info) << "Configured media link on device " << device->entity()->name()<br>
> + << " on pad " << srcPad->index();<br>
> + }<br>
> +<br>
> return ret;<br>
> }<br>
><br>
> @@ -1098,6 +1138,13 @@ int PipelineHandlerRPi::registerCamera(MediaDevice *unicam, MediaDevice *isp, Me<br>
> if (data->sensor_->init())<br>
> return -EINVAL;<br>
><br>
> + /*<br>
> + * Enumerate all the Video Mux/Bridge devices across the sensor -> unicam<br>
> + * link. There may be a cascade of devices in this link!<br>
> + */<br>
> + MediaLink *link = sensorEntity->getPadByIndex(0)->links()[0];<br>
> + data->enumerateVideoDevices(link);<br>
> +<br>
> data->sensorFormats_ = populateSensorFormats(data->sensor_);<br>
><br>
> ipa::RPi::SensorConfig sensorConfig;<br>
> @@ -1447,6 +1494,98 @@ int RPiCameraData::configureIPA(const CameraConfiguration *config)<br>
> return 0;<br>
> }<br>
><br>
> +/*<br>
> + * enumerateVideoDevices() iterates over the Media Controller topology, starting<br>
> + * at the sensor and finishing at Unicam. For each sensor, RPiCameraData stores<br>
> + * a unique list of any intermediate video mux or bridge devices connected in a<br>
> + * cascade, together with the entity to entity link.<br>
> + *<br>
> + * Entity pad configuration and link enabling happens at the end of configure().<br>
> + * We first disables all pad links on each entity device in the chain, and then<br>
<br>
s/disables/disable/<br>
<br>
> + * selectively enabling the specific links to link sensor to Unicam across all<br>
> + * intermediate muxes and bridges.<br>
> + *<br>
> + * In the cascaded topology below, if Sensor1 is used, the Mux2 -> Mux1 link<br>
> + * will be disabled, and Sensor1 -> Mux1 -> Unicam links enabled. Alternatively,<br>
> + * if Sensor3 is used, the Sensor2 -> Mux2 and Sensor1 -> Mux1 links are disabled,<br>
> + * and Sensor3 -> Mux2 -> Mux1 -> Unicam links are enabled. All other links will<br>
> + * remain unchanged.<br>
> + *<br>
> + * +----------+<br>
> + * | Unicam |<br>
> + * +-----^----+<br>
> + * |<br>
> + * +---+---+<br>
> + * | Mux1 <-------+<br>
> + * +--^----+ |<br>
> + * | |<br>
> + * +-----+---+ +---+---+<br>
> + * | Sensor1 | | Mux2 |<--+<br>
> + * +---------+ +-^-----+ |<br>
> + * | |<br>
> + * +-------+-+ +---+-----+<br>
> + * | Sensor2 | | Sensor3 |<br>
> + * +---------+ +---------+<br>
> + */<br>
> +void RPiCameraData::enumerateVideoDevices(MediaLink *link)<br>
> +{<br>
> + const MediaPad *sinkPad = link->sink();<br>
> + const MediaEntity *entity = sinkPad->entity();<br>
> + bool unicamFound = false;<br>
> +<br>
> + /* We only deal with Video Mux and Bridge devices in cascade. */<br>
> + if (entity->function() != MEDIA_ENT_F_VID_MUX &&<br>
> + entity->function() != MEDIA_ENT_F_VID_IF_BRIDGE)<br>
> + return;<br>
> +<br>
> + /* Find the source pad for this Video Mux or Bridge device. */<br>
> + const MediaPad *entitySourcePad = nullptr;<br>
> + for (const MediaPad *pad : entity->pads()) {<br>
> + if (pad->flags() & MEDIA_PAD_FL_SOURCE) {<br>
> + /*<br>
> + * We can only deal with devices that have a single source<br>
> + * pad. If this device has multple source pads, ignore it<br>
> + * and this branch in the cascade.<br>
> + */<br>
> + if (entitySourcePad)<br>
> + return;<br>
> +<br>
> + entitySourcePad = pad;<br>
> + }<br>
> + }<br>
> +<br>
> + LOG(RPI, Info) << "Found video mux device " << entity->name()<br>
> + << " linked to sink pad " << sinkPad->index();<br>
<br>
I can see the value during development, but might "Info" otherwise get<br>
a bit wordy? Don't really mind though, I guess it won't bother<br>
"regular" users.<br></blockquote><div><br></div><div>Will do!</div><div><br></div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
But it all LGTM, so:<br>
<br>
Reviewed-by: David Plowman <<a href="mailto:david.plowman@raspberrypi.com" target="_blank">david.plowman@raspberrypi.com</a>><br></blockquote><div><br></div><div>Thanks!</div><div>Naush</div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
Thanks!<br>
David<br>
<br>
> +<br>
> + bridgeDevices_.emplace_back(std::make_unique<V4L2Subdevice>(entity), link);<br>
> + bridgeDevices_.back().first->open();<br>
> +<br>
> + /*<br>
> + * Iterate through all the sink pad links down the cascade to find any<br>
> + * other Video Mux and Bridge devices.<br>
> + */<br>
> + for (MediaLink *l : entitySourcePad->links()) {<br>
> + enumerateVideoDevices(l);<br>
> + /* Once we reach the Unicam entity, we are done. */<br>
> + if (l->sink()->entity()->name() == "unicam-image") {<br>
> + unicamFound = true;<br>
> + break;<br>
> + }<br>
> + }<br>
> +<br>
> + /* This identifies the end of our entity enumeration recursion. */<br>
> + if (link->source()->entity()->function() == MEDIA_ENT_F_CAM_SENSOR) {<br>
> + /*<br>
> + * If Unicam is not at the end of this cascade, we cannot configure<br>
> + * this topology automatically, so remove all entity references.<br>
> + */<br>
> + if (!unicamFound) {<br>
> + LOG(RPI, Warning) << "Cannot automatically configure this MC topology!";<br>
> + bridgeDevices_.clear();<br>
> + }<br>
> + }<br>
> +}<br>
> +<br>
> void RPiCameraData::statsMetadataComplete(uint32_t bufferId, const ControlList &controls)<br>
> {<br>
> if (state_ == State::Stopped)<br>
> --<br>
> 2.25.1<br>
><br>
</blockquote></div></div>