[libcamera-devel] [PATCH 5/5] media: v4l2-async: Create links during v4l2_async_match_notify()

[Daniel Scally]

Hi Laurent

On 14/12/2021 23:01, Laurent Pinchart wrote:
> Hi Daniel,
>
> On Tue, Dec 14, 2021 at 10:36:01PM +0000, Daniel Scally wrote:
>> On 14/12/2021 22:22, Laurent Pinchart wrote:
>>> On Mon, Dec 13, 2021 at 11:28:49PM +0000, Daniel Scally wrote:
>>>> Upon an async fwnode match, there's some typical behaviour that the
>>>> notifier and matching subdev will want to do. For example, a notifier
>>>> representing a sensor matching to an async subdev representing its
>>>> VCM will want to create an ancillary link to expose that relationship
>>>> to userspace.
>>>>
>>>> To avoid lots of code in individual drivers, try to build these links
>>>> within v4l2 core.
>>>>
>>>> Signed-off-by: Daniel Scally <djrscally at gmail.com>
>>>> ---
>>>> Changes since the rfc:
>>>>
>>>> 	- None
>>>>
>>>>  drivers/media/v4l2-core/v4l2-async.c | 51 ++++++++++++++++++++++++++++
>>>>  1 file changed, 51 insertions(+)
>>>>
>>>> diff --git a/drivers/media/v4l2-core/v4l2-async.c b/drivers/media/v4l2-core/v4l2-async.c
>>>> index 0404267f1ae4..6575b1cbe95f 100644
>>>> --- a/drivers/media/v4l2-core/v4l2-async.c
>>>> +++ b/drivers/media/v4l2-core/v4l2-async.c
>>>> @@ -275,6 +275,45 @@ v4l2_async_nf_try_complete(struct v4l2_async_notifier *notifier)
>>>>  static int
>>>>  v4l2_async_nf_try_all_subdevs(struct v4l2_async_notifier *notifier);
>>>>  
>>>> +static int
>>>> +__v4l2_async_create_ancillary_link(struct v4l2_async_notifier *notifier,
>>>> +				   struct v4l2_subdev *sd)
>>>> +{
>>>> +	struct media_link *link;
>>>> +
>>>> +	if (sd->entity.function != MEDIA_ENT_F_LENS &&
>>>> +	    sd->entity.function != MEDIA_ENT_F_FLASH)
>>>> +		return -EINVAL;
>>>> +
>>>> +	link = media_create_ancillary_link(&notifier->sd->entity, &sd->entity,
>>> Is there a guarantee at this point that notifier->sd->entity has already
>>> been registered with the media_device ? That's done by
>>> media_device_register_entity() called from
>>> v4l2_device_register_subdev().
>> v4l2_async_match_notify() calls v4l2_device_register_subdev() before the
>> point that I've added the call to v4l2_async_try_create_links(), so I
>> think that's covered there.
> It calls it on sd, not notifier->sd. It's the latter that concerns me.


Ah, you're right of course...I guess in that case the notifier->sd would
get registered during the v4l2_async_match_notify() where the sensor
driver's subdev is sd, but I don't think there's any guarantee that that
would happen first...I haven't traced it through but my guess is that it
would depend on the order in which the ipu3-cio2, sensor and lens
controller drivers probed. I'll check to try and be sure how it works
tomorrow

>
>>>> +					   MEDIA_LNK_FL_ENABLED |
>>>> +					   MEDIA_LNK_FL_IMMUTABLE);
>>>> +
>>>> +	return IS_ERR(link) ? PTR_ERR(link) : 0;
>>>> +}
>>>> +
>>>> +/*
>>>> + * Setup links on behalf of the notifier and subdev, where it's obvious what
>>> s/Setup/Create/ ("link setup" refers to another operation, enabling and
>>> disabling links at runtime)
>> Yes, good point; although that too is a piece of terminology I find a
>> bit jarring to be honest; I would have named that function
>> media_entity_configure_link()
>>
>>>> + * should be done. At the moment, we only support cases where the notifier
>>>> + * is a sensor and the subdev is a lens.
>>> s/sensor/camera sensor/
>>> s/lens/lens controller/
>>>
>>>> + *
>>>> + * TODO: Setup pad links if the notifier's function is MEDIA_ENT_F_VID_IF_BRIDGE
>>>> + * and the subdev's is MEDIA_ENT_F_CAM_SENSOR
>>>> + */
>>>> +static int v4l2_async_try_create_links(struct v4l2_async_notifier *notifier,
>>>> +				       struct v4l2_subdev *sd)
>>>> +{
>>>> +	if (!notifier->sd)
>>>> +		return 0;
>>>> +
>>>> +	switch (notifier->sd->entity.function) {
>>>> +	case MEDIA_ENT_F_CAM_SENSOR:
>>>> +		return __v4l2_async_create_ancillary_link(notifier, sd);
>>>> +	default:
>>>> +		return 0;
>>>> +	}
>>>> +}
>>>> +
>>>>  static int v4l2_async_match_notify(struct v4l2_async_notifier *notifier,
>>>>  				   struct v4l2_device *v4l2_dev,
>>>>  				   struct v4l2_subdev *sd,
>>>> @@ -293,6 +332,18 @@ static int v4l2_async_match_notify(struct v4l2_async_notifier *notifier,
>>>>  		return ret;
>>>>  	}
>>>>  
>>>> +	/*
>>>> +	 * Depending of the function of the entities involved, we may want to
>>>> +	 * create links between them (for example between a sensor and its lens
>>>> +	 * or between a sensor's source pad and the connected device's sink
>>>> +	 * pad)
>>> s/)/)./
>>>
>>>> +	 */
>>>> +	ret = v4l2_async_try_create_links(notifier, sd);
>>>> +	if (ret) {
>>>> +		v4l2_device_unregister_subdev(sd);
>>>> +		return ret;
>>>> +	}
>>>> +
>>>>  	/* Remove from the waiting list */
>>>>  	list_del(&asd->list);
>>>>  	sd->asd = asd;