[libcamera-devel] [PATCH] gstreamer: src: Send image-orientation tag

[Umang Jain]

Hi Robert,

On 6/14/23 9:51 PM, Robert Mader via libcamera-devel wrote:
> Matching the validated Transform of the configuration, if the transform
> is not the Identity. As we currently always request a configuration with
> `Identity` transform, this can be the case if camera has a rotation
> property (90/180/270 degree) and the sensor can't do rotation in
> question. Sending the orientation-tag allows downstream elements to
> compensate accordingly.
>
> This matches current pipewiresrc behavior and can be tested with e.g.:
> gst-launch-1.0 libcamerasrc ! videoconvert ! videoflip video-direction=auto ! gtksink
>
> Signed-off-by: Robert Mader <robert.mader at collabora.com>
> ---
>   src/gstreamer/gstlibcamera-utils.cpp | 24 ++++++++++++++++++++++++
>   src/gstreamer/gstlibcamera-utils.h   |  3 +++
>   src/gstreamer/gstlibcamerasrc.cpp    | 11 +++++++++++
>   3 files changed, 38 insertions(+)
>
> diff --git a/src/gstreamer/gstlibcamera-utils.cpp b/src/gstreamer/gstlibcamera-utils.cpp
> index 750ec351..3bfbc354 100644
> --- a/src/gstreamer/gstlibcamera-utils.cpp
> +++ b/src/gstreamer/gstlibcamera-utils.cpp
> @@ -553,3 +553,27 @@ gst_libcamera_get_camera_manager(int &ret)
>   
>   	return cm;
>   }
> +
> +const char *
> +gst_libcamera_transform_to_tag_string(libcamera::Transform transform)
> +{
> +	switch (transform) {
> +	case Transform::Rot90:
> +		return "rotate-90";
> +	case Transform::Rot180:
> +		return "rotate-180";
> +	case Transform::Rot270:
> +		return "rotate-270";
> +	case Transform::HFlip:
> +		return "flip-rotate-0";
> +	case Transform::VFlip:
> +		return "flip-rotate-180";
> +	case Transform::Transpose:
> +		return "flip-rotate-270";
> +	case Transform::Rot180Transpose:
> +		return "flip-rotate-90";
> +	case Transform::Identity:
> +	default:
> +		return "rotate-0";
> +	}
> +}
> diff --git a/src/gstreamer/gstlibcamera-utils.h b/src/gstreamer/gstlibcamera-utils.h
> index fd304a8b..499533c4 100644
> --- a/src/gstreamer/gstlibcamera-utils.h
> +++ b/src/gstreamer/gstlibcamera-utils.h
> @@ -11,6 +11,7 @@
>   #include <libcamera/camera_manager.h>
>   #include <libcamera/controls.h>
>   #include <libcamera/stream.h>
> +#include <libcamera/transform.h>
>   
>   #include <gst/gst.h>
>   #include <gst/video/video.h>
> @@ -30,6 +31,8 @@ gboolean gst_task_resume(GstTask *task);
>   #endif
>   std::shared_ptr<libcamera::CameraManager> gst_libcamera_get_camera_manager(int &ret);
>   
> +const char *gst_libcamera_transform_to_tag_string(libcamera::Transform transform);
> +
>   /**
>    * \class GLibLocker
>    * \brief A simple scoped mutex locker for GMutex
> diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp
> index a10cbd4f..b41f52d7 100644
> --- a/src/gstreamer/gstlibcamerasrc.cpp
> +++ b/src/gstreamer/gstlibcamerasrc.cpp
> @@ -461,6 +461,7 @@ gst_libcamera_src_task_enter(GstTask *task, [[maybe_unused]] GThread *thread,
>   	GLibRecLocker lock(&self->stream_lock);
>   	GstLibcameraSrcState *state = self->state;
>   	GstFlowReturn flow_ret = GST_FLOW_OK;
> +	const char *transform_tag;
>   	gint ret;
>   
>   	g_autoptr(GstStructure) element_caps = gst_structure_new_empty("caps");
> @@ -519,6 +520,16 @@ gst_libcamera_src_task_enter(GstTask *task, [[maybe_unused]] GThread *thread,
>   		goto done;
>   	}
>   
> +	transform_tag = gst_libcamera_transform_to_tag_string(state->config_->transform);

Reading the transform isn't enough. I think the camera property 
'rotation' should be reported instead when the transform is not 
requested (or it can be set to identity for default)

There will be cases I believe when the image-orientation should be set 
to transform as well - those is when users supplies a transform and it 
applies to the camera as expected. For cases were the transform cannot 
be applied, we need to detect that and report property::rotation again? 
I am not sure on the latter front right now...

> +	for (gsize i = 0; i < state->srcpads_.size(); i++) {
> +		GstPad *srcpad = state->srcpads_[i];
> +		GstEvent *tag_event;
> +
> +		tag_event = gst_event_new_tag(gst_tag_list_new(GST_TAG_IMAGE_ORIENTATION,
> +							       transform_tag, NULL));
> +		gst_pad_push_event(srcpad, tag_event);
> +	}
> +
>   	ret = state->cam_->configure(state->config_.get());
>   	if (ret) {
>   		GST_ELEMENT_ERROR(self, RESOURCE, SETTINGS,