[libcamera-devel] [PATCH 3/3] gstreamer: Provide colorimetry <> libcamera::ColorSpace mappings
Nicolas Dufresne
nicolas.dufresne at collabora.com
Wed Jul 27 16:50:24 CEST 2022
Le dimanche 24 juillet 2022 à 23:48 +0530, Umang Jain via libcamera-devel a
écrit :
> Hi Rishi,
>
> On 7/24/22 22:03, Rishikesh Donadkar wrote:
> > > default:
> > > + GST_WARNING("Colorspace YcbcrEncoding not mapped in GstLibcameraSrc");
> > > + colorimetry.matrix = GST_VIDEO_COLOR_MATRIX_UNKNOWN;
> > > + }
> > Back in v1 of colorspace and colorimetry integration nicolas pointed
> > out that we should not use the
> > _UNKNOWN to set the fields.
> > https://lists.libcamera.org/pipermail/libcamera-devel/2022-July/031941.html
> > I am not sure whether we should set the field to _UNKNOWN in the
> > default switch cases.
>
>
> That's what is in happening here. It's a parsing error hence we use
> _UNKNOWN along with a warning.
>
> This means a identifier exists in the libcamera(probably a new
> identifier is added to libcamera) but not mapped in gstlibcamerasrc,
> hence, it shall pop the warning and the mapping for the new identifier
> needs to be added in gstlibcamerasrc-utils :-)
>
> As discussed on the IRC, we are working on case-by-case basis here, so
> as we get reports of missing identifiers, we shall extend the mappings.
> Hence the entire mapping/warnings has been set up that way.
Perhaps you should add some more info to the trace in you want to "get reports
of missing identifiers" ?
As said in previous answers, WARNING should come with proper mitigation, UNKNOWN
may lead to a pipeline error later, as its an invalid value.
> >
> >
> > On Sun, Jul 24, 2022 at 8:14 PM Umang Jain <umang.jain at ideasonboard.com> wrote:
> > > From: Rishikesh Donadkar <rishikeshdonadkar at gmail.com>
> > >
> > > Provide colorimetry <=> libcamera::ColorSpace mappings via:
> > > - GstVideoColorimetry colorimetry_from_colorspace(colorspace);
> > > - ColorSpace colorspace_from_colorimetry(colorimetry);
> > >
> > > Read the colorimetry field from caps into the stream configuration.
> > > After stream validation, the sensor supported colorimetry will
> > > be retrieved and the caps will be updated accordingly.
> > >
> > > Colorimetry support for gstlibcamerasrc currently undertakes only one
> > > argument. Multiple colorimetry support shall be introduced in
> > > subsequent commits.
> > >
> > > Signed-off-by: Rishikesh Donadkar <rishikeshdonadkar at gmail.com>
> > > Signed-off-by: Umang Jain <umang.jain at ideasonboard.com>
> > > ---
> > > src/gstreamer/gstlibcamera-utils.cpp | 175 +++++++++++++++++++++++++++
> > > 1 file changed, 175 insertions(+)
> > >
> > > diff --git a/src/gstreamer/gstlibcamera-utils.cpp b/src/gstreamer/gstlibcamera-utils.cpp
> > > index c97c0d43..fb4f0e5c 100644
> > > --- a/src/gstreamer/gstlibcamera-utils.cpp
> > > +++ b/src/gstreamer/gstlibcamera-utils.cpp
> > > @@ -45,6 +45,157 @@ static struct {
> > > /* \todo NV42 is used in libcamera but is not mapped in GStreamer yet. */
> > > };
> > >
> > > +static GstVideoColorimetry
> > > +colorimetry_from_colorspace(const ColorSpace &colorSpace)
> > > +{
> > > + GstVideoColorimetry colorimetry;
> > > +
> > > + switch (colorSpace.primaries) {
> > > + case ColorSpace::Primaries::Rec709:
> > > + colorimetry.primaries = GST_VIDEO_COLOR_PRIMARIES_BT709;
> > > + break;
> > > + case ColorSpace::Primaries::Rec2020:
> > > + colorimetry.primaries = GST_VIDEO_COLOR_PRIMARIES_BT2020;
> > > + break;
> > > + case ColorSpace::Primaries::Smpte170m:
> > > + colorimetry.primaries = GST_VIDEO_COLOR_PRIMARIES_SMPTE170M;
> > > + break;
> > > + default:
> > > + GST_WARNING("ColorSpace primaries not mapped in GstLibcameraSrc");
> > > + colorimetry.primaries = GST_VIDEO_COLOR_PRIMARIES_UNKNOWN;
> > > + }
> > > +
> > > + switch (colorSpace.transferFunction) {
> > > + case ColorSpace::TransferFunction::Rec709:
> > > + colorimetry.transfer = GST_VIDEO_TRANSFER_BT709;
> > > + break;
> > > + case ColorSpace::TransferFunction::Srgb:
> > > + colorimetry.transfer = GST_VIDEO_TRANSFER_SRGB;
> > > + break;
> > > + case ColorSpace::TransferFunction::Linear:
> > > + colorimetry.transfer = GST_VIDEO_TRANSFER_GAMMA10;
> > > + break;
> > > + default:
> > > + GST_WARNING("ColorSpace transfer function not mapped in GstLibcameraSrc");
> > > + colorimetry.transfer = GST_VIDEO_TRANSFER_UNKNOWN;
> > > + }
> > > +
> > > + switch (colorSpace.ycbcrEncoding) {
> > > + case ColorSpace::YcbcrEncoding::Rec709:
> > > + colorimetry.matrix = GST_VIDEO_COLOR_MATRIX_BT709;
> > > + break;
> > > + case ColorSpace::YcbcrEncoding::Rec2020:
> > > + colorimetry.matrix = GST_VIDEO_COLOR_MATRIX_BT2020;
> > > + break;
> > > + case ColorSpace::YcbcrEncoding::Rec601:
> > > + colorimetry.matrix = GST_VIDEO_COLOR_MATRIX_BT601;
> > > + break;
> > > + default:
> > > + GST_WARNING("Colorspace YcbcrEncoding not mapped in GstLibcameraSrc");
> > > + colorimetry.matrix = GST_VIDEO_COLOR_MATRIX_UNKNOWN;
> > > + }
> > > +
> > > + switch (colorSpace.range) {
> > > + case ColorSpace::Range::Full:
> > > + colorimetry.range = GST_VIDEO_COLOR_RANGE_0_255;
> > > + break;
> > > + case ColorSpace::Range::Limited:
> > > + colorimetry.range = GST_VIDEO_COLOR_RANGE_16_235;
> > > + break;
> > > + default:
> > > + GST_WARNING("Colorspace range not mapped in GstLibcameraSrc");
> > > + colorimetry.range = GST_VIDEO_COLOR_RANGE_UNKNOWN;
> > > + }
> > > +
> > > + return colorimetry;
> > > +}
> > > +
> > > +static std::optional<ColorSpace>
> > > +colorspace_from_colorimetry(GstVideoColorimetry *colorimetry)
> > > +{
> > > + std::optional<ColorSpace> colorspace = ColorSpace::Default;
> > > +
> > > + switch (colorimetry->primaries) {
> > > + case GST_VIDEO_COLOR_PRIMARIES_BT709:
> > > + colorspace->primaries = ColorSpace::Primaries::Rec709;
> > > + break;
> > > + case GST_VIDEO_COLOR_PRIMARIES_BT2020:
> > > + colorspace->primaries = ColorSpace::Primaries::Rec2020;
> > > + break;
> > > + case GST_VIDEO_COLOR_PRIMARIES_SMPTE170M:
> > > + colorspace->primaries = ColorSpace::Primaries::Smpte170m;
> > > + break;
> > > + case GST_VIDEO_COLOR_PRIMARIES_UNKNOWN:
> > > + colorspace->primaries = ColorSpace::Primaries::Default;
> > > + break;
> > > + default:
> > > + GST_WARNING("Unknown primaries in colorimetry %d", colorimetry->primaries);
> > > + }
> > > +
> > > + switch (colorimetry->transfer) {
> > > + /* Tranfer function mappings inspired from v4l2src plugin */
> > > + case GST_VIDEO_TRANSFER_GAMMA18:
> > > + case GST_VIDEO_TRANSFER_GAMMA20:
> > > + case GST_VIDEO_TRANSFER_GAMMA22:
> > > + case GST_VIDEO_TRANSFER_GAMMA28:
> > > + GST_WARNING("GAMMA 18, 20, 22, 28 transfer functions not supported");
> > > + /* fallthrough */
> > > + case GST_VIDEO_TRANSFER_GAMMA10:
> > > + colorspace->transferFunction = ColorSpace::TransferFunction::Linear;
> > > + break;
> > > + case GST_VIDEO_TRANSFER_BT601:
> > > + case GST_VIDEO_TRANSFER_BT2020_12:
> > > + case GST_VIDEO_TRANSFER_BT2020_10:
> > > + case GST_VIDEO_TRANSFER_BT709:
> > > + colorspace->transferFunction = ColorSpace::TransferFunction::Rec709;
> > > + break;
> > > + case GST_VIDEO_TRANSFER_SRGB:
> > > + colorspace->transferFunction = ColorSpace::TransferFunction::Srgb;
> > > + break;
> > > + case GST_VIDEO_TRANSFER_UNKNOWN:
> > > + colorspace->transferFunction = ColorSpace::TransferFunction::Default;
> > > + break;
> > > + default:
> > > + GST_WARNING("Unknown colorimetry transfer %d", colorimetry->transfer);
> > > + }
> > > +
> > > + switch (colorimetry->matrix) {
> > > + /* FCC is about the same as BT601 with less digit */
> > > + case GST_VIDEO_COLOR_MATRIX_FCC:
> > > + case GST_VIDEO_COLOR_MATRIX_BT601:
> > > + colorspace->ycbcrEncoding = ColorSpace::YcbcrEncoding::Rec601;
> > > + break;
> > > + case GST_VIDEO_COLOR_MATRIX_BT709:
> > > + colorspace->ycbcrEncoding = ColorSpace::YcbcrEncoding::Rec709;
> > > + break;
> > > + case GST_VIDEO_COLOR_MATRIX_BT2020:
> > > + colorspace->ycbcrEncoding = ColorSpace::YcbcrEncoding::Rec2020;
> > > + break;
> > > + case GST_VIDEO_COLOR_MATRIX_RGB:
> > > + case GST_VIDEO_COLOR_MATRIX_UNKNOWN:
> > > + colorspace->ycbcrEncoding = ColorSpace::YcbcrEncoding::Default;
> > > + break;
> > > + default:
> > > + GST_WARNING("Unknown colorimetry matrix %d", colorimetry->matrix);
> > > + }
> > > +
> > > + switch (colorimetry->range) {
> > > + case GST_VIDEO_COLOR_RANGE_0_255:
> > > + colorspace->range = ColorSpace::Range::Full;
> > > + break;
> > > + case GST_VIDEO_COLOR_RANGE_16_235:
> > > + colorspace->range = ColorSpace::Range::Limited;
> > > + break;
> > > + case GST_VIDEO_COLOR_RANGE_UNKNOWN:
> > > + colorspace->range = ColorSpace::Range::Default;
> > > + break;
> > > + default:
> > > + GST_WARNING("Unknown range in colorimetry %d", colorimetry->range);
> > > + }
> > > +
> > > + return colorspace;
> > > +}
> > > +
> > > static GstVideoFormat
> > > pixel_format_to_gst_format(const PixelFormat &format)
> > > {
> > > @@ -139,6 +290,17 @@ gst_libcamera_stream_configuration_to_caps(const StreamConfiguration &stream_cfg
> > > "width", G_TYPE_INT, stream_cfg.size.width,
> > > "height", G_TYPE_INT, stream_cfg.size.height,
> > > nullptr);
> > > +
> > > + if (stream_cfg.colorSpace) {
> > > + GstVideoColorimetry colorimetry = colorimetry_from_colorspace(stream_cfg.colorSpace.value());
> > > + gchar *colorimetry_str = gst_video_colorimetry_to_string(&colorimetry);
> > > +
> > > + if (colorimetry_str != nullptr)
> > > + gst_structure_set(s, "colorimetry", G_TYPE_STRING, colorimetry_str, nullptr);
> > > + else
> > > + g_warning("libcamera::ColorSpace found but GstVideoColorimetry unknown");
> > > + }
> > > +
> > > gst_caps_append_structure(caps, s);
> > >
> > > return caps;
> > > @@ -222,6 +384,19 @@ gst_libcamera_configure_stream_from_caps(StreamConfiguration &stream_cfg,
> > > gst_structure_get_int(s, "height", &height);
> > > stream_cfg.size.width = width;
> > > stream_cfg.size.height = height;
> > > +
> > > + /* Configure colorimetry */
> > > + if (gst_structure_has_field(s, "colorimetry")) {
> > > + const gchar *colorimetry_caps = gst_structure_get_string(s, "colorimetry");
> > > + GstVideoColorimetry colorimetry;
> > > +
> > > + if(gst_video_colorimetry_from_string(&colorimetry, colorimetry_caps)) {
> > > + std::optional<ColorSpace> colorSpace = colorspace_from_colorimetry(&colorimetry);
> > > + stream_cfg.colorSpace = colorSpace;
> > > + } else {
> > > + g_print("Invalid colorimetry %s", colorimetry_caps);
> > > + }
> > > + }
> > > }
> > >
> > > #if !GST_CHECK_VERSION(1, 17, 1)
> > > --
> > > 2.31.1
> > >
More information about the libcamera-devel
mailing list