[libcamera-devel] [PATCH] qcam: Support OpenGL ES 2.0
Laurent Pinchart
laurent.pinchart at ideasonboard.com
Tue Aug 3 22:38:19 CEST 2021
Hi Andrey,
On Tue, Aug 03, 2021 at 11:28:30PM +0300, Andrey Konovalov wrote:
> Hi Laurent,
>
> Making qcam to work with OpenGL ES 2.0 is a nice improvement!
>
> On 03.08.2021 12:41, Laurent Pinchart wrote:
> > On Tue, Aug 03, 2021 at 10:31:23AM +0100, Kieran Bingham wrote:
> >> On 03/08/2021 10:23, Laurent Pinchart wrote:
> >>> The GL_RG and GL_RED texture formats are not supported in OpenGL ES
> >>> prior to 3.0. In order to be compatible with OpenGL ES 2.0, use
> >>> GL_LUMINANCE_ALPHA and GL_LUMINANCE instead. The shader code needs to be
> >>> updated accordingly for GL_RG, as the second component is now stored in
> >>
> >> /in alpha/in the alpha/
> >>
> >>> alpha component instead of the green component. Usage of the red
> >>> component is fine, the luminance value is stored in the red, green and
> >>> blue components.
> >>
> >> So these seem to be just fairly arbitrary identifiers of the texture
> >> maps to use, which as long as they are consistently referenced that's
> >> all that matters?
> >
> > They describe how the texture is stored in memory, and how it's
> > converted to RGBA before going through the shaders. See
> > https://docs.gl/es2/glTexImage2D for more information. In particular,
> >
> > GL_LUMINANCE
> >
> > Each element is a single luminance value. The GL converts it to
> > floating point, then assembles it into an RGBA element by
> > replicating the luminance value three times for red, green, and blue
> > and attaching 1 for alpha. Each component is then clamped to the
> > range [0,1].
> >
> > GL_LUMINANCE_ALPHA
> >
> > Each element is a luminance/alpha pair. The GL converts it to
> > floating point, then assembles it into an RGBA element by
> > replicating the luminance value three times for red, green, and
> > blue. Each component is then clamped to the range [0,1].
> >
> > In ES3, we additionally have
> > (https://www.khronos.org/registry/OpenGL-Refpages/es3.0/html/glTexImage2D.xhtml),
> >
> > GL_RED
> >
> > Each element is a single red component. For fixed point normalized
> > components, the GL converts it to floating point, clamps to the
> > range [0,1], and assembles it into an RGBA element by attaching 0.0
> > for green and blue, and 1.0 for alpha.
> >
> > GL_RG
> >
> > Each element is a red/green double. For fixed point normalized
> > components, the GL converts each component to floating point, clamps
> > to the range [0,1], and assembles them into an RGBA element by
> > attaching 0.0 for blue, and 1.0 for alpha.
> >
> > GL_RED maps the value to [v, 0.0, 0.0, 1.0] while GL_LUMINANCE maps it
> > to [v, v, v, 1.0]. As the shaders use the red component only, they're
> > equivalent. For GL_RG, however, when moving to GL_LUMINANCE_ALPHA, we
> > need to replace usage for the green component with the alpha component.
>
> Thanks for the explanation and the docs links!
>
> >>> Signed-off-by: Laurent Pinchart <laurent.pinchart at ideasonboard.com>
> >>> ---
> >>> Tested with vivid, using NV12, YV420, YVU420 and SGRBG8, which should
> >>> cover all code paths.
>
> Reviewed-by: Andrey Konovalov <andrey.konovalov at linaro.org>
>
> (Tested on RB3 board with camss using SGRBG10_CSI2P - just in case)
That's the only format family that I haven't been able to test :-)
Thanks.
> >>> ---
> >>> src/qcam/assets/shader/YUV_2_planes.frag | 4 +--
> >>> src/qcam/viewfinder_gl.cpp | 40 ++++++++++++------------
> >>> 2 files changed, 22 insertions(+), 22 deletions(-)
> >>>
> >>> diff --git a/src/qcam/assets/shader/YUV_2_planes.frag b/src/qcam/assets/shader/YUV_2_planes.frag
> >>> index 125f1c850a33..254463c05cac 100644
> >>> --- a/src/qcam/assets/shader/YUV_2_planes.frag
> >>> +++ b/src/qcam/assets/shader/YUV_2_planes.frag
> >>> @@ -26,9 +26,9 @@ void main(void)
> >>> yuv.x = texture2D(tex_y, textureOut).r - 0.063;
> >>> #if defined(YUV_PATTERN_UV)
> >>> yuv.y = texture2D(tex_u, textureOut).r - 0.500;
> >>> - yuv.z = texture2D(tex_u, textureOut).g - 0.500;
> >>> + yuv.z = texture2D(tex_u, textureOut).a - 0.500;
> >>> #elif defined(YUV_PATTERN_VU)
> >>> - yuv.y = texture2D(tex_u, textureOut).g - 0.500;
> >>> + yuv.y = texture2D(tex_u, textureOut).a - 0.500;
> >>> yuv.z = texture2D(tex_u, textureOut).r - 0.500;
> >>> #else
> >>> #error Invalid pattern
> >>> diff --git a/src/qcam/viewfinder_gl.cpp b/src/qcam/viewfinder_gl.cpp
> >>> index e7c8620c7024..77a6437e56fd 100644
> >>> --- a/src/qcam/viewfinder_gl.cpp
> >>> +++ b/src/qcam/viewfinder_gl.cpp
> >>> @@ -481,11 +481,11 @@ void ViewFinderGL::doRender()
> >>> configureTexture(*textures_[0]);
> >>> glTexImage2D(GL_TEXTURE_2D,
> >>> 0,
> >>> - GL_RED,
> >>> + GL_LUMINANCE,
> >>> size_.width(),
> >>> size_.height(),
> >>> 0,
> >>> - GL_RED,
> >>> + GL_LUMINANCE,
> >>> GL_UNSIGNED_BYTE,
> >>> data_);
> >>> shaderProgram_.setUniformValue(textureUniformY_, 0);
> >>> @@ -495,11 +495,11 @@ void ViewFinderGL::doRender()
> >>> configureTexture(*textures_[1]);
> >>> glTexImage2D(GL_TEXTURE_2D,
> >>> 0,
> >>> - GL_RG,
> >>> + GL_LUMINANCE_ALPHA,
> >>> size_.width() / horzSubSample_,
> >>> size_.height() / vertSubSample_,
> >>> 0,
> >>> - GL_RG,
> >>> + GL_LUMINANCE_ALPHA,
> >>> GL_UNSIGNED_BYTE,
> >>> data_ + size_.width() * size_.height());
> >>> shaderProgram_.setUniformValue(textureUniformU_, 1);
> >>> @@ -511,11 +511,11 @@ void ViewFinderGL::doRender()
> >>> configureTexture(*textures_[0]);
> >>> glTexImage2D(GL_TEXTURE_2D,
> >>> 0,
> >>> - GL_RED,
> >>> + GL_LUMINANCE,
> >>> size_.width(),
> >>> size_.height(),
> >>> 0,
> >>> - GL_RED,
> >>> + GL_LUMINANCE,
> >>> GL_UNSIGNED_BYTE,
> >>> data_);
> >>> shaderProgram_.setUniformValue(textureUniformY_, 0);
> >>> @@ -525,11 +525,11 @@ void ViewFinderGL::doRender()
> >>> configureTexture(*textures_[1]);
> >>> glTexImage2D(GL_TEXTURE_2D,
> >>> 0,
> >>> - GL_RED,
> >>> + GL_LUMINANCE,
> >>> size_.width() / horzSubSample_,
> >>> size_.height() / vertSubSample_,
> >>> 0,
> >>> - GL_RED,
> >>> + GL_LUMINANCE,
> >>> GL_UNSIGNED_BYTE,
> >>> data_ + size_.width() * size_.height());
> >>> shaderProgram_.setUniformValue(textureUniformU_, 1);
> >>> @@ -539,11 +539,11 @@ void ViewFinderGL::doRender()
> >>> configureTexture(*textures_[2]);
> >>> glTexImage2D(GL_TEXTURE_2D,
> >>> 0,
> >>> - GL_RED,
> >>> + GL_LUMINANCE,
> >>> size_.width() / horzSubSample_,
> >>> size_.height() / vertSubSample_,
> >>> 0,
> >>> - GL_RED,
> >>> + GL_LUMINANCE,
> >>> GL_UNSIGNED_BYTE,
> >>> data_ + size_.width() * size_.height() * 5 / 4);
> >>> shaderProgram_.setUniformValue(textureUniformV_, 2);
> >>> @@ -555,11 +555,11 @@ void ViewFinderGL::doRender()
> >>> configureTexture(*textures_[0]);
> >>> glTexImage2D(GL_TEXTURE_2D,
> >>> 0,
> >>> - GL_RED,
> >>> + GL_LUMINANCE,
> >>> size_.width(),
> >>> size_.height(),
> >>> 0,
> >>> - GL_RED,
> >>> + GL_LUMINANCE,
> >>> GL_UNSIGNED_BYTE,
> >>> data_);
> >>> shaderProgram_.setUniformValue(textureUniformY_, 0);
> >>> @@ -569,11 +569,11 @@ void ViewFinderGL::doRender()
> >>> configureTexture(*textures_[2]);
> >>> glTexImage2D(GL_TEXTURE_2D,
> >>> 0,
> >>> - GL_RED,
> >>> + GL_LUMINANCE,
> >>> size_.width() / horzSubSample_,
> >>> size_.height() / vertSubSample_,
> >>> 0,
> >>> - GL_RED,
> >>> + GL_LUMINANCE,
> >>> GL_UNSIGNED_BYTE,
> >>> data_ + size_.width() * size_.height());
> >>> shaderProgram_.setUniformValue(textureUniformV_, 2);
> >>> @@ -583,11 +583,11 @@ void ViewFinderGL::doRender()
> >>> configureTexture(*textures_[1]);
> >>> glTexImage2D(GL_TEXTURE_2D,
> >>> 0,
> >>> - GL_RED,
> >>> + GL_LUMINANCE,
> >>> size_.width() / horzSubSample_,
> >>> size_.height() / vertSubSample_,
> >>> 0,
> >>> - GL_RED,
> >>> + GL_LUMINANCE,
> >>> GL_UNSIGNED_BYTE,
> >>> data_ + size_.width() * size_.height() * 5 / 4);
> >>> shaderProgram_.setUniformValue(textureUniformU_, 1);
> >>> @@ -674,18 +674,18 @@ void ViewFinderGL::doRender()
> >>> case libcamera::formats::SRGGB12_CSI2P:
> >>> /*
> >>> * Raw Bayer 8-bit, and packed raw Bayer 10-bit/12-bit formats
> >>> - * are stored in GL_RED texture.
> >>> - * The texture width is equal to the stride.
> >>> + * are stored in GL_LUMINANCE texture. The texture width is
> >>
> >> I know you're modifying an existing sentence, but should this read either:
> >>
> >> - are stored in a GL_LUMINANCE texture
> >> or
> >> - are stored in the GL_LUMINANCE texture
> >>
> >> depending on if there is only one GL_LUMINANCE texture, or if there
> >> might be more and this is specifically allocated.
> >
> > I'll fix it, using "a GL_LUMINANCE texture".
> >
> >> (That extra clarification would help me understand those identifiers
> >> too... such power from a single word addition).
> >>
> >> Anyway, aside from / with that:
> >>
> >> Reviewed-by: Kieran Bingham <kieran.bingham at ideasonboard.com>
> >>
> >>> + * equal to the stride.
> >>> */
> >>> glActiveTexture(GL_TEXTURE0);
> >>> configureTexture(*textures_[0]);
> >>> glTexImage2D(GL_TEXTURE_2D,
> >>> 0,
> >>> - GL_RED,
> >>> + GL_LUMINANCE,
> >>> stride_,
> >>> size_.height(),
> >>> 0,
> >>> - GL_RED,
> >>> + GL_LUMINANCE,
> >>> GL_UNSIGNED_BYTE,
> >>> data_);
> >>> shaderProgram_.setUniformValue(textureUniformY_, 0);
> >>>
--
Regards,
Laurent Pinchart
More information about the libcamera-devel
mailing list