[libcamera-devel] [PATCH] qcam: Support OpenGL ES 2.0
Laurent Pinchart
laurent.pinchart at ideasonboard.com
Tue Aug 3 11:41:57 CEST 2021
Hi Kieran,
On Tue, Aug 03, 2021 at 10:31:23AM +0100, Kieran Bingham wrote:
> On 03/08/2021 10:23, Laurent Pinchart wrote:
> > The GL_RG and GL_RED texture formats are not supported in OpenGL ES
> > prior to 3.0. In order to be compatible with OpenGL ES 2.0, use
> > GL_LUMINANCE_ALPHA and GL_LUMINANCE instead. The shader code needs to be
> > updated accordingly for GL_RG, as the second component is now stored in
>
> /in alpha/in the alpha/
>
> > alpha component instead of the green component. Usage of the red
> > component is fine, the luminance value is stored in the red, green and
> > blue components.
>
> So these seem to be just fairly arbitrary identifiers of the texture
> maps to use, which as long as they are consistently referenced that's
> all that matters?
They describe how the texture is stored in memory, and how it's
converted to RGBA before going through the shaders. See
https://docs.gl/es2/glTexImage2D for more information. In particular,
GL_LUMINANCE
Each element is a single luminance value. The GL converts it to
floating point, then assembles it into an RGBA element by
replicating the luminance value three times for red, green, and blue
and attaching 1 for alpha. Each component is then clamped to the
range [0,1].
GL_LUMINANCE_ALPHA
Each element is a luminance/alpha pair. The GL converts it to
floating point, then assembles it into an RGBA element by
replicating the luminance value three times for red, green, and
blue. Each component is then clamped to the range [0,1].
In ES3, we additionally have
(https://www.khronos.org/registry/OpenGL-Refpages/es3.0/html/glTexImage2D.xhtml),
GL_RED
Each element is a single red component. For fixed point normalized
components, the GL converts it to floating point, clamps to the
range [0,1], and assembles it into an RGBA element by attaching 0.0
for green and blue, and 1.0 for alpha.
GL_RG
Each element is a red/green double. For fixed point normalized
components, the GL converts each component to floating point, clamps
to the range [0,1], and assembles them into an RGBA element by
attaching 0.0 for blue, and 1.0 for alpha.
GL_RED maps the value to [v, 0.0, 0.0, 1.0] while GL_LUMINANCE maps it
to [v, v, v, 1.0]. As the shaders use the red component only, they're
equivalent. For GL_RG, however, when moving to GL_LUMINANCE_ALPHA, we
need to replace usage for the green component with the alpha component.
> > Signed-off-by: Laurent Pinchart <laurent.pinchart at ideasonboard.com>
> > ---
> > Tested with vivid, using NV12, YV420, YVU420 and SGRBG8, which should
> > cover all code paths.
> > ---
> > src/qcam/assets/shader/YUV_2_planes.frag | 4 +--
> > src/qcam/viewfinder_gl.cpp | 40 ++++++++++++------------
> > 2 files changed, 22 insertions(+), 22 deletions(-)
> >
> > diff --git a/src/qcam/assets/shader/YUV_2_planes.frag b/src/qcam/assets/shader/YUV_2_planes.frag
> > index 125f1c850a33..254463c05cac 100644
> > --- a/src/qcam/assets/shader/YUV_2_planes.frag
> > +++ b/src/qcam/assets/shader/YUV_2_planes.frag
> > @@ -26,9 +26,9 @@ void main(void)
> > yuv.x = texture2D(tex_y, textureOut).r - 0.063;
> > #if defined(YUV_PATTERN_UV)
> > yuv.y = texture2D(tex_u, textureOut).r - 0.500;
> > - yuv.z = texture2D(tex_u, textureOut).g - 0.500;
> > + yuv.z = texture2D(tex_u, textureOut).a - 0.500;
> > #elif defined(YUV_PATTERN_VU)
> > - yuv.y = texture2D(tex_u, textureOut).g - 0.500;
> > + yuv.y = texture2D(tex_u, textureOut).a - 0.500;
> > yuv.z = texture2D(tex_u, textureOut).r - 0.500;
> > #else
> > #error Invalid pattern
> > diff --git a/src/qcam/viewfinder_gl.cpp b/src/qcam/viewfinder_gl.cpp
> > index e7c8620c7024..77a6437e56fd 100644
> > --- a/src/qcam/viewfinder_gl.cpp
> > +++ b/src/qcam/viewfinder_gl.cpp
> > @@ -481,11 +481,11 @@ void ViewFinderGL::doRender()
> > configureTexture(*textures_[0]);
> > glTexImage2D(GL_TEXTURE_2D,
> > 0,
> > - GL_RED,
> > + GL_LUMINANCE,
> > size_.width(),
> > size_.height(),
> > 0,
> > - GL_RED,
> > + GL_LUMINANCE,
> > GL_UNSIGNED_BYTE,
> > data_);
> > shaderProgram_.setUniformValue(textureUniformY_, 0);
> > @@ -495,11 +495,11 @@ void ViewFinderGL::doRender()
> > configureTexture(*textures_[1]);
> > glTexImage2D(GL_TEXTURE_2D,
> > 0,
> > - GL_RG,
> > + GL_LUMINANCE_ALPHA,
> > size_.width() / horzSubSample_,
> > size_.height() / vertSubSample_,
> > 0,
> > - GL_RG,
> > + GL_LUMINANCE_ALPHA,
> > GL_UNSIGNED_BYTE,
> > data_ + size_.width() * size_.height());
> > shaderProgram_.setUniformValue(textureUniformU_, 1);
> > @@ -511,11 +511,11 @@ void ViewFinderGL::doRender()
> > configureTexture(*textures_[0]);
> > glTexImage2D(GL_TEXTURE_2D,
> > 0,
> > - GL_RED,
> > + GL_LUMINANCE,
> > size_.width(),
> > size_.height(),
> > 0,
> > - GL_RED,
> > + GL_LUMINANCE,
> > GL_UNSIGNED_BYTE,
> > data_);
> > shaderProgram_.setUniformValue(textureUniformY_, 0);
> > @@ -525,11 +525,11 @@ void ViewFinderGL::doRender()
> > configureTexture(*textures_[1]);
> > glTexImage2D(GL_TEXTURE_2D,
> > 0,
> > - GL_RED,
> > + GL_LUMINANCE,
> > size_.width() / horzSubSample_,
> > size_.height() / vertSubSample_,
> > 0,
> > - GL_RED,
> > + GL_LUMINANCE,
> > GL_UNSIGNED_BYTE,
> > data_ + size_.width() * size_.height());
> > shaderProgram_.setUniformValue(textureUniformU_, 1);
> > @@ -539,11 +539,11 @@ void ViewFinderGL::doRender()
> > configureTexture(*textures_[2]);
> > glTexImage2D(GL_TEXTURE_2D,
> > 0,
> > - GL_RED,
> > + GL_LUMINANCE,
> > size_.width() / horzSubSample_,
> > size_.height() / vertSubSample_,
> > 0,
> > - GL_RED,
> > + GL_LUMINANCE,
> > GL_UNSIGNED_BYTE,
> > data_ + size_.width() * size_.height() * 5 / 4);
> > shaderProgram_.setUniformValue(textureUniformV_, 2);
> > @@ -555,11 +555,11 @@ void ViewFinderGL::doRender()
> > configureTexture(*textures_[0]);
> > glTexImage2D(GL_TEXTURE_2D,
> > 0,
> > - GL_RED,
> > + GL_LUMINANCE,
> > size_.width(),
> > size_.height(),
> > 0,
> > - GL_RED,
> > + GL_LUMINANCE,
> > GL_UNSIGNED_BYTE,
> > data_);
> > shaderProgram_.setUniformValue(textureUniformY_, 0);
> > @@ -569,11 +569,11 @@ void ViewFinderGL::doRender()
> > configureTexture(*textures_[2]);
> > glTexImage2D(GL_TEXTURE_2D,
> > 0,
> > - GL_RED,
> > + GL_LUMINANCE,
> > size_.width() / horzSubSample_,
> > size_.height() / vertSubSample_,
> > 0,
> > - GL_RED,
> > + GL_LUMINANCE,
> > GL_UNSIGNED_BYTE,
> > data_ + size_.width() * size_.height());
> > shaderProgram_.setUniformValue(textureUniformV_, 2);
> > @@ -583,11 +583,11 @@ void ViewFinderGL::doRender()
> > configureTexture(*textures_[1]);
> > glTexImage2D(GL_TEXTURE_2D,
> > 0,
> > - GL_RED,
> > + GL_LUMINANCE,
> > size_.width() / horzSubSample_,
> > size_.height() / vertSubSample_,
> > 0,
> > - GL_RED,
> > + GL_LUMINANCE,
> > GL_UNSIGNED_BYTE,
> > data_ + size_.width() * size_.height() * 5 / 4);
> > shaderProgram_.setUniformValue(textureUniformU_, 1);
> > @@ -674,18 +674,18 @@ void ViewFinderGL::doRender()
> > case libcamera::formats::SRGGB12_CSI2P:
> > /*
> > * Raw Bayer 8-bit, and packed raw Bayer 10-bit/12-bit formats
> > - * are stored in GL_RED texture.
> > - * The texture width is equal to the stride.
> > + * are stored in GL_LUMINANCE texture. The texture width is
>
> I know you're modifying an existing sentence, but should this read either:
>
> - are stored in a GL_LUMINANCE texture
> or
> - are stored in the GL_LUMINANCE texture
>
> depending on if there is only one GL_LUMINANCE texture, or if there
> might be more and this is specifically allocated.
I'll fix it, using "a GL_LUMINANCE texture".
> (That extra clarification would help me understand those identifiers
> too... such power from a single word addition).
>
> Anyway, aside from / with that:
>
> Reviewed-by: Kieran Bingham <kieran.bingham at ideasonboard.com>
>
> > + * equal to the stride.
> > */
> > glActiveTexture(GL_TEXTURE0);
> > configureTexture(*textures_[0]);
> > glTexImage2D(GL_TEXTURE_2D,
> > 0,
> > - GL_RED,
> > + GL_LUMINANCE,
> > stride_,
> > size_.height(),
> > 0,
> > - GL_RED,
> > + GL_LUMINANCE,
> > GL_UNSIGNED_BYTE,
> > data_);
> > shaderProgram_.setUniformValue(textureUniformY_, 0);
> >
--
Regards,
Laurent Pinchart
More information about the libcamera-devel
mailing list