[libcamera-devel] [PATCH] qcam: Support OpenGL ES 2.0
Andrey Konovalov
andrey.konovalov at linaro.org
Tue Aug 3 22:28:30 CEST 2021
Hi Laurent,
Making qcam to work with OpenGL ES 2.0 is a nice improvement!
On 03.08.2021 12:41, Laurent Pinchart wrote:
> Hi Kieran,
>
> On Tue, Aug 03, 2021 at 10:31:23AM +0100, Kieran Bingham wrote:
>> On 03/08/2021 10:23, Laurent Pinchart wrote:
>>> The GL_RG and GL_RED texture formats are not supported in OpenGL ES
>>> prior to 3.0. In order to be compatible with OpenGL ES 2.0, use
>>> GL_LUMINANCE_ALPHA and GL_LUMINANCE instead. The shader code needs to be
>>> updated accordingly for GL_RG, as the second component is now stored in
>>
>> /in alpha/in the alpha/
>>
>>> alpha component instead of the green component. Usage of the red
>>> component is fine, the luminance value is stored in the red, green and
>>> blue components.
>>
>> So these seem to be just fairly arbitrary identifiers of the texture
>> maps to use, which as long as they are consistently referenced that's
>> all that matters?
>
> They describe how the texture is stored in memory, and how it's
> converted to RGBA before going through the shaders. See
> https://docs.gl/es2/glTexImage2D for more information. In particular,
>
> GL_LUMINANCE
>
> Each element is a single luminance value. The GL converts it to
> floating point, then assembles it into an RGBA element by
> replicating the luminance value three times for red, green, and blue
> and attaching 1 for alpha. Each component is then clamped to the
> range [0,1].
>
> GL_LUMINANCE_ALPHA
>
> Each element is a luminance/alpha pair. The GL converts it to
> floating point, then assembles it into an RGBA element by
> replicating the luminance value three times for red, green, and
> blue. Each component is then clamped to the range [0,1].
>
> In ES3, we additionally have
> (https://www.khronos.org/registry/OpenGL-Refpages/es3.0/html/glTexImage2D.xhtml),
>
> GL_RED
>
> Each element is a single red component. For fixed point normalized
> components, the GL converts it to floating point, clamps to the
> range [0,1], and assembles it into an RGBA element by attaching 0.0
> for green and blue, and 1.0 for alpha.
>
> GL_RG
>
> Each element is a red/green double. For fixed point normalized
> components, the GL converts each component to floating point, clamps
> to the range [0,1], and assembles them into an RGBA element by
> attaching 0.0 for blue, and 1.0 for alpha.
>
> GL_RED maps the value to [v, 0.0, 0.0, 1.0] while GL_LUMINANCE maps it
> to [v, v, v, 1.0]. As the shaders use the red component only, they're
> equivalent. For GL_RG, however, when moving to GL_LUMINANCE_ALPHA, we
> need to replace usage for the green component with the alpha component.
Thanks for the explanation and the docs links!
>>> Signed-off-by: Laurent Pinchart <laurent.pinchart at ideasonboard.com>
>>> ---
>>> Tested with vivid, using NV12, YV420, YVU420 and SGRBG8, which should
>>> cover all code paths.
Reviewed-by: Andrey Konovalov <andrey.konovalov at linaro.org>
(Tested on RB3 board with camss using SGRBG10_CSI2P - just in case)
Thanks,
Andrey
>>> ---
>>> src/qcam/assets/shader/YUV_2_planes.frag | 4 +--
>>> src/qcam/viewfinder_gl.cpp | 40 ++++++++++++------------
>>> 2 files changed, 22 insertions(+), 22 deletions(-)
>>>
>>> diff --git a/src/qcam/assets/shader/YUV_2_planes.frag b/src/qcam/assets/shader/YUV_2_planes.frag
>>> index 125f1c850a33..254463c05cac 100644
>>> --- a/src/qcam/assets/shader/YUV_2_planes.frag
>>> +++ b/src/qcam/assets/shader/YUV_2_planes.frag
>>> @@ -26,9 +26,9 @@ void main(void)
>>> yuv.x = texture2D(tex_y, textureOut).r - 0.063;
>>> #if defined(YUV_PATTERN_UV)
>>> yuv.y = texture2D(tex_u, textureOut).r - 0.500;
>>> - yuv.z = texture2D(tex_u, textureOut).g - 0.500;
>>> + yuv.z = texture2D(tex_u, textureOut).a - 0.500;
>>> #elif defined(YUV_PATTERN_VU)
>>> - yuv.y = texture2D(tex_u, textureOut).g - 0.500;
>>> + yuv.y = texture2D(tex_u, textureOut).a - 0.500;
>>> yuv.z = texture2D(tex_u, textureOut).r - 0.500;
>>> #else
>>> #error Invalid pattern
>>> diff --git a/src/qcam/viewfinder_gl.cpp b/src/qcam/viewfinder_gl.cpp
>>> index e7c8620c7024..77a6437e56fd 100644
>>> --- a/src/qcam/viewfinder_gl.cpp
>>> +++ b/src/qcam/viewfinder_gl.cpp
>>> @@ -481,11 +481,11 @@ void ViewFinderGL::doRender()
>>> configureTexture(*textures_[0]);
>>> glTexImage2D(GL_TEXTURE_2D,
>>> 0,
>>> - GL_RED,
>>> + GL_LUMINANCE,
>>> size_.width(),
>>> size_.height(),
>>> 0,
>>> - GL_RED,
>>> + GL_LUMINANCE,
>>> GL_UNSIGNED_BYTE,
>>> data_);
>>> shaderProgram_.setUniformValue(textureUniformY_, 0);
>>> @@ -495,11 +495,11 @@ void ViewFinderGL::doRender()
>>> configureTexture(*textures_[1]);
>>> glTexImage2D(GL_TEXTURE_2D,
>>> 0,
>>> - GL_RG,
>>> + GL_LUMINANCE_ALPHA,
>>> size_.width() / horzSubSample_,
>>> size_.height() / vertSubSample_,
>>> 0,
>>> - GL_RG,
>>> + GL_LUMINANCE_ALPHA,
>>> GL_UNSIGNED_BYTE,
>>> data_ + size_.width() * size_.height());
>>> shaderProgram_.setUniformValue(textureUniformU_, 1);
>>> @@ -511,11 +511,11 @@ void ViewFinderGL::doRender()
>>> configureTexture(*textures_[0]);
>>> glTexImage2D(GL_TEXTURE_2D,
>>> 0,
>>> - GL_RED,
>>> + GL_LUMINANCE,
>>> size_.width(),
>>> size_.height(),
>>> 0,
>>> - GL_RED,
>>> + GL_LUMINANCE,
>>> GL_UNSIGNED_BYTE,
>>> data_);
>>> shaderProgram_.setUniformValue(textureUniformY_, 0);
>>> @@ -525,11 +525,11 @@ void ViewFinderGL::doRender()
>>> configureTexture(*textures_[1]);
>>> glTexImage2D(GL_TEXTURE_2D,
>>> 0,
>>> - GL_RED,
>>> + GL_LUMINANCE,
>>> size_.width() / horzSubSample_,
>>> size_.height() / vertSubSample_,
>>> 0,
>>> - GL_RED,
>>> + GL_LUMINANCE,
>>> GL_UNSIGNED_BYTE,
>>> data_ + size_.width() * size_.height());
>>> shaderProgram_.setUniformValue(textureUniformU_, 1);
>>> @@ -539,11 +539,11 @@ void ViewFinderGL::doRender()
>>> configureTexture(*textures_[2]);
>>> glTexImage2D(GL_TEXTURE_2D,
>>> 0,
>>> - GL_RED,
>>> + GL_LUMINANCE,
>>> size_.width() / horzSubSample_,
>>> size_.height() / vertSubSample_,
>>> 0,
>>> - GL_RED,
>>> + GL_LUMINANCE,
>>> GL_UNSIGNED_BYTE,
>>> data_ + size_.width() * size_.height() * 5 / 4);
>>> shaderProgram_.setUniformValue(textureUniformV_, 2);
>>> @@ -555,11 +555,11 @@ void ViewFinderGL::doRender()
>>> configureTexture(*textures_[0]);
>>> glTexImage2D(GL_TEXTURE_2D,
>>> 0,
>>> - GL_RED,
>>> + GL_LUMINANCE,
>>> size_.width(),
>>> size_.height(),
>>> 0,
>>> - GL_RED,
>>> + GL_LUMINANCE,
>>> GL_UNSIGNED_BYTE,
>>> data_);
>>> shaderProgram_.setUniformValue(textureUniformY_, 0);
>>> @@ -569,11 +569,11 @@ void ViewFinderGL::doRender()
>>> configureTexture(*textures_[2]);
>>> glTexImage2D(GL_TEXTURE_2D,
>>> 0,
>>> - GL_RED,
>>> + GL_LUMINANCE,
>>> size_.width() / horzSubSample_,
>>> size_.height() / vertSubSample_,
>>> 0,
>>> - GL_RED,
>>> + GL_LUMINANCE,
>>> GL_UNSIGNED_BYTE,
>>> data_ + size_.width() * size_.height());
>>> shaderProgram_.setUniformValue(textureUniformV_, 2);
>>> @@ -583,11 +583,11 @@ void ViewFinderGL::doRender()
>>> configureTexture(*textures_[1]);
>>> glTexImage2D(GL_TEXTURE_2D,
>>> 0,
>>> - GL_RED,
>>> + GL_LUMINANCE,
>>> size_.width() / horzSubSample_,
>>> size_.height() / vertSubSample_,
>>> 0,
>>> - GL_RED,
>>> + GL_LUMINANCE,
>>> GL_UNSIGNED_BYTE,
>>> data_ + size_.width() * size_.height() * 5 / 4);
>>> shaderProgram_.setUniformValue(textureUniformU_, 1);
>>> @@ -674,18 +674,18 @@ void ViewFinderGL::doRender()
>>> case libcamera::formats::SRGGB12_CSI2P:
>>> /*
>>> * Raw Bayer 8-bit, and packed raw Bayer 10-bit/12-bit formats
>>> - * are stored in GL_RED texture.
>>> - * The texture width is equal to the stride.
>>> + * are stored in GL_LUMINANCE texture. The texture width is
>>
>> I know you're modifying an existing sentence, but should this read either:
>>
>> - are stored in a GL_LUMINANCE texture
>> or
>> - are stored in the GL_LUMINANCE texture
>>
>> depending on if there is only one GL_LUMINANCE texture, or if there
>> might be more and this is specifically allocated.
>
> I'll fix it, using "a GL_LUMINANCE texture".
>
>> (That extra clarification would help me understand those identifiers
>> too... such power from a single word addition).
>>
>> Anyway, aside from / with that:
>>
>> Reviewed-by: Kieran Bingham <kieran.bingham at ideasonboard.com>
>>
>>> + * equal to the stride.
>>> */
>>> glActiveTexture(GL_TEXTURE0);
>>> configureTexture(*textures_[0]);
>>> glTexImage2D(GL_TEXTURE_2D,
>>> 0,
>>> - GL_RED,
>>> + GL_LUMINANCE,
>>> stride_,
>>> size_.height(),
>>> 0,
>>> - GL_RED,
>>> + GL_LUMINANCE,
>>> GL_UNSIGNED_BYTE,
>>> data_);
>>> shaderProgram_.setUniformValue(textureUniformY_, 0);
>>>
>
More information about the libcamera-devel
mailing list