[libcamera-devel] [PATCH 1/1] Add CrOS JEA implementation
Laurent Pinchart
laurent.pinchart at ideasonboard.com
Tue Apr 26 00:40:00 CEST 2022
Hi Harvey,
Thank you for the patch, and sorry for the late reply. Catching up with
e-mail after travel is painful. Next time I'll try to get the whole
world to travel at the same time, maybe I'll get less e-mails :-)
On Wed, Apr 06, 2022 at 05:41:30PM +0800, Harvey Yang via libcamera-devel wrote:
> This CL uses CrOS JpegCompressor with potential HW accelerator to do
> JPEG encoding.
>
> As CrOS JpegCompressor might need file descriptors to get the source
> data and pass the jpeg result, this CL extends FrameBuffer in the
> android source code as Android_FrameBuffer, which stores the
> buffer_handle_t when constructing the frame buffer, and adds a
> getter function to access it.
>
> This CL also redefines src/android/jpeg/encoder interfaces and adds
> Encoder::generateThumbnail, which might also be accelerated by CrOS
> HW. It simplifies PostProcessorJpeg's logic when generating the
> thumbnail. The original implementation is then moved into the
> EncoderLibJpeg::generateThumbnail.
This is missing a Signed-off-by line, see
Documentation/contributing.rst.
> ---
> include/libcamera/framebuffer.h | 3 +-
> src/android/android_framebuffer.cpp | 32 ++++++++
> src/android/android_framebuffer.h | 28 +++++++
> src/android/camera_device.cpp | 3 +-
> src/android/cros/camera3_hal.cpp | 3 +
> src/android/frame_buffer_allocator.h | 37 +++++----
> src/android/jpeg/cros_post_processor_jpeg.cpp | 14 ++++
> src/android/jpeg/encoder.h | 9 +-
> src/android/jpeg/encoder_jea.cpp | 82 +++++++++++++++++++
> src/android/jpeg/encoder_jea.h | 35 ++++++++
> src/android/jpeg/encoder_libjpeg.cpp | 70 ++++++++++++++++
> src/android/jpeg/encoder_libjpeg.h | 21 ++++-
> .../jpeg/generic_post_processor_jpeg.cpp | 14 ++++
> src/android/jpeg/meson.build | 16 ++++
> src/android/jpeg/post_processor_jpeg.cpp | 60 ++------------
> src/android/jpeg/post_processor_jpeg.h | 11 +--
> src/android/meson.build | 6 +-
> .../mm/cros_frame_buffer_allocator.cpp | 13 +--
> .../mm/generic_frame_buffer_allocator.cpp | 11 +--
There are lots of changes here, making this hard to review. Could you
please split this patch in pieces, with one logical change by patch, and
bundle them as a series ? Candidates are
- Drop the final keyword from FrameBuffer and make the destructor
virtual
- Add AndroidFrameBuffer and use it in the HAL (you could even split
that in two if desired, but bundling a new class with its user(s) can
make review easier, if the result isn't too big)
- Rework the JPEG encoder API and implementation to prepare for the
needs of JEA
- Add the JEA implementation
> 19 files changed, 367 insertions(+), 101 deletions(-)
> create mode 100644 src/android/android_framebuffer.cpp
> create mode 100644 src/android/android_framebuffer.h
> create mode 100644 src/android/jpeg/cros_post_processor_jpeg.cpp
> create mode 100644 src/android/jpeg/encoder_jea.cpp
> create mode 100644 src/android/jpeg/encoder_jea.h
> create mode 100644 src/android/jpeg/generic_post_processor_jpeg.cpp
> create mode 100644 src/android/jpeg/meson.build
>
> diff --git a/include/libcamera/framebuffer.h b/include/libcamera/framebuffer.h
> index de172d97..c902cc18 100644
> --- a/include/libcamera/framebuffer.h
> +++ b/include/libcamera/framebuffer.h
> @@ -46,7 +46,7 @@ private:
> std::vector<Plane> planes_;
> };
>
> -class FrameBuffer final : public Extensible
> +class FrameBuffer : public Extensible
> {
> LIBCAMERA_DECLARE_PRIVATE()
>
> @@ -61,6 +61,7 @@ public:
> FrameBuffer(const std::vector<Plane> &planes, unsigned int cookie = 0);
> FrameBuffer(std::unique_ptr<Private> d,
> const std::vector<Plane> &planes, unsigned int cookie = 0);
> + virtual ~FrameBuffer() {}
>
> const std::vector<Plane> &planes() const { return planes_; }
> Request *request() const;
> diff --git a/src/android/android_framebuffer.cpp b/src/android/android_framebuffer.cpp
> new file mode 100644
> index 00000000..1ff7018e
> --- /dev/null
> +++ b/src/android/android_framebuffer.cpp
> @@ -0,0 +1,32 @@
> +/* SPDX-License-Identifier: LGPL-2.1-or-later */
> +/*
> + * Copyright (C) 2022, Google Inc.
> + *
> + * android_framebuffer.cpp - Android Frame buffer handling
> + */
> +
> +#include "android_framebuffer.h"
> +
> +#include <hardware/camera3.h>
> +
> +AndroidFrameBuffer::AndroidFrameBuffer(
> + buffer_handle_t handle,
> + std::unique_ptr<Private> d,
> + const std::vector<Plane> &planes,
> + unsigned int cookie)
> + : FrameBuffer(std::move(d), planes, cookie), handle_(handle)
> +{
> +}
> +
> +AndroidFrameBuffer::AndroidFrameBuffer(
> + buffer_handle_t handle,
> + const std::vector<Plane> &planes,
> + unsigned int cookie)
> + : FrameBuffer(planes, cookie), handle_(handle)
> +{
> +}
> +
> +buffer_handle_t AndroidFrameBuffer::getHandle() const
> +{
> + return handle_;
> +}
> diff --git a/src/android/android_framebuffer.h b/src/android/android_framebuffer.h
> new file mode 100644
> index 00000000..49df9756
> --- /dev/null
> +++ b/src/android/android_framebuffer.h
> @@ -0,0 +1,28 @@
> +/* SPDX-License-Identifier: LGPL-2.1-or-later */
> +/*
> + * Copyright (C) 2022, Google Inc.
> + *
> + * android_framebuffer.h - Android Frame buffer handling
> + */
> +
> +#pragma once
> +
> +#include "libcamera/internal/framebuffer.h"
> +
> +#include <hardware/camera3.h>
> +
> +class AndroidFrameBuffer final : public libcamera::FrameBuffer
> +{
> +public:
> + AndroidFrameBuffer(
> + buffer_handle_t handle, std::unique_ptr<Private> d,
> + const std::vector<Plane> &planes,
> + unsigned int cookie = 0);
> + AndroidFrameBuffer(buffer_handle_t handle,
> + const std::vector<Plane> &planes,
> + unsigned int cookie = 0);
> + buffer_handle_t getHandle() const;
> +
> +private:
> + buffer_handle_t handle_ = nullptr;
> +};
> diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp
> index 00d48471..643b4dee 100644
> --- a/src/android/camera_device.cpp
> +++ b/src/android/camera_device.cpp
> @@ -25,6 +25,7 @@
>
> #include "system/graphics.h"
>
> +#include "android_framebuffer.h"
> #include "camera_buffer.h"
> #include "camera_hal_config.h"
> #include "camera_ops.h"
> @@ -754,7 +755,7 @@ CameraDevice::createFrameBuffer(const buffer_handle_t camera3buffer,
> planes[i].length = buf.size(i);
> }
>
> - return std::make_unique<FrameBuffer>(planes);
> + return std::make_unique<AndroidFrameBuffer>(camera3buffer, planes);
> }
>
> int CameraDevice::processControls(Camera3RequestDescriptor *descriptor)
> diff --git a/src/android/cros/camera3_hal.cpp b/src/android/cros/camera3_hal.cpp
> index fb863b5f..ea5577f0 100644
> --- a/src/android/cros/camera3_hal.cpp
> +++ b/src/android/cros/camera3_hal.cpp
> @@ -9,8 +9,11 @@
>
> #include "../camera_hal_manager.h"
>
> +cros::CameraMojoChannelManagerToken *g_cros_camera_token = nullptr;
> +
> static void set_up([[maybe_unused]] cros::CameraMojoChannelManagerToken *token)
> {
> + g_cros_camera_token = token;
> }
>
> static void tear_down()
> diff --git a/src/android/frame_buffer_allocator.h b/src/android/frame_buffer_allocator.h
> index 5d2eeda1..e26422a3 100644
> --- a/src/android/frame_buffer_allocator.h
> +++ b/src/android/frame_buffer_allocator.h
> @@ -13,9 +13,10 @@
> #include <libcamera/base/class.h>
>
> #include <libcamera/camera.h>
> -#include <libcamera/framebuffer.h>
> #include <libcamera/geometry.h>
>
> +#include "android_framebuffer.h"
> +
> class CameraDevice;
>
> class PlatformFrameBufferAllocator : libcamera::Extensible
> @@ -31,25 +32,25 @@ public:
> * Note: The returned FrameBuffer needs to be destroyed before
> * PlatformFrameBufferAllocator is destroyed.
> */
> - std::unique_ptr<libcamera::FrameBuffer> allocate(
> + std::unique_ptr<AndroidFrameBuffer> allocate(
> int halPixelFormat, const libcamera::Size &size, uint32_t usage);
> };
>
> -#define PUBLIC_FRAME_BUFFER_ALLOCATOR_IMPLEMENTATION \
> -PlatformFrameBufferAllocator::PlatformFrameBufferAllocator( \
> - CameraDevice *const cameraDevice) \
> - : Extensible(std::make_unique<Private>(cameraDevice)) \
> -{ \
> -} \
> -PlatformFrameBufferAllocator::~PlatformFrameBufferAllocator() \
> -{ \
> -} \
> -std::unique_ptr<libcamera::FrameBuffer> \
> -PlatformFrameBufferAllocator::allocate(int halPixelFormat, \
> - const libcamera::Size &size, \
> - uint32_t usage) \
> -{ \
> - return _d()->allocate(halPixelFormat, size, usage); \
> -}
> +#define PUBLIC_FRAME_BUFFER_ALLOCATOR_IMPLEMENTATION \
> + PlatformFrameBufferAllocator::PlatformFrameBufferAllocator( \
> + CameraDevice *const cameraDevice) \
> + : Extensible(std::make_unique<Private>(cameraDevice)) \
> + { \
> + } \
> + PlatformFrameBufferAllocator::~PlatformFrameBufferAllocator() \
> + { \
> + } \
> + std::unique_ptr<AndroidFrameBuffer> \
> + PlatformFrameBufferAllocator::allocate(int halPixelFormat, \
> + const libcamera::Size &size, \
> + uint32_t usage) \
> + { \
> + return _d()->allocate(halPixelFormat, size, usage); \
> + }
>
> #endif /* __ANDROID_FRAME_BUFFER_ALLOCATOR_H__ */
> diff --git a/src/android/jpeg/cros_post_processor_jpeg.cpp b/src/android/jpeg/cros_post_processor_jpeg.cpp
> new file mode 100644
> index 00000000..7020f0d0
> --- /dev/null
> +++ b/src/android/jpeg/cros_post_processor_jpeg.cpp
> @@ -0,0 +1,14 @@
> +/* SPDX-License-Identifier: LGPL-2.1-or-later */
> +/*
> + * Copyright (C) 2022, Google Inc.
> + *
> + * cros_post_processor_jpeg.cpp - CrOS JPEG Post Processor
> + */
> +
> +#include "encoder_jea.h"
> +#include "post_processor_jpeg.h"
> +
> +void PostProcessorJpeg::SetEncoder()
> +{
> + encoder_ = std::make_unique<EncoderJea>();
> +}
> diff --git a/src/android/jpeg/encoder.h b/src/android/jpeg/encoder.h
> index b974d367..6d527d91 100644
> --- a/src/android/jpeg/encoder.h
> +++ b/src/android/jpeg/encoder.h
> @@ -12,14 +12,19 @@
> #include <libcamera/framebuffer.h>
> #include <libcamera/stream.h>
>
> +#include "../camera_request.h"
> +
> class Encoder
> {
> public:
> virtual ~Encoder() = default;
>
> virtual int configure(const libcamera::StreamConfiguration &cfg) = 0;
> - virtual int encode(const libcamera::FrameBuffer &source,
> - libcamera::Span<uint8_t> destination,
> + virtual int encode(Camera3RequestDescriptor::StreamBuffer *streamBuffer,
> libcamera::Span<const uint8_t> exifData,
> unsigned int quality) = 0;
> + virtual int generateThumbnail(const libcamera::FrameBuffer &source,
> + const libcamera::Size &targetSize,
> + unsigned int quality,
> + std::vector<unsigned char> *thumbnail) = 0;
> };
> diff --git a/src/android/jpeg/encoder_jea.cpp b/src/android/jpeg/encoder_jea.cpp
> new file mode 100644
> index 00000000..838e8647
> --- /dev/null
> +++ b/src/android/jpeg/encoder_jea.cpp
> @@ -0,0 +1,82 @@
> +/* SPDX-License-Identifier: LGPL-2.1-or-later */
> +/*
> + * Copyright (C) 2022, Google Inc.
> + *
> + * encoder_jea.cpp - JPEG encoding using CrOS JEA
> + */
> +
> +#include "encoder_jea.h"
> +
> +#include <libcamera/base/log.h>
> +
> +#include "libcamera/internal/mapped_framebuffer.h"
> +
> +#include <cros-camera/camera_mojo_channel_manager_token.h>
> +
> +#include "../android_framebuffer.h"
> +
> +extern cros::CameraMojoChannelManagerToken *g_cros_camera_token;
> +
> +EncoderJea::EncoderJea() = default;
> +
> +EncoderJea::~EncoderJea() = default;
> +
> +int EncoderJea::configure(const libcamera::StreamConfiguration &cfg)
> +{
> + size_ = cfg.size;
> +
> + if (jpeg_compressor_.get())
> + return 0;
> +
> + if (g_cros_camera_token == nullptr)
> + return -ENOTSUP;
> +
> + jpeg_compressor_ = cros::JpegCompressor::GetInstance(g_cros_camera_token);
> +
> + return 0;
> +}
> +
> +int EncoderJea::encode(Camera3RequestDescriptor::StreamBuffer *streamBuffer,
> + libcamera::Span<const uint8_t> exifData,
> + unsigned int quality)
> +{
> + if (!jpeg_compressor_.get())
> + return -1;
> +
> + uint32_t out_data_size = 0;
> +
> + if (!jpeg_compressor_->CompressImageFromHandle(
> + dynamic_cast<const AndroidFrameBuffer *>(
> + streamBuffer->srcBuffer)
> + ->getHandle(),
> + *streamBuffer->camera3Buffer, size_.width, size_.height, quality,
> + exifData.data(), exifData.size(), &out_data_size)) {
> + return -1;
> + }
> +
> + return out_data_size;
> +}
> +
> +int EncoderJea::generateThumbnail(const libcamera::FrameBuffer &source,
> + const libcamera::Size &targetSize,
> + unsigned int quality,
> + std::vector<unsigned char> *thumbnail)
> +{
> + if (!jpeg_compressor_.get())
> + return -1;
> +
> + libcamera::MappedFrameBuffer frame(&source, libcamera::MappedFrameBuffer::MapFlag::Read);
> +
> + if (frame.planes().empty())
> + return 0;
> +
> + uint32_t out_data_size = 0;
> +
> + if (!jpeg_compressor_->GenerateThumbnail(frame.planes()[0].data(),
> + size_.width, size_.height, targetSize.width, targetSize.height,
> + quality, thumbnail->size(), thumbnail->data(), &out_data_size)) {
> + return -1;
> + }
> +
> + return out_data_size;
> +}
> diff --git a/src/android/jpeg/encoder_jea.h b/src/android/jpeg/encoder_jea.h
> new file mode 100644
> index 00000000..d5c9f1f7
> --- /dev/null
> +++ b/src/android/jpeg/encoder_jea.h
> @@ -0,0 +1,35 @@
> +/* SPDX-License-Identifier: LGPL-2.1-or-later */
> +/*
> + * Copyright (C) 2022, Google Inc.
> + *
> + * encoder_jea.h - JPEG encoding using CrOS JEA
> + */
> +
> +#pragma once
> +
> +#include <libcamera/geometry.h>
> +
> +#include <cros-camera/jpeg_compressor.h>
> +
> +#include "encoder.h"
> +
> +class EncoderJea : public Encoder
> +{
> +public:
> + EncoderJea();
> + ~EncoderJea();
> +
> + int configure(const libcamera::StreamConfiguration &cfg) override;
> + int encode(Camera3RequestDescriptor::StreamBuffer *streamBuffer,
> + libcamera::Span<const uint8_t> exifData,
> + unsigned int quality) override;
> + int generateThumbnail(const libcamera::FrameBuffer &source,
> + const libcamera::Size &targetSize,
> + unsigned int quality,
> + std::vector<unsigned char> *thumbnail) override;
> +
> +private:
> + libcamera::Size size_;
> +
> + std::unique_ptr<cros::JpegCompressor> jpeg_compressor_;
> +};
> diff --git a/src/android/jpeg/encoder_libjpeg.cpp b/src/android/jpeg/encoder_libjpeg.cpp
> index 21a3b33d..b5591e33 100644
> --- a/src/android/jpeg/encoder_libjpeg.cpp
> +++ b/src/android/jpeg/encoder_libjpeg.cpp
> @@ -24,6 +24,8 @@
> #include "libcamera/internal/formats.h"
> #include "libcamera/internal/mapped_framebuffer.h"
>
> +#include "../camera_buffer.h"
> +
> using namespace libcamera;
>
> LOG_DECLARE_CATEGORY(JPEG)
> @@ -82,8 +84,17 @@ EncoderLibJpeg::~EncoderLibJpeg()
> }
>
> int EncoderLibJpeg::configure(const StreamConfiguration &cfg)
> +{
> + thumbnailer_.configure(cfg.size, cfg.pixelFormat);
> + cfg_ = cfg;
> +
> + return internalConfigure(cfg);
> +}
> +
> +int EncoderLibJpeg::internalConfigure(const StreamConfiguration &cfg)
> {
> const struct JPEGPixelFormatInfo info = findPixelInfo(cfg.pixelFormat);
> +
> if (info.colorSpace == JCS_UNKNOWN)
> return -ENOTSUP;
>
> @@ -178,6 +189,65 @@ void EncoderLibJpeg::compressNV(const std::vector<Span<uint8_t>> &planes)
> }
> }
>
> +int EncoderLibJpeg::encode(Camera3RequestDescriptor::StreamBuffer *streamBuffer,
> + libcamera::Span<const uint8_t> exifData,
> + unsigned int quality)
> +{
> + internalConfigure(cfg_);
> + return encode(*streamBuffer->srcBuffer, streamBuffer->dstBuffer.get()->plane(0), exifData, quality);
> +}
> +
> +int EncoderLibJpeg::generateThumbnail(
> + const libcamera::FrameBuffer &source,
> + const libcamera::Size &targetSize,
> + unsigned int quality,
> + std::vector<unsigned char> *thumbnail)
> +{
> + /* Stores the raw scaled-down thumbnail bytes. */
> + std::vector<unsigned char> rawThumbnail;
> +
> + thumbnailer_.createThumbnail(source, targetSize, &rawThumbnail);
> +
> + StreamConfiguration thCfg;
> + thCfg.size = targetSize;
> + thCfg.pixelFormat = thumbnailer_.pixelFormat();
> + int ret = internalConfigure(thCfg);
> +
> + if (!rawThumbnail.empty() && !ret) {
> + /*
> + * \todo Avoid value-initialization of all elements of the
> + * vector.
> + */
> + thumbnail->resize(rawThumbnail.size());
> +
> + /*
> + * Split planes manually as the encoder expects a vector of
> + * planes.
> + *
> + * \todo Pass a vector of planes directly to
> + * Thumbnailer::createThumbnailer above and remove the manual
> + * planes split from here.
> + */
> + std::vector<Span<uint8_t>> thumbnailPlanes;
> + const PixelFormatInfo &formatNV12 = PixelFormatInfo::info(formats::NV12);
> + size_t yPlaneSize = formatNV12.planeSize(targetSize, 0);
> + size_t uvPlaneSize = formatNV12.planeSize(targetSize, 1);
> + thumbnailPlanes.push_back({ rawThumbnail.data(), yPlaneSize });
> + thumbnailPlanes.push_back({ rawThumbnail.data() + yPlaneSize, uvPlaneSize });
> +
> + int jpeg_size = encode(thumbnailPlanes, *thumbnail, {}, quality);
> + thumbnail->resize(jpeg_size);
> +
> + LOG(JPEG, Debug)
> + << "Thumbnail compress returned "
> + << jpeg_size << " bytes";
> +
> + return jpeg_size;
> + }
> +
> + return -1;
> +}
> +
> int EncoderLibJpeg::encode(const FrameBuffer &source, Span<uint8_t> dest,
> Span<const uint8_t> exifData, unsigned int quality)
> {
> diff --git a/src/android/jpeg/encoder_libjpeg.h b/src/android/jpeg/encoder_libjpeg.h
> index 1b3ac067..56b27bae 100644
> --- a/src/android/jpeg/encoder_libjpeg.h
> +++ b/src/android/jpeg/encoder_libjpeg.h
> @@ -15,6 +15,8 @@
>
> #include <jpeglib.h>
>
> +#include "thumbnailer.h"
> +
> class EncoderLibJpeg : public Encoder
> {
> public:
> @@ -22,19 +24,32 @@ public:
> ~EncoderLibJpeg();
>
> int configure(const libcamera::StreamConfiguration &cfg) override;
> + int encode(Camera3RequestDescriptor::StreamBuffer *streamBuffer,
> + libcamera::Span<const uint8_t> exifData,
> + unsigned int quality) override;
> + int generateThumbnail(
> + const libcamera::FrameBuffer &source,
> + const libcamera::Size &targetSize,
> + unsigned int quality,
> + std::vector<unsigned char> *thumbnail) override;
> +
> +private:
> + int internalConfigure(const libcamera::StreamConfiguration &cfg);
> +
> int encode(const libcamera::FrameBuffer &source,
> libcamera::Span<uint8_t> destination,
> libcamera::Span<const uint8_t> exifData,
> - unsigned int quality) override;
> + unsigned int quality);
> int encode(const std::vector<libcamera::Span<uint8_t>> &planes,
> libcamera::Span<uint8_t> destination,
> libcamera::Span<const uint8_t> exifData,
> unsigned int quality);
>
> -private:
> void compressRGB(const std::vector<libcamera::Span<uint8_t>> &planes);
> void compressNV(const std::vector<libcamera::Span<uint8_t>> &planes);
>
> + libcamera::StreamConfiguration cfg_;
> +
> struct jpeg_compress_struct compress_;
> struct jpeg_error_mgr jerr_;
>
> @@ -42,4 +57,6 @@ private:
>
> bool nv_;
> bool nvSwap_;
> +
> + Thumbnailer thumbnailer_;
> };
> diff --git a/src/android/jpeg/generic_post_processor_jpeg.cpp b/src/android/jpeg/generic_post_processor_jpeg.cpp
> new file mode 100644
> index 00000000..890f6972
> --- /dev/null
> +++ b/src/android/jpeg/generic_post_processor_jpeg.cpp
> @@ -0,0 +1,14 @@
> +/* SPDX-License-Identifier: LGPL-2.1-or-later */
> +/*
> + * Copyright (C) 2022, Google Inc.
> + *
> + * generic_post_processor_jpeg.cpp - Generic JPEG Post Processor
> + */
> +
> +#include "encoder_libjpeg.h"
> +#include "post_processor_jpeg.h"
> +
> +void PostProcessorJpeg::SetEncoder()
> +{
> + encoder_ = std::make_unique<EncoderLibJpeg>();
> +}
> diff --git a/src/android/jpeg/meson.build b/src/android/jpeg/meson.build
> new file mode 100644
> index 00000000..8606acc4
> --- /dev/null
> +++ b/src/android/jpeg/meson.build
> @@ -0,0 +1,16 @@
> +# SPDX-License-Identifier: CC0-1.0
> +
> +android_hal_sources += files([
> + 'exif.cpp',
> + 'post_processor_jpeg.cpp'])
> +
> +platform = get_option('android_platform')
> +if platform == 'generic'
> + android_hal_sources += files(['encoder_libjpeg.cpp',
> + 'generic_post_processor_jpeg.cpp',
> + 'thumbnailer.cpp'])
> +elif platform == 'cros'
> + android_hal_sources += files(['cros_post_processor_jpeg.cpp',
> + 'encoder_jea.cpp'])
> + android_deps += [dependency('libcros_camera')]
> +endif
> diff --git a/src/android/jpeg/post_processor_jpeg.cpp b/src/android/jpeg/post_processor_jpeg.cpp
> index d72ebc3c..7ceba60e 100644
> --- a/src/android/jpeg/post_processor_jpeg.cpp
> +++ b/src/android/jpeg/post_processor_jpeg.cpp
> @@ -9,10 +9,10 @@
>
> #include <chrono>
>
> +#include "../android_framebuffer.h"
> #include "../camera_device.h"
> #include "../camera_metadata.h"
> #include "../camera_request.h"
> -#include "encoder_libjpeg.h"
> #include "exif.h"
>
> #include <libcamera/base/log.h>
> @@ -44,60 +44,11 @@ int PostProcessorJpeg::configure(const StreamConfiguration &inCfg,
>
> streamSize_ = outCfg.size;
>
> - thumbnailer_.configure(inCfg.size, inCfg.pixelFormat);
> -
> - encoder_ = std::make_unique<EncoderLibJpeg>();
> + SetEncoder();
>
> return encoder_->configure(inCfg);
> }
>
> -void PostProcessorJpeg::generateThumbnail(const FrameBuffer &source,
> - const Size &targetSize,
> - unsigned int quality,
> - std::vector<unsigned char> *thumbnail)
> -{
> - /* Stores the raw scaled-down thumbnail bytes. */
> - std::vector<unsigned char> rawThumbnail;
> -
> - thumbnailer_.createThumbnail(source, targetSize, &rawThumbnail);
> -
> - StreamConfiguration thCfg;
> - thCfg.size = targetSize;
> - thCfg.pixelFormat = thumbnailer_.pixelFormat();
> - int ret = thumbnailEncoder_.configure(thCfg);
> -
> - if (!rawThumbnail.empty() && !ret) {
> - /*
> - * \todo Avoid value-initialization of all elements of the
> - * vector.
> - */
> - thumbnail->resize(rawThumbnail.size());
> -
> - /*
> - * Split planes manually as the encoder expects a vector of
> - * planes.
> - *
> - * \todo Pass a vector of planes directly to
> - * Thumbnailer::createThumbnailer above and remove the manual
> - * planes split from here.
> - */
> - std::vector<Span<uint8_t>> thumbnailPlanes;
> - const PixelFormatInfo &formatNV12 = PixelFormatInfo::info(formats::NV12);
> - size_t yPlaneSize = formatNV12.planeSize(targetSize, 0);
> - size_t uvPlaneSize = formatNV12.planeSize(targetSize, 1);
> - thumbnailPlanes.push_back({ rawThumbnail.data(), yPlaneSize });
> - thumbnailPlanes.push_back({ rawThumbnail.data() + yPlaneSize, uvPlaneSize });
> -
> - int jpeg_size = thumbnailEncoder_.encode(thumbnailPlanes,
> - *thumbnail, {}, quality);
> - thumbnail->resize(jpeg_size);
> -
> - LOG(JPEG, Debug)
> - << "Thumbnail compress returned "
> - << jpeg_size << " bytes";
> - }
> -}
> -
> void PostProcessorJpeg::process(Camera3RequestDescriptor::StreamBuffer *streamBuffer)
> {
> ASSERT(encoder_);
> @@ -164,8 +115,8 @@ void PostProcessorJpeg::process(Camera3RequestDescriptor::StreamBuffer *streamBu
>
> if (thumbnailSize != Size(0, 0)) {
> std::vector<unsigned char> thumbnail;
> - generateThumbnail(source, thumbnailSize, quality, &thumbnail);
> - if (!thumbnail.empty())
> + ret = encoder_->generateThumbnail(source, thumbnailSize, quality, &thumbnail);
> + if (ret > 0 && !thumbnail.empty())
> exif.setThumbnail(thumbnail, Exif::Compression::JPEG);
> }
>
> @@ -194,8 +145,7 @@ void PostProcessorJpeg::process(Camera3RequestDescriptor::StreamBuffer *streamBu
> const uint8_t quality = ret ? *entry.data.u8 : 95;
> resultMetadata->addEntry(ANDROID_JPEG_QUALITY, quality);
>
> - int jpeg_size = encoder_->encode(source, destination->plane(0),
> - exif.data(), quality);
> + int jpeg_size = encoder_->encode(streamBuffer, exif.data(), quality);
> if (jpeg_size < 0) {
> LOG(JPEG, Error) << "Failed to encode stream image";
> processComplete.emit(streamBuffer, PostProcessor::Status::Error);
> diff --git a/src/android/jpeg/post_processor_jpeg.h b/src/android/jpeg/post_processor_jpeg.h
> index 98309b01..a09f8798 100644
> --- a/src/android/jpeg/post_processor_jpeg.h
> +++ b/src/android/jpeg/post_processor_jpeg.h
> @@ -8,11 +8,11 @@
> #pragma once
>
> #include "../post_processor.h"
> -#include "encoder_libjpeg.h"
> -#include "thumbnailer.h"
>
> #include <libcamera/geometry.h>
>
> +#include "encoder.h"
> +
> class CameraDevice;
>
> class PostProcessorJpeg : public PostProcessor
> @@ -25,14 +25,9 @@ public:
> void process(Camera3RequestDescriptor::StreamBuffer *streamBuffer) override;
>
> private:
> - void generateThumbnail(const libcamera::FrameBuffer &source,
> - const libcamera::Size &targetSize,
> - unsigned int quality,
> - std::vector<unsigned char> *thumbnail);
> + void SetEncoder();
>
> CameraDevice *const cameraDevice_;
> std::unique_ptr<Encoder> encoder_;
> libcamera::Size streamSize_;
> - EncoderLibJpeg thumbnailEncoder_;
> - Thumbnailer thumbnailer_;
> };
> diff --git a/src/android/meson.build b/src/android/meson.build
> index 75b4bf20..026b8b3c 100644
> --- a/src/android/meson.build
> +++ b/src/android/meson.build
> @@ -38,6 +38,7 @@ endif
> android_deps += [libyuv_dep]
>
> android_hal_sources = files([
> + 'android_framebuffer.cpp',
> 'camera3_hal.cpp',
> 'camera_capabilities.cpp',
> 'camera_device.cpp',
> @@ -47,10 +48,6 @@ android_hal_sources = files([
> 'camera_ops.cpp',
> 'camera_request.cpp',
> 'camera_stream.cpp',
> - 'jpeg/encoder_libjpeg.cpp',
> - 'jpeg/exif.cpp',
> - 'jpeg/post_processor_jpeg.cpp',
> - 'jpeg/thumbnailer.cpp',
> 'yuv/post_processor_yuv.cpp'
> ])
>
> @@ -58,6 +55,7 @@ android_cpp_args = []
>
> subdir('cros')
> subdir('mm')
> +subdir('jpeg')
>
> android_camera_metadata_sources = files([
> 'metadata/camera_metadata.c',
> diff --git a/src/android/mm/cros_frame_buffer_allocator.cpp b/src/android/mm/cros_frame_buffer_allocator.cpp
> index 52e8c180..163c5d75 100644
> --- a/src/android/mm/cros_frame_buffer_allocator.cpp
> +++ b/src/android/mm/cros_frame_buffer_allocator.cpp
> @@ -14,6 +14,7 @@
>
> #include "libcamera/internal/framebuffer.h"
>
> +#include "../android_framebuffer.h"
> #include "../camera_device.h"
> #include "../frame_buffer_allocator.h"
> #include "cros-camera/camera_buffer_manager.h"
> @@ -47,11 +48,11 @@ public:
> {
> }
>
> - std::unique_ptr<libcamera::FrameBuffer>
> + std::unique_ptr<AndroidFrameBuffer>
> allocate(int halPixelFormat, const libcamera::Size &size, uint32_t usage);
> };
>
> -std::unique_ptr<libcamera::FrameBuffer>
> +std::unique_ptr<AndroidFrameBuffer>
> PlatformFrameBufferAllocator::Private::allocate(int halPixelFormat,
> const libcamera::Size &size,
> uint32_t usage)
> @@ -80,9 +81,11 @@ PlatformFrameBufferAllocator::Private::allocate(int halPixelFormat,
> plane.length = cros::CameraBufferManager::GetPlaneSize(handle, i);
> }
>
> - return std::make_unique<FrameBuffer>(
> - std::make_unique<CrosFrameBufferData>(std::move(scopedHandle)),
> - planes);
> + auto fb = std::make_unique<AndroidFrameBuffer>(handle,
> + std::make_unique<CrosFrameBufferData>(std::move(scopedHandle)),
> + planes);
> +
> + return fb;
> }
>
> PUBLIC_FRAME_BUFFER_ALLOCATOR_IMPLEMENTATION
> diff --git a/src/android/mm/generic_frame_buffer_allocator.cpp b/src/android/mm/generic_frame_buffer_allocator.cpp
> index acb2fa2b..c79b7b10 100644
> --- a/src/android/mm/generic_frame_buffer_allocator.cpp
> +++ b/src/android/mm/generic_frame_buffer_allocator.cpp
> @@ -18,6 +18,7 @@
> #include <hardware/gralloc.h>
> #include <hardware/hardware.h>
>
> +#include "../android_framebuffer.h"
> #include "../camera_device.h"
> #include "../frame_buffer_allocator.h"
>
> @@ -77,7 +78,7 @@ public:
>
> ~Private() override;
>
> - std::unique_ptr<libcamera::FrameBuffer>
> + std::unique_ptr<AndroidFrameBuffer>
> allocate(int halPixelFormat, const libcamera::Size &size, uint32_t usage);
>
> private:
> @@ -92,7 +93,7 @@ PlatformFrameBufferAllocator::Private::~Private()
> gralloc_close(allocDevice_);
> }
>
> -std::unique_ptr<libcamera::FrameBuffer>
> +std::unique_ptr<AndroidFrameBuffer>
> PlatformFrameBufferAllocator::Private::allocate(int halPixelFormat,
> const libcamera::Size &size,
> uint32_t usage)
> @@ -135,9 +136,9 @@ PlatformFrameBufferAllocator::Private::allocate(int halPixelFormat,
> offset += planeSize;
> }
>
> - return std::make_unique<FrameBuffer>(
> - std::make_unique<GenericFrameBufferData>(allocDevice_, handle),
> - planes);
> + return std::make_unique<AndroidFrameBuffer>(handle,
> + std::make_unique<GenericFrameBufferData>(allocDevice_, handle),
> + planes);
> }
>
> PUBLIC_FRAME_BUFFER_ALLOCATOR_IMPLEMENTATION
--
Regards,
Laurent Pinchart
More information about the libcamera-devel
mailing list