[libcamera-devel] [PATCH 1/1] Add CrOS JEA implementation
Cheng-Hao Yang
chenghaoyang at chromium.org
Tue Apr 26 11:18:29 CEST 2022
Thanks Laurent!
No worries about the schedule. I'm not in a hurry :)
I've splitted it into four CLs as you suggested in my PATCH v3 (Sorry for
the spam as I forgot to add the Signed-off-by line :'( )
Please check :)
BR,
Harvey
On Tue, Apr 26, 2022 at 6:40 AM Laurent Pinchart <
laurent.pinchart at ideasonboard.com> wrote:
> Hi Harvey,
>
> Thank you for the patch, and sorry for the late reply. Catching up with
> e-mail after travel is painful. Next time I'll try to get the whole
> world to travel at the same time, maybe I'll get less e-mails :-)
>
> On Wed, Apr 06, 2022 at 05:41:30PM +0800, Harvey Yang via libcamera-devel
> wrote:
> > This CL uses CrOS JpegCompressor with potential HW accelerator to do
> > JPEG encoding.
> >
> > As CrOS JpegCompressor might need file descriptors to get the source
> > data and pass the jpeg result, this CL extends FrameBuffer in the
> > android source code as Android_FrameBuffer, which stores the
> > buffer_handle_t when constructing the frame buffer, and adds a
> > getter function to access it.
> >
> > This CL also redefines src/android/jpeg/encoder interfaces and adds
> > Encoder::generateThumbnail, which might also be accelerated by CrOS
> > HW. It simplifies PostProcessorJpeg's logic when generating the
> > thumbnail. The original implementation is then moved into the
> > EncoderLibJpeg::generateThumbnail.
>
> This is missing a Signed-off-by line, see
> Documentation/contributing.rst.
>
> > ---
> > include/libcamera/framebuffer.h | 3 +-
> > src/android/android_framebuffer.cpp | 32 ++++++++
> > src/android/android_framebuffer.h | 28 +++++++
> > src/android/camera_device.cpp | 3 +-
> > src/android/cros/camera3_hal.cpp | 3 +
> > src/android/frame_buffer_allocator.h | 37 +++++----
> > src/android/jpeg/cros_post_processor_jpeg.cpp | 14 ++++
> > src/android/jpeg/encoder.h | 9 +-
> > src/android/jpeg/encoder_jea.cpp | 82 +++++++++++++++++++
> > src/android/jpeg/encoder_jea.h | 35 ++++++++
> > src/android/jpeg/encoder_libjpeg.cpp | 70 ++++++++++++++++
> > src/android/jpeg/encoder_libjpeg.h | 21 ++++-
> > .../jpeg/generic_post_processor_jpeg.cpp | 14 ++++
> > src/android/jpeg/meson.build | 16 ++++
> > src/android/jpeg/post_processor_jpeg.cpp | 60 ++------------
> > src/android/jpeg/post_processor_jpeg.h | 11 +--
> > src/android/meson.build | 6 +-
> > .../mm/cros_frame_buffer_allocator.cpp | 13 +--
> > .../mm/generic_frame_buffer_allocator.cpp | 11 +--
>
> There are lots of changes here, making this hard to review. Could you
> please split this patch in pieces, with one logical change by patch, and
> bundle them as a series ? Candidates are
>
> - Drop the final keyword from FrameBuffer and make the destructor
> virtual
> - Add AndroidFrameBuffer and use it in the HAL (you could even split
> that in two if desired, but bundling a new class with its user(s) can
> make review easier, if the result isn't too big)
> - Rework the JPEG encoder API and implementation to prepare for the
> needs of JEA
> - Add the JEA implementation
>
> > 19 files changed, 367 insertions(+), 101 deletions(-)
> > create mode 100644 src/android/android_framebuffer.cpp
> > create mode 100644 src/android/android_framebuffer.h
> > create mode 100644 src/android/jpeg/cros_post_processor_jpeg.cpp
> > create mode 100644 src/android/jpeg/encoder_jea.cpp
> > create mode 100644 src/android/jpeg/encoder_jea.h
> > create mode 100644 src/android/jpeg/generic_post_processor_jpeg.cpp
> > create mode 100644 src/android/jpeg/meson.build
> >
> > diff --git a/include/libcamera/framebuffer.h
> b/include/libcamera/framebuffer.h
> > index de172d97..c902cc18 100644
> > --- a/include/libcamera/framebuffer.h
> > +++ b/include/libcamera/framebuffer.h
> > @@ -46,7 +46,7 @@ private:
> > std::vector<Plane> planes_;
> > };
> >
> > -class FrameBuffer final : public Extensible
> > +class FrameBuffer : public Extensible
> > {
> > LIBCAMERA_DECLARE_PRIVATE()
> >
> > @@ -61,6 +61,7 @@ public:
> > FrameBuffer(const std::vector<Plane> &planes, unsigned int cookie
> = 0);
> > FrameBuffer(std::unique_ptr<Private> d,
> > const std::vector<Plane> &planes, unsigned int cookie
> = 0);
> > + virtual ~FrameBuffer() {}
> >
> > const std::vector<Plane> &planes() const { return planes_; }
> > Request *request() const;
> > diff --git a/src/android/android_framebuffer.cpp
> b/src/android/android_framebuffer.cpp
> > new file mode 100644
> > index 00000000..1ff7018e
> > --- /dev/null
> > +++ b/src/android/android_framebuffer.cpp
> > @@ -0,0 +1,32 @@
> > +/* SPDX-License-Identifier: LGPL-2.1-or-later */
> > +/*
> > + * Copyright (C) 2022, Google Inc.
> > + *
> > + * android_framebuffer.cpp - Android Frame buffer handling
> > + */
> > +
> > +#include "android_framebuffer.h"
> > +
> > +#include <hardware/camera3.h>
> > +
> > +AndroidFrameBuffer::AndroidFrameBuffer(
> > + buffer_handle_t handle,
> > + std::unique_ptr<Private> d,
> > + const std::vector<Plane> &planes,
> > + unsigned int cookie)
> > + : FrameBuffer(std::move(d), planes, cookie), handle_(handle)
> > +{
> > +}
> > +
> > +AndroidFrameBuffer::AndroidFrameBuffer(
> > + buffer_handle_t handle,
> > + const std::vector<Plane> &planes,
> > + unsigned int cookie)
> > + : FrameBuffer(planes, cookie), handle_(handle)
> > +{
> > +}
> > +
> > +buffer_handle_t AndroidFrameBuffer::getHandle() const
> > +{
> > + return handle_;
> > +}
> > diff --git a/src/android/android_framebuffer.h
> b/src/android/android_framebuffer.h
> > new file mode 100644
> > index 00000000..49df9756
> > --- /dev/null
> > +++ b/src/android/android_framebuffer.h
> > @@ -0,0 +1,28 @@
> > +/* SPDX-License-Identifier: LGPL-2.1-or-later */
> > +/*
> > + * Copyright (C) 2022, Google Inc.
> > + *
> > + * android_framebuffer.h - Android Frame buffer handling
> > + */
> > +
> > +#pragma once
> > +
> > +#include "libcamera/internal/framebuffer.h"
> > +
> > +#include <hardware/camera3.h>
> > +
> > +class AndroidFrameBuffer final : public libcamera::FrameBuffer
> > +{
> > +public:
> > + AndroidFrameBuffer(
> > + buffer_handle_t handle, std::unique_ptr<Private> d,
> > + const std::vector<Plane> &planes,
> > + unsigned int cookie = 0);
> > + AndroidFrameBuffer(buffer_handle_t handle,
> > + const std::vector<Plane> &planes,
> > + unsigned int cookie = 0);
> > + buffer_handle_t getHandle() const;
> > +
> > +private:
> > + buffer_handle_t handle_ = nullptr;
> > +};
> > diff --git a/src/android/camera_device.cpp
> b/src/android/camera_device.cpp
> > index 00d48471..643b4dee 100644
> > --- a/src/android/camera_device.cpp
> > +++ b/src/android/camera_device.cpp
> > @@ -25,6 +25,7 @@
> >
> > #include "system/graphics.h"
> >
> > +#include "android_framebuffer.h"
> > #include "camera_buffer.h"
> > #include "camera_hal_config.h"
> > #include "camera_ops.h"
> > @@ -754,7 +755,7 @@ CameraDevice::createFrameBuffer(const
> buffer_handle_t camera3buffer,
> > planes[i].length = buf.size(i);
> > }
> >
> > - return std::make_unique<FrameBuffer>(planes);
> > + return std::make_unique<AndroidFrameBuffer>(camera3buffer, planes);
> > }
> >
> > int CameraDevice::processControls(Camera3RequestDescriptor *descriptor)
> > diff --git a/src/android/cros/camera3_hal.cpp
> b/src/android/cros/camera3_hal.cpp
> > index fb863b5f..ea5577f0 100644
> > --- a/src/android/cros/camera3_hal.cpp
> > +++ b/src/android/cros/camera3_hal.cpp
> > @@ -9,8 +9,11 @@
> >
> > #include "../camera_hal_manager.h"
> >
> > +cros::CameraMojoChannelManagerToken *g_cros_camera_token = nullptr;
> > +
> > static void set_up([[maybe_unused]] cros::CameraMojoChannelManagerToken
> *token)
> > {
> > + g_cros_camera_token = token;
> > }
> >
> > static void tear_down()
> > diff --git a/src/android/frame_buffer_allocator.h
> b/src/android/frame_buffer_allocator.h
> > index 5d2eeda1..e26422a3 100644
> > --- a/src/android/frame_buffer_allocator.h
> > +++ b/src/android/frame_buffer_allocator.h
> > @@ -13,9 +13,10 @@
> > #include <libcamera/base/class.h>
> >
> > #include <libcamera/camera.h>
> > -#include <libcamera/framebuffer.h>
> > #include <libcamera/geometry.h>
> >
> > +#include "android_framebuffer.h"
> > +
> > class CameraDevice;
> >
> > class PlatformFrameBufferAllocator : libcamera::Extensible
> > @@ -31,25 +32,25 @@ public:
> > * Note: The returned FrameBuffer needs to be destroyed before
> > * PlatformFrameBufferAllocator is destroyed.
> > */
> > - std::unique_ptr<libcamera::FrameBuffer> allocate(
> > + std::unique_ptr<AndroidFrameBuffer> allocate(
> > int halPixelFormat, const libcamera::Size &size, uint32_t
> usage);
> > };
> >
> > -#define PUBLIC_FRAME_BUFFER_ALLOCATOR_IMPLEMENTATION \
> > -PlatformFrameBufferAllocator::PlatformFrameBufferAllocator( \
> > - CameraDevice *const cameraDevice) \
> > - : Extensible(std::make_unique<Private>(cameraDevice)) \
> > -{ \
> > -} \
> > -PlatformFrameBufferAllocator::~PlatformFrameBufferAllocator()
> \
> > -{ \
> > -} \
> > -std::unique_ptr<libcamera::FrameBuffer>
> \
> > -PlatformFrameBufferAllocator::allocate(int halPixelFormat, \
> > - const libcamera::Size &size, \
> > - uint32_t usage) \
> > -{ \
> > - return _d()->allocate(halPixelFormat, size, usage); \
> > -}
> > +#define PUBLIC_FRAME_BUFFER_ALLOCATOR_IMPLEMENTATION
> \
> > + PlatformFrameBufferAllocator::PlatformFrameBufferAllocator(
> \
> > + CameraDevice *const cameraDevice)
> \
> > + : Extensible(std::make_unique<Private>(cameraDevice))
> \
> > + {
> \
> > + }
> \
> > + PlatformFrameBufferAllocator::~PlatformFrameBufferAllocator()
> \
> > + {
> \
> > + }
> \
> > + std::unique_ptr<AndroidFrameBuffer>
> \
> > + PlatformFrameBufferAllocator::allocate(int halPixelFormat,
> \
> > + const libcamera::Size
> &size, \
> > + uint32_t usage)
> \
> > + {
> \
> > + return _d()->allocate(halPixelFormat, size, usage);
> \
> > + }
> >
> > #endif /* __ANDROID_FRAME_BUFFER_ALLOCATOR_H__ */
> > diff --git a/src/android/jpeg/cros_post_processor_jpeg.cpp
> b/src/android/jpeg/cros_post_processor_jpeg.cpp
> > new file mode 100644
> > index 00000000..7020f0d0
> > --- /dev/null
> > +++ b/src/android/jpeg/cros_post_processor_jpeg.cpp
> > @@ -0,0 +1,14 @@
> > +/* SPDX-License-Identifier: LGPL-2.1-or-later */
> > +/*
> > + * Copyright (C) 2022, Google Inc.
> > + *
> > + * cros_post_processor_jpeg.cpp - CrOS JPEG Post Processor
> > + */
> > +
> > +#include "encoder_jea.h"
> > +#include "post_processor_jpeg.h"
> > +
> > +void PostProcessorJpeg::SetEncoder()
> > +{
> > + encoder_ = std::make_unique<EncoderJea>();
> > +}
> > diff --git a/src/android/jpeg/encoder.h b/src/android/jpeg/encoder.h
> > index b974d367..6d527d91 100644
> > --- a/src/android/jpeg/encoder.h
> > +++ b/src/android/jpeg/encoder.h
> > @@ -12,14 +12,19 @@
> > #include <libcamera/framebuffer.h>
> > #include <libcamera/stream.h>
> >
> > +#include "../camera_request.h"
> > +
> > class Encoder
> > {
> > public:
> > virtual ~Encoder() = default;
> >
> > virtual int configure(const libcamera::StreamConfiguration &cfg) =
> 0;
> > - virtual int encode(const libcamera::FrameBuffer &source,
> > - libcamera::Span<uint8_t> destination,
> > + virtual int encode(Camera3RequestDescriptor::StreamBuffer
> *streamBuffer,
> > libcamera::Span<const uint8_t> exifData,
> > unsigned int quality) = 0;
> > + virtual int generateThumbnail(const libcamera::FrameBuffer &source,
> > + const libcamera::Size &targetSize,
> > + unsigned int quality,
> > + std::vector<unsigned char>
> *thumbnail) = 0;
> > };
> > diff --git a/src/android/jpeg/encoder_jea.cpp
> b/src/android/jpeg/encoder_jea.cpp
> > new file mode 100644
> > index 00000000..838e8647
> > --- /dev/null
> > +++ b/src/android/jpeg/encoder_jea.cpp
> > @@ -0,0 +1,82 @@
> > +/* SPDX-License-Identifier: LGPL-2.1-or-later */
> > +/*
> > + * Copyright (C) 2022, Google Inc.
> > + *
> > + * encoder_jea.cpp - JPEG encoding using CrOS JEA
> > + */
> > +
> > +#include "encoder_jea.h"
> > +
> > +#include <libcamera/base/log.h>
> > +
> > +#include "libcamera/internal/mapped_framebuffer.h"
> > +
> > +#include <cros-camera/camera_mojo_channel_manager_token.h>
> > +
> > +#include "../android_framebuffer.h"
> > +
> > +extern cros::CameraMojoChannelManagerToken *g_cros_camera_token;
> > +
> > +EncoderJea::EncoderJea() = default;
> > +
> > +EncoderJea::~EncoderJea() = default;
> > +
> > +int EncoderJea::configure(const libcamera::StreamConfiguration &cfg)
> > +{
> > + size_ = cfg.size;
> > +
> > + if (jpeg_compressor_.get())
> > + return 0;
> > +
> > + if (g_cros_camera_token == nullptr)
> > + return -ENOTSUP;
> > +
> > + jpeg_compressor_ =
> cros::JpegCompressor::GetInstance(g_cros_camera_token);
> > +
> > + return 0;
> > +}
> > +
> > +int EncoderJea::encode(Camera3RequestDescriptor::StreamBuffer
> *streamBuffer,
> > + libcamera::Span<const uint8_t> exifData,
> > + unsigned int quality)
> > +{
> > + if (!jpeg_compressor_.get())
> > + return -1;
> > +
> > + uint32_t out_data_size = 0;
> > +
> > + if (!jpeg_compressor_->CompressImageFromHandle(
> > + dynamic_cast<const AndroidFrameBuffer *>(
> > + streamBuffer->srcBuffer)
> > + ->getHandle(),
> > + *streamBuffer->camera3Buffer, size_.width,
> size_.height, quality,
> > + exifData.data(), exifData.size(), &out_data_size)) {
> > + return -1;
> > + }
> > +
> > + return out_data_size;
> > +}
> > +
> > +int EncoderJea::generateThumbnail(const libcamera::FrameBuffer &source,
> > + const libcamera::Size &targetSize,
> > + unsigned int quality,
> > + std::vector<unsigned char> *thumbnail)
> > +{
> > + if (!jpeg_compressor_.get())
> > + return -1;
> > +
> > + libcamera::MappedFrameBuffer frame(&source,
> libcamera::MappedFrameBuffer::MapFlag::Read);
> > +
> > + if (frame.planes().empty())
> > + return 0;
> > +
> > + uint32_t out_data_size = 0;
> > +
> > + if (!jpeg_compressor_->GenerateThumbnail(frame.planes()[0].data(),
> > + size_.width,
> size_.height, targetSize.width, targetSize.height,
> > + quality,
> thumbnail->size(), thumbnail->data(), &out_data_size)) {
> > + return -1;
> > + }
> > +
> > + return out_data_size;
> > +}
> > diff --git a/src/android/jpeg/encoder_jea.h
> b/src/android/jpeg/encoder_jea.h
> > new file mode 100644
> > index 00000000..d5c9f1f7
> > --- /dev/null
> > +++ b/src/android/jpeg/encoder_jea.h
> > @@ -0,0 +1,35 @@
> > +/* SPDX-License-Identifier: LGPL-2.1-or-later */
> > +/*
> > + * Copyright (C) 2022, Google Inc.
> > + *
> > + * encoder_jea.h - JPEG encoding using CrOS JEA
> > + */
> > +
> > +#pragma once
> > +
> > +#include <libcamera/geometry.h>
> > +
> > +#include <cros-camera/jpeg_compressor.h>
> > +
> > +#include "encoder.h"
> > +
> > +class EncoderJea : public Encoder
> > +{
> > +public:
> > + EncoderJea();
> > + ~EncoderJea();
> > +
> > + int configure(const libcamera::StreamConfiguration &cfg) override;
> > + int encode(Camera3RequestDescriptor::StreamBuffer *streamBuffer,
> > + libcamera::Span<const uint8_t> exifData,
> > + unsigned int quality) override;
> > + int generateThumbnail(const libcamera::FrameBuffer &source,
> > + const libcamera::Size &targetSize,
> > + unsigned int quality,
> > + std::vector<unsigned char> *thumbnail)
> override;
> > +
> > +private:
> > + libcamera::Size size_;
> > +
> > + std::unique_ptr<cros::JpegCompressor> jpeg_compressor_;
> > +};
> > diff --git a/src/android/jpeg/encoder_libjpeg.cpp
> b/src/android/jpeg/encoder_libjpeg.cpp
> > index 21a3b33d..b5591e33 100644
> > --- a/src/android/jpeg/encoder_libjpeg.cpp
> > +++ b/src/android/jpeg/encoder_libjpeg.cpp
> > @@ -24,6 +24,8 @@
> > #include "libcamera/internal/formats.h"
> > #include "libcamera/internal/mapped_framebuffer.h"
> >
> > +#include "../camera_buffer.h"
> > +
> > using namespace libcamera;
> >
> > LOG_DECLARE_CATEGORY(JPEG)
> > @@ -82,8 +84,17 @@ EncoderLibJpeg::~EncoderLibJpeg()
> > }
> >
> > int EncoderLibJpeg::configure(const StreamConfiguration &cfg)
> > +{
> > + thumbnailer_.configure(cfg.size, cfg.pixelFormat);
> > + cfg_ = cfg;
> > +
> > + return internalConfigure(cfg);
> > +}
> > +
> > +int EncoderLibJpeg::internalConfigure(const StreamConfiguration &cfg)
> > {
> > const struct JPEGPixelFormatInfo info =
> findPixelInfo(cfg.pixelFormat);
> > +
> > if (info.colorSpace == JCS_UNKNOWN)
> > return -ENOTSUP;
> >
> > @@ -178,6 +189,65 @@ void EncoderLibJpeg::compressNV(const
> std::vector<Span<uint8_t>> &planes)
> > }
> > }
> >
> > +int EncoderLibJpeg::encode(Camera3RequestDescriptor::StreamBuffer
> *streamBuffer,
> > + libcamera::Span<const uint8_t> exifData,
> > + unsigned int quality)
> > +{
> > + internalConfigure(cfg_);
> > + return encode(*streamBuffer->srcBuffer,
> streamBuffer->dstBuffer.get()->plane(0), exifData, quality);
> > +}
> > +
> > +int EncoderLibJpeg::generateThumbnail(
> > + const libcamera::FrameBuffer &source,
> > + const libcamera::Size &targetSize,
> > + unsigned int quality,
> > + std::vector<unsigned char> *thumbnail)
> > +{
> > + /* Stores the raw scaled-down thumbnail bytes. */
> > + std::vector<unsigned char> rawThumbnail;
> > +
> > + thumbnailer_.createThumbnail(source, targetSize, &rawThumbnail);
> > +
> > + StreamConfiguration thCfg;
> > + thCfg.size = targetSize;
> > + thCfg.pixelFormat = thumbnailer_.pixelFormat();
> > + int ret = internalConfigure(thCfg);
> > +
> > + if (!rawThumbnail.empty() && !ret) {
> > + /*
> > + * \todo Avoid value-initialization of all elements of the
> > + * vector.
> > + */
> > + thumbnail->resize(rawThumbnail.size());
> > +
> > + /*
> > + * Split planes manually as the encoder expects a vector of
> > + * planes.
> > + *
> > + * \todo Pass a vector of planes directly to
> > + * Thumbnailer::createThumbnailer above and remove the
> manual
> > + * planes split from here.
> > + */
> > + std::vector<Span<uint8_t>> thumbnailPlanes;
> > + const PixelFormatInfo &formatNV12 =
> PixelFormatInfo::info(formats::NV12);
> > + size_t yPlaneSize = formatNV12.planeSize(targetSize, 0);
> > + size_t uvPlaneSize = formatNV12.planeSize(targetSize, 1);
> > + thumbnailPlanes.push_back({ rawThumbnail.data(),
> yPlaneSize });
> > + thumbnailPlanes.push_back({ rawThumbnail.data() +
> yPlaneSize, uvPlaneSize });
> > +
> > + int jpeg_size = encode(thumbnailPlanes, *thumbnail, {},
> quality);
> > + thumbnail->resize(jpeg_size);
> > +
> > + LOG(JPEG, Debug)
> > + << "Thumbnail compress returned "
> > + << jpeg_size << " bytes";
> > +
> > + return jpeg_size;
> > + }
> > +
> > + return -1;
> > +}
> > +
> > int EncoderLibJpeg::encode(const FrameBuffer &source, Span<uint8_t>
> dest,
> > Span<const uint8_t> exifData, unsigned int
> quality)
> > {
> > diff --git a/src/android/jpeg/encoder_libjpeg.h
> b/src/android/jpeg/encoder_libjpeg.h
> > index 1b3ac067..56b27bae 100644
> > --- a/src/android/jpeg/encoder_libjpeg.h
> > +++ b/src/android/jpeg/encoder_libjpeg.h
> > @@ -15,6 +15,8 @@
> >
> > #include <jpeglib.h>
> >
> > +#include "thumbnailer.h"
> > +
> > class EncoderLibJpeg : public Encoder
> > {
> > public:
> > @@ -22,19 +24,32 @@ public:
> > ~EncoderLibJpeg();
> >
> > int configure(const libcamera::StreamConfiguration &cfg) override;
> > + int encode(Camera3RequestDescriptor::StreamBuffer *streamBuffer,
> > + libcamera::Span<const uint8_t> exifData,
> > + unsigned int quality) override;
> > + int generateThumbnail(
> > + const libcamera::FrameBuffer &source,
> > + const libcamera::Size &targetSize,
> > + unsigned int quality,
> > + std::vector<unsigned char> *thumbnail) override;
> > +
> > +private:
> > + int internalConfigure(const libcamera::StreamConfiguration &cfg);
> > +
> > int encode(const libcamera::FrameBuffer &source,
> > libcamera::Span<uint8_t> destination,
> > libcamera::Span<const uint8_t> exifData,
> > - unsigned int quality) override;
> > + unsigned int quality);
> > int encode(const std::vector<libcamera::Span<uint8_t>> &planes,
> > libcamera::Span<uint8_t> destination,
> > libcamera::Span<const uint8_t> exifData,
> > unsigned int quality);
> >
> > -private:
> > void compressRGB(const std::vector<libcamera::Span<uint8_t>>
> &planes);
> > void compressNV(const std::vector<libcamera::Span<uint8_t>>
> &planes);
> >
> > + libcamera::StreamConfiguration cfg_;
> > +
> > struct jpeg_compress_struct compress_;
> > struct jpeg_error_mgr jerr_;
> >
> > @@ -42,4 +57,6 @@ private:
> >
> > bool nv_;
> > bool nvSwap_;
> > +
> > + Thumbnailer thumbnailer_;
> > };
> > diff --git a/src/android/jpeg/generic_post_processor_jpeg.cpp
> b/src/android/jpeg/generic_post_processor_jpeg.cpp
> > new file mode 100644
> > index 00000000..890f6972
> > --- /dev/null
> > +++ b/src/android/jpeg/generic_post_processor_jpeg.cpp
> > @@ -0,0 +1,14 @@
> > +/* SPDX-License-Identifier: LGPL-2.1-or-later */
> > +/*
> > + * Copyright (C) 2022, Google Inc.
> > + *
> > + * generic_post_processor_jpeg.cpp - Generic JPEG Post Processor
> > + */
> > +
> > +#include "encoder_libjpeg.h"
> > +#include "post_processor_jpeg.h"
> > +
> > +void PostProcessorJpeg::SetEncoder()
> > +{
> > + encoder_ = std::make_unique<EncoderLibJpeg>();
> > +}
> > diff --git a/src/android/jpeg/meson.build b/src/android/jpeg/meson.build
> > new file mode 100644
> > index 00000000..8606acc4
> > --- /dev/null
> > +++ b/src/android/jpeg/meson.build
> > @@ -0,0 +1,16 @@
> > +# SPDX-License-Identifier: CC0-1.0
> > +
> > +android_hal_sources += files([
> > + 'exif.cpp',
> > + 'post_processor_jpeg.cpp'])
> > +
> > +platform = get_option('android_platform')
> > +if platform == 'generic'
> > + android_hal_sources += files(['encoder_libjpeg.cpp',
> > + 'generic_post_processor_jpeg.cpp',
> > + 'thumbnailer.cpp'])
> > +elif platform == 'cros'
> > + android_hal_sources += files(['cros_post_processor_jpeg.cpp',
> > + 'encoder_jea.cpp'])
> > + android_deps += [dependency('libcros_camera')]
> > +endif
> > diff --git a/src/android/jpeg/post_processor_jpeg.cpp
> b/src/android/jpeg/post_processor_jpeg.cpp
> > index d72ebc3c..7ceba60e 100644
> > --- a/src/android/jpeg/post_processor_jpeg.cpp
> > +++ b/src/android/jpeg/post_processor_jpeg.cpp
> > @@ -9,10 +9,10 @@
> >
> > #include <chrono>
> >
> > +#include "../android_framebuffer.h"
> > #include "../camera_device.h"
> > #include "../camera_metadata.h"
> > #include "../camera_request.h"
> > -#include "encoder_libjpeg.h"
> > #include "exif.h"
> >
> > #include <libcamera/base/log.h>
> > @@ -44,60 +44,11 @@ int PostProcessorJpeg::configure(const
> StreamConfiguration &inCfg,
> >
> > streamSize_ = outCfg.size;
> >
> > - thumbnailer_.configure(inCfg.size, inCfg.pixelFormat);
> > -
> > - encoder_ = std::make_unique<EncoderLibJpeg>();
> > + SetEncoder();
> >
> > return encoder_->configure(inCfg);
> > }
> >
> > -void PostProcessorJpeg::generateThumbnail(const FrameBuffer &source,
> > - const Size &targetSize,
> > - unsigned int quality,
> > - std::vector<unsigned char>
> *thumbnail)
> > -{
> > - /* Stores the raw scaled-down thumbnail bytes. */
> > - std::vector<unsigned char> rawThumbnail;
> > -
> > - thumbnailer_.createThumbnail(source, targetSize, &rawThumbnail);
> > -
> > - StreamConfiguration thCfg;
> > - thCfg.size = targetSize;
> > - thCfg.pixelFormat = thumbnailer_.pixelFormat();
> > - int ret = thumbnailEncoder_.configure(thCfg);
> > -
> > - if (!rawThumbnail.empty() && !ret) {
> > - /*
> > - * \todo Avoid value-initialization of all elements of the
> > - * vector.
> > - */
> > - thumbnail->resize(rawThumbnail.size());
> > -
> > - /*
> > - * Split planes manually as the encoder expects a vector of
> > - * planes.
> > - *
> > - * \todo Pass a vector of planes directly to
> > - * Thumbnailer::createThumbnailer above and remove the
> manual
> > - * planes split from here.
> > - */
> > - std::vector<Span<uint8_t>> thumbnailPlanes;
> > - const PixelFormatInfo &formatNV12 =
> PixelFormatInfo::info(formats::NV12);
> > - size_t yPlaneSize = formatNV12.planeSize(targetSize, 0);
> > - size_t uvPlaneSize = formatNV12.planeSize(targetSize, 1);
> > - thumbnailPlanes.push_back({ rawThumbnail.data(),
> yPlaneSize });
> > - thumbnailPlanes.push_back({ rawThumbnail.data() +
> yPlaneSize, uvPlaneSize });
> > -
> > - int jpeg_size = thumbnailEncoder_.encode(thumbnailPlanes,
> > - *thumbnail, {},
> quality);
> > - thumbnail->resize(jpeg_size);
> > -
> > - LOG(JPEG, Debug)
> > - << "Thumbnail compress returned "
> > - << jpeg_size << " bytes";
> > - }
> > -}
> > -
> > void PostProcessorJpeg::process(Camera3RequestDescriptor::StreamBuffer
> *streamBuffer)
> > {
> > ASSERT(encoder_);
> > @@ -164,8 +115,8 @@ void
> PostProcessorJpeg::process(Camera3RequestDescriptor::StreamBuffer *streamBu
> >
> > if (thumbnailSize != Size(0, 0)) {
> > std::vector<unsigned char> thumbnail;
> > - generateThumbnail(source, thumbnailSize, quality,
> &thumbnail);
> > - if (!thumbnail.empty())
> > + ret = encoder_->generateThumbnail(source,
> thumbnailSize, quality, &thumbnail);
> > + if (ret > 0 && !thumbnail.empty())
> > exif.setThumbnail(thumbnail,
> Exif::Compression::JPEG);
> > }
> >
> > @@ -194,8 +145,7 @@ void
> PostProcessorJpeg::process(Camera3RequestDescriptor::StreamBuffer *streamBu
> > const uint8_t quality = ret ? *entry.data.u8 : 95;
> > resultMetadata->addEntry(ANDROID_JPEG_QUALITY, quality);
> >
> > - int jpeg_size = encoder_->encode(source, destination->plane(0),
> > - exif.data(), quality);
> > + int jpeg_size = encoder_->encode(streamBuffer, exif.data(),
> quality);
> > if (jpeg_size < 0) {
> > LOG(JPEG, Error) << "Failed to encode stream image";
> > processComplete.emit(streamBuffer,
> PostProcessor::Status::Error);
> > diff --git a/src/android/jpeg/post_processor_jpeg.h
> b/src/android/jpeg/post_processor_jpeg.h
> > index 98309b01..a09f8798 100644
> > --- a/src/android/jpeg/post_processor_jpeg.h
> > +++ b/src/android/jpeg/post_processor_jpeg.h
> > @@ -8,11 +8,11 @@
> > #pragma once
> >
> > #include "../post_processor.h"
> > -#include "encoder_libjpeg.h"
> > -#include "thumbnailer.h"
> >
> > #include <libcamera/geometry.h>
> >
> > +#include "encoder.h"
> > +
> > class CameraDevice;
> >
> > class PostProcessorJpeg : public PostProcessor
> > @@ -25,14 +25,9 @@ public:
> > void process(Camera3RequestDescriptor::StreamBuffer *streamBuffer)
> override;
> >
> > private:
> > - void generateThumbnail(const libcamera::FrameBuffer &source,
> > - const libcamera::Size &targetSize,
> > - unsigned int quality,
> > - std::vector<unsigned char> *thumbnail);
> > + void SetEncoder();
> >
> > CameraDevice *const cameraDevice_;
> > std::unique_ptr<Encoder> encoder_;
> > libcamera::Size streamSize_;
> > - EncoderLibJpeg thumbnailEncoder_;
> > - Thumbnailer thumbnailer_;
> > };
> > diff --git a/src/android/meson.build b/src/android/meson.build
> > index 75b4bf20..026b8b3c 100644
> > --- a/src/android/meson.build
> > +++ b/src/android/meson.build
> > @@ -38,6 +38,7 @@ endif
> > android_deps += [libyuv_dep]
> >
> > android_hal_sources = files([
> > + 'android_framebuffer.cpp',
> > 'camera3_hal.cpp',
> > 'camera_capabilities.cpp',
> > 'camera_device.cpp',
> > @@ -47,10 +48,6 @@ android_hal_sources = files([
> > 'camera_ops.cpp',
> > 'camera_request.cpp',
> > 'camera_stream.cpp',
> > - 'jpeg/encoder_libjpeg.cpp',
> > - 'jpeg/exif.cpp',
> > - 'jpeg/post_processor_jpeg.cpp',
> > - 'jpeg/thumbnailer.cpp',
> > 'yuv/post_processor_yuv.cpp'
> > ])
> >
> > @@ -58,6 +55,7 @@ android_cpp_args = []
> >
> > subdir('cros')
> > subdir('mm')
> > +subdir('jpeg')
> >
> > android_camera_metadata_sources = files([
> > 'metadata/camera_metadata.c',
> > diff --git a/src/android/mm/cros_frame_buffer_allocator.cpp
> b/src/android/mm/cros_frame_buffer_allocator.cpp
> > index 52e8c180..163c5d75 100644
> > --- a/src/android/mm/cros_frame_buffer_allocator.cpp
> > +++ b/src/android/mm/cros_frame_buffer_allocator.cpp
> > @@ -14,6 +14,7 @@
> >
> > #include "libcamera/internal/framebuffer.h"
> >
> > +#include "../android_framebuffer.h"
> > #include "../camera_device.h"
> > #include "../frame_buffer_allocator.h"
> > #include "cros-camera/camera_buffer_manager.h"
> > @@ -47,11 +48,11 @@ public:
> > {
> > }
> >
> > - std::unique_ptr<libcamera::FrameBuffer>
> > + std::unique_ptr<AndroidFrameBuffer>
> > allocate(int halPixelFormat, const libcamera::Size &size, uint32_t
> usage);
> > };
> >
> > -std::unique_ptr<libcamera::FrameBuffer>
> > +std::unique_ptr<AndroidFrameBuffer>
> > PlatformFrameBufferAllocator::Private::allocate(int halPixelFormat,
> > const libcamera::Size
> &size,
> > uint32_t usage)
> > @@ -80,9 +81,11 @@ PlatformFrameBufferAllocator::Private::allocate(int
> halPixelFormat,
> > plane.length =
> cros::CameraBufferManager::GetPlaneSize(handle, i);
> > }
> >
> > - return std::make_unique<FrameBuffer>(
> > -
> std::make_unique<CrosFrameBufferData>(std::move(scopedHandle)),
> > - planes);
> > + auto fb = std::make_unique<AndroidFrameBuffer>(handle,
> > +
> std::make_unique<CrosFrameBufferData>(std::move(scopedHandle)),
> > + planes);
> > +
> > + return fb;
> > }
> >
> > PUBLIC_FRAME_BUFFER_ALLOCATOR_IMPLEMENTATION
> > diff --git a/src/android/mm/generic_frame_buffer_allocator.cpp
> b/src/android/mm/generic_frame_buffer_allocator.cpp
> > index acb2fa2b..c79b7b10 100644
> > --- a/src/android/mm/generic_frame_buffer_allocator.cpp
> > +++ b/src/android/mm/generic_frame_buffer_allocator.cpp
> > @@ -18,6 +18,7 @@
> > #include <hardware/gralloc.h>
> > #include <hardware/hardware.h>
> >
> > +#include "../android_framebuffer.h"
> > #include "../camera_device.h"
> > #include "../frame_buffer_allocator.h"
> >
> > @@ -77,7 +78,7 @@ public:
> >
> > ~Private() override;
> >
> > - std::unique_ptr<libcamera::FrameBuffer>
> > + std::unique_ptr<AndroidFrameBuffer>
> > allocate(int halPixelFormat, const libcamera::Size &size, uint32_t
> usage);
> >
> > private:
> > @@ -92,7 +93,7 @@ PlatformFrameBufferAllocator::Private::~Private()
> > gralloc_close(allocDevice_);
> > }
> >
> > -std::unique_ptr<libcamera::FrameBuffer>
> > +std::unique_ptr<AndroidFrameBuffer>
> > PlatformFrameBufferAllocator::Private::allocate(int halPixelFormat,
> > const libcamera::Size
> &size,
> > uint32_t usage)
> > @@ -135,9 +136,9 @@ PlatformFrameBufferAllocator::Private::allocate(int
> halPixelFormat,
> > offset += planeSize;
> > }
> >
> > - return std::make_unique<FrameBuffer>(
> > - std::make_unique<GenericFrameBufferData>(allocDevice_,
> handle),
> > - planes);
> > + return std::make_unique<AndroidFrameBuffer>(handle,
> > +
> std::make_unique<GenericFrameBufferData>(allocDevice_, handle),
> > + planes);
> > }
> >
> > PUBLIC_FRAME_BUFFER_ALLOCATOR_IMPLEMENTATION
>
> --
> Regards,
>
> Laurent Pinchart
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.libcamera.org/pipermail/libcamera-devel/attachments/20220426/3b315142/attachment-0001.htm>
More information about the libcamera-devel
mailing list