[libcamera-devel] [PATCH v5 3/3] android: camera_device: Reorder configurations before requesting
Hirokazu Honda
hiroh at chromium.org
Fri Dec 11 07:02:28 CET 2020
Hi, Kieran, Laurent and Umang
Thanks for reviewing!
On Fri, Dec 11, 2020 at 7:25 AM Laurent Pinchart
<laurent.pinchart at ideasonboard.com> wrote:
>
> Hi Hiro,
>
> Thank you for the patch.
>
> On Wed, Dec 09, 2020 at 05:54:10AM +0000, Hirokazu Honda wrote:
> > This reorders Camera3Configs before executing
> > CameraConfiguration::validate() to make it easier for the Camera
> > to satisfy the Android framework request.
> >
> > Signed-off-by: Hirokazu Honda <hiroh at chromium.org>
> > Reviewed-by: Jacopo Mondi <jacopo at jmondi.org>
> > ---
> > src/android/camera_device.cpp | 109 +++++++++++++++++++++++++++++++++-
> > 1 file changed, 107 insertions(+), 2 deletions(-)
> >
> > diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp
> > index b7bf3d88..c9e0ec98 100644
> > --- a/src/android/camera_device.cpp
> > +++ b/src/android/camera_device.cpp
> > @@ -9,6 +9,7 @@
> > #include "camera_ops.h"
> > #include "post_processor.h"
> >
> > +#include <optional>
> > #include <sys/mman.h>
> > #include <tuple>
> > #include <vector>
> > @@ -27,6 +28,8 @@
> >
> > using namespace libcamera;
> >
> > +LOG_DECLARE_CATEGORY(HAL)
> > +
> > namespace {
> >
> > /*
> > @@ -140,9 +143,109 @@ struct Camera3StreamConfig {
> > std::vector<CameraStream::Type> types;
> > StreamConfiguration config;
> > };
> > -} /* namespace */
> >
> > -LOG_DECLARE_CATEGORY(HAL)
> > +/*
> > + * Reorder the configurations so that libcamera::Camera can accept them as much
> > + * as possible. The sort rule is as follows.
> > + * 1.) The configuration for NV12 request whose resolution is the largest.
> > + * 2.) The configuration for JPEG request.
> > + * 3.) Others. Larger resolutions and different formats are put earlier.
> > + */
> > +std::vector<Camera3StreamConfig> sortCamera3StreamConfigs(
> > + std::vector<Camera3StreamConfig> unsortedConfigs,
> > + const camera3_stream_t *jpegStream)
>
> How about making this a private member function ?
>
I would keep here so that we don't expose Camera3StreamConfig in
camera_device.h.
Regards,
-Hiro
> > +{
> > + const size_t unsortedSize = unsortedConfigs.size();
> > + std::optional<Camera3StreamConfig> jpegConfig = std::nullopt;
>
> std::nullopt is the default value, so you can write
>
> std::optional<Camera3StreamConfig> jpegConfig;
>
> > +
> > + if (jpegStream) {
> > + for (size_t i = 0; i < unsortedSize; ++i) {
> > + const auto &streams = unsortedConfigs[i].streams;
> > + if (std::find(streams.begin(), streams.end(),
> > + jpegStream) != streams.end()) {
> > + jpegConfig = std::move(unsortedConfigs[i]);
> > + unsortedConfigs.erase(unsortedConfigs.begin() + i);
> > + break;
> > + }
> > + }
> > + if (!jpegConfig)
> > + LOG(HAL, Fatal) << "No Camera3StreamConfig is found for Jpeg";
> > + }
> > +
> > + std::map<uint32_t, std::vector<Camera3StreamConfig>> formatToConfigs;
>
> Should the key be a PixelFormat instead of a uint32_t ?
>
> > + for (const auto &streamConfig : unsortedConfigs) {
> > + const StreamConfiguration &config = streamConfig.config;
> > + formatToConfigs[config.pixelFormat].push_back(streamConfig);
> > +
> > + }
>
> I wonder if this could be simplified to
>
> const Camera3StreamConfig *jpegConfig = nullptr;
>
> std::map<PixelFormat, std::vector<Camera3StreamConfig>> formatToConfigs;
> for (const auto &streamConfig : unsortedConfigs) {
> if (jpegStream && !jpegConfig) {
> const auto &streams = streamConfig.streams;
> if (std::find(streams.begin(), streams.end(),
> jpegStream) != streams.end()) {
> jpegConfig = &streamConfig;
> continue;
> }
> }
>
> const StreamConfiguration &config = streamConfig.config;
> formatToConfigs[config.pixelFormat].push_back(streamConfig);
> }
>
> if (jpegStream && !jpegConfig)
> LOG(HAL, Fatal) << "No Camera3StreamConfig is found for JPEG";
>
> The advantage is that unsortedStream would now be const. formatToConfigs
> could then be a map of PixelFormat to vector of pointers to
> Camera3StreamConfig, avoiding a copy. The only copy would be done when
> creating the entries in sortedConfigs.
>
> > + for (auto &[format, streamConfigs] : formatToConfigs) {
> > + /* Sorted by resolution. Smaller is put first. */
> > + std::sort(streamConfigs.begin(), streamConfigs.end(),
> > + [](const auto &streamConfigA, const auto &streamConfigB) {
> > + const Size &sizeA = streamConfigA.config.size;
> > + const Size &sizeB = streamConfigB.config.size;
> > + return sizeA < sizeB;
> > + });
> > + }
> > +
> > + std::vector<Camera3StreamConfig> sortedConfigs;
> > + sortedConfigs.reserve(unsortedSize);
>
> A blank line would be nice here.
>
> > + /*
> > + * NV12 is the most prioritized format. Put the configuration with NV12
> > + * and the largest resolution first.
> > + */
> > + const auto nv12It = formatToConfigs.find(formats::NV12);
> > + if (nv12It != formatToConfigs.end()) {
> > + auto &nv12Configs = nv12It->second;
> > + const Size &nv12LargestSize = nv12Configs.back().config.size;
>
> Here too.
>
> > + /*
> > + * If JPEG will be created from NV12 and the size is larger than
> > + * the largest NV12 configurations, then put the NV12
> > + * configuration for JPEG first.
> > + */
> > + if (jpegConfig && jpegConfig->config.pixelFormat == formats::NV12) {
> > + const Size &nv12SizeForJpeg = jpegConfig->config.size;
> > +
> > + if (nv12LargestSize < nv12SizeForJpeg) {
> > + LOG(HAL, Debug) << "Insert " << jpegConfig->config.toString();
> > + sortedConfigs.push_back(std::move(*jpegConfig));
> > + jpegConfig = std::nullopt;
> > + }
> > + }
>
> And here too.
>
> > + LOG(HAL, Debug) << "Insert " << nv12Configs.back().config.toString();
> > + sortedConfigs.push_back(std::move(nv12Configs.back()));
> > + nv12Configs.pop_back();
> > + }
> > +
> > + /* If the configuration for JPEG is there, then put it. */
> > + if (jpegConfig) {
> > + LOG(HAL, Debug) << "Insert " << jpegConfig->config.toString();
> > + sortedConfigs.push_back(std::move(*jpegConfig));
> > + jpegConfig = std::nullopt;
> > + }
> > +
> > + /*
> > + * Put configurations with different formats and larger resolutions
> > + * earlier.
> > + */
> > + while (!formatToConfigs.empty()) {
> > + for (auto it = formatToConfigs.begin(); it != formatToConfigs.end();) {
> > + auto& configs = it->second;
>
> auto &configs = it->second;
>
> > + if (configs.empty()) {
> > + it = formatToConfigs.erase(it);
> > + continue;
> > + }
>
> This could be moved to the end, ...
>
> > + LOG(HAL, Debug) << "Insert " << configs.back().config.toString();
> > + sortedConfigs.push_back(std::move(configs.back()));
> > + configs.pop_back();
> > + it++;
>
> ... it would become
>
> configs.pop_back();
> if (configs.empty())
> it = formatToConfigs.erase(it);
> else
> it++;
>
> > + }
> > + }
> > + assert(sortedConfigs.size() == unsortedSize);
> > +
> > + return sortedConfigs;
> > +}
> > +} /* namespace */
> >
> > MappedCamera3Buffer::MappedCamera3Buffer(const buffer_handle_t camera3buffer,
> > int flags)
> > @@ -1333,6 +1436,8 @@ int CameraDevice::configureStreams(camera3_stream_configuration_t *stream_list)
> > streamConfigs[index].types.push_back(type);
> > }
> >
> > + streamConfigs = sortCamera3StreamConfigs(std::move(streamConfigs),
> > + jpegStream);
>
> This is a bit of a weird function signature. Could we sort the vector
> in-place ?
>
> sortCamera3StreamConfigs(streamConfigs, jpegStream);
>
> It doesn't mean that the implementation must be in-place,
> sortCamera3StreamConfigs() can make an internal copy.
>
> As I wanted to test my suggestions, I've ended up reworking the code to
> make sure it would compile. I've pushed the result to
> https://git.libcamera.org/libcamera/pinchartl/libcamera.git/log/?h=android/hiro
> in case it could be useful.
>
> > for (const auto &streamConfig : streamConfigs) {
> > config_->addConfiguration(streamConfig.config);
> > for (size_t i = 0; i < streamConfig.streams.size(); ++i) {
>
> --
> Regards,
>
> Laurent Pinchart
More information about the libcamera-devel
mailing list