[PATCH/RFC 18/32] libcamera: camera_sensor: Expose the Bayer order
Laurent Pinchart
laurent.pinchart at ideasonboard.com
Mon Mar 4 23:56:22 CET 2024
On Mon, Mar 04, 2024 at 06:50:02PM +0100, Jacopo Mondi wrote:
> And this can be fast-tracked as well imho
How about "[PATCH/RFC 14/32] libcamera: camera_sensor: Move related
classes to subdirectory" ? Patches 15/32 to 18/32 depend on it. I could
rebase, but...
You have also reviewed the refactoring of V4L2Subdevice (01/32 to
09/32), and acked most of it. I'd like to merge that too if possible.
03/32, 04/32, 07/32 and 09/32 are missing your Rb.
Does anyone else want to review all those patches too ?
> Reviewed-by: Jacopo Mondi <jacopo.mondi at ideasonboard.com>
>
> On Fri, Mar 01, 2024 at 11:21:07PM +0200, Laurent Pinchart wrote:
> > Pipeline handlers may need to know the Bayer order produced by the
> > sensor when a Transform is applied (horizontal or vertical flip). This
> > is currently implemented manually in the Raspberry Pi pipeline handler.
> > Move the implementation to the CameraSensor class to make it usable in
> > other pipeline handlers.
> >
> > Signed-off-by: Laurent Pinchart <laurent.pinchart at ideasonboard.com>
> > ---
> > include/libcamera/internal/camera_sensor.h | 4 +-
> > .../pipeline/rpi/common/pipeline_base.cpp | 54 +++----------------
> > .../pipeline/rpi/common/pipeline_base.h | 6 +--
> > src/libcamera/sensor/camera_sensor.cpp | 37 ++++++++++++-
> > 4 files changed, 45 insertions(+), 56 deletions(-)
> >
> > diff --git a/include/libcamera/internal/camera_sensor.h b/include/libcamera/internal/camera_sensor.h
> > index 750d6d729cac..d05f48ebeebe 100644
> > --- a/include/libcamera/internal/camera_sensor.h
> > +++ b/include/libcamera/internal/camera_sensor.h
> > @@ -22,12 +22,12 @@
> >
> > #include <libcamera/ipa/core_ipa_interface.h>
> >
> > +#include "libcamera/internal/bayer_format.h"
> > #include "libcamera/internal/formats.h"
> > #include "libcamera/internal/v4l2_subdevice.h"
> >
> > namespace libcamera {
> >
> > -class BayerFormat;
> > class CameraLens;
> > class MediaEntity;
> > class SensorConfiguration;
> > @@ -69,6 +69,7 @@ public:
> > const ControlList &properties() const { return properties_; }
> > int sensorInfo(IPACameraSensorInfo *info) const;
> > Transform computeTransform(Orientation *orientation) const;
> > + BayerFormat::Order bayerOrder(Transform t) const;
> >
> > const ControlInfoMap &controls() const;
> > ControlList getControls(const std::vector<uint32_t> &ids);
> > @@ -114,6 +115,7 @@ private:
> > Rectangle activeArea_;
> > const BayerFormat *bayerFormat_;
> > bool supportFlips_;
> > + bool flipsAlterBayerOrder_;
> > Orientation mountingOrientation_;
> >
> > ControlList properties_;
> > diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp
> > index 9449c3dc458c..7e420b3f90a4 100644
> > --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp
> > +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp
> > @@ -235,24 +235,16 @@ CameraConfiguration::Status RPiCameraConfiguration::validate()
> > for (auto &raw : rawStreams_) {
> > StreamConfiguration *rawStream = raw.cfg;
> >
> > - /* Adjust the RAW stream to match the computed sensor format. */
> > - BayerFormat sensorBayer = BayerFormat::fromMbusCode(sensorFormat_.code);
> > -
> > /*
> > - * Some sensors change their Bayer order when they are h-flipped
> > - * or v-flipped, according to the transform. If this one does, we
> > - * must advertise the transformed Bayer order in the raw stream.
> > - * Note how we must fetch the "native" (i.e. untransformed) Bayer
> > - * order, because the sensor may currently be flipped!
> > + * Some sensors change their Bayer order when they are
> > + * h-flipped or v-flipped, according to the transform. Adjust
> > + * the RAW stream to match the computed sensor format by
> > + * applying the sensor Bayer order resulting from the transform
> > + * to the user request.
> > */
> > - if (data_->flipsAlterBayerOrder_) {
> > - sensorBayer.order = data_->nativeBayerOrder_;
> > - sensorBayer = sensorBayer.transform(combinedTransform_);
> > - }
> >
> > - /* Apply the sensor adjusted Bayer order to the user request. */
> > BayerFormat cfgBayer = BayerFormat::fromPixelFormat(rawStream->pixelFormat);
> > - cfgBayer.order = sensorBayer.order;
> > + cfgBayer.order = data_->sensor_->bayerOrder(combinedTransform_);
> >
> > if (rawStream->pixelFormat != cfgBayer.toPixelFormat()) {
> > rawStream->pixelFormat = cfgBayer.toPixelFormat();
> > @@ -840,40 +832,6 @@ int PipelineHandlerBase::registerCamera(std::unique_ptr<RPi::CameraData> &camera
> > */
> > data->properties_.set(properties::ScalerCropMaximum, Rectangle{});
> >
> > - /*
> > - * We cache two things about the sensor in relation to transforms
> > - * (meaning horizontal and vertical flips): if they affect the Bayer
> > - * ordering, and what the "native" Bayer order is, when no transforms
> > - * are applied.
> > - *
> > - * If flips are supported verify if they affect the Bayer ordering
> > - * and what the "native" Bayer order is, when no transforms are
> > - * applied.
> > - *
> > - * We note that the sensor's cached list of supported formats is
> > - * already in the "native" order, with any flips having been undone.
> > - */
> > - const V4L2Subdevice *sensor = data->sensor_->device();
> > - const struct v4l2_query_ext_ctrl *hflipCtrl = sensor->controlInfo(V4L2_CID_HFLIP);
> > - if (hflipCtrl) {
> > - /* We assume it will support vflips too... */
> > - data->flipsAlterBayerOrder_ = hflipCtrl->flags & V4L2_CTRL_FLAG_MODIFY_LAYOUT;
> > - }
> > -
> > - /* Look for a valid Bayer format. */
> > - BayerFormat bayerFormat;
> > - for (const auto &iter : data->sensorFormats_) {
> > - bayerFormat = BayerFormat::fromMbusCode(iter.first);
> > - if (bayerFormat.isValid())
> > - break;
> > - }
> > -
> > - if (!bayerFormat.isValid()) {
> > - LOG(RPI, Error) << "No Bayer format found";
> > - return -EINVAL;
> > - }
> > - data->nativeBayerOrder_ = bayerFormat.order;
> > -
> > ret = platformRegister(cameraData, frontend, backend);
> > if (ret)
> > return ret;
> > diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h
> > index 267eef1102f1..0608bbe5f0c7 100644
> > --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h
> > +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h
> > @@ -48,7 +48,7 @@ class CameraData : public Camera::Private
> > public:
> > CameraData(PipelineHandler *pipe)
> > : Camera::Private(pipe), state_(State::Stopped),
> > - flipsAlterBayerOrder_(false), dropFrameCount_(0), buffersAllocated_(false),
> > + dropFrameCount_(0), buffersAllocated_(false),
> > ispOutputCount_(0), ispOutputTotal_(0)
> > {
> > }
> > @@ -131,10 +131,6 @@ public:
> >
> > std::queue<Request *> requestQueue_;
> >
> > - /* Store the "native" Bayer order (that is, with no transforms applied). */
> > - bool flipsAlterBayerOrder_;
> > - BayerFormat::Order nativeBayerOrder_;
> > -
> > /* For handling digital zoom. */
> > IPACameraSensorInfo sensorInfo_;
> > Rectangle ispCrop_; /* crop in ISP (camera mode) pixels */
> > diff --git a/src/libcamera/sensor/camera_sensor.cpp b/src/libcamera/sensor/camera_sensor.cpp
> > index 402025566544..5c4f35324055 100644
> > --- a/src/libcamera/sensor/camera_sensor.cpp
> > +++ b/src/libcamera/sensor/camera_sensor.cpp
> > @@ -58,7 +58,7 @@ LOG_DEFINE_CATEGORY(CameraSensor)
> > CameraSensor::CameraSensor(const MediaEntity *entity)
> > : entity_(entity), pad_(UINT_MAX), staticProps_(nullptr),
> > bayerFormat_(nullptr), supportFlips_(false),
> > - properties_(properties::properties)
> > + flipsAlterBayerOrder_(false), properties_(properties::properties)
> > {
> > }
> >
> > @@ -271,9 +271,14 @@ int CameraSensor::validateSensorDriver()
> > const struct v4l2_query_ext_ctrl *hflipInfo = subdev_->controlInfo(V4L2_CID_HFLIP);
> > const struct v4l2_query_ext_ctrl *vflipInfo = subdev_->controlInfo(V4L2_CID_VFLIP);
> > if (hflipInfo && !(hflipInfo->flags & V4L2_CTRL_FLAG_READ_ONLY) &&
> > - vflipInfo && !(vflipInfo->flags & V4L2_CTRL_FLAG_READ_ONLY))
> > + vflipInfo && !(vflipInfo->flags & V4L2_CTRL_FLAG_READ_ONLY)) {
> > supportFlips_ = true;
> >
> > + if (hflipInfo->flags & V4L2_CTRL_FLAG_MODIFY_LAYOUT ||
> > + vflipInfo->flags & V4L2_CTRL_FLAG_MODIFY_LAYOUT)
> > + flipsAlterBayerOrder_ = true;
> > + }
> > +
> > if (!supportFlips_)
> > LOG(CameraSensor, Debug)
> > << "Camera sensor does not support horizontal/vertical flip";
> > @@ -1041,6 +1046,34 @@ Transform CameraSensor::computeTransform(Orientation *orientation) const
> > return transform;
> > }
> >
> > +/**
> > + * \brief Compute the Bayer order that results from the given Transform
> > + * \param[in] t The Transform to apply to the sensor
> > + *
> > + * Some sensors change their Bayer order when they are h-flipped or v-flipped.
> > + * This function computes and returns the Bayer order that would result from the
> > + * given transform applied to the sensor.
> > + *
> > + * This function is valid only when the sensor produces raw Bayer formats.
> > + *
> > + * \return The Bayer order produced by the sensor when the Transform is applied
> > + */
> > +BayerFormat::Order CameraSensor::bayerOrder(Transform t) const
> > +{
> > + /* Return a defined by meaningless value for non-Bayer sensors. */
> > + if (!bayerFormat_)
> > + return BayerFormat::Order::BGGR;
> > +
> > + if (!flipsAlterBayerOrder_)
> > + return bayerFormat_->order;
> > +
> > + /*
> > + * Apply the transform to the native (i.e. untransformed) Bayer order,
> > + * using the rest of the Bayer format supplied by the caller.
> > + */
> > + return bayerFormat_->transform(t).order;
> > +}
> > +
> > /**
> > * \brief Retrieve the supported V4L2 controls and their information
> > *
--
Regards,
Laurent Pinchart
More information about the libcamera-devel
mailing list