[libcamera-devel] [PATCH 2/2] ipa: raspberrypi: Add support for imx290/imx327 sensors
Dave Stevenson
dave.stevenson at raspberrypi.com
Tue Mar 9 13:34:48 CET 2021
Hi Laurent & David
On Mon, 8 Mar 2021 at 22:20, Laurent Pinchart
<laurent.pinchart at ideasonboard.com> wrote:
>
> Hi David,
>
> On Mon, Mar 08, 2021 at 10:11:35PM +0000, David Plowman wrote:
> > On Mon, 8 Mar 2021 at 19:31, Laurent Pinchart wrote:
> > > On Thu, Mar 04, 2021 at 03:31:20PM +0000, David Plowman wrote:
> > > > imx290 and imx327 share the same kernel driver (imx290.c) and are
> > > > therefore both recognised here as "imx290".
> > >
> > > Are there any differences between the two sensors that may need to be
> > > handled anywhere in libcamera ?
> >
> > Good question, and the answer is... I don't really know. I don't have
> > any modules that I know to have an imx290 in them, but at some point
> > I'm sure we or someone else will come across one and we'll find out.
> > At some point some we're going to have to start coping with different
> > modules with the same (or indistinguishable sensors) in them too.
>
> There are two issues here, differences between the IMX290 and IMX327
> sensors, and differences between modules. For the former, would it make
> sense to already add a sony,imx327 compatible string to the imx290
> driver, and make use of it ? To be clear, it's not a blocker for this
> patch.
I have a mono IMX290 from Vision Components here, as well as several
IMX327's. Vision Components appear to use literally the same PCB for
the two modules - they are both silkscreened IMX327.
IMX290 is the bigger brother to IMX327. It can do 1080p120 (10bit
only, 4lane CSI2 at 891Mb/s/lane), whilst IMX327 can only do 1080p60
(10 or 12bit, 2lane CSI2 at 891Mb/s/lane, or 4lane at 445.5Mb/s/lane).
Programming interface is identical except for allowing frsel=0 for
120fps on IMX290.
The driver is largely Andrey Konovalov's mainline driver with controls
added, and support for an alternate source clock frequency
(74.250MHz).
It switches mode based on link frequency, and doesn't support the
1080p120fps mode, so is actually independent of whether it is 327 or
290. Based on the link frequency numbers it's a max of 1080p30 on
either lane configuration, but I thought I'd had 1080p60.
One to investigate to make the driver more flexible, but shouldn't
affect libcamera (other than wanting mono support!). The Pi ISP won't
support 1080p120 so is actually of limited interest to us.
Dave
> > > > We add the necessary
> > > > CamHelper for these sensors, as well as a camera tuning file.
> > > >
> > > > The tuning was done with an Innomaker STARVIS IMX327LQR module. These
> > > > have no IR cut filter so there is no proper colour tuning. However,
> > > > you should obtain reasonable results for most modules using this
> > > > sensor. Specific tunings for further modules can always be added
> > > > subsequently.
> > > >
> > > > To use this sensor on the Raspberry Pi platform, please add
> > > >
> > > > dtoverlay=imx290,clock-frequency=74250000
> > > >
> > > > into your /boot/config.txt file.
> > > >
> > > > Signed-off-by: David Plowman <david.plowman at raspberrypi.com>
> > > > ---
> > > > src/ipa/raspberrypi/cam_helper_imx290.cpp | 67 +++++++++
> > > > src/ipa/raspberrypi/data/imx290.json | 165 ++++++++++++++++++++++
> > > > src/ipa/raspberrypi/data/meson.build | 1 +
> > > > src/ipa/raspberrypi/meson.build | 1 +
> > > > 4 files changed, 234 insertions(+)
> > > > create mode 100644 src/ipa/raspberrypi/cam_helper_imx290.cpp
> > > > create mode 100644 src/ipa/raspberrypi/data/imx290.json
> > > >
> > > > diff --git a/src/ipa/raspberrypi/cam_helper_imx290.cpp b/src/ipa/raspberrypi/cam_helper_imx290.cpp
> > > > new file mode 100644
> > > > index 00000000..6f412e40
> > > > --- /dev/null
> > > > +++ b/src/ipa/raspberrypi/cam_helper_imx290.cpp
> > > > @@ -0,0 +1,67 @@
> > > > +/* SPDX-License-Identifier: BSD-2-Clause */
> > > > +/*
> > > > + * Copyright (C) 2021, Raspberry Pi (Trading) Limited
> > > > + *
> > > > + * cam_helper_imx290.cpp - camera helper for imx290 sensor
> > > > + */
> > > > +
> > > > +#include <math.h>
> > > > +
> > > > +#include "cam_helper.hpp"
> > > > +
> > > > +using namespace RPiController;
> > > > +
> > > > +class CamHelperImx290 : public CamHelper
> > > > +{
> > > > +public:
> > > > + CamHelperImx290();
> > > > + uint32_t GainCode(double gain) const override;
> > > > + double Gain(uint32_t gain_code) const override;
> > > > + void GetDelays(int &exposure_delay, int &gain_delay,
> > > > + int &vblank_delay) const override;
> > > > + unsigned int HideFramesModeSwitch() const override;
> > > > +
> > > > +private:
> > > > + /*
> > > > + * Smallest difference between the frame length and integration time,
> > > > + * in units of lines.
> > > > + */
> > > > + static constexpr int frameIntegrationDiff = 2;
> > > > +};
> > > > +
> > > > +CamHelperImx290::CamHelperImx290()
> > > > + : CamHelper(nullptr, frameIntegrationDiff)
> > > > +{
> > > > +}
> > > > +
> > > > +uint32_t CamHelperImx290::GainCode(double gain) const
> > > > +{
> > > > + int code = 66.6667 * log10(gain);
> > > > + return std::max(0, std::min(code, 0xf0));
> > > > +}
> > > > +
> > > > +double CamHelperImx290::Gain(uint32_t gain_code) const
> > > > +{
> > > > + return pow(10, 0.015 * gain_code);
> > > > +}
> > > > +
> > > > +void CamHelperImx290::GetDelays(int &exposure_delay, int &gain_delay,
> > > > + int &vblank_delay) const
> > > > +{
> > > > + exposure_delay = 2;
> > > > + gain_delay = 2;
> > > > + vblank_delay = 2;
> > > > +}
> > > > +
> > > > +unsigned int CamHelperImx290::HideFramesModeSwitch() const
> > > > +{
> > > > + /* After a mode switch, we seem to get 1 bad frame. */
> > > > + return 1;
> > > > +}
> > > > +
> > > > +static CamHelper *Create()
> > > > +{
> > > > + return new CamHelperImx290();
> > > > +}
> > > > +
> > > > +static RegisterCamHelper reg("imx290", &Create);
> > > > diff --git a/src/ipa/raspberrypi/data/imx290.json b/src/ipa/raspberrypi/data/imx290.json
> > > > new file mode 100644
> > > > index 00000000..6fb92cc4
> > > > --- /dev/null
> > > > +++ b/src/ipa/raspberrypi/data/imx290.json
> > > > @@ -0,0 +1,165 @@
> > > > +{
> > > > + "rpi.black_level":
> > > > + {
> > > > + "black_level": 3840
> > > > + },
> > > > + "rpi.dpc":
> > > > + {
> > > > + },
> > > > + "rpi.lux":
> > > > + {
> > > > + "reference_shutter_speed": 6813,
> > > > + "reference_gain": 1.0,
> > > > + "reference_aperture": 1.0,
> > > > + "reference_lux": 890,
> > > > + "reference_Y": 12900
> > > > + },
> > > > + "rpi.noise":
> > > > + {
> > > > + "reference_constant": 0,
> > > > + "reference_slope": 2.67
> > > > + },
> > > > + "rpi.geq":
> > > > + {
> > > > + "offset": 187,
> > > > + "slope": 0.00842
> > > > + },
> > > > + "rpi.sdn":
> > > > + {
> > > > + },
> > > > + "rpi.awb":
> > > > + {
> > > > + "bayes": 0
> > > > + },
> > > > + "rpi.agc":
> > > > + {
> > > > + "speed": 0.2,
> > > > + "metering_modes":
> > > > + {
> > > > + "matrix":
> > > > + {
> > > > + "weights":
> > > > + [
> > > > + 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1
> > > > + ]
> > > > + },
> > > > + "centre-weighted":
> > > > + {
> > > > + "weights":
> > > > + [
> > > > + 3, 3, 3, 2, 2, 2, 2, 1, 1, 1, 1, 0, 0, 0, 0
> > > > + ]
> > > > + },
> > > > + "spot":
> > > > + {
> > > > + "weights":
> > > > + [
> > > > + 2, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
> > > > + ]
> > > > + }
> > > > + },
> > > > + "exposure_modes":
> > > > + {
> > > > + "normal":
> > > > + {
> > > > + "shutter":
> > > > + [
> > > > + 10, 30000, 60000
> > > > + ],
> > > > + "gain":
> > > > + [
> > > > + 1.0, 2.0, 8.0
> > > > + ]
> > > > + },
> > > > + "sport":
> > > > + {
> > > > + "shutter":
> > > > + [
> > > > + 10, 5000, 10000, 20000, 120000
> > > > + ],
> > > > + "gain":
> > > > + [
> > > > + 1.0, 2.0, 4.0, 6.0, 6.0
> > > > + ]
> > > > + }
> > > > + },
> > > > + "constraint_modes":
> > > > + {
> > > > + "normal":
> > > > + [
> > > > + ],
> > > > + "highlight":
> > > > + [
> > > > + {
> > > > + "bound": "LOWER", "q_lo": 0.98, "q_hi": 1.0, "y_target":
> > > > + [
> > > > + 0, 0.5, 1000, 0.5
> > > > + ]
> > > > + },
> > > > + {
> > > > + "bound": "UPPER", "q_lo": 0.98, "q_hi": 1.0, "y_target":
> > > > + [
> > > > + 0, 0.8, 1000, 0.8
> > > > + ]
> > > > + }
> > > > + ]
> > > > + },
> > > > + "y_target":
> > > > + [
> > > > + 0, 0.16, 1000, 0.16, 10000, 0.16
> > > > + ]
> > > > + },
> > > > + "rpi.alsc":
> > > > + {
> > > > + "omega": 1.3,
> > > > + "n_iter": 100,
> > > > + "luminance_strength": 0.7,
> > > > + "luminance_lut":
> > > > + [
> > > > + 2.844, 2.349, 2.018, 1.775, 1.599, 1.466, 1.371, 1.321, 1.306, 1.316, 1.357, 1.439, 1.552, 1.705, 1.915, 2.221,
> > > > + 2.576, 2.151, 1.851, 1.639, 1.478, 1.358, 1.272, 1.231, 1.218, 1.226, 1.262, 1.335, 1.438, 1.571, 1.766, 2.067,
> > > > + 2.381, 2.005, 1.739, 1.545, 1.389, 1.278, 1.204, 1.166, 1.153, 1.161, 1.194, 1.263, 1.356, 1.489, 1.671, 1.943,
> > > > + 2.242, 1.899, 1.658, 1.481, 1.329, 1.225, 1.156, 1.113, 1.096, 1.107, 1.143, 1.201, 1.289, 1.423, 1.607, 1.861,
> > > > + 2.152, 1.831, 1.602, 1.436, 1.291, 1.193, 1.121, 1.069, 1.047, 1.062, 1.107, 1.166, 1.249, 1.384, 1.562, 1.801,
> > > > + 2.104, 1.795, 1.572, 1.407, 1.269, 1.174, 1.099, 1.041, 1.008, 1.029, 1.083, 1.146, 1.232, 1.364, 1.547, 1.766,
> > > > + 2.104, 1.796, 1.572, 1.403, 1.264, 1.171, 1.097, 1.036, 1.001, 1.025, 1.077, 1.142, 1.231, 1.363, 1.549, 1.766,
> > > > + 2.148, 1.827, 1.594, 1.413, 1.276, 1.184, 1.114, 1.062, 1.033, 1.049, 1.092, 1.153, 1.242, 1.383, 1.577, 1.795,
> > > > + 2.211, 1.881, 1.636, 1.455, 1.309, 1.214, 1.149, 1.104, 1.081, 1.089, 1.125, 1.184, 1.273, 1.423, 1.622, 1.846,
> > > > + 2.319, 1.958, 1.698, 1.516, 1.362, 1.262, 1.203, 1.156, 1.137, 1.142, 1.171, 1.229, 1.331, 1.484, 1.682, 1.933,
> > > > + 2.459, 2.072, 1.789, 1.594, 1.441, 1.331, 1.261, 1.219, 1.199, 1.205, 1.232, 1.301, 1.414, 1.571, 1.773, 2.052,
> > > > + 2.645, 2.206, 1.928, 1.728, 1.559, 1.451, 1.352, 1.301, 1.282, 1.289, 1.319, 1.395, 1.519, 1.685, 1.904, 2.227
> > > > + ],
> > > > + "sigma": 0.005,
> > > > + "sigma_Cb": 0.005
> > > > + },
> > > > + "rpi.contrast":
> > > > + {
> > > > + "ce_enable": 1,
> > > > + "gamma_curve":
> > > > + [
> > > > + 0, 0, 1024, 5040, 2048, 9338, 3072, 12356, 4096, 15312, 5120, 18051, 6144, 20790, 7168, 23193,
> > > > + 8192, 25744, 9216, 27942, 10240, 30035, 11264, 32005, 12288, 33975, 13312, 35815, 14336, 37600, 15360, 39168,
> > > > + 16384, 40642, 18432, 43379, 20480, 45749, 22528, 47753, 24576, 49621, 26624, 51253, 28672, 52698, 30720, 53796,
> > > > + 32768, 54876, 36864, 57012, 40960, 58656, 45056, 59954, 49152, 61183, 53248, 62355, 57344, 63419, 61440, 64476,
> > > > + 65535, 65535
> > > > + ]
> > > > + },
> > > > + "rpi.sharpen":
> > > > + {
> > > > + },
> > > > + "rpi.ccm":
> > > > + {
> > > > + "ccms":
> > > > + [
> > > > + {
> > > > + "ct": 3900, "ccm":
> > > > + [
> > > > + 1.54659, -0.17707, -0.36953, -0.51471, 1.72733, -0.21262, 0.06667, -0.92279, 1.85612
> > > > + ]
> > > > + }
> > > > + ]
> > > > + },
> > > > + "rpi.focus":
> > > > + {
> > > > + }
> > > > +}
> > > > diff --git a/src/ipa/raspberrypi/data/meson.build b/src/ipa/raspberrypi/data/meson.build
> > > > index 5236bf1e..509ad58b 100644
> > > > --- a/src/ipa/raspberrypi/data/meson.build
> > > > +++ b/src/ipa/raspberrypi/data/meson.build
> > > > @@ -3,6 +3,7 @@
> > > > conf_files = files([
> > > > 'imx219.json',
> > > > 'imx477.json',
> > > > + 'imx290.json',
> > >
> > > Alphabetical order please.
> > >
> > > > 'ov5647.json',
> > > > 'uncalibrated.json',
> > > > ])
> > > > diff --git a/src/ipa/raspberrypi/meson.build b/src/ipa/raspberrypi/meson.build
> > > > index 59e49686..7e88f8e0 100644
> > > > --- a/src/ipa/raspberrypi/meson.build
> > > > +++ b/src/ipa/raspberrypi/meson.build
> > > > @@ -21,6 +21,7 @@ rpi_ipa_sources = files([
> > > > 'cam_helper_ov5647.cpp',
> > > > 'cam_helper_imx219.cpp',
> > > > 'cam_helper_imx477.cpp',
> > > > + 'cam_helper_imx290.cpp',
> > >
> > > Here too.
> >
> > Will do! Version 2 of this set incoming shortly.
> >
> > > Reviewed-by: Laurent Pinchart <laurent.pinchart at ideasonboard.com>
> > >
> > > > 'controller/controller.cpp',
> > > > 'controller/histogram.cpp',
> > > > 'controller/algorithm.cpp',
>
> --
> Regards,
>
> Laurent Pinchart
> _______________________________________________
> libcamera-devel mailing list
> libcamera-devel at lists.libcamera.org
> https://lists.libcamera.org/listinfo/libcamera-devel
More information about the libcamera-devel
mailing list