[libcamera-devel] [PATCH v3 5/8] ipa: raspberrypi: Allow long exposure modes for imx477.
Naushir Patuck
naush at raspberrypi.com
Mon Jul 5 14:43:37 CEST 2021
Hi David,
Thank you for your review feedback.
On Mon, 5 Jul 2021 at 12:47, David Plowman <david.plowman at raspberrypi.com>
wrote:
> Hi Naush
>
> Thanks for this patch!
>
> On Fri, 2 Jul 2021 at 16:09, Naushir Patuck <naush at raspberrypi.com> wrote:
> >
> > Update the imx477 CamHelper to use long exposure modes if needed.
> > This is done by overloading the CamHelper::GetVBlanking function to
> return a
> > frame length (and vblank value) computed using a scaling factor when the
> value
> > would be larger than what the sensor register could otherwise hold.
> >
> > CamHelperImx477::Prepare is also overloaded to ensure that the
> "device.status"
> > metadata returns the right value if the long exposure scaling factor is
> used.
> > The scaling factor is unfortunately not returned back in metadata.
> >
> > With the current imx477 driver, we can achieve a maximum exposure time
> of approx
> > 127 seconds since the HBLANK control is read-only.
> >
> > Signed-off-by: Naushir Patuck <naush at raspberrypi.com>
> > ---
> > src/ipa/raspberrypi/cam_helper_imx477.cpp | 95 +++++++++++++++++++++++
> > 1 file changed, 95 insertions(+)
> >
> > diff --git a/src/ipa/raspberrypi/cam_helper_imx477.cpp
> b/src/ipa/raspberrypi/cam_helper_imx477.cpp
> > index 91d05d9226ff..ddf0863c11b4 100644
> > --- a/src/ipa/raspberrypi/cam_helper_imx477.cpp
> > +++ b/src/ipa/raspberrypi/cam_helper_imx477.cpp
> > @@ -6,14 +6,23 @@
> > */
> >
> > #include <assert.h>
> > +#include <cmath>
> > #include <stddef.h>
> > #include <stdio.h>
> > #include <stdlib.h>
> >
> > +#include <libcamera/base/log.h>
> > +
> > #include "cam_helper.hpp"
> > #include "md_parser.hpp"
> >
> > using namespace RPiController;
> > +using namespace libcamera;
> > +using libcamera::utils::Duration;
> > +
> > +namespace libcamera {
> > +LOG_DECLARE_CATEGORY(IPARPI)
> > +}
> >
> > /*
> > * We care about two gain registers and a pair of exposure registers.
> Their
> > @@ -34,6 +43,9 @@ public:
> > CamHelperImx477();
> > uint32_t GainCode(double gain) const override;
> > double Gain(uint32_t gain_code) const override;
> > + void Prepare(libcamera::Span<const uint8_t> buffer, Metadata
> &metadata) override;
> > + uint32_t GetVBlanking(Duration &exposure, Duration
> minFrameDuration,
> > + Duration maxFrameDuration) const override;
> > void GetDelays(int &exposure_delay, int &gain_delay,
> > int &vblank_delay) const override;
> > bool SensorEmbeddedDataPresent() const override;
> > @@ -44,6 +56,10 @@ private:
> > * in units of lines.
> > */
> > static constexpr int frameIntegrationDiff = 22;
> > + /* Maximum frame length allowable for long exposure
> calculations. */
> > + static constexpr int frameLengthMax = 0xffdc;
> > + /* Largest long exposure scale factor given as a left shift on
> the frame length. */
> > + static constexpr int longExposureShiftMax = 7;
> >
> > void PopulateMetadata(const MdParser::RegisterMap ®isters,
> > Metadata &metadata) const override;
> > @@ -64,6 +80,85 @@ double CamHelperImx477::Gain(uint32_t gain_code) const
> > return 1024.0 / (1024 - gain_code);
> > }
> >
> > +void CamHelperImx477::Prepare(libcamera::Span<const uint8_t> buffer,
> Metadata &metadata)
> > +{
> > + MdParser::RegisterMap registers;
> > + Metadata parsedMetadata;
> > +
> > + if (buffer.empty())
> > + return;
> > +
> > + if (parser_->Parse(buffer, registers) != MdParser::Status::OK) {
> > + LOG(IPARPI, Error) << "Embedded data buffer parsing
> failed";
> > + return;
> > + }
> > +
> > + PopulateMetadata(registers, parsedMetadata);
> > + metadata.Merge(parsedMetadata);
> > +
> > + DeviceStatus deviceStatus, parsedDeviceStatus;
> > + if (metadata.Get("device.status", deviceStatus) ||
> > + parsedMetadata.Get("device.status", parsedDeviceStatus)) {
> > + LOG(IPARPI, Error) << "DeviceStatus not found";
> > + return;
> > + }
> > +
> > + /*
> > + * The DeviceStatus struct is first populated with values
> obtained from
> > + * DelayedControls. If this reports frame length is >
> frameLengthMax,
> > + * it means we are using a long exposure mode. Since the long
> exposure
> > + * scale factor is not returned back through embedded data, we
> must rely
> > + * on the existing exposure lines and frame length values
> returned by
> > + * DelayedControls.
> > + *
> > + * Otherwise, all values are updated with what is reported in the
> > + * embedded data.
> > + */
> > + if (deviceStatus.frame_length <= frameLengthMax) {
> > + deviceStatus.shutter_speed =
> parsedDeviceStatus.shutter_speed;
> > + deviceStatus.frame_length =
> parsedDeviceStatus.frame_length;
> > + }
> > + deviceStatus.analogue_gain = parsedDeviceStatus.analogue_gain;
> > +
> > + LOG(IPARPI, Debug) << "Metadata updated - " << deviceStatus;
> > +
> > + metadata.Set("device.status", deviceStatus);
> > +}
>
> I guess my only feeling about this is how much it copies from the base
> class version (well, parseEmbeddedData, if that still exists?). I
> expect you've stared at this yourself for far too long and are
> probably quite sick of it by now!!
>
> Here's one last thought, but please feel free to ignore it because
> I'll be fine with what you have above!
>
> The thing is, our default parseEmbeddedData method gives embedded data
> a chance to overwrite what came from the delayed controls.
I do agree that CamHelperImx477::Prepare() does look slightly similar to
the default CamHelper::parseEmbeddedData(). In fact, in the last revision,
the former would call the latter! I decided to do it the above way to be
slightly more efficient in how many times we touch the various metadata
buffers flying around within the general Prepare() operation. This way does
save 2 accesses, but that does not seem like much to be honest.
What if we
> derived PopulateMetadata instead. I think we'd have to pass it the
> regular metadata too, but then it would get a chance to populate the
> parsedMetadata from whichever source it liked better. And everything
> else would be untouched. Possibly.
>
I wouldn't mind doing this, except I don't really like passing in two
metadata
buffers into PopulateMetadata(), it just feels wrong to me.
Perhaps I should put back the original logic where
CamHelperImx477::Prepare()
would call CamHelper::parseEmbeddedData() but at the expense of a
few more accesses to the metadata buffers?
> But please don't feel obliged. Maybe that's just more devious than we
> want - I'll leave it up to you!
>
> > +
> > +uint32_t CamHelperImx477::GetVBlanking(Duration &exposure,
> > + Duration minFrameDuration,
> > + Duration maxFrameDuration) const
> > +{
> > + uint32_t frameLength, exposureLines;
> > + unsigned int shift = 0;
> > +
> > + frameLength = mode_.height + CamHelper::GetVBlanking(exposure,
> minFrameDuration,
> > +
> maxFrameDuration);
> > + /*
> > + * Check if the frame length calculated needs to be setup for
> long
> > + * exposure mode. This will require us to use a long exposure
> scale
> > + * factor provided by a shift operation in the sensor.
> > + */
> > + while (frameLength > frameLengthMax) {
> > + if (++shift > longExposureShiftMax) {
> > + shift = longExposureShiftMax;
> > + frameLength = frameLengthMax;
> > + break;
> > + }
> > + frameLength >>= 1;
> > + }
> > +
> > + if (shift) {
> > + /* Account for any rounding in the scaled frame length
> value. */
> > + frameLength <<= shift;
> > + exposureLines = CamHelper::ExposureLines(exposure);
> > + exposureLines = std::min(exposureLines, frameLength -
> frameIntegrationDiff);
> > + exposure = CamHelper::Exposure(exposureLines);
>
> I think I asked once before about qualifying these calls with the base
> class name when it isn't necessary. Do we do it elsewhere? Does it
> make you worry that these methods might be virtual when they aren't?
> But I really don't mind, and am happy either way.
>
You are right, it should be fine to remove the base class name from this
call. I will amend that in the next revision.
Regards,
Naush
>
> As I said, I'm fine with this as it stands, so:
>
> Reviewed-by: David Plowman <david.plowman at raspberrypi.com>
>
> Thanks
> David
>
> > + }
> > +
> > + return frameLength - mode_.height;
> > +}
> > +
> > void CamHelperImx477::GetDelays(int &exposure_delay, int &gain_delay,
> > int &vblank_delay) const
> > {
> > --
> > 2.25.1
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.libcamera.org/pipermail/libcamera-devel/attachments/20210705/a203c978/attachment-0001.htm>
More information about the libcamera-devel
mailing list