[libcamera-devel] [PATCH 6/6] ipa: raspberrypi: Generalise the SMIA metadata parser
Naushir Patuck
naush at raspberrypi.com
Tue Jun 15 11:46:44 CEST 2021
Hi Laurent,
Thank you for your feedback.
On Tue, 15 Jun 2021 at 04:09, Laurent Pinchart <
laurent.pinchart at ideasonboard.com> wrote:
> Hi Naush,
>
> Thank you for the patch.
>
> On Mon, Jun 14, 2021 at 10:53:40AM +0100, Naushir Patuck wrote:
> > Instead of having each CamHelper subclass the MdParserSmia, change the
> > implementation of MdParserSmia to be more generic. The MdParserSmia now
> gets
> > given a list of registers to search for and helper functions are used to
> compute
> > exposure lines and gain codes from these registers.
> >
> > Update the imx219 and imx477 CamHelpers by using this new mechanism.
> >
> > As a drive-by change, fixup a possible buffer overrun in the parsing
> code.
>
> Could that be split to a separate patch ?
Sure, will do.
>
> > Signed-off-by: Naushir Patuck <naush at raspberrypi.com>
> > ---
> > src/ipa/raspberrypi/cam_helper_imx219.cpp | 117 ++++----------------
> > src/ipa/raspberrypi/cam_helper_imx477.cpp | 125 ++++------------------
> > src/ipa/raspberrypi/md_parser.hpp | 42 ++++++--
> > src/ipa/raspberrypi/md_parser_smia.cpp | 89 ++++++++++++---
> > 4 files changed, 153 insertions(+), 220 deletions(-)
> >
> > diff --git a/src/ipa/raspberrypi/cam_helper_imx219.cpp
> b/src/ipa/raspberrypi/cam_helper_imx219.cpp
> > index 36dbe8cd941a..72c1042ad6be 100644
> > --- a/src/ipa/raspberrypi/cam_helper_imx219.cpp
> > +++ b/src/ipa/raspberrypi/cam_helper_imx219.cpp
> > @@ -23,21 +23,13 @@
> >
> > using namespace RPiController;
> >
> > -/* Metadata parser implementation specific to Sony IMX219 sensors. */
> > -
> > -class MdParserImx219 : public MdParserSmia
> > -{
> > -public:
> > - MdParserImx219();
> > - Status Parse(libcamera::Span<const uint8_t> buffer) override;
> > - Status GetExposureLines(unsigned int &lines) override;
> > - Status GetGainCode(unsigned int &gain_code) override;
> > -private:
> > - /* Offset of the register's value in the metadata block. */
> > - int reg_offsets_[3];
> > - /* Value of the register, once read from the metadata block. */
> > - int reg_values_[3];
> > -};
> > +/*
> > + * We care about one gain register and a pair of exposure registers.
> Their I2C
> > + * addresses from the Sony IMX219 datasheet:
> > + */
> > +constexpr uint32_t gainReg = 0x157;
> > +constexpr uint32_t expHiReg = 0x15A;
>
> Lowercase for hex constants please.
>
Ack.
>
> > +constexpr uint32_t expLoReg = 0x15B;
> >
> > class CamHelperImx219 : public CamHelper
> > {
> > @@ -55,11 +47,23 @@ private:
> > */
> > static constexpr int frameIntegrationDiff = 4;
> >
> > - MdParserImx219 imx219_parser;
> > + MdParserSmia imx219_parser;
> > +
> > + static uint32_t ParseExposureLines(const MdParserSmia::RegMap &map)
> > + {
> > + return map.at(expHiReg).value * 256 + map.at
> (expLoReg).value;
> > + }
> > +
> > + static uint32_t ParseGainCode(const MdParserSmia::RegMap &map)
> > + {
> > + return map.at(gainReg).value;
> > + }
> > };
> >
> > CamHelperImx219::CamHelperImx219()
> > - : CamHelper(frameIntegrationDiff)
> > + : CamHelper(frameIntegrationDiff),
> > + imx219_parser({ expHiReg, expLoReg, gainReg },
> > + ParseGainCode, ParseExposureLines)
>
> I'm not thrilled by this to be honest, as we'll have to add more
> functions when we'll want to parse more than just the exposure time and
> gain. Could we use a mechanism that would scale better ? Decoupling the
> data parser from the data interpreter would be one way to achieve this.
> You could pass the list of registers you're interested in to the
> MdParserSmia constructor. It would the parse the embedded data in
> findRegs() (which should be renamed) and return a map of register
> address to register value. The CamHelper would then be responsible for
> extracting the register values and converting them.
>
Yes, I do agree that this is somewhat restrictive. Here's a way to decouple
the parsing and supplying values:
1) MdParser::Parse and MdParserSmia::Parse now return the register to offset
map, instead of keeping this internal:
Status Parse(libcamera::Span<const uint8_t> buffer, RegMap &map);
2) CamHelper::parseEmbeddedData calls MdParser::Parse() as it currently
does,
and this will now call a new virtual method CamHelper::fillMetadata() that
will
fill our metadata buffer with whatever embedded data bits we think useful.
This way, all the sensor specific dealing is hidden behind the sensor
derived
CamHelper class.
I did not get your point about renaming findRegs(), could you elaborate on
that
a bit?
I'll prepare this change and we can discuss further once I have some code
written.
Regards,
Naush
> > {
> > #if ENABLE_EMBEDDED_DATA
> > parser_ = &imx219_parser;
> > @@ -97,82 +101,3 @@ static CamHelper *Create()
> > }
> >
> > static RegisterCamHelper reg("imx219", &Create);
> > -
> > -/*
> > - * We care about one gain register and a pair of exposure registers.
> Their I2C
> > - * addresses from the Sony IMX219 datasheet:
> > - */
> > -#define GAIN_REG 0x157
> > -#define EXPHI_REG 0x15A
> > -#define EXPLO_REG 0x15B
> > -
> > -/*
> > - * Index of each into the reg_offsets and reg_values arrays. Must be in
> > - * register address order.
> > - */
> > -#define GAIN_INDEX 0
> > -#define EXPHI_INDEX 1
> > -#define EXPLO_INDEX 2
> > -
> > -MdParserImx219::MdParserImx219()
> > -{
> > - reg_offsets_[0] = reg_offsets_[1] = reg_offsets_[2] = -1;
> > -}
> > -
> > -MdParser::Status MdParserImx219::Parse(libcamera::Span<const uint8_t>
> buffer)
> > -{
> > - bool try_again = false;
> > -
> > - if (reset_) {
> > - /*
> > - * Search again through the metadata for the gain and
> exposure
> > - * registers.
> > - */
> > - assert(bits_per_pixel_);
> > - /* Need to be ordered */
> > - uint32_t regs[3] = { GAIN_REG, EXPHI_REG, EXPLO_REG };
> > - reg_offsets_[0] = reg_offsets_[1] = reg_offsets_[2] = -1;
> > - int ret = static_cast<int>(findRegs(buffer,
> > - regs, reg_offsets_,
> 3));
> > - /*
> > - * > 0 means "worked partially but parse again next time",
> > - * < 0 means "hard error".
> > - */
> > - if (ret > 0)
> > - try_again = true;
> > - else if (ret < 0)
> > - return ERROR;
> > - }
> > -
> > - for (int i = 0; i < 3; i++) {
> > - if (reg_offsets_[i] == -1)
> > - continue;
> > -
> > - reg_values_[i] = buffer[reg_offsets_[i]];
> > - }
> > -
> > - /* Re-parse next time if we were unhappy in some way. */
> > - reset_ = try_again;
> > -
> > - return OK;
> > -}
> > -
> > -MdParser::Status MdParserImx219::GetExposureLines(unsigned int &lines)
> > -{
> > - if (reg_offsets_[EXPHI_INDEX] == -1 || reg_offsets_[EXPLO_INDEX]
> == -1)
> > - return NOTFOUND;
> > -
> > - lines = reg_values_[EXPHI_INDEX] * 256 + reg_values_[EXPLO_INDEX];
> > -
> > - return OK;
> > -}
> > -
> > -MdParser::Status MdParserImx219::GetGainCode(unsigned int &gain_code)
> > -{
> > - if (reg_offsets_[GAIN_INDEX] == -1)
> > - return NOTFOUND;
> > -
> > - gain_code = reg_values_[GAIN_INDEX];
> > -
> > - return OK;
> > -}
> > diff --git a/src/ipa/raspberrypi/cam_helper_imx477.cpp
> b/src/ipa/raspberrypi/cam_helper_imx477.cpp
> > index 038a8583d311..7a1100c25afc 100644
> > --- a/src/ipa/raspberrypi/cam_helper_imx477.cpp
> > +++ b/src/ipa/raspberrypi/cam_helper_imx477.cpp
> > @@ -15,21 +15,14 @@
> >
> > using namespace RPiController;
> >
> > -/* Metadata parser implementation specific to Sony IMX477 sensors. */
> > -
> > -class MdParserImx477 : public MdParserSmia
> > -{
> > -public:
> > - MdParserImx477();
> > - Status Parse(libcamera::Span<const uint8_t> buffer) override;
> > - Status GetExposureLines(unsigned int &lines) override;
> > - Status GetGainCode(unsigned int &gain_code) override;
> > -private:
> > - /* Offset of the register's value in the metadata block. */
> > - int reg_offsets_[4];
> > - /* Value of the register, once read from the metadata block. */
> > - int reg_values_[4];
> > -};
> > +/*
> > + * We care about two gain registers and a pair of exposure registers.
> Their
> > + * I2C addresses from the Sony IMX477 datasheet:
> > + */
> > +constexpr uint32_t expHiReg = 0x0202;
> > +constexpr uint32_t expLoReg = 0x0203;
> > +constexpr uint32_t gainHiReg = 0x0204;
> > +constexpr uint32_t gainLoReg = 0x0205;
> >
> > class CamHelperImx477 : public CamHelper
> > {
> > @@ -48,11 +41,23 @@ private:
> > */
> > static constexpr int frameIntegrationDiff = 22;
> >
> > - MdParserImx477 imx477_parser;
> > + MdParserSmia imx477_parser;
> > +
> > + static uint32_t ParseExposureLines(const MdParserSmia::RegMap &map)
> > + {
> > + return map.at(expHiReg).value * 256 + map.at
> (expLoReg).value;
> > + }
> > +
> > + static uint32_t ParseGainCode(const MdParserSmia::RegMap &map)
> > + {
> > + return map.at(gainHiReg).value * 256 + map.at
> (gainLoReg).value;
> > + }
> > };
> >
> > CamHelperImx477::CamHelperImx477()
> > - : CamHelper(frameIntegrationDiff)
> > + : CamHelper(frameIntegrationDiff),
> > + imx477_parser({ expHiReg, expLoReg, gainHiReg, gainLoReg },
> > + ParseGainCode, ParseExposureLines)
> > {
> > parser_ = &imx477_parser;
> > }
> > @@ -86,89 +91,3 @@ static CamHelper *Create()
> > }
> >
> > static RegisterCamHelper reg("imx477", &Create);
> > -
> > -/*
> > - * We care about two gain registers and a pair of exposure registers.
> Their
> > - * I2C addresses from the Sony IMX477 datasheet:
> > - */
> > -#define EXPHI_REG 0x0202
> > -#define EXPLO_REG 0x0203
> > -#define GAINHI_REG 0x0204
> > -#define GAINLO_REG 0x0205
> > -
> > -/*
> > - * Index of each into the reg_offsets and reg_values arrays. Must be in
> register
> > - * address order.
> > - */
> > -#define EXPHI_INDEX 0
> > -#define EXPLO_INDEX 1
> > -#define GAINHI_INDEX 2
> > -#define GAINLO_INDEX 3
> > -
> > -MdParserImx477::MdParserImx477()
> > -{
> > - reg_offsets_[0] = reg_offsets_[1] = reg_offsets_[2] =
> reg_offsets_[3] = -1;
> > -}
> > -
> > -MdParser::Status MdParserImx477::Parse(libcamera::Span<const uint8_t>
> buffer)
> > -{
> > - bool try_again = false;
> > -
> > - if (reset_) {
> > - /*
> > - * Search again through the metadata for the gain and
> exposure
> > - * registers.
> > - */
> > - assert(bits_per_pixel_);
> > - /* Need to be ordered */
> > - uint32_t regs[4] = {
> > - EXPHI_REG,
> > - EXPLO_REG,
> > - GAINHI_REG,
> > - GAINLO_REG
> > - };
> > - reg_offsets_[0] = reg_offsets_[1] = reg_offsets_[2] =
> reg_offsets_[3] = -1;
> > - int ret = static_cast<int>(findRegs(buffer,
> > - regs, reg_offsets_,
> 4));
> > - /*
> > - * > 0 means "worked partially but parse again next time",
> > - * < 0 means "hard error".
> > - */
> > - if (ret > 0)
> > - try_again = true;
> > - else if (ret < 0)
> > - return ERROR;
> > - }
> > -
> > - for (int i = 0; i < 4; i++) {
> > - if (reg_offsets_[i] == -1)
> > - continue;
> > -
> > - reg_values_[i] = buffer[reg_offsets_[i]];
> > - }
> > -
> > - /* Re-parse next time if we were unhappy in some way. */
> > - reset_ = try_again;
> > -
> > - return OK;
> > -}
> > -
> > -MdParser::Status MdParserImx477::GetExposureLines(unsigned int &lines)
> > -{
> > - if (reg_offsets_[EXPHI_INDEX] == -1 || reg_offsets_[EXPLO_INDEX]
> == -1)
> > - return NOTFOUND;
> > -
> > - lines = reg_values_[EXPHI_INDEX] * 256 + reg_values_[EXPLO_INDEX];
> > -
> > - return OK;
> > -}
> > -
> > -MdParser::Status MdParserImx477::GetGainCode(unsigned int &gain_code)
> > -{
> > - if (reg_offsets_[GAINHI_INDEX] == -1 || reg_offsets_[GAINLO_INDEX]
> == -1)
> > - return NOTFOUND;
> > -
> > - gain_code = reg_values_[GAINHI_INDEX] * 256 +
> reg_values_[GAINLO_INDEX];
> > -
> > - return OK;
> > -}
> > diff --git a/src/ipa/raspberrypi/md_parser.hpp
> b/src/ipa/raspberrypi/md_parser.hpp
> > index 25ba0e7c9400..6bbcdec0830b 100644
> > --- a/src/ipa/raspberrypi/md_parser.hpp
> > +++ b/src/ipa/raspberrypi/md_parser.hpp
> > @@ -6,6 +6,10 @@
> > */
> > #pragma once
> >
> > +#include <functional>
> > +#include <map>
> > +#include <vector>
> > +
> > #include <libcamera/span.h>
> >
> > /* Camera metadata parser class. Usage as shown below.
> > @@ -16,7 +20,8 @@
> > * application code doesn't have to worry which to kind to instantiate.
> But for
> > * the sake of example let's suppose we're parsing imx219 metadata.
> > *
> > - * MdParser *parser = new MdParserImx219(); // for example
> > + * MdParser *parser = new MdParserSmia({ expHiReg, expLoReg, gainReg },
> > + ParseGainCode, ParseExposureLines));
> > * parser->SetBitsPerPixel(bpp);
> > * parser->SetLineLengthBytes(pitch);
> > * parser->SetNumLines(2);
> > @@ -113,14 +118,32 @@ protected:
> > * md_parser_imx219.cpp for an example).
> > */
> >
> > -class MdParserSmia : public MdParser
> > +class MdParserSmia final : public MdParser
> > {
> > public:
> > - MdParserSmia() : MdParser()
> > - {
> > - }
> > + struct Register {
> > + Register()
> > + : offset(0), value(0), found(false)
> > + {
> > + }
> > +
> > + uint32_t offset;
> > + uint32_t value;
> > + bool found;
> > + };
> >
> > -protected:
> > + /* Maps register address to offset in the buffer. */
> > + using RegMap = std::map<uint32_t, Register>;
> > + using GetFn = std::function<uint32_t(const RegMap&)>;
> > +
> > + MdParserSmia(const std::vector<uint32_t> ®s, GetFn gain_fn,
> > + GetFn exposureFn);
> > +
> > + MdParser::Status Parse(libcamera::Span<const uint8_t> buffer)
> override;
> > + Status GetExposureLines(unsigned int &lines) override;
> > + Status GetGainCode(unsigned int &gain_code) override;
> > +
> > +private:
> > /*
> > * Note that error codes > 0 are regarded as non-fatal; codes < 0
> > * indicate a bad data buffer. Status codes are:
> > @@ -138,8 +161,11 @@ protected:
> > BAD_PADDING = -5
> > };
> >
> > - ParseStatus findRegs(libcamera::Span<const uint8_t> buffer,
> uint32_t regs[],
> > - int offsets[], unsigned int num_regs);
> > + ParseStatus findRegs(libcamera::Span<const uint8_t> buffer);
> > +
> > + RegMap map_;
> > + GetFn gain_fn_;
> > + GetFn exposure_fn_;
> > };
> >
> > } // namespace RPi
> > diff --git a/src/ipa/raspberrypi/md_parser_smia.cpp
> b/src/ipa/raspberrypi/md_parser_smia.cpp
> > index 65ffbe00c76e..f4748dd535d0 100644
> > --- a/src/ipa/raspberrypi/md_parser_smia.cpp
> > +++ b/src/ipa/raspberrypi/md_parser_smia.cpp
> > @@ -8,9 +8,11 @@
> > #include <map>
> > #include <string>
> >
> > +#include "libcamera/internal/log.h"
> > #include "md_parser.hpp"
> >
> > using namespace RPiController;
> > +using namespace libcamera;
> >
> > /*
> > * This function goes through the embedded data to find the offsets (not
> > @@ -28,18 +30,79 @@ constexpr unsigned int REG_LOW_BITS = 0xa5;
> > constexpr unsigned int REG_VALUE = 0x5a;
> > constexpr unsigned int REG_SKIP = 0x55;
> >
> > -MdParserSmia::ParseStatus MdParserSmia::findRegs(libcamera::Span<const
> uint8_t> buffer,
> > - uint32_t regs[], int
> offsets[],
> > - unsigned int num_regs)
> > +MdParserSmia::MdParserSmia(const std::vector<uint32_t> ®isters,
> > + GetFn gain_fn, GetFn exposure_fn)
> > + : gain_fn_(gain_fn), exposure_fn_(exposure_fn)
> > {
> > - assert(num_regs > 0);
> > + for (auto r : registers)
> > + map_[r] = {};
> > +}
> > +
> > +MdParser::Status MdParserSmia::Parse(libcamera::Span<const uint8_t>
> buffer)
> > +{
> > + if (reset_) {
> > + /*
> > + * Search again through the metadata for the gain and
> exposure
> > + * registers.
> > + */
> > + ASSERT(bits_per_pixel_);
> > +
> > + ParseStatus ret = findRegs(buffer);
> > + /*
> > + * > 0 means "worked partially but parse again next time",
> > + * < 0 means "hard error".
> > + *
> > + * In either case, we retry parsing on the next frame.
> > + */
> > + if (ret != PARSE_OK)
> > + return ERROR;
> > +
> > + reset_ = false;
> > + }
> > +
> > + /* Populate the register values requested. */
> > + for (auto &kv : map_) {
> > + Register ® = kv.second;
> > +
> > + if (!reg.found) {
> > + reset_ = true;
> > + return NOTFOUND;
> > + }
> > +
> > + reg.value = buffer[reg.offset];
> > + }
> > +
> > + return OK;
> > +}
> > +
> > +MdParser::Status MdParserSmia::GetExposureLines(unsigned int &lines)
> > +{
> > + if (reset_)
> > + return NOTFOUND;
> > +
> > + lines = exposure_fn_(map_);
> > + return OK;
> > +}
> > +
> > +MdParser::Status MdParserSmia::GetGainCode(unsigned int &gain_code)
> > +{
> > + if (reset_)
> > + return NOTFOUND;
> > +
> > + gain_code = gain_fn_(map_);
> > + return OK;
> > +}
> > +
> > +MdParserSmia::ParseStatus MdParserSmia::findRegs(libcamera::Span<const
> uint8_t> buffer)
> > +{
> > + ASSERT(map_.size());
> >
> > if (buffer[0] != LINE_START)
> > return NO_LINE_START;
> >
> > unsigned int current_offset = 1; /* after the LINE_START */
> > unsigned int current_line_start = 0, current_line = 0;
> > - unsigned int reg_num = 0, first_reg = 0;
> > + unsigned int reg_num = 0, regs_done = 0;
> >
> > while (1) {
> > int tag = buffer[current_offset++];
> > @@ -73,8 +136,8 @@ MdParserSmia::ParseStatus
> MdParserSmia::findRegs(libcamera::Span<const uint8_t>
> > return NO_LINE_START;
> > } else {
> > /* allow a zero line length to mean "hunt
> for the next line" */
> > - while (buffer[current_offset] !=
> LINE_START &&
> > - current_offset < buffer.size())
> > + while (current_offset < buffer.size() &&
> > + buffer[current_offset] !=
> LINE_START)
> > current_offset++;
> >
> > if (current_offset == buffer.size())
> > @@ -91,13 +154,13 @@ MdParserSmia::ParseStatus
> MdParserSmia::findRegs(libcamera::Span<const uint8_t>
> > else if (tag == REG_SKIP)
> > reg_num++;
> > else if (tag == REG_VALUE) {
> > - while (reg_num >=
> > - /* assumes registers are in
> order... */
> > - regs[first_reg]) {
> > - if (reg_num == regs[first_reg])
> > - offsets[first_reg] =
> current_offset - 1;
> > + auto reg = map_.find(reg_num);
> > +
> > + if (reg != map_.end()) {
> > + map_[reg_num].offset =
> current_offset - 1;
> > + map_[reg_num].found = true;
> >
> > - if (++first_reg == num_regs)
> > + if (++regs_done == map_.size())
> > return PARSE_OK;
> > }
> > reg_num++;
>
> --
> Regards,
>
> Laurent Pinchart
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.libcamera.org/pipermail/libcamera-devel/attachments/20210615/1dfb91d5/attachment-0001.htm>
More information about the libcamera-devel
mailing list