[libcamera-devel] [PATCH v3 10/38] libcamera: Add IPADataSerializer
Jacopo Mondi
jacopo at jmondi.org
Tue Oct 27 12:30:19 CET 2020
Hi Paul,
On Tue, Oct 27, 2020 at 07:14:03PM +0900, paul.elder at ideasonboard.com wrote:
> Hi Jacopo,
>
> Thank you for the thorough review!
>
Thank you for the enormous work
> On Mon, Oct 26, 2020 at 05:07:52PM +0100, Jacopo Mondi wrote:
> > Hi Paul,
> >
> > On Fri, Oct 02, 2020 at 11:31:26PM +0900, Paul Elder wrote:
> > > Add an IPADataSerializer which implments de/serialization of built-in
> > > (PODs, vector, map, string) and libcamera data structures. This is
> > > intended to be used by the proxy and the proxy worker in the IPC layer.
> > >
> > > Signed-off-by: Paul Elder <paul.elder at ideasonboard.com>
> > >
> > > ---
> > > Changes in v3:
> > > - reimplement append/readUInt with memcpy (intead of bit shifting)
> > > - change DECLARE_INTEGRAL_SERIALIZER with DECLARE_POD_SERIALIZER
> > > - use this for int64_t, bool, float, and double
> > > - fix comment style
> > >
> > > Changes in v2:
> > > - added serializers for all integer types, bool, and string
> > > - use string serializer for IPASettings serializer
> > > - add documentation
> > > - add serializer for const ControlList
> > > ---
> > > .../libcamera/internal/ipa_data_serializer.h | 1014 +++++++++++++++++
> > > src/libcamera/ipa_data_serializer.cpp | 154 +++
> > > src/libcamera/meson.build | 1 +
> > > 3 files changed, 1169 insertions(+)
> > > create mode 100644 include/libcamera/internal/ipa_data_serializer.h
> > > create mode 100644 src/libcamera/ipa_data_serializer.cpp
> > >
> > > diff --git a/include/libcamera/internal/ipa_data_serializer.h b/include/libcamera/internal/ipa_data_serializer.h
> > > new file mode 100644
> > > index 00000000..9cd35c4c
> > > --- /dev/null
> > > +++ b/include/libcamera/internal/ipa_data_serializer.h
> > > @@ -0,0 +1,1014 @@
> > > +/* SPDX-License-Identifier: LGPL-2.1-or-later */
> > > +/*
> > > + * Copyright (C) 2020, Google Inc.
> > > + *
> > > + * ipa_data_serializer.h - Image Processing Algorithm data serializer
> > > + */
> > > +#ifndef __LIBCAMERA_INTERNAL_IPA_DATA_SERIALIZER_H__
> > > +#define __LIBCAMERA_INTERNAL_IPA_DATA_SERIALIZER_H__
> > > +
> > > +#include <deque>
> > > +#include <iostream>
> > > +#include <string.h>
> > > +#include <tuple>
> > > +#include <vector>
> > > +
> > > +#include <libcamera/buffer.h>
> > > +#include <libcamera/control_ids.h>
> > > +#include <libcamera/geometry.h>
> > > +#include <libcamera/ipa/ipa_interface.h>
> > > +
> > > +#include "libcamera/internal/byte_stream_buffer.h"
> > > +#include "libcamera/internal/camera_sensor.h"
> > > +#include "libcamera/internal/control_serializer.h"
> > > +#include "libcamera/internal/log.h"
> > > +
> > > +namespace libcamera {
> > > +
> > > +LOG_DECLARE_CATEGORY(IPADataSerializer)
> > > +
> >
> > should these be wrapped in an anonymous namespace ? They are used by
> > the generated proxyes, but the header is included. I tried it locally
> > and complier seems to digest it
>
> Ah yes, probably.
>
> > > +template<typename T>
> > > +void appendUInt(std::vector<uint8_t> &vec, T val)
> > > +{
> > > + size_t byteWidth = sizeof(val);
> > > + std::vector<uint8_t> v(byteWidth);
> > > + memcpy(&v[0], &val, byteWidth);
> > > +
> > > + vec.insert(vec.end(), v.begin(), v.end());
> > > +}
> > > +
> > > +template<typename T>
> > > +T readUInt(std::vector<uint8_t> &vec, size_t pos)
> > > +{
> > > + T ret = 0;
> > > + size_t byteWidth = sizeof(ret);
> > > + if (pos + byteWidth > vec.size())
> > > + return ret;
> > > +
> > > + memcpy(&ret, &vec.data()[pos], byteWidth);
> >
> > Just &vec[pos] should be enough
>
> It's fine even for memcpying?
>
std::vector::operator[] returns a T&, so taking its address should be
equivalent to &std::vector::data()[pos]
> > > + return ret;
> > > +}
> > > +
> > > +template<typename T>
> > > +T readUInt(std::vector<uint8_t>::iterator it)
> > > +{
> > > + T ret = 0;
> > > + size_t byteWidth = sizeof(ret);
> > > +
> > > + std::vector<uint8_t> v(it, it + byteWidth);
> > > + memcpy(&ret, v.data(), byteWidth);
> > > + return ret;
> > > +}
> >
> > These new versions looks much better!
>
> \o/
>
> Ah, so later on you're saying that I should rename these, since they're
> not limited to uints?
>
Yes, I think removing UInt from the names might be useful.
As said below this apply to most POD types, so appendPOD() and
readPOD() might be more appropriate. But even just 'append()' and
'read()' might be enough and nicer to read (assuming they're wrapped
in an anonymous namespace). Ah wait, 'read()' -might- already be used :)
We could live with that, as the read syscall shall be prefixed with ::
This, in example, works with g++ 10.2.0 and clang++ 10.0.1
#include <unistd.h>
#include <iostream>
namespace {
void read(char b)
{
std::cout << b << std::endl;
}
};
int main()
{
char a;
::read(1, &a, 1);
read(a);
return 0;
}
I'm not sure we want to go there though.
Otherwise, name the namespace that wraps readUInt() ? You can use
'detail' as the namespace name as it's done for Span<> ?
> > > +
> > > +template<typename T>
> > > +class IPADataSerializer
> > > +{
> > > +#ifdef __DOXYGEN__
> > > +public:
> > > + static std::tuple<std::vector<uint8_t>, std::vector<int32_t>>
> > > + serialize(T data, ControlSerializer *cs);
> > > +
> > > + static T deserialize(std::vector<uint8_t> &data,
> > > + ControlSerializer *cs);
> > > + static T deserialize(std::vector<uint8_t>::iterator it1,
> > > + std::vector<uint8_t>::iterator it2,
> > > + ControlSerializer *cs);
> > > +
> >
> > Intentional empty line ?
>
> Yes, the first two are data only, and the bottom two are data+fd.
>
> > > + static T deserialize(std::vector<uint8_t> &data,
> > > + std::vector<int32_t> &fds,
> > > + ControlSerializer *cs);
> > > + static T deserialize(std::vector<uint8_t>::iterator data_it1,
> > > + std::vector<uint8_t>::iterator data_it2,
> > > + std::vector<int32_t>::iterator fds_it1,
> > > + std::vector<int32_t>::iterator fds_it2,
> > > + ControlSerializer *cs);
> >
> > Aren't these better called data_start, data_end and fds_start, fds_end
> > ?
>
> Yeah, that's better.
>
> > > +#endif /* __DOXYGEN__ */
> >
> > Do you need this ifdef __DOXYGEN__ ? you want these parsed, don't you ?
> > It's usually #ifndef __DOXYGEN__ to exclude from documentation parsing
> > some parts of the code, like you're done below
>
> I do need this ifdef. I want doxygen to document just the functions in
> the base serializer, since it's the same interface for all serializers,
But won't doxygen parse it anyway. I tried removing the #ifdef and I
have documentation generated for IPADataSerializer::serialize() and
friends generated.
> but I don't want the base serializer template to be compiled.
That's ok. And that's why you need the below #ifndef
Or didn't I get what you mean with 'base serializer' ?
>
> > > +};
> > > +
> > > +#ifndef __DOXYGEN__
> > > +
> > > +template<typename V>
> > > +class IPADataSerializer<std::vector<V>>
> > > +{
> > > +public:
> > > + static std::tuple<std::vector<uint8_t>, std::vector<int32_t>>
> > > + serialize(const std::vector<V> &data, ControlSerializer *cs = nullptr)
> > > + {
> > > + std::vector<uint8_t> data_vec;
> >
> > We can reserve data.size() * sizeof(V) in the vector
>
> I'm not sure we can do that...
>
> > > + std::vector<int32_t> fds_vec;
> > > +
> > > + /* Serialize the length. */
> > > + uint32_t vec_len = data.size();
> > > + appendUInt<uint32_t>(data_vec, vec_len);
> > > +
> > > + /* Serialize the members. */
> > > + for (auto it = data.begin(); it != data.end(); ++it) {
> >
> > for (auto const &it : data) ?
>
> ack
>
> > > + std::vector<uint8_t> dvec;
> > > + std::vector<int32_t> fvec;
> > > +
> > > + std::tie(dvec, fvec) =
> > > + IPADataSerializer<V>::serialize(*it, cs);
> > > +
> > > + appendUInt<uint32_t>(data_vec, dvec.size());
> >
> > Why is the size (in bytes) of each serialized entry recorded in the
> > serialized data buffer ? Aren't all the members of the same size ?
>
> ...because the members of the vector aren't necessarily the same size.
> The vector could contains variable-size objects, like other vectors, or
> maps, or strings, etc, so we need to tag the size of every element.
The fact you've pointed out that, in example, CameraSensor contains a
string of variable length makes me agree with you and the fact we have
to prefix the actual data with the expected size in the serialized
format. The same for pre-reserving the vectors, sizeof() won't surely
help with !POD data.
>
> > > + appendUInt<uint32_t>(data_vec, fvec.size());
> >
[snip]
> > > +#define DECLARE_POD_SERIALIZER(type) \
> > > +template<> \
> > > +class IPADataSerializer<type> \
> > > +{ \
> > > +public: \
> > > + static std::tuple<std::vector<uint8_t>, std::vector<int32_t>> \
> > > + serialize(const type data, \
> > > + [[maybe_unused]] ControlSerializer *cs = nullptr) \
> > > + { \
> > > + std::vector<uint8_t> data_vec; \
> >
> > Do you think pre-allocating sizeof(type) in the vector might help ?
>
> tbh, yeah.
>
> > > + appendUInt<type>(data_vec, data); \
>
> Oh, then this needs a writeUInt cousin.
>
Why so ?
> > > + \
> > > + return {data_vec, {}}; \
> > > + } \
> > > + \
> > > + static type deserialize(std::vector<uint8_t> &data, \
> > > + [[maybe_unused]] ControlSerializer *cs = nullptr)\
> > > + { \
> > > + return IPADataSerializer<type>::deserialize(data.begin(),\
> > > + data.end());\
> > > + } \
> > > + \
> > > + static type deserialize(std::vector<uint8_t>::iterator it1, \
> > > + [[maybe_unused]] std::vector<uint8_t>::iterator it2,\
> > > + [[maybe_unused]] ControlSerializer *cs = nullptr)\
> > > + { \
> > > + return readUInt<type>(it1); \
> > > + } \
> > > + \
> > > + static type deserialize(std::vector<uint8_t> &data, \
> > > + [[maybe_unused]] std::vector<int32_t> &fds,\
> > > + [[maybe_unused]] ControlSerializer *cs = nullptr)\
> > > + { \
> > > + return IPADataSerializer<type>::deserialize(data.begin(),\
> > > + data.end());\
> > > + } \
> > > + \
> > > + static type deserialize(std::vector<uint8_t>::iterator data_it1,\
> > > + std::vector<uint8_t>::iterator data_it2,\
> > > + [[maybe_unused]] std::vector<int32_t>::iterator fds_it1,\
> > > + [[maybe_unused]] std::vector<int32_t>::iterator fds_it2,\
> > > + [[maybe_unused]] ControlSerializer *cs = nullptr)\
> > > + { \
> > > + return IPADataSerializer<type>::deserialize(data_it1, \
> > > + data_it2); \
> > > + } \
> > > +};
> > > +
> > > +DECLARE_POD_SERIALIZER(bool)
> > > +DECLARE_POD_SERIALIZER(uint8_t)
> > > +DECLARE_POD_SERIALIZER(uint16_t)
> > > +DECLARE_POD_SERIALIZER(uint32_t)
> > > +DECLARE_POD_SERIALIZER(uint64_t)
> > > +DECLARE_POD_SERIALIZER(int8_t)
> > > +DECLARE_POD_SERIALIZER(int16_t)
> > > +DECLARE_POD_SERIALIZER(int32_t)
> > > +DECLARE_POD_SERIALIZER(int64_t)
> >
> > The I wonder if keeping here and in the documentation the term 'uint'
> > is correct and it shouldn't just be replaced with 'int'
>
> I'm guessing you're referring to {append,read}UInt()?
>
Yes
> > > +DECLARE_POD_SERIALIZER(float)
> > > +DECLARE_POD_SERIALIZER(double)
> >
> > Or even POD
>
> I'm not sure what you're referring to here :/
>
Still on the name, see above for the readPOD() appendPOD() discussion
(these two are awful names tbh)
> > > +
> > > +template<>
> > > +class IPADataSerializer<std::string>
> > > +{
> > > +public:
> > > + static std::tuple<std::vector<uint8_t>, std::vector<int32_t>>
> > > + serialize(const std::string &data, [[maybe_unused]] ControlSerializer *cs = nullptr)
> > > + {
> > > + std::vector<uint8_t> data_vec(data.begin(), data.end());
> > > +
> > > + return {data_vec, {}};
> > > + }
> > > +
> > > + static std::string deserialize(std::vector<uint8_t> &data,
> > > + [[maybe_unused]] ControlSerializer *cs = nullptr)
> > > + {
> > > + return IPADataSerializer<std::string>::deserialize(data.begin(), data.end());
> > > + }
> > > +
> > > + static std::string deserialize(std::vector<uint8_t>::iterator it1,
> > > + std::vector<uint8_t>::iterator it2,
> > > + [[maybe_unused]] ControlSerializer *cs = nullptr)
> > > + {
> > > + std::string str(it1, it2);
> > > +
> > > + return str;
> >
> > Can you just
> > return {it1, it2};
> > ?
>
> Ah, yes. I had it (and everything below) implemented in this manner
> because I wanted to algorithmize the serdes code generation, so I was
> practicing it on myself here. I guess it's about time for optimizations;
> thank you for the pointers.
>
> > > + }
> > > +
> > > + static std::string deserialize(std::vector<uint8_t> &data,
> > > + [[maybe_unused]] std::vector<int32_t> &fds,
> > > + [[maybe_unused]] ControlSerializer *cs = nullptr)
> > > + {
> > > + return IPADataSerializer<std::string>::deserialize(data.begin(), data.end());
> >
> > Maybe
> > return {data.being(), data.end()};
> >
> > would save a function call ?
> > Same for other overloaded methods (and possibly not only in the
> > <string> specialization).
> >
>
> ack
>
> > > + }
> > > +
> > > + static std::string deserialize(std::vector<uint8_t>::iterator data_it1,
> > > + std::vector<uint8_t>::iterator data_it2,
> > > + [[maybe_unused]] std::vector<int32_t>::iterator fds_it1,
> > > + [[maybe_unused]] std::vector<int32_t>::iterator fds_it2,
> > > + [[maybe_unused]] ControlSerializer *cs = nullptr)
> > > + {
> > > + return IPADataSerializer<std::string>::deserialize(data_it1, data_it2);
> > > + }
> > > +};
> > > +
> > > +template<>
> > > +class IPADataSerializer<FileDescriptor>
> > > +{
> > > +public:
> > > + static std::tuple<std::vector<uint8_t>, std::vector<int32_t>>
> > > + serialize(const FileDescriptor &data, [[maybe_unused]] ControlSerializer *cs = nullptr)
> > > + {
> > > + std::vector<uint8_t> data_vec = { data.isValid() };
> > > + std::vector<int32_t> fd_vec;
> > > + if (data.isValid())
> > > + fd_vec.push_back(data.fd());
> > > +
> > > + return {data_vec, fd_vec};
> > > + }
> > > +
> > > + static FileDescriptor deserialize(std::vector<uint8_t> &data, std::vector<int32_t> &fds,
> > > + [[maybe_unused]] ControlSerializer *cs = nullptr)
> > > + {
> > > + return IPADataSerializer<FileDescriptor>::deserialize(data.begin(), data.end(),
> > > + fds.begin(), fds.end());
> > > + }
> > > +
> > > + static FileDescriptor deserialize(std::vector<uint8_t>::iterator data_it1,
> > > + std::vector<uint8_t>::iterator data_it2,
> > > + std::vector<int32_t>::iterator fds_it1,
> > > + std::vector<int32_t>::iterator fds_it2,
> > > + [[maybe_unused]] ControlSerializer *cs = nullptr)
> > > + {
> > > + if (std::distance(data_it1, data_it2) < 1)
> > > + LOG(IPADataSerializer, Fatal)
> > > + << "Invalid data to deserialize FileDescriptor";
> > > +
> > > + bool valid = *data_it1;
> >
> > in C this would be
> > bool valid = !!*data_it1;
> > not sure about C++ and not even sure if the !! is just a paranoid
> > kernel convention, as I don't think even C requires that
>
> Oh... I'm not sure. It passed my de/serialization tests.
>
> > > +
> > > + if (valid && std::distance(fds_it1, fds_it2) < 1)
> >
> > if (valid && std::distance(fds_it1, fds_it2) < 1) {
> > LOG(IPADataSerializer, Fatal)
> > << "Invalid fds to deserialize FileDescriptor";
> > return {}
> > }
> >
> > return FileDescriptor(*fds_it1);
> >
> > Or is returning {} when std::distance() < 1 wrong ? I don't think so,
> > as even if it is "valid" there's not fds serialized.
>
> If std::distance() < 1 then *fds_it1 will segfault. So if valid and
> std::distance() < 1, then there's a big problem -> fatal. If not valid,
> then std::distance() doesn't matter, and we return a new uninitialized
> FileDescriptor, but we can't construct it using *fds_it1, since the
> iterator shouldn't be valid.
>
I've missed 'Fatal' in the LOG() call. Sorry about this.
> It's perfectly fine to send an invalid FileDescriptor, for example if
> the FileDescriptor field isn't used, or uninitialized. The problem with
> sending it directly is that sendmsg() EINVALs if we try to send -1, so
> we need this valid flag in the data vector.
>
> >
> > > + LOG(IPADataSerializer, Fatal)
> > > + << "Invalid fds to deserialize FileDescriptor";
> > > +
> > > + return valid ? FileDescriptor(*fds_it1) : FileDescriptor();
> > > + }
> > > +};
> > > +
> > > +template<>
> > > +class IPADataSerializer<IPASettings>
> > > +{
> > > +public:
> > > + static std::tuple<std::vector<uint8_t>, std::vector<int32_t>>
> > > + serialize(const IPASettings &data, [[maybe_unused]] ControlSerializer *cs = nullptr)
> > > + {
> > > + return IPADataSerializer<std::string>::serialize(data.configurationFile);
> > > + }
> > > +
> > > + static IPASettings deserialize(std::vector<uint8_t> &data,
> > > + [[maybe_unused]] ControlSerializer *cs = nullptr)
> > > + {
> > > + return IPADataSerializer<IPASettings>::deserialize(data.begin(), data.end());
> > > + }
> > > +
> > > + static IPASettings deserialize(std::vector<uint8_t>::iterator it1,
> > > + std::vector<uint8_t>::iterator it2,
> > > + [[maybe_unused]] ControlSerializer *cs = nullptr)
> > > + {
> > > + IPASettings ret;
> > > + ret.configurationFile = IPADataSerializer<std::string>::deserialize(it1, it2);
> > > +
> > > + return ret;
> >
> > return { IPADataSerializer<std::string>::deserialize(it1, it2) };
> >
> > ?
>
> Ah yes, much needed optimization.
>
> > > + }
> > > +
> > > + static IPASettings deserialize(std::vector<uint8_t> &data,
> > > + [[maybe_unused]] std::vector<int32_t> &fds,
> > > + [[maybe_unused]] ControlSerializer *cs = nullptr)
> > > + {
> > > + return IPADataSerializer<IPASettings>::deserialize(data.begin(), data.end());
> >
> > return { IPADataSerializer<std::string>::deserialize(data.begin(), data.end() };
> >
> > ?
>
> ack
>
> > > + }
> > > +
> > > + static IPASettings deserialize(std::vector<uint8_t>::iterator data_it1,
> > > + std::vector<uint8_t>::iterator data_it2,
> > > + [[maybe_unused]] std::vector<int32_t>::iterator fds_it1,
> > > + [[maybe_unused]] std::vector<int32_t>::iterator fds_it2,
> > > + [[maybe_unused]] ControlSerializer *cs = nullptr)
> > > + {
> > > + return IPADataSerializer<IPASettings>::deserialize(data_it1, data_it2);
> >
> > same
>
> ack
>
> > > + }
> > > +};
> > > +
> > > +template<>
> > > +class IPADataSerializer<CameraSensorInfo>
> > > +{
> > > +public:
> > > + static std::tuple<std::vector<uint8_t>, std::vector<int32_t>>
> > > + serialize(const CameraSensorInfo &data, [[maybe_unused]] ControlSerializer *cs = nullptr)
> > > + {
> > > + std::vector<uint8_t> data_vec;
> >
> > Reserving sizeof(CameraSensorInfo) might help
>
> It's got a string in it though, so we don't know sizeof(CameraSensorInfo).
>
It could be calculated though. Nevermind
> > I just noticed and I wonder why all this code uses this_naming_style
> > in place of the libcamera standard namingStyle
>
> Ah yes, I should probably fix that...
>
Thanks
> > > +
> > > + uint32_t str_len = data.model.size();
> > > + appendUInt<uint32_t>(data_vec, str_len);
> > > +
> > > + data_vec.insert(data_vec.end(), data.model.begin(), data.model.end());
> >
> > Shouldn't this be serialized using the IPADataSerializer<std::string>
> > if it's not needed, do we need the specialization then ?
>
> It should be :)
>
> > > +
> > > + appendUInt<uint32_t>(data_vec, data.bitsPerPixel);
> > > +
> > > + appendUInt<uint32_t>(data_vec, data.activeAreaSize.width);
> > > + appendUInt<uint32_t>(data_vec, data.activeAreaSize.height);
> > > +
> > > + appendUInt<uint32_t>(data_vec, static_cast<uint32_t>(data.analogCrop.x));
> > > + appendUInt<uint32_t>(data_vec, static_cast<uint32_t>(data.analogCrop.y));
> > > + appendUInt<uint32_t>(data_vec, data.analogCrop.width);
> > > + appendUInt<uint32_t>(data_vec, data.analogCrop.height);
> > > +
> > > + appendUInt<uint32_t>(data_vec, data.outputSize.width);
> > > + appendUInt<uint32_t>(data_vec, data.outputSize.height);
> > > +
> > > + appendUInt<uint64_t>(data_vec, data.pixelRate);
> > > +
> > > + appendUInt<uint32_t>(data_vec, data.lineLength);
> > > +
> > > + return {data_vec, {}};
> > > + }
> > > +
> > > + static CameraSensorInfo deserialize(std::vector<uint8_t> &data,
> > > + [[maybe_unused]] ControlSerializer *cs = nullptr)
> > > + {
> > > + return IPADataSerializer<CameraSensorInfo>::deserialize(data.begin(), data.end());
> > > + }
> > > +
> > > + static CameraSensorInfo deserialize(std::vector<uint8_t>::iterator it1,
> > > + [[maybe_unused]] std::vector<uint8_t>::iterator it2,
> > > + [[maybe_unused]] ControlSerializer *cs = nullptr)
> > > + {
> > > + CameraSensorInfo ret;
> > CameraSensorInfo ret{};
> >
> > Also for other statically allocated fields that are returned by other
> > specializations.
>
> Do we need to, since all the fields are populated below?
>
Technically not, and for all the supported types you fill in all
fields, you're right. But to be honest I won't mind a
0-initialization anyway, mostly because if we happen to add support
for new types, and if they won't be fully-filled, we keep the good
practice of {} and we're safe. A bit paranoid maybe :)
> > > +
> > > + uint32_t str_len = readUInt<uint32_t>(it1);
> > > + std::string str(it1 + 4, it1 + 4 + str_len);
> > > + ret.model = str;
> > > +
> > > + std::vector<uint8_t>::iterator it = it1 + 4 + str_len;
> > > +
> > > + ret.bitsPerPixel = readUInt<uint32_t>(it);
> > > +
> > > + ret.activeAreaSize.width = readUInt<uint32_t>(it + 4);
> > > + ret.activeAreaSize.height = readUInt<uint32_t>(it + 8);
> > > +
> > > + ret.analogCrop.x = static_cast<int32_t>(readUInt<uint32_t>(it + 12));
> > > + ret.analogCrop.y = static_cast<int32_t>(readUInt<uint32_t>(it + 16));
> > > + ret.analogCrop.width = readUInt<uint32_t>(it + 20);
> > > + ret.analogCrop.height = readUInt<uint32_t>(it + 24);
> > > +
> > > + ret.outputSize.width = readUInt<uint32_t>(it + 28);
> > > + ret.outputSize.height = readUInt<uint32_t>(it + 32);
> > > +
> > > + ret.pixelRate = readUInt<uint64_t>(it + 36);
> > > +
> > > + ret.lineLength = readUInt<uint64_t>(it + 44);
> >
> > lineLength is a 32 bits integer
>
> Ah, yes.
>
> > > +
> > > + return ret;
> > > + }
> > > +
> > > + static CameraSensorInfo deserialize(std::vector<uint8_t> &data,
> > > + [[maybe_unused]] std::vector<int32_t> &fds,
> > > + [[maybe_unused]] ControlSerializer *cs = nullptr)
> > > + {
> > > + return IPADataSerializer<CameraSensorInfo>::deserialize(data.begin(), data.end());
> > > + }
> > > +
> > > + static CameraSensorInfo deserialize(std::vector<uint8_t>::iterator data_it1,
> > > + std::vector<uint8_t>::iterator data_it2,
> > > + [[maybe_unused]] std::vector<int32_t>::iterator fds_it1,
> > > + [[maybe_unused]] std::vector<int32_t>::iterator fds_it2,
> > > + [[maybe_unused]] ControlSerializer *cs = nullptr)
> > > + {
> > > + return IPADataSerializer<CameraSensorInfo>::deserialize(data_it1, data_it2);
> > > + }
> > > +};
> > > +
> > > +template<>
> > > +class IPADataSerializer<IPAStream>
> > > +{
> > > +public:
> > > + static std::tuple<std::vector<uint8_t>, std::vector<int32_t>>
> > > + serialize(const IPAStream &data, [[maybe_unused]] ControlSerializer *cs = nullptr)
> > > + {
> > > + std::vector<uint8_t> data_vec;
> > > +
> > > + appendUInt<uint32_t>(data_vec, data.pixelFormat);
> > > +
> > > + appendUInt<uint32_t>(data_vec, data.size.width);
> > > + appendUInt<uint32_t>(data_vec, data.size.height);
> > > +
> > > + return {data_vec, {}};
> > > + }
> > > +
> > > + static IPAStream deserialize(std::vector<uint8_t> &data,
> > > + [[maybe_unused]] ControlSerializer *cs = nullptr)
> > > + {
> > > + return IPADataSerializer<IPAStream>::deserialize(data.begin(), data.end());
> > > + }
> > > +
> > > + static IPAStream deserialize(std::vector<uint8_t>::iterator it1,
> > > + [[maybe_unused]] std::vector<uint8_t>::iterator it2,
> > > + [[maybe_unused]] ControlSerializer *cs = nullptr)
> > > + {
> > > + IPAStream ret;
> > > +
> > > + ret.pixelFormat = readUInt<uint32_t>(it1);
> > > +
> > > + ret.size.width = readUInt<uint32_t>(it1 + 4);
> > > + ret.size.height = readUInt<uint32_t>(it1 + 8);
> > > +
> > > + return ret;
> > > + }
> > > +
> > > + static IPAStream deserialize(std::vector<uint8_t> &data,
> > > + [[maybe_unused]] std::vector<int32_t> &fds,
> > > + [[maybe_unused]] ControlSerializer *cs = nullptr)
> > > + {
> > > + return IPADataSerializer<IPAStream>::deserialize(data.begin(), data.end());
> > > + }
> > > +
> > > + static IPAStream deserialize(std::vector<uint8_t>::iterator data_it1,
> > > + std::vector<uint8_t>::iterator data_it2,
> > > + [[maybe_unused]] std::vector<int32_t>::iterator fds_it1,
> > > + [[maybe_unused]] std::vector<int32_t>::iterator fds_it2,
> > > + [[maybe_unused]] ControlSerializer *cs = nullptr)
> > > + {
> > > + return IPADataSerializer<IPAStream>::deserialize(data_it1, data_it2);
> > > + }
> > > +};
> > > +
> > > +template<>
> > > +class IPADataSerializer<const ControlList>
> > > +{
> > > +public:
> > > + /* map arg will be generated, since it's per-pipeline anyway. */
> > > + static std::tuple<std::vector<uint8_t>, std::vector<int32_t>>
> > > + serialize(const ControlList &data, const ControlInfoMap &map,
> > > + ControlSerializer *cs)
> > > + {
> > > + if (!cs)
> > > + LOG(IPADataSerializer, Fatal)
> > > + << "ControlSerializer not provided for serialization of ControlList";
> > > +
> > > + size_t size = cs->binarySize(map);
> > > + std::vector<uint8_t> infoData(size);
> > > + ByteStreamBuffer buffer(infoData.data(), infoData.size());
> > > + int ret = cs->serialize(map, buffer);
> > > +
> > > + if (ret < 0 || buffer.overflow()) {
> > > + LOG(IPADataSerializer, Error) << "Failed to serialize ControlList's ControlInfoMap";
> > > + return {{}, {}};
> > > + }
> > > +
> > > + size = cs->binarySize(data);
> > > + std::vector<uint8_t> listData(size);
> > > + buffer = ByteStreamBuffer(listData.data(), listData.size());
> > > + ret = cs->serialize(data, buffer);
> > > +
> > > + if (ret < 0 || buffer.overflow()) {
> > > + LOG(IPADataSerializer, Error) << "Failed to serialize ControlList";
> > > + return {{}, {}};
> > > + }
> > > +
> > > + std::vector<uint8_t> data_vec;
> >
> > You know the sizes and can reserve ?
>
> At this point. yes.
>
> > > + appendUInt<uint32_t>(data_vec, infoData.size());
> > > + appendUInt<uint32_t>(data_vec, listData.size());
> > > + data_vec.insert(data_vec.end(), infoData.begin(), infoData.end());
> > > + data_vec.insert(data_vec.end(), listData.begin(), listData.end());
> > > +
> > > + return {data_vec, {}};
> > > + }
> > > +
> > > + static const ControlList deserialize(std::vector<uint8_t> &data, ControlSerializer *cs)
> > > + {
> > > + return IPADataSerializer<const ControlList>::deserialize(data.begin(), data.end(), cs);
> > > + }
> > > +
> > > + static const ControlList deserialize(std::vector<uint8_t>::iterator it1,
> > > + [[maybe_unused]] std::vector<uint8_t>::iterator it2,
> > > + ControlSerializer *cs)
> > > + {
> > > + if (!cs)
> > > + LOG(IPADataSerializer, Fatal)
> > > + << "ControlSerializer not provided for deserialization of ControlList";
> > > +
> > > + uint32_t infoDataSize = readUInt<uint32_t>(it1);
> > > + uint32_t listDataSize = readUInt<uint32_t>(it1 + 4);
> > > +
> > > + std::vector<uint8_t>::iterator it = it1 + 8;
> > > +
> > > + std::vector<uint8_t> infoData(it, it + infoDataSize);
> > > + std::vector<uint8_t> listData(it + infoDataSize, it + infoDataSize + listDataSize);
> > > +
> > > + ByteStreamBuffer buffer(const_cast<const uint8_t *>(infoData.data()), infoData.size());
> > > + ControlInfoMap map = cs->deserialize<ControlInfoMap>(buffer);
> > > + /* It's fine if map is empty. */
> > > + if (buffer.overflow()) {
> > > + LOG(IPADataSerializer, Error)
> > > + << "Failed to deserialize ControlLists's ControlInfoMap: buffer overflow";
> > > + return ControlList();
> > > + }
> > > +
> > > + buffer = ByteStreamBuffer(const_cast<const uint8_t *>(listData.data()), listData.size());
> > > + ControlList list = cs->deserialize<ControlList>(buffer);
> > > + if (buffer.overflow())
> > > + LOG(IPADataSerializer, Error) << "Failed to deserialize ControlList: buffer overflow";
> > > + if (list.empty())
> > > + LOG(IPADataSerializer, Error) << "Failed to deserialize ControlList: empty list";
> >
> > Should we return {} in these cases or is it fine ?
>
> ControlSerializer::deserialize() returns {} in these cases anyway, so it
> should be fine.
>
> > > +
> > > + return list;
> > > + }
> > > +
> > > + static const ControlList deserialize(std::vector<uint8_t> &data,
> > > + [[maybe_unused]] std::vector<int32_t> &fds,
> > > + ControlSerializer *cs)
> > > + {
> > > + return IPADataSerializer<const ControlList>::deserialize(data.begin(), data.end(), cs);
> > > + }
> > > +
> > > + static const ControlList deserialize(std::vector<uint8_t>::iterator data_it1,
> > > + std::vector<uint8_t>::iterator data_it2,
> > > + [[maybe_unused]] std::vector<int32_t>::iterator fds_it1,
> > > + [[maybe_unused]] std::vector<int32_t>::iterator fds_it2,
> > > + ControlSerializer *cs)
> > > + {
> > > + return IPADataSerializer<const ControlList>::deserialize(data_it1, data_it2, cs);
> > > + }
> > > +};
> > > +
> > > +template<>
> > > +class IPADataSerializer<ControlList>
> > > +{
> > > +public:
> > > + static std::tuple<std::vector<uint8_t>, std::vector<int32_t>>
> > > + serialize(const ControlList &data, const ControlInfoMap &map,
> > > + ControlSerializer *cs)
> > > + {
> > > + const ControlList &list_const = const_cast<const ControlList &>(data);
> > > +
> > > + std::vector<uint8_t> data_vec;
> > > + std::tie(data_vec, std::ignore) =
> > > + IPADataSerializer<const ControlList>::serialize(list_const, map, cs);
> > > +
> > > + return {data_vec, {}};
> > > + }
> > > +
> > > + static ControlList deserialize(std::vector<uint8_t> &data,
> > > + ControlSerializer *cs)
> > > + {
> > > + ControlList ret;
> > > + const ControlList &list = ret;
> > > + const_cast<ControlList &>(list) =
> > > + IPADataSerializer<const ControlList>::deserialize(data, cs);
> > > +
> > > + return ret;
> > > + }
> > > +
> > > + static ControlList deserialize(std::vector<uint8_t>::iterator it1,
> > > + [[maybe_unused]] std::vector<uint8_t>::iterator it2,
> > > + ControlSerializer *cs)
> > > + {
> > > + ControlList ret;
> > > + const ControlList &list = ret;
> > > + const_cast<ControlList &>(list) =
> > > + IPADataSerializer<const ControlList>::deserialize(it1, it2, cs);
> >
> > Why not
> > const ControlList list =
> > IPADataSerializer<const ControlList>::deserialize(it1, it2, cs);
> > return const_cast<ControlList>(list)
>
> My understanding is that to remove a const with const_cast, the original
> variable must not be const.
You're right, this won't work, cast should happen on a pointer or
reference.
This works with gcc and clang:
const ControlList &list =
IPADataSerializer<const ControlList>::deserialize(data, cs);
return const_cast<ControlList &>(list);
As the function returns by value it should be ok.
Tests pass as well, but I'm not sure if this case is tested though.
>
> > > +
> > > + return ret;
> > > + }
> > > +
> > > + */
[snip]
> > > +
> > > +/**
> > > + * \fn template<typename T> T readUInt(std::vector<uint8_t> &vec, size_t pos)
> > > + * \brief Read uint from byte vector, in little-endian order
> > > + * \tparam T Type of uint to read
> > > + * \param[in] vec Byte vector to read from
> > > + * \param[in] pos Index in vec to start reading from
> > > + *
> >
> > No need for an empty line if no additional description is provided
>
> ack
>
> > > + * \return The uint read from \a vec, or 0 if reading goes past end of \a vec
> >
> > I wonder if we should not pass a T * as an output parameter and return
> > an error code. Returning 0 makes it impossible to detect failure, as 0
> > is a valid value
>
> I was actually wondering about that since when I wrote it in the first
> place. This might be a good solution.
Up to you. Do you check for the return value or do you need to do so ?
>
> > > + */
> > > +
> > > +/**
> > > + * \fn template<typename T> T readUInt(std::vector<uint8_t>::iterator it)
> > > + * \brief Read uint from byte vector, in little-endian order
> > > + * \tparam T Type of uint to read
> > > + * \param[in] it Iterator of byte vector to read from
> > > + *
> >
> > No need to empty line
>
> ack
>
> > > + * \return The uint read from \a vec
> > > + */
> > > +
> > > +/**
> > > + * \fn template<typename T> IPADataSerializer<T>::serialize(
> > > + * T data,
> > > + * ControlSerializer *cs = nullptr)
> > > + * \brief Serialize an object into byte vector and fd vector
> > > + * \tparam T Type of object to serialize
> > > + * \param[in] data Object to serialize
> > > + * \param[in] cs ControlSerializer
> > > + *
> > > + * \a cs is only necessary if the object type \a T or its members contain
> > > + * ControlList or ControlInfoMap.
> > > + *
> > > + * \return Tuple of byte vector and fd vector, that is the serialized form
> > > + * of \a data
> >
> > Maybe later, but I think for each template specialization, we want to
> > document the serialization format, in example that for vectors, the
> > length is the first 4 bytes and so on (even more for custom libcamera
> > data types)
>
> Good idea.
Although it won't be parsed by doxygen and will remain in-code only.
But knowing how each container will be serialized might help
validating it.
Actually, how does it work for closed source IPAs ? Before this series
the pipeline handler and the IPA were in charge of
serializing/deserializing with custom code, as the close source blob,
possibly written in C, wouldn't be able to link against libcamera and
use its helpers (better, they might chose not to, as from a compliance
pov it's fine linking a blob against L-GPL code). Is it still possible
? In that case, documenting how things gets serialized might be
even more important.
Thanks
j
More information about the libcamera-devel
mailing list