[libcamera-devel] [RFC PATCH 01/17] IPA: IPC: raspberrypi: Add data definition and generated header
Paul Elder
paul.elder at ideasonboard.com
Wed Aug 26 13:09:10 CEST 2020
This patch shows what the data definition for the IPA interface for
raspberrypi might look like, as well as the corresponding generated
header, with the data definition file as the source. This first patch is
meant to show what the pipeline/IPA author would have to write/agree
upon, and what the corresponding generated header would look like.
The data defition file, as seen in raspberrypi.mojom, uses the mojo
syntax [1]. To define the IPA interface, functions must be defined as
shown in interface RPiIPAInterface. These functions are for the pipeline
handler to call. For functions that the IPA calls, via signals, are
defined in interface RPiIPACallbackInterface (we'll probably pick up
which interface is for which by magic names and extract the pipeline/IPA
name).
Any functions and any data structures can be used. Data structures that
that are specific to the pipeline/IPA must be defined in the same file
(eg. RPiConfigureParams, RPiEventParams, RPiActionParams, etc). Any
other data structures (eg. IPAStream, IPABuffer, ControlInfoMap) must
have de/serializers implemented. I have implemented de/serializers for a
few data structures that I think are important.
There are some rules to the interface definition:
- any functions that the pipeline calls after start() must not return
anything (eg. processEvent)
- any functions that the IPA calls, ever, must not return anything (eg.
queueFrameAction)
- any input values, if not POD, will have const and reference in the
generated function prototype
- if there is only one output and it is a POD, then it will be a simple
return value. Otherwise the output(s) will be via pointer output
parameter(s).
- maps and arrays are allowed (they will be translated to std::map and
std::vector, respectively)
raspberrypi_wrapper.h shows the generated header. For this RFC, it is
hand-generated by me, but I'll make a compiler to automatically generate
it, given the mojo data definition file. I think this generated file is
fairly straightforward. It contains the structs and enums as defined in
the mojo file. This header is meant to be included by pipelines and IPAs
so that they can know how to access the data structures passed around in
the IPA interface. It will also be used by the generated IPA proxy to
figure out how to de/serialize the nested structures.
In raspberrypi.h we first move the enum away into the mojo definition
and the generated header. The last hunk is meant to be generated by the
compiler as well, though I'm not sure how best to fit it in, since the
rest of this header is hand-written. RPiCMD will be generated based on
the IPA interface defined in the mojo file. Note that both the main
functions and the callbacks share the same enumspace. The
IPARPiInterface, which implements and IPAInterface, is defined as well,
also based on the IPA interface defined in the mojo file. It is a
virtual class, since both IPAProxyRPi and IPARPi must implement it since
that's how the stack will work:
pipeline handler -> IPAProxyRPi --thread--> IPARPi
\
\-> IPAIPC --IPA--> IPAProxyRPiWorker -> IPARPi
Signed-off-by: Paul Elder <paul.elder at ideasonboard.com>
---
include/libcamera/ipa/raspberrypi.h | 61 ++++++---
include/libcamera/ipa/raspberrypi.mojom | 97 ++++++++++++++
include/libcamera/ipa/raspberrypi_wrapper.h | 136 ++++++++++++++++++++
3 files changed, 276 insertions(+), 18 deletions(-)
create mode 100644 include/libcamera/ipa/raspberrypi.mojom
create mode 100644 include/libcamera/ipa/raspberrypi_wrapper.h
diff --git a/include/libcamera/ipa/raspberrypi.h b/include/libcamera/ipa/raspberrypi.h
index ca62990e..fa7efdbc 100644
--- a/include/libcamera/ipa/raspberrypi.h
+++ b/include/libcamera/ipa/raspberrypi.h
@@ -9,24 +9,7 @@
#include <libcamera/control_ids.h>
#include <libcamera/controls.h>
-
-enum RPiConfigParameters {
- RPI_IPA_CONFIG_LS_TABLE = (1 << 0),
- RPI_IPA_CONFIG_STAGGERED_WRITE = (1 << 1),
- RPI_IPA_CONFIG_SENSOR = (1 << 2),
-};
-
-enum RPiOperations {
- RPI_IPA_ACTION_V4L2_SET_STAGGERED = 1,
- RPI_IPA_ACTION_V4L2_SET_ISP,
- RPI_IPA_ACTION_STATS_METADATA_COMPLETE,
- RPI_IPA_ACTION_RUN_ISP,
- RPI_IPA_ACTION_RUN_ISP_AND_DROP_FRAME,
- RPI_IPA_ACTION_EMBEDDED_COMPLETE,
- RPI_IPA_EVENT_SIGNAL_STAT_READY,
- RPI_IPA_EVENT_SIGNAL_ISP_PREPARE,
- RPI_IPA_EVENT_QUEUE_REQUEST,
-};
+#include <libcamera/ipa/ipa_interface.h>
enum RPiIpaMask {
ID = 0x0ffff,
@@ -40,6 +23,10 @@ enum RPiIpaMask {
namespace libcamera {
+struct RPiConfigureParams;
+struct RPiEventParams;
+struct RPiActionParams;
+
/* List of controls handled by the Raspberry Pi IPA */
static const ControlInfoMap RPiControls = {
{ &controls::AeEnable, ControlInfo(false, true) },
@@ -59,6 +46,44 @@ static const ControlInfoMap RPiControls = {
{ &controls::ColourCorrectionMatrix, ControlInfo(-16.0f, 16.0f) },
};
+// TODO this needs to be automatically generated too
+enum RPiCMD {
+ CMD_INIT = 1,
+ CMD_EXIT,
+ CMD_START,
+ CMD_STOP,
+ CMD_CONFIGURE,
+ CMD_MAPBUFFERS,
+ CMD_UNMAPBUFFERS,
+ CMD_PROCESSEVENT,
+ CMD_QUEUEFRAMEACTION,
+};
+
+// This needs to be here so that the rpi proxy and worker can include it
+// Still needs to inherit from IPAInterface for IPAModule to work
+// TODO This needs to be automatically generated... so put in a different
+// header too, probably
+class IPARPiInterface : public IPAInterface
+{
+public:
+ virtual ~IPARPiInterface() {};
+
+ virtual int init(const IPASettings &settings) = 0;
+ virtual int start() = 0;
+ virtual void stop() = 0;
+
+ virtual void configure(const CameraSensorInfo &sensorInfo,
+ const std::map<unsigned int, IPAStream> &streamConfig,
+ const std::map<unsigned int, const ControlInfoMap> &entityControls,
+ const RPiConfigureParams &data,
+ RPiConfigureParams *response) = 0;
+ virtual void mapBuffers(const std::vector<IPABuffer> &buffers) = 0;
+ virtual void unmapBuffers(const std::vector<unsigned int> &ids) = 0;
+ virtual void processEvent(const RPiEventParams &event) = 0;
+
+ Signal<unsigned int, const RPiActionParams &> queueFrameAction;
+};
+
} /* namespace libcamera */
#endif /* __LIBCAMERA_IPA_INTERFACE_RASPBERRYPI_H__ */
diff --git a/include/libcamera/ipa/raspberrypi.mojom b/include/libcamera/ipa/raspberrypi.mojom
new file mode 100644
index 00000000..3baceb37
--- /dev/null
+++ b/include/libcamera/ipa/raspberrypi.mojom
@@ -0,0 +1,97 @@
+// automatically add const and & to all inputs (that are not primitives)
+// automatically add pointer to all outputs (that are not primitives)
+// if return value is single and primitive, then return as ret value
+// if return value is multiple or non-primitive, then return as output param
+interface RPiIPAInterface {
+ init(IPASettings settings) => (int32);
+ start() => (int32);
+ stop() => ();
+
+ configure(CameraSensorInfo sensorInfo,
+ map<uint32, IPAStream> streamConfig,
+ map<uint32, ControlInfoMap> entityControls,
+ RPiConfigureParams ipaConfig)
+ => (RPiConfigureParams result);
+
+ // arrays get turned into vectors
+ mapBuffers(array<IPABuffer> buffers) => (int32);
+ unmapBuffers(array<uint32> ids) => ();
+
+ processEvent(RPiEventParams data) => ();
+}
+
+// these shall not return anything
+interface RPiIPACallbackInterface {
+ queueFrameAction(uint32, RPiActionPayload) => ();
+}
+
+enum RPiConfigParameters {
+ RPI_IPA_CONFIG_LS_TABLE,
+ RPI_IPA_CONFIG_STAGGERED_WRITE,
+ RPI_IPA_CONFIG_SENSOR,
+ RPI_IPA_CONFIG_SEND_FD,
+};
+
+enum RPiEvents {
+ RPI_IPA_EVENT_SIGNAL_STAT_READY,
+ RPI_IPA_EVENT_SIGNAL_ISP_PREPARE,
+ RPI_IPA_EVENT_QUEUE_REQUEST,
+ RPI_IPA_EVENT_SEND_FD,
+};
+
+enum RPiActions {
+ RPI_IPA_ACTION_V4L2_SET_STAGGERED,
+ RPI_IPA_ACTION_V4L2_SET_ISP,
+ RPI_IPA_ACTION_STATS_METADATA_COMPLETE,
+ RPI_IPA_ACTION_RUN_ISP,
+ RPI_IPA_ACTION_RUN_ISP_AND_DROP_FRAME,
+ RPI_IPA_ACTION_EMBEDDED_COMPLETE,
+};
+
+// Custom Data containers
+
+struct RPiStaggeredWritePayload {
+ uint32 gainDelay;
+ uint32 exposureDelay;
+ uint32 sensorMetadata;
+};
+
+struct RPiIspPreparePayload {
+ uint32 embeddedbufferId;
+ uint32 bayerbufferId;
+};
+
+struct RPiStatsCompletePayload {
+ uint32 bufferId;
+ ControlList controls;
+};
+
+struct RPiConfigurePayload {
+ RPiConfigParameters op;
+ FileDescriptor lsTableHandle;
+ int32 lsTableHandleStatic;
+ RPiStaggeredWritePayload staggeredWriteResult;
+ ControlList controls;
+ int32 bufferFd;
+};
+
+// First level payload
+
+struct RPiConfigureParams {
+ array<RPiConfigurePayload> payload;
+};
+
+struct RPiEventParams {
+ RPiEvents ev;
+ uint32 bufferId;
+ RPiIspPreparePayload ispPrepare;
+ ControlList controls;
+ int32 bufferFd;
+};
+
+struct RPiActionParams {
+ RPiActions op;
+ uint32 bufferId;
+ RPiStatsCompletePayload statsComplete;
+ ControlList controls;
+};
diff --git a/include/libcamera/ipa/raspberrypi_wrapper.h b/include/libcamera/ipa/raspberrypi_wrapper.h
new file mode 100644
index 00000000..ba44526d
--- /dev/null
+++ b/include/libcamera/ipa/raspberrypi_wrapper.h
@@ -0,0 +1,136 @@
+// automatically generated by custom compiler
+
+#ifndef __LIBCAMERA_IPA_INTERFACE_RASPBERRYPI_FB_H__
+#define __LIBCAMERA_IPA_INTERFACE_RASPBERRYPI_FB_H__
+
+#include "libcamera/internal/byte_stream_buffer.h"
+#include "libcamera/internal/control_serializer.h"
+#include "libcamera/internal/ipa_data_serializer.h"
+
+#include <libcamera/ipa/ipa_interface.h>
+#include <libcamera/ipa/raspberrypi.h>
+
+#include <deque>
+#include <vector>
+
+namespace libcamera {
+
+enum RPiConfigParameters {
+ RPI_IPA_CONFIG_LS_TABLE = 1,
+ RPI_IPA_CONFIG_STAGGERED_WRITE = 2,
+ RPI_IPA_CONFIG_SENSOR = 3,
+ RPI_IPA_CONFIG_SEND_FD = 4,
+};
+
+enum RPiEvents {
+ RPI_IPA_EVENT_SIGNAL_STAT_READY = 1,
+ RPI_IPA_EVENT_SIGNAL_ISP_PREPARE = 2,
+ RPI_IPA_EVENT_QUEUE_REQUEST = 3,
+ RPI_IPA_EVENT_SEND_FD = 4,
+};
+
+enum RPiActions {
+ RPI_IPA_ACTION_V4L2_SET_STAGGERED = 1,
+ RPI_IPA_ACTION_V4L2_SET_ISP = 2,
+ RPI_IPA_ACTION_STATS_METADATA_COMPLETE = 3,
+ RPI_IPA_ACTION_RUN_ISP = 4,
+ RPI_IPA_ACTION_RUN_ISP_AND_DROP_FRAME = 5,
+ RPI_IPA_ACTION_EMBEDDED_COMPLETE = 6,
+};
+
+struct RPiStaggeredWritePayload
+{
+public:
+ RPiStaggeredWritePayload() : gainDelay_(0), exposureDelay_(0), sensorMetadata_(0) {}
+ ~RPiStaggeredWritePayload() {}
+
+ RPiStaggeredWritePayload(uint32_t gainDelay, uint32_t exposureDelay, uint32_t sensorMetadata) : gainDelay_(gainDelay), exposureDelay_(exposureDelay), sensorMetadata_(sensorMetadata) {}
+
+ uint32_t gainDelay_;
+ uint32_t exposureDelay_;
+ uint32_t sensorMetadata_;
+};
+
+struct RPiIspPreparePayload
+{
+public:
+ RPiIspPreparePayload() : embeddedbufferId_(0), bayerbufferId_(0) {}
+ ~RPiIspPreparePayload() {}
+
+ RPiIspPreparePayload(uint32_t embeddedbufferId, uint32_t bayerbufferId) : embeddedbufferId_(embeddedbufferId), bayerbufferId_(bayerbufferId) {}
+
+ uint32_t embeddedbufferId_;
+ uint32_t bayerbufferId_;
+};
+
+struct RPiStatsCompletePayload
+{
+public:
+ RPiStatsCompletePayload() : bufferId_(0) {}
+ ~RPiStatsCompletePayload() {}
+
+ RPiStatsCompletePayload(uint32_t bufferId, ControlList &controls) : bufferId_(bufferId), controls_(controls) {}
+
+ uint32_t bufferId_;
+ ControlList controls_;
+};
+
+struct RPiConfigurePayload
+{
+public:
+ RPiConfigurePayload() : op_(static_cast<RPiConfigParameters>(0)), lsTableHandle_(-1), bufferFd_(0) {}
+ ~RPiConfigurePayload() {}
+
+ RPiConfigurePayload(enum RPiConfigParameters op, uint32_t lsTableHandle, uint32_t lsTableHandleStatic, RPiStaggeredWritePayload staggeredWriteResult, ControlList controls, int32_t bufferFd) : op_(op), lsTableHandle_(lsTableHandle), lsTableHandleStatic_(lsTableHandleStatic), staggeredWriteResult_(staggeredWriteResult), controls_(controls), bufferFd_(bufferFd) {}
+
+ enum RPiConfigParameters op_;
+ FileDescriptor lsTableHandle_;
+ int32_t lsTableHandleStatic_;
+ RPiStaggeredWritePayload staggeredWriteResult_;
+ ControlList controls_;
+ int32_t bufferFd_;
+};
+
+struct RPiConfigureParams
+{
+public:
+ RPiConfigureParams() {}
+ ~RPiConfigureParams() {}
+
+ RPiConfigureParams(std::vector<RPiConfigurePayload> payload) : payload_(payload) {}
+
+ std::vector<RPiConfigurePayload> payload_;
+};
+
+struct RPiEventParams
+{
+public:
+ RPiEventParams() : ev_(static_cast<RPiEvents>(0)), bufferId_(0), bufferFd_(0) {}
+ ~RPiEventParams() {}
+
+ RPiEventParams(enum RPiEvents ev, uint32_t bufferId, RPiIspPreparePayload ispPrepare, ControlList controls, int32_t bufferFd) : ev_(ev), bufferId_(bufferId), ispPrepare_(ispPrepare), controls_(controls), bufferFd_(bufferFd) {}
+
+ enum RPiEvents ev_;
+ uint32_t bufferId_;
+ RPiIspPreparePayload ispPrepare_;
+ ControlList controls_;
+ int32_t bufferFd_;
+};
+
+struct RPiActionParams
+{
+public:
+ RPiActionParams() : op_(static_cast<RPiActions>(0)), bufferId_(0) {}
+ ~RPiActionParams() {}
+
+ RPiActionParams(enum RPiActions op, uint32_t bufferId, RPiStatsCompletePayload statsComplete, ControlList controls) : op_(op), bufferId_(bufferId), statsComplete_(statsComplete), controls_(controls) {}
+
+ enum RPiActions op_;
+ uint32_t bufferId_;
+ RPiStatsCompletePayload statsComplete_;
+ ControlList controls_;
+};
+
+} /* namespace libcamera */
+
+#endif /* __LIBCAMERA_IPA_INTERFACE_RASPBERRYPI_FB_H__ */
--
2.27.0
More information about the libcamera-devel
mailing list