[libcamera-devel] Draft building an IPA module outline

[Chris Chinchilla]

Hey all,

I’d appreciate some input on the outline below for creating the “building an IPA module” guide. It’s fairly high level right now as I am lacking some of implementation detail (I mostly found looking at RKISP1 the most useful), and this is why I wanted to get feedback. In the meantime I will make a start on writing the steps for creating the file structure skeleton and adding it the build system etc, but any further insights or comments would be helpful. I’d especially appreciate some input on the specifics of “doing something” with an IPA module, which is the part lacking most right now.

The overall intention is to add this IPA module to the vivid pipeline handler example and manipulate the output somehow.

Some parts are also taken from current docs, I’ll add and reword in context of the guide.


Thanks, Chris

# Building an IPA module

## What is an IPA module

Generally written as part of the pipeline handler, the IPA module manager looks for a module matching the handler. You add matching modules by creating an appropriate folder in src/ipa, adding the source to the build system, and searching for a match in the corresponding pipeline handler.

## Module isolation

The IPA module manager discovers IPA modules from disk, queries and loads them, and creates IPA contexts. It supports isolation of the modules in a separate process with IPC communication and offers a unified IPAInterface view of the IPA contexts to pipeline handlers regardless of whether the modules are isolated or loaded in the same process.

Module isolation is based on the module licence. Open-source modules are loaded without isolation, while closed-source module are forcefully isolated. The isolation mechanism ensures that no code from a closed-source module is ever run in the libcamera process.

All interactions with the IPA context go the same interface regardless of process isolation. In all cases the data passed to the IPAInterface methods is serialized to Plain Old Data, either for the purpose of passing it to the IPA context plain C API, or to transmit the data to the isolated process through IPC.

## Creating a skeleton IPA module

• Create an instance of the IPAInterface base class
• create class member methods: init, start, stop etc
    • init - set values with config, can be in method or file (if so, add file to build)
    • start/stop - prepare/release resources, which are what?
    • configure - Called by Pipeline handler to configure the IPA stream and sensor settings.
    • map/unmap buffers - inform ipa module of the buffers shared from the pipeline handler
    • process - process by which pipeline handler informs a module of events in an on-going capture operation. The events you define in the pipeline handler
    • queueframeaction - Queue an action associated with a frame to the pipeline handler.
    • any private methods you need

## Create an IPA context

• Create in camera data during pipeline handler match process
• Start and stop IPA module generally when starting and stopping streams

## Manipulate output

Kieran and I discussed adding something very visual here, ramping gain and/or white balance very high etc, so it’s very clear what is happening.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.libcamera.org/pipermail/libcamera-devel/attachments/20200730/c1d3f3e0/attachment.htm>