[libcamera-devel] libcamera transform control

[David Plowman]

Replying to my own post to add comments from Naush and Jacopo!

David

On Mon, 6 Jul 2020 at 14:32, David Plowman
<david.plowman at raspberrypi.com> wrote:
>
> Hi everyone
>
> This email takes the previous discussion of image transformations and
> moves it into a thread of its own.
>
> The problem at hand is how to specify the 2D planar transformation to
> be applied to a camera image. These are well known to us as flips,
> rotations and so on (the usual eight members of dihedral group D4).
>
> Laurent's suggestion was to apply them as standard libcamera controls.
> Some platforms may be able to apply them on a per-frame basis; others
> will not (they will require the transform to be set before the camera
> is started - once this mechanism exists). (Is there anyway to signal a
> platform's capabilities in this regard?)
>
> For the time being we are ignoring the possibility that some platforms
> might be able to apply different transformations to different streams
> (I believe there is in any case no mechanism for per-stream controls).
>
> Note also that raw streams will always have the orientation with which
> they come out of the camera, though again I don't believe we have a
> convenient way for a platform to indicate whether this includes the
> transform or not  (perhaps a stream could indicate its transform
> relative to the requested transform?).
>
> We propose to represent the transforms as "int" controls, in fact
> being members of an enum with eight entries. Further, we suggest that
> the first four entries are "identity", "hflip", "vflip", "h+vflip",
> meaning the control's maximum value can indicate whether transforms
> that involve a transposition are excluded.

> Naush:
>
> ... the pipeline handlers need to maintain a list of
> controls supported, like in include/libcamera/ipa/raspberrypi.h
> (RPiControls).  This then gets exported to the CameraData base class
> member controlInfo_.  ARequest will not allow setting a
> libcamera::control that is not listed in CameraData::controlInfo_

Jacopo:

That's correct, it seems to me from this discussion, a ControlInfo
could be augmented with information about a control being supported
per-frame or just at at configuration time (or else...)

> Notes on the Raspberry Pi implementation:
> Only transpose-less transforms will be allowed, and only before the
> camera is started. We will support them by setting the corresponding
> control bits in the camera driver (so raw streams will include the
> transform). This means the ALSC (Auto Lens Shading) algorithm will
> have to know whether camera images are oriented differently from the
> calibration images.
>
> Thoughts?
>
> Best regards
> David