[libcamera-devel] Custom automatic image processing

[Laurent Pinchart]

Hi Marco,

On Wed, Apr 29, 2020 at 04:49:51PM +0200, Marco Felsch wrote:
> Hi all,
> 
> first of all I'm very new to this project. I noticed the project since
> the ELCE2018 but never got intouch with it until now. A customer of us
> wants to implement a custom automatic image processing unit within the
> gstreamer pipeline. This isn't the right place for it and we proposed
> libcamera but I don't know how much work we need to implement it. Here
> are the facts:
>   - Platform: imx6

Which i.MX6 is that ? There are important variations in the camera
hardware between different SoCs in the i.MX6 family.

>   - Camera: AR0237 (I will send a driver in the next weeks)

Please make sure to support at least the following features in that
driver:

- .get_selection() with the V4L2_SEL_TGT_CROP_BOUNDS,
  V4L2_SEL_TGT_NATIVE_SIZE and V4L2_SEL_TGT_CROP targets
  (V4L2_SEL_TGT_NATIVE_SIZE isn't required today but we're considering
  switching from V4L2_SEL_TGT_CROP_BOUNDS to V4L2_SEL_TGT_NATIVE_SIZE)
- V4L2_CID_PIXEL_RATE in read-only mode
- V4L2_CID_HBLANK in read-only mode (read-write is fine too)

The driver should also support the following features, even if they're
not mandatory at the moment:

- Direct control of binning and cropping through the .set_selection()
  and .set_fmt() operations (no hardcoded list of modes with register
  lists)
- V4L2_CID_VBLANK in read-write mode

>   - Gstreamer as media framework

Thanks to Nicolas we have GStreamer support for libcamera :-)

>   - Custom embedded OS (no Android)
> 
> The sensor can embed sensor statistics into the frame at the beginning
> and the end. We need to extract those data, calc all necessary values
> (like gain, exposure, ...) and adjust the sensor using v4l2-ctrls if
> necessary.
> 
> Pls, can someone provide me some information what we can expect from the
> current master state. What is working? How much work would you expect
> for the above use-case?

I few questions before providing you with answers.

- Unless I'm mistaken, the i.MX6 doesn't support colour interpolation
  (de-bayering) in hardware, and the AR0237 can only produce raw Bayer
  data, right ?

- Is the i.MX6 you're using capable of capturing the embedded data and
  embedded statistics to a different buffer than the image data
  (splitting the incoming frame based on line numbers to different DMA
  engines) ?

-- 
Regards,

Laurent Pinchart