[libcamera-devel] Debayering

[Laurent Pinchart]

Hi Dorota,

On Sat, Nov 13, 2021 at 11:56:04AM +0100, Dorota Czaplejewicz wrote:
> On Fri, 12 Nov 2021 18:25:55 +0900 paul.elder at ideasonboard.com wrote:
> > On Fri, Nov 12, 2021 at 02:09:43AM +0200, Laurent Pinchart wrote:
> > > On Thu, Nov 11, 2021 at 11:56:07AM +0000, Kieran Bingham wrote:  
> > > > Quoting Jacopo Mondi (2021-11-10 20:32:29)  
> > > > > On Wed, Nov 10, 2021 at 02:01:23PM +0100, Dorota Czaplejewicz wrote:  
> > > > > > Hi all,
> > > > > >
> > > > > > I'm making sure that Librem 5 cameras can be easily used via
> > > > > > libcamera. One of the steps that are needed for the "easily" part is
> > > > > > having a builtin debayering mechanism.
> > > > > >
> > > > > > Before I jump head-in, I wanted to ask for some guidance about where
> > > > > > to place this support, and how to expose it to the consumer.
> > > > > >
> > > > > > For a simplest implementation, supported cameras could add the RGB
> > > > > > format to the list of supported formats, and transparently convert
> > > > > > the bayer data to RGB if configured for RGB. That opens up a
> > > > > > question about the opt-in mechanism, but is otherwise simple enough.  
> > > > 
> > > > I think this should be 'opt in' only by the platform (not the 'camera'
> > > > or 'sensor'). To me that means it's a pipeline handler decision as to
>
> What is the definition of "platform", "camera" and "sensor" used here?
> Is platform=pipeline?

"Platform" more or less means the SoC type. It's roughly equivalent to
the pipeline handler, except for the simple pipeline handler that
supports multiple platforms. As some of them don't have a GPU, we can't
assume that all platforms supported by the simple pipeline handler would
use the GPU-based ISP.

"Sensor" is the imaging sensor (S5K3L6XX or YACG4D0C9SHC in your case).

"Camera" is the combination of an imaging sensor and all other related
devices (ISP, GPU-based ISP, ...) that together implement the ability to
capture and process frames.

> > > > whether it exposes a software/GPU ISP. (a CPU ISP should only be
> > > > considered a shortcut to GPU accelleration later I believe).
> > > > 
> > > > > > I intend to start by adding a CPU-based debayering implementation
> > > > > > for the sake of building it up quickly. Would a simple
> > > > > > implementation like I outlined above suitable for upstreaming?  
> > > > 
> > > > That would depend on how it's architectured. I don't think we'd like
> > > > 'quick hacks' in place internally to be merged.
>
> "Quick hacks" is not what I intend to merge. Rather, I found the
> libcamera codebase difficult to learn, and I don't see the point in

Is there anything in particular that would have made it easier ?

> trying to architect any significant part of the solution from the
> outset. I'd rather have a quick first step to confirm the sanity of
> the approach, and then iterate on that.

No disagreement there, it's best to work in an iterative manner instead
of spending a large amount of time designing something that won't fit in
the end.

> > > > Although it has an 'optional' SimpleConvertor so it's also this:
>
> I'll take a look at this.
>
> > The current interface design that Siyuan has has statistics and image
> > processing as separate components, so this should be supported
> > out-of-the-box. We'll have to solidify the requirements of the
> > interface, though, to mandate that soft ISP implementations make sure
> > that those two are separate and dont't have dependencies under-the-hood.
> 
> Is that design available anywhere?

The latest version I know of is available at
https://lists.libcamera.org/pipermail/libcamera-devel/2021-October/025580.html.
It's work in progress, and nothing is set in stone, so you can comment
on anything that doesn't feel right, or propose alternatives. Paul is
following the work more closely than I do.

> > > > > Can I ask how you have envisioned your capture pipeline ? I
> > > > > assume you have a raw sensor connected to a pipeline handler that has
> > > > > no ISP. What component would be driving the adjustment of the sensor
> > > > > parameters in such an architecture ?  
> 
> I don't have enough familiarity with libcamera for any concrete ideas,
> so I only made vague guesses before asking the list for guidance.
>
> Now that I received the guidance, I will start with something like the
> SimpleConverter (assuming that it does what I expect it to do), and
> not concern myself with parameter adjustment.
> 
> Once I dig in and have more questions, I'll send updates.

Looking forward to that :-)

-- 
Regards,

Laurent Pinchart