[libcamera-devel] "Software ISP" -- where to put it, neccessary hooks

[Laurent Pinchart]

Hello,

On Wed, Jul 05, 2023 at 06:05:55PM +0100, Kieran Bingham wrote:
> Quoting Pavel Machek via libcamera-devel (2023-06-30 14:25:18)
> > Hi!
> > 
> > I'm looking for guidance. I have software-only auto-exposure code, but
> > it is in wrong places. Code is at
> > 
> > https://gitlab.com/tui/libcamera/-/tree/hacks-ae?ref_type=heads
> > 
> > I tried to move the code to src/ipa/libipa/simple_ae.cpp , but I could
> > not get the result to link if I did that.
> > 
> > Currently code is called from SimpleCameraData::bufferReady , is that
> > okay thing to do?
> 
> So the tricky thing here is that you are trying to call libipa from a
> pipeline handler - that's not quite right.
> 
> So we likely need to determine if the simple-pipeline handler should
> (optionally?) instantiate an IPA module which can then incorporate the
> software ISP implementation. I expect this would only instantiate and
> use the IPA when a RAW sensor is connected.

That's what I would recommend too. We should have a soft ISP component
for the pipeline handler to instantiate, with a corresponding IPA
module. The interface of the soft ISP should be generic, defined as an
abstract class with pure virtual functions, with a derived class that
implements a CPU-based soft ISP. Later, we'll develop a GPU-based soft
ISP as a second implementation of the interface, and the simple pipeline
handler will be able to use either.

> I suspect we would have a new IPA under src/ipa/softisp/ which would do
> the implementation of 3a using a CPU/GPU depending on resource
> availability.

The code in src/ipa/softisp/ should only control the ISP algorithms, not
implement the CPU-based image processing. Ideally that IPA module would
be used with either the CPU- and GPU-based implementations on the
pipeline handler side.

> From there any helpers that might be shared across IPA modules could
> live in libipa, but anything that is specific to the new 'softisp' would
> be under src/ipa/softisp/ ... so you would have
> src/ipa/softisp/algorithms/cpu_ae.cpp perhaps.
> 
> > My divideUpExposure code has origins in rapsberry pi code; I could not
> > find a way to share it nicely so I cut&pasted. If there's a better way
> > (or better code to reuse), I'd like to know.
> 
> I believe code reuse from the RPi code is fine, but you would need to
> ensure that the new code file matches the RPi IPA licence which is BSD I
> believe.

-- 
Regards,

Laurent Pinchart