[libcamera-devel] Sharpness control proposal

[David Plowman]

Hi Laurent

Thanks for the comments.

On Tue, 16 Jun 2020 at 04:13, Laurent Pinchart
<laurent.pinchart at ideasonboard.com> wrote:
>
> Hi David,
>
> On Mon, Jun 15, 2020 at 12:20:15PM +0100, David Plowman wrote:
> > Hi
> >
> > We're busy trying to produce libcamera-based alternatives to existing
> > camera applications. One feature they quite often have is the ability to let
> > the user change the image sharpness.
> >
> > I note that V4L2 has a sharpness control, wherein the "minimum value"
> > means no sharpening, and "larger values increase the amount of
> > sharpening". Slightly in line with this, I'd like to suggest:
> >
> > A sharpening control taking a float value, where:
> > * A value of zero means no sharpening.
> > * 1.0 applies a "reasonable" default level of sharpening.
> > * We can stipulate some fairly arbitrary maximum.
>
> Do you think we need floating point values for this, or would an integer
> give enough precision ?

I would agree that a float is more precision than we need. On the other
hand, it saves us from defining something that might have less precision
than some future pipeline handler wants, and it also has a natural
interpretation - you don't have to think what the scale really is.

>
> > Beyond this, we could recommend that the value here is loosely
> > "proportional" to the amount of sharpening applied. (For the purposes
> > of this definition, we might refer to the amount of sharpening as
> > being the difference between a sharpened and unsharpened image.)
> >
> > What do people think? I feel there is some benefit in trying to be a
> > bit more prescriptive about what the control means; otherwise it
> > becomes more difficult for a cross-platform application to know how to
> > use it.
>
> Overall I think your proposal is good. I also would prefer being more
> prescriptive if possible. As libcamera has support for reporting default
> values, we may want to specify a fixed maximum value here, and let the
> default being device-specific. Or do you think we could specify both a
> fixed default (1.0) and a fixed maximum ?

So I imagine two places where the sharpness value gets interpreted.

Firstly, there's the application. This may present a GUI with a slider
returning (for example) values from 0 to 100, where 0 = no sharpening,
100 = some very large amount and maybe 50 = "default".

The application would have to transform this into the libcamera
parameter. It will have to make a decision on how the 0 to 50 range
maps onto 0.0 to 1.0 (or whatever), and how 50 to 100 maps onto
1.0 to "maximum".

Secondly the pipeline handler or IPA (or someone!) will turn this into
something the hardware understands, which is probably beyond our
concern.

So my feeling is that it wouldn't hurt to allow pipeline handlers to pick
their own default values and scale for the sharpness parameter, but
I don't really see that it's necessary - why push complexity (OK, only a
little complexity!) to the application if the pipeline handler can deal
with it anyway?

But certainly, even if an application has to look up default values before
generating the sharpness parameter that libcamera wants, it's hardly a
big deal, so I don't feel terribly strongly.

As regards a maximum value, we could certainly impose one. For
example, once you've reached, say, 4x the default, you've got crazy
amounts of sharpening, though there's never a theoretical limit as to
how high it could go.

It would be interesting if other pipeline handler / ISP developers had
any thoughts on the matter.

>
> How is this implemented in the Rasperry Pi case, do you use an unsharpen
> filter, or something different ?

Not too sure what I can say, but clearly there will be high pass
filters involved. These responses will get converted into a certain
amount of sharpening and added to the image. But you're right, an
unsharp mask is a way of making a high pass filter so it can be
thought of in these terms.

>
> --
> Regards,
>
> Laurent Pinchart