[libcamera-devel] [PATCH v2 0/3] raspberrypi: FPS control

[Laurent Pinchart]

Hi Nicolas,

On Mon, May 25, 2020 at 08:04:03AM -0400, Nicolas Dufresne wrote:
> Le mercredi 13 mai 2020 à 10:11 +0100, Naushir Patuck a écrit :
> > Hi,
> > 
> > This is the v2 patchset for framerate control on the Raspberry Pi platform.
> > 
> > There are quite substantial updates from the v1 patch:
> > - Added min/max frame duration controls to libcamera.
> 
> The cover letter says FPS control, this patch adds frame duration
> control. Did we chose frame duration simply because V4L2 works that
> way?

No, V4L2 wasn't considered to make this decision.

> What do we want to standard on? How is sensor programmed in this
> regard?

In most cases, sensors are programmed with a pixel rate (which directly
derives from the input clock), an analog crop rectangle and horizontal
and vertical blanking. The frame rate is equal to the pixel rate divided
by the product of the total vertical and total horizontal sizes.

"Smart" sensors integrate an ISP and a microcontroller that runs an
auto-exposure algorithm. The microcode configures the blanking values,
based on higher-level parameters such as a target frame rate or minimum
and maximum frame rates (sometimes with a step). How those frame rates
are expressed is sensor-specific, some sensors use frame durations, some
sensors use frame rates expressed as a fraction (sometimes with
constraints on the numerator and denominator, possible with the
numerator being fixed to 1).

The V4L2 subdev API has an operation to set the frame rate as a fraction
(this comes from TV capture devices), and an API to configure the pixel
rate and blanking (though V4L2 controls). The former is abused by sensor
drivers that should really use the latter, and I've called for the
V4L2 FPS-based API to be deprecated for camera sensors in most cases.

> What is the most natural configuration? An FPS fraction, or a
> frame duration ?

See
https://lists.libcamera.org/pipermail/libcamera-devel/2020-May/008741.html
for discussions on this topic.

> > - Control vblanking for both min and max frame duration.
> > - Moved GetVBlanking() into the CamHelper base class.
> > - Updated tunings for the imx219, imx477 and ov5647 sensors to limit maximum exposure times.
> > 
> > Thanks,
> > Naush
> > 
> > Naushir Patuck (3):
> >   libcamera: controls: Add frame duration control
> >   libcamera: raspberrypi: Add control of sensor vblanking
> >   ipa: raspberrypi: config: Update shutter speeds for imx219/477 and
> >     ov5647
> > 
> >  include/ipa/raspberrypi.h                     |  1 +
> >  src/ipa/raspberrypi/cam_helper.cpp            | 38 ++++++++++++++-
> >  src/ipa/raspberrypi/cam_helper.hpp            | 15 +++++-
> >  src/ipa/raspberrypi/cam_helper_imx219.cpp     | 11 ++++-
> >  src/ipa/raspberrypi/cam_helper_imx477.cpp     | 11 ++++-
> >  src/ipa/raspberrypi/cam_helper_ov5647.cpp     | 11 ++++-
> >  src/ipa/raspberrypi/data/imx219.json          | 15 +++++-
> >  src/ipa/raspberrypi/data/imx477.json          | 15 +++++-
> >  src/ipa/raspberrypi/data/ov5647.json          | 15 +++++-
> >  src/ipa/raspberrypi/raspberrypi.cpp           | 48 ++++++++++++++++---
> >  src/libcamera/control_ids.yaml                | 14 ++++++
> >  .../pipeline/raspberrypi/raspberrypi.cpp      |  2 +
> >  12 files changed, 177 insertions(+), 19 deletions(-)

-- 
Regards,

Laurent Pinchart