[libcamera-devel] [PATCH v6] libcamera: properties: Define pixel array properties

Niklas Söderlund niklas.soderlund at ragnatech.se
Fri Jun 5 21:53:44 CEST 2020


Hi Jacopo,

Nice work!

On 2020-06-04 17:31:22 +0200, Jacopo Mondi wrote:
> Add definition of pixel array related properties.
> 
> Signed-off-by: Jacopo Mondi <jacopo at jmondi.org>

Reviewed-by: Niklas Söderlund <niklas.soderlund at ragnatech.se>

> ---
>  src/libcamera/property_ids.yaml | 263 ++++++++++++++++++++++++++++++++
>  1 file changed, 263 insertions(+)
> 
> diff --git a/src/libcamera/property_ids.yaml b/src/libcamera/property_ids.yaml
> index ce627fa042ba..762d60881568 100644
> --- a/src/libcamera/property_ids.yaml
> +++ b/src/libcamera/property_ids.yaml
> @@ -386,4 +386,267 @@ controls:
>                                |                    |
>                                |                    |
>                                +--------------------+
> +
> +  - UnitCellSize:
> +      type: Size
> +      description: |
> +        The pixel unit cell physical size, in nanometers.
> +
> +        The UnitCellSize properties defines the horizontal and vertical sizes
> +        of a single pixel unit, including its active and non-active parts.
> +
> +        The property can be used to calculate the physical size of the sensor's
> +        pixel array area and for calibration purposes.
> +
> +  - PixelArrayPhysicalSize:
> +      type: Size
> +      description: |
> +        The camera sensor full pixel array size, in nanometers.
> +
> +        The PixelArrayPhysicalSize property reports the physical dimensions
> +        (width x height) of the full pixel array matrix, including readable
> +        and non-readable pixels.
> +
> +        \todo Rename this property to PhysicalSize once we will have property
> +              categories (i.e. Properties::PixelArray::PhysicalSize)
> +
> +  - PixelArraySize:
> +      type: Size
> +      description: |
> +        The camera sensor pixel array readable area vertical and horizontal
> +        sizes, in pixels.
> +
> +        The PixelArraySize property defines the size in pixel units of the
> +        readable part of full pixel array matrix, including optically black
> +        pixels used for calibration, pixels which are not considered valid for
> +        capture and active pixels valid for image capture.
> +
> +        The property describes a rectangle whose top-left corner is placed
> +        in position (0, 0) and whose vertical and horizontal sizes are defined
> +        by the Size element transported by the property.
> +
> +        The property describes the maximum size of the raw data produced by
> +        the sensor, which might not correspond to the physical size of the
> +        sensor pixel array matrix, as some portions of the physical pixel
> +        array matrix are not accessible and cannot be transmitted out.
> +
> +        For example, a pixel array matrix assembled as follow
> +
> +             +--------------------------------------------------+
> +             |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx|
> +             |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx|
> +             |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx|
> +             |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx|
> +             |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx|
> +             |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx|
> +             |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx|
> +             |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx|
> +             ...          ...           ...      ...          ...
> +
> +             ...          ...           ...      ...          ...
> +             |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx|
> +             |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx|
> +             |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx|
> +             |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx|
> +             +--------------------------------------------------+
> +
> +        composed by two lines of non-readable pixels (x) followed by N lines of
> +        readable data (D) surrounded by two columns of non-readable pixels on
> +        each side, only the readable portion is transmitted to the receiving
> +        side, defining the sizes of the largest possible buffer of raw data
> +        that can be presented to applications.
> +
> +                               PixelArraySize[0]
> +               /----------------------------------------------/
> +               +----------------------------------------------+ /
> +               |DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD| |
> +               |DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD| |
> +               |DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD| |
> +               |DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD| |
> +               |DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD| |
> +               |DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD| | PixelArraySize[1]
> +               ...        ...           ...      ...        ...
> +               ...        ...           ...      ...        ...
> +               |DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD| |
> +               |DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD| |
> +               +----------------------------------------------+ /
> +
> +        All other rectangles that describes portions of the pixel array, such as
> +        the optical black pixels rectangles and active pixel areas are defined
> +        relatively to the rectangle described by this property.
> +
> +        \todo Rename this property to Size once we will have property
> +              categories (i.e. Properties::PixelArray::Size)
> +
> +  - PixelArrayOpticalBlackRectangles:
> +      type: Rectangle
> +      size: [1 x n]
> +      description: |
> +        The raw data buffer regions which contains optical black pixels
> +        considered valid for calibration purposes.
> +
> +        The PixelArrayOpticalBlackRectangles describes (possibly multiple)
> +        rectangular areas of the raw data buffer, where optical black pixels are
> +        located and could be accessed for calibration and black level
> +        correction.
> +
> +        This property describes the position and size of optically black pixel
> +        rectangles relatively to their position in the raw data buffer as stored
> +        in memory, which might differ from their actual physical location in the
> +        pixel array matrix.
> +
> +        It is important to note, in facts, that camera sensor might
> +        automatically re-order, shuffle or skip portions of their pixels array
> +        matrix when transmitting data to the receiver.
> +
> +        The pixel array contains several areas with different purposes,
> +        interleaved by lines and columns which are said not to be valid for
> +        capturing purposes. Invalid lines and columns are defined as invalid as
> +        they could be positioned too close to the chip margins or to the optical
> +        blank shielding placed on top of optical black pixels.
> +
> +                                PixelArraySize[0]
> +               /----------------------------------------------/
> +                  x1                                       x2
> +               +--o---------------------------------------o---+ /
> +               |IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII| |
> +               |IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII| |
> +            y1 oIIOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOII| |
> +               |IIOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOII| |
> +               |IIOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOII| |
> +            y2 oIIOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOII| |
> +               |IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII| |
> +               |IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII| |
> +            y3 |IIOOPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPOOII| |
> +               |IIOOPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPOOII| | PixelArraySize[1]
> +               |IIOOPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPOOII| |
> +               ...          ...           ...     ...       ...
> +               ...          ...           ...     ...       ...
> +            y4 |IIOOPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPOOII| |
> +               |IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII| |
> +               |IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII| |
> +               +----------------------------------------------+ /
> +
> +        The readable pixel array matrix is composed by
> +        2 invalid lines (I)
> +        4 lines of valid optically black pixels (O)
> +        2 invalid lines (I)
> +        n lines of valid pixel data (P)
> +        2 invalid lines (I)
> +
> +        And the position of the optical black pixel rectangles is defined by
> +
> +            PixelArrayOpticalBlackRectangles = {
> +               { x1, y1, x2 - x1 + 1, y2 - y1 + },
> +               { x1, y3, 2, y4 - y3 + 1 },
> +               { x2, y3, 2, y4 - y3 + 1 },
> +            };
> +
> +        If the sensor, when required to output the full pixel array matrix,
> +        automatically skip the invalid lines and columns, producing the
> +        following data buffer, when captured to memory
> +
> +                                    PixelArraySize[0]
> +               /----------------------------------------------/
> +                                                           x1
> +               +--------------------------------------------o-+ /
> +               |OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO| |
> +               |OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO| |
> +               |OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO| |
> +               |OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO| |
> +            y1 oOOPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPOO| |
> +               |OOPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPOO| |
> +               |OOPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPOO| | PixelArraySize[1]
> +               ...       ...          ...       ...         ... |
> +               ...       ...          ...       ...         ... |
> +               |OOPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPOO| |
> +               |OOPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPOO| |
> +               +----------------------------------------------+ /
> +
> +        The invalid lines and columns should not be reported as part of the
> +        PixelArraySize property in first place.
> +
> +        In this case, the position of the black pixel rectangles will be
> +
> +            PixelArrayOpticalBlackRectangles = {
> +               { 0, 0, y1 + 1, PixelArraySize[0] },
> +               { 0, y1, 2, PixelArraySize[1] - y1 + 1 },
> +               { x1, y1, 2, PixelArraySize[1] - y1 + 1 },
> +            };
> +
> +        \todo Rename this property to Size once we will have property
> +              categories (i.e. Properties::PixelArray::OpticalBlackRectangles)
> +
> +  - PixelArrayActiveAreas:
> +      type: Rectangle
> +      size: [1 x n]
> +      description: |
> +        The PixelArrayActiveAreas property defines the (possibly multiple and
> +        overlapping) portions of the camera sensor readable pixel matrix
> +        which are considered valid for image acquisition purposes.
> +
> +        Each rectangle is defined relatively to the PixelArraySize rectangle,
> +        with its top-left corner defined by its horizontal and vertical
> +        distances from the PixelArraySize rectangle top-left corner, placed in
> +        position (0, 0).
> +
> +        This property describes an arbitrary number of overlapping rectangles,
> +        with each rectangle representing the maximum image size that the camera
> +        sensor can produce for a particular aspect ratio.
> +
> +        When multiple rectangles are reported, they shall be ordered from the
> +        tallest to the shortest.
> +
> +        Example 1
> +        A camera sensor which only produces images in the 4:3 image resolution
> +        will report a single PixelArrayActiveAreas rectangle, from which all
> +        other image formats are obtained by either cropping the field-of-view
> +        and/or applying pixel sub-sampling techniques such as pixel skipping or
> +        binning.
> +
> +                     PixelArraySize[0]
> +                    /----------------/
> +                      x1          x2
> +            (0,0)-> +-o------------o-+  /
> +                 y1 o +------------+ |  |
> +                    | |////////////| |  |
> +                    | |////////////| |  | PixelArraySize[1]
> +                    | |////////////| |  |
> +                 y2 o +------------+ |  |
> +                    +----------------+  /
> +
> +        The property reports a single rectangle
> +
> +                 PixelArrayActiveAreas = (x1, y1, x2 - x1 + 1, y2 - y1 + 1)
> +
> +        Example 2
> +        A camera sensor which can produce images in different native
> +        resolutions will report several overlapping rectangles, one for each
> +        natively supported resolution.
> +
> +                     PixelArraySize[0]
> +                    /------------------/
> +                      x1  x2    x3  x4
> +            (0,0)-> +o---o------o---o+  /
> +                 y1 o    +------+    |  |
> +                    |    |//////|    |  |
> +                 y2 o+---+------+---+|  |
> +                    ||///|//////|///||  | PixelArraySize[1]
> +                 y3 o+---+------+---+|  |
> +                    |    |//////|    |  |
> +                 y4 o    +------+    |  |
> +                    +----+------+----+  /
> +
> +        The property reports two rectangles
> +
> +                PixelArrayActiveAreas = ((x2, y1, x3 - x2 + 1, y4 - y1 + 1),
> +                                         (x1, y2, x4 - x1 + 1, y3 - y2 + 1))
> +
> +        The first rectangle describes the maximum field-of-view of all image
> +        formats in the 4:3 resolutions, while the second one describes the
> +        maximum field of view for all image formats in the 16:9 resolutions.
> +
> +        \todo Rename this property to ActiveAreas once we will have property
> +              categories (i.e. Properties::PixelArray::ActiveAreas)
> +
>  ...
> --
> 2.26.2
> 
> _______________________________________________
> libcamera-devel mailing list
> libcamera-devel at lists.libcamera.org
> https://lists.libcamera.org/listinfo/libcamera-devel

-- 
Regards,
Niklas Söderlund


More information about the libcamera-devel mailing list