[PATCH v8 2/3] libcamera: Add face detection controls

[Jacopo Mondi]

Hi Harvey

On Wed, Sep 25, 2024 at 08:12:25AM GMT, Harvey Yang wrote:
> From: Yudhistira Erlandinata <yerlandinata at chromium.org>
>
> Add FaceDetectMode, FaceDetectFaceRectangles, FaceDetectFaceScores,
> and FaceDetectFaceLandmark. Also add ControlTypePoint for supporting
> FaceDetectFaceLandmark.
>
> Signed-off-by: Yudhistira Erlandinata <yerlandinata at chromium.org>
> Co-developed-by: becker hsieh <beckerh at chromium.org>
> Co-developed-by: Harvey Yang <chenghaoyang at chromium.org>
> Reviewed-by: Jacopo Mondi <jacopo.mondi at ideasonboard.com>
> ---
>  include/libcamera/controls.h         |  6 ++
>  src/libcamera/control_ids_draft.yaml | 88 ++++++++++++++++++++++++++++
>  src/libcamera/controls.cpp           |  6 ++
>  3 files changed, 100 insertions(+)
>
> diff --git a/include/libcamera/controls.h b/include/libcamera/controls.h
> index 7c2bb287..bf1b8609 100644
> --- a/include/libcamera/controls.h
> +++ b/include/libcamera/controls.h
> @@ -34,6 +34,7 @@ enum ControlType {
>  	ControlTypeString,
>  	ControlTypeRectangle,
>  	ControlTypeSize,
> +	ControlTypePoint,
>  };
>
>  namespace details {
> @@ -87,6 +88,11 @@ struct control_type<Size> {
>  	static constexpr ControlType value = ControlTypeSize;
>  };
>
> +template<>
> +struct control_type<Point> {
> +	static constexpr ControlType value = ControlTypePoint;
> +};
> +
>  template<typename T, std::size_t N>
>  struct control_type<Span<T, N>> : public control_type<std::remove_cv_t<T>> {
>  };
> diff --git a/src/libcamera/control_ids_draft.yaml b/src/libcamera/control_ids_draft.yaml
> index 9bef5bf1..6a472b48 100644
> --- a/src/libcamera/control_ids_draft.yaml
> +++ b/src/libcamera/control_ids_draft.yaml
> @@ -227,4 +227,92 @@ controls:
>              value. All of the custom test patterns will be static (that is the
>              raw image must not vary from frame to frame).
>
> +  - FaceDetectMode:
> +      type: uint8_t

This is not valid anymore since the introduction of
e6da224926b0 ("libcamera: controls: Handle enum values without a cast")

I'll make it a int32_t and slightly tweak the text descriptions to
align them to existing draft controls.

I'll re-send a new version of these patches on top of an upate to the
Rectangle's top-left point documentation.

Thanks
  j

> +      description: |
> +        Reporting mode of face detection.
> +
> +        Currently identical to ANDROID_STATISTICS_FACE_DETECT_MODE.
> +
> +        \sa FaceDetectFaceRectangles
> +        \sa FaceDetectFaceScores
> +        \sa FaceDetectFaceLandmarks
> +        \sa FaceDetectFaceIds
> +
> +      enum:
> +        - name: FaceDetectModeOff
> +          value: 0
> +          description: |
> +            Pipeline should not report face detection result.
> +
> +        - name: FaceDetectModeSimple
> +          value: 1
> +          description: |
> +            Pipeline should at least report FaceDetectFaceRectangles and
> +            FaceDetectFaceScores for each detected face.
> +            FaceDetectFaceLandmarks and FaceDetectFaceIds are optional.
> +
> +        - name: FaceDetectModeFull
> +          value: 2
> +          description: |
> +            Pipeline should report all face controls, including
> +            FaceDetectFaceRectangles, FaceDetectFaceScores,
> +            FaceDetectFaceLandmarks, and FaceDeteceFaceIds.
> +
> +  - FaceDetectFaceRectangles:
> +      type: Rectangle
> +      description: |
> +        Boundary rectangles of the detected faces.
> +        The number of values should be the number of faces reported in
> +        FaceDetectFaceRectangles.
> +
> +        The FaceDetectFaceRectangles control can only be returned in metadata.
> +
> +        Currently identical to ANDROID_STATISTICS_FACE_RECTANGLES.
> +
> +      size: [n]
> +
> +  - FaceDetectFaceScores:
> +      type: uint8_t
> +      description: |
> +        Confidence score of each of the detected faces by face detector.
> +        The range of score is [0, 100].
> +        The FaceDetectFaceScores control can only be returned in metadata.
> +        The number of values should be the number of faces reported in
> +        FaceDetectFaceRectangles.
> +
> +        Currently identical to ANDROID_STATISTICS_FACE_SCORES.
> +
> +      size: [n]
> +
> +  - FaceDetectFaceLandmarks:
> +      type: Point
> +      description: |
> +        Array of human face landmark coordinates in format:
> +        [..., left_eye_i, right_eye_i, mouth_i, left_eye_i+1, ...],
> +        with i = index of face.
> +        The number of values should be 3 * the number of faces reported in
> +        FaceDetectFaceRectangles.
> +
> +        The FaceDetectFaceLandmarks control can only be returned in metadata.
> +
> +        Currently identical to ANDROID_STATISTICS_FACE_LANDMARKS.
> +
> +      size: [n]
> +
> +  - FaceDetectFaceIds:
> +      type: int32_t
> +      description: |
> +        Each detected face is given a unique ID that is valid for as long as
> +        the face is visible to the camera device. A face that leaves the field
> +        of view and later returns may be assigned a new ID.
> +        The number of values should be the number of faces reported in
> +        FaceDetectFaceRectangles.
> +
> +        The FaceDetectFaceIds control can only be returned in metadata.
> +
> +        Currently identical to ANDROID_STATISTICS_FACE_IDS.
> +
> +      size: [n]
> +
>  ...
> diff --git a/src/libcamera/controls.cpp b/src/libcamera/controls.cpp
> index dba74404..c452e607 100644
> --- a/src/libcamera/controls.cpp
> +++ b/src/libcamera/controls.cpp
> @@ -60,6 +60,7 @@ static constexpr size_t ControlValueSize[] = {
>  	[ControlTypeString]		= sizeof(char),
>  	[ControlTypeRectangle]		= sizeof(Rectangle),
>  	[ControlTypeSize]		= sizeof(Size),
> +	[ControlTypePoint]		= sizeof(Point),
>  };
>
>  } /* namespace */
> @@ -254,6 +255,11 @@ std::string ControlValue::toString() const
>  			str += value->toString();
>  			break;
>  		}
> +		case ControlTypePoint: {
> +			const Point *value = reinterpret_cast<const Point *>(data);
> +			str += value->toString();
> +			break;
> +		}
>  		case ControlTypeNone:
>  		case ControlTypeString:
>  			break;
> --
> 2.46.0.792.g87dc391469-goog
>