[PATCH v3 2/3] libcamera: Add face detection controls

[Jacopo Mondi]

Hi

On Fri, Aug 30, 2024 at 09:00:20PM GMT, Harvey Yang wrote:
> From: Yudhistira Erlandinata <yerlandinata at chromium.org>
>
> Add FaceDetectMode, FaceDetectFaceRectangles, FaceDetectFaceScores,
> and FaceDetectFaceLandmark. Also add ControlTypePoint for supporting
> FaceDetectFaceLandmark.
>
> Signed-off-by: Yudhistira Erlandinata <yerlandinata at chromium.org>
> Co-developed-by: becker hsieh <beckerh at chromium.org>
> Co-developed-by: Harvey Yang <chenghaoyang at chromium.org>
> ---
>  include/libcamera/controls.h         |  6 +++
>  src/libcamera/control_ids_draft.yaml | 63 ++++++++++++++++++++++++++++
>  src/libcamera/controls.cpp           |  6 +++
>  3 files changed, 75 insertions(+)
>
> diff --git a/include/libcamera/controls.h b/include/libcamera/controls.h
> index 7c2bb2872..bf1b8609c 100644
> --- a/include/libcamera/controls.h
> +++ b/include/libcamera/controls.h
> @@ -34,6 +34,7 @@ enum ControlType {
>  	ControlTypeString,
>  	ControlTypeRectangle,
>  	ControlTypeSize,
> +	ControlTypePoint,
>  };
>
>  namespace details {
> @@ -87,6 +88,11 @@ struct control_type<Size> {
>  	static constexpr ControlType value = ControlTypeSize;
>  };
>
> +template<>
> +struct control_type<Point> {
> +	static constexpr ControlType value = ControlTypePoint;
> +};
> +
>  template<typename T, std::size_t N>
>  struct control_type<Span<T, N>> : public control_type<std::remove_cv_t<T>> {
>  };
> diff --git a/src/libcamera/control_ids_draft.yaml b/src/libcamera/control_ids_draft.yaml
> index 9bef5bf15..23e09bff8 100644
> --- a/src/libcamera/control_ids_draft.yaml
> +++ b/src/libcamera/control_ids_draft.yaml
> @@ -227,4 +227,67 @@ controls:
>              value. All of the custom test patterns will be static (that is the
>              raw image must not vary from frame to frame).
>
> +  - FaceDetectMode:
> +      type: uint8_t
> +      description: |
> +        Reporting mode of face detection.
> +
> +      enum:
> +        - name: FaceDetectModeOff
> +          value: 0
> +          description: |
> +            Pipeline should not report face detection result.
> +        - name: FaceDetectModeSimple
> +          value: 1
> +          description: |
> +            Pipeline should at least report faces boundary rectangles, in
> +            metadata FaceDetectFaceRectangles, and confidence score,
> +            in metadata FaceDetectFaceScores, for each of them.
> +            FaceDetectFaceLandmark is optional.
> +
> +            The number of faces in each list must be the same.
> +
> +            \sa FaceDetectFaceRectangles
> +            \sa FaceDetectFaceScores
> +            \sa FaceDetectFaceLandmark
> +
> +  - FaceDetectFaceRectangles:
> +      type: Rectangle
> +      description: |
> +        Boundary rectangle of the detected faces in format:
> +        [..., xmin_i, ymin_i, xmax_i, ymax_i, ...], where (0,0) is top-left
> +        of active pixel area.
> +        The number of values should be 3 * the number of faces.

Aren't there 4 entries per face ?

> +
> +        The FaceDetectFaceRectangles control can only be returned in metadata.
> +
> +      size: [n]
> +
> +  - FaceDetectFaceScores:
> +      type: uint8_t
> +      description: |
> +        Confidence score of each of the detected faces by face detector.
> +        The range of score is [0, 100].
> +        The FaceDetectFaceScores control can only be returned in metadata.
> +        The number of values should be the number of faces.
> +
> +        Currently identical to ANDROID_STATISTICS_FACE_SCORES.
> +
> +      size: [n]
> +
> +  - FaceDetectFaceLandmark:
> +      type: Point
> +      description: |
> +        Array of human face landmark coordinates in format:
> +        [..., left_eye_i, right_eye_i, mouth_i, left_eye_i+1, ...],
> +        with i = index of face.
> +        The number of values should be 6 * the number of faces.

I count 3. but looking at the implementation each landmark is a point
in (x, y) form.

> +
> +        The FaceDetectFaceLandmark control can only be returned in metadata.
> +
> +        Currently identical to ANDROID_STATISTICS_FACE_LANDMARKS.
> +
> +      size: [n]
> +
> +
>  ...
> diff --git a/src/libcamera/controls.cpp b/src/libcamera/controls.cpp
> index 11d35321c..ce73ae9d7 100644
> --- a/src/libcamera/controls.cpp
> +++ b/src/libcamera/controls.cpp
> @@ -61,6 +61,7 @@ static constexpr size_t ControlValueSize[] = {
>  	[ControlTypeString]		= sizeof(char),
>  	[ControlTypeRectangle]		= sizeof(Rectangle),
>  	[ControlTypeSize]		= sizeof(Size),
> +	[ControlTypePoint]		= sizeof(Point),
>  };
>
>  } /* namespace */
> @@ -255,6 +256,11 @@ std::string ControlValue::toString() const
>  			str += value->toString();
>  			break;
>  		}
> +		case ControlTypePoint: {
> +			const Point *value = reinterpret_cast<const Point *>(data);
> +			str += value->toString();
> +			break;
> +		}
>  		case ControlTypeNone:
>  		case ControlTypeString:
>  			break;
> --
> 2.46.0.469.g59c65b2a67-goog
>