[PATCH v3 2/3] libcamera: Add face detection controls

[Cheng-Hao Yang]

Hi Jacopo,

The updates will be applied in the next patch.

On Sat, Aug 31, 2024 at 10:19 PM Jacopo Mondi <jacopo.mondi at ideasonboard.com>
wrote:

> Hi
>
> On Fri, Aug 30, 2024 at 09:00:20PM GMT, Harvey Yang wrote:
> > From: Yudhistira Erlandinata <yerlandinata at chromium.org>
> >
> > Add FaceDetectMode, FaceDetectFaceRectangles, FaceDetectFaceScores,
> > and FaceDetectFaceLandmark. Also add ControlTypePoint for supporting
> > FaceDetectFaceLandmark.
> >
> > Signed-off-by: Yudhistira Erlandinata <yerlandinata at chromium.org>
> > Co-developed-by: becker hsieh <beckerh at chromium.org>
> > Co-developed-by: Harvey Yang <chenghaoyang at chromium.org>
> > ---
> >  include/libcamera/controls.h         |  6 +++
> >  src/libcamera/control_ids_draft.yaml | 63 ++++++++++++++++++++++++++++
> >  src/libcamera/controls.cpp           |  6 +++
> >  3 files changed, 75 insertions(+)
> >
> > diff --git a/include/libcamera/controls.h b/include/libcamera/controls.h
> > index 7c2bb2872..bf1b8609c 100644
> > --- a/include/libcamera/controls.h
> > +++ b/include/libcamera/controls.h
> > @@ -34,6 +34,7 @@ enum ControlType {
> >       ControlTypeString,
> >       ControlTypeRectangle,
> >       ControlTypeSize,
> > +     ControlTypePoint,
> >  };
> >
> >  namespace details {
> > @@ -87,6 +88,11 @@ struct control_type<Size> {
> >       static constexpr ControlType value = ControlTypeSize;
> >  };
> >
> > +template<>
> > +struct control_type<Point> {
> > +     static constexpr ControlType value = ControlTypePoint;
> > +};
> > +
> >  template<typename T, std::size_t N>
> >  struct control_type<Span<T, N>> : public
> control_type<std::remove_cv_t<T>> {
> >  };
> > diff --git a/src/libcamera/control_ids_draft.yaml
> b/src/libcamera/control_ids_draft.yaml
> > index 9bef5bf15..23e09bff8 100644
> > --- a/src/libcamera/control_ids_draft.yaml
> > +++ b/src/libcamera/control_ids_draft.yaml
> > @@ -227,4 +227,67 @@ controls:
> >              value. All of the custom test patterns will be static (that
> is the
> >              raw image must not vary from frame to frame).
> >
> > +  - FaceDetectMode:
> > +      type: uint8_t
> > +      description: |
> > +        Reporting mode of face detection.
> > +
> > +      enum:
> > +        - name: FaceDetectModeOff
> > +          value: 0
> > +          description: |
> > +            Pipeline should not report face detection result.
> > +        - name: FaceDetectModeSimple
> > +          value: 1
> > +          description: |
> > +            Pipeline should at least report faces boundary rectangles,
> in
> > +            metadata FaceDetectFaceRectangles, and confidence score,
> > +            in metadata FaceDetectFaceScores, for each of them.
> > +            FaceDetectFaceLandmark is optional.
> > +
> > +            The number of faces in each list must be the same.
> > +
> > +            \sa FaceDetectFaceRectangles
> > +            \sa FaceDetectFaceScores
> > +            \sa FaceDetectFaceLandmark
> > +
> > +  - FaceDetectFaceRectangles:
> > +      type: Rectangle
> > +      description: |
> > +        Boundary rectangle of the detected faces in format:
> > +        [..., xmin_i, ymin_i, xmax_i, ymax_i, ...], where (0,0) is
> top-left
> > +        of active pixel area.
> > +        The number of values should be 3 * the number of faces.
>
> Aren't there 4 entries per face ?
>
Right, sorry. Updated.


>
> > +
> > +        The FaceDetectFaceRectangles control can only be returned in
> metadata.
> > +
> > +      size: [n]
> > +
> > +  - FaceDetectFaceScores:
> > +      type: uint8_t
> > +      description: |
> > +        Confidence score of each of the detected faces by face detector.
> > +        The range of score is [0, 100].
> > +        The FaceDetectFaceScores control can only be returned in
> metadata.
> > +        The number of values should be the number of faces.
> > +
> > +        Currently identical to ANDROID_STATISTICS_FACE_SCORES.
> > +
> > +      size: [n]
> > +
> > +  - FaceDetectFaceLandmark:
> > +      type: Point
> > +      description: |
> > +        Array of human face landmark coordinates in format:
> > +        [..., left_eye_i, right_eye_i, mouth_i, left_eye_i+1, ...],
> > +        with i = index of face.
> > +        The number of values should be 6 * the number of faces.
>
> I count 3. but looking at the implementation each landmark is a point
> in (x, y) form.
>
Right, updated to [left_eye_i_x, left_eye_i_y, ...].


>
> > +
> > +        The FaceDetectFaceLandmark control can only be returned in
> metadata.
> > +
> > +        Currently identical to ANDROID_STATISTICS_FACE_LANDMARKS.
> > +
> > +      size: [n]
> > +
> > +
> >  ...
> > diff --git a/src/libcamera/controls.cpp b/src/libcamera/controls.cpp
> > index 11d35321c..ce73ae9d7 100644
> > --- a/src/libcamera/controls.cpp
> > +++ b/src/libcamera/controls.cpp
> > @@ -61,6 +61,7 @@ static constexpr size_t ControlValueSize[] = {
> >       [ControlTypeString]             = sizeof(char),
> >       [ControlTypeRectangle]          = sizeof(Rectangle),
> >       [ControlTypeSize]               = sizeof(Size),
> > +     [ControlTypePoint]              = sizeof(Point),
> >  };
> >
> >  } /* namespace */
> > @@ -255,6 +256,11 @@ std::string ControlValue::toString() const
> >                       str += value->toString();
> >                       break;
> >               }
> > +             case ControlTypePoint: {
> > +                     const Point *value = reinterpret_cast<const Point
> *>(data);
> > +                     str += value->toString();
> > +                     break;
> > +             }
> >               case ControlTypeNone:
> >               case ControlTypeString:
> >                       break;
> > --
> > 2.46.0.469.g59c65b2a67-goog
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.libcamera.org/pipermail/libcamera-devel/attachments/20240903/a185a9b8/attachment.htm>