[PATCH v3 2/3] libcamera: Add face detection controls
Cheng-Hao Yang
chenghaoyang at chromium.org
Tue Sep 3 09:59:42 CEST 2024
Sorry, Jacopo, I regret:
On Tue, Sep 3, 2024 at 3:43 PM Cheng-Hao Yang <chenghaoyang at chromium.org>
wrote:
> Hi Jacopo,
>
> The updates will be applied in the next patch.
>
> On Sat, Aug 31, 2024 at 10:19 PM Jacopo Mondi <
> jacopo.mondi at ideasonboard.com> wrote:
>
>> Hi
>>
>> On Fri, Aug 30, 2024 at 09:00:20PM GMT, Harvey Yang wrote:
>> > From: Yudhistira Erlandinata <yerlandinata at chromium.org>
>> >
>> > Add FaceDetectMode, FaceDetectFaceRectangles, FaceDetectFaceScores,
>> > and FaceDetectFaceLandmark. Also add ControlTypePoint for supporting
>> > FaceDetectFaceLandmark.
>> >
>> > Signed-off-by: Yudhistira Erlandinata <yerlandinata at chromium.org>
>> > Co-developed-by: becker hsieh <beckerh at chromium.org>
>> > Co-developed-by: Harvey Yang <chenghaoyang at chromium.org>
>> > ---
>> > include/libcamera/controls.h | 6 +++
>> > src/libcamera/control_ids_draft.yaml | 63 ++++++++++++++++++++++++++++
>> > src/libcamera/controls.cpp | 6 +++
>> > 3 files changed, 75 insertions(+)
>> >
>> > diff --git a/include/libcamera/controls.h b/include/libcamera/controls.h
>> > index 7c2bb2872..bf1b8609c 100644
>> > --- a/include/libcamera/controls.h
>> > +++ b/include/libcamera/controls.h
>> > @@ -34,6 +34,7 @@ enum ControlType {
>> > ControlTypeString,
>> > ControlTypeRectangle,
>> > ControlTypeSize,
>> > + ControlTypePoint,
>> > };
>> >
>> > namespace details {
>> > @@ -87,6 +88,11 @@ struct control_type<Size> {
>> > static constexpr ControlType value = ControlTypeSize;
>> > };
>> >
>> > +template<>
>> > +struct control_type<Point> {
>> > + static constexpr ControlType value = ControlTypePoint;
>> > +};
>> > +
>> > template<typename T, std::size_t N>
>> > struct control_type<Span<T, N>> : public
>> control_type<std::remove_cv_t<T>> {
>> > };
>> > diff --git a/src/libcamera/control_ids_draft.yaml
>> b/src/libcamera/control_ids_draft.yaml
>> > index 9bef5bf15..23e09bff8 100644
>> > --- a/src/libcamera/control_ids_draft.yaml
>> > +++ b/src/libcamera/control_ids_draft.yaml
>> > @@ -227,4 +227,67 @@ controls:
>> > value. All of the custom test patterns will be static
>> (that is the
>> > raw image must not vary from frame to frame).
>> >
>> > + - FaceDetectMode:
>> > + type: uint8_t
>> > + description: |
>> > + Reporting mode of face detection.
>> > +
>> > + enum:
>> > + - name: FaceDetectModeOff
>> > + value: 0
>> > + description: |
>> > + Pipeline should not report face detection result.
>> > + - name: FaceDetectModeSimple
>> > + value: 1
>> > + description: |
>> > + Pipeline should at least report faces boundary rectangles,
>> in
>> > + metadata FaceDetectFaceRectangles, and confidence score,
>> > + in metadata FaceDetectFaceScores, for each of them.
>> > + FaceDetectFaceLandmark is optional.
>> > +
>> > + The number of faces in each list must be the same.
>> > +
>> > + \sa FaceDetectFaceRectangles
>> > + \sa FaceDetectFaceScores
>> > + \sa FaceDetectFaceLandmark
>> > +
>> > + - FaceDetectFaceRectangles:
>> > + type: Rectangle
>> > + description: |
>> > + Boundary rectangle of the detected faces in format:
>> > + [..., xmin_i, ymin_i, xmax_i, ymax_i, ...], where (0,0) is
>> top-left
>> > + of active pixel area.
>> > + The number of values should be 3 * the number of faces.
>>
>> Aren't there 4 entries per face ?
>>
> Right, sorry. Updated.
>
It should be 1 * the number of faces, as the unit we use is Rectangle.
We don't need to describe the boundary again here as well, as it's
Android's definition.
>
>
>>
>> > +
>> > + The FaceDetectFaceRectangles control can only be returned in
>> metadata.
>> > +
>> > + size: [n]
>> > +
>> > + - FaceDetectFaceScores:
>> > + type: uint8_t
>> > + description: |
>> > + Confidence score of each of the detected faces by face
>> detector.
>> > + The range of score is [0, 100].
>> > + The FaceDetectFaceScores control can only be returned in
>> metadata.
>> > + The number of values should be the number of faces.
>> > +
>> > + Currently identical to ANDROID_STATISTICS_FACE_SCORES.
>> > +
>> > + size: [n]
>> > +
>> > + - FaceDetectFaceLandmark:
>> > + type: Point
>> > + description: |
>> > + Array of human face landmark coordinates in format:
>> > + [..., left_eye_i, right_eye_i, mouth_i, left_eye_i+1, ...],
>> > + with i = index of face.
>> > + The number of values should be 6 * the number of faces.
>>
>> I count 3. but looking at the implementation each landmark is a point
>> in (x, y) form.
>>
> Right, updated to [left_eye_i_x, left_eye_i_y, ...].
>
It should be 3, and the format remains as
`[..., left_eye_i, right_eye_i, mouth_i, left_eye_i+1, ...]`.
>
>
>>
>> > +
>> > + The FaceDetectFaceLandmark control can only be returned in
>> metadata.
>> > +
>> > + Currently identical to ANDROID_STATISTICS_FACE_LANDMARKS.
>> > +
>> > + size: [n]
>> > +
>> > +
>> > ...
>> > diff --git a/src/libcamera/controls.cpp b/src/libcamera/controls.cpp
>> > index 11d35321c..ce73ae9d7 100644
>> > --- a/src/libcamera/controls.cpp
>> > +++ b/src/libcamera/controls.cpp
>> > @@ -61,6 +61,7 @@ static constexpr size_t ControlValueSize[] = {
>> > [ControlTypeString] = sizeof(char),
>> > [ControlTypeRectangle] = sizeof(Rectangle),
>> > [ControlTypeSize] = sizeof(Size),
>> > + [ControlTypePoint] = sizeof(Point),
>> > };
>> >
>> > } /* namespace */
>> > @@ -255,6 +256,11 @@ std::string ControlValue::toString() const
>> > str += value->toString();
>> > break;
>> > }
>> > + case ControlTypePoint: {
>> > + const Point *value = reinterpret_cast<const Point
>> *>(data);
>> > + str += value->toString();
>> > + break;
>> > + }
>> > case ControlTypeNone:
>> > case ControlTypeString:
>> > break;
>> > --
>> > 2.46.0.469.g59c65b2a67-goog
>> >
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.libcamera.org/pipermail/libcamera-devel/attachments/20240903/5fdac37c/attachment.htm>
More information about the libcamera-devel
mailing list