[libcamera-devel] [PATCH v3 0/4] lc-compliance: Refactor using Googletest
Nícolas F. R. A. Prado
nfraprado at collabora.com
Fri May 14 15:16:48 CEST 2021
This is a prototype of the refactoring of lc-compliance using Googletest as the
framework.
Patches 1 and 2 are just to simplify the cleanup path from the tests. Patch 3
does the actual refactor. Patch 4 adds two new parameters, --list and --filter.
lc-compliance is built around testing a single camera (and possibly optional
secondary cameras in the future), which is passed through the -c flag. The list
of tests can be queried with:
lc-compliance --list
By default all tests are run, but it's possible to select which tests to run by
using:
lc-compliance -c '<camera_name>' --filter '<test_filter>'
where the test_filter is in the same format as Googletest's --gtest_filter.
Quoting its help:
--gtest_filter=POSTIVE_PATTERNS[-NEGATIVE_PATTERNS]
Run only the tests whose name matches one of the positive patterns but
none of the negative patterns. '?' matches any single character; '*'
matches any substring; ':' separates two patterns.
This is a snippet of a successful run:
[==========] Running 120 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 120 tests from RolesAndRequests/FixedRequestsTest
[ RUN ] RolesAndRequests/FixedRequestsTest.BalancedSingleCycle/raw1
[51:19:15.995527857] [339440] INFO Camera camera.cpp:915 configuring streams: (0) 1280x720-MJPEG
Camera needs 4 requests, can't test only 1
[ SKIPPED ] RolesAndRequests/FixedRequestsTest.BalancedSingleCycle/raw1 (63 ms)
[...]
[ RUN ] RolesAndRequests/FixedRequestsTest.Unbalanced/viewfinder89
[51:22:08.969389923] [339440] INFO Camera camera.cpp:915 configuring streams: (0) 1280x720-MJPEG
[ OK ] RolesAndRequests/FixedRequestsTest.Unbalanced/viewfinder89 (3144 ms)
[----------] 120 tests from RolesAndRequests/FixedRequestsTest (176118 ms total)
[----------] Global test environment tear-down
[==========] 120 tests from 1 test suite ran. (176118 ms total)
[ PASSED ] 96 tests.
[ SKIPPED ] 24 tests, listed below:
[ SKIPPED ] RolesAndRequests/FixedRequestsTest.BalancedSingleCycle/raw1
[ SKIPPED ] RolesAndRequests/FixedRequestsTest.BalancedSingleCycle/raw2
[ SKIPPED ] RolesAndRequests/FixedRequestsTest.BalancedSingleCycle/raw3
[ SKIPPED ] RolesAndRequests/FixedRequestsTest.BalancedSingleCycle/still1
[...]
And how a failure looks like:
[ RUN ] RolesAndRequests/FixedRequestsTest.BalancedSingleCycle/raw5
[51:30:10.270474586] [340392] INFO Camera camera.cpp:915 configuring streams: (0) 1280x720-MJPEG
../src/lc-compliance/simple_capture.cpp:117: Failure
Expected equality of these values:
captureCount_
Which is: 5
captureLimit_
Which is: 6
[ FAILED ] RolesAndRequests/FixedRequestsTest.BalancedSingleCycle/raw5, where GetParam() = (0, 5) (317 ms)
The current complete list of tests is shown below:
RolesAndRequests/FixedRequestsTest.
BalancedSingleCycle/raw1 # GetParam() = (0, 1)
BalancedSingleCycle/raw2 # GetParam() = (0, 2)
BalancedSingleCycle/raw3 # GetParam() = (0, 3)
BalancedSingleCycle/raw5 # GetParam() = (0, 5)
BalancedSingleCycle/raw8 # GetParam() = (0, 8)
BalancedSingleCycle/raw13 # GetParam() = (0, 13)
BalancedSingleCycle/raw21 # GetParam() = (0, 21)
BalancedSingleCycle/raw34 # GetParam() = (0, 34)
BalancedSingleCycle/raw55 # GetParam() = (0, 55)
BalancedSingleCycle/raw89 # GetParam() = (0, 89)
BalancedSingleCycle/still1 # GetParam() = (1, 1)
BalancedSingleCycle/still2 # GetParam() = (1, 2)
BalancedSingleCycle/still3 # GetParam() = (1, 3)
BalancedSingleCycle/still5 # GetParam() = (1, 5)
BalancedSingleCycle/still8 # GetParam() = (1, 8)
BalancedSingleCycle/still13 # GetParam() = (1, 13)
BalancedSingleCycle/still21 # GetParam() = (1, 21)
BalancedSingleCycle/still34 # GetParam() = (1, 34)
BalancedSingleCycle/still55 # GetParam() = (1, 55)
BalancedSingleCycle/still89 # GetParam() = (1, 89)
BalancedSingleCycle/video1 # GetParam() = (2, 1)
BalancedSingleCycle/video2 # GetParam() = (2, 2)
BalancedSingleCycle/video3 # GetParam() = (2, 3)
BalancedSingleCycle/video5 # GetParam() = (2, 5)
BalancedSingleCycle/video8 # GetParam() = (2, 8)
BalancedSingleCycle/video13 # GetParam() = (2, 13)
BalancedSingleCycle/video21 # GetParam() = (2, 21)
BalancedSingleCycle/video34 # GetParam() = (2, 34)
BalancedSingleCycle/video55 # GetParam() = (2, 55)
BalancedSingleCycle/video89 # GetParam() = (2, 89)
BalancedSingleCycle/viewfinder1 # GetParam() = (3, 1)
BalancedSingleCycle/viewfinder2 # GetParam() = (3, 2)
BalancedSingleCycle/viewfinder3 # GetParam() = (3, 3)
BalancedSingleCycle/viewfinder5 # GetParam() = (3, 5)
BalancedSingleCycle/viewfinder8 # GetParam() = (3, 8)
BalancedSingleCycle/viewfinder13 # GetParam() = (3, 13)
BalancedSingleCycle/viewfinder21 # GetParam() = (3, 21)
BalancedSingleCycle/viewfinder34 # GetParam() = (3, 34)
BalancedSingleCycle/viewfinder55 # GetParam() = (3, 55)
BalancedSingleCycle/viewfinder89 # GetParam() = (3, 89)
BalancedMultiCycle/raw1 # GetParam() = (0, 1)
BalancedMultiCycle/raw2 # GetParam() = (0, 2)
BalancedMultiCycle/raw3 # GetParam() = (0, 3)
BalancedMultiCycle/raw5 # GetParam() = (0, 5)
BalancedMultiCycle/raw8 # GetParam() = (0, 8)
BalancedMultiCycle/raw13 # GetParam() = (0, 13)
BalancedMultiCycle/raw21 # GetParam() = (0, 21)
BalancedMultiCycle/raw34 # GetParam() = (0, 34)
BalancedMultiCycle/raw55 # GetParam() = (0, 55)
BalancedMultiCycle/raw89 # GetParam() = (0, 89)
BalancedMultiCycle/still1 # GetParam() = (1, 1)
BalancedMultiCycle/still2 # GetParam() = (1, 2)
BalancedMultiCycle/still3 # GetParam() = (1, 3)
BalancedMultiCycle/still5 # GetParam() = (1, 5)
BalancedMultiCycle/still8 # GetParam() = (1, 8)
BalancedMultiCycle/still13 # GetParam() = (1, 13)
BalancedMultiCycle/still21 # GetParam() = (1, 21)
BalancedMultiCycle/still34 # GetParam() = (1, 34)
BalancedMultiCycle/still55 # GetParam() = (1, 55)
BalancedMultiCycle/still89 # GetParam() = (1, 89)
BalancedMultiCycle/video1 # GetParam() = (2, 1)
BalancedMultiCycle/video2 # GetParam() = (2, 2)
BalancedMultiCycle/video3 # GetParam() = (2, 3)
BalancedMultiCycle/video5 # GetParam() = (2, 5)
BalancedMultiCycle/video8 # GetParam() = (2, 8)
BalancedMultiCycle/video13 # GetParam() = (2, 13)
BalancedMultiCycle/video21 # GetParam() = (2, 21)
BalancedMultiCycle/video34 # GetParam() = (2, 34)
BalancedMultiCycle/video55 # GetParam() = (2, 55)
BalancedMultiCycle/video89 # GetParam() = (2, 89)
BalancedMultiCycle/viewfinder1 # GetParam() = (3, 1)
BalancedMultiCycle/viewfinder2 # GetParam() = (3, 2)
BalancedMultiCycle/viewfinder3 # GetParam() = (3, 3)
BalancedMultiCycle/viewfinder5 # GetParam() = (3, 5)
BalancedMultiCycle/viewfinder8 # GetParam() = (3, 8)
BalancedMultiCycle/viewfinder13 # GetParam() = (3, 13)
BalancedMultiCycle/viewfinder21 # GetParam() = (3, 21)
BalancedMultiCycle/viewfinder34 # GetParam() = (3, 34)
BalancedMultiCycle/viewfinder55 # GetParam() = (3, 55)
BalancedMultiCycle/viewfinder89 # GetParam() = (3, 89)
Unbalanced/raw1 # GetParam() = (0, 1)
Unbalanced/raw2 # GetParam() = (0, 2)
Unbalanced/raw3 # GetParam() = (0, 3)
Unbalanced/raw5 # GetParam() = (0, 5)
Unbalanced/raw8 # GetParam() = (0, 8)
Unbalanced/raw13 # GetParam() = (0, 13)
Unbalanced/raw21 # GetParam() = (0, 21)
Unbalanced/raw34 # GetParam() = (0, 34)
Unbalanced/raw55 # GetParam() = (0, 55)
Unbalanced/raw89 # GetParam() = (0, 89)
Unbalanced/still1 # GetParam() = (1, 1)
Unbalanced/still2 # GetParam() = (1, 2)
Unbalanced/still3 # GetParam() = (1, 3)
Unbalanced/still5 # GetParam() = (1, 5)
Unbalanced/still8 # GetParam() = (1, 8)
Unbalanced/still13 # GetParam() = (1, 13)
Unbalanced/still21 # GetParam() = (1, 21)
Unbalanced/still34 # GetParam() = (1, 34)
Unbalanced/still55 # GetParam() = (1, 55)
Unbalanced/still89 # GetParam() = (1, 89)
Unbalanced/video1 # GetParam() = (2, 1)
Unbalanced/video2 # GetParam() = (2, 2)
Unbalanced/video3 # GetParam() = (2, 3)
Unbalanced/video5 # GetParam() = (2, 5)
Unbalanced/video8 # GetParam() = (2, 8)
Unbalanced/video13 # GetParam() = (2, 13)
Unbalanced/video21 # GetParam() = (2, 21)
Unbalanced/video34 # GetParam() = (2, 34)
Unbalanced/video55 # GetParam() = (2, 55)
Unbalanced/video89 # GetParam() = (2, 89)
Unbalanced/viewfinder1 # GetParam() = (3, 1)
Unbalanced/viewfinder2 # GetParam() = (3, 2)
Unbalanced/viewfinder3 # GetParam() = (3, 3)
Unbalanced/viewfinder5 # GetParam() = (3, 5)
Unbalanced/viewfinder8 # GetParam() = (3, 8)
Unbalanced/viewfinder13 # GetParam() = (3, 13)
Unbalanced/viewfinder21 # GetParam() = (3, 21)
Unbalanced/viewfinder34 # GetParam() = (3, 34)
Unbalanced/viewfinder55 # GetParam() = (3, 55)
Unbalanced/viewfinder89 # GetParam() = (3, 89)
Changes in v3:
- Thanks to Niklas:
- Went back to static test registration, and created a Singleton Environment
class to store the camera global variable
- Added patch 4 implementing list and filter parameters
- Added a nameParameters() function to give meaningful names to the test
parameters, removing the need to register each test suite for every role
v2: https://lists.libcamera.org/pipermail/libcamera-devel/2021-May/020056.html
Changes in v2:
- Added patch 1 to make Camera::stop() idempotent as well from [1].
- Made test registration dynamic in patch 3
- Some other minor fixes
[1] https://lists.libcamera.org/pipermail/libcamera-devel/2021-April/019735.html
v1: https://lists.libcamera.org/pipermail/libcamera-devel/2021-April/019702.html
Nícolas F. R. A. Prado (4):
libcamera: camera: Make stop() idempotent
lc-compliance: Make SimpleCapture::stop() idempotent
lc-compliance: Refactor using Googletest
lc-compliance: Add list and filter parameters
src/lc-compliance/main.cpp | 152 +++++++++++++++++++++------
src/lc-compliance/meson.build | 3 +
src/lc-compliance/simple_capture.cpp | 95 ++++++-----------
src/lc-compliance/simple_capture.h | 8 +-
src/lc-compliance/single_stream.cpp | 148 +++++++++++++-------------
src/lc-compliance/tests.h | 19 +++-
src/libcamera/camera.cpp | 20 +++-
7 files changed, 275 insertions(+), 170 deletions(-)
--
2.31.1
More information about the libcamera-devel
mailing list