[libcamera-devel] [RFC PATCH v2 0/3] lc-compliance: Refactor using Googletest
Nícolas F. R. A. Prado
nfraprado at collabora.com
Mon May 3 21:33:04 CEST 2021
This is a prototype of the refactoring of lc-compliance using Googletest as the
framework.
Patches 1 and 2 are just to simplify the cleanup path from the tests. Patch 3
does the actual refactor.
In v1 the tests were registered statically which simplified the work needed to
add new tests, but it required the camera pointer to be a global variable, which
wouldn't scale for usage with multiple cameras.
For v2 the tests are registered dynamically so that a camera pointer can be
passed to tests being instantiated. This requires manual registration of each
parametrization of the tests, which I tried to make less cumbersome by defining
the REGISTER_TESTS() macro (although for tests with different parameters,
different macros would be needed...). It also requires each new test to declare
a new class and be described in the TestBody() function of the class. Perhaps
this could also be made easier with a macro?
Also, since now a test is created for each of the cameras in the system, the -c
flag was removed, as the camera to be tested can be selected by filtering
only tests for that camera with --gtest_filter. Although we could re-add this
flag later for convenience.
The parameters for Gtest have yet to be wired up, but can currently be passed
after double dashes, like
lc-compliance -c '<camera_name>' -- --gtest_list_tests
to list all current tests defined. The parameter --gtest_filter can be used to
filter the individual tests or test suites to run. It uses string matching and
globing for the test names so it's easy to run a whole test suite MyTestSuite
with --gtest_list_tests='*MyTestSuite*'.
This is a snippet of a successful run:
[==========] Running 240 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 240 tests from SingleStream
[ RUN ] SingleStream.BalancedSingleCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Raw/1
[20:42:43.656235633] [105482] INFO Camera camera.cpp:915 configuring streams: (0) 1280x720-MJPEG
Camera needs 4 requests, can't test only 1
[ SKIPPED ] SingleStream.BalancedSingleCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Raw/1 (63 ms)
[...]
[ RUN ] SingleStream.Unbalanced/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.2-0bda:5511/Viewfinder/89
[20:51:53.678488142] [105482] INFO Camera camera.cpp:915 configuring streams: (0) 576x360-R8
[ OK ] SingleStream.Unbalanced/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.2-0bda:5511/Viewfinder/89 (6390 ms)
[----------] 240 tests from SingleStream (556412 ms total)
[----------] Global test environment tear-down
[==========] 240 tests from 1 test suite ran. (556412 ms total)
[ PASSED ] 192 tests.
[ SKIPPED ] 48 tests, listed below:
[ SKIPPED ] SingleStream.BalancedSingleCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Raw/1
[ SKIPPED ] SingleStream.BalancedSingleCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Raw/2
[...]
And how a failure looks like:
[ RUN ] SingleStream.BalancedSingleCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Raw/5
[20:55:13.218088360] [106518] INFO Camera camera.cpp:915 configuring streams: (0) 1280x720-MJPEG
../src/lc-compliance/simple_capture.cpp:117: Failure
Expected equality of these values:
captureCount_
Which is: 5
captureLimit_
Which is: 6
[ FAILED ] SingleStream.BalancedSingleCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Raw/5 (318 ms)
The current complete list of tests is shown below for a single camera (each test
will show once for each of the cameras, which makes the list very lengthy if
multiple cameras are present):
SingleStream.
BalancedSingleCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Raw/1
BalancedSingleCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Raw/2
BalancedSingleCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Raw/3
BalancedSingleCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Raw/5
BalancedSingleCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Raw/8
BalancedSingleCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Raw/13
BalancedSingleCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Raw/21
BalancedSingleCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Raw/34
BalancedSingleCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Raw/55
BalancedSingleCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Raw/89
BalancedSingleCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Still/1
BalancedSingleCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Still/2
BalancedSingleCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Still/3
BalancedSingleCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Still/5
BalancedSingleCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Still/8
BalancedSingleCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Still/13
BalancedSingleCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Still/21
BalancedSingleCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Still/34
BalancedSingleCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Still/55
BalancedSingleCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Still/89
BalancedSingleCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Video/1
BalancedSingleCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Video/2
BalancedSingleCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Video/3
BalancedSingleCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Video/5
BalancedSingleCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Video/8
BalancedSingleCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Video/13
BalancedSingleCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Video/21
BalancedSingleCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Video/34
BalancedSingleCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Video/55
BalancedSingleCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Video/89
BalancedSingleCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Viewfinder/1
BalancedSingleCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Viewfinder/2
BalancedSingleCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Viewfinder/3
BalancedSingleCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Viewfinder/5
BalancedSingleCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Viewfinder/8
BalancedSingleCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Viewfinder/13
BalancedSingleCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Viewfinder/21
BalancedSingleCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Viewfinder/34
BalancedSingleCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Viewfinder/55
BalancedSingleCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Viewfinder/89
BalancedMultiCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Raw/1
BalancedMultiCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Raw/2
BalancedMultiCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Raw/3
BalancedMultiCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Raw/5
BalancedMultiCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Raw/8
BalancedMultiCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Raw/13
BalancedMultiCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Raw/21
BalancedMultiCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Raw/34
BalancedMultiCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Raw/55
BalancedMultiCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Raw/89
BalancedMultiCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Still/1
BalancedMultiCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Still/2
BalancedMultiCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Still/3
BalancedMultiCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Still/5
BalancedMultiCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Still/8
BalancedMultiCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Still/13
BalancedMultiCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Still/21
BalancedMultiCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Still/34
BalancedMultiCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Still/55
BalancedMultiCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Still/89
BalancedMultiCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Video/1
BalancedMultiCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Video/2
BalancedMultiCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Video/3
BalancedMultiCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Video/5
BalancedMultiCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Video/8
BalancedMultiCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Video/13
BalancedMultiCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Video/21
BalancedMultiCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Video/34
BalancedMultiCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Video/55
BalancedMultiCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Video/89
BalancedMultiCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Viewfinder/1
BalancedMultiCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Viewfinder/2
BalancedMultiCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Viewfinder/3
BalancedMultiCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Viewfinder/5
BalancedMultiCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Viewfinder/8
BalancedMultiCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Viewfinder/13
BalancedMultiCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Viewfinder/21
BalancedMultiCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Viewfinder/34
BalancedMultiCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Viewfinder/55
BalancedMultiCycle/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Viewfinder/89
Unbalanced/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Raw/1
Unbalanced/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Raw/2
Unbalanced/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Raw/3
Unbalanced/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Raw/5
Unbalanced/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Raw/8
Unbalanced/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Raw/13
Unbalanced/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Raw/21
Unbalanced/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Raw/34
Unbalanced/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Raw/55
Unbalanced/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Raw/89
Unbalanced/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Still/1
Unbalanced/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Still/2
Unbalanced/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Still/3
Unbalanced/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Still/5
Unbalanced/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Still/8
Unbalanced/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Still/13
Unbalanced/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Still/21
Unbalanced/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Still/34
Unbalanced/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Still/55
Unbalanced/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Still/89
Unbalanced/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Video/1
Unbalanced/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Video/2
Unbalanced/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Video/3
Unbalanced/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Video/5
Unbalanced/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Video/8
Unbalanced/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Video/13
Unbalanced/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Video/21
Unbalanced/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Video/34
Unbalanced/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Video/55
Unbalanced/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Video/89
Unbalanced/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Viewfinder/1
Unbalanced/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Viewfinder/2
Unbalanced/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Viewfinder/3
Unbalanced/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Viewfinder/5
Unbalanced/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Viewfinder/8
Unbalanced/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Viewfinder/13
Unbalanced/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Viewfinder/21
Unbalanced/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Viewfinder/34
Unbalanced/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Viewfinder/55
Unbalanced/\_SB_.PCI0.XHC_.RHUB.HS09-9:1.0-0bda:5511/Viewfinder/89
Changes in v2:
- Added patch 1 to make Camera::stop() idempotent as well from [1].
- Made test registration dynamic in patch 3
- Some other minor fixes
[1] https://lists.libcamera.org/pipermail/libcamera-devel/2021-April/019735.html
v1: https://lists.libcamera.org/pipermail/libcamera-devel/2021-April/019702.html
Nícolas F. R. A. Prado (3):
libcamera: camera: Make stop() idempotent
lc-compliance: Make SimpleCapture::stop() idempotent
lc-compliance: Refactor using Googletest
src/lc-compliance/main.cpp | 99 +++++++++---------
src/lc-compliance/meson.build | 3 +
src/lc-compliance/simple_capture.cpp | 95 +++++++----------
src/lc-compliance/simple_capture.h | 8 +-
src/lc-compliance/single_stream.cpp | 151 ++++++++++++++-------------
src/lc-compliance/tests.h | 15 ++-
src/libcamera/camera.cpp | 20 +++-
7 files changed, 199 insertions(+), 192 deletions(-)
--
2.31.1
More information about the libcamera-devel
mailing list