[libcamera-ci] [RFC PATCH v1 3/3] Add job to run lc-compliance on the virtual pipeline handler

[Laurent Pinchart]

On Mon, Dec 16, 2024 at 11:15:33AM +0100, Barnabás Pőcze wrote:
> 2024. 12. 15. 20:21 keltezéssel, Laurent Pinchart írta:
> > On Thu, Dec 12, 2024 at 07:16:55PM +0100, Barnabás Pőcze wrote:
> >> Add a new job "lc-compliance:virtual" that uses the build artifacts
> >> produced by the "build-test:debug" job to run lc-compliance in a
> >> virtual machine on the "Virtual0" camera.
> >>
> >> The `force_fallback_for=gtest` option is needed because `cpp_debustl`
> >> makes ABI incompatible changes in the STL containers, so googletest
> >> also needs to be compiled with these changes.
> >>
> >> Signed-off-by: Barnabás Pőcze <barnabas.pocze at ideasonboard.com>
> >> ---
> >>   .gitlab-ci/test-lc-compliance.sh | 27 +++++++++++++++++++++++++++
> >>   gitlab-ci.yml                    | 18 ++++++++++++++++--
> >>   2 files changed, 43 insertions(+), 2 deletions(-)
> >>   create mode 100755 .gitlab-ci/test-lc-compliance.sh
> >>
> >> diff --git a/.gitlab-ci/test-lc-compliance.sh b/.gitlab-ci/test-lc-compliance.sh
> >> new file mode 100755
> >> index 0000000..e6697a6
> >> --- /dev/null
> >> +++ b/.gitlab-ci/test-lc-compliance.sh
> >> @@ -0,0 +1,27 @@
> >> +#!/bin/bash
> > 
> > Please add an SPDX comment block, see other scripts for examples.
> > 
> >> +
> >> +set -e
> >> +
> >> +source "$(dirname "$0")/lib.sh"
> >> +
> >> +libcamera_compliance() {
> >> +	echo "Running libcamera compliance tests in a qemu VM"
> >> +
> >> +	virtme-ng \
> >> +		--verbose \
> >> +		--skip-modules \
> >> +		--force-9p \
> >> +		--rwdir "$PWD/build" \
> >> +		--run /opt/linux/bzImage \
> >> +		--exec "LIBCAMERA_LOG_LEVELS=*:DEBUG ./build/src/apps/lc-compliance/lc-compliance -c Virtual0; echo \\\$? > ./build/.test-status"
> >> +
> >> +	local status=$(cat build/.test-status)
> >> +	echo "Test result exit state: $status"
> >> +	rm build/.test-status
> >> +
> >> +	if [[ $status != 0 ]] ; then
> >> +		exit $status
> >> +	fi
> >> +}
> >> +
> >> +run libcamera_compliance
> >> diff --git a/gitlab-ci.yml b/gitlab-ci.yml
> >> index c7448b8..5e9e309 100644
> >> --- a/gitlab-ci.yml
> >> +++ b/gitlab-ci.yml
> >> @@ -382,9 +382,10 @@ build-test:debug:
> >>         -D cam=disabled
> >>         -D cpp_debugstl=true
> >>         -D documentation=disabled
> >> +      -D force_fallback_for=gtest
> >>         -D gstreamer=enabled
> >> -      -D lc-compliance=disabled
> >> -      -D pipelines=[]
> >> +      -D lc-compliance=enabled
> >> +      -D pipelines=virtual
> > 
> >        -D pipelines=[virtual]
> 
> I am fairly certain `pipelines=virtual` should work, or is this
> a coding style preference?

It's a style preference, as the option is an array. I should have made
that clearer, sorry.

> As far as I see tings are not consistent
> unfortunately, `build-libcamera-cros.sh` does not use the full
> array notation, while the other occurrences do.

Maybe something to fix ? :-)

> > but I think that's not strictly needed, as the virtual pipeline handler
> > should be selected automatically when the test option is enabled.
> 
> You're right. So should I remove it?

I'm fine dropping it (or to be exact keeping "-D pipelines=[]").

> >>         -D pycamera=enabled
> >>         -D qcam=disabled
> >>         -D test=true
> >> @@ -420,3 +421,16 @@ test-unit:
> >>     # artifacts:
> >>     #   reports:
> >>     #     junit: build/meson-logs/testlog.junit.xml
> >> +
> >> +lc-compliance:virtual:
> >> +  extends:
> >> +    - .fdo.distribution-image at debian
> >> +    - .libcamera-ci.debian:12
> >> +    - .libcamera-ci.scripts
> >> +  stage: test
> >> +  needs:
> >> +    - job: build-test:debug
> >> +  tags:
> >> +    - kvm
> >> +  script:
> >> +    - $CI_PROJECT_DIR/.gitlab-ci/test-lc-compliance.sh
> > 
> > I think gtest support generating junit reports. Can we do that, and
> > package the report in artifacts ?
> 
> Indeed, it can:
> https://gitlab.freedesktop.org/pobrn/libcamera/-/pipelines/1330595/test_report?job_name=lc-compliance%3Avirtual

-- 
Regards,

Laurent Pinchart