[libcamera-ci] [RFC PATCH v1 3/3] Add job to run lc-compliance on the virtual pipeline handler

[Laurent Pinchart]

Hi Barnabás,

Thank you for the patch.

On Thu, Dec 12, 2024 at 07:16:55PM +0100, Barnabás Pőcze wrote:
> Add a new job "lc-compliance:virtual" that uses the build artifacts
> produced by the "build-test:debug" job to run lc-compliance in a
> virtual machine on the "Virtual0" camera.
> 
> The `force_fallback_for=gtest` option is needed because `cpp_debustl`
> makes ABI incompatible changes in the STL containers, so googletest
> also needs to be compiled with these changes.
> 
> Signed-off-by: Barnabás Pőcze <barnabas.pocze at ideasonboard.com>
> ---
>  .gitlab-ci/test-lc-compliance.sh | 27 +++++++++++++++++++++++++++
>  gitlab-ci.yml                    | 18 ++++++++++++++++--
>  2 files changed, 43 insertions(+), 2 deletions(-)
>  create mode 100755 .gitlab-ci/test-lc-compliance.sh
> 
> diff --git a/.gitlab-ci/test-lc-compliance.sh b/.gitlab-ci/test-lc-compliance.sh
> new file mode 100755
> index 0000000..e6697a6
> --- /dev/null
> +++ b/.gitlab-ci/test-lc-compliance.sh
> @@ -0,0 +1,27 @@
> +#!/bin/bash

Please add an SPDX comment block, see other scripts for examples.

> +
> +set -e
> +
> +source "$(dirname "$0")/lib.sh"
> +
> +libcamera_compliance() {
> +	echo "Running libcamera compliance tests in a qemu VM"
> +
> +	virtme-ng \
> +		--verbose \
> +		--skip-modules \
> +		--force-9p \
> +		--rwdir "$PWD/build" \
> +		--run /opt/linux/bzImage \
> +		--exec "LIBCAMERA_LOG_LEVELS=*:DEBUG ./build/src/apps/lc-compliance/lc-compliance -c Virtual0; echo \\\$? > ./build/.test-status"
> +
> +	local status=$(cat build/.test-status)
> +	echo "Test result exit state: $status"
> +	rm build/.test-status
> +
> +	if [[ $status != 0 ]] ; then
> +		exit $status
> +	fi
> +}
> +
> +run libcamera_compliance
> diff --git a/gitlab-ci.yml b/gitlab-ci.yml
> index c7448b8..5e9e309 100644
> --- a/gitlab-ci.yml
> +++ b/gitlab-ci.yml
> @@ -382,9 +382,10 @@ build-test:debug:
>        -D cam=disabled
>        -D cpp_debugstl=true
>        -D documentation=disabled
> +      -D force_fallback_for=gtest
>        -D gstreamer=enabled
> -      -D lc-compliance=disabled
> -      -D pipelines=[]
> +      -D lc-compliance=enabled
> +      -D pipelines=virtual

      -D pipelines=[virtual]

but I think that's not strictly needed, as the virtual pipeline handler
should be selected automatically when the test option is enabled.

>        -D pycamera=enabled
>        -D qcam=disabled
>        -D test=true
> @@ -420,3 +421,16 @@ test-unit:
>    # artifacts:
>    #   reports:
>    #     junit: build/meson-logs/testlog.junit.xml
> +
> +lc-compliance:virtual:
> +  extends:
> +    - .fdo.distribution-image at debian
> +    - .libcamera-ci.debian:12
> +    - .libcamera-ci.scripts
> +  stage: test
> +  needs:
> +    - job: build-test:debug
> +  tags:
> +    - kvm
> +  script:
> +    - $CI_PROJECT_DIR/.gitlab-ci/test-lc-compliance.sh

I think gtest support generating junit reports. Can we do that, and
package the report in artifacts ?

-- 
Regards,

Laurent Pinchart