[libcamera-ci] [RFC PATCH v4] Add job to run lc-compliance on the virtual pipeline handler

[Laurent Pinchart]

Hi Barnabás,

Thank you for the patch.

On Thu, Dec 19, 2024 at 02:09:32PM +0100, Barnabás Pőcze wrote:
> Add a new job named `test-lc-compliance:virtual` that uses the
> build artifacts created by the `build-package:debug` job
> on amd64 to run `lc-compliance` on the "Virtual0" camera
> in a virtual machine.
> 
> Signed-off-by: Barnabás Pőcze <barnabas.pocze at ideasonboard.com>
> ---
> This needs libcamera changes to work and to run in reasonable time:
>   * https://patchwork.libcamera.org/project/libcamera/list/?series=4874
>   * https://patchwork.libcamera.org/patch/22379/
>   * https://patchwork.libcamera.org/patch/22406/
> 
> Changes in v4:
>   * add `set -x` to see the full commands run inside the virtual machine
>   * activate installed example configuration of the virtual pipeline handler
> 
> Changes in v2:
>   * publish lc-compliance results to gitlab
>   * add and use script to install libcamera binary package
> ---
>  .gitlab-ci/test-lc-compliance.sh  | 40 +++++++++++++++++++++++++++++++
>  .gitlab-ci/unpackage-libcamera.sh | 17 +++++++++++++
>  gitlab-ci.yml                     | 22 ++++++++++++++++-
>  3 files changed, 78 insertions(+), 1 deletion(-)
>  create mode 100755 .gitlab-ci/test-lc-compliance.sh
>  create mode 100755 .gitlab-ci/unpackage-libcamera.sh
> 
> diff --git a/.gitlab-ci/test-lc-compliance.sh b/.gitlab-ci/test-lc-compliance.sh
> new file mode 100755
> index 0000000..6208723
> --- /dev/null
> +++ b/.gitlab-ci/test-lc-compliance.sh
> @@ -0,0 +1,40 @@
> +#!/bin/bash
> +
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +# SPDX-FileCopyrightText: © 2024 Google Inc.
> +
> +set -e
> +
> +source "$(dirname "$0")/lib.sh"
> +
> +libcamera_compliance() {
> +	echo "Running libcamera compliance tests in a qemu VM"
> +
> +	virtme-ng \
> +		--verbose \
> +		--skip-modules \
> +		--force-9p \
> +		--rwdir "$PWD/build" \
> +		--run /opt/linux/bzImage \
> +		--exec " \
> +			set -x; \
> +			mv /usr/local/share/libcamera/pipeline/virtual/virtual.yaml.example \
> +			   /usr/local/share/libcamera/pipeline/virtual/virtual.yaml; \
> +			LIBCAMERA_LOG_LEVELS=*:DEBUG \
> +			ASAN_OPTIONS=halt_on_error=1:abort_on_error=1:print_summary=1 \
> +			UBSAN_OPTIONS=halt_on_error=1:abort_on_error=1:print_summary=1:print_stacktrace=1 \
> +			GTEST_OUTPUT=xml:./build/lc-compliance-report.xml \
> +			lc-compliance -c Virtual0; \
> +			echo \\\$? > ./build/.test-status; \
> +		"
> +
> +	local status=$(cat build/.test-status)
> +	echo "Test result exit state: $status"
> +	rm build/.test-status
> +
> +	if [[ $status != 0 ]] ; then
> +		exit $status
> +	fi
> +}
> +
> +run libcamera_compliance
> diff --git a/.gitlab-ci/unpackage-libcamera.sh b/.gitlab-ci/unpackage-libcamera.sh
> new file mode 100755
> index 0000000..9eb3192
> --- /dev/null
> +++ b/.gitlab-ci/unpackage-libcamera.sh
> @@ -0,0 +1,17 @@
> +#!/bin/bash
> +
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +# SPDX-FileCopyrightText: © 2024 Google Inc.
> +
> +set -e
> +
> +source "$(dirname "$0")/lib.sh"
> +
> +libcamera_unpackage() {
> +	echo "Unpackage libcamera binaries"
> +
> +	tar -xvf libcamera-${CI_COMMIT_SHA}.tar.xz -C /
> +	ldconfig -v
> +}
> +
> +run libcamera_unpackage
> diff --git a/gitlab-ci.yml b/gitlab-ci.yml
> index d268285..ec19f55 100644
> --- a/gitlab-ci.yml
> +++ b/gitlab-ci.yml
> @@ -280,7 +280,7 @@ build-package:debug:
>        -D b_sanitize=address,undefined
>        -D cpp_debugstl=true
>        -D force_fallback_for=['gtest']
> -      -D pipelines=['auto']
> +      -D pipelines=['auto','virtual']
>        -D auto_features=disabled
>        -D lc-compliance=enabled
>        -D test=false
> @@ -411,3 +411,23 @@ test-unit:
>    artifacts:
>      reports:
>        junit: build/meson-logs/testlog.junit.xml
> +
> +test-lc-compliance:virtual:
> +  extends:
> +    - .fdo.distribution-image at debian
> +    - .libcamera-ci.debian:12
> +    - .libcamera-ci.scripts
> +  stage: test
> +  needs:
> +    - job: build-package:debug
> +      parallel:
> +        matrix:
> +          - ARCH: amd64

Should we add

    - job: container-debian:12
      artifacts: false

here ? I know build-package:debug depends on it, but I think it's a good
practice to have an explicit dependency on the container needed to run
this job.

The rest looks good, so

Reviewed-by: Laurent Pinchart <laurent.pinchart at ideasonboard.com>

If you agree with the comment I can add the dependency when applying the
patch, there's no need to resubmit.

> +  tags:
> +    - kvm
> +  script:
> +    - $CI_PROJECT_DIR/.gitlab-ci/unpackage-libcamera.sh
> +    - $CI_PROJECT_DIR/.gitlab-ci/test-lc-compliance.sh
> +  artifacts:
> +    reports:
> +      junit: build/lc-compliance-report.xml

-- 
Regards,

Laurent Pinchart