[libcamera-ci] [RFC PATCH v1 3/3] Add job to run lc-compliance on the virtual pipeline handler

[Barnabás Pőcze]

Hi


2024. 12. 15. 20:21 keltezéssel, Laurent Pinchart írta:
> Hi Barnabás,
> 
> Thank you for the patch.
> 
> On Thu, Dec 12, 2024 at 07:16:55PM +0100, Barnabás Pőcze wrote:
>> Add a new job "lc-compliance:virtual" that uses the build artifacts
>> produced by the "build-test:debug" job to run lc-compliance in a
>> virtual machine on the "Virtual0" camera.
>>
>> The `force_fallback_for=gtest` option is needed because `cpp_debustl`
>> makes ABI incompatible changes in the STL containers, so googletest
>> also needs to be compiled with these changes.
>>
>> Signed-off-by: Barnabás Pőcze <barnabas.pocze at ideasonboard.com>
>> ---
>>   .gitlab-ci/test-lc-compliance.sh | 27 +++++++++++++++++++++++++++
>>   gitlab-ci.yml                    | 18 ++++++++++++++++--
>>   2 files changed, 43 insertions(+), 2 deletions(-)
>>   create mode 100755 .gitlab-ci/test-lc-compliance.sh
>>
>> diff --git a/.gitlab-ci/test-lc-compliance.sh b/.gitlab-ci/test-lc-compliance.sh
>> new file mode 100755
>> index 0000000..e6697a6
>> --- /dev/null
>> +++ b/.gitlab-ci/test-lc-compliance.sh
>> @@ -0,0 +1,27 @@
>> +#!/bin/bash
> 
> Please add an SPDX comment block, see other scripts for examples.
> 
>> +
>> +set -e
>> +
>> +source "$(dirname "$0")/lib.sh"
>> +
>> +libcamera_compliance() {
>> +	echo "Running libcamera compliance tests in a qemu VM"
>> +
>> +	virtme-ng \
>> +		--verbose \
>> +		--skip-modules \
>> +		--force-9p \
>> +		--rwdir "$PWD/build" \
>> +		--run /opt/linux/bzImage \
>> +		--exec "LIBCAMERA_LOG_LEVELS=*:DEBUG ./build/src/apps/lc-compliance/lc-compliance -c Virtual0; echo \\\$? > ./build/.test-status"
>> +
>> +	local status=$(cat build/.test-status)
>> +	echo "Test result exit state: $status"
>> +	rm build/.test-status
>> +
>> +	if [[ $status != 0 ]] ; then
>> +		exit $status
>> +	fi
>> +}
>> +
>> +run libcamera_compliance
>> diff --git a/gitlab-ci.yml b/gitlab-ci.yml
>> index c7448b8..5e9e309 100644
>> --- a/gitlab-ci.yml
>> +++ b/gitlab-ci.yml
>> @@ -382,9 +382,10 @@ build-test:debug:
>>         -D cam=disabled
>>         -D cpp_debugstl=true
>>         -D documentation=disabled
>> +      -D force_fallback_for=gtest
>>         -D gstreamer=enabled
>> -      -D lc-compliance=disabled
>> -      -D pipelines=[]
>> +      -D lc-compliance=enabled
>> +      -D pipelines=virtual
> 
>        -D pipelines=[virtual]

I am fairly certain `pipelines=virtual` should work, or is this
a coding style preference? As far as I see tings are not consistent
unfortunately, `build-libcamera-cros.sh` does not use the full
array notation, while the other occurrences do.


> 
> but I think that's not strictly needed, as the virtual pipeline handler
> should be selected automatically when the test option is enabled.

You're right. So should I remove it?


> 
>>         -D pycamera=enabled
>>         -D qcam=disabled
>>         -D test=true
>> @@ -420,3 +421,16 @@ test-unit:
>>     # artifacts:
>>     #   reports:
>>     #     junit: build/meson-logs/testlog.junit.xml
>> +
>> +lc-compliance:virtual:
>> +  extends:
>> +    - .fdo.distribution-image at debian
>> +    - .libcamera-ci.debian:12
>> +    - .libcamera-ci.scripts
>> +  stage: test
>> +  needs:
>> +    - job: build-test:debug
>> +  tags:
>> +    - kvm
>> +  script:
>> +    - $CI_PROJECT_DIR/.gitlab-ci/test-lc-compliance.sh
> 
> I think gtest support generating junit reports. Can we do that, and
> package the report in artifacts ?
> 

Indeed, it can:
https://gitlab.freedesktop.org/pobrn/libcamera/-/pipelines/1330595/test_report?job_name=lc-compliance%3Avirtual


Regards,
Barnabás Pőcze