[libcamera-devel] [PATCH] meson: disable tests by default

[Kieran Bingham]

Quoting Laurent Pinchart (2022-09-05 11:35:12)
> On Mon, Sep 05, 2022 at 10:51:40AM +0100, Kieran Bingham wrote:
> > Quoting Umang Jain via libcamera-devel (2022-09-05 07:41:50)
> > > On 9/5/22 3:29 AM, Laurent Pinchart via libcamera-devel wrote:
> > > > On Sun, Sep 04, 2022 at 06:27:15PM +0200, Christian Rauch via libcamera-devel wrote:
> > > >> Compiling with tests adds a significant amount of build targets to the
> > > >> meson build. However, tests are only of interest when a feature has been
> > > >> implemented or for continuous integration.
> > > >> Disable tests by default to reduce the computational resource requirements
> > > >> and the duration of a complete libcamera built.
> > > >>
> > > >> Signed-off-by: Christian Rauch <Rauch.Christian at gmx.de>
> > > > As most people building libcamera won't run tests, this seems reasonable
> > > > to me. I'm however a bit worried that it would lead to lower
> > > > compilation-test coverage, as people may send patches without realizing
> > > > they break compilation of tests. We already have that problem for
> > > 
> > > I'm concerned about this as well.
> > 
> > When I first saw this patch over the weekend, indeed my reaction was
> > "Ahhh" ... but I also agree this is a better default to have off right
> > now.
> > 
> > We would like patches to be submitted after having all tests run, but
> > the majority of people who compile libcamera (currently) may not be
> > developers. We have lots of people compiling on RPi / IPU3 and low power
> > devices directly, and they certainly don't care for running tests.
> > 
> > So to improve user experience, I do think this should be off by default
> > (at least now) and that stance may change depending on our upcoming
> > release strategies ... but developers should be able to enable tests
> > anyway.
> > 
> > Ideally of course these tests will then be enabled and run on any branch
> > push to the common public hosting services, so I think we should
> > integrate a gitlab and github CI script that could build and run the
> > unit tests. (These could skip the kernel related tests initially).
> 
> Many of our tests indeed depend on vimc, vivid and vim2m, so that's a
> blocker for git..b. I'd be happy if we started with test runners that
> performed compile testing only, with all options enabled.

For that I'd really like to continue to investigate (or for anyone else
to continue to investigate) running the tests under a compiled kernel in
either UML (which seemed to be a deadend for linux-media driver support
due to large rabbit holes) - or as a microvm, with a custom minimal
kernel that contains VIMC/VIVID/VIM2M.

It's something I started to research, but haven't been able to complete,
but I beleive would be a way of fully automating the tests with the
virtual drivers. If anyone wants to explore this, I'm happy to share
where I got to on it.

--
Kieran


> 
> > Anyway,
> > 
> > Yes - 
> > 
> > Reviewed-by: Kieran Bingham <kieran.bingham at ideasonboard.com>
> > 
> > > > runtime breakages of tests, and it's most likely be addressed by test
> > > > automation on patches submitted upstream, so I think I'm fine with the
> > > > change.
> > > >
> > > > Reviewed-by: Laurent Pinchart <laurent.pinchart at ideasonboard.com>
> > > 
> > > Reviewed-by: Umang Jain <umang.jain at ideasonboard.com>
> > > 
> > > >> ---
> > > >>   meson_options.txt | 1 +
> > > >>   1 file changed, 1 insertion(+)
> > > >>
> > > >> diff --git a/meson_options.txt b/meson_options.txt
> > > >> index 7a9aecfc..f1d67808 100644
> > > >> --- a/meson_options.txt
> > > >> +++ b/meson_options.txt
> > > >> @@ -47,6 +47,7 @@ option('qcam',
> > > >>
> > > >>   option('test',
> > > >>           type : 'boolean',
> > > >> +        value : false,
> > > >>           description: 'Compile and include the tests')
> > > >>
> > > >>   option('tracing',
> 
> -- 
> Regards,
> 
> Laurent Pinchart