The GitLab Continuous Integration test stage jobs can fail like this:
$ make check
...
Making check in help
make[1]: Entering directory `/builds/mfleetwo/gparted/help'
...
xmllint --noout --xinclude --dtdvalid 'http://scrollkeeper.sourceforge.net/dtds/scrollkeeper-omf-1.0/scrollkeeper-omf.dtd' gparted-C.omf
warning: failed to load external entity "http://scrollkeeper.sourceforge.net/dtds/scrollkeeper-omf-1.0/scrollkeeper-omf.dtd"
Could not parse DTD http://scrollkeeper.sourceforge.net/dtds/scrollkeeper-omf-1.0/scrollkeeper-omf.dtd
xmllint --noout --xinclude --dtdvalid 'http://scrollkeeper.sourceforge.net/dtds/scrollkeeper-omf-1.0/scrollkeeper-omf.dtd' gparted-cs.omf
...
make[1]: *** [check-doc-omf] Error 2
make[1]: Leaving directory `/builds/mfleetwo/gparted/help'
make: *** [check-recursive] Error 1
ERROR: Job failed: exit code 1
It fails when the scrollkeeper.sourceforge.net site reports that
SourceForge is undergoing maintenance or is temporarily unavailable. I
have seen this occur on 3 separate occasions in the last 4 weeks since I
started experimenting with GitLab CI, which is rather too often.
Xmllint comes from the GNOME 2 gnome-doc-utils.make rules used to build
and validate GNOME 2 documentation.
Fragment of useful Debian bug report 730688:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=730688
--disable-scrollkeeper requieres scrollkeeper installed
"You can reproduce the problem in mdbtoools version 0.7.1-1 with no
network, and rarian-compat not installed.
When the network is available, buildd downloads the DTD for checks.
When there is no network, gnome-doc-utils fails.
"
Fix by:
(1) adding the rarian-compat package to the CI Docker images, which
provides a local copy of the scrollkeeper-omf.dtd file;
(2) adding the xmllint --nonet option to prevent fetching of DTDs
remotely.
With reference to earlier commit:
0eb9f1fcfb
Reduce dependency on scrollkeeper (#743318)
That commit allowed GParted to be installed on GNOME 3 desktops without
requiring rarian-compat package to be installed. This commit adds
rarian-compat to the CI images so that 'make check' can succeed without
accessing the Internet. Just the intricate path to continue to build
and test a GNOME 2 application in a world of GNOME 3 desktops with
beginning to be reduced backward compatibility.
Closes#9 - CI test jobs occasionally fail with xmllint not loading
external entity http://scrollkeeper.sourceforce.net/dtds/
scrollkeeper-omf-1.0/scrollkeeper-omf.dtd
Unfortunately parallelising 'make distcheck' causes it to fail like
this:
$ nproc=`grep -c '^processor' /proc/cpuinfo` || nproc=1
$ echo nproc=$nproc
nproc=8
...
$ make -j $nproc distcheck
...
make[1]: Entering directory '/builds/mfleetwo/gparted/gparted-0.31.0-git/_build/sub'
ERROR: files left after uninstall:
./share/icons/hicolor/icon-theme.cache
Makefile:896: recipe for target 'distuninstallcheck' failed
make[1]: Leaving directory '/builds/mfleetwo/gparted/gparted-0.31.0-git/_build/sub'
make[1]: *** [distuninstallcheck] Error 1
make: *** [distcheck] Error 1
Makefile:840: recipe for target 'distcheck' failed
ERROR: Job failed: exit code 1
Therefore go back to serial 'make distcheck'.
Closes!6 - Reduce the time taken by the GitLab CI jobs
Reduce the time taken by the GitLab Continuous Integration jobs by
parallelising make to use all available CPUs in the Docker CI image
when it is building GParted code. This includes 'make diskcheck'
because that also does a second build of the GParted code in a separate
subdirectory.
Closes!6 - Reduce the time taken by the GitLab CI jobs
Prepare the GitLab Continuous Integration configuration for also
building and testing GParted on a Ubuntu image. The definition of the
image and before_script, which so far specify the CentOS Docker image
and how to install the required RPM packages, need to move from being
top level nodes to being defined per job. Namely within jobs
'centos_build' and 'centos_test'.
To avoid duplicating various nodes within multiple jobs, YAML anchors
(&LABEL) and references (*LABEL) are used. They are defined in ignored
jobs, job names starting with a dot (.).
Closes!4 - Add GitLab CI jobs to build and test GParted
Ready for adding additional Continuous Integration jobs using different
distribution Docker images. Rename thus:
build -> centos_build
test -> centos_test
Closes!4 - Add GitLab CI jobs to build and test GParted
Fragment of the tests/test-suite.log from the Docker CI image showing
details of the unit test failure:
Running main() from gtest_main.cc
[==========] Running 26 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 26 tests from BlockSpecialTest
...
[ RUN ] BlockSpecialTest.NamedBlockSpecialObjectBySymlinkMatches
test_BlockSpecial.cc:137: Failure
Failed
get_link_name(): Failed to open directory '/dev/disk/by-id'
test_BlockSpecial.cc:168: Failure
Failed
follow_link_name(): Failed to resolve symbolic link ''
test_BlockSpecial.cc:255: Failure
Expected: (lnk.m_name.c_str()) != (bs.m_name.c_str()), actual: "" vs ""
[ FAILED ] BlockSpecialTest.NamedBlockSpecialObjectBySymlinkMatches (0 ms)
...
[==========] 26 tests from 1 test case ran. (1 ms total)
[ PASSED ] 25 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] BlockSpecialTest.NamedBlockSpecialObjectBySymlinkMatches
1 FAILED TEST
So the code is trying to find a symbolic link to a block device to use
in the test. It is trying to read the directory /dev/disk/by-id to find
a symbolic link, but the directory doesn't exist in the Docker CI image.
The used directory was recently changed [1] to use one which existed on
all distributions. Docker images don't even have the /dev/disk
directory. Exclude just this specific test.
[1] 7fe4148074
Use /dev/disk/by-id/ to get device symlink in test_BlockSpecial
Closes!4 - Add GitLab CI jobs to build and test GParted
Recursively list all the files below /dev as part of the 'test' job as
certain block device names are needed by the failing test_BlockSpecial
unit test.
The artifact captures all the files from the directory in which the CI
script runs to build and test GParted. It creates a ZIP file which can
be downloaded after the job finishes, whether the job succeeds of fails.
This is to capture logs from the failure of the test_BlockSpecial unit
test.
Closes!4 - Add GitLab CI jobs to build and test GParted
Add GitLab Continuous Integration job named 'test' which runs the
GParted unit tests and distcheck. Note that the job starts from a fresh
official CentOS Docker image so also has to rebuild GParted too.
So far this job fails on unit test test_BlockSpecial. Fragment of the
CI job log:
make check-TESTS
make[2]: Entering directory `/builds/mfleetwo/gparted/tests'
make[3]: Entering directory `/builds/mfleetwo/gparted/tests'
PASS: test_dummy
FAIL: test_BlockSpecial
PASS: test_PasswordRAMStore
PASS: test_PipeCapture
make[4]: Entering directory `/builds/mfleetwo/gparted/tests'
make[4]: Nothing to be done for `all'.
make[4]: Leaving directory `/builds/mfleetwo/gparted/tests'
============================================================================
Testsuite summary for gparted 0.31.0-git
============================================================================
# TOTAL: 4
# PASS: 3
# SKIP: 0
# XFAIL: 0
# FAIL: 1
# XPASS: 0
# ERROR: 0
============================================================================
See tests/test-suite.log
Please report to https://bugzilla.gnome.org/enter_bug.cgi?product=gparted
============================================================================
Closes!4 - Add GitLab CI jobs to build and test GParted
Initial GitLab Continuous Integration configuration with a single job
named 'build' which just confirms GParted can be built and installed on
the latest official CentOS Docker image.
Closes!4 - Add GitLab CI jobs to build and test GParted
Mike Fleetwood