gparted/.gitlab-ci.yml

127 lines
4.2 KiB
YAML
Raw Normal View History

stages:
- build
- test
.alpine_image_template: &alpine_image_definition
# Use official Alpine image https://hub.docker.com/_/alpine/.
image: alpine:3.17
before_script:
- cat /proc/version
- cat /etc/os-release
- apk update
- apk add gnome-common yelp-tools automake autoconf glib-dev libtool g++
parted-dev gtkmm3-dev itstool make git polkit-dev
# Extra packages only needed during the test stage.
- apk add btrfs-progs btrfs-progs-extra e2fsprogs e2fsprogs-extra exfatprogs
dosfstools mtools f2fs-tools jfsutils cryptsetup lvm2 udftools
Speed up signature erasing unit test in Alpine Linux CI test job (#220) Each test in test_EraseFileSystemSignatures is taking just over 10 seconds to run in the Alpine Linux CI image: [ RUN ] EraseFileSystemSignaturesTest.IntelSoftwareRAIDAligned [ OK ] EraseFileSystemSignaturesTest.IntelSoftwareRAIDAligned (10045 ms) [ RUN ] EraseFileSystemSignaturesTest.IntelSoftwareRAIDUnaligned ... [ FAILED ] EraseFileSystemSignaturesTest.IntelSoftwareRAIDUnaligned (10048 ms) [----------] 2 tests from EraseFileSystemSignaturesTest (20093 ms total) [----------] Global test environment tear-down [==========] 2 tests from 1 test case ran. (20093 ms total) This is because the udevadm command is not found and so settle_device() waits for 10 seconds in this call chain: erase_filesystem_signatures() settle_device(SETTLE_DEVICE_APPLY_MAX_WAIT_SECONDS) sleep(10) Install udevadm command into the Alpine Linux CI job docker image to fix this. Now it's on a par with the time taken in the other distro CI test jobs: [ RUN ] EraseFileSystemSignaturesTest.IntelSoftwareRAIDAligned [ OK ] EraseFileSystemSignaturesTest.IntelSoftwareRAIDAligned (417 ms) [ RUN ] EraseFileSystemSignaturesTest.IntelSoftwareRAIDUnaligned ... [ FAILED ] EraseFileSystemSignaturesTest.IntelSoftwareRAIDUnaligned (165 ms) [----------] 2 tests from EraseFileSystemSignaturesTest (582 ms total) [----------] Global test environment tear-down [==========] 2 tests from 1 test case ran. (582 ms total) Closes #220 - Format to Cleared not clearing "pdc" ataraid signature
2023-02-08 02:51:33 -07:00
xfsprogs xfsprogs-extra xvfb-run kmod gzip eudev
.rocky_image_template: &rocky_image_definition
# Use official Rocky Linux image https://hub.docker.com/_/rockylinux/.
image: rockylinux:8
before_script:
- cat /proc/version
- cat /etc/os-release
- dnf update -y
# Enable PowerTools repo for development packages.
- dnf install -y 'dnf-command(config-manager)'
- dnf config-manager --set-enabled powertools
- dnf install -y which gnome-common yelp-tools glib2-devel gcc-c++
libuuid-devel parted-devel gtkmm30-devel make polkit file
polkit-devel gettext-devel
Add initial create ext2 only FileSystem interface class test (!49) This is the first step of adding testing of the derived FileSystem interface classes which call the file system specific executables. Rather than mocking command execution and returned output the tests run the real commands, effectively making this integration testing. Test case setup determines the file system supported actions using get_filesystem_support() and individual tests are skipped if a feature is not supported, just as GParted does for it's actions. Each test creates it's own sparse image file and a fresh file system, performs a test on one FileSystem interface call and deletes the image file. This makes each test independent and allows them to run as a non-root user, provided the file system command itself doesn't require root. Errors reported for a failed interface call will include the GParted OperationDetails, which in turn includes the file system specific command used and stdout and stderr from it's execution. For example, temporarily breaking the test code to create a 10 KiB image file instead of 256 MiB one produces this: $ ./test_ext2 Running main() from test_ext2.cc [==========] Running 1 test from 1 test case. [----------] Global test environment set-up. [----------] 1 test from ext2Test [ RUN ] ext2Test.Create test_ext2.cc:199: Failure Value of: s_ext2_obj->create(m_partition, m_operation_detail) Actual: false Expected: true Operation details: <b><i>mkfs.ext2 -F -L &apos;&apos; &apos;/home/centos/programming/c/gparted/tests/test_ext2.img&apos;</i></b> 00:00:00 (ERROR) <i></i> <i>mke2fs 1.42.9 (28-Dec-2013) /home/centos/programming/c/gparted/tests/test_ext2.img: Not enough space to build proposed filesystem while setting up superblock </i> [ FAILED ] ext2Test.Create (25 ms) [----------] 1 test from ext2Test (25 ms total) [----------] Global test environment tear-down [==========] 1 test from 1 test case ran. (30 ms total) [ PASSED ] 0 tests. [ FAILED ] 1 test, listed below: [ FAILED ] ext2Test.Create 1 FAILED TEST $ echo $? 1 Also as Utils:: and FileSystem::execute_command() are needed, this requires linking with GParted_Core for GParted_Core::mainthread and therefore with most of the non-UI classes in gpartedbin. Closes !49 - Add file system interface tests
2019-07-17 06:08:08 -06:00
# Extra packages only needed during the test stage.
# Install EPEL repo first for exfatprogs and ntfsprogs.
- dnf install -y epel-release
- dnf install -y e2fsprogs dosfstools exfatprogs mtools util-linux
cryptsetup device-mapper lvm2 ntfsprogs udftools xfsprogs
xfsdump xorg-x11-server-Xvfb kmod
.ubuntu_image_template: &ubuntu_image_definition
# Use official Ubuntu image https://hub.docker.com/_/ubuntu/.
image: ubuntu:latest
before_script:
- cat /proc/version
- cat /etc/os-release
Prevent tzdata install hang in GitLab CI Ubuntu image (!60) The Ubuntu based GitLab CI jobs have recently started being terminated after the default 1 hour timeout. Installing / updating packages in the image is updating the tzdata package which is prompting for input which it will never receive, hence the hang. The end of output from the job looks like this: Setting up tzdata (2020a-0ubuntu0.20.04) ... debconf: unable to initialize frontend: Dialog debconf: (TERM is not set, so the dialog frontend is not usable.) debconf: falling back to frontend: Readline Configuring tzdata ------------------ Please select the geographic area in which you live. Subsequent configuration questions will narrow this down by presenting a list of cities, representing the time zones in which they are located. 1. Africa 4. Australia 7. Atlantic 10. Pacific 13. Etc 2. America 5. Arctic 8. Europe 11. SystemV 3. Antarctica 6. Asia 9. Indian 12. US Geographic area: ... ERROR: Job failed: execution took longer than 1h0m0s seconds This is a well known issue [1][2][3]. Probably occurring now because of a new release of tzdata not included in the base Ubuntu image we are using. Fix by telling the underlying dpkg tools this installation is non-interactive. [1] Avoiding user interaction with tzdata when installing certbot in a docker container https://askubuntu.com/questions/909277/avoiding-user-interaction-with-tzdata-when-installing-certbot-in-a-docker-contai [2] How to install tzdata on a ubuntu docker image? https://serverfault.com/questions/949991/how-to-install-tzdata-on-a-ubuntu-docker-image [3] apt-get install tzdata noninteractive https://stackoverflow.com/questions/44331836/apt-get-install-tzdata-noninteractive Closes !60 - Fix GitLab CI job failures following Ubuntu docker image updates
2020-05-22 08:59:35 -06:00
- export DEBIAN_FRONTEND=noninteractive
- apt update
- apt install -y build-essential gnome-common yelp-tools libglib2.0-dev-bin
uuid-dev libparted-dev libgtkmm-3.0-dev policykit-1
libpolkit-gobject-1-dev
Add initial create ext2 only FileSystem interface class test (!49) This is the first step of adding testing of the derived FileSystem interface classes which call the file system specific executables. Rather than mocking command execution and returned output the tests run the real commands, effectively making this integration testing. Test case setup determines the file system supported actions using get_filesystem_support() and individual tests are skipped if a feature is not supported, just as GParted does for it's actions. Each test creates it's own sparse image file and a fresh file system, performs a test on one FileSystem interface call and deletes the image file. This makes each test independent and allows them to run as a non-root user, provided the file system command itself doesn't require root. Errors reported for a failed interface call will include the GParted OperationDetails, which in turn includes the file system specific command used and stdout and stderr from it's execution. For example, temporarily breaking the test code to create a 10 KiB image file instead of 256 MiB one produces this: $ ./test_ext2 Running main() from test_ext2.cc [==========] Running 1 test from 1 test case. [----------] Global test environment set-up. [----------] 1 test from ext2Test [ RUN ] ext2Test.Create test_ext2.cc:199: Failure Value of: s_ext2_obj->create(m_partition, m_operation_detail) Actual: false Expected: true Operation details: <b><i>mkfs.ext2 -F -L &apos;&apos; &apos;/home/centos/programming/c/gparted/tests/test_ext2.img&apos;</i></b> 00:00:00 (ERROR) <i></i> <i>mke2fs 1.42.9 (28-Dec-2013) /home/centos/programming/c/gparted/tests/test_ext2.img: Not enough space to build proposed filesystem while setting up superblock </i> [ FAILED ] ext2Test.Create (25 ms) [----------] 1 test from ext2Test (25 ms total) [----------] Global test environment tear-down [==========] 1 test from 1 test case ran. (30 ms total) [ PASSED ] 0 tests. [ FAILED ] 1 test, listed below: [ FAILED ] ext2Test.Create 1 FAILED TEST $ echo $? 1 Also as Utils:: and FileSystem::execute_command() are needed, this requires linking with GParted_Core for GParted_Core::mainthread and therefore with most of the non-UI classes in gpartedbin. Closes !49 - Add file system interface tests
2019-07-17 06:08:08 -06:00
# Extra packages only needed during the test stage.
- apt install -y btrfs-progs e2fsprogs exfatprogs f2fs-tools dosfstools
mtools hfsutils hfsprogs jfsutils util-linux cryptsetup-bin
dmsetup lvm2 nilfs-tools ntfs-3g reiser4progs reiserfsprogs
udftools xfsprogs xfsdump xvfb kmod
.build_stage_template: &build_stage_definition
stage: build
script:
- ./autogen.sh
- nproc=`grep -c '^processor' /proc/cpuinfo` || nproc=1
- echo nproc=$nproc
- make -j $nproc
- make install
# Save all files on job failure for investigation.
artifacts:
when: on_failure
name: "$CI_PROJECT_NAME-ci-job-$CI_JOB_ID-$CI_JOB_NAME"
untracked: true
paths:
- ./
expire_in: 1 week
.test_stage_template: &test_stage_definition
stage: test
script:
- ./autogen.sh
- nproc=`grep -c '^processor' /proc/cpuinfo` || nproc=1
- echo nproc=$nproc
- make -j $nproc
Exclude unit tests needing losetup in Docker CI image (!59) test_SupportedFileSystems is another unit test that has been failing since 23-Feb-2020 in GNOME GitLab Continuous Integration test jobs. Fragments from tests/test-suite.log from a failed test CI job: FAIL: test_SupportedFileSystems =============================== ... [ RUN ] My/SupportedFileSystemsTest.Create/lvm2pv test_SupportedFileSystems.cc:387: Failure Failed create_loopdev(): Execute: losetup --find --show 'test_SupportedFileSystems.img' losetup: test_SupportedFileSystems.img: failed to set up loop device: No such file or directory create_loopdev(): Losetup failed with exit status 1 create_loopdev(): Failed to create required loop device Error: Could not stat device - No such file or directory. test_SupportedFileSystems.cc:446: Failure Value of: lp_device != NULL Actual: false Expected: true test_SupportedFileSystems.cc:490: Failure Value of: m_fs_object->create(m_partition, m_operation_detail) Actual: false Expected: true Operation details: lvm pvcreate -M 2 '' 00:00:00 (ERROR) WARNING: Failed to connect to lvmetad. Falling back to device scanning. Device not found. detach_loopdev(): Execute: losetup --detach '' losetup: /dev/: detach failed: Inappropriate ioctl for device detach_loopdev(): Losetup failed with exit status 1 detach_loopdev(): Failed to detach loop device. Test NOT affected [ FAILED ] My/SupportedFileSystemsTest.Create/lvm2pv, where GetParam() = 20 (64 ms) ... [ RUN ] My/SupportedFileSystemsTest.CreateAndReadUsage/btrfs test_SupportedFileSystems.cc:387: Failure Failed create_loopdev(): Execute: losetup --find --show 'test_SupportedFileSystems.img' losetup: test_SupportedFileSystems.img: failed to set up loop device: No such file or directory create_loopdev(): Losetup failed with exit status 1 create_loopdev(): Failed to create required loop device Error: Could not stat device - No such file or directory. test_SupportedFileSystems.cc:446: Failure Value of: lp_device != NULL Actual: false Expected: true test_SupportedFileSystems.cc:503: Failure Value of: m_fs_object->create(m_partition, m_operation_detail) Actual: false Expected: true Operation details: mkfs.btrfs -L '' '' 00:00:00 (ERROR) btrfs-progs v4.9.1 See http://btrfs.wiki.kernel.org for more information. ERROR: failed to check size for : No such file or directory detach_loopdev(): Execute: losetup --detach '' losetup: /dev/: detach failed: Inappropriate ioctl for device detach_loopdev(): Losetup failed with exit status 1 detach_loopdev(): Failed to detach loop device. Test NOT affected [ FAILED ] My/SupportedFileSystemsTest.CreateAndReadUsage/btrfs, where GetParam() = 7 (11 ms) All the test_SupportedFileSystems unit tests which need a loop device are failing when running losetup like this: losetup: <IMAGE_NAME>: failed to setup loop device: No such file or directory losetup uses /dev/loop-control [1][2] which no longer exists in the Docker image. However even after creating /dev/loop-control in the image, losetup continues to fail. Also tried stracing losetup but that fails like this, presumably because it is not allowed inside the Docker image: $ strace losetup --find --show /tmp/disk.img strace: ptrace(PTRACE_TRACEME, ...): Operation not permitted +++ exited with 1 +++ For now I have run out of ways to investigate and resolve this, so just exclude the 12 tests which required loop devices. All the tests still execute when run locally outside the restricted GNOME GitLab CI Docker setup. Closes !59 - Fix GNOME GitLab CI test job failures because of missing /dev entries
2020-03-11 01:55:53 -06:00
# Exclude specific unit tests which fail without being able to create
# loop devices in Docker images.
- export GTEST_FILTER=`tests/exclude_loopdev_tests.sh tests/test_SupportedFileSystems.cc`
- echo $GTEST_FILTER
- fgrep -v nodev /proc/filesystems | sort
- cat /proc/partitions
- ls -l /dev
Create block special devices needed by test_BlockSpecial in GitLab CI jobs (!59) From 23-Feb-2020 onwards, GNOME GitLab Continuous Integration test jobs have been failing running unit tests which previously succeeded. With some extra debugging added into test_BlockSpecial to print 'bname' and 'bs' values in the failing tests, here are fragments from tests/test-suite.log for the the test_BlockSpecial failures in a test CI job: FAIL: test_BlockSpecial ======================= ... [ RUN ] BlockSpecialTest.NamedBlockSpecialObjectBlockDevice bname="/dev/sr0" bs=BlockSpecial{"/dev/sr0",0,0} test_BlockSpecial.cc:218: Failure Value of: bs.m_major > 0 || bs.m_minor > 0 Actual: false Expected: true [ FAILED ] BlockSpecialTest.NamedBlockSpecialObjectBlockDevice (0 ms) ... [ RUN ] BlockSpecialTest.TwoNamedBlockSpecialObjectBlockDevices bname1="/dev/sr0" bname2="/dev/sda" bs1=BlockSpecial{"/dev/sr0",0,0} bs2=BlockSpecial{"/dev/sda",0,0} test_BlockSpecial.cc:250: Failure Value of: bs1.m_major != bs2.m_major || bs1.m_minor != bs2.m_minor Actual: false Expected: true [ FAILED ] BlockSpecialTest.TwoNamedBlockSpecialObjectBlockDevices (1 ms) Contents of /proc/partitions inside the Docker image when this test CI job failed: $ cat /proc/partitions major minor #blocks name 11 0 1048575 sr0 8 0 573367448 sda 8 1 573366407 sda1 And the listing of /dev/: $ ls -l /dev/ total 0 lrwxrwxrwx 1 root root 11 Mar 3 09:00 core -> /proc/kcore lrwxrwxrwx 1 root root 13 Mar 3 09:00 fd -> /proc/self/fd crw-rw-rw- 1 root root 1, 7 Mar 3 09:00 full drwxrwxrwt 2 root root 40 Mar 3 09:00 mqueue crw-rw-rw- 1 root root 1, 3 Mar 3 09:00 null lrwxrwxrwx 1 root root 8 Mar 3 09:00 ptmx -> pts/ptmx drwxr-xr-x 2 root root 0 Mar 3 09:00 pts crw-rw-rw- 1 root root 1, 8 Mar 3 09:00 random drwxrwxrwt 2 root root 40 Mar 3 09:00 shm lrwxrwxrwx 1 root root 15 Mar 3 09:00 stderr -> /proc/self/fd/2 lrwxrwxrwx 1 root root 15 Mar 3 09:00 stdin -> /proc/self/fd/0 lrwxrwxrwx 1 root root 15 Mar 3 09:00 stdout -> /proc/self/fd/1 crw-rw-rw- 1 root root 5, 0 Mar 3 09:00 tty crw-rw-rw- 1 root root 1, 9 Mar 3 09:00 urandom crw-rw-rw- 1 root root 1, 5 Mar 3 09:00 zero See how the test_BlockSpecial fixtures are getting major=0 and minor=0 for the block special devices they are testing with. This is happening because there aren't any entries in /dev for those disks and partitions listed in /proc/partitions. Assume that Docker in GNOME GitLab has changed and that unneeded and unwanted devices in /dev are no longer being created inside images. In the test CI jobs execute new script, tests/makedev.sh, to create just the first two block special devices mentioned in /proc/partitions needed by test_BlockSpecial. Closes !59 - Fix GNOME GitLab CI test job failures because of missing /dev entries
2020-03-08 03:34:15 -06:00
# Create needed /dev entries for unit tests in Docker images.
- tests/makedev.sh
- make check
- make distcheck
- fgrep -v nodev /proc/filesystems | sort
# Save all files on job failure for investigation.
artifacts:
when: on_failure
name: "$CI_PROJECT_NAME-ci-job-$CI_JOB_ID-$CI_JOB_NAME"
untracked: true
paths:
- ./
expire_in: 1 week
# Check GParted can be built and installed on Alpine Linux, CentOS and Ubuntu.
alpine_build:
<<: *alpine_image_definition
<<: *build_stage_definition
rocky_build:
<<: *rocky_image_definition
<<: *build_stage_definition
ubuntu_build:
<<: *ubuntu_image_definition
<<: *build_stage_definition
# Check GParted unit tests and distcheck pass on Alpine Linux, CentOS and
# Ubuntu.
alpine_test:
<<: *alpine_image_definition
<<: *test_stage_definition
rocky_test:
<<: *rocky_image_definition
<<: *test_stage_definition
ubuntu_test:
<<: *ubuntu_image_definition
<<: *test_stage_definition