Fragment of a failed CI test job from a GiLab job runner which didn't
allow creation of block special devices looked like:
$ tests/makedev.sh
mknod -m 0660 /dev/sda b 8 0
mknod: '/dev/sda': Operation not permitted
chown: cannot access '/dev/sda': No such file or directory
mknod -m 0660 /dev/sda1 b 8 1
mknod: '/dev/sda1': Operation not permitted
chown: cannot access '/dev/sda1': No such file or directory
mkdir: created directory '/dev/disk'
mkdir: created directory '/dev/disk/by-id/'
'/dev/disk/by-id/gparted-sda' -> '/dev/sda'
test/makedev.sh attempted to create two block devices it wanted for
testing, but that failed with "Operation not permitted". It then
created dangling symbolic link /dev/disk/by-id/gparted-sda -> /dev/sda;
gparted-sda pointed to a name which didn't exist.
Despite the previous commit testing and skipping every test where the
block device doesn't exist this unit test still failed:
[ RUN ] BlockSpecialTest.NamedBlockSpecialObjectBySymlinkMatches
test_BlockSpecial.cc:186: Failure
Failed
follow_link_name(): Failed to resolve symbolic link '/dev/disk/by-id/gparted-sda'
test_BlockSpecial.cc:271: Skip test. Block device '' does not exist
[ FAILED ] BlockSpecialTest.NamedBlockSpecialObjectBySymlinkMatches (0 ms)
The unit test called get_link_name() which read the directory
/dev/disk/by-id and found symbolic link gparted-sda. It then called
follow_link_name() passing /dev/disk/by-id/gparted-sda which used
realpath(3) to get the canonicalised absolute pathname, which includes
following links. But as gparted-sda pointed to a non-existent file it
failed and reported message "Failed to resolve symbolic link ...". Then
after that the unit test skips the non-existent block device, but the
test has already failed at that point.
Fix the unit test by also checking the symbolic link points to an
existing block device before calling follow_link_name() on it. This
works because SKIP_IF_BLOCK_DEVICE_DOESNT_EXIST() uses stat(3), which
follows symbolic links, in it's verification.
Also put SKIP_IF_BLOCK_DEVICE_DOESNT_EXIST() immediately after each
initialisation of a block device name for some sort of consistency with
it's need in this fixed NamedBlockSpecialObjectBySymlinkMatches unit
test.
Closed!113 - Fix occasional GitLab CI test jobs failures on
BlockSpecial unit tests
Since November 2022 test_BlockSpecial has been occasionally failing in
GNOME GitLab Docker CI test jobs like this:
[ RUN ] BlockSpecialTest.NamedBlockSpecialObjectBlockDevice
test_BlockSpecial.cc:216: Failure
Value of: bs.m_major > 0 || bs.m_minor > 0
Actual: false
Expected: true
[ FAILED ] BlockSpecialTest.NamedBlockSpecialObjectBlockDevice (0 ms)
...
[ RUN ] BlockSpecialTest.TwoNamedBlockSpecialObjectBlockDevices
test_BlockSpecial.cc:244: Failure
Value of: bs1.m_major != bs2.m_major || bs1.m_minor != bs2.m_minor
Actual: false
Expected: true
[ FAILED ] BlockSpecialTest.TwoNamedBlockSpecialObjectBlockDevices (0 ms)
[ RUN ] BlockSpecialTest.NamedBlockSpecialObjectBySymlinkMatches
test_BlockSpecial.cc:170: Failure
Failed
follow_link_name(): Failed to resolve symbolic link '/dev/disk/by-id/gparted-sda'
[ FAILED ] BlockSpecialTest.NamedBlockSpecialObjectBySymlinkMatches (0 ms)
...
3 FAILED TESTS
FAIL test_BlockSpecial (exit status: 1)
As identified previously [1] the Docker CI images no longer have any
block devices in /dev. test/makedev.sh script was added to create block
devices test_BlockSpecial needs for it's testing. Now a subset of the
GNOME GitLab job runners additionally prevent creation of block special
device nodes. test/makedev.sh reports this:
$ tests/makedev.sh
mknod -m 0660 /dev/sda b 8 0
mknod: /dev/sda: Operation not permitted
chown: cannot access '/dev/sda': No such file or directory
mknod -m 0660 /dev/sda1 b 8 1
mknod: /dev/sda1: Operation not permitted
chown: cannot access '/dev/sda1': No such file or directory
Alternative rejected solutions:
1. Use fakeroot [2]. Package is available for the 3 distributions used
in CI jobs. Does fake stat() call. Works when run like this in the
CI test jobs:
fakeroot -s test/fakeroot.env tests/makedev.sh
fakeroot -i test/fakeroot.env make check
fakeroot -i test/fakeroot.env make distcheck
But if you run fakeroot ... make check on our development machines
as a non-root user it causes the test_SupportedFileSystems unit
tests which use losetup to fail. This is because
test_SupportedFileSystems thinks it's root inside the fakeroot
environment but fakeroot doesn't fake enough for losetup to work.
This makes running tests in the GitLab CI jobs different from how we
would have to run them on our development machines. Prefer not to
do that.
2. Use GNU ld --wrap [3] to call our own __wrap_stat() allowing
test_BlockSpecial to provide mocked results to the stat() call in
constructor BlockSpecial::BlockSpecial(). This works with
GNU C Library >= 2.33, released 01-Feb-2021, and musl libc,
therefore it works on CI tested distributions Ubuntu LTS >= 22.04
and Alpine Linux respectively. However this fails on earlier glibc
releases, so will fail on CentOS 7 CI image, as the compiler emits a
call to __xstat() rather than stat(). This is something to do with
how glibc's /usr/include/sys/stat.h supported multiple versions of
stat(). Don't use this as it's doesn't work everywhere.
Additional useful implementation hints. [4][5]
Choose to fix by just skipping unit tests which need block special names
to exist in the file system, but don't exist. This is the same
technique that test_SupportedFileSystems uses. So tests/makedev.sh
creates block devices if it can in the GNOME GitLab CI test images [1]
and now if that fails the individual unit tests are skipped.
[1] 57983b9fc2
Create block special devices needed by test_BlockSpecial in GitLab
CI jobs (!59)
[2] FakeRoot
https://wiki.debian.org/FakeRoot
[3] ld(1) - Linux manual page
https://man7.org/linux/man-pages/man1/ld.1.html
"--wrap=symbol
Use a wrapper function for symbol. Any undefined reference to
symbol will be resolved to "__wrap_symbol". Any undefined
reference to "__real_symbol" will be resolved to symbol.
This can be used to provide a wrapper for a system function.
The wrapper function should be called "__wrap_symbol". If it
wishes to call the system function, it should call
"__real_symbol".
...
"
[4] gcc: error: unrecognized option --wrap
https://stackoverflow.com/questions/33278164/gcc-error-unrecognized-option-wrap
[5] C++ ld linker --wrap option does not work for internal function calls
https://stackoverflow.com/questions/44464961/c-ld-linker-wrap-option-does-not-work-for-internal-function-callsClosed!113 - Fix occasional GitLab CI test jobs failures on
BlockSpecial unit tests
More recent g++ versions produce these warnings:
test_PasswordRAMStore.cc: In member function ‘virtual void GParted::PasswordRAMStoreTest_TotalErasure_Test::TestBody()’:
test_PasswordRAMStore.cc:61:32: warning: ‘ ’ directive output truncated writing 20 bytes into a region of size 10 [-Wformat-truncation=]
snprintf( buf, sizeof( buf ), "password%03u ", i );
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
test_PasswordRAMStore.cc:61:10: note: ‘snprintf’ output 32 bytes into a destination of size 21
snprintf( buf, sizeof( buf ), "password%03u ", i );
~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
snprintf() [1] truncates the printed string to the specified size so
didn't overflow the buffer. However clear the warning by making the
formatted string always exactly 20 characters long, followed by the
terminating NUL character to exactly fill the buffer.
[1] print3(f) - Linux manual page
https://man7.org/linux/man-pages/man3/snprintf.3.html
"The functions snprintf() and vsnprintf() write at most size bytes
(including the terminating null byte ('\0')) to str.
"
Since the previous commit "Also erase all Promise FastTrack RAID
signatures" the previous failing IntelSoftwareRAIDUnaligned test now
passes along with the new PromiseFastTrackRaid* tests.
$ ./test_EraseFileSystemSignatures
Running main() from test_EraseFileSystemSignatures.cc
DISPLAY=":0.0"
[==========] Running 4 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 4 tests from EraseFileSystemSignaturesTest
[ RUN ] EraseFileSystemSignaturesTest.IntelSoftwareRAIDAligned
[ OK ] EraseFileSystemSignaturesTest.IntelSoftwareRAIDAligned (158 ms)
[ RUN ] EraseFileSystemSignaturesTest.IntelSoftwareRAIDUnaligned
[ OK ] EraseFileSystemSignaturesTest.IntelSoftwareRAIDUnaligned (81 ms)
[ RUN ] EraseFileSystemSignaturesTest.PromiseFastTrackRAIDAligned
[ OK ] EraseFileSystemSignaturesTest.PromiseFastTrackRAIDAligned (74 ms)
[ RUN ] EraseFileSystemSignaturesTest.PromiseFastTrackRAIDUnaligned
[ OK ] EraseFileSystemSignaturesTest.PromiseFastTrackRAIDUnaligned (74 ms)
[----------] 4 tests from EraseFileSystemSignaturesTest (387 ms total)
[----------] Global test environment tear-down
[==========] 4 tests from 1 test case ran. (387 ms total)
[ PASSED ] 4 tests.
Closes#220 - Format to Cleared not clearing "pdc" ataraid signature
Move common testing code which doesn't need linking with GParted objects
into the common module. Move the remaining common code used to print
GParted objects using the insertion operator (operator<<) into the
insertion_operators module. Split the common code like this so that the
operator<<(std::ostream&, const OperationDetail&) function is not
included in test_PipeCapture and it is not forced to link with all the
non-UI related GParted objects.
The Automake manual provides guidance that when a header belongs to a
single program it is recommended to be listed in the program's _SOURCES
variable and for a directory only containing header files listing them
in the noinst_HEADERS variable is the right variable to use [1].
However the guidance doesn't cover this case for common.h and
insertion_operators.h; header files in a directory with other files and
used by multiple programs. So just because we have gparted_core_OBJECTS
(normal Makefile, not Automake special variable) listing objects to link
with, choose to use noinst_HEADERS Automake variable to list needed
headers.
[1] GNU Automake manual, 9.2 Header files
https://www.gnu.org/software/automake/manual/html_node/Headers.html
"Usually, only header files that accompany installed libraries
need to be installed. Headers used by programs or convenience
libraries are not installed. The noinst_HEADERS variable can be
used for such headers. However, when the header belongs to a
single convenience library or program, we recommend listing it
in the program's or library's _SOURCES variable (see Defining
program sources) instead of in noinst_HEADERS. This is clearer
for the Makefile.am reader. noinst_HEADERS would be the right
variable to use in a directory containing only headers and no
associated library or program.
All header files must be listed somewhere; in a _SOURCES
variable or in a _HEADERS variable. Missing ones will not
appear in the distribution.
"
Closes#220 - Format to Cleared not clearing "pdc" ataraid signature
Initially just testing erasing of Intel Software RAID signatures.
Chosen because it was expected to work, but turned out not to be true in
all cases.
The code needs to initialise GParted_Core::mainthread, construct
Gtk::Main() and execute xvfb-run because of this call chain:
GParted_Core::erase_filesystem_signatures()
GParted_Core::settle_device()
Utils::execute_command ("udevadm settle ...")
status.foreground = (Glib::Thread::self() == GParted_Core::mainthread)
Gtk::Main::run()
This was also needed when testing file system interface classes as
discussed in commits [1][2].
The test fails like this:
$ ./test_EraseFileSystemSignatures
...
[ RUN ] EraseFileSystemSignaturesTest.IntelSoftwareRAIDAligned
[ OK ] EraseFileSystemSignaturesTest.IntelSoftwareRAIDAligned (155 ms)
[ RUN ] EraseFileSystemSignaturesTest.IntelSoftwareRAIDUnaligned
test_EraseFileSystemSignatures.cc:286: Failure
Failed
image_contains_all_zeros(): First non-zero bytes:
0x00001A00 "Intel Raid ISM C" 49 6E 74 65 6C 20 52 61 69 64 20 49 53 4D 20 43
test_EraseFileSystemSignatures.cc:320: Failure
Value of: image_contains_all_zeros()
Actual: false
Expected: true
[ FAILED ] EraseFileSystemSignaturesTest.IntelSoftwareRAIDUnaligned (92 ms)
Manually write the same test image:
$ python << 'EOF'
signature = b'Intel Raid ISM Cfg Sig. '
import os
fd = os.open('/tmp/test.img', os.O_CREAT|os.O_WRONLY)
os.ftruncate(fd, 16*1024*1024 - 512)
os.lseek(fd, -(2*512), os.SEEK_END)
os.write(fd, signature)
os.close(fd)
EOF
Run gpartedbin /tmp/test.img and Format to > Cleared. GParted continues
to display the the image file as containing an ataraid signature.
$ blkid /tmp/test.img
/tmp/test.img: TYPE="isw_raid_member"
$ hexdump -C /tmp/test.img
00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
00fffa00 49 6e 74 65 6c 20 52 61 69 64 20 49 53 4d 20 43 |Intel Raid ISM C|
00fffa10 66 67 20 53 69 67 2e 20 00 00 00 00 00 00 00 00 |fg Sig. ........|
00fffa20 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
00fffe00
This signature is not being cleared when the device/partition/image size
is 512 bytes smaller than a whole MiB because the last 3.5 KiB is left
unwritten. This is because the last block of zeros written is 8 KiB
aligned to 4 KiB at the end of the device.
[1] a97c23c57c
Add initial create ext2 only FileSystem interface class test (!49)
[2] 8db9a83b39
Run test program under xvfb-run to satisfy need for an X11 display (!49)
Closes#220 - Format to Cleared not clearing "pdc" ataraid signature
From 27-Nov-2022 the alpine_test GitLab CI job started failing,
reporting errors creating XFS file systems in the
test_SupportedFileSystems unit test like this:
[ RUN ] My/SupportedFileSystemsTest.Create/xfs
test_SupportedFileSystems.cc:501: Failure
Value of: m_fs_object->create(m_partition, m_operation_detail)
Actual: false
Expected: true
Operation details:
mkfs.xfs -f -L '' '/builds/GNOME/gparted/tests/test_SupportedFileSystems.img' 00:00:00 (ERROR)
Filesystem must be larger than 300MB.
...
This is because Docker image "alpine:latest" has updated to Alpine Linux
3.17 which includes xfsprogs 6.0.0 which includes this change (first
released in xfsprogs 5.19.0):
https://git.kernel.org/pub/scm/fs/xfs/xfsprogs-dev.git/commit/?id=6e0ed3d19c54603f0f7d628ea04b550151d8a262
mkfs: stop allowing tiny filesystems
Refuse to format a filesystem that are "too small", because these
configurations are known to have performance and redundancy problems
that are not present on the volume sizes that XFS is best at
handling.
Specifically, this means that we won't allow logs smaller than 64MB,
we won't allow single-AG filesystems, and we won't allow volumes
smaller than 300MB.
Increase the default unit test file system image size from 256 MiB to
256+64 = 320 MiB to avoid this error.
Closes#217 - GitLab CI test job failing with new mkfs.xfs error
"Filesystem must be larger than 300MB."
Avoid having to manually maintain the list of excluded File System tests
in the GitLab Docker CI image. Scan the unit test source extracting
those tests marked with SKIP_IF_NOT_ROOT_FOR_REQUIRED_LOOPDEV_FOR_FS()
to automatically construct the setting for the GTEST_FILTER environment
variable.
Closes!105 - Update used btrfs file system commands, new minimum is
btrfs-progs 4.5
Now that reading btrfs usage, UUID and label can be performed on a file
system image remove the need for a loop device for the relevant unit
tests.
Closes!105 - Update used btrfs file system commands, new minimum is
btrfs-progs 4.5
For FAT16/32 and XFS file systems clearing the label uses different
command options and code path in file system specific ::write_label()
method. Therefore extend this unit test to also test clearing the
label.
During review and testing of this patchset it was discovered that using
GParted to set a new UUID on a FAT16 or FAT32 file system that there was
a new unwanted side effect of clearing the label.
Add unit test to cover this error scenario. It does the following:
1. Creates a file system with a known label;
2. Writes a new UUID;
3. Reads the label and confirms it matches the initial label.
This new unit test captures the fault like this:
$ ./test_SupportedFileSystems --gtest_filter='*CreateAndWriteUUIDAndReadLabel*'
...
[ RUN ] My/SupportedFileSystemsTest.CreateAndWriteUUIDAndReadLabel/fat16
test_SupportedFileSystems.cc:645: Failure
Expected equality of these values:
fs_label
Which is: "TEST_LABEL"
m_partition.get_filesystem_label().c_str()
Which is: ""
[ FAILED ] My/SupportedFileSystemsTest.CreateAndWriteUUIDAndReadLabel/fat16, where GetParam() = 13 (21 ms)
[ RUN ] My/SupportedFileSystemsTest.CreateAndWriteUUIDAndReadLabel/fat32
test_SupportedFileSystems.cc:645: Failure
Expected equality of these values:
fs_label
Which is: "TEST_LABEL"
m_partition.get_filesystem_label().c_str()
Which is: ""
[ FAILED ] My/SupportedFileSystemsTest.CreateAndWriteUUIDAndReadLabel/fat32, where GetParam() = 14 (22 ms)
Don't forget to exclude this unit test for file systems which need a
loop device but which fails to be created inside the docker CI image.
Reference:
39fdfe51da
Exclude unit tests needing losetup in Docker CI image (!59)
Closes!104 - Add Alpine Linux CI jobs and resolve label and UUID issues
with FAT16/32
Make (format as) bcache backing device (-B) and cache device (-C) and
implicitly attach the backing device to the cache to enable caching, all
in one.
# bcache make -B /dev/sdb1 -C /dev/sdc1
# bcache show
Name Type State Bname AttachToDev
/dev/sdb1 1 (data) clean(running) bcache0 /dev/sdc1
/dev/sdc1 3 (cache) active N/A N/A
After experimenting with 'bcache unregister', 'bcache register' and
stracing 'bcache show' the bcache kernel module creates the sysfs
directory /sys/block/DEV[/PTN]/bcache and it's contents only when the
bcache device is registered with the kernel (bcache component is
active). Use this to identify whether any bcache device (component)
should be displayed as active or not in GParted.
# ls -ld /sys/block/sd?/sd?1/bcache
drwxr-xr-x. 6 root root 0 Jan 7 10:08 /sys/block/sdb/sdb1/bcache
drwxr-xr-x. 2 root root 0 Jan 7 10:08 /sys/block/sdc/sdc1/bcache
Closes#183 - Basic support for bcache
This is the final piece which enables GParted to pass the LUKS
passphrase when resizing an open LUKS encryption mapping when
'cryptsetup resize' will prompt for it.
Closes#59 - Resize of LUKS2 encrypted file system fails with "Nothing
to read on input"
The GitLab CI ubuntu_test job has occasionally been failing like this,
perhaps once every few weeks or so.
[ RUN ] My/SupportedFileSystemsTest.CreateAndReadUUID/reiser4
test_SupportedFileSystems.cc:569: Failure
Expected: (m_partition.uuid.size()) >= (9U), actual: 0 vs 9
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadUUID/reiser4, where GetParam() = 24 (17 ms)
[----------] 1 test from My/SupportedFileSystemsTest (17 ms total)
Turns out there are 2 bugs in resier4progs. One causes debugfs.reiser4
to report a null UUID if the first byte of the UUID happens to be zero
[1], and the other cases mkfs.resier4 to write a corrupted UUID,
sometimes a null (all zeros) UUID [2].
There is a 1 in 256 chance of getting a null UUID [2] when creating and
reading a reiser4 file system, hence the occasional failure of the CI
job. The centos_test job isn't affected because CentOS doesn't have the
reiser4progs package.
Fix this by detecting when reiser4 reports a null UUID and assign a
dummy UUID to make the test pass. This does mean that there is a 1 in
256 chance of not detecting a true failure. However that still means
there is a 255 in 256 chance of detecting a true failure. That's good
odds. When a null UUID is detected for a reiser4 file system the test
output looks like this:
[ RUN ] My/SupportedFileSystemsTest.CreateAndReadUUID/reiser4
test_SupportedFileSystems.cc:580: Ignore test failure of a null UUID.
[ OK ] My/SupportedFileSystemsTest.CreateAndReadUUID/reiser4 (46 ms)
[1] 4802cdb18a
Fix up repair_master_print()
[2] 44cc024f39
Stop occasionally making file systems with null UUIDs
Closes#145 - Sporadic failure of test case
My/SupportedFileSystemsTest.CreateAndReadUUID/reiser4
This previous commit [1] excluded unit test
BlockSpecialTest.NamedBlockSpecialObjectBySymlinkMatches because GNOME
GitLab Docker CI images don't have /dev/disk hierarchy and so no
symbolic links to block devices.
Create the /dev/disk/by-id directory and a symlink for this unit test to
use in the new tests/makedev.sh script.
[1] fe2fc33e67
Exclude unit test which fails in Docker CI image (!4)
From 23-Feb-2020 onwards, GNOME GitLab Continuous Integration test jobs
have been failing running unit tests which previously succeeded. With
some extra debugging added into test_BlockSpecial to print 'bname' and
'bs' values in the failing tests, here are fragments from
tests/test-suite.log for the the test_BlockSpecial failures in a test CI
job:
FAIL: test_BlockSpecial
=======================
...
[ RUN ] BlockSpecialTest.NamedBlockSpecialObjectBlockDevice
bname="/dev/sr0"
bs=BlockSpecial{"/dev/sr0",0,0}
test_BlockSpecial.cc:218: Failure
Value of: bs.m_major > 0 || bs.m_minor > 0
Actual: false
Expected: true
[ FAILED ] BlockSpecialTest.NamedBlockSpecialObjectBlockDevice (0 ms)
...
[ RUN ] BlockSpecialTest.TwoNamedBlockSpecialObjectBlockDevices
bname1="/dev/sr0"
bname2="/dev/sda"
bs1=BlockSpecial{"/dev/sr0",0,0}
bs2=BlockSpecial{"/dev/sda",0,0}
test_BlockSpecial.cc:250: Failure
Value of: bs1.m_major != bs2.m_major || bs1.m_minor != bs2.m_minor
Actual: false
Expected: true
[ FAILED ] BlockSpecialTest.TwoNamedBlockSpecialObjectBlockDevices (1 ms)
Contents of /proc/partitions inside the Docker image when this test CI
job failed:
$ cat /proc/partitions
major minor #blocks name
11 0 1048575 sr0
8 0 573367448 sda
8 1 573366407 sda1
And the listing of /dev/:
$ ls -l /dev/
total 0
lrwxrwxrwx 1 root root 11 Mar 3 09:00 core -> /proc/kcore
lrwxrwxrwx 1 root root 13 Mar 3 09:00 fd -> /proc/self/fd
crw-rw-rw- 1 root root 1, 7 Mar 3 09:00 full
drwxrwxrwt 2 root root 40 Mar 3 09:00 mqueue
crw-rw-rw- 1 root root 1, 3 Mar 3 09:00 null
lrwxrwxrwx 1 root root 8 Mar 3 09:00 ptmx -> pts/ptmx
drwxr-xr-x 2 root root 0 Mar 3 09:00 pts
crw-rw-rw- 1 root root 1, 8 Mar 3 09:00 random
drwxrwxrwt 2 root root 40 Mar 3 09:00 shm
lrwxrwxrwx 1 root root 15 Mar 3 09:00 stderr -> /proc/self/fd/2
lrwxrwxrwx 1 root root 15 Mar 3 09:00 stdin -> /proc/self/fd/0
lrwxrwxrwx 1 root root 15 Mar 3 09:00 stdout -> /proc/self/fd/1
crw-rw-rw- 1 root root 5, 0 Mar 3 09:00 tty
crw-rw-rw- 1 root root 1, 9 Mar 3 09:00 urandom
crw-rw-rw- 1 root root 1, 5 Mar 3 09:00 zero
See how the test_BlockSpecial fixtures are getting major=0 and minor=0
for the block special devices they are testing with. This is happening
because there aren't any entries in /dev for those disks and partitions
listed in /proc/partitions. Assume that Docker in GNOME GitLab has
changed and that unneeded and unwanted devices in /dev are no longer
being created inside images.
In the test CI jobs execute new script, tests/makedev.sh, to create just
the first two block special devices mentioned in /proc/partitions needed
by test_BlockSpecial.
Closes!59 - Fix GNOME GitLab CI test job failures because of missing
/dev entries
So far the read file system usage figures, read via the file system
interface classes using file system specific tools, have been checked to
the exact sector for:
0 <= used <= size
0 <= unused <= size
unallocated = 0
used + unused = size
However for JFS and NTFS this fails like this:
# ./test_SupportedFileSystems --gtest_filter='*ReadUsage/*' | fgrep ' ms'
[ OK ] My/SupportedFileSystemsTest.CreateAndReadUsage/btrfs (335 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndReadUsage/exfat (0 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndReadUsage/ext2 (38 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndReadUsage/ext3 (131 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndReadUsage/ext4 (32 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndReadUsage/f2fs (47 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndReadUsage/fat16 (19 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndReadUsage/fat32 (48 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndReadUsage/hfs (0 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndReadUsage/hfsplus (0 ms)
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadUsage/jfs, where GetParam() = 17 (73 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndReadUsage/linuxswap (20 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndReadUsage/luks (0 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndReadUsage/lvm2pv (410 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndReadUsage/minix (0 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndReadUsage/nilfs2 (226 ms)
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadUsage/ntfs, where GetParam() = 23 (56 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndReadUsage/reiser4 (49 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndReadUsage/reiserfs (139 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndReadUsage/udf (34 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndReadUsage/xfs (67 ms)
[----------] 21 tests from My/SupportedFileSystemsTest (1726 ms total)
[==========] 21 tests from 1 test case ran. (1726 ms total)
# ./test_SupportedFileSystems --gtest_filter='*ReadUsage/jfs:*ReadUsage/ntfs'
Running main() from test_SupportedFileSystems.cc
Note: Google Test filter = *ReadUsage/jfs:*ReadUsage/ntfs
[==========] Running 2 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 2 tests from My/SupportedFileSystemsTest
[ RUN ] My/SupportedFileSystemsTest.CreateAndReadUsage/jfs
test_SupportedFileSystems.cc:465: Failure
Expected equality of these values:
m_partition.sectors_unallocated
Which is: 2472
0
test_SupportedFileSystems.cc:517: Failure
Expected equality of these values:
m_partition.sectors_used + m_partition.sectors_unused
Which is: 521816
m_partition.get_sector_length()
Which is: 524288
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadUsage/jfs, where GetParam() = 17 (36 ms)
[ RUN ] My/SupportedFileSystemsTest.CreateAndReadUsage/ntfs
test_SupportedFileSystems.cc:465: Failure
Expected equality of these values:
m_partition.sectors_unallocated
Which is: 8
0
test_SupportedFileSystems.cc:517: Failure
Expected equality of these values:
m_partition.sectors_used + m_partition.sectors_unused
Which is: 524280
m_partition.get_sector_length()
Which is: 524288
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadUsage/ntfs, where GetParam() = 23 (35 ms)
[----------] 2 tests from My/SupportedFileSystemsTest (71 ms total)
[----------] Global test environment tear-down
[==========] 2 tests from 1 test case ran. (72 ms total)
[ PASSED ] 0 tests.
[ FAILED ] 2 tests, listed below:
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadUsage/jfs, where GetParam() = 17
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadUsage/ntfs, where GetParam() = 23
2 FAILED TESTS
So JFS is reporting 2472 unallocated sectors in a size of 524288 sectors
and NTFS is reporting 8 unallocated sectors in the same size. This
exact issue is already solved for GParted so that it doesn't show a
small amount of unallocated space by commits [1][2] from Bug 499202 [3].
Fix the same way, use the accessors to the file system usage figures
which don't show unallocated space when it is below the significant
threshold.
[1] b5c80f18a9
Enhance calculation of significant unallocated space (#499202)
[2] 7ebedc4bb3
Don't show intrinsic unallocated space (#499202)
[3] Bug 499202 - gparted does not see the difference if partition size
differs from filesystem size
https://bugzilla.gnome.org/show_bug.cgi?id=499202Closes!49 - Add file system interface tests
Checking a MINIX V3 file system fails like this:
$ ./test_SupportedFileSystems --gtest_filter='*Check/minix'
...
[ RUN ] My/SupportedFileSystemsTest.CreateAndCheck/minix
test_SupportedFileSystems.cc:554: Failure
Value of: m_fs_object->check_repair(m_partition, m_operation_detail)
Actual: false
Expected: true
Operation details:
mkfs.minix -3 '/home/centos/programming/c/gparted/tests/test_SupportedFileSystems.img' 00:00:00 (SUCCESS)
87392 inodes
262144 blocks
Firstdatazone=5507 (5507)
Zonesize=1024
Maxsize=2147483647
fsck.minix '/home/centos/programming/c/gparted/tests/test_SupportedFileSystems.img' 00:00:00 (ERROR)
fsck.minix from util-linux 2.23.2
bad magic number in super-block
[ FAILED ] My/SupportedFileSystemsTest.CreateAndCheck/minix, where GetParam() = 21 (182 ms)
fsck.minix doesn't support checking MINIX V3 file systems until this
commit, first included in util-linux 2.27 released 2015-09-07.
https://git.kernel.org/pub/scm/utils/util-linux/util-linux.git/commit/?id=86a9f3dad58addb50eca9daa9d233827a005dad7
fsck.minix: add minix v3 support
CentOS 7 only includes util-linux 2.23.2 so is affected by this, however
Ubuntu 18.04 LTS includes util-linux 2.31.1 so is not affected.
Just always skip this test for now. Plan to re-enable later when the
oldest supported distributions and GitLab CI images include the needed
util-linux release.
Closes!49 - Add file system interface tests
The tests were failing like this:
$ ./test_SupportedFileSystems --gtest_filter='*CreateAndReadUUID/fat16'
....
[ RUN ] My/SupportedFileSystemsTest.CreateAndReadUUID/fat16
test_SupportedFileSystems.cc:552: Failure
Expected equality of these values:
m_partition.uuid.size()
Which is: 9
36U
Which is: 36
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadUUID/fat16, where GetParam() = 13 (45 ms)
This is because the test was expecting a full 36 character UUID as used
by Linux file systems. Also accept shorter 9 character "UUID"s as used
by FAT16/32 file systems.
Closes!49 - Add file system interface tests
For NILFS2 the read and write tests which use nilfs-tune all fail using
an image file, even when run as root, however the other tests succeed.
Selected output from the test program:
# ./test_SupportedFileSystems --gtest_filter='*/nilfs2' | fgrep ' ms'
[ OK ] My/SupportedFileSystemsTest.Create/nilfs2 (22 ms)
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadUsage/nilfs2, where GetParam() = 22 (31 ms)
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadLabel/nilfs2, where GetParam() = 22 (30 ms)
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadUUID/nilfs2, where GetParam() = 22 (30 ms)
[ FAILED ] My/SupportedFileSystemsTest.CreateAndWriteLabel/nilfs2, where GetParam() = 22 (37 ms)
[ FAILED ] My/SupportedFileSystemsTest.CreateAndWriteUUID/nilfs2, where GetParam() = 22 (39 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndCheck/nilfs2 (0 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndRemove/nilfs2 (0 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndGrow/nilfs2 (386 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndShrink/nilfs2 (345 ms)
[----------] 10 tests from My/SupportedFileSystemsTest (920 ms total)
[==========] 10 tests from 1 test case ran. (920 ms total)
nilfs-tune fails like this when given an image file:
# truncate -s 256M test.img
# mkfs.nilfs2 test.img
mkfs.nilfs2 (nilfs-utils 2.2.7)
Start writing file system initial data to the device
Blocksize:4096 Device:test.img Device Size:268435456
File system initialization succeeded !!
# nilfs-tune -l test.img
nilfs-tune 2.2.7
nilfs-tune: test.img: cannot open NILFS
# echo $?
1
However using nilfs-tune via a loop device works:
# losetup --show --find /dev/loop0
/dev/loop0
# nilfs-tune -l /dev/loop0
nilfs-tune 2.2.7
Filesystem volume name: (none)
Filesystem UUID: fc49912c-4d39-4672-8610-1e1185d0db5f
Filesystem magic number: 0x3434
Filesystem revision #: 2.0
Filesystem features: (none)
Filesystem state: valid
Filesystem OS type: Linux
Block size: 4096
...
So nilfs-tune only works with block devices. Fix by making these tests
require a loop device and therefore make them root only. Now these
tests are skipped as non-root user and pass as root.
Closes!49 - Add file system interface tests
File systems BTRFS, JFS, NILFS2 and XFS can only be resized while
mounted, but only root can mount file systems. Therefore these tests
fail. Also BTRFS resize uses 'btrfs filesystem show' to discover the
devid, which also fails as described in the previous commit message.
Note that root can mount a file system image directly, but that it
implicitly creates loop device:
# truncate -s 256M test.img
# mkfs.xfs test.img
# mount test.img /mnt/1
# fgrep /mnt/1 /proc/mounts
/dev/loop0 /mnt/1 xfs rw,seclabel,relatime,attr2,inode64,noquota 0 0
# losetup -a
/dev/loop0: [64768]:35826659 (/root/test.img)
Therefore make these tests root only and require an explicit loop
device. Now these file system resize tests succeed as root and are
skipped as non-root.
Closes!49 - Add file system interface tests
For BTRFS the read (and resize) tests fail when using an image file,
however the create, write and check tests pass. Selected output from
the test program:
$ ./test_SupportedFileSystems --gtest_filter='*/btrfs' | fgrep ' ms'
[ OK ] My/SupportedFileSystemsTest.Create/btrfs (43 ms)
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadUsage/btrfs, where GetParam() = 7 (95 ms)
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadLabel/btrfs, where GetParam() = 7 (158 ms)
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadUUID/btrfs, where GetParam() = 7 (164 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndWriteLabel/btrfs (164 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndWriteUUID/btrfs (132 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndCheck/btrfs (129 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndRemove/btrfs (0 ms)
[ FAILED ] My/SupportedFileSystemsTest.CreateAndGrow/btrfs, where GetParam() = 7 (155 ms)
[ FAILED ] My/SupportedFileSystemsTest.CreateAndShrink/btrfs, where GetParam() = 7 (97 ms)
[----------] 10 tests from My/SupportedFileSystemsTest (1137 ms total)
[==========] 10 tests from 1 test case ran. (1137 ms total)
The read operations fail because 'btrfs filesystem show' doesn't work on
am image file:
$ truncate -s 256M test.img
$ mkfs.btrfs test.img
btrfs-progs v4.9.1
See http://btrfs.wiki.kernel.org for more information.
Label: (null)
UUID: de1624ae-39bb-4796-aee4-7ee1fa24c06a
Node side: 16384
Sector size: 4096
Filesystem size: 256.00MiB
Block group profiles:
Data: single
Metadata: DUP
System: DUP
SSD detected: no
Incompat features: extref, skinny-metadata
Number of devices: 1
Devices:
ID SIZE PATH
1 256.00MiB test.img
$ btrfs filesystem show test.img
ERROR: not a valid btrfs filesystem: /home/centos/programming/c/gparted/tests/test.img
$ echo $1
1
Querying a BTRFS image file also fails as root:
$ su
Password:
# btrfs filesystem show test.img
ERROR: not a valid btrfs filesystem: /home/centos/programming/c/gparted/tests/test.img
# echo $1
1
However querying the BTRFS via a loop device succeeds:
# losetup --show --find test.img
/dev/loop0
# btrfs filesystem show /dev/loop0
Label: none uuid: de1624ae-39bb-4796-aee4-7ee1fa24c06a
Total devices 1 FS bytes used 112.00KiB
devid 1 size 256.00MiB used 88.00MiB path /root/test.img
There must be some kernel level BTRFS file system device discovery
happening because now after creating a loop device for the image file,
the BTRFS can be shown via the image file directly:
# btrfs filesystem show test.img
Label: none uuid: de1624ae-39bb-4796-aee4-7ee1fa24c06a
Total devices 1 FS bytes used 112.00KiB
devid 1 size 256.00MiB used 88.00MiB path /root/test.img
Anyway for the BTRFS reading tests make them required a loop device and
therefore root only. Now these tests are skipped as non-root user and
pass as root.
Addressing BTRFS resizing test failures will be handled in a following
commit.
Closes!49 - Add file system interface tests
Creating an LVM2 PV as a non-root user on an image file fails like this:
$ truncate -s 256M test.img
$ lvm pvcreate `pwd`/test.img
WARNING: Running as a non-root user. Functionality may be unavailable.
/run/lvm/lvmetad.socket: access failed: Permission denied
WARNING: Failed to connect to lvmetad. Falling back to device scanning.
/run/lock/lvm/P_orphans:aux: open failed: Permission denied
Can't get lock for orphan PVs.
$ echo $?
5
Trying the same as root also fails:
# truncate -s 256M test.img
# lvm pvcreate `pwd`/test.img
Device /root/test.img not found.
# echo $?
5
LVM seems strongly predicated on only using block devices [1]. LVM can
use loop devices though, but loop devices can only be created by root.
# truncate -s 256M test.img
# losetup -f --show `pwd`/test.img
/dev/loop0
# lvm pvcreate /dev/loop0
Physical volume "/dev/loop0" successfully created.
# echo $?
0
Make the LVM2 PV tests require user root and use loop device over the
test image. Tests for the other file system types still directly uses
the image file. This makes the LVM2 PV tests pass when run as root, or
successfully skipped when run as non-root.
[1] lvmconfig --typeconfig default --withcomments --withspace | less
From the "devices" section of the commented default configuration,
LVM uses block devices found below /dev, devices provided by udev
and/or found in sysfs.
Closes!49 - Add file system interface tests
Extend testing to all fully supported file systems, those with an
implemented FileSystem derived class.
Note that in main() GParted threading needs to now be initialised before
InitGoogleTest() because it calls INSTANTIATE_TEST_CASE_P() which in
turn calls get_supported_fstypes() which eventually constructs all the
individual file system interface objects and discovers available
support, some of which use execute_command(). Example call chain:
InitGoogleTest()
INSTANTIATE_TEST_CASE_P()
get_supported_fstypes()
setup_supported_filesystems()
{SupportedFileSystems}->find_supported_filesystems()
{btrfs}->get_filesystem_support()
Utils::execute_command()
In the CentOS 7 GitLab CI image the EPEL (Extra Packages for Enterprise
Linux) repository is added to provide f2fs-tools and ntfsprogs.
23 of 210 tests fail on CentOS 7 and 22 on Ubuntu 18.04 LTS. The
following commits will resolve these test failures.
$ ./test_SupportedFileSystems
Running main() from test_SupportedFileSystems.cc
[==========] Running 210 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 210 tests from My/SupportedFileSystemsTest
...
[----------] 210 tests from My/SupportedFileSystemsTest (11066 ms total)
[----------] Global test environment tear-down
[==========] 210 tests from 1 test case ran. (11067 ms total)
[ PASSED ] 187 tests.
[ FAILED ] 23 tests, listed below:
[ FAILED ] My/SupportedFileSystemsTest.Create/lvm2pv, where GetParam() = 20
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadUsage/btrfs, where GetParam() = 7
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadUsage/jfs, where GetParam() = 17
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadUsage/lvm2pv, where GetParam() = 20
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadUsage/nilfs2, where GetParam() = 22
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadUsage/ntfs, where GetParam() = 23
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadLabel/btrfs, where GetParam() = 7
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadLabel/nilfs2, where GetParam() = 22
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadUUID/btrfs, where GetParam() = 7
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadUUID/fat16, where GetParam() = 13
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadUUID/fat32, where GetParam() = 14
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadUUID/jfs, where GetParam() = 17
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadUUID/nilfs2, where GetParam() = 22
[ FAILED ] My/SupportedFileSystemsTest.CreateAndWriteLabel/nilfs2, where GetParam() = 22
[ FAILED ] My/SupportedFileSystemsTest.CreateAndWriteUUID/nilfs2, where GetParam() = 22
[ FAILED ] My/SupportedFileSystemsTest.CreateAndCheck/lvm2pv, where GetParam() = 20
[ FAILED ] My/SupportedFileSystemsTest.CreateAndCheck/minix, where GetParam() = 21
[ FAILED ] My/SupportedFileSystemsTest.CreateAndRemove/lvm2pv, where GetParam() = 20
[ FAILED ] My/SupportedFileSystemsTest.CreateAndGrow/btrfs, where GetParam() = 7
[ FAILED ] My/SupportedFileSystemsTest.CreateAndGrow/lvm2pv, where GetParam() = 20
[ FAILED ] My/SupportedFileSystemsTest.CreateAndGrow/xfs, where GetParam() = 27
[ FAILED ] My/SupportedFileSystemsTest.CreateAndShrink/btrfs, where GetParam() = 7
[ FAILED ] My/SupportedFileSystemsTest.CreateAndShrink/lvm2pv, where GetParam() = 20
23 FAILED TESTS
Closes!49 - Add file system interface tests
Until now the parameterised test values are printed as part of the test
names as just 0, 1, etc. like this:
$ ./test_SupportedFileSystems
Running main() from test_SupportedFileSystems.cc
[==========] Running 20 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 20 tests from My/SupportedFileSystemsTest
[ RUN ] My/SupportedFileSystemsTest.Create/0
[ OK ] My/SupportedFileSystemsTest.Create/0 (48 ms)
[ RUN ] My/SupportedFileSystemsTest.Create/1
[ OK ] My/SupportedFileSystemsTest.Create/1 (11 ms)
Provide the file system types as the names for the parameterised test
values [1]. Now the test names are printed like this:
$ ./test_SupportedFileSystems
Running main() from test_SupportedFileSystems.cc
[==========] Running 20 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 20 tests from My/SupportedFileSystemsTest
[ RUN ] My/SupportedFileSystemsTest.Create/ext2
[ OK ] My/SupportedFileSystemsTest.Create/ext2 (51 ms)
[ RUN ] My/SupportedFileSystemsTest.Create/linuxswap
[ OK ] My/SupportedFileSystemsTest.Create/linuxswap (11 ms)
Also use these Google Test name friendly ASCII alphanumeric only names
everywhere the file system type needs to be reported in this test
program.
[1] Specifying Names for Value-Parameterized Test Parameters
https://github.com/google/googletest/blob/v1.8.x/googletest/docs/advanced.md#specifying-names-for-value-parameterized-test-parametersCloses!49 - Add file system interface tests
Use Google Test Value-Parameterised to call every test for both ext2
and linux-swap.
https://github.com/google/googletest/blob/v1.8.x/googletest/docs/advanced.md#value-parameterized-tests
Running the test now looks like this:
$ ./test_SupportedFileSystems
Running main() from test_SupportedFileSystems.cc
[==========] Running 20 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 20 tests from My/SupportedFileSystemsTest
[ RUN ] My/SupportedFileSystemsTest.Create/0
[ OK ] My/SupportedFileSystemsTest.Create/0 (97 ms)
[ RUN ] My/SupportedFileSystemsTest.Create/1
[ OK ] My/SupportedFileSystemsTest.Create/1 (15 ms)
[ RUN ] My/SupportedFileSystemsTest.CreateAndReadUsage/0
[ OK ] My/SupportedFileSystemsTest.CreateAndReadUsage/0 (106 ms)
[ RUN ] My/SupportedFileSystemsTest.CreateAndReadUsage/1
[ OK ] My/SupportedFileSystemsTest.CreateAndReadUsage/1 (14 ms)
[ RUN ] My/SupportedFileSystemsTest.CreateAndReadLabel/0
[ OK ] My/SupportedFileSystemsTest.CreateAndReadLabel/0 (95 ms)
[ RUN ] My/SupportedFileSystemsTest.CreateAndReadLabel/1
[ OK ] My/SupportedFileSystemsTest.CreateAndReadLabel/1 (23 ms)
[ RUN ] My/SupportedFileSystemsTest.CreateAndReadUUID/0
[ OK ] My/SupportedFileSystemsTest.CreateAndReadUUID/0 (99 ms)
[ RUN ] My/SupportedFileSystemsTest.CreateAndReadUUID/1
[ OK ] My/SupportedFileSystemsTest.CreateAndReadUUID/1 (22 ms)
[ RUN ] My/SupportedFileSystemsTest.CreateAndWriteLabel/0
[ OK ] My/SupportedFileSystemsTest.CreateAndWriteLabel/0 (102 ms)
[ RUN ] My/SupportedFileSystemsTest.CreateAndWriteLabel/1
[ OK ] My/SupportedFileSystemsTest.CreateAndWriteLabel/1 (22 ms)
[ RUN ] My/SupportedFileSystemsTest.CreateAndWriteUUID/0
[ OK ] My/SupportedFileSystemsTest.CreateAndWriteUUID/0 (101 ms)
[ RUN ] My/SupportedFileSystemsTest.CreateAndWriteUUID/1
[ OK ] My/SupportedFileSystemsTest.CreateAndWriteUUID/1 (21 ms)
[ RUN ] My/SupportedFileSystemsTest.CreateAndCheck/0
[ OK ] My/SupportedFileSystemsTest.CreateAndCheck/0 (153 ms)
[ RUN ] My/SupportedFileSystemsTest.CreateAndCheck/1
test_SupportedFileSystems.cc:424: Skip test. check not supported or support not found
[ OK ] My/SupportedFileSystemsTest.CreateAndCheck/1 (0 ms)
[ RUN ] My/SupportedFileSystemsTest.CreateAndRemove/0
test_SupportedFileSystems.cc:437: Skip test. remove not supported or support not found
[ OK ] My/SupportedFileSystemsTest.CreateAndRemove/0 (0 ms)
[ RUN ] My/SupportedFileSystemsTest.CreateAndRemove/1
test_SupportedFileSystems.cc:437: Skip test. remove not supported or support not found
[ OK ] My/SupportedFileSystemsTest.CreateAndRemove/1 (0 ms)
[ RUN ] My/SupportedFileSystemsTest.CreateAndGrow/0
[ OK ] My/SupportedFileSystemsTest.CreateAndGrow/0 (266 ms)
[ RUN ] My/SupportedFileSystemsTest.CreateAndGrow/1
[ OK ] My/SupportedFileSystemsTest.CreateAndGrow/1 (32 ms)
[ RUN ] My/SupportedFileSystemsTest.CreateAndShrink/0
[ OK ] My/SupportedFileSystemsTest.CreateAndShrink/0 (111 ms)
[ RUN ] My/SupportedFileSystemsTest.CreateAndShrink/1
[ OK ] My/SupportedFileSystemsTest.CreateAndShrink/1 (28 ms)
[----------] 20 tests from My/SupportedFileSystemsTest (1311 ms total)
[----------] Global test environment tear-down
[==========] 20 tests from 1 test case ran. (1342 ms total)
[ PASSED ] 20 tests.
Closes!49 - Add file system interface tests
Replace directly using ext2 derived FileSystem interface class with
using the SupportedFileSystems class. This is a step in getting ready
for testing all the GParted file system interface classes in one go.
Closes!49 - Add file system interface tests
GParted_Core::FILESYSTEMS and ::FILESYSTEM_MAP and the methods that
query and manipulate them are self-contained. Therefore move them into
a separate SupportedFileSystems module.
Also having a single class maintaining all FileSystem interface objects
will make testing all the file system types much easier as there will be
no need to duplicate this functionality in the test.
Closes!49 - Add file system interface tests
Here are the errors reported in the deliberately broken
CreateAndReadLabel test from the previous commit message:
[ RUN ] ext2Test.CreateAndReadLabel
test_ext2.cc:311: Failure
Value of: m_partition.get_messages().empty()
Actual: false
Expected: true
Partition messages:
e2label: No such file or directory while trying to open /does_not_exist/test_ext2.img
Couldn't find valid filesystem superblock.
[ FAILED ] ext2Test.CreateAndReadLabel (77 ms)
Even though the test was deliberately broken by setting the wrong path
for the file system image and the e2label command failed, apparently
testing for the expected label still passed. What happened was that the
desired "TEST_LABEL" has to be in the Partition object and then the file
system was created. Then reading the file system label failed, however
"TEST_LABEL" was already set in the Partition object so it matched.
Reading the label is unique among the read actions of usage, label and
UUID as the others don't need to be set before the file system is
created. GParted doesn't encounter this issue because when refreshing
devices it creates new blank Partition objects and then performs the
read actions to populate them.
Fix by resetting the Partition object back to only containing basic
information before all the reading file system information tests, even
though it is only needed in the read label case. This also better
reflects how GParted works.
Now with the same deliberate brokenness the test also reports the label
does not match it's expected value:
$ ./test_ext2 --gtest_filter='ext2Test.CreateAndReadLabel'
Running main() from test_ext2.cc
Note: Google Test filter = ext2Test.CreateAndReadLabel
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from ext2Test
[ RUN ] ext2Test.CreateAndReadLabel
test_ext2.cc:322: Failure
Expected equality of these values:
fs_label
Which is: "TEST_LABEL"
m_partition.get_filesystem_label().c_str()
Which is: ""
test_ext2.cc:272: Failure
Value of: m_partition.get_messages().empty()
Actual: false
Expected: true
Partition messages:
e2label: No such file or directory while trying to open /does_not_exist/test_ext2.img
Couldn't find valid filesystem superblock.
[ FAILED ] ext2Test.CreateAndReadLabel (70 ms)
[----------] 1 test from ext2Test (70 ms total)
[----------] Global test environment tear-down
[==========] 1 test from 1 test case ran. (75 ms total)
[ PASSED ] 0 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] ext2Test.CreateAndReadLabel
1 FAILED TEST
Closes!49 - Add file system interface tests
The file system reading methods report errors into Partition messages
because they are used as part of the GParted refresh, rather than
reporting errors into OperationDetails used when applying operations.
Therefore test for no messages for success and print the messages on
failure.
For example, temporarily breaking the read label test code by setting
the wrong file system image name produces this:
$ ./test_ext2 --gtest_filter='ext2Test.CreateAndReadLabel'
Running main() from test_ext2.cc
Note: Google Test filter = ext2Test.CreateAndReadLabel
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from ext2Test
[ RUN ] ext2Test.CreateAndReadLabel
test_ext2.cc:311: Failure
Value of: m_partition.get_messages().empty()
Actual: false
Expected: true
Partition messages:
e2label: No such file or directory while trying to open /does_not_exist/test_ext2.img
Couldn't find valid filesystem superblock.
[ FAILED ] ext2Test.CreateAndReadLabel (77 ms)
[----------] 1 test from ext2Test (77 ms total)
[----------] Global test environment tear-down
[==========] 1 test from 1 test case ran. (85 ms total)
[ PASSED ] 0 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] ext2Test.CreateAndReadLabel
1 FAILED TEST
$ echo $?
1
Closes!49 - Add file system interface tests
As seen in the first commit message, operation detail text is XML
encoded. This makes it harder to read, especially commands which often
have single quotes encoded as '. For example:
<b><i>mkfs.ext2 -F -L '' '/home/centos/programming/c/gparted/tests/test_ext2.img'</i></b>
Strip this encoding when printing the operation details. Now the same
example looks like:
mkfs.ext2 -F -L '' '/home/centos/programming/c/gparted/tests/test_ext2.img'
Closes!49 - Add file system interface tests
Running test_ext2 in GitLab Continuous Integration environment fails like
this:
(test_ext2:6338): Gtk-WARNING **: 09:06:17.576: cannot open display:
Running main() from test_ext2.cc
Obviously the GitLab CI environment doesn't have an X11 display, but
unfortunately this test case code requires one.
Utils::execute_command() calls Gtk::Main::run() so requires a Gtk::Main
object constructing and therefore an X11 display, even though this
program never displays anything graphical. The call chain is:
main() test_ext2.cc
Gtk::Main::Main() gtkmm/gtk/src/main.ccg
Gtk::Main::init() [1]
gtk_init() gtk/gtk/gtkmain.c [2]
which exits with a non-zero exit status when the DISPLAY environment
variable is unset.
Looked at deriving from Gtk::Main class and writing a replacement init()
method which calls gtk_init_check() instead of gtk_init() but
Gtk::Main::instance_ is a private member so not accessible in a derived
class.
Tried using Glib::MainLoop instead of Gtk::Main, but that doesn't
initialise everything that Gtk::Main(), so the program crashes.
Therefore use xvfb-run [3][4] to run this test program against a virtual
X11 display when a real display isn't available. Coded execution of
xvfb-run into this test program so that it can simply be executed on the
command line like the other test programs, without having to remember to
run "xvfb-run ./test_ext2 ...".
[1] Gtk::Main::init()
https://gitlab.gnome.org/GNOME/gtkmm/blob/3.10.1/gtk/src/main.ccg#L287
[2] gtk_init()
https://gitlab.gnome.org/GNOME/gtk/blob/3.10.9/gtk/gtkmain.c#L1000
[3] how to run gtk app on linux without an x server
https://superuser.com/questions/624918/how-to-run-gtk-app-on-linux-without-an-x-server
[4] Using GTK without DISPLAY
https://stackoverflow.com/questions/11694278/using-gtk-without-displayCloses!49 - Add file system interface tests
This is the first step of adding testing of the derived FileSystem
interface classes which call the file system specific executables.
Rather than mocking command execution and returned output the tests run
the real commands, effectively making this integration testing.
Test case setup determines the file system supported actions using
get_filesystem_support() and individual tests are skipped if a feature
is not supported, just as GParted does for it's actions.
Each test creates it's own sparse image file and a fresh file system,
performs a test on one FileSystem interface call and deletes the image
file. This makes each test independent and allows them to run as a
non-root user, provided the file system command itself doesn't require
root. Errors reported for a failed interface call will include the
GParted OperationDetails, which in turn includes the file system
specific command used and stdout and stderr from it's execution.
For example, temporarily breaking the test code to create a 10 KiB image
file instead of 256 MiB one produces this:
$ ./test_ext2
Running main() from test_ext2.cc
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from ext2Test
[ RUN ] ext2Test.Create
test_ext2.cc:199: Failure
Value of: s_ext2_obj->create(m_partition, m_operation_detail)
Actual: false
Expected: true
Operation details:
<b><i>mkfs.ext2 -F -L '' '/home/centos/programming/c/gparted/tests/test_ext2.img'</i></b> 00:00:00 (ERROR)
<i></i>
<i>mke2fs 1.42.9 (28-Dec-2013)
/home/centos/programming/c/gparted/tests/test_ext2.img: Not enough space to build proposed filesystem while setting up superblock
</i>
[ FAILED ] ext2Test.Create (25 ms)
[----------] 1 test from ext2Test (25 ms total)
[----------] Global test environment tear-down
[==========] 1 test from 1 test case ran. (30 ms total)
[ PASSED ] 0 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] ext2Test.Create
1 FAILED TEST
$ echo $?
1
Also as Utils:: and FileSystem::execute_command() are needed, this
requires linking with GParted_Core for GParted_Core::mainthread and
therefore with most of the non-UI classes in gpartedbin.
Closes!49 - Add file system interface tests
Was attempting to erase the wrong key "key-long" for the test. It
should be erasing "key-too-long". Fix this. Given that that line in
the test was to erase a non-existent key and confirm failure; the test
passed before with the wrong non-existent key, and still passes with the
right non-existent key. Clearly a cut and paste error as a result of
copying the previous LongPassword test when initially added in:
c6657aab9e
Add unit tests for PasswordRAMStore module (#795617)
We have to copy the dentry->d_name before calling closedir(). If not,
the string points to nothing and the test fails (It does not fail all
the time, but only by chance).
Confirmed using valgrind. Selected output from running the unit test
under valgrind:
$ valgrind --track-origins=yes ./test_blockSpecial
==25110== Memcheck, a memory error detector
...
==25110== Command: ./test_BlockSpecial
==25110==
Running main() from src/gtest_main.cc
[==========] Running 26 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 26 tests from BlockSpecialTest
...
[ RUN ] BlockSpecialTest.NamedBlockSpecialObjectBySymlinkMatches
==25110== Invalid read of size 1
==25110== at 0x4C2C9B2: strlen (vg_replace_strmem.c:458)
==25110== by 0x40E7C4: length (char_traits.h:259)
==25110== by 0x40E7C4: append (basic_string.h:1009)
==25110== by 0x40E7C4: operator+<char, std::char_traits<char>, std::allocator<char> > (basic_string.h:2468)
==25110== by 0x40E7C4: get_link_name (test_BlockSpecial.cc:156)
==25110== by 0x40E7C4: GParted::BlockSpecialTest_NamedBlockSpecialObjectBySymlinkMatches_Test::TestBody() (test_BlockSpecial.cc:247)
...
=25110== Address 0x1231ea93 is 115 bytes inside a block of size 32,816 free'd
==25110== at 0x4C2ACBD: free (vg_replace_malloc.c:530)
==25110== by 0x9F773AC: closedir (in /usr/lib64/libc-2.17.so)
==25110== by 0x40E7AC: get_link_name (test_BlockSpecial.cc:153)
==25110== by 0x40E7AC: GParted::BlockSpecialTest_NamedBlockSpecialObjectBySymlinkMatches_Test::TestBody() (test_BlockSpecial.cc:247)
...
==25110== Block was alloc'd at
==25110== at 0x4C29BC3: malloc (vg_replace_malloc.c:299)
==25110== by 0x9F77280: __alloc_dir (in /usr/lib64/libc-2.17.so)
==25110== by 0x40E746: get_link_name (test_BlockSpecial.cc:134)
==25110== by 0x40E746: GParted::BlockSpecialTest_NamedBlockSpecialObjectBySymlinkMatches_Test::TestBody() (test_BlockSpecial.cc:247)
...
==25110== Invalid read of size 1
==25110== at 0x4C2E220: memcpy@@GLIBC_2.14 (vg_replace_strmem.c:1022)
==25110== by 0x953A997: std::string::append(char const*, unsigned long) (in /usr/lib64/libstdc++.so.6.0.19)
==25110== by 0x40E7D2: append (basic_string.h:1009)
==25110== by 0x40E7D2: operator+<char, std::char_traits<char>, std::allocator<char> > (basic_string.h:2468)
==25110== by 0x40E7D2: get_link_name (test_BlockSpecial.cc:156)
==25110== by 0x40E7D2: GParted::BlockSpecialTest_NamedBlockSpecialObjectBySymlinkMatches_Test::TestBody() (test_BlockSpecial.cc:247)
...
==25110== Address 0x1231ea93 is 115 bytes inside a block of size 32,816 free'd
==25110== at 0x4C2ACBD: free (vg_replace_malloc.c:530)
==25110== by 0x9F773AC: closedir (in /usr/lib64/libc-2.17.so)
==25110== by 0x40E7AC: get_link_name (test_BlockSpecial.cc:153)
==25110== by 0x40E7AC: GParted::BlockSpecialTest_NamedBlockSpecialObjectBySymlinkMatches_Test::TestBody() (test_BlockSpecial.cc:247)
...
==25110== Block was alloc'd at
==25110== at 0x4C29BC3: malloc (vg_replace_malloc.c:299)
==25110== by 0x9F77280: __alloc_dir (in /usr/lib64/libc-2.17.so)
==25110== by 0x40E746: get_link_name (test_BlockSpecial.cc:134)
==25110== by 0x40E746: GParted::BlockSpecialTest_NamedBlockSpecialObjectBySymlinkMatches_Test::TestBody() (test_BlockSpecial.cc:247)
...
[ OK ] BlockSpecialTest.NamedBlockSpecialObjectBySymlinkMatches (50 ms)
Selected lines from test_BlockSpecial.cc:
132 static std::string get_link_name()
133 {
134 DIR * dir = opendir( "/dev/disk/by-id" );
...
141 bool found = false;
142 struct dirent * dentry;
143 // Silence GCC [-Wparentheses] warning with double parentheses
144 while ( ( dentry = readdir( dir ) ) )
145 {
146 if ( strcmp( dentry->d_name, "." ) != 0 &&
147 strcmp( dentry->d_name, ". " ) != 0 )
148 {
149 found = true;
150 break;
151 }
152 }
153 closedir( dir );
154
155 if ( found )
156 return std::string( "/dev/disk/by-id/" ) + dentry->d_name;
So the memory referred to by dentry was allocated on line 134, freed on
153 and accessed after freed on 156.
Closes!41 - Fix test (dentry->d_name is invalidated by closedir...)
Use of Glib::thread_init() was deprecated in glibmm 2.32 [1]. The
oldest supported distributions have these versions:
Debian 8 glibmm 2.42.0
RHEL / CentOS 7 glibmm 2.56.0
SLES 12 glibmm 2.38.1
Ubuntu 14.04 LTS glibmm 2.39.93
Checking further the glibmm 2.32 reference manual says this about
Glib::thread_init() [2]:
Initializes the GLib thread system.
Deprecated:
Calling thread_init() is no longer necessary and no longer has any
effect.
However only some of the glibmm example programs had Glib::thread_init()
removed, others had it replaced by Glib::init() [3]. Again the glibmm
2.32 reference manual says this about Glib::init() [4]:
Initialize glibmm.
You may call this more than once. You do not need to call this if
you are using Glib::MainLoop or Gtk::Main, because they call it for
you.
GParted does call Gtk::Main and test_PipeCapture does call
Glib::MainLoop. Therefore just raise the minimum version to glibmm 2.32
and remove both calls to Glib::thread_init().
[1] Glibmm 2.32 NEWS file
https://gitlab.gnome.org/GNOME/glibmm/blob/2.32.0/NEWS#L207
[2] glibmm 2.32, glibmm: Glib Namespace Reference, Glib::thread_init()
https://developer.gnome.org/glibmm/2.32/namespaceGlib.html#ab26d01c776801f1fff00753e97af4fc7
[3] glibmm commit "Avoid use of deprecates API in tests and examples."
3e0fbb22c0
[4] glibmm 2.32, glibmm: Glib Namespace Reference, Glib::init()
https://developer.gnome.org/glibmm/2.32/namespaceGlib.html#ac90aee10d0b90e3d8a96a86b5394f87bCloses!22 - Increase minimums to libparted 2.2 and glibmm 2.32
Replace the insert() method (which reports an error when inserting a
password with a key which already exists) with the store() method which
replaces or inserts the password depending on whether the key already
exists or not respectively. There is also an optimisation that nothing
is changed if the password to be replaced is the same as the one already
stored. The code in Win_GParted::open_encrypted_partition() is
simplified now it doesn't have to implement this pattern of behaviour
itself.
Bug 795617 - Implement opening and closing of LUKS mappings
Use private access into the PasswordRAMStore class to directly obtain
the address of the locked memory, rather than inferring it from the
address of the first stored password. This simplifies
PasswordRAMStoreTest::SetUpTestCase() and avoids encoding most of the
implementation knowledge that the first password will be stored at the
start of the protected memory.
Bug 795617 - Implement opening and closing of LUKS mappings
Test that all passwords are zeroed by PasswordRAMStore::erase_all(), the
same method as used in the PasswordRAMStore destructor.
Bug 795617 - Implement opening and closing of LUKS mappings
As noted in comments:
1) This is white box testing because it uses implementation knowledge
to look through the API to the internals of the password store.
2) It is not currently possible to test that the passwords are zeroed
when the store is destroyed.
However zeroing of memory is being tested when individual passwords
are erased.
Bug 795617 - Implement opening and closing of LUKS mappings
Found that older but still supported distributions Debian 8 and
Ubuntu 14.04 LTS don't have directory /dev/disk/by-path/. This is used
by the BlockSpecial unit test as a source of a symbolic link to a block
special device.
This causes the unit test to fail like this:
$ cd tests
$ ./test_BlockSpecial
...
[ RUN ] BlockSpecialTest.NamedBlockSpecialObjectBySymlinkMatches
test_BlockSpecial.cc:137: Failure
Failed
get_link_name(): Failed to open directory '/dev/disk/by-path'
test_BlockSpecial.cc:168: Failure
Failed
follow_link_name(): Failed to resolve symbolic link ''
test_BlockSpecial.cc:255: Failure
Expected: (lnk.m_name.c_str()) != (bs.m_name.c_str()), actual: "" vs ""
[ FAILED ] BlockSpecialTest.NamedBlockSpecialObjectBySymlinkMatches (0 ms)
...
[ FAILED ] 1 test, listed below:
[ FAILED ] BlockSpecialTest.NamedBlockSpecialObjectBySymlinkMatches
1 FAILED TEST
Which in turn causes make check and make distcheck to fail.
Use directory /dev/disk/by-id/ instead as it always exists.
Make distcheck is failing like:
<snip>
Make distcheck
Making distclean in tests
make[2]: Entering directory
'/home/gedakc/workspace/gparted/gparted-0.29.0-beta1/_build/sub/tests'
Makefile:658: ../src/.deps/BlockSpecial.Po: No such file or directory
Makefile:659: ../src/.deps/PipeCapture.Po: No such file or directory
make[2]: *** No rule to make target '../src/.deps/PipeCapture.Po'. Stop.
make[2]: Leaving directory
'/home/gedakc/workspace/gparted/gparted-0.29.0-beta1/_build/sub/tests'
Makefile:596: recipe for target 'distclean-recursive' failed
make[1]: *** [distclean-recursive] Error 1
make[1]: Leaving directory
'/home/gedakc/workspace/gparted/gparted-0.29.0-beta1/_build/sub'
Makefile:805: recipe for target 'distcheck' failed
make: *** [distcheck] Error 1
Make distcheck started failing like this with this commit:
f31808989a
Silence subdir-objects warning from automake 1.14 (#781978)
However, also need to be referring to source files from a sibling
directory like this from tests/Makefile.am:
test_BlockSpecial_SOURCES = test_BlockSpecial.cc ../src/BlockSpecial.cc
^^^^^^^
First introduced by this commit:
c37be28148
Add unit tests for BlockSpecial constructors and internal caching (#781978)
This failure, if not exactly common, has been seen a number of times
before. Using this hint from a GNU Automake forum post:
converting to subdir-objects
http://gnu-automake.7480.n7.nabble.com/converting-to-subdir-objects-td21724.html
"I had the same issue and wouldn't care unless Automake 1.15 started
to warn about "source file in a subdirectory". In my case 'bar.o' is
also built from 'foo/Makefile' so I decided to skip using 'bar.c' as
SOURCE and instead link in 'foo/bar.o'.
http://dev.thep.lu.se/svndigest/changeset/1581
Cheers,
Peter
"
Fix by avoiding referring to source files from a sibling directory when
building the unit test programs, instead refer to object files instead.
Note that the automake subdir-objects option is still required to
silence the warning because of the use of source files in subdirectories
by lib/gtest/Makefile.am.
Bug 785308 - make distcheck fails with *.Po: No such file or directory
Mike Fleetwood