Avoid having to manually maintain the list of excluded File System tests
in the GitLab Docker CI image. Scan the unit test source extracting
those tests marked with SKIP_IF_NOT_ROOT_FOR_REQUIRED_LOOPDEV_FOR_FS()
to automatically construct the setting for the GTEST_FILTER environment
variable.
Closes!105 - Update used btrfs file system commands, new minimum is
btrfs-progs 4.5
Now that reading btrfs usage, UUID and label can be performed on a file
system image remove the need for a loop device for the relevant unit
tests.
Closes!105 - Update used btrfs file system commands, new minimum is
btrfs-progs 4.5
GParted has been using 'btrfs filesystem show' to report file system
usage but that doesn't work on a file system image so doesn't work in a
GitLab CI test job, as discussed earlier in this patchset.
There is 'btrfs inspect-internal min-dev-size' but:
1. That only works on a mounted file system and GParted isn't going to
mount an unmounted file system just to query it's used space, so by
extension won't work on image files.
2. It reports a figure which is almost the same as the chunk usage of
the device within the btrfs file system. However if some files have
been deleted leaving chunks partially used, then 'btrfs filesystem
resize' will successfully shrink a btrfs smaller than the reported
minimum device size.
And there is also 'btrfs filesystem usage' but that also only works on a
mounted file system.
So instead use 'btrfs inspect-internal dump-super' to report some of the
figures previously obtained from 'btrfs filesystem show'. For example
for a single device btrfs in an image file:
$ truncate -s 256M /tmp/test.img
$ mkfs.btrfs /tmp/test.img
$ btrfs inspect-internal dump-super /tmp/test.img | egrep 'total_bytes|bytes_used|sectorsize|devid'
total_bytes 268435456
bytes_used 114688
sectorsize 4096
dev_item.total_bytes 268435456
dev_item.bytes_used 92274688
dev_item.devid 1
Comparing with results from 'btrfs filesystem show' for the same file
system, after adding a loop device to allow 'btrfs filesystem show' to
succeed:
$ su -
# losetup --find --show /tmp/test.img
# btrfs filesystem show --raw /dev/loop0
Label: none uuid: 32a1eb31-4691-41ae-9ede-c45d723655a3
Total devices 1 FS bytes used 114688
devid 1 size 268435456 used 92274688 path /dev/loop0
This does bring a forced change in the calculation which affects multi-
device btrfs file systems. 'btrfs filesystem show' provided chunk
allocation information per device ("used" figure for each "devid"). The
file system wide used bytes ("FS bytes used") was apportioned according
to the fraction of the chunk allocation each device contained. However
'btrfs inspect-internal dump-super' doesn't provide chunk allocation
information for all devices, only for the current device
("dev_item.bytes_used"). Instead the calculation now has to apportion
the file system wide used bytes ("bytes_used") according to the fraction
of the size of the current device ("dev_item.total_bytes") within the
total size ("total_bytes").
This can't make any difference to a single device btrfs file system as
both fractions will be 1. It only affects how the file system wide used
bytes is distributed among multiple devices.
As an example to see the difference between calculation methods, create
a 2 GiB btrfs taking the defaults so getting duplicated metadata and
single data. Add another 2 GiB partition and populate with some files.
# mkfs.btrfs /dev/sdb1
btrfs-progs v4.15.1
See http://btrfs.wiki.kernel.org for more information.
Label: (null)
UUID: 68195e7e-c13f-4095-945f-675af4b1a451
Node size: 16384
Sector size: 4096
Filesystem size: 2.00GiB
Block group profiles:
Data: single 8.00MiB
Metadata: DUP 102.38MiB
System: DUP 8.00MiB
SSD detected: no
Incompat features: extref, skinny-metadata
Number of devices: 1
Devices:
ID SIZE PATH
1 2.00GiB /dev/sdb1
# mount /dev/sdb1 /mnt/1
# btrfs device add /dev/sdc1 /mnt/1
# cp -a /home/$USER/programming/c/gparted/ /mnt/1/
Usage figures using the old calculation apportioning file system wide
usage according to chunk allocation per device:
# btrfs filesystem show --raw /dev/sdb1
Label: none uuid: 68195e7e-c13f-4095-945f-675af4b1a451
Total devices 2 FS bytes used 178749440
devid 1 size 2147483648 used 239861760 path /dev/sdb1
devid 2 size 2147483648 used 436207616 path /dev/sdc1
sum_devid_used = 239861760 + 436207616
= 676069376
sdb1 usage = 178749440 * 239861760 / 676069376
= 63418277
sdc1 usage = 178749440 * 436207616 / 676069376
= 115331163
Usage figures using the new calculation apportioning file system wide
usage according to device sizes:
# btrfs inspect-internal dump-super /dev/sdb1 | egrep 'total_bytes|^bytes_used'
total_bytes 4294967296
bytes_used 178749440
dev_item.total_bytes 2147483648
# btrfs inspect-internal dump-super /dev/sdc1 | egrep 'total_bytes|^bytes_used'
total_bytes 4294967296
bytes_used 178749440
dev_item.total_bytes 2147483648
sdb1 usage = 178749440 * 2147483648 / 4294967296
= 89374720
sdc1 usage = 178749440 * 2147483648 / 4294967296
= 89374720
Both calculation methods ignore that btrfs allocates chunks at the
volume manager level. So when fully compacted the last chunk for
metadata and data for each storage profile (RAID level) will be
partially filled and this is not accounted for.
Also for multi-device btrfs file systems the new calculation provides
different results. However given that shrinking a device in a multi-
device btrfs file system can and does relocate extents to other devices
(redundancy requirements of chunks permitting) it's minimum size is
virtually impossible to calculate and may not restrict how small the
btrfs device can be shrunk anyway. If it turns out that this new
calculation causes problems it's been made a separate commit from the
previous commit for easier reverting.
Closes!105 - Update used btrfs file system commands, new minimum is
btrfs-progs 4.5
'btrfs filesystem show' only used to report rounded human readable size
figures. Therefore the actual figure could have been anywhere within
the rounding limit. GParted also applied a heuristic to snap the file
system size figure to the partition size if the partition size was
within the rounding limit of the reported file system size [1].
btrfs-progs v4.1 added the --raw option to print the figures in bytes
[2][3][4].
# btrfs filesystem show --raw /dev/sdb1
Label: none uuid: 003a619e-856f-4b9c-bd29-4d0ae0296d66
Total devices 2 FS bytes used 178765824
devid 1 size 2147483648 used 239861760 path /dev/sdb1
devid 2 size 2147483648 used 436207616 path /dev/sdc1
Since the oldest supported distributions now use btrfs-progs v4.5.3 and
later (see the distribution End-of-Life table in the previous commit
message), unconditionally use this to get accurate figures.
[1] 7fc16a1b69
Handle btrfs tools rounding of figures (#499202)
[2] btrfs-progs: Allow "filesystem show" command to handle different units
https://git.kernel.org/pub/scm/linux/kernel/git/kdave/btrfs-progs.git/commit/?id=15379fa2257bf937cf7830c0b1b79f2daf5df72c
[3] btrfs-progs: docs: new size options for fi show
https://git.kernel.org/pub/scm/linux/kernel/git/kdave/btrfs-progs.git/commit/?id=81225f11d9ea58590476612e69211113ddb9b943
[4] Btrfs progs release 4.1
https://lore.kernel.org/linux-btrfs/20150622150023.GX6761@twin.jikos.cz/Closes!105 - Update used btrfs file system commands, new minimum is
btrfs-progs 4.5
GParted so far uses 'btrfs filesystem show' to read the file system
UUID. But this doesn't work on a file system image so doesn't work in
the GitLab CI test jobs, as discussed in the earlier commit "Use btrfs
filesystem label to read the FS label (!105)".
$ truncate -s 256M /tmp/test.img
$ mkfs.btrfs /tmp/test.img
...
UUID: 5ea62f88-fef3-4ece-a726-b88f3d81fe1c
...
$ btrfs filesystem show /tmp/test.img
ERROR: not a valid btrfs filesystem: /tmp/test.img
Instead use 'btrfs inspect-internal dump-super' which works on image
files too.
$ btrfs inspect-internal dump-super /tmp/test.img
...
fsid 5ea62f88-fef3-4ece-a726-b88f3d81fe1c
...
'btrfs inspect-internal dump-super' was added in btrfs-progs 4.5 [1][2]
so is available in the oldest supported distributions and can be used
unconditionally.
Distro EOL btrfs --version
Debian 10 2022-Jun v4.20
RHEL / CentOS 7 2024-Jun v4.9.1
Ubuntu 18.04 LTS 2023-Apr v4.15.1
SLES 12 SP5 2024-Oct v4.5.3 [3][4]
Unfortunately it returns 0 status on failure so use non-empty stderr to
identify failure.
$ rm /tmp/test.img
$ truncate -s 256M /tmp/test.img
$ btrfs inspect-internal dump-super /tmp/test.img 1> /dev/null
ERROR: bad magic on superblock on /tmp/test.img at 65536
$ echo $?
0
[1] btrfs-progs: introduce inspect-internal dump-super
https://git.kernel.org/pub/scm/linux/kernel/git/kdave/btrfs-progs.git/commit/?id=eaa93e3b0295fc94c774ec73056559a6b8c78b42
[2] Btrfs progs release 4.5
https://lore.kernel.org/linux-btrfs/20160320235330.GG21722@suse.cz/
"* new/moved commands
* btrfs-show-super -> btrfs inspect-internal dump-super
"
[3] SUSE Long Term Service Pack Support
https://links.imagerelay.com/cdn/3404/ql/f3a083e9bcd34c76addd096d7f60ec00/long_term_service_pack_support_flyer.pdf
[4] SUSE package search
https://scc.suse.com/packages?name=SUSE%20Linux%20Enterprise%20Server&version=12.5&arch=x86_64&query=btrfsprogs&module=Closes!105 - Update used btrfs file system commands, new minimum is
btrfs-progs 4.5
Create function to replace repeated code which optionally removes
trailing new line character from a string.
Closes!105 - Update used btrfs file system commands, new minimum is
btrfs-progs 4.5
Until now GParted has run 'btrfs filesystem show' to read the file
system label. This has a number of issues and limitations:
1. Doesn't work on a file system image so can't be unit tested in the
GitLab CI test job where a loop device can't be created [1] or as a
non-root user.
2. Reported failed exit status 1 when successfully showing a mounted
btrfs, but only when using btrfs-progs v3.14 and 3.14.1 [2][3].
3. Failed to distinguish between label set to "none" and no label
reported as none, but only when mounted and with btrfs-progs v3.12
[3].
As non-root user run btrfs read label unit test:
$ tests/test_SupportedFileSystems --gtest_filter='*CreateAndReadLabel/btrfs'
...
[ RUN ] My/SupportedFileSystemsTest.CreateAndReadLabel/btrfs
test_SupportedFileSystem.cc:539: Skip test. Not root to be able to create required loop device
[ OK ] My/SupportedFileSystemsTest.CreateAndReadLabel/btrfs (0 ms)
Even as root, 'btrfs filesystem show' fails to work on an image file:
# truncate -s 256M /tmp/test.img
# mkfs.btrfs /tmp/test.img
# btrfs filesystem show /tmp/test.img
ERROR: not a valid btrfs filesystem: /tmp/test.img
# echo $?
1
# rm /tmp/test.img
Instead use 'btrfs filesystem label' to read the label. It also works
on an image file, the exit status is informative and output is just the
label followed by a new line character [4] so very simple to parse.
Error case:
$ truncate -s 256M /tmp/test.img
$ btrfs filesystem label /tmp/test.img
No valid Btrfs found on /tmp/test.img
$ echo $?
255
No label case:
$ mkfs.btrfs /tmp/test.img
$ btrfs filesystem label /tmp/test.img
$ echo $?
0
$ btrfs filesystem label /tmp/test.img | hexdump -C
00000000 0a |.|
00000001
Label case:
$ mkfs.btrfs -L 'label with
> new line and trailing space ' /tmp/test.img
$ btrfs filesystem label /tmp/test.img
label with
new line and trailing space
$ echo $?
0
$ btrfs filesystem label /tmp/test.img | hexdump -C
00000000 6c 61 62 65 6c 20 77 69 74 68 0a 6e 65 77 20 6c |label with.new l|
00000010 69 6e 65 20 61 6e 64 20 74 72 61 69 6c 69 6e 67 |ine and trailing|
00000020 20 73 70 61 63 65 20 0a | space .|
00000028
Run 'btrfs filesystem label' always passing the block device as this
works for both mounted and unmounted file systems. This is in
contrast to writing the label for a mounted btrfs where the mount
mount must be used [5].
# mkfs.btrfs -L 'test label' /dev/sdb1
# btrfs filesystem label /dev/sdb1
test label
# mount /dev/sdb1 /mnt/1
# btrfs filesystem label /dev/sdb1
test label
[1] 07ad43a107
Create loop devices for BTRFS read file system interface tests (!49)
[2] 82c6265fa5
Update parsing of btrfs filesystem show for the UUID (#733601)
[3] eca732fb0c
Update parsing of btrfs filesystem show for the label (#733601)
[4] btrfs-progs v3.14, cmd_label() function
https://git.kernel.org/pub/scm/linux/kernel/git/kdave/btrfs-progs.git/tree/cmds-filesystem.c?h=v3.14#n984
[5] eb034b1759
Add labelling of mounted btrfs (#163)
Closes!105 - Update used btrfs file system commands, new minimum is
btrfs-progs 4.5
Now that the docker image for ubuntu:latest has updated to Ubuntu 22.04
LTS [1] and exfatprogs is available [2] add the package to the CI job to
include it in the testing.
[1] Ubuntu Docker official image
https://hub.docker.com/_/ubuntu/
"Supported tags and respective Dockerfile links
...
22.04, ..., latest, ...
"
[2] Ubuntu > Packages > jammy (22.04LTS) > otherosfs > exfatprogs
https://packages.ubuntu.com/jammy/exfatprogs
[3] 3783cb4173
Add unit testing of GParted exFAT interface (!30)
For FAT16/32 and XFS file systems clearing the label uses different
command options and code path in file system specific ::write_label()
method. Therefore extend this unit test to also test clearing the
label.
Now fix the error with GParted clearing the label when setting a new
UUID on a FAT16/32 file system. Reproduce the issue on the command
line:
# mkfs.fat -F 16 -v -I -n TEST_LABEL /dev/sdb1
# mdir -f -i /dev/sdb1 ::/
Volume in drive : is TEST_LABEL
Volume Serial Number is 5D4C-6E6E
...
# mlabel -n -i /dev/sdb1 ::
# mdir -f -i /dev/sdb1 ::/
Volume in drive : has no label
Volume Serial Number is 77BB-A883
...
This was broken by commit "Fix writing FAT16/32 FS UUID on Alpine Linux
(!104)" earlier in this patchset, which included this comment:
"... Also drop the '-s' option
as showing the current label is unrelated to writing a new UUID."
It is not mentioned in the mlabel[1] manual page that option -s is
needed in order to avoid clearing the label when assigning a new UUID.
Anyway add the option back.
[1] mlabel(1)
https://linux.die.net/man/1/mlabel
"s Shows the existing label, without prompting the user.
n Assigns a new (random) serial number to the disk
"
Closes!104 - Add Alpine Linux CI jobs and resolve label and UUID issues
with FAT16/32
During review and testing of this patchset it was discovered that using
GParted to set a new UUID on a FAT16 or FAT32 file system that there was
a new unwanted side effect of clearing the label.
Add unit test to cover this error scenario. It does the following:
1. Creates a file system with a known label;
2. Writes a new UUID;
3. Reads the label and confirms it matches the initial label.
This new unit test captures the fault like this:
$ ./test_SupportedFileSystems --gtest_filter='*CreateAndWriteUUIDAndReadLabel*'
...
[ RUN ] My/SupportedFileSystemsTest.CreateAndWriteUUIDAndReadLabel/fat16
test_SupportedFileSystems.cc:645: Failure
Expected equality of these values:
fs_label
Which is: "TEST_LABEL"
m_partition.get_filesystem_label().c_str()
Which is: ""
[ FAILED ] My/SupportedFileSystemsTest.CreateAndWriteUUIDAndReadLabel/fat16, where GetParam() = 13 (21 ms)
[ RUN ] My/SupportedFileSystemsTest.CreateAndWriteUUIDAndReadLabel/fat32
test_SupportedFileSystems.cc:645: Failure
Expected equality of these values:
fs_label
Which is: "TEST_LABEL"
m_partition.get_filesystem_label().c_str()
Which is: ""
[ FAILED ] My/SupportedFileSystemsTest.CreateAndWriteUUIDAndReadLabel/fat32, where GetParam() = 14 (22 ms)
Don't forget to exclude this unit test for file systems which need a
loop device but which fails to be created inside the docker CI image.
Reference:
39fdfe51da
Exclude unit tests needing losetup in Docker CI image (!59)
Closes!104 - Add Alpine Linux CI jobs and resolve label and UUID issues
with FAT16/32
Unit test writing FAT16/32 file system UUIDs fails on Alpine Linux like
this:
$ ./test_SupportedFileSystems --gtest_filter='*CreateAndWriteUUID/fat16'
...
[ RUN ] My/SupportedFileSystemsTest.CreateAndWriteUUID/fat16
test_SupportedFileSystems.cc:616: Failure
Value of: m_fs_object->write_uuid(m_partition, m_operation_detail)
Actual: false
Expected: true
Operation details:
mkfs.fat -F16 -v -I '/home/alpine/programming/c/gparted/tests/test_SupportedFileSystems.img' 00:00:00 (SUCCESS)
...
mlabel -s -n :: -i '/home/alpine/programming/c/gparted/tests/test_SupportedFileSystems.img' 00:00:00 (ERROR)
Mtools version 4.0.39, dated April 10th, 2022
Usage: mlabel [-vscVn] [-N serial] drive:
[ FAILED ] My/SupportedFileSystemsTest.CreateAndWriteUUID/fat16, where GetParam() = 13 (38 ms)
Using GParted on Alpine Linux to perform the same action produces the
same error in the operation results. Reproduce this on the command
line:
# mkfs.fat -F 16 -v -I /dev/sdb1
# mlabel -s -n :: -i /dev/sdb1
Mtools version 4.0.39, dated April 10th, 2022
Usage: mlabel [-vscVn] [-N serial] drive:
# echo $?
1
Again fix the same way, by moving the non-option '::' drive
specification to the end of the command line. Also drop the '-s' option
as showing the current label is unrelated to writing a new UUID.
# mdir -f -i /dev/sdb1 ::/ | grep 'Volume Serial Number is'
Volume Serial Number is B97E-59A3
# mlabel -n -i /dev/sdb1 ::
# echo $?
0
# mdir -f -i /dev/sdb1 ::/ | grep 'Volume Serial Number is'
Volume Serial Number is 1552-96A6
Closes!104 - Add Alpine Linux CI jobs and resolve label and UUID issues
with FAT16/32
Unit test reading FAT16/32 file system UUIDs fails on Alpine Linux like
this:
$ ./test_SupportedFileSystems --gtest_filter='*CreateAndReadUUID/fat16'
....
[ RUN ] My/SupportedFileSystemsTest.CreateAndReadUUID/fat16
test_SupportedFileSystems.cc:581: Failure
Expected: (m_partition.uuid.size()) >= (9U), actual: 0 vs 9
test_SupportedFileSystems.cc:584: Failure
Value of: m_partition.get_messages().empty()
Actual: false
Expected: true
Partition messages:
Drive '::' not supported
Cannot initialize '::'
Drive 'A:' not supported
Cannot initialize 'A:'
Drive 'A:' not supported
Cannot initialize 'A:'
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadUUID/fat16, where GetParam() = 13 (28 ms)
This doesn't normally affect GParted because it uses blkid as first
choice to read file system UUIDs, only using file system specific
commands when blkid isn't available. Reproduce this on the command
line:
# mkfs.fat -F 16 -v -I /dev/sdb1
# mdir -f :: -i /dev/sdb1
Drive '::' not supported
Cannot initialize '::'
Drive 'A:' not supported
Cannot initialize 'A:'
Drive 'A:' not supported
Cannot initialize 'A:'
Again, this is caused by having non-option '::' drive specification
before all the options on the mdir command line, which isn't supported
by the POSIX strict getopt(3) on Alpine Linux. Apply the same fix of
moving the non-option argument to the end.
# mdir -f -i /dev/sdb1 ::/
Volume is drive : has no label
Volume Serial Number is 7DC9-BCD9
Director for ::/
No files
# echo $?
0
Closes!104 - Add Alpine Linux CI jobs and resolve label and UUID issues
with FAT16/32
Unit test writing FAT16/32 file system labels fails on Alpine Linux like
this:
$ ./test_SupportedFileSystems --gtest_filter='*CreateAndWriteLabel/fat16'
...
[ RUN ] My/SupportedFileSystemsTest.CreateAndWriteLabel/fat16
test_SupportedFileSystems.cc:601: Failure
Value of: m_fs_object->write_label(m_partition, m_operation_detail)
Actual: false
Expected: true
Operation details:
mkfs.fat -F16 -v -I -n 'FIRST ' '/home/alpine/programming/c/gparted/tests/test_SupportedFileSystems.img' 00:00:00 (SUCCESS)
...
mlabel ::'SECOND ' -i '/home/alpine/programming/c/gparted/tests/test_SupportedFileSystems.img' 00:00:00 (ERROR)
Mtools version 4.0.39, dated April 10th, 2022
Usage: mlabel [-vscVn] [-N serial] drive:
[ FAILED ] My/SupportedFileSystemsTest.CreateAndWriteLabel/fat16, where GetParam() = 13 (29 ms)
Using GParted on Alpine Linux to perform the same action produces the
same error in the operation results. Reproduce this on the command
line:
# mkfs.fat -F 16 -v -I -n FIRST /dev/sdb1
# mlabel ::SECOND -i /dev/sdb1
Mtools version 4.0.39, dated April 10th, 2022
Usage: mlabel [-vscVn] [-N serial] drive:
# echo $?
1
Again, this is because musl libc's getopt(3) is POSIX compliant and
stops parsing options at '::', the first non-option argument. Apply the
same fix of moving the non-option argument to the end of the mlabel
command line:
# mlabel -i /dev/sdb1 ::SECOND
# echo $?
0
# mlabel -s -i /dev/sdb1
Volume label is SECOND
And for the clearing label case:
# mlabel -c -i /dev/sdb1 ::
# echo $?
0
# mlabel -s -i /dev/sdb1
Volume has no label
Closes!104 - Add Alpine Linux CI jobs and resolve label and UUID issues
with FAT16/32
Several of the FAT16/32 file system unit tests fail on Alpine Linux. In
this commit we are just looking at the failure to read the label. The
test fails like this:
$ ./test_SupportedFileSystems --gtest_filter='*CreateAndReadLabel/fat16'
...
[ RUN ] My/SupportedFileSystemsTest.CreateAndReadLabel/fat16
test_SupportedFileSystems.cc:551: Failure
Expected equality of these values:
fs_label
Which is: "TEST_LABEL"
m_partition.get_filesystem_label().c_str()
Which is: ""
test_SupportedFileSystems.cc:554: Failure
Value of: m_partition.get_messages().empty()
Actual: false
Expected: true
Partition messages:
Mtools version 4.0.39, dated April 10th, 2022
Usage: mlabel [-vscVn] [-N serial] drive:
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadLabel/fat16, where GetParam() = 13 (21 ms)
The same error can be seen by using GParted to display a FAT16 or FAT32
file system on Alpine Linux. The Partition Information dialog displays
this warning:
Mtools version 4.0.39, dated April 10th, 2022
Usage: mlabel [-vscVn] [-N serial] drive:
Reproduce this on the command line:
# mkfs.fat -F 16 -v -I -n TEST_LABEL /dev/sdb1
# mlabel -s :: -i /dev/sdb1
Mtools version 4.0.39, dated April 10th, 2022
Usage: mlabel [-vscVn] [-N serial] drive:
# echo $?
1
The mlabel.c source [1] uses getopt(3) to parse the command line
arguments. musl libc's [2] getopt(3) must be strictly POSIX compliant
[3][4] and stops reading options at the first non-option argument, '::'
in this case. Move the non-option argument to the end of the command
line and it works:
# mlabel -s -i /dev/sdb1 ::
Volume label is TEST_LABEL
Where as GNU Libc's getopt(3) [5] says that by default it reorders argv
eventually moving all non-option arguments to the end, hence why this
has worked on every Linux distribution using GNU Libc. This can be
broken on any Linux distribution using GNU Libc by enforcing strict
POSIX behaviour from getopt(3). For example on Fedora 36:
# mkfs.fat -F 16 -v -I -n TEST_LABEL /dev/sdb1
# export POSIXLY_CORRECT=1
# mlabel -s :: -i /dev/sdb1
Mtools version 4.0.39, dated April 10th, 2022
Usage: mlabel [-vscVn] [-N serial] drive:
# echo $?
1
# mlabel -s -i /dev/sdb1 ::
Hidden (2048) does not match sectors (63)
Volume label is TEST_LABEL
# echo $?
0
Fix by moving the non-option (image file drive specification) '::' to
the end of the mlabel command line.
[1] Mtools
https://www.gnu.org/software/mtools/
[2] musl libc
https://musl.libc.org/
"musl is an implementation of the C standard library built on top of
the Linux system call API, including interfaces defined in the base
language standard, POSIX, and widely agreed-upon extensions.
"
[3] POSIX.1-2017, Functions, getopt
https://pubs.opengroup.org/onlinepubs/9699919799/functions/getopt.html
[4] getopt(3p)
https://man7.org/linux/man-pages/man3/getopt.3p.html
[5] getopt(3)
https://www.man7.org/linux/man-pages/man3/getopt.3.html
"By default, getopt() permutes the contents of argv as it scans, so
that eventually all the nonoptions are at the end. Two other
scanning modes are also implemented. If the first character of
optstring is '+' or the environment variable POSIXLY_CORRECT is set,
then option processing stops as soon as a nonoption argument is
encountered.
"
Closes!104 - Add Alpine Linux CI jobs and resolve label and UUID issues
with FAT16/32
Follows the "Return Early" design pattern making the code easier to
understand without having to remember cases for elses or cascading ifs.
Refactor before the following commit's fix so that capture of output on
failure can be confirmed as still working.
Closes!104 - Add Alpine Linux CI jobs and resolve label and UUID issues
with FAT16/32
The test CI job on Alpine Linux fails like this:
$ GTEST_FILTER+=':My/SupportedFileSystemsTest.CreateAndReadUsage/btrfs'
/bin/sh: eval: line 135: GTEST_FILTER+=:My/SupportedFileSystemsTest.CreateAndReadUsage/btrfs: not found
This is because the busybox ash shell in Alpine Linux doesn't support +=
syntax for variable concatenation. Use plain variable assignment
instead.
Closes!104 - Add Alpine Linux CI jobs and resolve label and UUID issues
with FAT16/32
There have been a number of GParted build issues [1][2] recently on
Alpine Linux because it uses musl libc [3] which is stricter to POSIX,
rather than the GNU C Library (glibc) which has numerous enhancements.
Glibc is used by most Linux distributions, including CentOS and Ubuntu
already used in the GNOME Continuous Integration jobs. So add a GParted
build job on Alpine Linux to catch these issues in future. Uses the
docker image of the latest Alpine Linux release.
[1] 3d4b1c1e7b
Fix NULL == 0 assumption in call to ped_partition_flag_next() (!100)
[2] 45c00927b7
Use POSIX basename() in BCache_Info.cc (!99)
[3] musl libc
https://musl.libc.org/Closes!104 - Add Alpine Linux CI jobs and resolve label and UUID issues
with FAT16/32
When the CentOS 7 CI jobs were failing on a subset of the job runners
[1] during March to May 2022, the docker image would hang even before
the packages were fully installed so cat /proc/version and cat
/etc/os-release were never run. Move them to the first thing done in
the docker image.
[1] Hanging of GitLab CI jobs on a subset of job runners
https://discourse.gnome.org/t/hanging-of-gitlab-ci-jobs-on-a-subset-of-job-runners/9931
Even though this is fixed the execution of configure as part of make
distcheck outputs this:
checking whether po/Makefile.in.in deletes intltool cache lock file... /usr/bin/grep: po/Makefile.in.in: No such file or directory
/usr/bin/sed: can't read po/Makefile.in.in: No such file or directory
/usr/bin/grep: po/Makefile.in.in: No such file or directory
no
make distcheck [1] performs a VPATH build with a read-only srcdir and
a separate writable build directory with files split between the two.
The relevant layout looks like:
./gparted-1.4.0-git/configure
./gparted-1.4.0-git/po/Makefile.in.in
./gparted-1.4.0-git/_build/sub/
And make distcheck runs configure like this:
cd ./gparted-1.4.0-git/_build/sub
../../configure --srcdir=../..
The file is ../../po/Makefile.in.in in this case, so not found by the
existing check. A simple investigation technique is to run make
distcheck, kill it shortly after configure completes and examine the
build tree. Definitely before make distcheck completes successfully and
deletes everything.
Fix by using $srcdir prefix to access the file. Also handle the case of
po/Makefile.in.in not existing, although this doesn't now occur in the
scenario fixed by this commit. And only patch the file if it's
writable, another case that doesn't occur in this scenario.
Relevant output line from configure run by make distcheck now looks
like:
checking whether po/Makefile.in.in deletes intltool cache lock file... yes
[1] GNU Automake, 14.4 Checking the Distribution
https://www.gnu.org/software/automake/manual/html_node/Checking-the-Distribution.htmlCloses!103 - Fix make distcheck failure found in GitLab CI job
unbuntu_test
On Ubuntu 22.04 LTS make distcheck fails like this:
$ make distcheck
...
ERROR: files left in build directory after distclean:
./po/.intltool-merge-cache.lock
make[1]: *** [Makefile:920: distcleancheck] Error 1
make[1]: Leaving directory '/builds/GNOME/gparted/gparted-1.4.0-git/_build/sub'
make: *** [Makefile:849: distcheck] Error 1
This was picked up by the GitLab ubuntu_test CI job after the Ubuntu
22.04 LTS release and the official Ubuntu docker image labelled latest
was updated to match, circa April 2022. This is a known issue with
intltool >= 0.51.0-5.1 [1][2][3], first included in Ubuntu 22.04 LTS.
The pending proposed fix is to also delete the left behind
.intltool-merge-cache.lock along with the associated cache file itself
in the intltool provided Makefile.in.in [4].
Applying a fix to the GitLab ubuntu_test CI job does nothing for fixing
it for us maintainers on our distributions. po/Makefile.in.in is not
part of the GParted git repository, instead it is copied from
/usr/share/intltool/Makefile.in.in by ./autogen.sh -> gnome-autogen.sh
-> intltoolize --force --copy --automake. Add a configure check which
patches po/Makefile.in.in as needed. This will fix it for those
building from git, and be a harmless check for those building from a tar
release. Configure output line looks like:
checking whether po/Makefile.in.in deletes intltool cache lock file... fixed
[1] Ubuntu bug 1712194 - Error when running make distcheck
https://bugs.launchpad.net/intltool/+bug/1712194
[2] Debian bug #991623 - intltool: make distcheck broken
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=991623
[3] Arch Linux bug FS#67098 - [intltool] latest patch for race condition
breaks some builds
https://bugs.archlinux.org/task/67098
[4] Remove cache lock file in mostlyclean
https://code.launchpad.net/~danbnicholson/intltool/intltool/+merge/406321Closes!103 - Fix make distcheck failure found in GitLab CI job
unbuntu_test
add_mountpoint_entry() doesn't modify the passed strings so use
pass-by-constant-reference. This avoids pass-by-value and having to
construct copies of the strings just to pass them to this method.
A user received the following error when attempting to resize a mounted
btrfs file system on their NixOS distribution:
Shrink /dev/nvme0n1p3 from 933.38 GiB to 894.32 GiB (ERROR)
+ calibrate /dev/nvme0n1p3 00:00:00 (SUCCESS)
+ btrfs filesystem resize 1:937759744K '/etc/machine-id' (ERROR)
ERROR: not a directory: /etc/machine-id
ERROR: resize works on mounted filesystems and accepts only
directories as argument. Passing file containing a btrfs image
would resize the underlying filesystem instead of the image.
In the partition table section of the gparted_details /dev/nvme0n1p3 was
reported with these mount points:
/etc/machine-id, /etc/NetworkManager/system-connections,
/etc/ssh/ssh_host_ed25519_key, /etc/ssh/ssh_host_ed25519_key.pub,
/etc/ssh/ssh_host_rsa_key, /etc/ssh/ssh_host_rsa_key.pub, /home,
/nix, /nix/store, /state, /var
The user had a common configuration of NixOS which boots with an empty
tmpfs as root with a few bind mounted files and directories to provide
the needed persistent data [1][2].
Re-create an equivalent situation:
1. Create a btrfs file system and mount it:
# mkfs.btrfs /dev/sdb1
# mkdir /mnt/store
# mount /dev/sdb1 /mnt/store
2. Bind mount a file from this file system else where in the hierarchy.
The only criteria is that this mount point sorts before /mnt/store.
# echo 'Test contents' > /mnt/store/test
# touch /boot/test
# mount --bind /mnt/store/test /boot/test
The kernel reports these mount mounts:
# grep sdb1 /proc/mounts
/dev/sdb1 /mnt/store btrfs rw,seclabel,relatime,space_cache=v2,subvolid=5,subvol=/ 0 0
/dev/sdb1 /boot/test btrfs rw,seclabel,relatime,space_cache=v2,subvolid=5,subvol=/ 0 0
3. Use GParted to resize this mounted btrfs file system. It fails with
the above error.
GParted read the mount points from /proc/mounts and sorted them. (See
the end of Mount_Info::load_cache() for the sorting). When resizing the
btrfs file system GParted just used the first sorted mount point. This
was the file /etc/machine-id for the user and file /boot/test in the
re-creation, hence the error.
Fix by selecting the first directory mount point to pass to the btrfs
resize command.
[1] NixOS tmpfs as root
https://elis.nu/blog/2020/05/nixos-tmpfs-as-root/
[2] Erase your darlings
https://grahamc.com/blog/erase-your-darlingsCloses#193 - path used to resize btrfs needs to be a directory
GParted fails to build on Alpine Linux Edge (development tree for the
next release) like this:
GParted_Core.cc: In constructor 'GParted::GParted_Core::GParted_Core()':
GParted_Core.cc:75:64: error: invalid 'static_cast' from type 'std::nullptr_t' to type 'PedPartitionFlag'
75 | for ( PedPartitionFlag flag = ped_partition_flag_next( static_cast<PedPartitionFlag>( NULL ) ) ;
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The code is failing to compile now because musl libc 1.2.3 has became
more C++11 strict [1][2] by defining NULL [3] as nullptr [4] rather than
as 0. The parameter to ped_partition_flag_next() [5] should always have
been numeral 0 cast to an enumeration and never the NULL pointer.
Fixes this commit [6] from 2004-12-27 which changed the parameter from 0
to NULL.
[1] define NULL as nullptr when used in C++11 or later
https://git.musl-libc.org/cgit/musl/commit?id=98e688a9da5e7b2925dda17a2d6820dddf1fb28
[2] NULL vs nullptr (Why was it replaced?) [duplicate]
https://stackoverflow.com/questions/20509734/null-vs-nullptr-why-was-it-replaced
[3] C++ reference, NULL
https://en.cppreference.com/w/cpp/types/NULL
[4] C++ reference, nullptr
https://en.cppreference.com/w/cpp/language/nullptr
[5] libparted Documentation, ped_partition_flag_next()
https://www.gnu.org/software/parted/api/group__PedPartition.html#g0ce9ce4247b320011bc8e9d957c8cdbb
[6] Added cylsize to Device and made Operation contain a Device instead
commit 174f0cff77Closes!100 - Fix NULL == 0 assumption in call to
ped_partition_flag_next()
Musl libc [1][2] doesn't implement the GNU variant of basename() [3][4],
obtained via #include <string.h>. Therefore GParted fails to build on
such distributions:
fdebug-prefix-map=TOPDIR/build/tmp/work/cortexa57-yoe-linux-musl/gparted/1.4.0-r0/recipe-sysroot-native=-fvisibility-inlines-hidden -c -o ../../gparted-1.4.0/src/BCache_Info.cc:52:33:
error: use of undeclared identifier 'basename'; did you mean 'g_basename'?
return "/dev/" + Glib::ustring(basename(buf));
^~~~~~~~
g_basename
Fix by using the POSIX implementation of basename() [5] instead,
obtained via #include <libgen.h>, which musl libc does implement [6].
Note that the POSIX implementation of basename() is allowed to modify
the string passed to it. This is okay because
BCache_Info::get_bcache_device() is using a modifiable local character
buffer.
[1] musl libc
https://musl.libc.org/
[2] Projects using musl
https://wiki.musl-libc.org/projects-using-musl.html
[3] The GNU C Library, 5.10 Finding Tokens in a String
https://www.gnu.org/software/libc/manual/html_node/Finding-Tokens-in-a-String.html
[4] basename(3) - Linux manual page
https://man7.org/linux/man-pages/man3/basename.3.html
[5] POSIX basename()
https://pubs.opengroup.org/onlinepubs/009695399/functions/basename.html
[6] musl source, basename.c
http://git.musl-libc.org/cgit/musl/tree/src/misc/basename.cCloses!99 - Fix undeclared identifier 'basename' build failure with
musl libc
Balázs Úr