'btrfs filesystem show' only used to report rounded human readable size
figures. Therefore the actual figure could have been anywhere within
the rounding limit. GParted also applied a heuristic to snap the file
system size figure to the partition size if the partition size was
within the rounding limit of the reported file system size [1].
btrfs-progs v4.1 added the --raw option to print the figures in bytes
[2][3][4].
# btrfs filesystem show --raw /dev/sdb1
Label: none uuid: 003a619e-856f-4b9c-bd29-4d0ae0296d66
Total devices 2 FS bytes used 178765824
devid 1 size 2147483648 used 239861760 path /dev/sdb1
devid 2 size 2147483648 used 436207616 path /dev/sdc1
Since the oldest supported distributions now use btrfs-progs v4.5.3 and
later (see the distribution End-of-Life table in the previous commit
message), unconditionally use this to get accurate figures.
[1] 7fc16a1b69
Handle btrfs tools rounding of figures (#499202)
[2] btrfs-progs: Allow "filesystem show" command to handle different units
https://git.kernel.org/pub/scm/linux/kernel/git/kdave/btrfs-progs.git/commit/?id=15379fa2257bf937cf7830c0b1b79f2daf5df72c
[3] btrfs-progs: docs: new size options for fi show
https://git.kernel.org/pub/scm/linux/kernel/git/kdave/btrfs-progs.git/commit/?id=81225f11d9ea58590476612e69211113ddb9b943
[4] Btrfs progs release 4.1
https://lore.kernel.org/linux-btrfs/20150622150023.GX6761@twin.jikos.cz/Closes!105 - Update used btrfs file system commands, new minimum is
btrfs-progs 4.5
Create function to replace repeated code which optionally removes
trailing new line character from a string.
Closes!105 - Update used btrfs file system commands, new minimum is
btrfs-progs 4.5
add_mountpoint_entry() doesn't modify the passed strings so use
pass-by-constant-reference. This avoids pass-by-value and having to
construct copies of the strings just to pass them to this method.
A user received the following error when attempting to resize a mounted
btrfs file system on their NixOS distribution:
Shrink /dev/nvme0n1p3 from 933.38 GiB to 894.32 GiB (ERROR)
+ calibrate /dev/nvme0n1p3 00:00:00 (SUCCESS)
+ btrfs filesystem resize 1:937759744K '/etc/machine-id' (ERROR)
ERROR: not a directory: /etc/machine-id
ERROR: resize works on mounted filesystems and accepts only
directories as argument. Passing file containing a btrfs image
would resize the underlying filesystem instead of the image.
In the partition table section of the gparted_details /dev/nvme0n1p3 was
reported with these mount points:
/etc/machine-id, /etc/NetworkManager/system-connections,
/etc/ssh/ssh_host_ed25519_key, /etc/ssh/ssh_host_ed25519_key.pub,
/etc/ssh/ssh_host_rsa_key, /etc/ssh/ssh_host_rsa_key.pub, /home,
/nix, /nix/store, /state, /var
The user had a common configuration of NixOS which boots with an empty
tmpfs as root with a few bind mounted files and directories to provide
the needed persistent data [1][2].
Re-create an equivalent situation:
1. Create a btrfs file system and mount it:
# mkfs.btrfs /dev/sdb1
# mkdir /mnt/store
# mount /dev/sdb1 /mnt/store
2. Bind mount a file from this file system else where in the hierarchy.
The only criteria is that this mount point sorts before /mnt/store.
# echo 'Test contents' > /mnt/store/test
# touch /boot/test
# mount --bind /mnt/store/test /boot/test
The kernel reports these mount mounts:
# grep sdb1 /proc/mounts
/dev/sdb1 /mnt/store btrfs rw,seclabel,relatime,space_cache=v2,subvolid=5,subvol=/ 0 0
/dev/sdb1 /boot/test btrfs rw,seclabel,relatime,space_cache=v2,subvolid=5,subvol=/ 0 0
3. Use GParted to resize this mounted btrfs file system. It fails with
the above error.
GParted read the mount points from /proc/mounts and sorted them. (See
the end of Mount_Info::load_cache() for the sorting). When resizing the
btrfs file system GParted just used the first sorted mount point. This
was the file /etc/machine-id for the user and file /boot/test in the
re-creation, hence the error.
Fix by selecting the first directory mount point to pass to the btrfs
resize command.
[1] NixOS tmpfs as root
https://elis.nu/blog/2020/05/nixos-tmpfs-as-root/
[2] Erase your darlings
https://grahamc.com/blog/erase-your-darlingsCloses#193 - path used to resize btrfs needs to be a directory
Continuing from the state in the previous commit, create an ext4 file
system using the previously created external journal and mount it.
# mke2fs -t ext4 -J device=/dev/sdb1 -L test-ext4 /dev/sdb2
# mount /dev/sdb2 /mnt/2
Did some experimenting with trying to create a second file system using
the same external journal which is already in use.
# mke2fs -t ext4 -J device=/dev/sdb1 -L 2nd-test-ext4 /dev/sdb3
...
/dev/sdb1 is apparently in use by the system; will not make a journal here!
# exit $?
1
Examined the source code of mke2fs and found that it performs an
exclusive read-only open of the named journal block device to check if
it is in use by the system or not [1]. Use the same method in GParted.
Not used alternative method would be to mark the jbd active when the
ext3/4 file system using it is active, but that requires working out the
linkage between them. That can be done using either blkid or dumpe2fs
output but that involves parsing more fields and caching more data so is
much more code than just testing the block device busy status using the
same method which mke2fs uses.
Matching UUIDs via blkid output.
# blkid /dev/sdb1 /dev/sdb2
/dev/sdb1: LABEL="test-jbd" UUID="6e52858e-0479-432f-80a1-de42f9a4093e" TYPE="jbd"
/dev/sdb2: LABEL="test-ext4" UUID="cea5c2cd-b21c-4abf-a497-8c073bb12300" EXT_JOURNAL="6e52858e-0479-432f-80a1-de42f9a4093e" TYPE="ext4"
Matching UUIDs via dumpe2fs output.
# dumpe2fs -h /dev/sdb1 | egrep 'Filesystem UUID|Journal users'
dumpe2fs 1.46.3 (27-Jul-2021)
Filesystem UUID: 6e52858e-0479-432f-80a1-de42f9a4093e
Journal users: cea5c2cd-b21c-4abf-a497-8c073bb12300
# dumpe2fs -h /dev/sdb2 | egrep 'Filesystem UUID|Journal UUID'
dumpe2fs 1.46.3 (27-Jul-2021)
Filesystem UUID: cea5c2cd-b21c-4abf-a497-8c073bb12300
Journal UUID: 6e52858e-0479-432f-80a1-de42f9a4093e
If GParted was going to show the journal to file system linkage in the
UI then doing this would be needed. However so far there has only been
a single reported case of a GParted user using an external journal,
therefore adding the code complexity for this feature is not currently
justified. The simple busy detection method used by mke2fs is all that
is needed.
[1] mke2fs source code
https://git.kernel.org/pub/scm/fs/ext2/e2fsprogs.git/
misc/mke2fs.c:main()
check_mount(journal_device, force, _("journal"));
misc/util.c:check_mount()
ext2fs_check_if_mounted(device, &mount_flags);
lib/ext2fs/ismounted.c:ext2fs_check_if_mounted()
ext2fs_check_mount_point(file, mount_flags, NULL, 0);
lib/ext2fs/ismounted.c:ext2fs_check_if_mounted()
if (stat(device, &st_buf) == 0 &&
ext2fsP_is_disk_device(st_buf.st_mode)) {
int fd = open(device, O_RDONLY | O_EXCL);
if (fd >= 0) {
/*
* The device is not busy so it's
* definitelly not mounted. No need to
* to perform any more checks.
*/
close(fd);
*mount_flags = 0;
return 0;
} else if (errno == EBUSY) {
busy = 1;
}
}
Closes#89 - GParted doesn't recognise EXT4 fs journal partition
A user reported that they were using an external journal with an ext4
file system, but that GParted didn't recognise it. (They had the jbd
on an Intel Optane drive and the ext4 file system on an SSD).
Create a jbd like this:
# mke2fs -O journal_dev -L test-jbd /dev/sdb1
# blkid /dev/sdb1
/dev/sdb1: LABEL="test-jbd" UUID="6e52858e-0479-432f-80a1-de42f9a4093e" TYPE="jbd"
Add recognition of jbd. Use Blue Shadow colour, the same as ext4,
because jbd is primarily used by ext3/4 [1][2]. jbd is also used by
ocfs2 [1][3] and lustre [4][5] clustered file systems, but they are very
unlikely to encountered by GParted users. Also xfs [6] and jfs [7] can
have external journals so if recognition of them is ever added they will
get the same colour as their respective file systems too.
[1] Journaling block device
https://en.wikipedia.org/wiki/Journaling_block_device
"JBD is filesystem-independent. ext3, ext4 and OCFS2 are known to
use JBD"
[2] https://ext4.wiki.kernel.org/index.php/Frequently_Asked_Questions#What_are_the_key_differences_between_jbd_and_jbd2.3F
[3] OCFS2: The Oracle Clustered File System, Version 2
https://www.kernel.org/doc/ols/2006/ols2006v1-pages-289-302.pdf
"Metadata journaling is done on a per node basis with JBD"
[4] Efficient Object Storage Journaling in a Distributed Parallel File
System
https://www.usenix.org/legacy/event/fast10/tech/full_papers/oral.pdf
[5] Lustre Software Release 2.x Operations Manual
https://doc.lustre.org/lustre_manual.pdf
6.4.2. Choosing Parameters for an External Journal
[6] mkfs.xfs(8) - construct an XFS filesystem
https://man7.org/linux/man-pages/man8/mkfs.xfs.8.html
"OPTIONS
...
logdev=device
This is used to specify that the log section should reside on
the device separate from the data section. The internal=1 and
logdev options are mutually exclusive.
"
[7] jfs_mkfs(8) - create a JFS formatted partition
https://manpages.debian.org/testing/jfsutils/jfs_mkfs.8.en.html
"OPTIONS
...
-j journal_device
Create the external JFS journal on journal_device, ...
"
Closes#89 - GParted doesn't recognise EXT4 fs journal partition
A bcache device provides accelerated access to a backing device in a one
to one relationship. Multiple bcache backing devices can be attached to
and accelerated by the same cache device. Extending the setup from the
previous commit, create an additional backing device and attach it to
the same cache.
# bcache make -B /dev/sdb2
# bcache attach /dev/sdc1 /dev/sdb2
# bcache show
Name Type State Bname AttachToDev
/dev/sdb2 1 (data) clean(running) bcache1 /dev/sdc1
/dev/sdb1 1 (data) clean(running) bcache0 /dev/sdc1
/dev/sdc1 3 (cache) active N/A N/A
List a couple of bcache specific sysfs files which identify registered
(active) bcache devices (components).
# ls -l /sys/block/sd?/sd??/bcache/{dev,set}
lrwxrwxrwx. 1 root root 0 Jan 7 10:08 /sys/block/sdb/sdb1/bcache/dev -> ../../../../../../../../../../virtual/block/bcache0
lrwxrwxrwx. 1 root root 0 Jan 7 11:53 /sys/block/sdb/sdb2/bcache/dev -> ../../../../../../../../../../virtual/block/bcache1
lrwxrwxrwx. 1 root root 0 Jan 7 11:53 /sys/block/sdc/sdc1/bcache/set -> ../../../../../../../../../../../fs/bcache/9945e165-0604-4f29-94bd-b155d01080ad
As was done with previous software block devices [1][2][3][4] show the
bcache (access) device as the mount point of a backing device
(component). Use the /sys/block/DEV[/PTN]/bcache/dev sysfs symlinks to
provide the bcache device names. Bcache cache devices (components)
don't get mount points because they aren't accessible.
[1] commit 8083f11d84
Display LVM2 VGNAME as the PV's mount point (#160787)
[2] commit f6c2f00df7
Populate member mount point with SWRaid array device (#756829)
[3] commit 538c866d09
Display array device as mount point of mdadm started ATARAID members
(#75)
[4] commit 538c866d09
Display array device as mount point of mdadm started ATARAID members
(#75)
Closes#183 - Basic support for bcache
Make (format as) bcache backing device (-B) and cache device (-C) and
implicitly attach the backing device to the cache to enable caching, all
in one.
# bcache make -B /dev/sdb1 -C /dev/sdc1
# bcache show
Name Type State Bname AttachToDev
/dev/sdb1 1 (data) clean(running) bcache0 /dev/sdc1
/dev/sdc1 3 (cache) active N/A N/A
After experimenting with 'bcache unregister', 'bcache register' and
stracing 'bcache show' the bcache kernel module creates the sysfs
directory /sys/block/DEV[/PTN]/bcache and it's contents only when the
bcache device is registered with the kernel (bcache component is
active). Use this to identify whether any bcache device (component)
should be displayed as active or not in GParted.
# ls -ld /sys/block/sd?/sd?1/bcache
drwxr-xr-x. 6 root root 0 Jan 7 10:08 /sys/block/sdb/sdb1/bcache
drwxr-xr-x. 2 root root 0 Jan 7 10:08 /sys/block/sdc/sdc1/bcache
Closes#183 - Basic support for bcache
Use blkid to detect bcache formatted devices. Requires blkid from
util-linux >= 2.24 for detection of bcache devices [1].
Use util-linux's FS images when testing GParted detection.
# wget http://git.kernel.org/cgit/utils/util-linux/util-linux.git/plain/tests/ts/blkid/images-fs/bcache-B.img.xz
# xzcat bcache-B.img.xz > /dev/sdb1
# wget http://git.kernel.org/cgit/utils/util-linux/util-linux.git/plain/tests/ts/blkid/images-fs/bcache-C.img.xz
# xzcat bcache-C.img.xz > /dev/sdc1
# blkid /dev/sdb1 /dev/sdc1
/dev/sdb1: UUID="8fb7f716-4c19-4517-bfbb-6f4a2becad60" TYPE="bcache" PARTUUID="f8f1485e-01"
/dev/sdc1: UUID="7a343627-ac87-4bf0-b76f-46067cbc9b8c" TYPE="bcache" PARTUUID="f46e8c86-01"
To tidy-up after testing GParted detection, stop the bcache device in
case it was automatically started and wipe the signatures. This is to
prevent udev rules from automatically starting the bcache device on
every subsequent reboot.
# echo 1 > /sys/block/sdb/sdb1/bcache/stop
# wipefs -a /dev/sdb1 /dev/sdc1
Closes#183 - Basic support for bcache
Since the only use of SWRaid_Info::get_uuid() assign the returned value
this doesn't actually save any copy construction. Do it for consistency
with the other get_*() methods in SWRaid_Info.
Closes!94 - Make more getter methods use return-by-constant-reference
Have to use a second constant reference variable array_path_2 in
GParted_Core::set_mountpoints() because by design C++ does not implement
rebinding of references [1].
[1] why doesn't C++ allow rebinding a reference?
https://stackoverflow.com/questions/27037744/why-doesnt-c-allow-rebinding-a-referenceCloses!94 - Make more getter methods use return-by-constant-reference
All uses of get_description() copy construct to a local variable, not
assign to a reference, so this doesn't save anything. It is just being
done to be consistent with making other getters return a constant
reference.
Closes!94 - Make more getter methods use return-by-constant-reference
A number of GParted methods named get_*() are returning properties and
are return-by-value. For objects this implies the returned value is
copy constructed and later destroyed when it goes out of scope. Change
them to use return-by-constant-reference to avoid unnecessary object
duplication. Just have to make sure the reference goes out of scope
before the referenced object is destroyed to avoid having a reference
(pointer) to freed memory. Assigning to a local variable instead of
of a local reference still duplicates the object so can be used when the
lifetime of the gotten object needs to be longer that the referenced
object.
Previously done for other getters here:
d948cbcb91
Make get_custom_text() and get_generic_text() return by reference
This change just makes Device::get_path() return a constant reference
and updates uses to match to avoid copy constructing the returned value.
Closes!94 - Make more getter methods use return-by-constant-reference
Creating a grep process to check if a particular mount is still mounted
is an unnecessary overhead. All that is needed is for the Mount_Info
module to refresh it's copy of /proc/mounts and query that.
To keep the code as simple as possible just reload the whole of the
Mount_Info module and query the mount cache to determine if the
particular block device is still mounted at this particular mount point.
This therefore re-reads /proc/mounts (necessary) and /proc/swaps and
/etc/fstab (unnecessary). This is still much less overhead than
creating a separate grep process.
Closes!89 - Fix unmount error when unmounting below a bind mount point
Implemented the second half of the solution described in the previous
commit. Resolve UUID= and LABEL= references when searching in the
Mount_Info cache so that mount points of encrypted file systems listed
in /etc/fstab can be found using the later added FS_Info details.
Closes#162 - It is no longer possible to mount a LUKS encrypted file
system
Show support for online labelling using a second tick mark in the
Features dialog. This matches how online grow and shrink are shown.
Closes#163 - Feature request: set label on a mounted btrfs
Btrfs supports labelling of the file system while it is mounted. This
was added into Linux kernel 3.10 [1] and btrfs-progs 3.12 [2]. As the
oldest supported distributions have the needed versions or newer,
unconditionally enable without any checking for availability.
Distro EOL Linux kernel btrfs-progs
Debian 9 2022-Jun 4.19 4.7.3
RHEL / CentOS 7 2024-Jun 3.10.0 4.9.1
Ubuntu 18.04 LTS 2023-Apr 4.15.0 4.15.1
Unmounted btrfs is labelled via the block device containing it, where as
a mounted btrfs is labelled via it's mount point.
# mkfs.btrfs -L initial_label /dev/sdb1
# blkid /dev/sdb1
/dev/sdb1: LABEL="initial_label" ...
# btrfs filesystem label /dev/sdb1 unmounted_label_2
# blkid /dev/sdb1
/dev/sdb1: LABEL="unmounted_label_2" ...
# mount /dev/sdb1 /mnt/1
# btrfs filesystem label /dev/sdb1 mounted_label_3
# blkid /dev/sdb1
/dev/sdb1: LABEL="mounted_label_3" ...
[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=a8bfd4abea3da0e28f215e2a2b8c2f1ca27ebe80
Btrfs: set/change the label of a mounted file system
[2] https://git.kernel.org/pub/scm/linux/kernel/git/kdave/btrfs-progs.git/commit/?id=619dc61cae1420da2dec48f689d49b9b346abc15
Btrfs-progs: Change the label of a mounted file system
Closes#163 - Feature request: set label on a mounted btrfs
So far when prompting for the LUKS passphrase the dialog always looks
like this:
+------------------------------------------------+
| LUKS Passphrase /dev/sdb1 |
+------------------------------------------------+
| Enter LUKS passphrase to open /dev/sdb1 |
| Passphrase: [ ] |
| |
| |
| [ Cancel ] [ Unlock ] |
+------------------------------------------------+
Specifically the first line of the dialog says the reason to provide the
passphrase is to open the encryption mapping. Now the passphrase may
also be requested when resizing the encryption mapping, as part of a
resize of check operation, show the appropriate reason in the password
dialog.
Closes#59 - Resize of LUKS2 encrypted file system fails with "Nothing
to read on input"
This is the equivalent to what was previously done when adding opening
of LUKS mappings. Namely to add a way to pass the LUKS passphrase to
'cryptsetup luksOpen' via standard input. Previously the functionality
was added to Utils::execute_command() [1]. Now it is also needed to
pass the LUKS passphrase to 'cryptsetup resize', which is executed as
part of applying resize and check operations to an encrypted file
system. So add this functionality to FileSystem::execute_command().
For now writing to stdin is only needed for the one variant of
FileSystem::execute_command() which doesn't have progress tracking
callbacks. Writing to stdin can easily be added to the other progress
tracking callback variants of execute_command() when needed.
[1] 8dff80edc6
Add ability for small writes to stdin of child processes (#795617)
Closes#59 - Resize of LUKS2 encrypted file system fails with "Nothing
to read on input"
When composing a resize operation on an open encryption mapping, use the
existing LUKS password dialog to prompt for the passphrase, if and only
if 'cryptsetup resize' will prompt and GParted doesn't already have a
password. 'cryptsetup resize' will prompt for a LUKS passphrase when
the passphrase was stored in the kernel keyring service,
key_loc == KEYLOC_KeyRing. See the previous commit "Capture LUKS
mapping master encryption key location (#59)" for more details.
As commented in the code GParted specifically doesn't support the case
where the LUKS passphrase is changed while GParted is running and it
knew the old passphrase. When resizing an open encryption mapping
GParted will just pass the old out of date passphrase it knows and the
resize will fail like this:
# cryptsetup status sdb2_crypt | egrep 'type|key location'
type: LUKS2
key location: keyring
# dmsetup table --target crypt
sdb2_crypt: 0 491520 crypt aes-xts-plain64 :64:logon:cryptsetup:3d040240-97ba-4559-af98-72c3be500498-d0 0 8:18 32768
# echo -n oldpassword | cryptsetup -v --size 491520 resize sdb2_crypt
No key available with this passphrase.
Command failed with code -2 (no permission or bad passphrase).
# echo $?
2
To work around this either close and restart GParted or close and reopen
the encryption mapping. The former because GParted doesn't save
passwords across a restart so will prompt and the latter because GParted
will use the wrong old passphrase to try to open the mapping and then
prompt for the correct passphrase until successfully opened.
Closes#59 - Resize of LUKS2 encrypted file system fails with "Nothing
to read on input"
ISSUE OVERVIEW
When GParted tries to resize an open LUKS encryption mapping and the
volume (master) key was stored in the kernel keyring service [1] it
fails like this:
Check and repair file system ([Encrypted] ext4) on /dev/...(ERROR)
+ calibrate /dev/sdd1 (SUCCESS)
+ check file system on /dev/mapper/sdd1_crypt for errors...(SUCCESS)
+ grow encryption volume to fill the partition (ERROR)
+ cryptsetup -v resize 'sdd1_crypt' (ERROR)
Command failed with code -1 (wrong or missing parameters).
Nothing to read on input.
This error occurs with cryptsetup >= 2.0, kernel >= 4.10 and LUKS2
format because the crypt Device-Mapper target no longer has the volume
key so cryptsetup resize prompts for a passphrase, but GParted doesn't
provide it.
THIS COMMIT
Additionally capture the location of the volume (master) key location
for active encryption mappings. Do this the using the same method that
cryptsetup uses [2][3]. Namely if the first character of the KEY is a
":" then the key *was* stored in the kernel keyring service, otherwise
it *is* store in the Device-Mapper crypt target as previously.
# echo -n badpassword | cryptsetup luksFormat --type luks1 /dev/sdb1 -
# echo -n badpassword | cryptsetup luksOpen /dev/sdb1 sdb1_crypt
# cryptsetup status sdb1_crypt | egrep 'type|key location'
type: LUKS1
key location: dm-crypt
# echo -n badpassword | cryptsetup luksFormat --type luks2 /dev/sdb2 -
# echo -n badpassword | cryptsetup luksOpen /dev/sdb2 sdb2_crypt
# cryptsetup status sdb2_crypt | egrep 'type|key location'
type: LUKS2
key location: keyring
# dmsetup table --target crypt
sdb1_crypt: 0 520192 crypt aes-xts-plain64 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 0 8:17 4096
sdb2_crypt: 0 491520 crypt aes-xts-plain64 :64:logon:cryptsetup:3d040240-97ba-4559-af98-72c3be500498-d0 0 8:18 32768
^
First character of the KEY field --------------'
[1] Integration with the kernel keyring service
https://gitlab.com/cryptsetup/cryptsetup/blob/v2.0.0/docs/Keyring.txt
"
Starting with cryptsetup 2.0 we load [Volume Key] VK in kernel
keyring by default for LUKSv2 devices ...
In summary, the key description visible in dm-crypt table line is a
reference to VK that usually no longer exists in kernel keyring
service if you used cryptsetup to for device activation.
"
[2] cryptsetup/v2.3.5/lib/libdevmapper.c:_dm_target_query_crypt()
https://gitlab.com/cryptsetup/cryptsetup/-/blob/v2.3.5/lib/libdevmapper.c#L2031
if (key_[0] == ':')
*act_flags |= CRYPT_ACTIVATE_KEYRING_KEY;
[3] cryptsetup/v2.3.5/src/cryptsetup.c:action_status()
https://gitlab.com/cryptsetup/cryptsetup/-/blob/v2.3.5/src/cryptsetup.c#L839
log_std(" key location: %s\n", (cad.flags & CRYPT_ACTIVATE_KEYRING_KEY) ? "keyring" : "dm-crypt");
Closes#59 - Resize of LUKS2 encrypted file system fails with "Nothing
to read on input"
get_disk() is the wrapper around libparted's ped_disk_new() which reads
a disk label from the specified device and if successful creates the in
memory PedDisk object to represent it. In the case that libparted
doesn't recognise a disk label or a file system, having get_disk() go
and destroy the passed in PedDevice object via parameter lp_device is
very unexpected behaviour hence describing it as a coding landmine.
BACKGROUND
1. Early on GParted only worked with devices with valid disk labels.
FileSystem.h:open_device_and_disk() required both ped_device_get()
and ped_disk_new() to succeed or neither to succeed.
2. Commit [1] added support for devices which didn't yet have a disk
label. open_device_and_disk() had default parameter strict=true
added. While scanning strict=false was passed which allowed
open_device_and_disk() to return success if only ped_device_get()
succeeded and ped_disk_new() failed when the disk was empty. All
other times open_device_and_disk() was called with default
strict=true, still requiring both or neither to succeed.
3. Commit [2] added support for whole disk file systems. The now named
get_device_and_disk() had it's functionality split between
get_device() and get_disk(). This result in the code landmine being
left behind: get_disk() destroying the passed device object if
default parameter strict=true and no disk label or file system was
detected.
ANALYSIS
1. Since support for whole disk file systems [2] all current calls to
get_device_and_disk() let the strict parameter default to true and
are only called on known partitions within disk labels when applying
a change to that partition. Therefore they don't care about the
behaviour of get_disk(), just that get_device_and_disk() maintains
that both ped_device_get() and ped_disk_new() succeed or neither
succeed.
2. Two direct calls to get_disk() where the strict parameter defaults to
true, from calibrate_partition() and erase_filesystem_signatures(),
only do so on known partitions within disk labels as part of applying
a change to that partition. Therefore ped_disk_new() will succeed
and so PedDevice isn't deleted when not wanted.
3. The two remaining direct calls to get_disk() where the strict
parameter is explicitly set to false, from set_device_from_disk() and
detect_filesystem_in_encryption_mapping(), are when scanning. As the
pass strict=false they don't allow the PedDevice deletion to occur if
no recognised disk label is found.
FIX
Remove the strict parameter from get_disk() and get_device_and_disk() as
it's no longer needed. Remove the code landmine by removing the side
affect of destroying the PedDevice object if a disk label isn't found.
Make sure get_device_and_disk() maintains the all or nothing behaviour.
Also don't pass lp_device by reference to a pointer to get_disk() so the
code can't change where lp_device points.
[1] 038c5c5d99
P (special thanks to mantiena-baltix for bringing this issue to my
[2] 51ac4d5648
Split get_device_and_disk() into two (#743181)
Closes#152 - GParted crashed when trying to probe an encrypted
partition containing content that libparted doesn't
recognise
As discussed in the previous commit "Don't crash probing libparted
unrecognised encrypted file system (#152)", detect_filesystem() accepted
a NULL lp_device pointer and dereferenced it leading to the crash.
Document the requirement for lp_device parameter to be non-NULL via an
assert and also correctly const the parameters.
This forces needing to const the lp_partition parameter to
get_partition_path() too. Also assert it's non-NULL requirement.
Closes#152 - GParted crashed when trying to probe an encrypted
partition containing content that libparted doesn't
recognise
To avoid making set_luks_partition() more complicated extract the file
system detection portion into a new function.
Closes 148 - Encrypted file systems are no longer recognised
Since changes for issue #131 "GParted hangs when non-named device is
hung" FS_Info cache is initialised, cleared and loaded via one call to
load_cache_for_paths(). It runs blkid for named or found disk devices
and associated found partitions from /proc/partitions, rather than
running blkid and letting it report for all block devices.
To avoid the possibility of using blkid on an encryption mapping on a
non-specified and possibly hung block device GParted can't just specify
all encryption mappings. Instead only encryption mappings which belong
to the above identified block devices should be probed. That requires
identifying LUKS encryption data in the block devices first so will
require subsequently loading additional data into the FS_Info cache and
running blkid again.
To accommodate this make the FS_Info cache incrementally loadable,
rather than doing everything in a single call to load_cache_for_paths().
Have a separate clear_cache() call which initialises and clears the
cache and make load_cache_for_paths() just run blkid and insert data for
the named paths.
Closes 148 - Encrypted file systems are no longer recognised
... in class Dialog_Partition_New and slightly refactor the code in
build_filesystems_combo() method which sets it.
Change the name from first_creatable_fs to default_fs to be more
immediately obvious what the variable represents. As default_fs is used
to index the items in the combo_filesystem derived ComboBox, make it's
type an int to match the type of the parameter passed to
Gtk::ComboBox::set_active() [1]. Initialise default_fs to -1 (no
selection) in the class constructor [2], which also allows removal of
local variable set_first just used to track whether first_creatable_fs
had been assigned yet or not.
[1] gtkmm: Gtk::ComboBox Class Reference, set_active()
https://developer.gnome.org/gtkmm/stable/classGtk_1_1ComboBox.html#a4f23cf08e85733d23f120935b235096d
[2] C++ FAQ / Should my constructors use "initialization lists" or
"assignment"?
https://isocpp.org/wiki/faq/ctors#init-lists
"Type" was rather a generic name. Use "combo_type_changed" which makes
it clear that the boolean parameter indicates whether a change to
combo_type or one of the other ComboBoxes triggered this callback.
Extract call to GLib's g_utf8_get_char_validated() and the associated
workaround to also read NUL characters into a separate function to make
PipeCapture::OnReadable() a little smaller and simpler, so easier to
understand.
Add max_len > 0 clause into get_utf8_char_validated() like this:
if (uc == UTF8_PARTIAL && max_len > 0)
so that the NUL character reading workaround is only applied when
max_len specifies the maximum number of bytes to read, rather than
when -1 specifies reading a NUL termination string. This makes
get_utf8_char_validated() a complete wrapper of
g_utf8_get_char_validated() [1], even though GParted always specifies
the maximum number of bytes to read.
No longer describe the inability to read NUL characters as a bug [2]
since the GLib author's said it wasn't [3].
[1] GLib Reference Manual, Unicode Manipulation Functions,
g_utf8_get_char_validated ()
https://developer.gnome.org/glib/stable/glib-Unicode-Manipulation.html#g-utf8-get-char-validated
[2] 8dbbb47ce2
Workaround g_utf8_get_char_validate() bug with embedded NUL bytes
(#777973)
[3] Bug 780095 - g_utf8_get_char_validated() stopping at nul byte even
for length specified buffers
https://bugzilla.gnome.org/show_bug.cgi?id=780095#18
"If g_utf8_get_char_validated() encounters a nul byte in the
middle of a string of given longer length, it returns -2,
indicating a partial gunichar. That is not the obvious
behaviour, but since g_utf8_get_char_validated() has been API
for a long time, the behaviour cannot be changed.
"
Closes#136 - 1.2.0: test suite is failing in test_PipeCapture
Also with exfatprogs 1.1.0 [1], tune.exfat and exfatlabel gained the
capability to report and set the exFAT Volume Serial Number [2][3][4].
This is what blkid and therefore GParted reports as the UUID.
Report serial number:
# tune.exfat -i /dev/sdb1
exfatprogs version : 1.1.0
volume serial : 0x772ffe5d
# echo $?
0
# blkid /dev/sdb1
/dev/sdb1: LABEL="test exfat" UUID="772F-FE5D" TYPE="exfat" PTTYPE="dos"
Set serial number:
# tune.exfat -I 0xf96ef190 /dev/sdb1
exfatprogs version : 1.1.0
New volume serial : 0xf96ef190
# echo $?
0
tune.exfat exists in earlier releases of exfatprogs so check it has the
capability by searching for "Set volume serial" in the help output
before enabling this capability.
# tune.exfat
exfatprogs version : 1.1.0
Usage: tune.exfat
-l | --print-label Print volume label
-L | --set-label=label Set volume label
-i | --print-serial Print volume serial
-L | --set-serial=value Set volume serial
-V | --version Show version
-v | --verbose Print debug
-h | --help Show help
(Note the cut and paste error reporting the set volume serial flag as
'-L' rather than actually '-S').
[1] exfatprogs-1.1.0 version released
http://github.com/exfaoprogs/exfatprogs/releases/tag/1.1.0
[2] [tools][feature request] Allow To Change Volume Serial Number ("ID")
#138https://github.com/exfatprogs/exfatprogs/issues/138
[3] exfatlabel:add get/set volume serial option
b4d9c9eeb5
[4] exFAT file system specification, 3.1.11 VolumeSerialNumber Field
https://docs.microsoft.com/en-us/windows/win32/fileio/exfat-specification#3111-volumeserialnumber-fieldCloses!67 - Add support for reading exFAT usage and updating the UUID
exfatprogs 1.1.0 released 2021-02-09 [1] has gained support for
reporting file system usage [2][3] so add that capability to GParted.
It works like this:
# dump.exfat /dev/sdb1 | egrep 'Volume Length\(sectors\):|Sector Size Bits:|Sector per Cluster bits:|Free Clusters:'
Volume Length(sectors): 524288
Sector Size Bits: 9
Sector per Cluster bits: 3
Free Clusters: 23585
Unfortunately dump.exfat returns a non-zero status on success so that
can't be used to check for failure:
# dump.exfat /dev/sdb1
exfatprogs version : 1.1.0
-------------- Dump Boot sector region --------------
Volume Length(sectors): 524288
...
# echo $?
192
dump.exfat only writes errors to stderr, so use this to identify failure:
# dump.exfat /dev/sdb1 1> /dev/null
# echo $?
192
# dump.exfat /dev/zero 1> /dev/null
invalid block device size(/dev/zero)
bogus sector size bits : 0
# echo $?
234
[1] exfatprogs-1.1.0 version released
http://github.com/exfaoprogs/exfatprogs/releases/tag/1.1.0
[2] [feature request] File system usage reporting
https://github.com/exfatprogs/exfatprogs/issues/139
[3] exfatprogs: add dump.exfat
7ce9b2336bCloses!67 - Add support for reading exFAT usage and updating the UUID
This method is now only called from one location in the code so put it's
two lines of code there.
Closes#131 - GParted hangs when non-named device is hung
Now we always want to run blkid naming all paths, ensure the FS_Info
cache is explicitly loaded first. Report an error if not done so and
remove the cache loading code from running blkid without naming all
paths. Fewer code paths to consider and reason about.
Closes#131 - GParted hangs when non-named device is hung
Again on Fedora 31 with a slightly different disk layout to the previous
commit. sdb is partitioned with 1 empty partition and sdc remains
completely empty:
# lsblk -o name,maj:min,rm,size,ro,type,fstype,label,mountpoint
NAME MAJ:MIN RM SIZE RO TYPE FSTYPE LABEL MOUNTPOINT
sda 8:0 0 20G 0 disk
|-sda1 8:1 0 1G 0 part ext4 /boot
\-sda2 8:2 0 19G 0 part LVM2_member
|-fedora-root 253:0 0 17G 0 lvm ext4 /
\-fedora-swap 253:1 0 2G 0 lvm swap [SWAP]
sdb 8:16 0 8G 0 disk
\-sdb1 8:17 0 1G 0 part
sdc 8:32 0 8G 0 disk
sr0 11:0 1 1024M 0 rom
# blkid -v
blkid from util-linux 2.34 (libblkid 2.34.0, 14-Jun-2019)
# blkid /dev/sda /dev/sda1 /dev/sda2 /dev/sdb /dev/sdb1 /dev/sdc
/dev/sda: PTUUID="5012fb1f" PTTYPE="dos"
/dev/sda1: UUID="3cd48816-7817-4636-9fec-5f1afe76c1b2" TYPE="ext4" PARTUUID="5012fb1f-01"
/dev/sda2: UUID="PH94ej-C8xU-bnMJ-UIh8-ZimI-4B7f-dHlZxh" TYPE="LVM2_member" PARTUUID="5012fb1f-02"
/dev/sdb: PTUUID="1d120b57" PTTYPE="dos"
/dev/sdb1: PARTUUID="1d120b57-01"
Stracing GParted shows these executions of blkid:
# strace -f -q -bexecve -eexecve ./gpartedbin 2>&1 1> /dev/null | egrep -v 'ENOENT|SIGCHLD'
...
[pid 160040] execve("/usr/sbin/blkid", ["blkid", "/dev/sda", "/dev/sda1", "/dev/sda2", "/dev/sdb", "/dev/sdb1", "/dev/sdc"], 0xa4e1b0 /* 32 vars */ <detached ...>
[pid 160041] execve("/usr/sbin/blkid", ["blkid", "/dev/sdc"], 0xa4e1b0 /* 32 vars */ <detached ...>
...
On Fedora 31 with blkid from util-linux 2.34 it reports information for
sdb (partitioned drive) and sdb1 (empty partition) with only no
information for sdc (empty whole disk drive). Hence no FS_Info cache
entry and re-execution of blkid just for sdc.
On older CentOS 7 with the same disk layout blkid reports this:
# blkid -v
blkid from util-linux 2.23.2 (libblkid 2.23.0, 25-Apr-2013)
# blkid /dev/sda /dev/sda1 /dev/sda2 /dev/sdb /dev/sdb1 /dev/sdc
/dev/sda: PTTYPE="dos"
/dev/sda1: UUID="e7d559e4-3e1d-4fbc-b034-3fdeb498f44d" TYPE="xfs"
/dev/sda2: UUID="B7ODFx-BfTE-hq7N-UlrF-f5ML-CPRe-klSy26" TYPE="LVM2_member"
/dev/sdb: PTTYPE="dos"
And stracing GParted shows these executions of blkid:
# strace -f -q -bexecve -eexecve ./gpartedbin 2>&1 1> /dev/null | egrep -v 'ENOENT|SIGCHLD'
...
[pid 1889] execve("/sbin/blkid", ["blkid", "/dev/sda", "/dev/sda1", "/dev/sda2", "/dev/sdb", "/dev/sdb1", "/dev/sdc"], 0x10b8b10 /* 26 vars */ <detached ...>
[pid 1890] execve("/sbin/blkid", ["blkid", "/dev/sdb1"], 0x10b8b10 /* 26 vars */ <detached ...>
[pid 1891] execve("/sbin/blkid", ["blkid", "/dev/sdc"], 0x10b8b10 /* 26 vars */ <detached ...>
...
This time on CentOS 7 with blkid from util-linux 2.23.2 it reports
information for only sdb (partitioned drive), but not sdb1 (empty
partition) or sdc (empty whole disk drive). Hence no FS_info cache
entries and re-execution of blkid for both sdb1 and sdc.
GParted needs blkid identification of file system images, LVM Logical
Volumes or any other partitions named on the command line which it
wouldn't normally scan [1]. Now every name of interest is passed to
blkid, additional executions of blkid won't get any extra information
and are redundant. Therefore remove this unnecessary code.
Note that these last 2 commits remove creation of "blank" cache entries
(just block special with blank fstype and other attributes) when blkid
reports no information for a particular path. Those entry were needed
to suppress unnecessary additional execution of blkid. However now that
blkid is only executed once (excluding querying the label) this is no
longer necessary. All the getter functions return suitable blank values
when no cache entry is found.
[1] e8f0504b13
Make sure that FS_Info cache is loaded for all named paths (#787181)
Closes#131 - GParted hangs when non-named device is hung
Mike Fleetwood