A user reported that it was not possible to deactivate an active LVM
Physical Volume. They were using GParted app from GParted Live 1.6.0.
This behaviour has been reproduced on GParted Live 1.6.0-3 (Debian SID
as of 2024-04-08) and Fedora Rawhide as of 2024-08-15, both recent
development branches of their respective distributions. Was not able to
replicate this on these latest releases: Debian 12, Fedora 40 and
Ubuntu 24.04 LTS.
GParted did deactivate the LVM Volume Group containing the Physical
Volume but device refresh probing triggered a udev rule which
re-activated the Volume Group. Summary:
1. GParted read the partition table by calling ped_disk_get().
2. Libparted opened the whole disk and every partition read-write to
flush the caches for coherency. On closing the file handles ...
3. The kernel generated partition remove and add uevents.
4. Udev triggered the rule to start the LVM VG.
Details obtained with the help of udevadm monitor, strace and
journalctl:
GParted | set_devices_thread()
GParted | set_device_from_disk()
GParted | get_device()
GParted | get_disk()
libparted| ped_disk_new()
libparted| openat(AT_FDCWD, "/dev/sdb", O_RDWR) = 7
libparted| ioctl(7, BLKFLSBUF) = 0
libparted| openat(AT_FDCWD, "/dev/sdb1", O_RDWR) = 8
libparted| ioctl(8, BLKFLSBUF) = 0
libparted| fsync(8) = 0
libparted| close(8) = 0
KERNEL | change /devices/pci0000:00/.../block/sdb/sdb1 (block)
UDEV | change /devices/pci0000:00/.../block/sdb/sdb1 (block)
libparted| fsync(7) = 0
libparted| close(7) = 0
KERNEL | remove /devices/pci0000:00/.../block/sdb/sdb1 (block)
KERNEL | change /devices/pci0000:00/.../block/sdb (block)
KERNEL | add /devices/pci0000:00/.../block/sdb/sdb1 (block)
UDEV | remove /devices/pci0000:00/.../block/sdb/sdb1 (block)
UDEV | change /devices/pci0000:00/.../block/sdb (block)
UDEV | add /devices/pci0000:00/.../block/sdb/sdb1 (block)
SYSLOG | lvm[62502]: PV /dev/sdb1 online, VG testvg is complete.
KERNEL | add /devices/virtual/bdi/253:0 (block)
KERNEL | add /devices/virtual/block/dm-0 (block)
KERNEL | change /devices/virtual/block/dm-0 (block)
UDEV | add /devices/virtual/bdi/253:0 (block)
UDEV | add /devices/virtual/block/dm-0 (block)
UDEV | change /devices/virtual/block/dm-0 (block)
SYSLOG | systemd[1]: Started lvm-activate-testvg.service - /usr/sbin/lvm vgchange -aay --autoactivation event testvg.
SYSLOG | lvm[62504]: 1 logical volume(s) in volume group "testvg" now active
SYSLOG | systemd[1]: lvm-activate-testvg.service: Deactivated successfully.
# grep 'lvm vgchange -a' /usr/lib/udev/rules.d/*lvm*
/usr/lib/udev/rules.d/69-dm-lvm.rules:... RUN+="... /usr/sbin/lvm vgchange -aay --autoactivation event $env{LVM_VG_NAME_COMPLETE}"
Evaluation using systemd's Locking Block Device Access [1]. Took a BSD
file lock on /dev/sdb while GParted was probing the drive. Used Python
from another terminal:
# python
>>> import fcntl
>>> f = open('/dev/sdb', 'wb')
>>> fcntl.flock(f, fcntl.LOCK_EX|fcntl.LOCK_NB)
Ran GParted. Released the lock by closing the open file when GParted
display had updated.
>>> f.close()
The lock temporarily stopped the Volume Group being activated so GParted
displayed it as inactive, but as soon as the lock was released the udev
rule fired and the Volume Group was activated. This is an even worse
situation as GParted displayed the Volume Group as inactive but it was
actually active. Therefore GParted can't use this method.
This type of issue has been encountered before with bcache devices [2]
and Linux Software RAID arrays [3] being automatically started by device
probing. Fix using the same method of temporarily adding in a blank
override rule which does nothing.
[1] systemd - Locking Block Device Access
https://systemd.io/BLOCK_DEVICE_LOCKING/
[2] 8640f91a4f
Prevent GParted probe starting stopped bcache (#183)
[3] a255abf343
Prevent GParted starting stopped Linux Software RAID arrays (#709640)
Closes#259 - Trying to deactivate LVM PV fails
During device probing GParted always explicitly calls ped_device_sync()
to flush the caches for coherency between the whole disk device and the
partition devices [1]. As the GParted_Core::flush_device() comment
explains, since v3.1.61-gfb99ba5 [2], libparted flushes the devices
every time a device is opened [3], so also explicitly doing this in
GParted is unnecessary. Therefore stop explicitly flushing the devices
in GParted, except when using libparted 3.1 and older which doesn't do
it itself.
The ped_device_open() and ped_device_close() needed for the
ped_device_sync() is also a trigger of device changes and udev rule
execution performing LVM Volume Group activation, because libparted
opens the device read-write [4]. This is another reason to remove it
when possible. However even when eliminated it does not solve the issue
of LVM VG activation because other triggers remain. Do want this change
first though so that the sequence of libparted calls isn't changed
immediately after documenting them and fixing the issue and so that
there is no doubt that this change doesn't fix the issue.
Removing this extra device flush saves a little bit of time, depending
on the speed of the drive and number of partitions. Sample savings on
my desktop:
Drive type and partitions Saving (seconds)
----------------------------- ----------------
fast SSD with 3 partitions 0.05
slow SSD with 12 partitions 0.27
HDD with 1 partition 0.05
VHDD in VM with 1 partition 0.14
VHDD in VM with 10 partitions 0.58
Also the settle_device() call in flush_device() needs to be kept to wait
for device changes and udev rule execution whether it is GParted
explicitly flushing the device or just libparted automatically doing it
for cache coherency in ped_device_get(). This is because:
1. Libparted <= 3.2 opens the whole disk device read-write for
ped_device_get() [5]. (This was changed with parted
v3.2.26-g44d5ae0 [6] to open the whole disk device read-only).
2. Libparted up to and including the latest 3.6 release still opens all
partition devices read-write for ped_device_get() [7].
3. A whole disk device FAT32 file system looks enough like a partition
table that both the Linux kernel and libparted think it is
partitioned:
# mkfs.fat -F32 /dev/sdb
mkfs.fat 4.2 (2021-01-31)
# grep sdb /proc/partitions
8 16 8388608 sdb
8 17 8388607 sdb1
# parted /dev/sdb print
Model: ATA VBOX HARDDISK (scsi)
Disk /dev/sdb: 8590MB
Sector size (logical/physical): 512B/512B
Partition Table: loop
Disk Flags:
Number Start End Size File system Flags
1 0s 16777215s 16777216s fat32
So the ped_device_get() call on a whole disk device FAT32 file system
still triggers device change and udev rule execution which needs to be
waited for, as this is exactly the case fixed previously by commit:
1382e0b828
Wait for udev change on /dev/DISK when querying whole device FS (!46)
[1] 3bea067596
Flush devices when scanning to prevent reading stale signatures (#723842)
[2] Revert "linux-commit: do not unnecessarily open partition device nodes"
http://git.savannah.gnu.org/cgit/parted.git/commit/?id=fb99ba5ebd0dc34204fc9f1014131d5d494805bc
[3] parted libparted/arch/linux.c:_device_open()
https://git.savannah.gnu.org/cgit/parted.git/tree/libparted/arch/linux.c?h=v3.6#n1752
1709 static int
1710 linux_open (PedDevice* dev)
1711 {
1712 return _device_open (dev, RW_MODE);
1713 }
1714
1715 static int
1716 _device_open (PedDevice* dev, int flags)
...
1752 _flush_cache (dev);
[4] parted libparted/device.c:ped_device_open() v3.6
https://git.savannah.gnu.org/cgit/parted.git/tree/libparted/device.c?h=v3.6#n226
parted libparted/arch/linux.c v3.6
https://git.savannah.gnu.org/cgit/parted.git/tree/libparted/arch/linux.c?h=v3.6
ped_device_open(...)
ped_architecture->dev_ops->open(...) = linux_open(...)
_device_open(..., RW_MODE)
open(..., O_RDWR)
[5] parted libparted/device.c:ped_device_get() v3.2
https://git.savannah.gnu.org/cgit/parted.git/tree/libparted/device.c?h=v3.2#n149
parted libparted/arch/linux.c v3.2
https://git.savannah.gnu.org/cgit/parted.git/tree/libparted/arch/linux.c?h=v3.2
ped_device_get(...)
ped_architecture->dev_ops->_new(...) = linux_new()
init_ide(...)
init_scsi(...)
init_generic(...)
ped_device_open(...)
ped_architecture->dev_ops->open(...) = linux_open(...)
open(..., O_RDWR)
[6] libparted: Use read only when probing devices on linux (#1245144)
http://git.savannah.gnu.org/cgit/parted.git/commit/?id=44d5ae0115c4ecfe3158748309e9912c5aede92d
[7] parted libparted/arch/linux.v v3.6
http://git.savannah.gnu.org/cgit/parted.git/tree/libparted/arch/linux.c?h=v3.6#n1660
1660 static void
1661 _flush_cache (PedDevice* dev)
...
1673 for (i = 1; i < lpn; i++) {
...
1680 if (!_partition_is_mounted_by_path(name)) {
1681 fd = open (name, WR_MODE, 0);
Closes#259 - Trying to deactivate LVM PV fails
When printing a number of bytes using Utils::format_size() it always
formatted the value with 2 decimal digits and an IEC multiplier. This
can be seen in the details of any operation which includes clearing old
file system signatures. Fragment of operation details:
Format /dev/sdb1 as cleared
+ calibrate /dev/sdb1
+ clear old file system signatures in /dev/sdb1
write 512.00 KiB of zeros at byte offset 0
write 4.00 KiB of zeros at byte offset 67108864
>> write 512.00 B of zeros at byte offset 132537184
write 4.00 KiB of zeros at byte offset 1072693248
write 512.00 KiB of zeros at byte offset 133593440
flush operating system cache of /dev/sdb1
It doesn't make sense to be reporting 100ths of a byte. So when values
are below 1 KiB report numbers of bytes without any decimal digits.
Create an uneven used space multi-device bcachefs file system like this,
where sdb1 is about 1/2 used and sdb2 is almost empty:
# bcachefs format /dev/sdb1
# mount /dev/sdb1 /mnt/1
# dd if=/dev/zero bs=1M count=512 of=/mnt/1/zeros.bin
# bcachefs device add /mnt/1 /dev/sdb2
GParted reports both devices as having the same usage:
Partition File System Mount Point Size Used Unused Flags
/dev/sdb1 bcachefs /mnt/1 1.00 GiB 526.00 MiB 498.00 MiB
/dev/sdb2 bcachefs /mnt/1 1.00 GiB 526.00 MiB 498.00 MiB
When in fact the used space is significantly different for each device:
# bcachefs fs usage /mnt/1 | egrep ' \(device |free:|capacity:'
(no label) (device 0): sdb1 rw
free: 522190848 3984
capacity: 1073741824 8192
(no label) (device 1): sdb2 rw
free: 1061945344 8102
capacity: 1073741824 8192
This is because bcachefs::set_used_sectors() is always searching for the
first "free:" and "capacity:" figures. Fix by reading the figures for
the correct device.
[Only the options being used by GParted are quoted here from the help
output. More options are available.]
$ bcachefs fsck --help
bcachefs fsck - filesystem check and repair
Usage: bcachefs fsck [OPTION]... <device>
Options:
-y Assume "yes" to all questions
-f Force checking even if filesystem is marked clean
-v Be verbose
Closes!123 - Add support for bcachefs, single device file systems only
When the size of the partition being cleared is an exact multiple of
MiBs, likely given that GParted aligns partitions to whole MiBs by
default, then the same range will be zeroed 4 times in a row. Example
operation details from clearing a partition look like this:
Format /dev/sdb1 as cleared
+ calibrate /dev/sdb1
path: /dev/sdb1 (partition)
start: 2048
end: 2099199
size: 2097152 (1.00 GiB)
+ clear old file system signatures in /dev/sdb1
write 512.00 KiB of zeros at byte offset 0
write 4.00 KiB of zeros at byte offset 67108864
write 512.00 B of zeros at byte offset 1072161280
>> write 4.00 KiB of zeros at byte offset 1072693248
>> write 4.00 KiB of zeros at byte offset 1072693248
>> write 4.00 KiB of zeros at byte offset 1072693248
>> write 4.00 KiB of zeros at byte offset 1072693248
write 512.00 KiB of zeros at byte offset 1073217536
+ set partition type on /dev/sdb1
This is because the bcachefs backup super block is located at -1 MiB
from the end of the device, rounded down by the bcachefs bucket size.
The bucket size can be any of 128 KiB, 256 KiB, 512 KiB or 1 MiB,
depending on the size of the bcachefs file system. So when the
partition size is an exact multiple of MiBs all 4 possible rounding
sizes result in the same offset.
Avoid writing the same range of zeros multiple times by skipping a range
if it is identical to the previous range.
Closes!123 - Add support for bcachefs, single device file systems only
GParted doesn't clear all bcachefs file system signatures.
# truncate -s $((1*1024*1024*1024)) /tmp/disk.img
# losetup --find --show /tmp/disk.img
/dev/loop0
# bcachefs format /tmp/disk.img
# wipefs /dev/loop0
DEVICE OFFSET TYPE UUID LABEL
loop0 0x1018 bcachefs 15304edb-6dc4-476c-989e-74eaea6660e8
loop0 0x3ff00018 bcachefs 15304edb-6dc4-476c-989e-74eaea6660e8
Use GParted to format to cleared /dev/loop0.
# wipefs /dev/loop0
DEVICE OFFSET TYPE UUID LABEL
loop0 0x3ff00018 bcachefs 15304edb-6dc4-476c-989e-74eaea6660e8
The backup super block is located at -1 MiB before the end of the
device, rounded down to the file system's bucket size [1], where the
bucket size is one of: 128 KiB, 256 KiB, 512 KiB, 1024 KiB [2].
location = device size - 1 MiB
= hex(1*1024*1024*1024 - 1*1024*1024)
= 0x3ff00000
Bcachefs magic is at byte offset 24 (0x18) into the super block [3].
The backup super block was not erased because GParted only writes zeros
from -512 KiB to the end of the device. It is necessary to clear old
signatures before formatting with a new file system to avoid confusion
from detection of multiple incompatible signatures [4][5]. Fix this by
writing zeros over all possible locations of the bcachefs backup super
block.
[1] bcachefs-tools v1.6.4: c_src/libbcachefs.c:bch2_format()
https://evilpiepirate.org/git/bcachefs-tools.git/tree/c_src/libbcachefs.c?h=v1.6.4#n313
[This code locates the backup super block at device size - 1 MiB
rounded down to the bucket size.]
321 struct bch_sb_layout *l = &sb.sb->layout;
322 u64 backup_sb = size_sectors - (1 << l->sb_max_size_bits);
323
324 backup_sb = rounddown(backup_sb, i->bucket_size >> 9);
[2] bcachefs-tools v1.6.4: c_src/libbcachefs.c:bch2_pick_bucket_size()
https://evilpiepirate.org/git/bcachefs-tools.git/tree/c_src/libbcachefs.c?h=v1.6.4#n85
85 /* Want a bucket size of at least 128k, if possible: */
86 bucket_size = max(bucket_size, 128ULL << 10);
...
94 /* max bucket size 1 mb */
95 bucket_size = min(bucket_size * scale, 1ULL << 20);
[3] bcachefs-tools v1.6.4: libcachefs/bcachefs_format.h:struct bch_sb
https://evilpiepirate.org/git/bcachefs-tools.git/tree/libbcachefs/bcachefs_format.h?h=v1.6.4#n907
[4] 743968ef68
Add clearing of SWRaid metadata 0.90 and 1.0 super blocks (#756829)
[5] 3c75f3f5b1
Use wipefs to clear old signatures before creating new file systems (#688882)
Closes!123 - Add support for bcachefs, single device file systems only
Add reading of the bcachefs file system label, not the per device label,
and the external UUID. These match what blkid reports.
Example without a label:
# bcachefs format /dev/sdb1
# bcachefs show-super /dev/sdb1 | egrep -i 'Label:|UUID:|Device:'
External UUID: 3316bc9a-d129-42b6-a80e-9649874bca73
Internal UUID: 656eebe5-10a9-4f12-94c8-aab2fdc54732
Label:
Device: 0
Label: (none)
UUID: cd436a8d-82eb-4993-a317-b39ea0d6bd2e
# blkid /dev/sdb1
/dev/sdb1: UUID="3316bc9a-d129-42b6-a80e-9649874bca73" BLOCK_SIZE="512" UUID_SUB="cd436a8d-82eb-4993-a317-b39ea0d6bd2e" TYPE="bcachefs" PARTUUID="7962e584-34c9-4088-8a00-a651af517089"
Example with a label:
# bcachefs format --force -L 'test label' /dev/sdb1
# bcachefs show-super /dev/sdb1 | egrep -i 'Label:|UUID:|Device:'
External UUID: 3d7bdabe-2616-4545-affc-1aba0f8fb4a7
Internal UUID: 9cc95d3e-7991-4f78-9dd0-850cb9749e34
Label: test label
Device: 0
Label: (none)
UUID: 784d1bd0-5769-4fbb-ad32-07894d381bba
# blkid /dev/sdb1
/dev/sdb1: UUID="3d7bdabe-2616-4545-affc-1aba0f8fb4a7" LABEL="test label" BLOCK_SIZE="512" UUID_SUB="784d1bd0-5769-4fbb-ad32-07894d381bba" TYPE="bcachefs" PARTUUID="7962e584-34c9-4088-8a00-a651af517089"
Closes!123 - Add support for bcachefs, single device file systems only
For bcachefs file systems 19 MiB and smaller, the available space is
reported as a very large value. The calculation went negative in a
64-bit signed value but then was interpreted as an unsiged value.
Writing any significant amount of data to the file system hangs.
# truncate -s $((19*1024*1024)) /tmp/test.img
# losetup --find --show /tmp/test.img
/dev/loop0
# bcachefs format /dev/loop0
# mkdir /mnt/0
# mount /dev/loop0 /mnt/0
# strace -e statfs df -k /mnt/0
statfs("/mnt/0", {f_type=0xca451a4e, f_bsize=4096, f_blocks=2305843009213693856,
f_bfree=2305843009213693600, f_bavail=35474507834056483, f_files=18446744073709529090,
f_ffree=18446744073709529088, f_fsid={val=[0xddb6645d, 0x8560584]}, f_namelen=512,
f_frsize=4096, f_flags=ST_VALID|ST_RELATIME}) = 0
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/loop0 9223372036854775424 1024 141898031336225932 1% /mnt/0
For a 20 MiB bcachefs the available space is 0 so the file system
overhead is 100%.
# umount /mnt/0
# truncate -s $((20*1024*1024)) /tmp/disk.img
# losetup --set-capacity /dev/loop0
# bcachefs format --force /dev/loop0
# mount /dev/loop0 /mnt/0
# strace -e statfs df -k /mnt/0
statfs("/mnt/0", {f_type=0xca451a4e, f_bsize=512, f_blocks=1280, f_bfree=0, f_bavail=0,
f_files=2, f_ffree=0, f_fsid={val=[0x6b3e4926, 0x33f99a32]}, f_namelen=512, f_frsize=512,
f_flags=ST_VALID|ST_RELATIME}) = 0
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/loop0 640 640 0 100% /mnt/0
For a 32 MiB bcachefs the file system overhead is a more reasonable 9%
so use this as the minimum bcachefs file system size.
...
# truncate -s $((32*1024*1024)) /tmp/disk.img
...
# strace -e statfs df -k /mnt/0
statfs("/mnt/0", {f_type=0xca451a4e, f_bsize=512, f_blocks=24832, f_bfree=22784,
f_bavail=22433, f_files=182274, f_ffree=182272, f_fsid={val=[0xfdddedd3, 0xe90be3cb]},
f_namelen=512, f_frsize=512, f_flags=ST_VALID|ST_RELATIME}) = 0
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/loop0 12416 1024 11217 9% /mnt/0
Closes!123 - Add support for bcachefs, single device file systems only
Currently bcachefs-tools only provides a method to report the file
system usage while it is mounted. We won't make GParted mount a
bcachefs to read it's usage as we want to keep GParted's scanning as a
read-only activity. Therefore GParted can't report the usage of an
unmounted bcachefs.
# bcachefs format /dev/sdb1
# bcachefs fs usage /dev/sdb1
error opening /dev/sdb1: not a bcachefs filesystem
# echo $?
1
# bcachefs fs usage --help
bcachefs fs usage - display detailed filesystem usage
Usage: bcachefs fs usage [OPTION]... <mountpoint>
...
# mount /dev/sdb1 /mnt/1
# bcachefs fs usage /mnt/1
Filesystem: a61a8302-9a79-4c24-a9e6-486e7fcc78f5
Size: 987842560
Used: 12713984
Online reserved: 0
Data type Required/total Durability Devices
btree: 1/1 1 [sdb1] 1048576
(no label) (device 0): sdb1 rw
data buckets fragmented
free: 1061027840 8095
sb: 3149824 25 126976
journal: 8388608 64
btree: 1048576 8
user: 0 0
cached: 0 0
parity: 0 0
stripe: 0 0
need_gc_gens: 0 0
need_discard: 0 0
capacity: 1073741824 8192
# echo $?
0
Closes!123 - Add support for bcachefs, single device file systems only
Set the minimum file system size to 16 MiB as creating a bcachefs that
size succeeds:
$ truncate -s $((16*1024*1024)) /tmp/disk.img
$ bcachefs format /tmp/disk.img
...
initializing new filesystem
going read-write
initializing freespace
$ echo $?
0
Where as creating a smaller file system fails for most sizes below that:
$ rm /tmp/disk.img
$ truncate -s $((15*1024*1024)) /tmp/disk.img
$ bcachefs format /tmp/disk.img
...
mounting version 1.6: btree_subvolume_children
initializing new filesystem
going read-write
bch2_trans_mark_dev_sb(): error ENOSPC_disk_reservation
bch2_fs_initialize(): error marking superblocks ENOSPC_disk_reservation
bch2_fs_initialize(): error ENOSPC_disk_reservation
bch2_fs_start(): error starting filesystem ENOSPC_disk_reservation
error opening /tmp/disk.img: ENOSPC_disk_reservation
$ echo $?
1
Closes!123 - Add support for bcachefs, single device file systems only
Bcachefs [1] has many of the same capabilities as Btrfs [2] and ZFS [3]:
COW (Copy-on-Write), multi-device, multi-volume, snapshotting and many
more. Therefore when adding bcachefs use the same range of Orange
colours already used by Btrfs and ZFS [4]. As bcachefs is a native
Linux file system and ZFS is not, move ZFS to a darker shade of Orange
to allow bcachefs to be added in the middle:
Btrfs - Orange Medium (#E58749)
bcachefs - Orange Dark (#C26825)
ZFS - Orange Shadow (#984F18)
[1] bcachefs
https://bcachefs.org/
[2] Welcome to BTRFS documentation! > Introduction
https://btrfs.readthedocs.io/en/latest/Introduction.html
[3] ZFS
https://en.wikipedia.org/wiki/ZFS
[4] 8a4f9ad205
Adjust shades of aquamarine, cyan and orange
Closes!123 - Add support for bcachefs, single device file systems only
Utils::trim_trailing_new_line() name is self-explanatory and doesn't
require a comment. The comment became superfluous with commit:
1fbc8988ff
Extract repeated code into trim_trailing_new_line() (!105)
Restricting the udev query from all types of device data to only
properties slightly simplifies the output from udevadm info which has to
be parsed. From this:
$ udevadm info --query=all --name=/dev/dm-0 | grep DM_NAME
E: DM_NAME=isw_cbaccqjiij_myarray
to:
$ udevadm info --query=property --name=/dev/dm-0 | grep DM_NAME
DM_NAME=isw_cbaccqjiij_myarray
Use if false return early pattern for the udevadm command found check.
Don't bother to check the exit status from running the udevadm command
or whether the output string contains anything before trying to parse it
for the serial number using Utils::regexp_label(). If the output string
is empty because either the udevadm command failed or the device has no
serial number then Utils::regexp_label() will assign the empty string to
serial_number; exactly what is wanted for these failure cases. This
makes the code simpler and easier to understand and still produces the
desired results in the failure cases.
The device model displayed in GParted has a leading space character when
the kernel device driver vendor attribute is blank.
# lsblk -o KNAME,VENDOR,MODEL,SERIAL,TYPE,TRAN -d /dev/sde /dev/sdh
KNAME VENDOR MODEL SERIAL TYPE TRAN
sde SanDisk U3 Cruzer Micro 00001853E473AABA disk usb
sdh Patriot Memory 07082AA3E0944E31 disk usb
^^^^^^^^
# cd /sys/block/sde/device
# cat vendor
SanDisk
# cat model
U3 Cruzer Micro
# cd /sys/block/sdh/device
# cat vendor
>>
# cat model
Patriot Memory
# ls -l /dev/sde /dev/sdh
brw-rw----. 1 root disk 8, 64 Aug 9 18:26 /dev/sde
brw-rw----. 1 root disk 8, 112 Aug 9 18:26 /dev/sdh
Libparted treats USB keys as SCSI devices because they use the SCSI
major device number. This means the libparted device model string is
constructed from the kernel device driver vendor and model strings read
from /sys, stripped and separated by a space character [1]. Hence the
leading space in the libparted model string when vendor is blank.
# parted /dev/sde print | head -1
Model: SanDisk U3 Cruzer Micro (scsi)
# parted /dev/sdh print | head -1
Model: Patriot Memory (scsi)
^
Trim space characters from the libparted model string to remove the
leading space displayed in GParted.
[1] parted/libparted/arch/linux.c:init_scsi()
http://git.savannah.gnu.org/cgit/parted.git/tree/libparted/arch/linux.c?h=v3.6#n1189
if (scsi_get_product_info (dev, &vendor, &product)) {
sprintf (dev->model, "%.8s %.16s", vendor, product);
A user reported that GParted was displaying binary data for the serial
number of their USB key. This was because hdparm -I was reporting
binary data for the serial number. It looked like this (except the
question marks were binary non-printable bytes).
$ LANG=C sudo hdparm -I /dev/sdh
/dev/sdh:
ATA device, with non-removable media
Model Number: ?|??t??L??|??
Serial Number: ?@>??8
u
Firmware Revision: u???
Standards:
Likely used: 1
...
Back in 2015 when reporting of drive serial numbers was being added
[1][2], using hdparm -I was the best option because it reported the
serial number of all hard drives including those in virtual machines,
worked the same everywhere and was available by default in most
distributions, but didn't work for USB keys. Where as lsblk -o SERIAL
was a new option not available yet on some distributions and didn't
report the serial number of drives in virtual machines.
Now hdparm -I capabilities are still the same; only working for hard
drives, including SSDs, but not USB keys. Except for the above case
hdparm has always found to not report serial numbers for USB keys, like
this:
# hdparm -I /dev/sde
/dev/sde:
SG_IO: bad/missing sense data, sb[]: 70 00 05 00 00 00 00 15 00 00 00 00 20 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 70 00 00 00
ATA device, with non-removable media
Standards:
Likely used: 1
....
Now the lsblk -o SERIAL option is available on all distributions and
reports the serial numbers of all drives including for virtual machine
disks and USB keys.
From an older but still supported Ubuntu 20.04 LTS virtual machine:
# lsblk -o KNAME,VENDOR,MODEL,SERIAL,TYPE,TRAN -d
KNAME VENDOR MODEL SERIAL TYPE TRAN
sda ATA VBOX HARDDISK VB13d4e080-b35e62d3 disk sata
sdb ATA VBOX HARDDISK VB221202cf-092e5857 disk sata
sdc ATA VBOX HARDDISK VB04fefadd-5a185f96 disk sata
sr0 VBOX CD-ROM VB2-01700376 rom ata
From a physical machine running older but still supported Rocky Linux 8
with 2 SSDs, 2 HDDs, and 6 USB keys:
# lsblk -o KNAME,VENDOR,MODEL,SERIAL,TYPE,TRAN -d
KNAME VENDOR MODEL SERIAL TYPE TRAN
sda ATA Samsung SSD 860 S3Z9NY0M620872V disk sata
sdb ATA SAMSUNG SSD UM41 DCF4300940SE940B4507 disk sata
sdc ATA WDC WD10JFCX-68N WD-WXE1EB6C3MHY disk sata
sdd ATA MM0500EBKAE 9XF132MW disk sata
sde SanDisk U3 Cruzer Micro 00001853E473AABA disk usb
sdf Kingston DataTraveler 3.0 408D5C1658F7E31079051DB9 disk usb
sdg Kingston DataTraveler 2.0 1C6F654FF40FBE50D9341059 disk usb
sdh Patriot Memory 07082AA3E0944E31 disk usb
sdi Corsair Slider 3.0 12240400400016361397 disk usb
sdj SMI USB DISK SMI_USB_DISK-0:0 disk usb
sr0 ASUS DRW-24B3LT B5D0CL213444 rom sata
sr1 SanDisk U3 Cruzer Micro 00001853E473AABA rom usb
lsblk gets it's information from udev. lsblk reports the SERIAL column
from the udev device property ID_SERIAL_SHORT, except when that is blank
it uses the ID_SERIAL property instead [3]. Note that ID_SERIAL is
composed from vendor and model information [4].
USB key with a serial number:
# udevadm info --query=property --name=/dev/sde | grep SERIAL
ID_SERIAL=SanDisk_U3_Cruzer_Micro_00001853E473AABA-0:0
ID_SERIAL_SHORT=00001853E473AABA
# lsblk -o KNAME,VENDOR,MODEL,SERIAL,TYPE,TRAN -d /dev/sde
KNAME VENDOR MODEL SERIAL TYPE TRAN
sde SanDisk U3 Cruzer Micro 00001853E473AABA disk usb
USB key without a serial number:
# udevadm info --query=property --name=/dev/sdj | grep SERIAL
ID_SERIAL=SMI_USB_DISK-0:0
# lsblk -o KNAME,VENDOR,MODEL,SERIAL,TYPE,TRAN -d /dev/sdj
KNAME VENDOR MODEL SERIAL TYPE TRAN
sdj SMI USB DISK SMI_USB_DISK-0:0 disk usb
To only get the device serial number, or blank when not available, query
the udev ID_SERIAL_SHORT property directly using udevadm info rather
than using lsblk -o SERIAL. Previously GParted displayed "none" if
hdparm -I successfully found nothing, including for USB keys, and left
the serial number field blank on failure. Now the field will be blank
in both cases, no serial number and failure to query it.
[1] 4b72ecd44e
Display device serial numbers (#751251)
[2] Bug 751251 - Show serial number in device information
https://bugzilla.gnome.org/show_bug.cgi?id=751251
[3] misc-utils/lsblk-properties.c:get_properties_by_udev()
https://git.kernel.org/pub/scm/utils/util-linux/util-linux.git/tree/misc-utils/lsblk-properties.c?h=v2.40.2#n121
[4] what is the difference between ID_SERIAL and ID_SERIAL_SHORT udev
attributes for a USB descriptor
https://stackoverflow.com/questions/45940014/what-is-the-difference-between-id-serial-and-id-serial-short-udev-attributes-for
"The ID_SERIAL_SHORT value comes from the iSerial string (if
present) in the USB device descriptor. The ID_SERIAL value is
constructed in software and made up from various strings (vendor
or manufacturer, model or product, and serial number if present)
separated with _.
"
Closes#263 - Serial number for my USB key showing binary data
To make it clear they are all member variables. Add missing
m_fs_block_size POD (Plain Old Data) member to the constructor
initialisation list so that it's never uninitialised. And stop
unnecessarily assigning constant false to m_force_auto_64bit in
get_filesystem_support() because it is already initialised to false by
the constructor.
To make it clear it is a member variable and continue the theme that
variables of type FSType have fstype in their name [1][2].
[1] 58fb230fb0
Also rename FS.filesystem member to fstype (!52)
[2] b0f92be638
Rename Partition.filesystem member to fstype (!52)
To make it clear m_old_mkudffs is a member variable.
To make it clear they are xfs member variables. Also add a default
constructor with an initialisation list so POD (Plain Old Data) member
variable m_src_used is never left uninitialised, as it was previously
only assigned in xfs::copy(). m_dest_mount_point is a Glib::ustring
object which is default constructed to a zero length string so doesn't
need adding to the xfs default constructor initialisation list.
* C++ FAQ / Should my constructors use "initialization lists" or
"assignment"?
https://isocpp.org/wiki/faq/ctors#init-lists
"Initialization lists. In fact, constructors should initialize as
a rule all member objects in the initialization list."
To make it clear m_old_mkudffs is a member variable. Also stop
unnecessarily assigning constant false to m_old_mkudffs in
get_filesystem_support() because it is already initialised to false by
the default constructor.
Use smart tab alignment; indent with tabs and align with spaces. Also
remove entries for lvm2 pv and minix which don't support labels, adding
a comment as to why.
Simplify code in Win_GParted::remove_non_empty_lvm2_pv_dialog() by
replacing open coded concatenation of a vector of strings using a new
line separator with a call to Glib::build_path().
Most of the code doesn't assign a zero length string when constructing a
string object. However there were a few places where it did. This is
unnecessary because Glib::ustring [1] and std::string [2] objects are
constructed as the the empty string by default.
[1] Glib::ustring::ustring()
https://developer-old.gnome.org/glibmm/stable/classGlib_1_1ustring.html#a71802782f4c2c408ef7ac69c6564b370
"Glib::ustring::ustring()
Default constructor, which creates an empty string.
"
[2] std::string::string
https://cplusplus.com/reference/string/string/string/
"(1) empty string constructor (default constructor)
Constructs an empty string, with a length of zero characters.
"
Using GParted to display a mounted exFAT file system results in this
partition warning:
exfatprogs version : 1.2.4
open failed : /dev/sdb1, Device or resource busy
This is because tune.exfat fails when attempting to query a mounted file
system.
When unmounted tune.exfat works:
# mkfs.exfat -L test-label /dev/sdb1
# tune.exfat -l /dev/sdb1
exfatprogs version : 1.2.4
label: test-label
# tune.exfat -i /dev/sdb1
exfatprogs version : 1.2.4
volume serial : 0xefaf7e93
But when mounted tune.exfat fails:
# mount /dev/sdb1 /mnt/1
# tune.exfat -l /dev/sdb1
exfatprogs version : 1.2.4
open failed : /dev/sdb1, Device or resource busy
# echo $?
255
# tune.exfat -i /dev/sdb1
exfatprogs version : 1.2.4
open failed : /dev/sdb1, Device or resource busy
Where as blkid succeeds:
# blkid /dev/sdb1
/dev/sdb1: LABEL="test-label" UUID="EFAF-7E93" BLOCK_SIZE="512" TYPE="exfat" PARTUUID="11a7a9eb-680b-4c90-982a-73f671e67e4f"
This failure case must have been missed when exFAT support was first
added [1][2] because re-testing it now back on Fedora 33 does show the
error. Anyway fix this by just not attempting to read the file system
label or serial number in the file system specific methods while its
mounted. GParted will fall back to reading the label from the blkid
populated FS_Info cache. See
GParted_Core::set_partition_label_and_uuid() for details. The serial
number is already read from the cache first, falling back to the file
system specific method second.
[1] bd386f445d
Add exFAT support (!30)
[2] Add exFAT support
https://gitlab.gnome.org/GNOME/gparted/-/merge_requests/30
Exfatprogs 1.2.2 and earlier are included in a number of the latest
distributions. Therefore support reading exFAT usage from the earlier
dump.exfat style output.
Distribution exfatprogs
Alpine 3.20 1.2.2
Debian 12 1.2.0
RHEL / Rocky Linux / AlmaLinux 9 1.2.2
Ubuntu 24.04 LTS 1.2.2
On Debian 12, example dump.exfat output:
# dump.exfat /dev/sdb1 | egrep 'exfatprogs version|Volume Length\(sectors\):|Sector|Free Clusters:'
exfatprogs version : 1.2.0
Volume Length(sectors): 524288
Sector Size Bits: 9
Sector per Cluster bits: 3
Free Clusters: 65019
Make this a separate commit so that it can be easily reverted in future
when this is no longer required. It also results in simpler code as the
old shift bits style figures are converted into new direct multiplier
style figures. Therefore the final if statement checking all the
figures are set, computing and assigning partition usage remains simple
and unchanged.
Closes#261 - Unable to read the contents of exfat file system with
exfatprogs >= 1.2.3
exFAT file system usage is no longer readable when exfatprogs 1.2.3,
released 2024-05-23 [1], and later is installed. This is because the
output from dump.exfat changed as a result of commit [2].
On Fedora 39 with exfatprogs 1.2.3 installed and exfatprogs 1.2.2
compiled locally from git; old dump.exfat output:
# ~fedora/programming/c/exfatprogs/dump/dump.exfat /dev/sdb1 | egrep 'exfatprogs version|Volume Length\(sectors\):|Sector|Free Clusters:'
exfatprogs version : 1.2.2
Volume Length(sectors): 524288
>> Sector Size Bits: 9
>> Sector per Cluster bits: 3
Free Clusters: 65024
New dump.exfat output:
# dump.exfat /dev/sdb1 | egrep 'exfatprogs version|Volume Length\(sectors\):|Sector|Free Clusters:'
exfatprogs version : 1.2.3
Volume Length(sectors): 524288
>> Bytes per Sector: 512
>> Sectors per Cluster: 8
Free Clusters: 65024
Change the code to read exFAT usage only from dump.exfat 1.2.3 style
output. Don't parse older style output for now as that makes the code
simpler.
[1] exfatprogs-1.2.3 version released
https://github.com/exfatprogs/exfatprogs/releases/tag/1.2.3
[2] dump: Sector and Cluster units
41863cb0f5Closes#261 - Unable to read the contents of exfat file system with
exfatprogs >= 1.2.3
As CentOS has been discontinued remove references to it from the README.
Also update package manager command from yum to dnf now RHEL 8 is the
earliest supported release.
Now that the official Ubuntu latest docker image has updated to Ubuntu
24.04 LTS [1] the build CI job fails like this:
$ make -j $nproc
...
/usr/bin/msgfmt --desktop --template gparted.desktop.in -d ./po -o gparted.desktop
/usr/bin/msgfmt --xml --template org.gnome.gparted.policy.in -d ./po -o org.gnome.gparted.policy
/usr/bin/msgfmt: cannot locate ITS rules for org.gnome.gparted.policy.in
make[2]: *** [Makefile:1060: org.gnome.gparted.policy] Error 1
The needed fix is already documented in the README file [2]. Install
the required package into the Ubuntu CI docker images to fix this.
[1] Ubuntu Docker official image
https://hub.docker.com/_/ubuntu/
"Supported tags and respective Dockerfile links
...
24.04, ..., latest, ...
"
[2] 16c2533af0
Document future Debian/Ubuntu build time dependency in README (!121)
Closes!124 - Update CI jobs for Ubuntu 24.04 LTS and use Rocky Linux 8
Network Block Devices are not displayed in GParted as partitionable
devices. They do appear in /proc/partitions, are reported by
fdisk -l [1] and by ped_device_probe_all() from libparted. Therefore
include them.
Create NBD device for testing:
# truncate -s 1G /tmp/disk-1G.img
# nbd-server -C /dev/null 9000 /tmp/disk-1G.img
# nbd-client localhost 9000 /dev/nbd0
After creating a couple of partitions for testing, the contents of
/proc/partitions looks like this:
# egrep 'name|nbd' /proc/partitions
major minor #blocks name
43 0 1048576 nbd0
43 1 262144 nbd0p1
43 2 785408 nbd0p2
Listing all disks using fdisk:
# fdisk -l
...
Disk /dev/nbd0: 1 GiB, 1073741824 bytes, 2097152 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x081b1cd1
Device Boot Start End Sectors Size Id Type
/dev/nbd0p1 2048 526335 524288 256M 83 Linux
/dev/nbd0p2 526336 2097151 1570816 767M 83 Linux
Temporarily apply this patch to GParted so that it ignores the devices it
currently selects from /proc/partitions to use what get_device_probe_all()
reports. GParted shows NBDs.
$ git diff --unified=1
diff --git a/src/GParted_Core.cc b/src/GParted_Core.cc
index 1629f94f..abea7a0b 100644
--- a/src/GParted_Core.cc
+++ b/src/GParted_Core.cc
@@ -172,3 +172,3 @@ void GParted_Core::set_devices_thread( std::vector<Device> * pdevices )
//try to find all available devices if devices exist in /proc/partitions
- std::vector<Glib::ustring> temp_devices = Proc_Partitions_Info::get_device_paths();
+ std::vector<Glib::ustring> temp_devices;
if ( ! temp_devices .empty() )
Tidy-up NBD device:
# nbd-client -d /dev/ndb0
# killall nbd-server
# rm /tmp/disk-1G.img
[1] man fdisk
"-l, --list
List the partition tables for the specified devices and then
exit. If no devices are given, the devices mentioned in
/proc/partitions (if this file exists) are used.
"
Closes#247 - GParted does not list NBD (Network Block Device) devices
in the GUI
Mike Fleetwood