This patch changes the reading of the btrfs multi-device membership to
resolve issue 1/2 by ignoring the exit status from the 'btrfs filesystem
show' command and relying on parsing the required information to
determine success or failure.
Bug #733601 - Btrfs: Warnings and missing label with btrfs-progs 3.12
and 3.14
Patch 3/4 - btrfs::read_label()
This patch changes the btrfs file system usage reading code to resolve
issue 1/2 by ignoring the exit status from the 'btrfs filesystem show'
command and relying on parsing the required information to determine
success or failure.
Bug #733601 - Btrfs: Warnings and missing label with btrfs-progs 3.12
and 3.14
Issue 2/2 - GParted doesn't show label for mounted btrfs file systems
'btrfs filesystem show /dev/PTN' command is used to query details of a
btrfs file system including reading the file system label. When the
file system is mounted the label is no longer enclosed in single quotes,
but only when using btrfs-progs v3.12. This causes GParted to think the
label is blank when the file system is mounted and therefore no longer
display it.
File system label not enclosed in single quotes when mounted:
# fgrep sdb1 /proc/mounts
/dev/sdb1 /mnt/1 btrfs rw,relatime,space_cache 0 0
# btrfs filesystem show /dev/sdb1
Label: test1-btrfs uuid: 1f78fa38-2f85-41d3-9be6-ae0356ae9469
Total devices 1 FS bytes used 192.00KiB
devid 1 size 2.00GiB used 240.75MiB path /dev/sdb1
Btrfs v3.12
File system label enclosed in single quotes when unmounted:
# umount /dev/sdb1
# btrfs filesystem show /dev/sdb1
Label: 'test1-btrfs' uuid: 1f78fa38-2f85-41d3-9be6-ae0356ae9469
Total devices 1 FS bytes used 192.00KiB
devid 1 size 2.00GiB used 240.75MiB path /dev/sdb1
Btrfs v3.12
Removing the single quotes enclosing the label makes the output
identical to that from the older 'btrfs-show' command.
Fix by using a common parser to extract the label from both the
'btrfs filesystem show' and 'btrfs-show' commands which can read the
label with and without enclosing single quotes.
Patch 2/4 - btrfs::read_label()
This patch changes the btrfs file system label parsing code to resolve
issue 1/2 by ignoring the exit status from the 'btrfs filesystem show'
command and relying on parsing the required information to determine
success or failure. Issue 2/2 is also resolved as described above.
Bug #733601 - Btrfs: Warnings and missing label with btrfs-progs 3.12
and 3.14
Issue 1/2 - GParted shows warnings for mounted btrfs file systems
'btrfs filesystem show /dev/PTN' command is used to query details of a
btrfs file system. When the file system is mounted the command reports
failed exit status 1, but only when using btrfs-progs v3.14 and v3.14.1.
This causes GParted to: (1) report warnings from the failed commands for
a mounted btrfs file system, (2) fail to determine file system usage
figures and (3) fail to display the mount point and busy indicator for
non-mounting devices in multi-device btrfs file systems. The label is
also read using the secondary blkid method via the FS_Info cache.
Failed exit status 1 when the btrfs file system is mounted:
# fgrep sdb1 /proc/mounts
/dev/sdb1 /mnt/1 btrfs rw,seclabel,relatime,space_cache 0 0
# btrfs filesystem show /dev/sdb1
Label: 'test1-btrfs' uuid: 033e6b07-ee6a-4620-a585-8580a2b83275
Total devices 1 FS bytes used 192.00KiB
devid 1 size 2.00GiB used 240.75MiB path /dev/sdb1
Btrfs v3.14.1
# echo $?
1
Successful exit status 0 when the btrfs file system is unmounted:
# umount /dev/sdb1
# btrfs filesystem show /dev/sdb1
Label: 'test1-btrfs' uuid: 033e6b07-ee6a-4620-a585-8580a2b83275
Total devices 1 FS bytes used 192.00KiB
devid 1 size 2.00GiB used 240.75MiB path /dev/sdb1
Btrfs v3.14.1
# echo $?
0
Fix by ignoring the exit status of the 'btrfs filesystem show' command
and rely on parsing the required information to determine success or
failure. The output from the older 'btrfs-show' command is almost
identical so the same code will parse it in all cases.
Patch 1/4 - btrfs::read_uuid()
This patch changes the secondary method used to read the btrfs UUID to
resolve issue 1/2 as described above.
Bug #733601 - Btrfs: Warnings and missing label with btrfs-progs 3.12
and 3.14
While one partition is busy, reformat another partition from the command
line. Afterwards parted/libparted still detects the original file
system and GParted shows errors from the file system specific tools
reporting the new file system doesn't exist. Only limitation is that
the new new file system must be recognised by libparted (or by GParted's
fallback file system signature detection).
Case #1, File system reformatting:
# parted /dev/sdb print
Model: ATA SAMSUNG SSD UM41 (scsi)
Disk /dev/sdb: 8012MB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number Start End Size Type File system Flags
1 1049kB 2149MB 2147MB primary ext2
2 2149MB 4296MB 2147MB primary ext2
# mount | fgrep sdb
/dev/sdb1 on /mnt/1 type ext2 (rw)
# mkfs.xfs -f /dev/sdb2
# blkid /dev/sdb2
/dev/sdb2: UUID="c31823a2-b81b-46fa-8246-0a59695e4834" TYPE="xfs"
# parted /dev/sdb print
Model: ATA SAMSUNG SSD UM41 (scsi)
Disk /dev/sdb: 8012MB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number Start End Size Type File system Flags
1 1049kB 2149MB 2147MB primary ext2
2 2149MB 4296MB 2147MB primary ext2
# e2label /dev/sdb2
e2label: Bad magic number in super-block while trying to open /dev/sdb2
Couldn't find valid filesystem superblock.
# dumpe2fs /dev/sdb2
dumpe2fs 1.41.12 (17-May-2010)
dumpe2fs: Bad magic number in super-block while trying to open /dev/sdb2
Couldn't find valid filesystem superblock.
Case #2, Removing device from multi-device btrfs:
# btrfs filesystem show /dev/sdb1
Label: none uuid: a05db434-efd5-4e8c-902f-05f89a88b610
Total devices 2 FS bytes used 156.00KB
devid 2 size 2.00GB used 512.00MB path /dev/sdb2
devid 1 size 2.00GB used 240.75MB path /dev/sdb1
# mount /dev/sdb1 /mnt/1
# btrfs device delete /dev/sdb2
# btrfs filesystem show /dev/sdb1
Label: none uuid: a05db434-efd5-4e8c-902f-05f89a88b610
Total devices 1 FS bytes used 92.00KB
devid 1 size 2.00GB used 714.25MB path /dev/sdb1
# btrfs filesystem show /dev/sdb2
and GParted reports this error for partition /dev/sdb2:
Unable to read the contents of this file system!
Because of this some operations may be unavailable.
The cause might be a missing software package.
The following list of software packages is required for btrfs
file system support: btrfs-tools.
This is another case of libparted reading from the whole disk device
(/dev/sdb) yet the file system tools use the partition specific block
device (/dev/sdb2), and the Linux buffer cache not providing cache
coherency. Previous scenario was fixed with:
797f0b8eeb
Flush device after wiping a file system (#688882)
This affects libparted 2.0 to 3.1 inclusive and is fixed by:
http://git.savannah.gnu.org/cgit/parted.git/commit/?id=fb99ba5ebd0dc34204fc9f1014131d5d494805bc
Revert "linux-commit: do not unnecessarily open partition device nodes"
Fix by calling ped_device_sync() to guarantee cache coherency for each
device during scanning.
Bug #723842 - GParted resizes the wrong filesystem (does not pass the
devid to btrfs filesystem resize)
With linux 3.5 and later, the device used to mount a btrfs file system
is updated in /proc/mounts when the previous mounting device is removed
from the file system. Most recent distributions make /etc/mtab a
symbolic link to /proc/mounts. However some still have /etc/mtab as a
plain file only updated by mount and umount, thus showing the old device
name which is no longer part of the file system.
On Ubuntu 13.10, which has /etc/mtab as a plain file managed by mount
and umount:
# mkfs.btrfs /dev/sdb1
# mount /dev/sdb1 /mnt/1
# btrfs device add /dev/sdb2 /mnt/1
# btrfs device delete /dev/sdb1 /mnt/1
# sync
# btrfs filesystem show /dev/sdb1
# btrfs filesystem show /dev/sdb2
Label: none uuid: e47775a6-e5ad-4fb4-9ea4-1570aa5b4009
Total devices 2 FS bytes used 28.00KB
devid 2 size 2.00GB used 272.00MB path /dev/sdb2
# fgrep btrfs /proc/mounts
/dev/sdb2 /mnt/1 btrfs rw,relatime,space_cache 0 0
# ls -l /etc/mtab
-rw-r--r-- 1 root root 842 Apr 15 19:41 /etc/mtab
# fgrep btrfs /etc/mtab
/dev/sdb1 /mnt/1 btrfs rw 0 0
This causes GParted to report /dev/sdb1 as busy and mounted at /mnt/1
when it is no longer mounted. This effects recent releases of Ubuntu,
13.04, 13.10 and 14.04.
Either /etc/mtab is a symlink and is identical to /proc/mounts or
/etc/mtab is a plain file with wrong information. Fix by not reading
mounted file systems from /etc/mtab.
However old distributions only contain 'rootfs' and '/dev/root' device
names for the / (root) file system with '/dev/root' being a block device
rather than a symlink to the true device. For example from CentOS 5.x:
# fgrep ' / ' /proc/mounts
rootfs / rootfs rw 0 0
/dev/root / ext3 rw,data=ordered 0 0
# ls -l /dev/root
brw------- 1 root root 8, 3 Jun 4 2013 /dev/root
This prevents identification, and therefore busy detection, of the
device containing the / (root) file system. Used to read /etc/mtab to
get the root file system device name.
# fgrep ' / ' /etc/mtab
/dev/sda3 / ext3 rw 0 0
# ls -l /dev/sda3
brw-r----- 1 root disk 8, 3 Jun 4 2013 /dev/sda3
As per commit:
409096f739
improved scanning for root mountpoint (/) ...
but, as discussed above, this contains an out of date device name after
the mounting device has been dynamically removed from a multi-device
btrfs, thus identifying the wrong device as busy. Instead fall back to
reading mounted file systems from the output of the mount command, but
only when required.
# mount | fgrep ' / '
/dev/sda3 on / type ext3 (rw)
Bug #723842 - GParted resizes the wrong filesystem (does not pass the
devid to btrfs filesystem resize)
In a number of cases there may be no btrfs device cache entry. Some of
the reasons why this can occur:
1) Mounting device removed from btrfs on linux <= 3.4 so old mount
point in /proc/mounts;
2) btrfs and btrfs-show commands don't exist;
3) btrfs or btrfs-show command returned non-zero exit status;
4) get_cache_entry() failed to parse output from btrfs filesystem show
or btrfs-show.
Without a valid btrfs device cache entry, busy detection for all member
devices fails. Search the GParted internal mounted partitions map as
the fallback busy detection method. This can only determine if the
mounting device is mounted or not, not any of the other members of a
multi-device btrfs file system.
Bug #723842 - GParted resizes the wrong filesystem (does not pass the
devid to btrfs filesystem resize)
GParted doesn't specify the devid when resizing a btrfs file system, so
the kernel defaults to resizing devid 1. On a multi-device btrfs this
may not be the same partition which GParted is resizing. This will
result in file system truncation and corruption. Shrinking the wrong
partition example:
1) Create a btrfs file system spanning 2 partitions:
# mkfs.btrfs /dev/sdb1 /dev/sdb2
# btrfs filesystem show /dev/sdb1
Label: none uuid: 41654265-9840-45c4-aca1-55989da358d6
Total devices 2 FS bytes used 112.00KiB
devid 1 size 2.00GiB used 437.50MiB path /dev/sdb1
devid 2 size 2.00GiB used 417.50MiB path /dev/sdb2
2) Resize /dev/sdb2 down to 1 GiB using GParted. This command was
run:
btrfs filesystem resize 1048576K /tmp/gparted-ddyGRh
which resized devid 1 (/dev/sdb1) to 1 GiB:
# btrfs filesystem show /dev/sdb1
Label: none uuid: 41654265-9840-45c4-aca1-55989da358d6
Total devices 2 FS bytes used 256.00KiB
devid 1 size 1.00GiB used 437.50MiB path /dev/sdb1
devid 2 size 2.00GiB used 417.50MiB path /dev/sdb2
but GParted instead resized /dev/sdb2 to 1 GiB:
# sfdisk -s /dev/sdb1
2097152
# sfdisk -s /dev/sdb2
1048576
Even on a single device btrfs devid 1 may no longer exist if the file
system has had the initial device removed from it. Example:
1) Create a single btrfs file system, add a second device and
remove the first:
# mkfs.btrfs /dev/sdb1
# mount /dev/sdb1 /mnt/1
# btrfs device add /dev/sdb2 /mnt/1
# btrfs device remove /dev/sdb1 /mnt/1
# umount /mnt/1
# btrfs filesystem show /dev/sdb2
Label: none uuid: 2cbf3ac3-1344-472a-a0c7-1476d23bdc9f
Total devices 1 FS bytes used 256.00KiB
devid 2 size 2.00GiB used 480.00MiB path /dev/sdb2
2) Again resize /dev/sdb2 down to 1 GiB using GParted. This
command was run:
btrfs filesystem resize 1048576K /tmp/gparted-ddyGRh
but it failed with:
ERROR: unable to resize 'tmp/gparted-lEyGaY' - No such device
A more informative error message was written to syslog:
# tail -1 /var/log/messages
Mar 12 14:15:01 localhost kernel: btrfs: resizer unable to find device 1
This is with Linux kernel 3.13.5 on Fedora 20, circa March 2014.
Fix by specifying the devid when resizing (part of) a btrfs file system.
Example command specifying devid 2:
btrfs filesystem resize 2:1048576K /tmp/1
This will always work because it is the kernel which interprets the
devid colon size parameter and has always done so since btrfs was first
added to the kernel in version 2.6.32 [1].
Reference:
[1] linux v2.6.32 fs/btrfs/ioctl.c btrfs_ioctl_resize()
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/fs/btrfs/ioctl.c?id=v2.6.32#n578
Bug #723842 - GParted resizes the wrong filesystem (does not pass the
devid to btrfs filesystem resize)
Currently GParted fails to report the usage of a multi-device btrfs file
system if it is mounted or if the used space is larger than the size of
an individual member device. When GParted does display usage figures it
also incorrectly reports the file system wide used figure against every
member device.
Mounted case:
statvfs() provides an FS size which is larger than any individual
device so is rejected. See:
GParted_Core::mounted_set_used_sectors()
Utils::get_mounted_filesystem_usage()
partition .set_sector_usage()
Unmounted case, FS used > device size:
FS used figure is larger than any individual device so free space is
calculated as a negative number and rejected. See:
btrfs::set_used_sectors()
Btrfs has a volume manager layer within the file system which allows it
to provide multiple levels of data redundancy, RAID levels, and use
multiple devices both of which can be changed while the file system is
mounted. To achieve this btrfs has to allocate space at two different
level: (1) chunks of 256 MiB or more at the volume manager level; and
(2) extents at the file data level.
References:
* Btrfs: Working with multiple devices
https://lwn.net/Articles/577961/
* Btrfs wiki: Glossary
https://btrfs.wiki.kernel.org/index.php/Glossary
This makes the question of how much disk space is being used in an
individual device a complicated question to answer. Further, the
current btrfs tools don't provide the required information.
Btrfs filesystem show only provides space usage information at the chunk
level per device. At the file extent level only a single figure for the
whole file system is provided. It also reports size of the data and
metadata being stored, not the larger figure of the amount of space
taken after redundancy is applied. So it is impossible to answer the
question of how much disk space is being used in an individual device.
Example output:
Label: none uuid: 36eb51a2-2927-4c92-820f-b2f0b5cdae50
Total devices 2 FS bytes used 156.00KB
devid 2 size 2.00GB used 512.00MB path /dev/sdb2
devid 1 size 2.00GB used 240.75MB path /dev/sdb1
Fix by guesstimating the per device used figure as the fraction of the
file system wide extent usage based on chunk usage per device.
Calculation:
ptn fs used = total fs used * devid used / sum devid used
Positives:
1) Per device used figure will correctly be between zero and allocated
chunk size.
Known inaccuracies:
[for single and multi-device btrfs file systems]
1) Btrfs filesystem show reports file system wide file extent usage
without considering redundancy applied to that data. (By default
btrfs stores two copies of metadata and one copy of data).
2) At minimum size when all data has been consolidated there will be a
few partly filled chunks of 256 MiB or more for data and metadata of
each storage profile (RAID level).
[for multi-device btrfs file systems only]
3) Data may be far from evenly distributed between the chunks on
multiple devices.
4) Extents can be and are relocated to other devices within the file
system when shrinking a device.
Bug #723842 - GParted resizes the wrong filesystem (does not pass the
devid to btrfs filesystem resize)
Linux can only show a single device name in /proc/mounts and /etc/mtab
for each mounted btrfs, even if it is a multi-device file system. So
GParted only shows a mount point for one of the devices in the btrfs, no
matter how many devices are part of the file system.
# mkfs.btrfs /dev/sdb1 /dev/sdb2
# btrfs filesystem show /dev/sdb1
Label: none uuid: 36eb51a2-2927-4c92-820f-b2f0b5cdae50
Total devices 2 FS bytes used 156.00KB
devid 2 size 2.00GB used 512.00MB path /dev/sdb2
devid 1 size 2.00GB used 240.75MB path /dev/sdb1
# mount /dev/sdb1 /mnt/1
# grep btrfs /proc/mounts
/dev/sdb1 /mnt/1 btrfs rw,seclabel,relatime,ssd,space_cache 0 0
GParted only shows the mount point for /dev/sdb1 as /mnt/1, but nothing
for /dev/sdb2.
Make GParted report the same mount point for all devices included in a
multi-device btrfs file system.
Add btrfs specific get_mount_device() method to report the mounting
device, if any, for the btrfs file system in the occupying the device in
question. Uses the existing cache of btrfs file system device
membership. Also extract common code from GParted_Core::
set_mountpoints() into set_mountpoints_helper().
Bug #723842 - GParted resizes the wrong filesystem (does not pass the
devid to btrfs filesystem resize)
Busy detection of file systems works by checking if the device is
mounted (appears in the mount_info map). For a multi-device btrfs file
system this will only report one of the devices as busy, not all of
them.
# btrfs filesystem show /dev/sdb1
Label: none uuid: 36eb51a2-2927-4c92-820f-b2f0b5cdae50
Total devices 2 FS bytes used 156.00KB
devid 2 size 2.00GB used 512.00MB path /dev/sdb2
devid 1 size 2.00GB used 240.75MB path /dev/sdb1
# mount /dev/sdb1 /mnt/1
# grep btrfs /proc/mounts
/dev/sdb1 /mnt/1 btrfs rw,seclabel,relatime,ssd,space_cache 0 0
GParted will only report /dev/sdb1 as busy, but not /dev/sdb2.
Add btrfs specific is_busy() method which reports the device as busy if
any of the devices in the btrfs file system are mounted. This uses a
cache which maps device membership in all btrfs file systems. The cache
is cleared on GParted refresh and incrementally populated as each btrfs
partition is checked for busy status.
WARNING:
Removal of the mounting device from a btrfs file system makes it
impossible to determine whether the file system is mounted or not for
linux <= 3.4. This is because /proc/mounts continues to show the old
device which is no longer a member of the file system.
# btrfs device delete /dev/sdb1 /mnt/1
# sync
# grep btrfs /proc/mounts
/dev/sdb1 /mnt/1 btrfs rw,seclabel,relatime,ssd,space_cache 0 0
# btrfs filesystem show /dev/sdb1
# btrfs filesystem show /dev/sdb2
Label: none uuid: 36eb51a2-2927-4c92-820f-b2f0b5cdae50
Total devices 1 FS bytes used 28.00KB
devid 2 size 2.00GB used 1.02GB path /dev/sdb2
Fixed in linux 3.5 by commit:
Btrfs: implement ->show_devname
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=9c5085c147989d48dfe74194b48affc23f376650
Bug #723842 - GParted resizes the wrong filesystem (does not pass the
devid to btrfs filesystem resize)
Add static member function GParted_Core::is_dev_mounted() so that other
modules can determine if a particular partition contains a mounted file
system or not.
Make it a static member function so that it can be called without
needing the gparted_core object. Extend to make the group of
manipulated variables (mount_info, fstab_info) and manipulating
functions (init_maps(), read_mountpoints_from_file(),
read_mountpoints_from_file_swaps(), get_all_mountpoints()) static too.
Bug #723842 - GParted resizes the wrong filesystem (does not pass the
devid to btrfs filesystem resize)
Previous commit:
Make partition busy detection method selectable per file system
(#723842)
causes GParted to no longer detect an unknown mounted file system as
busy.
This is needed because there are many less popular file systems which
Linux can mount but GParted doesn't recognise, such as: Acorn ADFS,
AFFS, BeFS, BFS, etc. Encountering an unrecognised mounted device can
also occur with btrfs on Linux 3.4 and earlier when the mounting device
is deleted from a multi-device btrfs file system. Happens because
/proc/mounts continues to show the original mounting device even after
that device was deleted from the file system.
# mkfs.btrfs /dev/sdb1
# mount /dev/sdb1 /mnt/1
# btrfs device add /dev/sdb2 /mnt/1
# btrfs device delete /dev/sdb1 /mnt/1
# btrfs filesystem sync /mnt/1
# grep btrfs /proc/mounts
/dev/sdb1 /mnt/1 btrfs rw,seclabel,relatime,ssd,space_cache 0 0
# blkid /dev/sdb1
# btrfs filesystem show /dev/sdb1
# blkid /dev/sdb2
/dev/sdb2: UUID="9c75647c-217a-4718-bcc7-f3ccd8cc7dc6" UUID_SUB="b5d43630-80d4-42ac-b406-185e97cd5bbe" TYPE="btrfs"
# btrfs filesystem show /dev/sdb2
Label: none uuid: 9c75647c-217a-4718-bcc7-f3ccd8cc7dc6
Total devices 1 FS bytes used 28.00KB
devid 2 size 2.00GB used 1.02GB path /dev/sdb2
Reinstate the search for mounted partitions for busy detection even for
unrecognised file system types.
Bug #723842 - GParted resizes the wrong filesystem (does not pass the
devid to btrfs filesystem resize)
GParted's primary inbuilt busy detection method is "is the partition
mounted?". A custom method is used for LVM2 PV because its not a
mounted file system.
Make busy detection selectable per file system type.
.fs.busy = FS::NONE (default)
No busy detection.
.fs.busy = FS::GPARTED
Use internal GParted method which checks if the partition is
mounted.
.fs.busy = FS:EXTERNAL
Call the file system type's member function is_busy().
LVM2 PV busy detection changes from a special case to just electing to
call the lvm2_pv::is_busy() method. Linux Software RAID remains a
special case because it's only recognised, but not otherwise supported.
Bug #723842 - GParted resizes the wrong filesystem (does not pass the
devid to btrfs filesystem resize)
Currently ./configure passes successfully even if a C++ compiler is not
installed. The first time building the code fails is when make tries
to compile the first C++ source file and the compiler executable is not
found.
Also because there is no C++ compiler the autoconf generated test for
Gtk::Window::set_default_icon_name() will always fail, leaving
HAVE_SET_DEFAULT_ICON_NAME unset. Therefore as well as installing a
C++ compiler, ./configure must be run again to correctly test for
set_default_icon_name().
This is as a result of a little known consequence of having AC_PROG_CC
proceed AC_PROG_CXX in the autoconf file configure.ac.
Subject: AC_PROG_CXX behaviour when no C++ compiler is found
http://lists.gnu.org/archive/html/bug-autoconf/2010-05/msg00001.html
Reverse the order, placing AC_PROG_CXX before AC_PROG_CC, so that the
generated ./configure script fails with an error if there is no C++
compiler available.
Bug #732803 - ./configure is successful even when C++ compile is missing
Fragment of debugging and valgrind output:
D: tid=2193 main()
...
D: tid=2202 GParted_Core::set_devices_thread()
...
D: tid=2202 Utils::execute_command(command="dumpe2fs -h /dev/sda1", output, error, use_C_locale=1)
D: tid=2202 this=0x13fef4a0 PipeCapture::PipeCapture()
D: tid=2202 this=0x13fef4f0 PipeCapture::PipeCapture()
D: tid=2202 this=0x13fef4a0 PipeCapture::connect_signal()
D: sourceid=77
D: tid=2202 this=0x13fef4f0 PipeCapture::connect_signal()
D: sourceid=78
D: tid=2193 data=0x13fef4a0 PipeCapture::_OnReadable()
D: tid=2193 this=0x13fef4a0 PipeCapture::OnReadable()
D: signal_update.emit()
D: return true
D: tid=2193 data=0x13fef4f0 PipeCapture::_OnReadable()
D: tid=2193 this=0x13fef4f0 PipeCapture::OnReadable()
D: signal_update.emit()
D: return true
D: tid=2193 data=0x13fef4a0 PipeCapture::_OnReadable()
D: tid=2193 this=0x13fef4a0 PipeCapture::OnReadable()
D: signal_update.emit()
D: return true
D: tid=2193 data=0x13fef4f0 PipeCapture::_OnReadable()
D: tid=2193 this=0x13fef4f0 PipeCapture::OnReadable()
D: signal_eof.emit()
D: return false
D: (!rc) &(pc->sourceid)=0x13fef518
D: tid=2193 data=0x13fef4a0 PipeCapture::_OnReadable()
D: tid=2193 this=0x13fef4a0 PipeCapture::OnReadable()
D: signal_update.emit()
D: return true
D: tid=2193 data=0x13fef4a0 PipeCapture::_OnReadable()
D: tid=2193 this=0x13fef4a0 PipeCapture::OnReadable()
D: signal_update.emit()
D: return true
D: tid=2193 data=0x13fef4a0 PipeCapture::_OnReadable()
D: tid=2193 this=0x13fef4a0 PipeCapture::OnReadable()
D: signal_eof.emit()
D: tid=2202 this=0x13fef4f0 PipeCapture::~PipeCapture()
D: sourceid=0
D: tid=2202 this=0x13fef4a0 PipeCapture::~PipeCapture()
D: sourceid=77
D: return false
D: (!rc) &(pc->sourceid)=0x13fef4c8
==2193== Thread 1:
==2193== Invalid write of size 4
==2193== at 0x490580: GParted::PipeCapture::_OnReadable(_GIOChannel*, GIOCondition, void*) (PipeCapture.cc:56)
==2193== by 0x38662492A5: g_main_context_dispatch (gmain.c:3066)
==2193== by 0x3866249627: g_main_context_iterate.isra.24 (gmain.c:3713)
==2193== by 0x3866249A39: g_main_loop_run (gmain.c:3907)
==2193== by 0x3D7FD45C26: gtk_main (gtkmain.c:1257)
==2193== by 0x469743: GParted::GParted_Core::set_devices(std::vector<GParted::Device, std::allocator<GParted::Device> >&) (GParted_Core.cc:155)
==2193== by 0x4A78F1: GParted::Win_GParted::menu_gparted_refresh_devices() (Win_GParted.cc:1259)
==2193== by 0x4A7886: GParted::Win_GParted::on_show() (Win_GParted.cc:1253)
==2193== by 0x3D82B2009C: Gtk::Widget_Class::show_callback(_GtkWidget*) (widget.cc:3855)
==2193== by 0x3867210297: g_closure_invoke (gclosure.c:777)
==2193== by 0x3867221B86: signal_emit_unlocked_R (gsignal.c:3516)
==2193== by 0x386722A0F1: g_signal_emit_valist (gsignal.c:3330)
==2193== Address 0x13fef4c8 is not stack'd, malloc'd or (recently) free'd
==2193==
PipeCapture.cc (with debugging):
46 gboolean PipeCapture::_OnReadable( GIOChannel *source,
47 GIOCondition condition,
48 gpointer data )
49 {
50 std::cout << "D: tid=" << (long int)syscall(SYS_gettid) << " data=" << data << " PipeCapture::_OnReadable()" << std::endl;
51 PipeCapture *pc = static_cast<PipeCapture *>(data);
52 gboolean rc = pc->OnReadable( Glib::IOCondition(condition) );
53 if (!rc)
54 {
55 std::cout << "D: (!rc) &(pc->sourceid)=" << &(pc->sourceid) << std::endl;
56 pc->sourceid = 0;
57 }
58 return rc;
59 }
The use after free across threads only happens when an external program
is being executed from a thread other than the main() thread. This is
because by default glib registered callbacks are run by the glib main
loop, which is only called from the main() thread with Gtk::Main::run().
Event sequence:
tid=2193 tid=2202
main()
...
GParted_Core::set_devices()
Glib::Thread::create(... set_devices_thread ...)
Gtk::Main::run() GParted_Core::set_devices_thread()
...
Utils::execute_command("dumpe2fs ... /dev/sda1" ...)
Glib::spawn_async_with_pipes()
PipeCapture outputcapture(out, output)
outputcapture.connect_signal()
//Glib main loop runs callback
PipeCapture::_OnReadable()
pc->OnReadable()
//output read
signal_update.emit()
return true
...
//Glib main loop runs callback
PipeCapture::_OnReadable()
pc->OnReadable()
//eof reached
[1] signal_eof.emit()
return status.exit_status
[2] PipeCapture::~PipeCapture()
[3] return false
[4] pc->sourceid = 0
What is happening is that the PipeCapture destructor [2] is running in
the set_devices_thread() thread and freeing the object's memory as soon
as signal_eof.emit() [1] has been called. Then signal_eof.emit()
returns back to OnReadable() which then returns false [3] back to the
_OnReadable() callback function which then assigns 0 to sourceid member
variable [4] in the already freed object, detected by valgrind as:
Invalid write of size 4
at ... GParted::PipeCapture::_OnReadable(...) (PipeCapture.cc:56)
This is happening because PipeCapture member variable sourceid is being
saved, in a different thread, just so the _OnReadable() callback can be
removed. However a glib IOChannel callback, type GIOFunc(), returning
false will be automatically removed.
GLib Reference Manual 2.26 / IO Channels
https://developer.gnome.org/glib/2.26/glib-IO-Channels.html#GIOFunc
GIOFunc()
Returns : the function should return FALSE if the event source
should be removed
Therefore fix by just not saving the event sourceid at all, and not
calling g_source_remove() to manually remove the callback, but instead
letting glib automatically remove the callback when it returns false.
Bug #731752 - Write after free cross thread race in
PipeCapture::_OnReadable()
OperationDetail was storing its children in a std::vector. This means they
can be moved around in memory arbitrarily, going through indeterminate
lifetimes. This is generally a bad thing for any non trivial object and
in the case of OperationDetail, it created havoc with the way it maintains
pointers between parent/child objects for signal connections. It will now
keep only pointers to children in a std::vector instead, so their lifetime
can be controlled, fixing various crashes.
Bug 729139 - Refactor OperationDetail to address random behavior
Returning false from the OnReadable callback causes the source to
be destroyed, but PipeCapture::~PipeCapture was destroying it a
second time. Prevent this by zeroing out the sourceid.
Bug 729800 - Prevent GSource double-destroy warning messages
Place the LVM2 member names in a single string, separated by new lines,
so that they can be selected all together by the user. This was not
previously possible when each member name was placed in a separate
widget.
A LVM volume group can contain more than one member and comprise a
multi-line entry. As such set the "Members" field header and value
label pair to top vertical alignment.
As part of the work on bug 652044 - uses deprecated APIs, selectable
vertical alignment was defaulted to ALIGN_CENTER for all labels. The
relevant commits can be viewed in comment 26 of said bug report.
https://bugzilla.gnome.org/show_bug.cgi?id=652044#c26
For multi-line labels a vertical ALIGN_CENTER value is not
consistently aesthetically pleasing. This becomes obvious when a
single-line heading label is paired with a multi-line value label.
To improve the aesthetics, a vertical alignment of ALIGN_TOP is
preferred.
Hence re-add the ability to optionally specify a vertical alignment for
labels. If a yalign value is not specified a default vertical alignment
of ALIGN_CENTER is used.
With the volume group name displayed immediately below the status field,
the vgname is no longer needed in the status field.
Bug 690542 - Partition Information Dialog Warning not readable
When the partition information dialog is resized, the data below the
partition graphic would remain left aligned. Improve the aesthetics by
horizontally centering the data in the window similar to the partition
graphic.
Bug 690542 - Partition Information Dialog Warning not readable
Placing the LVM2 member names in a string list enables the member names
to be selected all at once by a user. This was not possible with
separate label widgets for each LVM2 member name.
Bug 690542 - Partition Information Dialog Warning not readable
Piotr Drąg