Users will expect to see that copying of LUKS is available in the File
System Support dialog, even if technically what is implemented is
copying of the file system within an open encryption mapping. There is
no other reason to do this change as these two previous commits have
fully enabled copying of encrypted content:
Implement copy operation of encrypted file systems (#774814)
Preview copy operation of encrypted file systems (#774818)
Set LUKS .copy capability so that the dialog shows copying availability,
but then disallow copying of closed LUKS. (Checking for the capability
and performing copying the content of an open LUKS encryption mapping is
inherent in Win_GParted::set_valid_operations() and GParted_Core::copy()
in the way that they access the block device containing the file system,
whether encrypted or not).
Bug 774818 - Implement LUKS read-write actions NOT requiring a
passphrase
Implement the copy operation by making the copy code work with the
Partition object directly containing the file system, instead of the
enclosing PartitionLUKS object containing the LUKS encryption mapping.
Bug 774818 - Implement LUKS read-write actions NOT requiring a
passphrase
Implement composing of the copy paste operation for encrypted file
systems.
Copying a closed LUKS partition would duplicate the LUKS header
containing the UUID, passphrase and master encryption key. From a
security point of view having additional copies of encrypted data with
the same master key is an extra risk, but it all depends on what is
going to happen with that copy. The Cryptsetup FAQ [1] talks about how
to make a backup at the file system level and block level, preferring
file system level with separate encryption if needed. It strongly
recommends separate encryption if the copy is removable or going
off-site [2]. Also in the case of cloning the data, cloning the LUKS
container is strongly discouraged [3].
Therefore copying of encrypted file systems will be implemented by
copying the file system inside an open LUKS encryption mapping and not
by copying a closed LUKS partition.
Also, while creating new LUKS encryption is not yet supported, copying
an encrypted file system into a new partition will not be permitted as
that will always decrypt the data. An encrypted file system will be
allowed to be copied into an existing plain partition, decrypting the
data, or into an existing open encrypted partition, keeping it
encrypted. Pasting over the top of a closed encrypted partition will
remove the LUKS encryption. (This is planned to be removed when
creating and removing LUKS encryption is implemented as part of full
LUKS read-write support).
Remember that when pasting into an existing partition the file system
must fit within the available space and that encryption has overhead
from the LUKS header. Therefore copying from a plain partition into a
partition of the same size with open an encryption mapping will not fit
for space reasons.
[1] The Cryptsetup FAQ, Backup and data Recovery section
https://gitlab.com/cryptsetup/cryptsetup/wikis/FrequentlyAskedQuestions
[2] 6.7 Does a backup compromise security?
"If you do network-backup or tape-backup, I strongly recommend to go the
filesystem backup path with independent encryption, as you typically
cannot reliably delete data in these scenarios, especially in a cloud
setting."
[3] 6.15 Can I clone a LUKS container?
"You can, but it breaks security, because the cloned container has the
same header and hence the same master key. You cannot change the master
key on a LUKS container, even if you change the passphrase(s), the
master key stays the same. That means whoever has access to one of the
clones can decrypt them all, completely bypassing the passphrases.
The right way to do this is to first luksFormat the target container,
then to clone the contents of the source container, with both containers
mapped, i.e. decrypted. You can clone the decrypted contents of a LUKS
container in binary mode, although you may run into secondary issues
with GUIDs in filesystems, partition tables, RAID-components and the
like. These are just the normal problems binary cloning causes.
Note that if you need to ship (e.g.) cloned LUKS containers with a
default passphrase, that is fine as long as each container was
individually created (and hence has its own master key). In this case,
changing the default passphrase will make it secure again."
Bug 774818 - Implement LUKS read-write actions NOT requiring a
passphrase
Ensure pre-condition of never being passed a Partition object containing
an open LUKS encryption mapping is met for format operation related
methods.
Bug 774818 - Implement LUKS read-write actions NOT requiring a
passphrase
Formatting an existing encrypted partition will format the file system
within the encrypted mapping. Formatting over the top of a closed
encrypted partition will remove the encryption. (The latter is planned
to be prevented when creating and removing LUKS encryption is
implemented as part of full LUKS read-write support).
Composing the format operation inside an open LUKS encryption mapping
also has to account for the size of that mapping and construct a
PartitionLUKS object containing the new file system. Implementing the
operation itself is as simple as passing the Partition object directly
containing the file system, instead of the outer PartitionLUKS object.
Bug 774818 - Implement LUKS read-write actions NOT requiring a
passphrase
Already have:
Utils::get_filesystem_string(FS_EXT2) -> "ext2"
virtual Partition::get_filesystem_string() -> "ext2"
virtual PartitionLUKS::get_filesystem_string() -> "[Encrypted] ext2"
Add these:
Utils::get_encrypted_string() -> "[Encrypted]"
Utils::get_filesystem_string(false, FS_EXT2) -> "ext2"
Utils::get_filesystem_string(true, FS_EXT2) -> "[Encrypted] ext2"
This is ready for use of Utils::get_filesystem_string(true, FS_EXT2)
when composing the preview of a format of an encrypted file system by
Win_GParted::activate_format().
Bug 774818 - Implement LUKS read-write actions NOT requiring a
passphrase
Adapts the swapon/swapoff, activate/deactivate Volume Group and mount/
unmount file system code to work with the Partition object directly
containing the file system.
Bug 774818 - Implement LUKS read-write actions NOT requiring a
passphrase
Ensure pre-condition of never being passed a Partition object containing
an open LUKS encryption mapping is met for check operation related
methods.
Bug 774818 - Implement LUKS read-write actions NOT requiring a
passphrase
Just update the code describing and implementing the check operation.
No need to update the code composing the operation in
Win_GParted::activate_check() as that is incredibly simple, just cloning
the Partition object and doesn't need changing.
Note that for encrypted file systems this does:
1) runs FSCK on the encrypted file system;
2) grows the file system to fill the encrypted mapping.
At this time it does not grow the encryption mapping to fill the
partition. That will be added after resize/move has been implemented.
Bug 774818 - Implement LUKS read-write actions NOT requiring a
passphrase
Ensure pre-condition of never being passed a Partition object containing
an open LUKS encryption mapping is met for label file system operation
related methods.
Bug 774818 - Implement LUKS read-write actions NOT requiring a
passphrase
Again, just need to change the code when composing, describing and
implementing the operation to query and set the Partition object
directly containing the file system, instead of the enclosing encryption
mapping to make it work.
Bug 774818 - Implement LUKS read-write actions NOT requiring a
passphrase
Ensure pre-condition of never being passed a Partition object containing
an open LUKS encryption mapping is met for change UUID operation related
methods.
Bug 774818 - Implement LUKS read-write actions NOT requiring a
passphrase
When composing, describing and implementing the operation just need the
code to query and set the Partition object directly containing the file
system, instead of the enclosing encryption mapping to make it work.
The operation details for setting a new UUID on an encrypted ext4 file
system become:
Set a new random UUID on [Encrypted] ext4 file system on /dev/sdb4
+ calibrate /dev/sdb4
+ Set UUID on /dev/mapper/sdb4_crypt to a new, random value
+ tune2fs -U random /dev/mapper/sdb4_crypt
tune2fs 1.41.12 (17-May-2010)
Also note the now documented rule in apply_operation_to_disk() which
says each operation must leave the status of the encryption mapping and
file system as it found it.
Bug 774818 - Implement LUKS read-write actions NOT requiring a
passphrase
Provide and use a single interface for getting the file system string
for display, regardless of whether the partition is encrypted or the
encryption mapping is active or not.
Example return values for get_filesystem_string() for different types
and states of Partition objects:
1) Plain ext4 file system: -> "ext4"
2) Closed encrypted: -> "[Encrypted]"
3) Open encrypted ext4 file system: -> "[Encrypted] ext4"
This simplifies the code in TreeView_Detail::create_row() which sets the
file system type displayed in the main window. The same method will
then also be used when setting the operation description as each
operation is updated to handle encrypted file systems.
Bug 774818 - Implement LUKS read-write actions NOT requiring a
passphrase
The previous commit changed how the code behind the main window
retrieved the file system label for display. This is the relevant
changes in TreeView_Detail::create_row():
+ const Partition & filesystem_ptn = partition.get_filesystem_partition();
...
- Glib::ustring temp_filesystem_label = partition.get_filesystem_label();
+ Glib::ustring temp_filesystem_label = filesystem_ptn.get_filesystem_label();
treerow[treeview_detail_columns.label] = temp_filesystem_label;
In the case of an encrypted file system get_filesystem_label() is now
called on the Partition object directly rather than on the outer
Partition object containing the LUKS encryption.
The code behind the Information dialog always obtained and used the
Partition object directly containing the file system to call
get_filesystem_label() since read-only LUKS support was added.
Therefore the virtualised PartitionLUKS::get_filesystem_label() is no
longer needed, so remove it.
Bug 774818 - Implement LUKS read-write actions NOT requiring a
passphrase
There are multiple cases of code wanting to work with the Partition
object directly containing the file system, regardless of whether it is
within a PartitionLUKS object or not. The code had to do something
similar to this to access it:
const Partition * filesystem_ptn = &partition;
if ( partition.filesystem == FS_LUKS && partition.busy )
filesystem_ptn = &dynamic_cast<const PartitionLUKS *>( &partition )->get_encrypted();
...
// Access Partition object directly containing the file system
filesystem_ptn-> ...
Implement and use virtual accessor get_filesystem_partition() which
allows the code to be simplified like this:
const Partition & filesystem_ptn = partition.get_filesystem_partition();
...
// Access Partition object directly containing the file system
filesystem_ptn. ...
Bug 774818 - Implement LUKS read-write actions NOT requiring a
passphrase
In the main window, display the busy status of the file system only
according to whether it is mounted or not, ignoring the status of the
encryption mapping.
Bug 774818 - Implement LUKS read-write actions NOT requiring a
passphrase
More recent versions of blkid don't report an ISO9660 file system on the
whole disk device if partitions can be reports for embedded partitions.
However when querying the whole disk device directly then the expected
ISO9660 file system is reported. For example on CentOS 7 with the
previous ISO images:
# wget http://git.kernel.org/cgit/utils/util-linux/util-linux.git/plain/tests/ts/isosize/sample.iso.gz
# dd if=/dev/zero bs=1M of=/dev/sdc
# zcat sample.iso.gz | dd of=/dev/sdc
# blkid -v
blkid from util-linux 2.23.2 (libblkid 2.23.0, 25-Apr-2013)
# blkid | fgrep /dev/sdc
/dev/sdc1: UUID="2013-01-04-22-05-45-00" LABEL="ARCH_201301" TYPE="iso9660" PTTYPE="dos"
# blkid /dev/sdc
/dev/sdc: UUID="2013-01-04-22-05-45-00" LABEL="ARCH_201301" TYPE="iso9660" PTTYPE="dos"
# wget http://cdimage.debian.org/debian-cd/8.6.0/amd64/iso-cd/debian-8.6.0-amd64-netinst.iso
# dd if=/dev/zero bs=1M of=/dev/sdc
# dd if=debian-8.6.0-amd64-netinst.iso bs=1M of=/dev/sdc
# blkid | fgrep /dev/sdc
/dev/sdc1: UUID="2016-09-17-14-23-48-00" LABEL="Debian 8.6.0 amd64 1" TYPE="iso9660" PTTYPE="dos"
/dev/sdc2: SEC_TYPE="msdos" UUID="17F3-1162" TYPE="vfat"
# blkid /dev/sdc
/dev/sdc: UUID="2016-09-17-14-23-48-00" LABEL="Debian 8.6.0 amd64 1" TYPE="iso9660" PTTYPE="dos"
This behavioural difference with blkid is probably as a result of newer
versions of udev informing the kernel of the partitions embedded within
the ISO9660 image, and not directly as a result of a change in blkid
itself. Older distributions don't have partition entries for the above
ISO images, but CentOS 7 (with udev 219) and later distributions do have
partition entries:
# fgrep sdc /proc/partitions
8 16 8388608 sdc
8 17 252928 sdc1
8 18 416 sdc2
Fix by ensuring that the blkid FS_Info cache has entries for whole disk
devices, even if each entry has all empty attributes because there is a
partition table and not a recognised file system.
Calling blkid on whole disk devices containing partition tables produces
output like this, with newer versions of blkid:
# blkid /dev/sda
/dev/sda: PTTYPE="dos"
# blkid /dev/sdb
/dev/sdb: PTTYPE="gpt"
This will be loaded into the FS_Info cache as a blank entry for the
device by run_blkid_load_cache(). There will be a path name but all the
other attributes will be blank because there are no TYPE, SEC_TYPE, UUID
or LABEL name value pairs. With older versions of blkid no output is
produced at all. In that case load_fs_info_cache_extra_for_path() will
create the same blank entry with just the path name defined.
Bug 771244 - gparted does not recognize the iso9660 file system in
cloned Ubuntu USB boot drives
ISO9660 images can contain embedded partitions which when encountered on
whole disk drives causes libparted to report various warnings and
errors. For example on CentOS 6 with upgraded libparted 2.4 the
following errors and warnings are encountered with various ISO images.
(Deliberately using an older distribution with older blkid to avoid
another issue with blkid addressed in the following patch).
Libparted error message 1:
# wget http://git.kernel.org/cgit/utils/util-linux/util-linux.git/plain/tests/ts/isosize/sample.iso.gz
# dd if=/dev/zero bs=1M of=/dev/sdc
# zcat sample.iso.gz | dd of=/dev/sdc
# blkid -v
blkid from util-linux-ng 2.17.2 (libblkid 2.17.0, 22-Mar-2010)
# blkid | fgrep /dev/sdc
/dev/sdc: LABEL="ARCH_201301" TYPE="iso9660"
# ./gpartedbin /dev/sdc
======================
libparted : 2.4
======================
Invalid partition table - recursive partition on /dev/sdc.
Libparted Error
(-) Invalid partition table - recursive partition on /dev/sdc.
[ Cancel ] [ Ignore ]
Libparted error message 2:
# wget http://cdimage.debian.org/debian-cd/8.6.0/amd64/iso-cd/debian-8.6.0-amd64-netinst.iso
# dd if=/dev/zero bs=1M of=/dev/sdc
# dd if=debian-8.6.0-amd64-netinst.iso bs=1M of=/dev/sdc
# blkid | fgrep /dev/sdc
/dev/sdc: LABEL="Debian 8.6.0 amd64 1" TYPE="iso9660"
# ./gpartedbin /dev/sdc
======================
libparted : 2.4
======================
/dev/sdc contains GPT signatures, indicating that it has a GPT
table. However, it does not have a valid fake msdos partition
table, as it should. Perhaps it was corrupted -- possibly by a
program that doesn't understand GPT partition tables. Or perhaps
you deleted the GPT table, and are now using an msdos partition
table. Is this a GPT partition table?
Libparted Warning
/!\ /dev/sdc contains GPT signatures, indicating that it has a GPT
table. However, it does not have a valid fake msdos partition
table, as it should. Perhaps it was corrupted -- possibly by a
program that doesn't understand GPT partition tables. Or
perhaps you deleted the GPT table, and are now using an msdos
partition table. Is this a GPT partition table?
[ Yes ] [ No ]
These messages are because GParted is calling ped_disk_new() to attempt
to read the partition table even before it has tried to recognise the
ISO9660 file system on the whole disk drive. Full call chain is:
set_devices_thread()
set_device_from_disk()
get_device_and_disk()
get_disk()
ped_disk_new()
Fix this by delaying the call to ped_disk_new() until after whole disk
drive recognition has been performed. Replace combined
get_device_and_disk() with separate get_device() and only call
get_disk() after no whole disk drive file system has been recognised.
This is similar to how calibrate_partition() and
erase_filesystem_signatures() are structured to also handle whole disk
drive file systems.
Bug 771244 - gparted does not recognize the iso9660 file system in
cloned Ubuntu USB boot drives
Move code from GParted_Core::set_devices_thread() performing top level
population of each Device object during the scan of the drives into new
set_device_from_disk() method.
Bug 771244 - gparted does not recognize the iso9660 file system in
cloned Ubuntu USB boot drives
Requires blkid.
Note that FS_LUKS was also moved to more closely match the order in
include/Utils.h
Bug 771244 - gparted does not recognize the iso9660 file system in
cloned Ubuntu USB boot drives
Glibc 2.25 is deprecating <sys/types.h> including <sys/sysmacros.h>.
https://sourceware.org/git/?p=glibc.git;a=commitdiff;h=dbab6577c6684c62bd2521c1c29dc25c3cac966f
Deprecate inclusion of <sys/sysmacros.h> by <sys/types.h>
Building on Fedora Rawhide with Glibc 2.24.90 produces these warnings:
BlockSpecial.cc:64:13: warning: In the GNU C Library, "major" is defined
by <sys/sysmacros.h>. For historical compatibility, it is
currently defined by <sys/types.h> as well, but we plan to
remove this soon. To use "major", include <sys/sysmacros.h>
directly. If you did not intend to use a system-defined macro
"major", you should undefine it after including <sys/types.h>.
m_major = major( sb.st_rdev );
^~~~~~~~~~~~~~~~~~~~
BlockSpecial.cc:65:13: warning: In the GNU C Library, "minor" is defined
by <sys/sysmacros.h>. For historical compatibility, it is
currently defined by <sys/types.h> as well, but we plan to
remove this soon. To use "minor", include <sys/sysmacros.h>
directly. If you did not intend to use a system-defined macro
"minor", you should undefine it after including <sys/types.h>.
m_minor = minor( sb.st_rdev );
^~~~~~~~~~~~~~~~~~~~
Code needing major and minor macros should include <sys/sysmacros.h>
directly. As both Glibc and musl-libc have always provided this header
and GParted is a Linux only application, just always include the header
and don't bother with an autoconf check for its existence.
Bug 776173 - Missing sys/sysmacros.h #include, needed for experimental
glibc versions
The details for a file system shrink operation look like this with the
"shrink file system" message being missing:
Shrink /dev/sdb1 from 128.00 MiB to 100.00 MiB
+ calibrate /dev/sdb1
+ check file system on /dev/sdb1 for errors and (if possible) fix them
+ e2fsck -f -y -v -C 0 /dev/sdb1
+ resize2fs -p /dev/sdb1 102400K
+ shrink partition from 128.00 MiB to 100.00 MiB
This earlier commit [1] in the series dropped the message while moving
code from resize_filesystem() to shrink_filesystem(). Re-add the
message back.
[1] a0158abbeb
Refactor resizing file system apply methods (#775932)
Bug 775932 - Refactor mostly applying of operations
Explain when and why shrinking file systems is disallowed.
Unnecessary history. Check added in 2004-12-15:
d100935b55
:Set_Valid_Operations()
Comment later added in 2006-07-30:
677a21f50a
improved errorhandling a bit. At the initialscan we store errors/warnings
It made the code look a little messy, is easily resolved in the build
system and made the dependencies more complicated than needed. Each
GParted header was tracked via multiple different names (different
numbers of "../include/" prefixes). For example just looking at how
DialogFeatures.o depends on Utils.h:
$ cd src
$ make DialogFeatures.o
$ egrep ' [^ ]*Utils.h' .deps/DialogFeatures.Po
../include/DialogFeatures.h ../include/../include/Utils.h \
../include/../include/../include/../include/../include/../include/Utils.h \
../include/../include/../include/Utils.h \
After removing "../include/" from the GParted header #includes, just
need to add "-I../include" to the compile command via the AM_CPPFLAGS in
src/Makefile.am. Now the dependencies on GParted header files are
tracked under a single name (with a single "../include/" prefix). Now
DialogFeatures.o only depends on a single name to Utils.h:
$ make DialogFeatures.o
$ egrep ' [^ ]*Utils.h' .deps/DialogFeatures.Po
../include/DialogFeatures.h ../include/Utils.h ../include/i18n.h \
Simplify conditions checking whether activate/deactivate of an LVM2 PV
is possible. Excluding extended partition type was unnecessary as it
only matters that the file system type is LVM2 PV or not.
Also remove activate/deactivate from the comment above as that check
only determines if the busy state of file systems and swap space can be
toggled.
Linux-swap is recreated as part of copy, resize and move operations and
the code was special cased to implement that by calling the linux-swap
specific resize method. However the displayed text always said "growing
file system" and then proceeded to recreate linux swap. Example
operation:
Copy /dev/sdb1 to /dev/sdb2
...
+ copy file system from /dev/sdb1 to /dev/sdb2
Partition copy action skipped because linux-swap file system does not contain data
+ grow file system to fill the partition
+ create new linux-swap file system
+ mkswap -L"" -U "77d939ef-54d6-427a-a2bf-a053da7eed4c" /dev/sdb2
Setting up swapspace version 1, size = 262140 KiB
LABEL=, UUID=77d939ef-54d6-427a-a2bf-a053da7eed4c
Fix by writing recreate_linux_swap_filesystem() method with better
messaging and use everywhere maximise_filesystem() was previously used
to recreate linux-swap. Also as this is a create step, erase the
partition first to prevent the possibility of any other file system
signatures being found afterwards. Now the operation steps are more
reflective of what is actually being performed.
Copy /dev/sdb1 to /dev/sdb2
...
+ copy file system from /dev/sdb1 to /dev/sdb2
Partition copy action skipped because linux-swap file system does not contain data
+ clear old file system signatures in /dev/sdb2
+ create new linux-swap file system
+ mkswap -L"" -U "77d939ef-54d6-427a-a2bf-a053da7eed4c" /dev/sdb2
Setting up swapspace version 1, size = 262140 KiB
LABEL=, UUID=77d939ef-54d6-427a-a2bf-a053da7eed4c
Bug 775932 - Refactor mostly applying of operations
Resizing a file system only checked the .grow and .shrink support
capabilities of the file system, even when perform online resizing.
This wasn't a issue because .online_grow and .online_shrink were always
set from the offline .grow and .shrink capabilities respectively, but
only when online resizing was possible.
However the Device Mapper encryption mapping used for LUKS can only be
resized when online, and not when offline. Therefore the correct
.online_grow and .online_shrink capabilities needs to be checked to
prevent a LUKS resize step failing as not implemented.
Bug 775932 - Refactor mostly applying of operations
resize_filesystem() was meeting two different needs:
1) when called with fill_partition = false it generated operation
details;
2) when called from maximize_filesystem() with fill_partition = true it
skipped generating any operation details;
then ran the switch statement to select the resize implementation. So
extract the common switch statement into new method
resize_filesystem_implement().
Then observe that the only time resize_filesystem() was called to grow
the file system was when re-creating linux-swap. Therefore change that
call to use maximize_filesystem() and rename to shrink_filesystem() and
modify the operation detail messages to match.
Bug 775932 - Refactor mostly applying of operations
Attempting to copy or move linux-swap partition was possible when the
mkswap command was unavailable. However as linux-swap is re-created as
part of these operations the mkswap command is a mandatory requirement.
Example copy operation results:
Copy /dev/sdb1 to /dev/sdb2
+ calibrate /dev/sdb1
+ check file system on /dev/sdb1 for errors and (if possible) fix them
+ create empty partition
+ set partition type on /dev/sdb2
+ copy file system from /dev/sdb1 to /dev/sdb2 [SUCCESS]
Partition copy action skipped because linux-swap file system does not contain data
+ grow file system to fill the partition [WARNING]
growing is not available for this file system
Because mkswap was not available the grow action didn't re-create
the linux-swap so the newly copied partition is left without any content
and therefore reported as an unknown partition.
Fix by making copying and moving linux-swap depend on mkswap being
available.
Bug 775932 - Refactor mostly applying of operations
Make the methods called below apply_operation_to_disk() follow a
standard naming convention:
* Contains "_partition"
Uses libparted to query or change the partition in the disk label
(partition table).
E.g.:
calibrate_partition()
create_partition()
delete_partition()
name_partition()
resize_move_partition()
set_partition_type()
* Contains "_filesystem"
Manipulates the file system within the partition, mostly using the
FileSystem and derived class methods.
E.g.:
create_filesystem()
remove_filesystem()
label_filesystem()
copy_filesystem()
erase_filesystem_signatures()
check_repair_filesystem()
resize_filesystem()
maximize_filesystem()
* Other
Compound method calling multiple partition and file system related
apply methods.
E.g.:
create()
format()
copy()
resize_move()
resize()
move()
Rename:
Delete() -> delete_partition()
change_uuid() -> change_filesystem_uuid()
Bug 775932 - Refactor mostly applying of operations
The initial check in each of GParted_Core::resize() and move() check for
impossible conditions. (See resize_move() where both functions are
called for confirmation). The old messages could have suggested the
user somehow composed the operation incorrectly if they had ever been
seen.
Make it clear these failures are programming bugs, thus expecting the
user to raise a bug report should they ever be seen. While for now the
checks are obviously impossible they document preconditions for the
functions. More precondition checks are expected to be added later.
Bug 775932 - Refactor mostly applying of operations
Simplify the move() function. Change it into an if-operation-fails-
return-false style and re-write the final overly complicated conditional
check repair and maximise file system.
The final condition said if the file system was linux-swap or the new
partition was larger, perform a check repair and maximize file system.
However this is a move only step with a check at the top of the move()
function ensuring the Partition objects are the same size. So that
simplifies to only checking for linux-swap. As the context is a move
and linux-swap is recreated, performing a check repair first is
unnecessary, so remove that too.
Bug 775932 - Refactor mostly applying of operations
Make the logic at the resize_move_partition() call sites a little
simpler by not having to avoid calling it for whole disk device
partitions. Make it a successful non-operation in that case.
Bug 775932 - Refactor mostly applying of operations
Make the logic at the check_repair_filesystem() call sites a little
easier by not having to avoid calling it for online file systems. The
function handles that itself as a silent non-operation.
Bug 775932 - Refactor mostly applying of operations
Make the logic at the set_partition_type() call sites a little simpler
by not having to avoid calling it for whole disk device Partition
objects. Make set_partition_type() handle those itself as a silent
non-operation.
This is similar to how calibrate_partition() could be called on an
UNALLOCATED Partition object, and update_bootsector() is called on
non-NTFS file system Partition objects. Both are successful and silent
non-operations.
Bug 775932 - Refactor mostly applying of operations
Until now an encryption mapping has been modelled as a Partition object
similar to a partition like this:
.encrypted.device_path = "/dev/sdb1"
.encrypted.path = "/dev/mapper/sdb1_crypt"
.encrypted.whole_device = false
.encrypted.sector_start = // start of the mapping in the partition
.encrypted.sector_end = // end of the mapping in the partition
However accessing device_path in the start to end sector range is not
equivalent to accessing the partition path as it doesn't provide access
to the encrypted data. Therefore existing functions which read and
write partition data (GParted file system copying and signature erasure)
via libparted using the device_path won't work and will in fact destroy
the encrypted data. This could be coded around with an extra case in
the device opening code, however it is not necessary.
An encrypted block special device /dev/mapper/CRYPTNAME looks just like
a whole disk device because it doesn't contain a partition and the file
system it contains starts at sector 0 and goes to the end. Therefore
model an encryption mapping in the same way a whole disk device is
modelled as a Partition object like this:
.encrypted.device_path = "/dev/mapper/sdb1_crypt"
.encrypted.path = "/dev/mapper/sdb1_crypt"
.encrypted.whole_device = true
.encrypted.sector_start = 0
.encrypted.sector_end = // size of the encryption mapping - 1
Now GParted file system copy and erasure will just work without any
change. Just need to additionally store the LUKS header size, which was
previous stored in the sector_start, for use in the
get_sectors_{used,unused,unallocated}() calculations.
Bug 775932 - Refactor mostly applying of operations
Split out the switch statement selecting the copy implementation and
associated copy file system operation detail message into a separate
copy_filesystem() method, matching how a number of other operations are
coded. This is why the previous copy_filesystem() methods needed
renaming.
Re-write the remaining copy() into if-operation-fails-return-false style
to simplify it. Re-write final complicated conditional check repair and
maximise file system into separate positive if conditions for swap and
larger partition to make it understandable.
The min_size parameter to copy() was queried from the partition_src
parameter also passed to copy(). Drop the parameter and query inside
copy() instead.
Bug 775932 - Refactor mostly applying of operations
Rename GParted_Core methods:
copy_filesystem(4 params) -> copy_filesystem_internal()
copy_filesystem(5 params) -> copy_filesystem_internal()
copy_filesystem(10 params) -> copy_blocks()
See the following commit for the desire to do this.
Bug 775932 - Refactor mostly applying of operations
Files were named Block_Copy and the class was named block_copy. Change
to the primary naming convention of CamelCase class name and matching
file names.
Also make CopyBlocks::copy_block() a private method as it is only used
internally within the class.
Bug 775932 - Refactor mostly applying of operations
These two methods had a lot of repeated and common code. Both determine
if the partition has any pending operations, notify the user that
changing the busy status can not be performed, and report any errors
when changing the status.
Extract the common code into sub-functions check_toggle_busy_allowed()
and show_toggle_failure_dialog() to handle showing the message dialogs.
Also refactor toggle_busy_state() to make it clear that it handles 5
cases of swapon, swapoff, activate VG, deactivate VG and unmount file
system.
Also remove out of date comment near the top of toggle_busy_state()
stating there can only be pending operations for inactive partitions is
out of date. Some file systems can be resized while online and
partition naming is allowed whatever the busy status of the file system.
Bug 775932 - Refactor mostly applying of operations
The primary reason to refactor unmount_partition() is to pass the
Partition object to be unmounted, rather than use member variable
selected_partition_ptr so that it doesn't have to handle the differences
between encrypted and non-encrypted Partition objects. The calling
function can deal with that instead. Then there were lots of small
reasons to change almost every other line too:
* Return success or failure rather than updating a passed pointer with
the result. Leftover from when the function used to be a separate
thread:
commit 52a2a9b00a
Reduce threading (#685740)
* Pass updated error string by reference rather than by pointer. Likely
another leftover.
* Stop borrowing the updated error string as a local variable for the
error output from the umount command. Use new umount_error local
variable instead. Was bad practice making the code harder to
understand.
* Rename failed_mountpoints to skipped_mountpoints to better reflect
that it contains the mount points not attempted to be unmounted
because two or more file systems are mounted at that point.
* Rename errors to umount_errors to better reflect it contains the
errors from each failed umount command.
* Document the reason for mount points being skipped.
* Update the skipped mount points message to state definitely why they
could not be unmounted rather than stating most likely.
* Simplify logic processing the error cases and return value.
* Made static because it no longer accesses any class members.
* Remove out dated "//threads.." comment from the header. Another
leftover from when the function use to be a separate thread.
Bug 775932 - Refactor mostly applying of operations
Reorder the parameters into the same order in which they occur in the
row, i.e. Name first, then Mount Point and finally Label. Rename local
variables in load_partitions(1 param) and parameters of
load_partitions(5 params) prefixing with "show_" to make it clearer the
variables track if that column will be displayed or not.
create_row() populates the values for each row to be displayed in the UI
from the relevant Partition object. However load_partitions(5 params)
independently decided if the Name, Mount Point and Label columns were
empty and should be displayed.
Getting the mount point value is more complex for encrypted file systems
because it has to call get_mountpoints() on the inner encrypted
Partition object. load_partitions(5 params) didn't account for this.
Fix by making create_row() both copy the values into each row and at the
same time check if they are empty to decide if they should be displayed
or not.
Bug 775475 - Mount Point column displayed for encrypted file systems
even when empty
Running Device > Attempt Data Rescue... hangs the GParted UI for as long
as gpart takes to scan the whole disk device looking for file system
signatures.
Originally when gpart support was added by commit [1] a separate thread
was created to run gpart. Then most threading was removed by commit [2]
which left gpart running in the main thread blocking the UI.
[1] ef37bdb7de
Added support to lost data recovery using gpart
[2] 52a2a9b00a
Reduce threading (#685740)
guess_partition_table() hand codes using Glib to run the gpart command
asynchronously reading standard output, but it just doesn't run the Gtk
main loop to process events, hence the UI hangs. Instead just use
Utils::execute_command() which handles everything already. It runs the
commands asynchronously, reading output and if being run in the main
thread also calls the Gtk main loop to keep the UI responsive.
Bug 772123 - GParted is unresponsive while gpart is running
If the GParted main window is closed before the initial device load
completed gpartedbin never exits. The main window closes but the
process sits there idle forever. Subsequently running GParted reports
this error:
# gparted
The process gpartedbin is already running.
Only one gpartedbin process is permitted.
If the user is running GParted from a desktop menu they will never see
this error so they will never know why GParted won't start any more.
More technically, it is if the main window is closed before the
Win_GParted::on_show() callback completes.
I assume the Gtk main loop doesn't setup the normal quit handling until
the on_show() callback finishes drawing the main window for the first
time. Following this hint [1], move the initial device load from the
on_show() callback to immediately after it completes by using a run once
idle callback setup in on_show().
This looks exactly the same to the user except now gpartedbin exits when
the main window is closed during the initial device load. Note that
GParted finished the device load before exiting. This is exactly the
same as happens when exiting during subsequent device refreshes.
[1] How to know when a Gtk Window is fully shown?
http://stackoverflow.com/questions/14663212/how-to-know-when-a-gtk-window-is-fully-shown
"If you want to know when your main window appears the first time,
it is far easier (and saner) add a g_idle_add after your show_all
call."
Bug 771816 - GParted never exits if the main window is closed before the
initial device load completes
Again this is to handle a long list of mount points for a single
partition. To prevent the file system status value of "Mounted on [list
of 20 mount points]" causing the dialog to become wider than the screen,
line wrap the text. This instead makes the dialog taller, which already
automatically scrolls vertically as needed.
Bug 771693 - Mount Point column is wider than the screen on openSUSE
using btrfs root with lots of mounted subvolumes
openSUSE 42.1 with default btrfs root installation is heavily using
subvolumes. (Think of btrfs as a storage pool and subvolumes as
individual file systems within the pool for a rough approximation).
Thus the root partition is mounted on many mount points:
# df -k
Filesystem 1K-blocks Used Available Use% Mounted on
...
/dev/sda2 19445760 5820080 13157104 31% /
/dev/sda2 19445760 5820080 13157104 31% /var/lib/libvirt/images
/dev/sda2 19445760 5820080 13157104 31% /var/lib/mysql
/dev/sda2 19445760 5820080 13157104 31% /.snapshots
/dev/sda2 19445760 5820080 13157104 31% /home
/dev/sda2 19445760 5820080 13157104 31% /var/opt
/dev/sda2 19445760 5820080 13157104 31% /usr/local
/dev/sda2 19445760 5820080 13157104 31% /var/tmp
/dev/sda2 19445760 5820080 13157104 31% /var/lib/named
/dev/sda2 19445760 5820080 13157104 31% /var/lib/mariadb
/dev/sda2 19445760 5820080 13157104 31% /var/spool
/dev/sda2 19445760 5820080 13157104 31% /var/lib/pgsql
/dev/sda2 19445760 5820080 13157104 31% /var/lib/mailman
/dev/sda2 19445760 5820080 13157104 31% /srv
/dev/sda2 19445760 5820080 13157104 31% /opt
/dev/sda2 19445760 5820080 13157104 31% /var/crash
/dev/sda2 19445760 5820080 13157104 31% /tmp
/dev/sda2 19445760 5820080 13157104 31% /boot/grub2/i386-pc
/dev/sda2 19445760 5820080 13157104 31% /var/log
/dev/sda2 19445760 5820080 13157104 31% /boot/grub2/x86_64-efi
As the Mount Point column contains all 20 mount points it makes the
column wider than the screen, requiring lots of horizontal scrolling to
see the following columns. (Truncated to just 7 mounts in this
example here).
Partition | File System | Mount Point
/dev/sda1 * | # swap |
/dev/sda2 * | # btrfs | /, /.snapshots, /boot/grub2/i386-pc, /boot/grub2/x86_64-efi, /home, /opt, /srv
Fix by making the Mount Point column resizable and display truncated
text with ellipsis. The column now takes the initial and minimum width
from the width of the "Mount Point" column header text.
Partition | File System | Mount Point | Size
/dev/sda1 * | # swap | | 1.45 GiB
/dev/sda2 * | # btrfs | /, /.snapsho... | 18.54 GiB
Bug 771693 - Mount Point column is wider than the screen on openSUSE
using btrfs root with lots of mounted subvolumes
This commit stopped setting the text colours in the Partition, File
System and Mount Point columns to avoid hard coding text colours making
them impossible to read when using GNOME's High Contrast Inverse theme:
ff2a6c00dd
Changes post gparted-0.3.6 - code recreation from Source Forge
* src/TreeView_Detail.cc: Removed text_color hard coding
- Removed hard coding of Partition and Filesystem text_color
which was based on if partition was TYPE_UNALLOCATED.
- Removed hard coding of Mount text_color which was based
on if partition was busy. Lock symbol provides this indicator.
- Closes GParted bug #413810 - Don't hardcode text colour in
partition list
Now remove the remaining vestiges left behind. Remove the unused color
text and mount_text_color columns from the tree model. Also remove
setting of the column attributes which set the colour of the text in the
tree view from those unused columns in the tree model.
Unnecessary history. Added by:
b179990dc9
show greyed-out mountpoint of unmounted partitions in the treeview
as an improved way to identify partitions
Bug #333027 - Displaying unmounted partitions' default mount points
in grey
and by commit only in CVS history:
Bart Hakvoort <...> 2004-08-22 15:06:45
Made text in Partition column darkgrey for unallocated. this offers
more visual difference between partitions and unallocated space
Trying to create a new partition table on a device with active
partitions reports the number of active partitions in the error dialog.
However when there is a busy logical partition the number of reported
busy partitions will be one less than the number of partitions in the
main UI showing the busy symbol.
GParted considers extended partitions as busy when any of the logical
partitions it contains as busy. Display in the main UI reflects this.
Fix Win_GParted::active_partitions_on_device_count() to not exclude
extended partitions from the count.
Found that in some cases usage of active encrypted swap was not working,
but only for the first encrypted swap partition. This only failed on
the first Device Mapper device, dm-0:
# ls -l /dev/mapper/ /dev/dm-*
brw-rw---- 1 root disk 254, 0 Oct 4 20:58 /dev/dm-0
brw-rw---- 1 root disk 254, 1 Oct 4 20:58 /dev/dm-1
/dev/mapper/:
total 0
crw------- 1 root root 10,236 Oct 4 19:48 control
lrwxrwxrwx 1 root root 7 Oct 4 20:58 sdb1_crypt -> ../dm-0
lrwxrwxrwx 1 root root 7 Oct 4 20:58 sdb2_crypt -> ../dm-1
# cat /proc/swaps
Filename Type Size Used Priority
/dev/sda1 partition 1524732 92356 -1
/dev/dm-0 partition 1046524 0 -2
/dev/dm-1 partition 1046524 0 -3
Was failing because the minor number of dm-0 was 0, causing BlockSpecial
operator==() to fall back to name comparison rather than major, minor
number, and GParted name /dev/mapper/sdb1_crypt doesn't match /dev/dm-0.
Found on openSUSE and Ubuntu which don't use LVM by default and don't
already have dm-0 used as an LVM Logical Volume which GParted doesn't
support.
The LINUX ALLOCATED DEVICES document [1] says block special device 0, 0
(major, minor) is not used "reserved as null device number". (Not to
be confused with 1, 3 /dev/null the Null device). All other
non-negative pairs are valid block special device numbers. Therefore
update BlockSpecial operator==() accordingly; compare by major, minor
number when either is greater than 0 falling back to string compare
otherwise. This still fits in with the BlockSpecial() constructor using
major, minor numbers 0, 0 to represent plain files.
[1] LINUX ALLOCATED DEVICES
https://www.kernel.org/doc/Documentation/devices.txt
Bug 771670 - Usage of active encrypted swap is not shown
GParted does not show the usage of active encrypted swap partitions,
instead showing partition warning "Unable to read the contents of this
file system! ...". OS setup:
# ls -l /dev/mapper/sdb4_crypt /dev/dm-3
brw-rw----. 1 root disk 253, 3 Sep 14 07:26 /dev/dm-3
lrwxrwxrwx. 1 root root 7 Sep 14 07:26 /dev/mapper/sdb4_crypt -> ../dm-3
# mkswap -L encrypted_swap /dev/mapper/sdb4_crypt
# swapon /dev/mapper/sdb4_crypt
# cat /proc/swaps
Filename Type Size Used Priority
/dev/sda2 partition 2097148 237632 -1
/dev/dm-3 partition 1046524 0 -2
This is because the code was performing a string compare between the
canonical /dev/mapper/sdb4_crypt name GParted is using and the /dev/dm-3
name reported by the kernel via /proc/swaps. Fix by creating
BlockSpecial objects from the names and compare those so that comparison
is done correctly using major, minor numbers.
Bug 771670 - Usage of active encrypted swap is not shown
The comment became completely unnecessary with the transfer of
mount_info and fstab_info into separate Mount_Info module by commit:
63ec73dfda
Split mount_info and fstab_info maps into separate Mount_Info module
It was never necessary to clear one of the mappings at the end of the
device refresh because it was reloaded at the start of the next device
refresh anyway and it is only a small amount of memory.
Have an unmounted file system within an open encrypted mapping and an
entry in /etc/fstab for the file system like this:
# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
...
sdb 8:16 0 8G 0 disk
+-sdb1 8:17 0 1G 0 part
+-sdb1_crypt 253:0 0 1022M 0 crypt
# blkid | grep sdb1
/dev/sdb1: TYPE="crypto_LUKS" ...
/dev/mapper/sdb1_crypt: TYPE="ext4" ...
# ls -l /dev/mapper/sdb1_crypt /dev/dm-0
brw-rw----. 1 root disk 253, 0 Sep 12 19:09 /dev/dm-0
lrwxrwxrwx. 1 root root 7 Sep 12 19:09 /dev/mapper/sdb1_crypt -> ../dm-0
# grep sdb1 /etc/fstab
/dev/mapper/sdb1_crypt /mnt/1 ext4 defaults 0 0
The mount point will be shown twice for the partition:
/mnt/1, /mnt/1
This is because add_node_and_mountpoint() adds two entries for both the
symbolic and real block special names:
map["/dev/mapper/sdb1_crypt"] = ["/mnt/1"]
map["/dev/dm-0"] = ["/mnt/1"]
This was needed for the old code which used string compare to match
block devices so that the mount point could be looked up by either name.
However since bug 767842 introduced major, minor number comparison it
became unnecessary. As both names refer to the same device the mount
point gets added twice to the same entry. Hence display of the double
mount.
map[BlockSpecial{"/dev/mapper/sdb1_crypt", 253, 0}] =
["/mnt/1", "/mnt/1"]
It is always going to be the case that the symbolic link and real block
special names have the same major, minor numbers. That was the
requirement of the BlockSpecial class and the reason for using stat() to
lookup the numbers. Therefore adding entries for both names will always
add duplicate entries. Fix by stop using realpath() to lookup the real
name and adding the duplicate entry.
Introduced by:
a800ca8b68
Add BlockSpecial into mount_info and fstab_info (#767842)
Bug 771323 - GParted is showing duplicate mount points for unmounted
encrypted file systems
The GParted_Core::mount_info and GParted_Core::fstab_info maps and the
methods that manipulate them are self-contained. Therefore move them to
a separate Mount_Info module and reduce the size of the monster
GParted_Core slightly.
The FS_Info module has a pseudo multi-object interface and used the
constructor to load the cache. However all the data in the class is
static. An FS_Info object doesn't contain any member variables, yet was
needed just to call the member functions.
Make all the member functions static removing the need to use any
FS_Info objects and provide an explicit load_cache() method.
Following earlier commit "Pre-populate BlockSpecial cache while reading
/proc/partitions (#767842)" load_proc_partitions_info_cache() has
extracted just the name field from each line of /proc/partitions.
Therefore simplify the regular expressions matching each type of whole
disk device to just matching in the name field rather than matching in
the whole line.
The Proc_Partitions_Info has a pseudo multi-object interface and uses
the constructor to load the cache. However all the data in the class is
static. A Proc_Partitions_Info object doesn't contain any member
variables, yet was needed just to call the member functions.
Make all the member functions static removing the need to use any
Proc_Partitions_Info objects and provide and explicit load_cache()
method.
Vol_id has been retired and removed from all supported distributions.
See earlier commit "Remove use of retired vol_id from FS_Info module
(#767842)" for more details. Therefore remove it's use from GParted
entirely.
GParted is already reading /proc/partitions to get whole disk device
names. The file also contains the major, minor device number of every
partition. Use this information to pre-populate the cache in the
BlockSpecial class.
# cat /proc/partitions
major minor #blocks name
8 0 20971520 sda
8 1 512000 sda1
8 2 20458496 sda2
...
9 3 1047552 md3
259 2 262144 md3p1
259 3 262144 md3p2
...
253 0 18317312 dm-0
253 1 2097152 dm-1
253 2 8383872 dm-2
253 3 1048576 dm-3
Note that for Device-Mapper names (dm-*) the kernel is not using the
canonical user space names (mapper/*). There is no harm in
pre-populating the cache with these names and will help if tools report
them too. It is just that for DMRaid, LVM and LUKS, GParted uses the
canonical /dev/mapper/* names so will still have to call stat() once for
each such name.
For plain disks (sd*) and Linux Software RAID arrays (md*) the kernel
name is the common user space name too, therefore matches what GParted
uses and pre-populating does avoid calling stat() at all for these
names.
Bug 767842 - File system usage missing when tools report alternate block
device names
Creation of every BlockSpecial object used to result in a stat() OS
call. On one of my test VMs debugging with 4 disks and a few partitions
on each, GParted refresh generated 541 calls to stat() in the
BlockSpecial(name) constructor. However there were only 45 unique
names. So on average each name was stat()ed approximately 12 times.
Cache the major, minor number associated with each name after starting
with a cleared cache for each GParted refresh. This reduces these
direct calls to stat() to just the 45 unique names.
Bug 767842 - File system usage missing when tools report alternate block
device names
The FS_Info cache is loaded from "blkid" output and compares block
special names. Therefore switch to using BlockSpecial objects so that
comparisons are performed by the major, minor device number instead.
Bug 767842 - File system usage missing when tools report alternate block
device names
FS_Info module caches the output from blkid as a single string and uses
regular expressions to find the line matching the requested block
special file name. This is not compatible with using BlockSpecial
objects to represent block special files, and perform matching by major,
minor device number. Therefore parse the blkid output into a vector of
structures containing the needed fields, ready for switching to
BlockSpecial objects in the following patch.
Interface to the module remains unchanged.
Bug 767842 - File system usage missing when tools report alternate block
device names
Vol_id was removed from udev 142, released 2009-05-13, and udev switched
to using blkid instead [1]. All currently supported distributions use
later versions of udev (or systemd after the udev merge), except for
RedHat / CentOS 5 with udev 095. However RedHat / CentOS 5 does provide
blkid and vol_id is found in udev specific /lib/udev directory not on
the PATH. Therefore effectively vol_id is not available on any
supported distribution and blkid is always available. Therefore remove
use of vol_id from the FS_Info module. Less code to refactor and test
in following changes.
[1] delete vol_id and require util-linux-ng's blkid
http://git.kernel.org/cgit/linux/hotplug/udev.git/commit/?id=f07996885dab45102492d7f16e7e2997e264c725
Bug 767842 - File system usage missing when tools report alternate block
device names
The SWRaid_Info cache is loaded from "mdadm" command output and
/proc/mdstat file. It contains the member name which is used to access
the cache, therefore switch to using BlockSpecial objects so that
comparison is performed using the major, minor device number.
Bug 767842 - File system usage missing when tools report alternate block
device names
The LUKS_Info module cache is loaded from "dmsetup" command and compares
block special files, therefore switch to using BlockSpecial objects so
that comparisons are performed by major, minor device number.
Bug 767842 - File system usage missing when tools report alternate block
device names
Small optimisation which avoids constructing an extra BlockSpecial
object when determining if a btrfs member is mounted. Rather than
extracting the name from the BlockSpecial object in
btrfs::get_mount_device() and re-constructing another BlockSpecial
object from that name in GParted_Core::is_dev_mounted(), pass the
BlockSpecial object directly.
Bug 767842 - File system usage missing when tools report alternate block
device names
There are no known errors which affect the remaining caches in GParted.
However those caches which compare block special devices will be changed
to use BlockSpecial objects so comparison is by major, minor device
number rather than by name.
Change btrfs member cache loaded from "btrfs filesystem show" output to
use BlockSpecial objects.
Bug 767842 - File system usage missing when tools report alternate block
device names
On some distributions having btrfs on top of LUKS encrypted partitions,
adding a second device and removing the first device used to mount the
file system causes GParted to no longer be able to report the file
system as busy or the mount points themselves.
For example, on CentOS 7, create a single btrfs file system and mount
it. The provided /dev/mapper/sdb1_crypt name is reported, via
/proc/mounts, as the mounting device:
# cryptsetup luksFormat --force-password /dev/sdb1
# cryptsetup luksOpen /dev/sdb1 sdb1_crypt
# mkfs.btrfs -L encrypted-btrfs /dev/mapper/sdb1_crypt
# mount /dev/mapper/sdb1_crypt /mnt/1
# ls -l /dev/mapper
total 0
lrwxrwxrwx. 1 root root 7 Jul 2 14:15 centos-root -> ../dm-1
lrwxrwxrwx. 1 root root 7 Jul 2 14:15 centos-swap -> ../dm-0
crw-------. 1 root root 10, 236 Jul 2 14:15 control
lrwxrwxrwx. 1 root root 7 Jul 2 15:14 sdb1_crypt -> ../dm-2
# fgrep btrfs /proc/mounts
/dev/mapper/sdb1_crypt /mnt/1 btrfs rw,seclabel,relatime,space_cache 0 0
Add a second device to the btrfs file system:
# cryptsetup luksFormat --force-password /dev/sdb2
# cryptsetup luksOpen /dev/sdb2 sdb2_crypt
# btrfs device add /dev/mapper/sdb2_crypt /mnt/1
# ls -l /dev/mapper
...
lrwxrwxrwx. 1 root root 7 Jul 2 15:12 sdb2_crypt -> ../dm-3
# btrfs filesystem show /dev/mapper/sdb1_crypt
Label: 'encrypted-btrfs' uuid: 45d7b1ef-820c-4ef8-8abd-c70d928afb49
Total devices 2 FS bytes used 32.00KiB
devid 1 size 1022.00MiB used 12.00MiB path /dev/mapper/sdb1_crypt
devid 2 size 1022.00MiB used 0.00B path /dev/mapper/sdb2_crypt
Remove the first mounting device from the btrfs file system. Now the
non-canonical name /dev/dm-3 is reported, via /proc/mounts, as the
mounting device:
# btrfs device delete /dev/mapper/sdb1_crypt /mnt/1
# btrfs filesystem show /dev/mapper/sdb2_crypt
Label: 'encrypted-btrfs' uuid: 45d7b1ef-820c-4ef8-8abd-c70d928afb49
Total devices 1 FS bytes used 96.00KiB
devid 2 size 1022.00MiB used 144.00MiB path /dev/mapper/sdb2_crypt
# fgrep btrfs /proc/mounts
/dev/dm-3 /mnt/1 btrfs rw,seclabel,relatime,space_cache 0 0
# ls -l /dev/dm-3
brw-rw----. 1 root disk 253, 3 Jul 2 15:12 /dev/dm-3
GParted loads the mount_info mapping from /proc/mounts and with it the
/dev/dm-3 name. When GParted is determining if the encrypted btrfs file
system is mounted or getting the mount points it is using the
/dev/mapper/sdb2_crypt name. Therefore no information is found and the
file system is incorrectly reported as unmounted.
Fix by changing mount_info and fstab_info to use BlockSpecial objects
instead of strings so that matching is performed by major, minor device
numbers rather than by string compare. Note that as BlockSpecial
objects are used as the key of std::map [1] mappings operator<() [2]
needs to be provided to order the key values.
[1] std::map
http://www.cplusplus.com/reference/map/map/
[2] std::map::key_comp
http://www.cplusplus.com/reference/map/map/key_comp/
Bug 767842 - File system usage missing when tools report alternate block
device names
In some cases creating an LVM2 Physical Volume on top of a DMRaid array
reports no usage information and this partition warning:
Unable to read the contents of this file system!
Because of this some operations may be unavailable.
The cause might be a missing software package.
The following list of software packages is required for lvm2
pv file system support: lvm2.
For example on Ubuntu 14.04 LTS (with GParted built with
--enable-libparted-dmraid) create an LVM2 PV in a DMRaid array
partition. GParted uses this command:
# lvm pvcreate -M 2 /dev/mapper/isw_bacdehijbd_MyArray0p2
But LVM reports the PV having a different name:
# lvm pvs
PV VG Fmt Attr PSize PFree
/dev/disk/by-id/dm-name-isw_bacdehijbd_MyArray0p2 lvm2 a-- 1.00g 1.00g
This alternate name is loaded into the LVM2_PV_Info module cache. Hence
when GParted queries partition /dev/mapper/isw_bacdehijbd_MyArray0p2 it
has no PV information against that name and reports unknown usage.
However they are actually the same block special device; major 252,
minor 2:
# ls -l /dev/mapper/isw_bacdehijbd_MyArray0p2
brw-rw---- 1 root disk 252, 2 Jul 2 11:09 /dev/mapper/isw_bacdehijbd_MyArray0p2
# ls -l /dev/disk/by-id/dm-name-isw_bacdehijbd_MyArray0p2
lrwxrwxrwx 1 root root 10 Jul 2 11:09 /dev/disk/by-id/dm-name-isw_bacdehijbd_MyArray0p2 -> ../../dm-2
# ls -l /dev/dm-2
brw-rw---- 1 root disk 252, 2 Jul 2 11:09 /dev/dm-2
To determine if two names refer to the same block special device their
major, minor numbers need to be compared, instead of string comparing
their names.
Implement class BlockSpecial which encapsulates the name and major,
minor numbers for a block special device. Also performs comparison as
needed. See bug 767842 comments 4 and 5 for further investigation and
decision for choosing to implement a class.
Replace name strings in the LVM2_PV_Info module with BlockSpecial
objects performing correct block special device comparison.
Bug 767842 - File system usage missing when tools report alternate block
device names
Now Device and Partition objects only have a single path,
get_alternate_paths() is never called. Remove the method and population
of the private alternate_paths_cache member that went with it.
Bug 767842 - File system usage missing when tools report alternate block
device names
To reflect that there is now only a single path in the Partition object
now. Also get rid of the now unneeded optional clear_paths parameter
which was only relevant when there was a vector of paths.
Bug 767842 - File system usage missing when tools report alternate block
device names
Now that Partition objects only have a single path, rather than a list
of paths, stop performing unnecessary actions in calibrate_partitions()
which added alternate paths reported from libparted. Just left with
setting the partition path name correctly, when the path name doesn't
exist. Happens when the path is set to "Copy of /dev/SRC" when the
partition was newly created by a copy-paste into unallocated space
earlier in the sequence of operations now being applied.
Bug 767842 - File system usage missing when tools report alternate block
device names
Change from a vector of paths to a single path member in the Partition
object. Remove add_paths() and get_paths() methods. Keep add_path()
and get_path().
Bug 767842 - File system usage missing when tools report alternate block
device names
To reflect that there is only a single path in the Device object now.
Also get rid of the now unneeded optional parameter which was only
relevant when there was a vector of paths.
Bug 767842 - File system usage missing when tools report alternate block
device names
Background
GParted stored a list of paths for Device and Partition objects. It
sorted this list [1][2] and treated the first specially as that is what
get_path() returned and was used almost everywhere; with the file system
specific tools, looked up in various *_Info caches, etc.
[1] Device::add_path(), ::add_paths()
[2] Partition::add_path(), ::add_paths()
Mount point display [3] was the only bit of GParted which really worked
with the path list. Busy file system detection [4] just used the path
provided by libparted, or for LUKS /dev/mapper/* names. It checked that
single path against the mounted file systems found from /proc/mounts,
expanded with additional block device names when symlinks were
encountered.
[3] GParted_Core::set_mountpoints() -> set_mountpoints_helper()
[4] GParted_Core::set_device_partitions() -> is_busy()
GParted_Core::set_device_one_partition() -> is_busy()
GParted_Core::set_luks_partition() -> is_busy()
Having the first path, by sort order, treated specially by being used
everywhere and virtually ignoring the others was wrong, complicated to
remember and difficult code with. As all the additional paths were
virtually unused and made no difference, remove them. The "improved
detection of mountpoins, free space, etc.." benefit from commit [5]
doesn't seem to exist. Therefore simplify to a single path for Device
and Partition objects.
[5] commit 6d8b169e73
changed the way devices and partitions store their device paths.
Instead of holding a 'realpath' and a symbolic path we store paths
in a list. This allows for improved detection of mountpoins, free
space, etc..
This patch
Simplify the Device object from a vector of paths to a single path.
Remove add_paths() and get_paths() methods. Keep add_path() and
get_path() for now.
Bug 767842 - File system usage missing when tools report alternate block
device names
Recognise GRUB2 core.img boot code written to a partition without a file
system. Such setups are possible/likely with GPT partitioned disks as
there is a specific partition type reserved for it [1][2]:
21686148-6449-6E6F-744E-656564454649 (BIOS Boot partition)
[1] GUID Partition Table, Partition types
https://en.wikipedia.org/wiki/GUID_Partition_Table#Partition_type_GUIDs
[2] BIOS boot partition
https://en.wikipedia.org/wiki/BIOS_boot_partition
Bug 766989 - zfsonline support - need file system name support for ZFS
type codes
Composing these operations caused GParted to abort on an assert failure:
(1) Delete an existing partition,
(2) Create a new partition,
(3) Delete new partition.
This is the equivalent issue as fixed in the previous commit, except with
the delete operation rather than the check operation:
Prevent assert failure from OperationCheck::get_partition_new() (#767233)
# ./gpartedbin
======================
libparted : 2.4
======================
**
ERROR:OperationDelete.cc:41:virtual GParted::Partition& GParted::OperationDelete::get_partition_new(): assertion failed: (false)
Aborted (core dumped)
# gdb ./gpartedbin core.19232 --batch --quiet --ex backtrace -ex quit
[New Thread 19232]
[New Thread 19234]
[Thread debugging using libthread_db enabled]
Core was generated by `./gpartedbin'.
Program terminated with signal 6, Aborted.
#0 0x000000361f2325e5 in raise (sig=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
64 return INLINE_SYSCALL (tgkill, 3, pid, selftid, sig);
#0 0x000000361f2325e5 in raise (sig=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
#1 0x000000361f233dc5 in abort () at abort.c:92
#2 0x0000003620a67324 in g_assertion_message (domain=<value optimized out>, file=<value optimized out>, line=<value optimized out>, func=0x50fcc0 "virtual GParted::Partition& GParted::OperationDelete::get_partition_new()", message=0x1b55f60 "assertion failed: (false)") at gtestutils.c:1358
#3 0x0000003620a678f0 in g_assertion_message_expr (domain=0x0, file=0x50fa68 "OperationDelete.cc", line=41, func=0x50fcc0 "virtual GParted::Partition& GParted::OperationDelete::get_partition_new()", expr=<value optimized out>) at gtestutils.c:1369
#4 0x000000000049aa0d in GParted::OperationDelete::get_partition_new (this=0x1b321b0) at OperationDelete.cc:41
#5 0x00000000004c6700 in GParted::Win_GParted::activate_delete (this=0x7ffc91403670) at Win_GParted.cc:2068
...
As before the crash is happened in Win_GParted::activate_delete() as it
was going through the list of operations removing those which applied to
the never created partition. It came across the delete operation of an
existing partition and called get_partition_new(). As partition_new was
not set or used by the delete operation this asserted false and crashed
GParted.
Unlike the check operation case, the delete operation doesn't have a
partition afterwards. (As GParted represents unallocated space with
partition objects then the delete operation could be populated with a
new partition by constructing the correctly merged unallocated space
partition object, but that is what OperationDelete::apply_to_visual()
does and having a place holder doesn't seem like the right thing to do).
Instead exclude delete operations, on existing partitions, when looking
for operations applying to this not yet created partition as they are
mutually exclusive.
Bug 767233 - GParted core dump on assert failure in
OperationDelete::get_partition_new()
Composing these operations caused GParted to abort on an assert failure:
(1) Check an existing partition,
(2) Create a new partition,
(3) Delete new partition.
# ./gpartedbin
======================
libparted : 2.4
======================
**
ERROR:OperationCheck.cc:40:virtual GParted::Partition& GParted::OperationCheck::get_partition_new(): assertion failed: (false)
Aborted (core dumped)
# gdb ./gpartedbin core.8876 --batch --quiet --ex backtrace -ex quit
[New Thread 8876]
[New Thread 8879]
[Thread debugging using libthread_db enabled]
Core was generated by `./gpartedbin'.
Program terminated with signal 6, Aborted.
#0 0x000000361f2325e5 in raise (sig=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
64 return INLINE_SYSCALL (tgkill, 3, pid, selftid, sig);
#0 0x000000361f2325e5 in raise (sig=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
#1 0x000000361f233dc5 in abort () at abort.c:92
#2 0x0000003620a67324 in g_assertion_message (domain=<value optimized out>, file=<value optimized out>, line=<value optimized out>, func=0x50f400 "virtual GParted::Partition& GParted::OperationCheck::get_partition_new()", message=0x1d37a00 "assertion failed: (false)") at gtestutils.c:1358
#3 0x0000003620a678f0 in g_assertion_message_expr (domain=0x0, file=0x50f1a8 "OperationCheck.cc", line=40, func=0x50f400 "virtual GParted::Partition& GParted::OperationCheck::get_partition_new()", expr=<value optimized out>) at gtestutils.c:1369
#4 0x0000000000498e21 in GParted::OperationCheck::get_partition_new (this=0x1d1bb30) at OperationCheck.cc:40
#5 0x00000000004c66ec in GParted::Win_GParted::activate_delete (this=0x7fff031c3e30) at Win_GParted.cc:2068
...
When Win_GParted::activate_delete() was stepping through the operation
list removing operations (2 & 3 in the above recreation steps) which
related to the new partition never to be created it called
get_partition_new() on all operations in the list. This included
calling get_partition_new() on the check operation (1 in the above
recreation steps). As partition_new was not set or used by the check
operation get_partition_new() asserted false and crashed GParted.
Fix by populating the partition_new member in OperationCheck objects,
thus allowing get_partition_new() to be called on the object. As a
check operation doesn't change any partition boundaries or file system
attributes, just duplicate the new partition from the original
partition.
Bug 767233 - GParted core dump on assert failure in
OperationDelete::get_partition_new()
E2fsprogs 1.42 adds ext4 64bit feature [1] allowing volume sizes larger
than 16 TiB. However only enable large volumes from e2fsprogs 1.42.9
when a large number of 64bit bugs were fixed [2]. (Also RHEL / CentOS 7
use e2fsprogs 1.42.9 and always enable 64bit feature by default).
[1] Release notes, E2fsprogs 1.42 (November 29, 2011)
http://e2fsprogs.sourceforge.net/e2fsprogs-release.html#1.42
"This release of e2fsprogs has support for file systems > 16TB.
Online resize requires kernel support which will hopefully be in
Linux version 3.2. Offline support is not yet available for > 16TB
file systems, but will be coming".
[2] Release notes, E2fsprogs 1.42.9 (December 28, 2013)
http://e2fsprogs.sourceforge.net/e2fsprogs-release.html#1.42.9
"Fixed a large number of bugs in resize2fs, e2fsck, debugfs, and
libext2fs to correctly handle bigalloc and 64-bit file systems.
There were many corner cases that had not been noticed in previous
uses of these file systems, since they are not as common. Some of
the bugs could cause file system corruption or data loss, so users
of 64-bit or bigalloc file systems are strongly urged to upgrade to
e2fsprogs 1.42.9".
Bug 766910 - Multiple boot loaders don't work on 64bit EXT4 file systems
Calling libparted ped_geometry_new() creates a new PedGeometry object
from malloced memory, however the corresponding ped_geometry_destroy()
is never called to destroy the object and free the memory.
Perform a resize of a FAT file system when running GParted under
valgrind identifies several memory blocks leaked via ped_geometry_new()
from resize_move_filesystem_using_libparted(). One such example:
# valgrind --track-origins=yes --leak-check=full ./gpartedbin
...
==32069== 32 bytes in 1 blocks are definitely lost in loss record 5,419 of 11,542
==32069== at 0x4C29BFD: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==32069== by 0x8ECD8C5: ped_malloc (libparted.c:231)
==32069== by 0x8ED23C1: ped_geometry_new (geom.c:79)
==32069== by 0x4764F3: GParted::GParted_Core::resize_move_filesystem_using_libparted(GParted::Partition const&, GParted::Partition const&, GParted::OperationDetail&) (GParted_Core.cc:2666)
==32069== by 0x478007: GParted::GParted_Core::resize_filesystem(GParted::Partition const&, GParted::Partition const&, GParted::OperationDetail&, bool) (GParted_Core.cc:2990)
==32069== by 0x478440: GParted::GParted_Core::maximize_filesystem(GParted::Partition const&, GParted::OperationDetail&) (GParted_Core.cc:3037)
==32069== by 0x4769A0: GParted::GParted_Core::resize(GParted::Partition const&, GParted::Partition const&, GParted::OperationDetail&) (GParted_Core.cc:2746)
==32069== by 0x47582B: GParted::GParted_Core::resize_move(GParted::Partition const&, GParted::Partition&, GParted::OperationDetail&) (GParted_Core.cc:2457)
==32069== by 0x46DDB2: GParted::GParted_Core::apply_operation_to_disk(GParted::Operation*) (GParted_Core.cc:767)
...
There is also a leak of a PedGeometry object from
resize_move_partition(). Fix by calling ped_geometry_destroy() to
delete all the allocated PedGeometry objects and free the memory.
Bug 767009 - PedGeometry objects are memory leaked
When replacing the list of paths for the other partition object involved
in either a Resize/Move or Format operation in apply_operation_to_disk()
should replace the whole list of partitions not just replace with the
first path. Copy the whole path list is the correct action. It makes
no material difference because secondary partition paths are only used
to discover mount points during refresh phase and not at the apply
phase, where only the primary path is used.
Bug 766349 - Resolve code ugliness with partition path getting set to
"copy of /dev/SRC"
Quoting the relevant comments from GParted_Core::calibrate_partition():
Re-add the real partition path from libparted.
When creating a copy operation by pasting into unallocated space the
list of paths for the partition object was set to
["Copy of /dev/SRC"] because the partition didn't yet exist before
the operations were applied. Additional operations on that new
partition also got the list of paths set to ["Copy of /dev/SRC"].
This re-adds the real path to the start of the list making it
["/dev/NEW", "Copy of /dev/SRC"]. This provides the real path for
file system specific tools used during those additional operations
such mkfs for the format operation or fsck and others for the
resize/move operation.
FIXME: Having this work just because "/dev/NEW" happens to sort
before "Copy of /dev/SRC" is ugly! Probably have a separate display
path which can be changed at will without affecting the list of real
paths for the partition.
Having a separate display path is overly complicated and unnecessary.
Just replace the list of paths with the real one reported by libparted
if it contained "Copy of /dev/SRC", determined by checking if the file
exists. Otherwise continue to add the libparted name as an alternate
path. Whole disk devices can never be named "Copy of /dev/SRC" because
they are not partitioned so never created or deleted by GParted, only
ever written to, hence don't need the extra exists test logic.
Bug 766349 - Resolve code ugliness with partition path getting set to
"copy of /dev/SRC"
When composing a copy operation it always named the destination
partition as "copy of /dev/SRC". For the case of pasting into
unallocated space creating a new partition this was the right thing to
do as the partition doesn't yet exist so the path is not yet known.
However for the case of pasting into an existing partition the path is
known and replacing it with "copy of /dev/SRC" is wrong. No other
operation when operating on an existing partition changes it path.
Given a set of existing partitions, sdb1 to sdb4, compose a set of copy
operations as: copy sdb1 to sdb2, copy sdb2 to sdb3 and copy sdb3 to
sdb4. The displayed partitions before being applied become:
/dev/sdb1
copy of /dev/sdb1
copy of copy of /dev/sdb1
copy of copy of copy of /dev/sdb1
And the pending operations are named:
Copy /dev/sdb1 to /dev/sdb2
Copy copy of /dev/sdb1 to /dev/sdb3
Copy copy of copy of /dev/sdb1 to /sev/sdb4
This is perverse. In the case of pasting into an existing partition
keep the real path name. This keeps the partitions being displayed as:
/dev/sdb1
/dev/sdb2
/dev/sdb3
/dev/sdb4
And the pending operations named as:
Copy /dev/sdb1 to /dev/sdb2
Copy /dev/sdb2 to /dev/sdb3
Copy /dev/sdb3 to /dev/sdb4
Much more understandable.
Also switch to an upper case "C" in "Copy of /dev/SRC" as the temporary
path name when pasting into unallocated space. Finally update the
comment in calibrate_partition() to describe the remaining cases when
re-adding the path is still required.
Bug 766349 - Resolve code ugliness with partition path getting set to
"copy of /dev/SRC"
Make the code a little more self documenting by adding the symbolic
constants:
SETTLE_DEVICE_APPLY_MAX_WAIT_SECONDS
SETTLE_DEVICE_PROBE_MAX_WAIT_SECONDS
which highlight that settle_device() is called in two different
contexts, device probe and apply operations, with two different timeout
values.
File system specific commands sometimes fail reporting that the
partition specific /dev entry doesn't exist. Example failing check
operation details:
Check and repair file system (ext4) on dev/sdb4
calibrate /dev/sdb4
path: /dev/sdb4 (partition)
start: 4196352
end: 6293503
size: 2097152 (1.00 GiB)
check file system on /dev/sdb4 for errors and (if possible) fix them
e2fsck -f -y -bv -C 0 /dev/sdb4
e2fsck 1.42.9 (28-Dec-2013)
e2fsck: No such file or directory while trying to open /dev/sdb4
Possibly non-existent device?
This has been reproduced on CentOS 7. Debugging shows that the
libparted calls used to re-read the partition details in
GParted_Core::calibrate_partition() leads to udev removing and re-adding
all the partition /dev entries for the disk.
# udevadm monitor &
# gpartedbin
...
16.480662 +12.618659 calibrate_partition() calling get_device("/dev/sdb", lp_device) ...
16.483644 +0.002982 calibrate_partition() get_device() returned
16.483678 +0.000034 calibrate_partition() calling get_disk(lp_device, lp_disk) ...
16.618113 +0.134435 calibrate_partition() get_disk() returned
KERNEL[19275.707968] remove /devices/pci0000:00/0000:00:0d.0/ata4/host3/target3:0:0/3:0:0:0/block/sdb/sdb1 (block)
16.618561 +0.000448 destroy_device_and_disk() calling ped_disk_destroy(lp_disk) ...
16.618584 +0.000023 destroy_device_and_disk() ped_disk_destroy() returned
16.618591 +0.000007 destroy_device_and_disk() calling ped_device_destroy(lp_disk) ...
16.618602 +0.000011 destroy_device_and_disk() ped_device_destroy() returned
16.618687 +0.000085 calibrate_partition() return true
16.618851 +0.000164 execute_command() e2fsck -f -y -v -C 0 /dev/sdb4
KERNEL[19275.708389] remove /devices/pci0000:00/0000:00:0d.0/ata4/host3/target3:0:0/3:0:0:0/block/sdb/sdb2 (block)
KERNEL[19275.708500] remove /devices/pci0000:00/0000:00:0d.0/ata4/host3/target3:0:0/3:0:0:0/block/sdb/sdb3 (block)
KERNEL[19275.708643] remove /devices/pci0000:00/0000:00:0d.0/ata4/host3/target3:0:0/3:0:0:0/block/sdb/sdb4 (block)
KERNEL[19275.768278] change /devices/pci0000:00/0000:00:0d.0/ata4/host3/target3:0:0/3:0:0:0/block/sdb (block)
KERNEL[19275.771171] add /devices/pci0000:00/0000:00:0d.0/ata4/host3/target3:0:0/3:0:0:0/block/sdb/sdb1 (block)
KERNEL[19275.771360] add /devices/pci0000:00/0000:00:0d.0/ata4/host3/target3:0:0/3:0:0:0/block/sdb/sdb2 (block)
KERNEL[19275.771542] add /devices/pci0000:00/0000:00:0d.0/ata4/host3/target3:0:0/3:0:0:0/block/sdb/sdb3 (block)
KERNEL[19275.775858] add /devices/pci0000:00/0000:00:0d.0/ata4/host3/target3:0:0/3:0:0:0/block/sdb/sdb4 (block)
UDEV [19275.820153] remove /devices/pci0000:00/0000:00:0d.0/ata4/host3/target3:0:0/3:0:0:0/block/sdb/sdb3 (block)
UDEV [19275.823152] remove /devices/pci0000:00/0000:00:0d.0/ata4/host3/target3:0:0/3:0:0:0/block/sdb/sdb4 (block)
UDEV [19275.828275] remove /devices/pci0000:00/0000:00:0d.0/ata4/host3/target3:0:0/3:0:0:0/block/sdb/sdb1 (block)
16.742735 +0.123884 execute_command() exit status 8
UDEV [19275.841425] remove /devices/pci0000:00/0000:00:0d.0/ata4/host3/target3:0:0/3:0:0:0/block/sdb/sdb2 (block)
UDEV [19275.905478] change /devices/pci0000:00/0000:00:0d.0/ata4/host3/target3:0:0/3:0:0:0/block/sdb (block)
UDEV [19276.013580] add /devices/pci0000:00/0000:00:0d.0/ata4/host3/target3:0:0/3:0:0:0/block/sdb/sdb3 (block)
UDEV [19276.034728] add /devices/pci0000:00/0000:00:0d.0/ata4/host3/target3:0:0/3:0:0:0/block/sdb/sdb4 (block)
UDEV [19276.174840] add /devices/pci0000:00/0000:00:0d.0/ata4/host3/target3:0:0/3:0:0:0/block/sdb/sdb1 (block)
UDEV [19276.237105] add /devices/pci0000:00/0000:00:0d.0/ata4/host3/target3:0:0/3:0:0:0/block/sdb/sdb2 (block)
So exactly when GParted is running the external e2fsck command, udev is
in the middle of removing and re-adding all the /dev partition entries
for the disk. Hence the above failure reporting that /dev/sdb4 didn't
exist. This error depends on the timing between GParted running the
external file system specific command and udev removing and re-adding
the entries, so sometimes it works and sometimes it fails.
Further debugging showed that simply opening and closing the whole disk
device read-write triggers the same removing and re-adding of all the
partition /dev entries with udev >= 219. Opening the whole disk device
read-write is what libparted has always done until this post
libparted 3.2 patch to make it open read-only when probing:
http://git.savannah.gnu.org/cgit/parted.git/commit/?id=44d5ae0115c4ecfe3158748309e9912c5aede92d
libparted: Use read only when probing devices on linux (#1245144)
To fix this simply wait for udev devices to settle in
calibrate_partitions(). The longest I have seen udev take to do this is
0.80 seconds in a VM. Wait up to 10 seconds as is done in commit() ->
commit_to_os(), also called when applying operations.
On configurations which don't have this issue execution of udevadm
settle, which will return immediately, adds at most 0.1 seconds to the
time taken for the calibrate step. This won't be noticed in the time
taken of the operation details so there is no point in trying to avoid
executing udevadm settle when not needed.
Bug 762941 - Operations sometimes failing with: No such file or
directory
Minor issues:
1) In the while loop reading from /proc/partitions into variable line,
just after the sscanf() call the variable was re-purposed to hold the
device name making the code unnecessarily hard to follow.
2) Variable c_str was a fixed sized buffer holding the device name read
from /proc/partitions.
3) Variable c_str name provides no meaning as to what data it holds.
4) Return value from all the Utils::regexp_label() calls is converted
from Glib::ustring to std::string to be stored in device variable.
Resolve by using Utils::regexp_label() to extract the device name from
each line in /proc/partitions and store in the variable device, already
used for this purpose and now changed to type Glib::ustring.
realpath(3) manual page says:
BUGS
The POSIX.1-2001 standard version of this function is broken by
design, since it is impossible to determine a suitable size for
the output buffer, resolved_path. According to POSIX.1-2001 a
buffer of size PATH_MAX suffices, but PATH_MAX need not be a
defined constant, and may have to be obtained using pathconf(3).
And asking pathconf(3) does not really help, since, on the one
hand POSIX warns that the result of pathconf(3) may be huge and
unsuitable for mallocing memory, and on the other hand
pathconf(3) may return -1 to signify that PATH_MAX is not
bounded. The resolved_path == NULL feature, not standardized in
POSIX.1-2001, but standardized in POSIX.1-2008, allows this
design problem to be avoided.
The resolved_path == NULL feature of realpath() has existed as a Glibc
extension since realpath() was first added to Glibc 1.90, released in
June 1996. Therefore it can be used unconditionally.
https://sourceware.org/git/?p=glibc.git;a=commitdiff;h=fa0bc87c32d02cd81ec4d0ae00e0d943c683e6e1
Bug 764369 - Use realpath() safely
The previous commit (Fix crash reading NTFS usage when there is no
/dev/PTN entry) identified that the FileSystem member variable "index"
is too small on 64-bit machines. Also this member variable stores no
FileSystem class information and was being used as a local variable.
Replace with local variables of the of the correct type, wide enough to
store the npos not found value.
Bug 764658 - GParted crashes when reading NTFS usage when there is no
/dev/PTN entry
On a 64-bit distribution, with an NTFS file system in a partition
without a /dev entry then GParted will crash when attempting to read
the file system usage. Not having a /dev entry for the partition is
rare and only known to occur for the disk devices used within Fake RAID
(dmraid) arrays, and then only on Ubuntu 12.04 LTS. Other/newer
distributions do create /dev entries for partitions found on disk
devices within Fake RAID arrays.
Create mirror Fake RAID array:
# dmraid -f isw -C MyArray --type 1 --disk /dev/sdc,/dev/sdd
# dmraid -ay
Create NTFS partition on the Fake RAID array. On refresh GParted
crashes:
# ./gpartedbin
(gpartedbin:590): glibmm-ERROR **:
unhandled exception (type std::exception) in signal handler:
what: basic_string::assign
Without a /dev/sdc1 device entry the ntfsresize command reports this:
# ntfsresize --info --force --no-progress-bar /dev/sdc1
ntfsresize v2015.3.14 (libntfs-3g)
ERROR(2): Failed to check '/dev/sdc1' mount state: No such file or directory
Probably /etc/mtab is missing. It's too risky to continue. You might try
an another Linux distro.
The problem code in ntfs::set_used_sectors():
145 index = output.find( "Cluster size" );
146 if ( index == output.npos ||
147 sscanf( output.substr( index ).c_str(), "Cluster size : %Ld", &S ) != 1 )
As "Cluster size" did not exist in the output find() returned the not
found token of string::npos [1], which in a 64-bit environment is
represented by 2^64-1 [2]. However it was saved in the variable index
of type unsigned integer, which is only a 32-bit integer, thus
truncating it to 2^32-1. Therefore the comparison failed and sscanf()
tried to parse the output starting at offset 2^32-1 which resulted in
the crash.
Introduced by commit:
324d99a172
Record file system block size where known (#760709)
Fix by following the same pattern of the other comparisons in
ntfs::set_used_sectors() which checks if index is less than the output
length.
References:
[1] std::string::find
http://www.cplusplus.com/reference/string/string/find/
[2] std::string::npos
http://www.cplusplus.com/reference/string/string/npos/
(Note that Glib::ustring is derived from std::string in the Standard C++
library and provides a compatible interface).
Bug 764658 - GParted crashes when reading NTFS usage when there is no
/dev/PTN entry
GParted is allowing creation of a FAT32 formatted partition of any size.
However with a 512 byte sector size the maximum volume size of a FAT32
file system is reported to be 2 TiB.
* Wikipedia: File Allocation Table / FAT32
https://en.wikipedia.org/wiki/File_Allocation_Table#FAT32
"The boot sector uses a 32-bit field for the sector count, limiting
the FAT32 volume size to 2 TB for a sector size of 512 bytes and 16 TB
for a sector size of 4,096 bytes."
* Microsoft: Default cluster size for NTFS, FAT, and exFAT / Default
cluster sizes for FAT32
https://support.microsoft.com/en-us/kb/140365
Trying to create a FAT32 file system in a partition larger than 2 TiB
results in unallocated space being left after the file system.
Nuances:
[1] Larger sector sizes allow larger maximum volume sizes up to 16 TiB
with 4096 byte sectors.
[2] mkdosfs/mkfs.fat has an -S SECTOR_SIZE option which allows changing
the "logical" sector size of the file system allowing the maximum
volume to be proportionally increased.
[3] mkfs.fat appears to have an signed overflow bug when the size of the
partition is larger than maximum signed 32-bit integer of logical(?)
sectors. (2 TiB for a sector size of 512 bytes). It reports the
partition size as minus size and creates a 1 TiB file system.
GParted wants a single maximum file system size and the code is not
ready for a differing maximum file system size for different sector
sizes.
In fat16::create() could specify larger "logical" sector sizes to
mkfs.fat when the partition is larger than 2 TiB to allow maximum volume
size to be increased further. However that will take a lot of cross
platform testing to ensure that all sorts of devices support "logical"
sector sizes other than 512 bytes on devices with a hardware sector size
of 512 bytes. This is too much effort.
Therefore implement a single FAT32 maximum volume size of 2 TiB.
Bug 763896 - GParted not restricting creation of FAT32 volume to maximum
size of 2 TiB
As part of the internal block copy operation 5 initial ranges of blocks
are copied using different block sizes to determine the fastest. Then
the remainder is copied using the fastest block size. Each of these
copies reports progress independently, so during the benchmarking phase
the progress bar flashes 5 times as it goes from 0 to 100% in a fraction
of a second, before showing the progress of the remainder.
This looks bad, so report a single progress bar for all the ranges of
blocks copied in a single copy operation.
Already have variables done and length which track progress within each
copied range; and total_done which records amount copied in previous
ranges. Just add total_length to allow overall progress to be reported.
Bug 762367 - Use a single progress bar for the whole of the internal
copy operation
Previously total_done was updated in copy_thread() after copying of the
blocks, but importantly before the last call to set_progress_info() to
update the progress bar. Having total_done varying during the copy of a
single range of blocks, single call to copy_blocks::copy(), is an
impediment to being able to report a single progress bar across the
whole internal copy operation.
Move updating of total_done to copy() immediately after copy_thread()
completes.
Bug 762367 - Use a single progress bar for the whole of the internal
copy operation
Copying of ntfs is performed using ntfsclone, which writes progress
indication to standard output like this:
# ntfsclone -f /dev/sdb2 /dev/sdb1 2> /dev/null
NTFS volume version: 3.1
Cluster size : 4096 bytes
Current volume size: 21474832384 bytes (21475 MB)
Current device size: 21474836480 bytes (21475 MB)
Scanning volume ...
100.00 percent completed
Accounting clusters ...
Space in use : 1832 MB (8.5%)
Cloning NTFS ...
100.00 percent completed
Syncing ...
Add ntfsclone progress tracker for ntfsclone command. Deliberately
doesn't stop the progress bar. See comment in ntfs::clone_progress()
for the explanation.
Bug 762366 - Add progress bar to NTFS file system specific copy method
Mostly the code is explicit and calls the emit() method when emitting a
signal [1], like this:
signal_name.emit();
However there are a few cases which use the function call operator on
the signal object [2], like this:
signal_name();
The behaviour is identical [3] but it is preferred to be explicit that a
signal callback is being initiated, and it also makes them much easier
to search for too.
[1] List explicit emit() signal calls
fgrep '.emit(' src/*.cc
[2] List function call operator emitted signals
egrep "`sed -n '/sigc::signal/s/.*sigc::signal.*> *\([a-zA-Z_]*\).*/\1/p' include/*.h | tr '\n' '|' | sed 's/\(.*\).$/[^a-zA-Z_](\1)\\\(/'`" src/*.cc
[3] Quote from the libsigc++ Reference Manual, class sigc::signal
https://developer.gnome.org/libsigc++/stable/classsigc_1_1signal7.html#ab37db0ecc788824d0baa3c301efc8dcd
result_type sigc::signal<...>::operator()(...)
Triggers the emission of the signal (see emit())
For non-progress tracked external commands the command being executed is
displayed in the Apply pending operations dialog, just below the top
pulsing progress bar. However for progress tracked external commands
the description of the parent operation detail is displayed instead.
Example 1: non-progress tracked xfs check repair:
TreePath
Check and repair file system (xfs) on /dev/sdc1 0
+ calibrate /dev/sdc1 0:0
+ check file system on /dev/sdc1 for errors and (if po... 0:1
+ xfs_repair -v /dev/sdc1 0:1:0
+ [empty stdout] 0:1:0:0
+ [stderr]Phase 1 - find and verify superblock... 0:1:0:1
"xfs_repair -v /dev/sdc1" (TreePath 0:1:0) is shown because it is the
latest updated operation detail which is timed (set to status
executing).
Example 2: progress tracked ext4 copy using e2image:
TreePath
Copy /dev/sdc2 to /dev/sdc3 0
+ calibrate /dev/sdc2 0:0
+ check file system on /dev/sdc2 for errors and (if po... 0:1
+ set partition type on /dev/sdc3 0:2
+ copy file system of /dev/sdc2 to /dev/sdc3 0:3
+ e2image -ra -p /dev/sdc2 /dev/sdc3 0:3:0
+ [stdout]Scanning inodes... 0:3:0:0
+ [stderr]e2image 1.42.9 (28-Dec-2013)... 0:3:0:1
"copy file system of /dev/sdc2 to /dev/sdc3" (TreePath 0:3) is shown
because that operation detail is also timed and it is being constantly
updated by the progress bar updates via it.
Change execute_command() to update the progress bar via the operation
detail it creates to hold the command being executed, instead of the
parent operation detail, to resolve the above. Also replaces calling
operationdetail.get_last_child() throughout the method.
Bug 760709 - Add progress bars to XFS and EXT2/3/4 file system specific
copy methods
Remove starting and stopping the progress bar in xfs::copy(). The
progress bar will be automatically started in xfs::copy_progress()
callback when run_progressbar() is called; and automatically stopped in
FileSystem::execute_command() when it calls stop_progress() at the end.
Note that this will now not initialise the progress bar from zero
immediately that the XFS copy is started, but instead 0.5 seconds into
the copy when xfs::copy_progress() timed callback is called for the
first time.
Bug 760709 - Add progress bars to XFS and EXT2/3/4 file system specific
copy methods
The timed progress tracking callback for execution of xfs copy follows
this pattern:
sigc::connection c;
...
c = Glib::signal_timeout().connect( ... sigc::mem_fun( *this, &xfs::copy_progress ) ..., 500 /*ms*/ );
... execute_command( ... );
c.disconnect();
As with output progress tracking callbacks for ext2/3/4 and ntfs file
system specific commands, pass the callback slot and a flag into
execute_command() and connect the timed callback inside. This
simplified the pattern to:
... execute_command( ...|EXEC_PROGRESS_TIMED,
static_cast<TimedSlot>( sigc::mem_fun( *this, &xfs::copy_progress ) ) );
NOTE:
The type of sigc::mem_fun() doesn't allow the compiler to choose between
the two overloaded variants of execute_command() with the fourth
parameter of either (full types without typedefs of StreamSlot and
TimedSlot respectively):
sigc::slot<void, OperationDetail *> stream_progress_slot
sigc::slot<bool, OperationDetail *> timed_progress_slot
Therefore have to cast the result of all callback slots to the relevant
type. Hence:
static_cast<StreamSlot>( sigc::mem_fun( *this, &{CLASS}::{NAME}_progress ) )
static_cast<TimedSlot>( sigc::mem_fun( *this, &xfs::copy_progress ) )
References:
* [sigc] Functor not resolving between overloaded methods with
different slot types
https://mail.gnome.org/archives/libsigc-list/2016-February/msg00000.html
* Bug 306705 - Can't overload methods based on different slot<>
parameters.
https://bugzilla.gnome.org/show_bug.cgi?id=306705
Bug 760709 - Add progress bars to XFS and EXT2/3/4 file system specific
copy methods
For the relevant stream from a file system specific command being
tracked, there were 2 callbacks attached: update_command_output() and
update_command_progress(). When called, update_command_progress() just
emitted signal_progress to call the file system specific progress
tracker callback. Like this:
signal_update.emit() -> update_command_output()
-> update_command_progress()
signal_progress.emit() -> {CLASS}::{NAME}_progress()
Instead just connect the file system specific progress tracker callback
directly to signal_update and bypass the unnecessary
update_command_progress() method and the signal_progress signal. Like
this:
signal_update.emit() -> update_command_output()
-> {CLASS}::{NAME}_progress()
Bug 760709 - Add progress bars to XFS and EXT2/3/4 file system specific
copy methods
All the output progress tracking callbacks for execution of ext2/3/4 and
ntfs file system specific commands followed this pattern:
sigc::connection c = signal_progress.connect( sigc::mem_fun( *this, &ext2::..._progress ) );
bool success = ! execute_command( ... );
c.disconnect();
return success;
Instead, pass the callback slot and a flag into execute_command() and
connect the callback inside. This simplifies the pattern to:
return ! execute_command( ...|EXEC_PROGRESS_STDOUT,
sigc::mem_fun( *this, &ext2::..._progress ) );
Note that as the progress tracking callbacks are only registered against
updates to the relevant stream from the tracked commands they won't be
called when the other stream is updated any more.
Also note that signal_progress is a member of the FileSystem class and
derived objects so lives as long as GParted is running, therefore the
progress tracking callbacks need explicitly disconnecting at the end of
execute_command(). However signal_update is a member of the PipeCapture
class of which the output and error local variables in execute_command()
are types. Therefore there is no need to explicitly disconnect the
signal_update callbacks as they will be destructed along with the
callback slots when they go out of scope at the end of the
execute_command() method.
Bug 760709 - Add progress bars to XFS and EXT2/3/4 file system specific
copy methods
Remove unused members: fraction and progress_text from the
OperationDetail class now that the ProgressBar class has superseded
their use. This also allows removal of timer_global member from the
copy_blocks class. Timer_global was only used to track the elapsed time
copying blocks and allow the remaining time to be estimated and written
into progress_text. The ProgressBar class also does this itself
internally.
Bug 760709 - Add progress bars to XFS and EXT2/3/4 file system specific
copy methods
Most of the file system specific command progress trackers followed this
pattern:
void {CLASS}::{NAME}_progress( OperationDetail *operationdetail )
{
ProgressBar & progressbar = operationdetail->get_progressbar();
// parse output for progress and target values
if ( // have progress and target values )
{
if ( ! progressbar.running() )
progressbar.start( target );
progressbar.update( progress );
operationdetail->signal_update( *operationdetail );
}
else if ( // found progress finished )
{
if ( progressbar.running() )
progressbar.stop();
operationdetail->signal_update( *operationdetail );
}
}
That is a lot of repetition handling progress bar updates and
OperationDetail object update signalling. Remove the need for direct
access to the single ProgressBar object and provide these two
OperationDetail methods instead:
// Start and update in one
run_progressbar( progress, target, optional text_mode );
stop_progressbar();
Now the file system specific command progress trackers can become:
void {CLASS}::{NAME}_progress( OperationDetail *operationdetail )
{
// parse output for progress and target values
if ( // have progress and target values )
{
operationdetail->run_progressbar( progress, target );
}
else if ( // found progress finished )
{
operationdetail->stop_progressbar();
}
}
Make ProgressBar::get_progressbar() a private method to enforce use of
the new way to access the progress bar via the run_progress() and
stop_progressbar() methods. Then make the Dialog_Progress a friend
class to OperationDetail so that the Apply pending operations dialog can
still access the single ProgressBar object for its querying needs.
Bug 760709 - Add progress bars to XFS and EXT2/3/4 file system specific
copy methods
Using e2image to copy a file system looks like this. (Intermediate
progress lines which are constantly overwritten are indicated with ">").
# e2image -ra -p /dev/sdb4 /dev/sdb5
e2image 1.42.13 (17-May-2015)
Scanning inodes...
> Copying 0 / 276510 blocks (0%)
> Copying 8845 / 276510 blocks (3%)
> Copying 48433 / 276510 blocks (18%)
> Copying 77135 / 276510 blocks (28%)
> Copying 111311 / 276510 blocks (40%)
> Copying 137039 / 276510 blocks (50%)
> Copying 166189 / 276510 blocks (60%) 00:00:03 remaining at 108.20 MB/s
> Copying 190285 / 276510 blocks (69%) 00:00:03 remaining at 106.19 MB/s
> Copying 209675 / 276510 blocks (76%) 00:00:02 remaining at 102.38 MB/s
> Copying 238219 / 276510 blocks (86%) 00:00:01 remaining at 103.39 MB/s
> Copying 256692 / 276510 blocks (93%) 00:00:00 remaining at 100.27 MB/s
Copied 276510 / 276510 blocks (100%) in 00:00:10 at 108.01 MB/s
Note that the copying figures are reported in file system block size
units and the progress information is written to stderr, hence needing
these two previous commits:
Record file system block size where known (#760709)
Call any FS specific progress trackers for stderr updates too (#760709)
Add progress tracking function for e2image command. Also tracks when
the text progress indicator has passed in the output so that the
progress bar can be stopped as well as started when needed.
Bug 760709 - Add progress bars to XFS and EXT2/3/4 file system specific
copy methods
So far the signal_progress callback slot was only emitted when standard
output from the file system specific command was updated. This was okay
as all the commands until now wrote their progress information to
stdout. However e2image writes its progress information to stderr,
therefore also emit signal_progress when stderr is updated too.
This does mean that the file system specific *_progress() tracking
callbacks will be called when either of the OperationDetail objects
containing stdout or stderr are updated. Therefore the trackers may be
called when there is no update to the stream from which it is parsing
the progress information. This is not a problem as the tracker will
just update the progress bar with the same information it already has.
Also it won't happen much as only e2image is known to write to both
streams, and then only one line to stdout and the updated progress
information to stderr. This is just an observation and not an issue
which needs coding around.
Bug 760709 - Add progress bars to XFS and EXT2/3/4 file system specific
copy methods
Record the file system block size in the Partition object. Only
implemented for file systems when set_used_sectors() method has already
parsed the value or can easily parse the value from the existing
executed command(s).
Needed for ext2/3/4 copies and moves performed using e2image so that
they can be tracked in bytes by the ProgressBar class as e2image reports
progress in file system block size units.
Bug 760709 - Add progress bars to XFS and EXT2/3/4 file system specific
copy methods
XFS uses a file system specific method to copy the partition using
"xfsdump | xfsrestore". Monitor progress by periodically querying the
destination file system usage and comparing to the source file system
usage. Use 0.5 seconds as the polling interval to match that used by
the internal block copying algorithm.
NOTE:
The number of used blocks in the source and destination file system will
be very close but may not match exactly. I have seen an XFS copy finish
with the following progress text:
1.54 GiB of 1.50 GiB copied (-00:00:02 remaining)
Allow the progress bar to overrun like this as it is informing the user
that it actually copied a little more data and took a little longer than
expected. Needs these two previous commits to correctly round and
format the negative time remaining:
Fix rounding of negative numbers (#760709)
Fix formatting of negative time values (#760709)
Bug 760709 - Add progress bars to XFS and EXT2/3/4 file system specific
copy methods
... to display a negative sign before the hours, minutes and seconds.
Before:
Utils::format_time(-1) = "00:00:0-1"
Utils::format_time(-119) = "00:0-1:0-59"
After:
Utils::format_time(-1) = "-00:00:01"
Utils::format_time(-119) = "-00:01:59"
Bug 760709 - Add progress bars to XFS and EXT2/3/4 file system specific
copy methods
Utils::round() was doing +0.5 then truncate. Correct for positive
values. Wrong for negative values.
E.G.
Utils::round(-1.4)
= trunc(-1.4 + 0.5)
= trunc(-0.9)
= 0
Round of -1.4 is definitely not 0. Fix this for negative values by
subtracting 0.5 then truncating.
Reference:
How can I convert a floating-point value to an integer in C?
https://www.cs.tut.fi/~jkorpela/round.html
Bug 760709 - Add progress bars to XFS and EXT2/3/4 file system specific
copy methods
When the ntfs resize operation had almost completed, percentage complete
was >= 99.9%, the progress tracker was passing 0.04 (4%) to the progress
bar. After reading the next chunk of output from the ntfsresize command
the last line contained this text:
" 4) set the bootable flag for the partit"
End of the ntfsresize command output for context:
Relocating needed data ...
100.00 percent completed
Updating $BadClust file ...
Updating $Bitmap file ...
Updating Boot record ...
Syncing device ...
Successfully resized NTFS on device '/dev/sdd4'.
You can go on to shrink the device for example with Linux fdisk.
IMPORTANT: When recreating the partition, make sure that you
1) create it at the same disk sector (use sector as the unit!)
2) create it with the same partition type (usually 7, HPFS/NTFS)
3) do not make it smaller than the new NTFS filesystem size
4) set the bootable flag for the partition if it existed before
Otherwise you won't be able to access NTFS or can't boot from the disk!
If you make a mistake and don't have a partition table backup then you
can recover the partition table by TestDisk or Parted's rescue mode.
This was occurring because *scanf() can't actually report failure to
match fixed text after conversion of the last variable. See code
comment in ntfs::resize_progress() for more details. Fix by using
.find() instead to match the required "percent completed" explicit text
of the progress information when it appears on the last line.
Bug 760709 - Add progress bars to XFS and EXT2/3/4 file system specific
copy methods
Adapt the ntfs resize progress tracker to use the new ProgressBar class.
Also make it track when the text progress indicator has passed in the
output so that the progress bar can be stopped as well as started when
needed.
Bug 760709 - Add progress bars to XFS and EXT2/3/4 file system specific
copy methods
Adapt the ext2 fsck progress tracker to use the new ProgressBar class.
Also make it track when the text progress bar has completely passed in
the output so that the progress bar can be stopped as well as started
when needed.
Bug 760709 - Add progress bars to XFS and EXT2/3/4 file system specific
copy methods
Adapt the ext2 create file system progress tracker to used the new
ProgressBar class. Also make it track when the text progress indicator
completes so that the progress bar can be stopped as well as started
when needed.
Bug 760709 - Add progress bars to XFS and EXT2/3/4 file system specific
copy methods
Adapt the ext2 resize progress tracker to the new ProgressBar class.
Also update the progress function to track when text progress bars have
completely passed in the output so that the progress bar can be stopped
as well as started when needed.
Bug 760709 - Add progress bars to XFS and EXT2/3/4 file system specific
copy methods
Change the Applying pending operations dialog so that it takes it source
of progress from the single ProgressBar object, rather than the fraction
value in every OperationDetail object. Also remove ProgressBar
debugging now that it is being used to drive the UI.
NOTE:
This temporarily causes the existing file system specific progress bars
to not be shown because they still update via the fraction member in
each OperationDetail object, rather than the new ProgressBar. This will
be corrected in following commits.
Bug 760709 - Add progress bars to XFS and EXT2/3/4 file system specific
copy methods
1) Multiple progress bars
The OperationDetail class contains member fraction which is used to feed
data to the current operation progress bar shown in the Applying pending
operations dialog. Dialog_Progress::on_signal_update() gets called for
every updated OperationDetail object and depending on whether fraction
is > 0.0 or not, switches between showing a growing or pulsing progress
bar. This leads to the conclusion that every OperationDetail object
currently being updated is effectively driving the single on screen
progress bar with different data.
The Copy_Blocks code is careful to update text and faction in a single
OperationDetail object and everything is good. The on screen progress
bar is switched into growing mode and then grows to 100%.
Since external command output is updated in real time [1] there are two
OperationDetail objects, one for stdout and one for stderr, which are
updated whenever data is read from the relevant stream. Also now that
progress is interpreted from some external command output [2][3][4] a
separate OperationDetail object is getting updated with the progress
fraction. (Actually the grandparent OperationDetail of the ones
receiving stdout and stderr updates as used by the file system specific
*_progress() methods). In the normal case of an external command
which is reporting it's progress two OperationDetails are constantly
being updated together, the OperationDetail object tracking stdout and
it's grandparent receiving progress fraction updates. This causes the
the code in Dialog_Progress::on_signal_update() to constantly switch
between growing and pulsing progress bar mode. The only reason this
doesn't flash the progress bar is because the stdout OperationDetail
object is updated first and before the 100 ms timeout fires to pulse the
bar, it's grandparent is updated with the new fraction to keep growing
the bar instead.
2) Common code
The Copy_Blocks code currently tracks the progress of blocks copied
against target amount, which it has to do anyway. That information is
then used to generate the text and fraction to update into the
OperationDetail object and drive the on screen progress bar. This same
level of tracking is wanted for the XFS and ext2/3/4 file system
specific copy methods.
Conclusion and solution
Having multiple sources of progress bar data is a problem and makes it
clear that there must be only one source of progress data. Also some
code can be shared for tracking the amount of blocks copied and
generating the display.
Therefore have a single ProgressBar object which is used everywhere.
This commit
It just creates a single ProgressBar object which is available via all
OperationDetail objects and Copy_Blocks is updated accordingly. Note
that the ProgressBar still contains debugging and that the GUI progress
bar of the current operation is still driven via the fraction member in
any OperationDetail object.
Referenced commits:
[1] 52a2a9b00a
Reduce threading (#685740)
[2] ae434579e1
Display progress for e2fsck (#467925)
[3] baea186138
Display progress for mke2fs (#467925)
[4] 57b028bb8e
Display progress during resize (#467925)
Bug 760709 - Add progress bars to XFS and EXT2/3/4 file system specific
copy methods
Write a generic progress bar class. Has the following features:
* Has separate progress and target numbers, rather than a single
completion fraction, to enable the the next feature.
* Optionally generates text reporting the amount of data copied using
the progress and target numbers like this:
"1.00 MiB of 16.00 MiB copied"
* After running for 5 seconds, also add estimated remaining time.
(Waits to allow the data copying rate to settle down a little before
estimating the remaining time). Looks like this:
"1.00 MiB of 16.00 MiB copied (00:01:59) remaining)"
The ProgressBar class is not driving the visual progress bar yet. It
has just been added into the internal block copy algorithm and generates
debug messages showing the progress bar is operating correctly.
Debugging looks like this:
DEBUG: ProgressBar::start(target=2.0636e+09, text_mode=PROGRESSBAR_TEXT_COPY_BYTES)
DEBUG: ProgressBar::update(progress=1.30023e+08) m_fraction=0.0630081 m_text="124.00 MiB of 1.92 GiB copied"
DEBUG: ProgressBar::update(progress=2.67387e+08) m_fraction=0.129573 m_text="255.00 MiB of 1.92 GiB copied"
DEBUG: ProgressBar::update(progress=4.0475e+08) m_fraction=0.196138 m_text="386.00 MiB of 1.92 GiB copied"
...
DEBUG: ProgressBar::update(progress=1.13351e+09) m_fraction=0.549289 m_text="1.06 GiB of 1.92 GiB copied (00:00:04 remaining)"
DEBUG: ProgressBar::update(progress=1.26249e+09) m_fraction=0.611789 m_text="1.18 GiB of 1.92 GiB copied (00:00:04 remaining)"
DEBUG: ProgressBar::update(progress=1.39041e+09) m_fraction=0.67378 m_text="1.29 GiB of 1.92 GiB copied (00:00:03 remaining)"
...
DEBUG: ProgressBar::update(progress=1.97552e+09) m_fraction=0.957317 m_text="1.84 GiB of 1.92 GiB copied (00:00:00 remaining)"
DEBUG: ProgressBar::update(progress=2.0636e+09) m_fraction=1 m_text="1.92 GiB of 1.92 GiB copied"
DEBUG: ProgressBar::stop()
Bug 760709 - Add progress bars to XFS and EXT2/3/4 file system specific
copy methods
Simplify code in Dialog_Partition_Info::Dialog_Info() which was open
coding concatenating together a vector of strings with a new line
between each. Replace with Glib::build_path(), as used elsewhere in
this method and other code.
Create and open a LUKS mapping but don't create any file system within.
# cryptsetup luksFormat /dev/sdb5
# cryptsetup luksOpen /dev/sdb5 sdb5_crypt
GParted was incorrectly reporting this warning:
Unable to read the contents of this file system!
Because of this some operations may be unavailable.
The cause might be a missing software package.
The following list of software packages is required for luks file
system support: dmsetup.
This is even though the usage figures for the on disk LUKS encryption
are fully known. See luks::set_used_sectors().
This was occurring because derived PartitionLUKS::set_usage_known()
was checking usage figures for the outer LUKS and the inner encrypted
file system, unknown in this case. Correct when displaying figures in
the UI, but not correct in GParted_Core::set_used_sectors() when only
checking the outer LUKS usage figures were set correctly. Fix this.
Bug 760080 - Implement read-only LUKS support
At the moment any messages for an encrypted file system aren't shown,
only messages from the outer PartitionLUKS object are shown. Also in
Win_GParted::activate_paste() the selected Partition object, possibly
a derived PartitionLUKS, is cloned and the messages cleared.
Therefore a set of accessor methods must be provided to query and modify
partition messages. Messages will be stored in the Partition object to
which they are added and retrieved from all. So in the case of a
derived PartitionLUKS they will be retrieved from the messages vector of
the PartitionLUKS object itself and the messages vector for the
encrypted file system it contains.
To replace code like this in GParted_Core:
partition_temp->messages = messages;
We might naturally provide a set_messages() method which assigns the
messages vector and is used like this:
partition_temp->set_messages( messages );
However on a PartitionLUKS object what should set_messages() do? By the
name it will replace any existing messages in the PartitionLUKS object
itself, but what should happen to the messages for the contained
encrypted Partition object? Should they be cleared or left alone?
Rather than implement set_messages() with unclear semantics implement
append_messages(), which in the PartitionLUKS object case will clearly
leave any messages for the contained encrypted Partition object alone.
Append_messages() is then used to add messages as the Partition or
PartitionLUKS objects when populating the data in GParted_Core.
Bug 760080 - Implement read-only LUKS support
For LUKS formatted partitions add an encryption section into the
Information dialog and display the type of encryption, path, UUID and
status of the encryption.
The file system section continues to display appropriate file system
details, including the partition graphic with the file system specific
border colour and correct usage. The details will either be of a plain
file system, an encrypted file system, or nothing when there is no open
dm-crypt mapping, leaving the encrypted file system inaccessible.
Should there be LUKS encryption directly within LUKS encryption then the
details of the inner encryption will be displayed in the file system
section. However this configuration will not be further supported by
GParted.
Bug 760080 - Implement read-only LUKS support
There is already the set of methods in the Partition class to report the
file system usage. Virtualise them and provide PartitionLUKS specific
implementations to calculate the usage of a file system wrapped in LUKS
encryption.
See the ascii art and comment in PartitionLUKS.cc for the details of
those calculations.
Bug 760080 - Implement read-only LUKS support
LUKS headers don't provide any concept of label. Also there is already
the method Partition::get_filesystem_label() for getting the *file
system* label, so virtualise it and provide an appropriate
implementation to get the label of an encrypted file system represented
within a derived PartitionLUKS object. This causes the label to be
displayed correctly in the main window.
It also happens to display the encrypted file system label in the
Information dialog for a LUKS formatted partition. However the whole
Information dialog will be addressed differently in a following commit.
Bug 760080 - Implement read-only LUKS support
In the main window disk graphic, when there is an open dm-crypt mapping,
display the colour of the encrypted file system.
Bug 760080 - Implement read-only LUKS support
In the File System column in the GUI, when there is an open dm-crypt
mapping, display the colour square for the encrypted file system within
and the text as "[Encrypted] FSTYPE". For closed mappings nothing can
be known about the encrypted file system within so continue to display a
purple square and the text "[Encrypted]".
Looks like:
Partition | File System
...
/dev/sdb3 # ext4
v /dev/sdb4 * # extended
/dev/sdb5 # [Encrypted]
/dev/sdb6 * # [Encrypted] unknown
/dev/sdb7 * # [Encrypted] ext4
Bug 760080 - Implement read-only LUKS support
Partition object represents a region of a disk and the file system
within. GParted always displays the colour base of the type of the file
system. Therefore remove the color member and always look it up from
the type of the file system as needed.
This makes one less member that will need virtual accessor methods with
different handling in the derived PartitionLUKS class.
Bug 760080 - Implement read-only LUKS support
When there exists an open dm-crypt mapping, populate the encrypted
Partition object representing the encrypted file system.
Bug 760080 - Implement read-only LUKS support
Absolute minimum implementation of a PartitionLUKS class which can be
constructed, polymorphically copied and destroyed. Contains an
"encrypted" member of type Partition to represent the encrypted file
system within the LUKS format.
Create PartitionLUKS objects instead of base Partition objects when a
LUKS formatted partition is found. Only the base Partition object
member values have been populated, and the "encrypted" member remains
blank at this point.
Bug 760080 - Implement read-only LUKS support
This is the equivalent change as made to set_mountpoints() in an earlier
commit. Change GParted_Core::set_used_sectors() from being called with
a vector of partitions and processing them all to being called per
partition. This is in preparation for calling set_used_sectors() on a
single Partition object inside a PartitionLUKS object.
Bug 760080 - Implement read-only LUKS support
Populate the canonical device name, /dev/mapper/NAME, used to access the
encrypted file system into the mount points of the Partition object.
This is the equivalent of what is already done for the Volume Group name
and SWRaid Array device.
This does get displayed in the Mount Point column in the main window,
which isn't wanted. However the data will be needed when displaying
details of the encryption mapping in the Information dialog. Both will
be dealt with in following commits.
Bug 760080 - Implement read-only LUKS support
Previously GParted_Core::set_mountpoints() was called with a vector of
partitions and processed them all. Now make set_mountpoints() process a
single partition and push the calls to it down one level from
set_devices_thread() into set_device_partitions() and
set_device_one_partition(). This is in preparation for having an
encrypted file system represented as a Partition object inside a
PartitionLUKS object and needing to call set_mountpoints() for the inner
single Partition object.
Bug 760080 - Implement read-only LUKS support
Only load the LUKS_Info cache of active dm-crypt mappings when the first
LUKS partition is encountered. Not needed from a performance point of
view as the longest that I have ever seen "dmsetup table --target crypt"
take to run is 0.05 seconds. Just means that the dmsetup command is
only run when there are LUKS partitions and the information is needed.
Bug 760080 - Implement read-only LUKS support
Populate the used, unused and unallocated figures in the Partition
object for a LUKS formatted partition. See comment in
luks::set_used_sectors() for the rational of what is used, unused and
unallocated.
As that rational mentions, a LUKS header does not store the size of the
encrypted data and is assumed to extend to the end of the partition by
the tools which start the mapping.
An underlying block device of 128 MiB (131072 KiB).
# sfdisk -s /dev/sde
131072
An active LUKS mapping at offset 2 MiB (4096 512-byte sectors) and
length 126 MiB (129024 KiB, 258048 512-byte sectors).
# sfdisk -s /dev/mapper/sde_crypt
129024
# cryptsetup status sde_crypt
/dev/mapper/sde_crypt is active.
type: LUKS1
cipher: aes-cbc-essiv:sha256
keysize: 256 bits
device: /dev/sde
offset: 4096 sectors
size: 258048 sectors
mode: read/write
No size/length reported when dumping the LUKS header, just (payload)
offset.
# cryptsetup luksDump /dev/sde
LUKS header information for /dev/sde
Version: 1
Cipher name: aes
Cipher mode: cbc-essiv:sha256
Hash spec: sha1
Payload offset: 4096
MK bits: 256
MK digest: 7f fb ba 40 7e ba e4 3b 2f c6 d0 93 7b f7 05 49 7b 72 d4 ad
MK salt: 4a 5b 54 f9 7b 67 af 6e ef 16 31 0a fe d9 7e 5f
c3 66 dc 8a ed e0 07 f4 45 c3 7c 1a 8d 7d ac f4
MK iterations: 37750
UUID: 0a337705-434a-4994-a842-5b4351cb3778
...
Shrink the LUKS mapping to 64 MiB (65536 KiB, 131072 512-byte sectors).
# cryptsetup resize --size 131072 sde_crypt
# sfdisk -s /dev/mapper/sde_crypt
65536
# cryptsetup status sde_crypt
/dev/mapper/sde_crypt is active.
type: LUKS1
cipher: aes-cbc-essiv:sha256
keysize: 256 bits
device: /dev/sde
offset: 4096 sectors
size: 131072 sectors
mode: read/write
Stop and start the LUKS mapping.
# cryptsetup luksClose sde_crypt
# cryptsetup luksOpen /dev/sde sde_crypt
The size of the LUKS mapping is back to 126 MiB (129024 KiB, 258048
512-byte sectors), extending to the end of the partition.
# sfdisk -s /dev/mapper/sde_crypt
129024
# cryptsetup status sde_crypt
/dev/mapper/sde_crypt is active.
type: LUKS1
cipher: aes-cbc-essiv:sha256
keysize: 256 bits
device: /dev/sde
offset: 4096 sectors
size: 258048 sectors
mode: read/write
Bug 760080 - Implement read-only LUKS support
Also load the starting offset and length of the active dm-crypt mapping
into the LUKS_Info module from the dmsetup output. This provides the
location and size of the encrypted data within the underlying block
device.
Note that dmsetup reports in units of 512 bytes sectors [1], the GParted
LUKS_Info module uses bytes and GParted Partition objects work in device
sector size units. However the actual sector size of a dm-crypt mapping
[2] is the same as that of the underlying block device [3].
# modprobe scsi_debug dev_size_mb=128 sector_size=4096
# fgrep scsi_debug /sys/block/*/device/model
/sys/block/sdd/device/model:scsi_debug
# parted /dev/sde print
Error: /dev/sde: unrecognised disk label
Model: Linux scsi_debug (scsi)
Disk /dev/sde: 134MB
[3] Sector size (logical/physical): 4096B/4096B
Partition Table: unknown
# cryptsetup luksFormat /dev/sde
# cryptsetup luksOpen /dev/sde sde_crypt
# parted /dev/mapper/sde_crypt print
Error: /dev/mapper/sde_crypt: unrecognised disk label
Model: Linux device-mapper (crypt) (dm)
Disk /dev/mapper/sde_crypt: 132MB
[2] Sector size (logical/physical): 4096B/4096B
Partition Table: unknown
# cryptsetup status sde_crypt
/dev/mapper/sde_crypt is active.
type: LUKS1
cipher: aes-cbc-essiv:sha256
keysize: 256 bits
device: /dev/sde
offset: 4096 sectors
size: 258048 sectors
mode: read/write
# dmsetup table --target crypt
...
sde_crypt: 0 258048 crypt aes-cbc-essiv:sha256 0000000000000000000000000000000000000000000000000000000000000000 0 8:64 4096
[1] Both cryptsetup and dmsetup report the offset as 4096 and the size/
length as 258048. 128 MiB / (4096+258048) = 512 byte units, even on a
4096 byte sector size device.
Update debugging of LUKS to this:
# ./gpartedbin
======================
libparted : 2.4
======================
DEBUG: /dev/sdb5: LUKS closed
DEBUG: /dev/sdb6: LUKS open mapping /dev/mapper/sdb6_crypt, offset=2097152, length=534773760
/dev/sde: unrecognised disk label
DEBUG: /dev/sde: LUKS open mapping /dev/mapper/sde_crypt, offset=2097152, length=132120576
Bug 760080 - Implement read-only LUKS support
The code currently allows attempting to mount and unmount a LUKS
partition. It is nonsense to directly try to mount and unmount a LUKS
partition and obviously doesn't work. For read-only LUKS support there
is no need to attempt to apply this to the encrypted file system within.
Therefore prevent these operations for LUKS partitions.
Bug 760080 - Implement read-only LUKS support
This patchset is adding read-only LUKS support. Creation of LUKS is
planned to be a tick box adding encryption in the Create New Partition
dialog. Therefore remove the greyed out crypt-luks entry in the Create
New Partition dialog and the Format menu.
Bug 760080 - Implement read-only LUKS support
Provide a minimal implementation of a luks file system class which only
does busy detection.
NOTE:
For now, read-only LUKS support, a LUKS partition will be busy when a
dm-crypt mapping exists. Later when read-write LUKS support is added
GParted will need to look at the busy status of the encrypted file
system within the open LUKS partition and map LUKS partition busy status
to encryption being open or closed.
Bug 760080 - Implement read-only LUKS support
Load basic details of active Device-mapper encryption mappings from the
kernel. Use dmsetup active targets.
# cryptsetup luksFormat /dev/sdb5
# cryptsetup luksFormat /dev/sdb6
# cryptsetup luksOpen /dev/sdb6 sdb6_crypt
# ls -l /dev/mapper/sdb6_crypt /dev/dm-0
lrwxrwxrwx. 1 root root 7 Nov 15 09:03 /dev/mapper/sdb6_crypt -> ../dm-0
brw-rw----. 1 root disk 253, 0 Nov 15 09:03 /dev/dm-0
# ls -l /dev/sdb6
brw-rw----. 1 root disk 8, 22 Nov 15 09:02 /dev/sdb6
# dmsetup table --target crypt
sdb6_crypt: 0 1044480 crypt aes-cbc-essiv:sha256 0000000000000000000000000000000000000000000000000000000000000000 0 8:22 4096
So far just load the mapping name and underlying block device reference
(path or major, minor pair).
Note that all supported kernels appear to report the underlying block
device as major, minor pair in the dmsetup output. Underlying block
device paths are added to the cache when found during a search to avoid
stat(2) call on subsequent searches for the same path.
Prints debugging to show results, like this:
# ./gpartedbin
======================
libparted : 2.4
======================
DEBUG: /dev/sdb5: LUKS closed
DEBUG: /dev/sdb6: LUKS open mapping /dev/mapper/sdb6_crypt
Bug 760080 - Implement read-only LUKS support
Renamed from DEV_MAP_PATH to DEV_MAPPER_PATH. Moved so that the
constant is logically intended for use outside of the DMRaid class.
Also specifically make the string constant have external linkage, rather
than the default internal (static) linkage for constants, so that there
is only one copy of the variable in the program, rather than one copy in
each compilation unit which included DMRaid.h. Namely DMRaid.cc and
GParted_Core.cc.
References:
[1] Proper way to do const std::string in a header file?
http://stackoverflow.com/questions/10201880/proper-way-to-do-const-stdstring-in-a-header-file
[2] What is external linkage and internal linkage in C++
http://stackoverflow.com/questions/1358400/what-is-external-linkage-and-internal-linkage-in-c/1358796#1358796
Bug 760080 - Implement read-only LUKS support
History:
1) The constructor was added by commit:
6d8b169e73
2006-03-14 21:37:47
changed the way devices and partitions store their devicepaths. Instead of
2) Removed from most of the file system specific ::Copy() methods by
commit:
ad9f2126e7
2006-03-19 15:30:20
fixed issues with copying (see also #335004) cleanups + added FIXME added
3) Removed from GParted_Core::copy() method by commit:
7bb7e8a84f
2006-05-23 22:17:34
Use ped_device_read and ped_device_write instead of 'dd' to copy
4) Finally removed from the last place in xfs::Copy() method by commit:
e414b71b73
2012-01-11 19:49:13
Update xfs resize and copy to use new helper functions
The Partition(path) constructor is no longer used. Remove.
The sector_size parameter is unnecessary as the value can be retrieved
from the sector size of the selected Partition object on which the
create new, copy & paste or resize/move operation is being performed.
For the create new and resize/move operations it is trivial as the
existing unallocated or in use Partition object on which the operation
is being perform already contains the correct sector size. For the copy
& paste operation, which can copy across disk devices of different
sector sizes, we merely have to use the sector size of the existing
selected (destination) Partition object rather than copied (source)
Partition object. Hence these relevant lines in the new code:
Dialog_Partition_Copy::set_data(selected_partition, copied_partition)
new_partition = copied_partition.clone();
...
new_partition->sector_size = selected_partition.sector_size;
The expressions used in the call to Set() were comparing
lp_partition->type to 0 for the type parameter and passing it as a bool
for the inside_extended parameter. Libparted lp_partition->type is
actually an enumeration. The code was only working because of the
specific values assigned to the symbolic names, PED_PARTITION_NORMAL = 0
and PED_PARTITION_EXTENDED is non-zero (true).
Make the code use the symbolic names and not depend on the actual
enumeration values, which should be considered changeable and private to
libparted.
When displaying an unallocated partition
Win_GParted::set_valid_operations() calls GParted_Core::get_fs() with
parameter FS_UNALLOCATED.
Before this change, get_fs() would fail to find file system capabilities
set for FS_UNALLOCATED and construct a not supported capabilities set
and return that.
Afterwards, find_supported_filesystems() creates a not supported
capabilities set from the NULL pointer for FS_UNALLOCATED and adds this
entry into the FILESYSTEMS vector. Then get_fs() finds that not
supported capabilities set for FS_UNALLOCATED in the FILESYSTEMS vector
and returns that.
This makes no functional difference. It just seems right as other
unsupported but used file system types have entries in FILESYSTEM_MAP.
The struct FS constructor initialised every member *except* filesystem
and busy. Then in *most* cases after declaring struct FS, assignments
followed like this:
FS fs;
fs.filesystem = FS_BTRFS;
fs.busy = FS::GPARTED;
But member busy wasn't always initialised.
Add initialisation of members filesystem and busy to the struct FS
constructor. Specify optional parameter to the constructor to set the
filesystem member, or when left off filesystem is initialised to
FS_UNKNOWN.
get_fs() used to work by (1) returning the supported capabilities of the
requested file system found in the FILESYSTEMS vector; (2) if not found
return the supported capabilities for file system FS_UNKNOWN; and (3)
if that wasn't found either, create a not supported capabilities set for
FS_UNKNOWN and return that.
This is more complicated that required. Also the not supported
capabilities set, as created by struct FS() constructor, is the same as
that created in file_supported_filesystems() local variable fs_notsupp.
Simplify get_fs() just using a single not found code path returning a
not supported capabilities set.
Final step for full polymorphic handling of Partition objects is to
implement a virtual copy constructor. C++ doesn't directly support
virtual copy constructors, so instead use the virtual copy constructor
idiom [1]. (Just a virtual method called clone() which is implemented
in every polymorphic class and creates a clone of the current object and
returns a pointer to it).
Then replace all calls to the (monomorphic) Partition object copy
constructor throughout the code, except in the clone() implementation
itself, with calls to the new virtual clone() method "virtual copy
constructor".
Also have to make the Partition destructor virtual too [2][3] so that
the derived class destructor is called when deleting using a base class
pointer. C++ supports this directly.
[1] Wikibooks: More C++ Idioms / Virtual Constructor
https://en.wikibooks.org/wiki/More_C%2B%2B_Idioms/Virtual_Constructor
[2] When to use virtual destructors?
http://stackoverflow.com/questions/461203/when-to-use-virtual-destructors
[3] Virtuality
Guideline #4: A base class destructor should be either public and
virtual, or protected and nonvirtual
http://www.gotw.ca/publications/mill18.htm
Bug 759726 - Implement Partition object polymorphism
SQUASH: When first using pointers to Partition and calling delete
Copy assignment of Partition objects is now only performed in a few
places in the Operation and OperationResizeMove classes when updating
the displayed PartitionVector. (From Refresh_Visual() when each
operation is visually applied to the display_partitions vector; the
new_partition from the operation is copy assigned over the top of the
relevant existing partition in the display_partitions vector).
In general polymorphic copy assignment is complicated [1], and is now
unnecessary given the above limited use. All that is needed is a way to
polymorphically replace one Partition object with another in a
PartitionVector.
First, prevent further use of Partition object copy assignment by
providing a private declaration and no implementation, so the compiler
enforces this. Second implement and use PartitionVector method
replace_at() which replaces a pointer to one Partition object with
another at the specified index in the PartitionVector.
[1] The Assignment Operator Revisited
[Section:] Virtual assignment
http://icu-project.org/docs/papers/cpp_report/the_assignment_operator_revisited.html
Bug 759726 - Implement Partition object polymorphism
Now use a pointer to the Partition object in Dialog_Base_Partition class
and derived classes, Dialog_Partition_{Copy,New,Resize_Move}. This is
equivalent to how the Partition objects are managed in the Operation and
derived classes.
The Partition object is allocated and copy constructed in each derived
classes' set_data() method, called from each constructor and deallocated
in the destructors. Considering the remaining Big 3, these classes are
never copy constructed or copy assigned so provide private definitions
and no implementations so the compiler enforces this.
Bug 759726 - Implement Partition object polymorphism
The code in on_view_clicked() copy constructed a Partition object and
then in the following 3 lines only read a couple of public member
variables from the new copy.
Making a copy of the partition is unnecessary. Change to just creating
a constant reference to the Partition instead.
(It would also be an impediment to polymorphically using Partition
objects, except for the fact that gpart doesn't recognise LUKS
signatures so will never have to create a PartitionLUKS object).
Bug 759726 - Implement Partition object polymorphism
A number of methods in GParted_Core and Win_GParted were using local
Partition objects. Change them into pointers so that Partition object
polymorphism can be implemented.
Bug 759726 - Implement Partition object polymorphism
Change Win_GParted::copied_partition from Partition object which is
copied by value into a pointer to a Partition object object which is
allocated, copy constructed and deleted. Required as part of the
polymorphic implementation of Partitions.
As before when managing the lifetime of pointers to objects in a class
the Big 3 of destructor, copy constructor and copy assignment operator
need to be considered. A destructor is added to finally delete
copied_partition. A single Win_GParted object is only ever created and
destroyed in main(). The class is never copy constructed or copy
assigned. Make the compiler enforce this with private declarations and
no implementations.
Bug 759726 - Implement Partition object polymorphism
Operation classes now internally use pointers to Partition objects and
take on management of their lifetimes. As before, with the
PartitionVector class, when storing pointers in a class the Big 3 of
destructor, copy constructor and copy assignment operator also have to
be considered.
First, all the Partition objects are allocated in the derived Operation*
class parameterised constructors and freed in the associated
destructors. However the Operation classes are never copy constructed
or copy assigned; they are only ever created and destroyed. Only
pointers to the derived Operations are copied into the vector of pending
operations. Therefore the copy construtor and copy assignment operator
aren't needed. To enforce this provide inaccessible private
declarations without any implementation so that the compiler will
enforce this [1][2].
This example code fragment:
1 OperationCheck o1( device, partition );
2 OperationCheck o2 = o1;
3 o2 = o1;
Does these OperationCheck calls:
1 Implemented parameterised construtor,
2 Disallowed copy constructor,
3 Disallowed copy assignment
Trying to compile the above code would fail with errors like these:
../include/OperationCheck.h: In member function 'void GParted::Win_GParted::activate_check()':
../include/OperationCheck.h:36:2: error: 'GParted::OperationCheck::OperationCheck(const GParted::OperationCheck&)' is private
OperationCheck( const OperationCheck & src ); // Not implemented copy constructor
^
test.cc:2:21: error: within this context
OperationCheck o2 = o1;
^
../include/OperationCheck.h:37:19: error: 'GParted::OperationCheck& GParted::OperationCheck::operator=(const GParted::OperationCheck&)' is private
OperationCheck & operator=( const OperationCheck & rhs ); // Not implemented copy assignment operator
^
test.cc:3:4: error: within this context
o2 = o1;
^
[1] Disable copy constructor
http://stackoverflow.com/questions/6077143/disable-copy-constructor
[2] Disable compiler-generated copy-assignment operator [duplicate]
http://stackoverflow.com/questions/7823845/disable-compiler-generated-copy-assignment-operator
Bug 759726 - Implement Partition object polymorphism
The Operation classes contain partition objects which are copied by
value. Need to replace these with pointers to Partition objects instead
and manage their lifetimes so that they can be used polymorphically.
First step is to protect the partition members partition_new,
partition_original, and for OperationCopy class only, partition_copied
within the Operation classes and provide accessor methods.
get_partition_new() and get_partition_original() accessors are
implemented in the Operation base class so all derived classes get an
implementation. get_partition_new() is also virtual so that
OperationCheck and OperationDelete can override the implementation and
assert that they don't use partition_new. get_partition_copied() is
provided for the OperationCopy class only so can only be accessed via an
OperationCopy type variable.
Bug 759726 - Implement Partition object polymorphism
Remove PartitionVector push_back() and insert() methods which copy
construct Partitions objects into the vector. All the code has already
been changed to dynamically allocate Partition objects and use the
adoption variants of these methods named, push_back_adopt() and
insert_adopt(). Remove the no longer used methods.
Bug 759726 - Implement Partition object polymorphism
Replace all the current code which uses push_back() and insert() of a
local Partition object and gets it copy constructed into a
PartitionVector. Instead allocate a Partition object on the heap and
adopt a pointer into the PartitionVector using push_back_adopt() and
insert_adopt().
Bug 759726 - Implement Partition object polymorphism
GParted_Core and Operation classes both have an insert_unallocated()
method which do the same thing with very nearly identical code. Both
methods insert unallocated partitions into the vector of partitions
within the specified range of sectors to fill in any gaps larger than
1 MiB. The only difference was how the two methods got the device path;
the GParted_Core class method got it via a parameter and the Operation
class method got it by calling get_path() on its device member variable.
The GParted_Core insert_unallocated() method gets called during device
scanning and the Operation one gets called when constructing the visual
for a pending operation.
Consolidate down to a single insert_unallocated() implementation by
making the Operation class method call the GParted_Core class method.
Make the GParted_Core class method static and public so that it can be
called using the class name from outside the class.
Bug 759726 - Implement Partition object polymorphism
The current code uses push_back() and insert() to copy Partition objects
into the vector of pointers. This has a few issues:
1) Unnecessary copying of Partition objects;
2) Hides the nature of the PartitionVector class as a manager of
pointers to Partition objects by providing copy semantics to add
items. It is generally better to be explicit;
3) C++ doesn't provide polymorphic copy construction directly, but this
is easily worked around by following the Virtual Constructor idiom
[1], which would allow PartitionLUKS derived class objects to be
copied into the vector.
Add push_back_adopt() and insert_adopt() methods which add a pointer to
a Partition object into the PartitionVector adopting ownership.
[1] Wikibooks: More C++ Idioms / Virtual Constructor
https://en.wikibooks.org/wiki/More_C%2B%2B_Idioms/Virtual_Constructor
Bug 759726 - Implement Partition object polymorphism
The PartitionVector class is now internally using pointers to Partition
objects and taking on management of their lifetimes. It therefore has
to implement the Big 3: destructor, copy constructor and copy assignment
operator [1][2]. This is because the implicitly-defined copy
constructor and assignment operator perform memberwise "shallow copying"
and the destructor does nothing. This not correct for classes which
contain non-class types such as raw pointers.
The semantics of the interface still copies each Partition object into
the PartitionVector when they are added with push_back() and insert().
Note that a PartitionVector object is explicitly copy assigned in
Win_GParted::Refresh_Visual(). They are also implicitly copied when
(1) the implementing vector is resized larger to allow it to hold more
pointers to Partition objects than it previously had capacity for; and
(2) a Partition object is copied including the logicals PartitionVector
member.
[1] The rule of three/five/zero
http://en.cppreference.com/w/cpp/language/rule_of_three
[2] Rule of Three
https://en.wikipedia.org/wiki/Rule_of_three_%28C%2B%2B_programming%29
Bug 759726 - Implement Partition object polymorphism
Lots of files which use the Partition class relied on the declaration
being included via other header files. This is bad practice.
Add #include "Partition.h" into every file which uses the Partition
class which doesn't already include it. Header file #include guards are
specifically to allow this.
get_partitions() method was returning a vector of partitions. However
the calling code only needed to know whether any partitions were found
or not. Replace with found_partitions() method reporting the needed
boolean.
Now use of std::vector<Partition> partitions is hidden within the
Dialog_Rescue_Data class implementation.
Bug 759726 - Implement Partition object polymorphism
GParted was also displaying the GPT partition names as the file system
labels for some type of file systems.
Create 2 empty partitions on a GPT partitioned disk and name them like
this. Then create a hfsplus file system with an empty label. (Actually
single space character but blkid treats it as empty. Can't use GParted
for this because when no label is specified mkfs.hfsplus sets the label
to "untitled").
# sgdisk -p /dev/sdb
/dev/sdb: 4194304 sectors, 2.0 GiB
Logical sector size: 512 bytes
...
Number Start (sector) End (sector) Size Code Name
1 2048 526335 256.0 MiB 8300 empty partition
2 526336 1050623 256.0 MiB 8300 hfsplus partition
# mkfs.hfsplus -v " " /dev/sdb2
Even though only the GPT partition names are set to "SOMETHING
partition", GParted will display them as the file system labels too.
Also in the Information dialog for the empty partition a file system
UUID is displayed as well even though none exists.
Blkid output looks like this:
# blkid -V
blkid from util-linux 2.27.1 (libblkid 2.27.0, 02-Nov-2015)
# blkid | grep /dev/sdb
/dev/sdb1: PARTLABEL="empty partition" PARTUUID="bf3d2085-65b7-4ae6-97da-5ff968ad7d2c"
/dev/sdb2: UUID="2c893037-ff76-38f2-9158-2352ef5dc8de" TYPE="hfsplus" PARTLABEL="hfsplus partition" PARTUUID="457e9c2b-e4f2-4235-833b-28208592aaac"
With blkid from util-linux >= 2.22, released September 2012, it is
including additional PARTLABEL and PARTUUID name and value pairs for
GPT partitions [1]. The FS_Info module regular expressions used to
match the file system LABEL and UUID names also happen to match these
new PARTLABEL and PARTUUID names too. Hence this bug when GParted has
to fall back to using the FS_Info module to read the file system label,
when there is no working file system specific method. Effects: exfat,
f2fs, hfs, hfsplus, ufs, unknown.
Fix by requiring all the regular expressions used to search the blkid
output to also match the space character in front of the name.
[1] Util-linux 2.22 Release Notes
https://git.kernel.org/cgit/utils/util-linux/util-linux.git/tree/Documentation/releases/v2.22-ReleaseNotes?h=v2.22
Bug 759972 - GParted displays partition names also as file system labels
with new blkid for some file systems
In the UI new partitions were being named "unallocated" instead of
"New Partition #1", etc. Also deleting a new partition not yet created
deletes all new partitions from the operation list because it matches
by name only and they were all named "unallocated". Broken by this
recent commit:
762cd1094a
Return class member from Dialog_Partition_New::Get_New_Partition() (#757671)
Prior to this commit in the create new partition dialog code did these
relevant steps:
Dialog_Partition_New::Get_New_Partition()
Partition part_temp;
...
part_temp.Set(..., "New Partition #%1", ...);
Create local Partition object using default constructor which calls
Partition::Reset() and clears the paths vector. It then calls Set()
which appends the new name making the vector:
paths = ["New Partition #%1"]
After the above commit the code did these steps:
Dialog_Partition_New::Dialog_Partition_New(..., selected_partition, ...)
set_data(..., selected_partition, ...);
new_partition = selected_partition;
Dialog_Partition_New::Get_New_Partition(...)
new_partition.Set(..., "New Partition #%1", ...);
New_partition is copied from the selected unallocated partition in which
the new partition will be created. So new_partition is now named
"unallocated". Then the Set() call appends the new name making the
vector:
paths = ["unallocated", "New Partition #%1"]
As get_path() returns the first name in the paths vector the path name
changed from "New Partition #%1" to "unallocated" causing this bug.
Fix by resetting the new_partition object to clear all vestiges of it
being a copy of the selected unallocated partition, Reset() call, before
then calling Set(). This then appends the new name to an empty vector
making it contain just the required new name:
paths = ["New Partition #%1"]
Bug 759488 - Pending create partitions are all getting named
"unallocated"
Capture and parse the progress reports of ntfsresize and resize2fs and
update the dialog progress bar.
Bug 467925 - gparted: add progress bar during operation
Return newly constructed partition object by reference rather than by
copy from the Copy, Resize/Move and New dialog classes. This is another
case of stopping copying partition objects in preparation for using
polymorphic Partition objects. In C++ polymorphism has to use pass by
pointer and reference and not pass by value, copying, to avoid object
slicing.
The returned reference to the partition is only valid until the dialog
object containing the new_partition member is destroyed. This is okay
because in all three cases the returned referenced partition is copied
into a context with new lifetime expectations before the dialog object
is destroyed.
Case 1: GParted_Core::activate_paste()
Referenced new_partition is copied in the OperationCopy constructor
before the dialog object goes out of scope.
Operation * operation = new OperationCopy( ...,
dialog.Get_New_Partition( ... ),
... );
Case 2: GParted_Core::activate_new()
Referenced new_partition is copied in the OperationCreate
constructor before the dialog object goes out of scope.
Operation * operation = new OperationCreate( ...,
dialog.Get_New_Partition( ... ) );
Case 3: GParted_Core::activate_resize()
Temporary partition object is copied from the referenced
new_partition before the dialog object goes out of scope.
Partition part_temp = dialog.Get_New_Partition( ... );
Bug 757671 - Rework Dialog_Partition_New::Get_New_Partition() a bit
Replace the use of local Partition variable with class member in
preparation for Dialog_Partition_New::Get_New_Partition() being changed
to return the new partition object by reference instead of by value.
part_temp -> new_partition
Bug 757671 - Rework Dialog_Partition_New::Get_New_Partition() a bit
When creating a new extended partition, the unallocated space within it
was being created before adjusting the partition boundaries for
alignment reasons. This must be wrong. Move creation of the
unallocated space to after adjusting the partition boundaries for
alignment reasons. First introduced by this commit:
a30f991ca5
Fix size reduced by 1 MiB when created after cylinder aligned partition
Also added a big FIXME comment explaining how further adjustments are
still made by snap_to_alignment() to the partition boundaries including
a test case where this too late adjustment causes overlapping boundaries
and apply to fail.
Bug 757671 - Rework Dialog_Partition_New::Get_New_Partition() a bit
This is just to make the parameter name in the Dialog_Partition_New
constructor and set_data() method match the name of the equivalent
parameter in the Dialog_Partition_Copy and Dialog_Partition_Resize_Move
classes. (All three classes inherit from Dialog_Base_Partition and have
similar interfaces).
Bug 757671 - Rework Dialog_Partition_New::Get_New_Partition() a bit
Blkid recognises many more file system types and RAID member signatures
than libparted. GParted already uses blkid detection instead of or
before libparted for whole disk devices [1] and for ext4 detection [2]
(only required with libparted < 1.9.0). Also GParted could only use
blkid detection on non-512 byte sector devices [3] before libparted was
fixed in version 3.2 [4]. Blkid was documented as a mandatory
requirement from GParted 0.24.0 [5].
Util-linux package, of which blkid command is a part, is a core piece of
Linux software which is very actively maintained and used by lots of
other packages. Parted package is much less active and has added
detection of fewer file systems and doesn't recognise any RAID members.
In cases of multiple signatures within a partition blkid and libparted
can report different results leading to confusion and issues for
GParted. This was the primary reason for bug 688882 "Improve clearing
of file system signatures" and a number of other changes to GParted.
Also as the mount command links with libblkid it uses it's detection
when telling the kernel the type of a file system to be mounted.
There aren't any current issues with GParted's file system detection but
given the above argument, switch to using blkid before libparted for
file system detection. Only falling back to libparted when blkid
doesn't report a result, notably for extended partitions. Order of
information sources for detection is now:
1) mdadm (for SWRaid members)
2) blkid
3) libparted
4) GParted internal code
References:
[1] f8faee6377
Avoid whole disk FAT being detected as MSDOS partition table (#743181)
[2] 533eb1bc03
Added support for ext4 file systems
[3] 9e5e9f5627
Enhance file system detection to use FS_Info method - blkid
[4] http://git.savannah.gnu.org/cgit/parted.git/commit/?id=80678bdd957cf49a9ccfc8b88ba3fb8b4c63fc12
Fix filesystem detection on non 512 byte sectors
[5] 749a249571
Document blkid command as a mandatory requirement (#753436)
Bug 757781 - Always use blkid file system detection before libparted
The GParted_Core code only interacts with derived Operation* objects
through the Operation base class interface and in one case the
OperationCopy class. Therefore include Operation.h and OperationCopy.h
headers and no others.
ZFS labels were not cleared by GParted when clearing old file system
signatures.
# wget https://git.kernel.org/cgit/utils/util-linux/util-linux.git/plain/tests/ts/blkid/images-fs/zfs.img.bz2
# bzip2 -dc zfs.img.bz2 > /dev/sdb1
[In GParted format to cleared /dev/sdb1]
# blkid /dev/sdb1
/dev/sdb1: LABEL="tank" UUID="1782036546311300980" UUID_SUB="13179280127379850514" TYPE="zfs_member"
Update to also zero all 4 ZFS labels.
NOTE:
GParted now writes a little over 1 MiB when clearing old file system
signatures. As this is performed in the main thread the UI is not able
to respond during this action. Testing this on a range of USB flash
keys and hard drives found the slowest normal time to write this was
0.25 seconds, with an occasional outlier up to 2.8 seconds from a USB
flash key. This is considered acceptable.
Following the previous commit "Add erasing of SWRaid metadata 0.90 and
1.0 super blocks (#756829)" signature zeroing specified to write 4 KiB
of zeros at position end - 64 KiB, aligned to 64 KiB. Example operation
details from formatting a 1 GiB partition to cleared:
Format /dev/sdb8 as cleared
+ calibrate /dev/sdb8
+ clear old file system signatures in /dev/sdb8
+ write 68.00 KiB of zeros as byte offset 0
+ wite 4.00 KiB of zeros at byte offset 67108864
+ wite 64.00 KiB of zeros at byte offset 1073676288
+ write 8.00 KiB of zeros at byte offset 1073733632
+ flush operating system cache of /dev/sdb
However it actually wrote 64 KiB. This is because the rounding /
alignment was also applied to the zeroing length. Before this commit
rounding / alignment was always less than or equal to the length so this
wasn't seen before. Instead just apply device sector size rounding up
to the length.
Bug 756829 - SWRaid member detection enhancements
The super blocks for Linux Software RAID arrays using metadata types
0.90 and 1.0 are stored at the end of the partition and not currently
cleared by GParted.
Create a SWRaid array, stop it and format it to cleared using GParted.
The signature remains.
# mdadm --create /dev/md1 --level=linear --raid-devices=1 --force --metadata=1.0 /dev/sdb1
# mdadm --stop /dev/md1
[In GParted format to cleared /dev/sdb1]
# blkid /dev/sdb1
/dev/sdb1: UUID="8ac947a7-063f-2266-5f2a-e5d178198139" UUID_SUB="49bd51d4-4c54-fb16-a45e-bd795f783f59" LABEL="rockover:1" TYPE="linux_raid_member"
As fixed in other cases before [1][2] it is necessary to clear all
signatures before formatting as a new file system to prevent recognition
issues. For example now format the partition as a FAT32 file system.
Now there are two signatures and libparted reports one type and blkid
reports another.
# mkdosfs -F 32 /dev/sdb1
# blkid /dev/sdb1
/dev/sdb1: UUID="8ac947a7-063f-2266-5f2a-e5d178198139" UUID_SUB="49bd51d4-4c54-fb16-a45e-bd795f783f59" LABEL="rockover:1" TYPE="linux_raid_member"
# parted /dev/sdb print
Model: ATA SAMSUNG SSD UM41 (scsi)
Disk /dev/sdb: 8012MB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number Start End Size Type File system Flags
1 1049kB 538MB 537MB primary fat32
(Deliberately avoided btrfs, ext2/3/4 and xfs as recent versions of
their mkfs tools clear other signatures first for the same reason).
[1] Bug 688882 - Improve clearing of file system signatures
[2] 3c75f3f5b1
Use wipefs to clear old signatures before creating new file systems (#688882)
Update erase_filesystem_signatures() to also zero the necessary sectors
to clear SWRaid metadata 0.90 and 1.0 super blocks.
Bug 756829 - SWRaid member detection enhancements
If the installation is unusual / broken such that the mdadm command is
not found but there are active SWRaid arrays, provide a fallback with
some of the information. In this case populate the cache with details
only available from /proc/mdstat. Information will be limited to active
arrays only and their members. No UUIDs or labels. There will be no
information about inactive arrays and GParted will use it's normal
libparted and blkid identification for those devices and partitions.
As mdadm has gained the capability to manage Fake/BIOS RAID arrays they
also appear in /proc/mdstat when mdadm is used to start them. Enhance
the parser of /proc/mdstat to only extract information for SWRaid arrays
with recognised metadata versions.
Bug 756829 - SWRaid member detection enhancements
Automatically load the cache of SWRaid information for the first time if
any of the querying methods are called before the first explicit
load_cache() call. Means we can't accidentally use the class and
incorrectly find no SWRaid members when they do exist.
Bug 756829 - SWRaid member detection enhancements
Busy file systems are accessed via a mount point, LVM Physical Volumes
are activated via the Volume Group name and busy SWRaid members are
accessed via the array device, /dev entry. Therefore choose to show the
array device in the mount point field for busy SWRaid members.
The kernel device name for an SWRaid array (without leading "/dev/") is
the same as used in /proc/mdstat and /proc/partitions. Therefore the
array device (with leading "/dev/") displayed in GParted will match
between the mount point for busy SWRaid members and the array itself as
used in the device combo box.
# cat /proc/mdstat
Personalities : [raid1]
md1 : active raid1 sda1[2] sdb1[3]
524224 blocks super 1.0 [2/2] [UU]
...
# cat /proc/partitions
major minor #blocks name
8 0 33554432 sda
8 1 524288 sda1
...
8 16 33554432 sdb
8 17 524288 sdb1
...
9 1 524224 md1
...
Bug 756829 - SWRaid member detection enhancements
In cases where blkid wrongly reports a file system instead of an SWRaid
member (sometimes confused by metadata 0.90/1.0 mirror array or old
version not recognising SWRaid members), the UUID and label are
obviously wrong too. Therefore have to use the UUID and label returned
by the mdadm query command and never anything reported by blkid or any
file system specific command.
Example of blkid reporting the wrong type, UUID and label for /dev/sda1
and the correct values for /dev/sdb1:
# blkid | egrep 'sd[ab]1'
/dev/sda1: UUID="10ab5f7d-7d8a-4171-8b6a-5e973b402501" TYPE="ext4" LABEL="chimney-boot"
/dev/sdb1: UUID="15224a42-c25b-bcd9-15db-60004e5fe53a" UUID_SUB="0a095e45-9360-1b17-0ad1-1fe369e22b98" LABEL="chimney:1" TYPE="linux_raid_member"
# mdadm -E -s -v
ARRAY /dev/md/1 level=raid1 metadata=1.0 num-devices=2 UUID=15224a42:c25bbcd9:15db6000:4e5fe53a name=chimney:1
devices=/dev/sda1,/dev/sdb1
...
ARRAY /dev/md127 level=raid1 num-devices=2 UUID=8dc7483c:d74ee0a8:b6a8dc3c:a57e43f8
devices=/dev/sdb6,/dev/sda6
...
NOTES:
* In mdadm terminology the label is called the array name, hence name=
parameter for array md/1 in the above output.
* Metadata 0.90 arrays don't support naming, hence the missing name=
parameter for array md127 in the above output.
Bug 756829 - SWRaid member detection enhancements
Add active attribute to the cache of SWRaid members. Move parsing of
/proc/mdstat to discover busy SWRaid members into the cache loading
code. New parsing code is a little different because it is finding all
members of active arrays rather than determining if a specific member is
active.
Bug 756829 - SWRaid member detection enhancements
Detection of Linux SWRaid members currently fails in a number of cases:
1) Arrays which use metadata type 0.90 or 1.0 store the super block at
the end of the partition. So file system signatures in at least
linear and mirrored arrays occur at the same offsets in the
underlying partitions. As libparted only recognises file systems
this is what is detected, rather than an SWRaid member.
# mdadm -E -s -v
ARRAY /dev/md/1 level=raid1 metadata=1.0 num-devices=2 UUID=15224a42:c25bbcd9:15db6000:4e5fe53a name=chimney:1
devices=/dev/sda1,/dev/sdb1
...
# wipefs /dev/sda1
offset type
----------------------------------------------------------------
0x438 ext4 [filesystem]
LABEL: chimney-boot
UUID: 10ab5f7d-7d8a-4171-8b6a-5e973b402501
0x1fffe000 linux_raid_member [raid]
LABEL: chimney:1
UUID: 15224a42-c25b-bcd9-15db-60004e5fe53a
# parted /dev/sda print
Model: ATA VBOX HARDDISK (scsi)
Disk /dev/sda: 34.4GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number Start End Size Type File system Flags
1 1049kB 538MB 537MB primary ext4 boot, raid
...
2) Again with metadata type 0.90 or 1.0 arrays blkid may report the
contained file system instead of an SWRaid member. Have a single
example of this configuration with a mirrored array containing the
/boot file system. Blkid reports one member as ext4 and the other as
SWRaid!
# blkid | egrep 'sd[ab]1'
/dev/sda1: UUID="10ab5f7d-7d8a-4171-8b6a-5e973b402501" TYPE="ext4" LABEL="chimney-boot"
/dev/sdb1: UUID="15224a42-c25b-bcd9-15db-60004e5fe53a" UUID_SUB="0a095e45-9360-1b17-0ad1-1fe369e22b98" LABEL="chimney:1" TYPE="linux_raid_member"
Bypassing the blkid cache gets the correct result.
# blkid -c /dev/null /dev/sda1
/dev/sda1: UUID="15224a42-c25b-bcd9-15db-60004e5fe53a" UUID_SUB="d0460f90-d11a-e80a-ee1c-3d104dae7e5d" LABEL="chimney:1" TYPE="linux_raid_member"
However this can't be used because if a user has a floppy configured
in the BIOS but no floppy attached, GParted will wait for minutes as
the kernel tries to access non-existent hardware on behalf of the
blkid query. See commit:
18f863151c
Fix long scan problem when BIOS floppy setting incorrect
3) Old versions of blkid don't recognise SWRaid members at all so always
report the file system when found. Occurs with blkid v1.0 on
RedHat / CentOS 5.
The only way I can see how to fix all these cases is to use the mdadm
command to query the configured arrays. Then use this information for
first choice when detecting partition content, making the order: SWRaid
members, libparted, blkid and internal.
GParted shell wrapper already creates temporary blank udev rules to
prevent Linux Software RAID arrays being automatically started when
GParted refreshes its device information[1]. However an administrator
could manually stop or start arrays or change their configuration
between refreshes so GParted must load this information every refresh.
On my desktop with 4 internal hard drives and 3 testing Linux Software
RAID arrays, running mdadm adds between 0.20 and 0.30 seconds to the
device refresh time.
[1] a255abf343
Prevent GParted starting stopped Linux Software RAID arrays (#709640)
Bug 756829 - SWRaid member detection enhancements
When multiple devices are named on the command line and (after sorting
and removing duplicates) the device following a non-existent or invalid
one is not checked for usability [1]. In most situations this isn't
noticed as the device gets skipped at the "Searching ... partitions"
step instead. However as seen in bug 755495 and commit [2]
checking usability matters.
For example (on CentOS 6.5) a large sector disk device can be edited
when it follows a non-existent or invalid device named on the command
line:
# modprobe scsi_debug dev_size_mb=128 sector_size=4096
# fgrep scsi_debug /sys/block/*/device/model
/sys/block/sdd/device/model:scsi_debug
# ./gpartedbin /dev/does-not-exist /dev/sdd
======================
libparted : 2.1
======================
Could not stat device /dev/does-not-exist - No such file or directory.
Device /dev/sdd has a logical sector size of 4096. Not all parts of GNU Parted support this at the moment, and the working code is HIGHLY EXPERIMENTAL.
/dev/sdd: unrecognised disk label
When erasing a device don't skip confirming the following device is
usable.
[1] Usable device as implemented by useable_device()
Must not have a large sector size when GParted is built with an old
version of libparted which doesn't support large sector sizes and
must be able to read the first sector.
[2] 362b2db331
Check disks named on the command line are safe to use too (#755495)
Bug 756434 - GParted dumps core when passing non-existent or invalid
device on the command line
A non-existent or invalid disk device named on the command line caused
two libparted dialogs to be displayed repeatedly on every refresh. This
was because the device was only removed from the 'device_paths' vector
when it wasn't usable [1]; not when it didn't exist or was invalid, when
the libparted ped_device_get() call failed. Fix this.
[1] Usable device as implemented by useable_device()
Must not have a large sector size when GParted is built with an old
version of libparted which doesn't support large sector sizes and
must be able to read the first sector.
Bug 756434 - GParted dumps core when passing non-existent or invalid
device on the command line
Naming a non-existent or invalid disk device on the command line causes
GParted to dump core. Non-existent device looks like this:
# ./gpartedbin /dev/does-not-exist
======================
libparted : 2.4
======================
Could not stat device /dev/does-not-exist - No such file or directory.
Could not stat device /dev/does-not-exist - No such file or directory.
Backtrace has 10 calls on stack:
10: /lib64/libparted.so.0(ped_assert+0x31) [0x7fcfd10b3e61]
9: /lib64/libparted.so.0(+0x3fdfc12a0c) [0x7fcfd10b4a0c]
8: /home/mike/bin/gpartedbin-0.23.0-master-63-g23b5ba4() [0x455028]
7: /home/mike/bin/gpartedbin-0.23.0-master-63-g23b5ba4() [0x455090]
6: /home/mike/bin/gpartedbin-0.23.0-master-63-g23b5ba4() [0x4550d5]
5: /home/mike/bin/gpartedbin-0.23.0-master-63-g23b5ba4() [0x46723f]
4: /usr/lib64/libglibmm-2.4.so.1() [0x3ff5834a8d]
3: /lib64/libglib-2.0.so.0() [0x3fe086a374]
2: /lib64/libpthread.so.0() [0x3fdf407a51]
1: /lib64/libc.so.6(clone+0x6d) [0x3fdf0e893d]
Assertion (dev != NULL) at device.c:227 in function ped_device_open() failed.
Aborted (core dumped)
And with an invalid device the output looks like this:
# ./gpartedbin /dev/zero
======================
libparted : 2.4
======================
The device /dev/zero is so small that it cannot possibly store a file system or partition table. Perhaps you selected the wrong device?
Error fsyncing/closing /dev/zero: Invalid argument
The device /dev/zero is so small that it cannot possibly store a file system or partition table. Perhaps you selected the wrong device?
Error fsyncing/closing /dev/zero: Invalid argument
Backtrace has 10 calls on stack:
...
[Same as above]
Bisected the cause to this commit from 2015-03-09 in GParted 0.22.0. It
claimed to make no functional change. That turned out not to be true.
51ac4d5648
Split get_device_and_disk() into two (#743181)
Fix by simply adding the missed if condition in get_device().
Bug 756434 - GParted dumps core when passing non-existent or invalid
device on the command line
For probed DMRaid devices (when not using libparted DMRaid support)
GParted waits up to 1 second for udev to have processed all events and
created the /dev entries after starting each DMRaid array. This was
added by this commit from 2009-09-02:
e7352a5000
Ensure /dev file system device entries created before adding device
Do the same for devices named on the command line too.
Order named disk devices so that they appear in the combo box in the
same order which they would when probed. Also remove duplicates so that
the same disk devices aren't scanned multiple times and appear
duplicated in the UI.
Try this; it used to take ages to load and looked weird:
# gparted /dev/sda /dev/sdb /dev/sda /dev/sdb /dev/sda /dev/sdb
Bug 755495 - GParted allowing partitioning of large sector devices
specified on the command line, when built with old
libparted which doesn't support it
When probing for disk devices GParted ensures that libparted is capable
of handling the sector size safely and that it is a real disk before it
is shown in the UI. However when disk devices are named on the command
line none of these checks are performed.
Libparted versions before v2.2 can only safely handle a sector size of
512 bytes. Therefore on old distributions with libparted < v2.2 GParted
allows unsafe editing of disk devices with larger sector sizes when they
are named on the command line. Known to affect these distributions:
RedHat/CentOS 5 (parted 1.8.1)
RedHat/CentOS 6 (parted 2.1)
For example (on CentOS 6.5) large sector disk device is ignored when
probing:
# modprobe scsi_debug dev_size_mb=128 sector_size=4096
# fgrep scsi_debug /sys/block/*/device/model
/sys/block/sdd/device/model:scsi_debug
# gparted
======================
libparted : 2.1
======================
Device /dev/sdd has a logical sector size of 4096. Not all parts of
GNU Parted support this at the moment, and the working code is
HIGHLY EXPERIMENTAL.
Ignoring device /dev/sdd with logical sector size of 4096 bytes.
GParted requires libparted version 2.2 or higher to support devices
with sector sizes larger than 512 bytes.
However when the device is named it is not ignored and can be edited:
# gparted /dev/sdd
======================
libparted : 2.1
======================
Device /dev/sdd has a logical sector size of 4096. Not all parts of
GNU Parted support this at the moment, and the working code is
HIGHLY EXPERIMENTAL.
/dev/sdd: unrecognised disk label
Apply the same validity checks to disk devices named on the command line
as to probed ones.
Bug 755495 - GParted allowing partitioning of large sector devices
specified on the command line, when built with old
libparted which doesn't support it
Abstract code checking sector size and ensuring the first sector of a
candidate disk device can be read into new
GParted_Core::useable_device() method.
Bug 755495 - GParted allowing partitioning of large sector devices
specified on the command line, when built with old
libparted which doesn't support it
I missed another case of 'index' may be used uninitialised warning in
OperationDelete::apply_to_visual(). Indent a code block within an if
clause so that the compiler can confirm that the 'index' local variable
isn't used uninitialised. Prevent this compiler warning:
OperationDelete.cc: In member function 'virtual void GParted::OperationDelete::apply_to_visual(std::vector<GParted::Partition, std::allocator<GParted::Partition> >&)':
OperationDelete.cc:34: warning: 'index' may be used uninitialized in this function
Bug 755214 - Refactor operation merging
GParted waits forever when attempting to set a FAT16/32 file system
label which contains prohibited characters [1][2]. This is because
mlabel asks a question and is waiting for input. Force cancelling the
operation doesn't work either as GParted sends signal 2 (interrupt i.e.
[Ctrl-C]) but mtools commands specifically ignores this and a number of
other signals. Have to kill mlabel with signal 9 (kill) to regain
control of GParted.
Mlabel command with prohibited characters in the label:
# export MTOOLS_SKIP_CHECK=1
# mlabel ::"MYLABEL/ " -i /dev/sdb10
Long file name "MYLABEL/ " contains illegal character(s).
a)utorename A)utorename-all r)ename R)ename-all
s)kip S)kip-all q)uit (aArRsSq):
Remove prohibited characters from FAT16/32 file systems labels when
creating and labelling them. Also upper case the label to meet label
requirements [1][2]. This silently corrects the label and the actual
label applied will be displayed when GParted refreshes after applying
the operation.
[1] Microsoft TechNet: Label
https://technet.microsoft.com/en-us/library/bb490925.aspx
[2] Replicated in Wikikedia: label (command)
https://en.wikipedia.org/wiki/Label_%28command%29
Bug 755608 - Labeling fat16/fat32 partitions hangs if certain characters
included in label
These member variables store no Operation class information and were
being used as local variables. Replace with local variables.
Also indent a code block within an if clause so that the compiler can
confirm that the new local variable isn't used uninitialised. Prevents
this compiler warning:
OperationResizeMove.cc: In member function 'void GParted::OperationResizeMove::apply_normal_to_visual(std::vector<GParted::Partition, std::allocator<GParted::Partition> >&)':
OperationResizeMove.cc:125: warning: 'index' may be used uninitialized in this function
Bug 755214 - Refactor operation merging
Since this commit earlier in the patchset the second optional parameter
of method Win_GParted::Add_Operation() is no longer used. Remove it.
Replace open coded merge of resize/move into create operation
(#755214)
Bug 755214 - Refactor operation merging
This is the equivalent case fixed in the earlier commit, but now using
copy/paste to create the second new partition rather than plain new.
Fix visually re-apply create operation in create-create-grow-first
sequence (#755214)
Start with an existing partition as a copy source. Then this sequence
of operations will cause the copy partition to disappear from the disk
graphic:
1) create new #1,
2) copy existing / paste into unallocated leaving space preceding,
3) resize new #1 larger.
There are two different types of copy operation. The first is copy into
unallocated space creating a new partition which needs treating the same
as create new operation. The second is copy into existing partition
which needs treating the same as the other operations which don't change
the boundaries of the partition. Fix apply_to_visual() accordingly.
Bug 755214 - Refactor operation merging
Move the code from OperationCreate::apply_to_visual() into new method
Operation::insert_new() in the parent class. This is in preparation for
the following commit.
Bug 755214 - Refactor operation merging
The apply_to_visual() method for the change UUID, format, label file
system and name partition operations duplicated identical code. This
code was just substituting the partition in the disk graphic vector with
the new partition recorded in the operation, as none of these operations
change the partition boundaries. Move this duplicate code into the
parent class in new method Operation::substitute_new().
Bug 755214 - Refactor operation merging
After previous commit "Replace open coded merge of resize/move into
create operation (#755214)" the second created partition would disappear
from the disk graphic in the following sequence: create new #1, create
new #2 leaving space preceding, resize #1 larger. The create new #2
operation still existed and was shown in the operation list. It was
just that it disappeared from the disk graphic.
Remember that when each operation is created it records the partition,
or the unallocated space, to which the operation is applied at the time
the operation is created in the partition_original member variable. In
the above sequence the resize #1 larger operation was merged back into
the create new #1 operation. When visually re-applying the create
new #1 operation to the disk graphic, it left a smaller unallocated
partition following it. This was smaller than the unallocated partition
recorded in the create new #2 operation, hence it failed to visually
re-apply to the disk graphic.
The insight to fix this is that it doesn't matter what size the
unallocated space was when the create new operation was constructed. It
only matters that the new partition to be created fits in the available
unallocated space currently in the disk graphic.
Bug 755214 - Refactor operation merging
This information is already documented in the existing comments
associated with the calls to merge_operations() and assignments to the
mergetype variables. The table just summaries the rules together in one
place.
Bug 755214 - Refactor operation merging
For the case of resizing/moving a new, not yet created partition,
activate_resize() open coded the merge operation. Again this code has
existed in GParted since before version 0.0.5 and the current code
history in Git.
Replace the necessary code so that an explicit merge_operations() call
is used instead; along with the other case of resizing/moving an
existing partition.
NOTES:
This commit changes the merge direction. The old coded merged forward
by removing the old create operation and adding a new create operation
with the new size. This was bad because with multiple pending create
operations, each merged resize operation reordered those create
operations. Then when the operations were applied the partitions were
created and therefore numbered in a different order to that shown in
disk graphic.
The new code merges backwards by updating the initial create operation
with the new size. This maintains the create operation order so that
when applied the partitions are numbered in the same order as shown in
the disk graphic.
Bug 755214 - Refactor operation merging
For the case of formatting a new, not yet created partition,
activate_format() open coded the merge operation. This code has existed
in GParted since before version 0.0.5 and the current code history in
Git.
Replace the necessary code so that an explicit merge_operations() call
is used instead; along with the other case of formatting an existing
partition.
Bug 755214 - Refactor operation merging
Creation of the various operations involved various implicit rules about
how the different types of operations were merged in different cases.
This was open coded in each ::activate_*() method. Abstract this into
new merge_operations() method and make the merging rules explicitly
specified.
NOTE:
The removal of operation type checking in the MERGE_LAST_WITH_ANY cases
is not a problem because all the Operation*::merge_operations() methods
ensure the operation types match as part of the merge attempt.
Bug 755214 - Refactor operation merging
Rename Win_GParted::Merge_Operations() to merge_two_operations(). To
reflect what it does and in preparation for further refactoring of the
code.
Be more strict on the validation of the first and second indexes. The
first operation must also be before the second operation in the
operation[] vector. (It is actually a programming bug if first and
second fail validation. However so far g_assert() is only being used to
validate pointers, which if wrong would likely cause the program to
eventually crash when dereferenced later. In this case a bug would
merely cause the incorrectly specified pair of operations to not be
merged).
Move validate_display_partition_ptr() declaration in the header file to
be in the same ordering as it's definition in the source file.
Bug 755214 - Refactor operation merging
Win_GParted::Merge_Operations() method was modifying the internals of
Operation* objects; in particular the partition_new member variable.
This is breaking data hiding and encapsulation tenant of object oriented
programming.
Implement exactly the same operation merge semantics, but hide the
manipulation of the internals of the Operation* objects within the
Operation* classes themselves.
Bug 755214 - Refactor operation merging
... before refactoring the code.
See the commit message from 2011-10-05 for details of what operations,
available at that time, on the same partition can be merged and in what
cases:
b10349ae37
Merge overlapping operations (#438573)
Bug 755214 - Refactor operation merging
As previous commit, display_partitions is now a Win_GParted member
variable so checking for the existence of an extended partition can be
localised where it is used.
Remove index_extended member variable and localise the same checking in
activate_new().
Now that display_partitions is a Win_GParted member variable and
therefore available throughout the class, since commit [1], calculation
of primary_count can be localised in max_amount_prim_reached() where it
is used.
Implements a FIXME and removes primary_count as a member variable.
[1] 545b75d957
Move vector of partition objects to a Win_GParted class member (#750168)
Perform a copy, reformat source and paste sequence in GParted. When the
source is a primary partition everything works as expected, with the
newly pasted partition reflecting the reformatted source. However when
the source is a logical partition GParted thinks it is pasting the
original source, rather than the reformatted source. The same is also
true for other file system manipulation operations: resize, file system
label and new UUID. It is just that reformatting the source to a
different file system type is the most obvious in the UI and causes the
most significantly wrong actions to be performed.
For example start with an ext4 logical partition, select it for copy,
format it to xfs and paste into a new partition. GParted thinks the
second operation will create a copy of an ext4 file system instead of
the xfs file system. When applied the operation details are:
Format /dev/sdd5 as xfs
+ calibrate /dev/sdd5
+ clear old file system signatures in /dev/sdd5
+ set partition type on /dev/sdd5
+ create new xfs file system
+ mkfs.xfs -f -L "" /dev/sdd5
Copy /dev/sdd5 to /dev/sdd (start at 131.00 MiB)
+ calibrate /dev/sdd5
+ check file system on /dev/sdd5 for errors and (if possible) fix them
+ e2fsck -f -y -v -C 0 /dev/sdd5
e2fsck: Subperblock invalid, trying backup blocks...
Resize inode not valid. Recreate? yes
...
/dev/sdd5: ***** FILE SYSTEM WAS MODIFIED *****
+ create empty partition
+ set partition type on /dev/sdd6
new partition type: ext4
+ copy file system of /dev/sdd5 to /dev/sdd6
using internal algorithm
...
GParted formatted sdd5 to xfs, but then the copy step ran e2fsck and
managed to resurrect the ext4 file system and then performed a block
copy of it to partition sdd6. The copy step should have ran xfs_repair
and used xfsdump | xfsrestore to copy the xfs file system. Afterwards
sdd5 contains both xfs and ext4 signatures.
# wipefs /dev/sdd5
offset type
----------------------------------------------------------------
0x438 ext4 [filesystem]
UUID: f0ed4247-76db-4d93-b3bc-c7da4a70f95e
0x0 xfs [filesystem]
UUID: 1ac8e7c3-0311-4c64-8e4a-b715a23ea0bd
This has been broken at least as far back as GParted 0.1.0.
Fix by simply refreshing the copy source partition object when it is a
logical partition too, as well as when it is a primary partition.
Bug 754827 - Copy, reformat source and paste a logical partition thinks
it's pasting the original file system
Command exit status is a 1 byte value between 0 and 255. [1][2] However
at the Unix API level the value is encoded as documented in the
waitpid(2) manual page. This is true for the Glib API too. [3] This is
why, for example, the comment in ext2::check_repair() reported receiving
undocumented exit status 256. It was actually receiving exit status 1
encoded as per the waitpid(2) method.
Add shell style exit status decoding [2] to execution of all external
commands. Return value from Utils::execute_command() and
FileSystem::execute_command() functions are now:
0 - 125 - Exit status from the command
126 - Error executing the command
127 - Command not found
128+N - Command terminated by signal N
255 - Unexpected waitpid(2) condition
Also adjust checking of the returned statuses as necessary.
[1] Advanced Bash-Scripting Guide: Appendix D. Exit Codes With Special
Meanings
http://www.linuxtopia.org/online_books/advanced_bash_scripting_guide/exitcodes.html
[2] Quote from the bash(1) manual page:
EXIT STATUS
... Exit statuses fall between 0 and 255, though as
explained below, the shell may use values above 125
specially. ...
... When a command terminates on a fatal signal N, bash uses
the value of 128+N as the exit status.
If a command is not found, the child process created to
execute it returns a status of 127. If a command is found
but is not executable, the return status is 126.
[3] Quote from the Glib Reference Manual, Spawning Processes section,
for function g_spawn_check_exit_status():
https://developer.gnome.org/glib/stable/glib-Spawning-Processes.html#g-spawn-check-exit-status
The g_spawn_sync() and g_child_watch_add() family of APIs return
an exit status for subprocesses encoded in a platform-specific
way. On Unix, this is guaranteed to be in the same format
waitpid() returns, ...
Bug 754684 - Updates to FileSystem:: and Utils::execute_command()
functions
Replace open coding of the creation of the operation details for the
mlabel command used to set the label and UUID with calls to
FileSystem::execute_command() which will do it all. This also results
in the commands getting a time and check mark displayed in the operation
details.
Bug 754684 - Updates to FileSystem:: and Utils::execute_command()
functions
There has been an undocumented rule that external commands displayed in
the operation details, as part of file system manipulations, only get a
time and check mark displayed when multiple commands are needed, and not
otherwise. (GParted checks whether all commands are successful or not
regardless of whether a check mark is displayed in the operation details
or not).
EXCEPTION 1: btrfs resize
Since the following commit [1] from 2013-02-22, GParted stopped
displaying the timing for the btrfs resize command in the operation
details. It being part of a multi-command sequence to perform the step.
This is because FileSystem::execute_command() since the commit can only
check the exit status for zero / non-zero while timing and checking the
command status but btrfs resize needs to consider some non-zero statuses
as successful.
[1] 52a2a9b00a
Reduce threading (#685740)
EXCEPTION 2: ext2/3/4 move and copy using e2image
When use of e2image was added [2] the single command steps were timed
and check.
[2] 86111fe12a
Use e2image to move/copy ext[234] file systems (#721516)
EXCEPTION 3: fat16/32 write label and UUID
Uses Utils::execute_command() rather than FileSystem::execute_command()
so can be separately changed. See the following commit for resolution
of the final commands not yet timed and check mark displayed.
CHANGE:
Lets make a simpler rule of always displaying the time and a check mark
for all external commands displayed in the operation details. However
this makes several of the other single command actions need special exit
status handling because zero success, non-zero failure is not correct
for every case. Specifically affects resizing of reiserfs and check
repair of ext2/3/4, fat16/32, jfs and reiserfs.
After this change all external commands run as file system actions must
follow one of these two patterns of using the EXEC_CHECK_STATUS flag or
separately calling FileSystem::set_status() to register success or
failure of the command:
exit_status = execute_command(cmd, od, EXEC_CHECK_STATUS...);
or:
exit_status = execute_command(cmd, od, ...);
bool success = (exit_status == 0 || exit_status == OTHER_SUCCESS_VALUE...);
set_status(od, success );
Bug 754684 - Updates to FileSystem:: and Utils::execute_command()
functions
Change the two optional boolean parameters into a single optional flags
parameter which uses symbolically defined names. Makes reading the
execute_command() calls much easier to understand. (Implemented as bit
field using the same technique as used for Glib::SpawnFlags [1]).
This changes the calls thus:
execute_command(cmd, od) -> (cmd, od)
execute_command(cmd, od, false) -> (cmd, od, EXEC_NONE) // [2]
execute_command(cmd, od, true ) -> (cmd, od, EXEC_CHECK_STATUS)
execute_command(cmd, od, false, true) -> (cmd, od, EXEC_CANCEL_SAFE)
execute_command(cmd, od, true , true) ->
(cmd, od, EXEC_CHECK_STATUS|EXEC_CANCEL_SAFE)
[1] SpawnFlags bitwise operators in
/usr/include/glibmm-2.4/glibmm/spawn.h.
[2] False and EXEC_NONE are the default values for the optional third
parameter before and after this change respectively and both mean
the same. This is being used in btrfs::resize() and being kept for
now despite it being the default.
Bug 754684 - Updates to FileSystem:: and Utils::execute_command()
functions
Add a pattern to recognise Non-Volatile Memory Express devices as valid
devices to work with. Devices are named by the Linux kernel device
driver like /dev/nvme0n1 [1] with partitions named like /dev/nvme0n1p1
[2].
[1] linux 3.3 drivers/block/nvme.c nvme_alloc_ns()
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/drivers/block/nvme.c?id=v3.3#n1351
[2] Contents of /proc/partitions for a partitioned NVME device
$ grep nvme /proc/partitions
259 0 390711384 nvme0n1
259 1 977 nvme0n1p1
259 2 31250000 nvme0n1p2
259 3 328209496 nvme0n1p3
259 4 31249408 nvme0n1p4
Bug 755022 - gparted doesn't recognize nvme devices
The ntfs-3g package previously provided the FUSE based NTFS driver to
mount the file system and ntfsprogs provided the user space tools. In
April 2011 the packages have merged [1] forming ntfs-3g_ntfsprogs. Arch
Linux / Debian / Slackware / Ubuntu now just have an ntfs-3g package
with everything; where as CentOS / Fedora / openSUSE are sticking with
the original two package names. Reverse the order of the needed
packages to:
ntfs-3g / ntfsprogs
[1] Release: NTFS-3G + NTFSPROGS 2011.4.12
http://www.tuxera.com/release-ntfs-3g-ntfsprogs-2011-4-12/
For reiserfs the upstream package is named reiserfsprogs. Arch Linux /
Debian / Slackware / Ubuntu use this name; but CentOS / Fedora name it
reiserfs-utils and openSUSE names it just reiserfs. Update the README
file with all 3 package names.
reiserfsprogs / reiserfs-utils / reiserfs
In the File System Support dialog just use the first 2 package names as
we don't want to make the dialog too wide with all 3 names and hopefully
on openSUSE it is more obvious that the reiserfs package is needed to
support the reiserfs file system.
NOTE:
Again this slightly increases the width of the File System Support
dialog on my CentOS 6 desktop with default fonts, now from 676 to 707
pixels. Again still well within the 800 wide target and still narrower
than the main window.
Bug 753436 - Update documentation of GParted software dependencies
In Fedora bug report:
Bug 1176108 - Warning shown on BTRFS partition because of missing btrfs-tools package
https://bugzilla.redhat.com/show_bug.cgi?id=1176108#c0
The user said:
However there is no btrfs-tools package in the standard Fedora repo.
There is a btrfs-progs package, which is already installed. It's
unclear whether this is a real error or simply a mismatched package
name.
The upstream software is named btrfs-progs. Arch Linux / CentOS /
Fedora / Slackware use the upstream name. Debian / Ubuntu name it
btrfs-tools and openSUSE calls it btrfsprogs (no dash).
Rename the needed software to:
btrfs-progs / btrfs-tools
Upstream name first separated by slash from alternative names
distributions use.
NOTE:
This slightly increases the width of the File System Support dialog on
my CentOS 6 desktop with default fonts, from 655 to 676 pixels. Still
well within the 800 wide target and still narrower that the main window.
Bug 753436 - Update documentation of GParted software dependencies
This message would be displayed as part of the details for an operation
to create or format a partition as LVM2 PV on a disk using partition
table types dvh or pc98, which don't have an lvm flag.
v Format /dev/sdb1 as lvm2 pv
> calibrate /dev/sdb1
> clear old file system signatures in /dev/sdb1
> set partition type on /dev/sdb1
Skip setting unsupported partition flag: lvm
> create new lvm2 pv file system
Translators found this message difficult to translate, as reported in
bug 752901 - Add translator comment or fix string. It is also only
standard practice to set the lvm flag on partitions containing LVM2 PVs,
and not required. In hindsight reporting that an operation was skipped,
when the operation is not a necessity, is probably bad practice as it
could lead to uncertainty on behalf of the user and questions as to why
the flag isn't supported. Just remove the message.
Correct the colour name comment for NILFS2.
Colour for LVM2 PV was a Medium Brown. This is close to Face Skin Dark
from the GNOME colour palette. Use this instead.
Make colour for ReFS a bit darker so it is more distinct from the colour
for NTFS.
Add translation comments to improve the clarity of some translatable
text strings regarding setting partition flags.
Bug 752901 - Add translator comment or fix string
Opening the Resize/Move dialog on a logical partition causes GParted to
crash. This crash affects current GParted GIT HEAD, but does not affect
GParted 0.22.0. Git bisect identifies that it was broken with the
following commit:
Remove Set_Data() from the copy, resize/move and new dialog class APIs
7a4a375ed6
The problem was trying to treat the reference display_partitions_ref
like a pointer, and in particular on line 1732 trying to make it refer
to the a different vector of partitions, .logicals sub-vector.
1721 void Win_GParted::activate_resize()
1722 {
...
1726 std::vector<Partition> & display_partitions_ref = display_partitions;
1727 if ( selected_partition_ptr->type == TYPE_LOGICAL )
1728 {
1729 unsigned int ext = 0 ;
1730 while ( ext < display_partitions.size() && display_partitions[ext].type != TYPE_EXTENDED )
1731 ext++;
* 1732 display_partitions_ref = display_partitions[ext].logicals;
1733 }
1734
1735 Dialog_Partition_Resize_Move dialog( gparted_core.get_fs( selected_partition_ptr->filesystem ),
1736 *selected_partition_ptr,
1737 display_partitions_ref );
What was actually happening was that the .logicals sub-vector was being
copied, replacing the display_partitions vector and freeing the original
sub-vector. This left selected_partition_ptr pointing to the original
memory where the selected partition use to exist in the .logicals
sub-vector. At some point in the Dialog_Partition_Resize_Move class
*selected_partition_ptr was referenced, accessing the freed memory.
Crash soon followed.
Fix by using a pointer instead of a reference, which can be assigned to
point to a different object.
Bug 752587 - GParted crashing when opening Resize/Move dialog on
logical partition
Since GParted commit 52a2a9b "Reduce threading (#685740)", released in
GParted 0.15.0, application of operations occurs in the main thread
running the UI, therefore long running libparted actions such as
resizing a FAT16 or FAT32 file system hang the UI for as long as it take
to complete the operation.
https://git.gnome.org/browse/gparted/commit/?id=52a2a9b00a32996921ace055e71d0e09fb33c5fe
Though this problem exists for all libparted actions, it is particularly
noticeable when performing a large resize of fat16/fat32/hfs/hfs+ file
systems.
To address this significant cause of an unresponsive GUI, this
enhancement adds threading to the libparted ped_file_system_resize
function call.
Bug 737022 - UI hangs while running libparted operations such as
FAT16/FAT32 resizing
GParted was also searching the PATH for the availability of the udevadm
and udevsettle commands for every device with a busy partition during a
refresh and for every applied operation effecting a partition table. As
with hdparm previously this was wasteful.
Again, remember the result of searching the PATH at startup and refresh
when clicking on the [Rescan For Supported Actions] button in the File
System Support dialog.
Previously on every refresh for every device, GParted was searching the
PATH to discover if the hdparm command existed. Stracing GParted showed
that calling Glib::find_program_in_path("hdparm") made the following OS
calls:
access("/usr/lib64/qt-3.3/bin/hdparm", X_OK) = -1 ENOENT (No such file or directory)
access("/usr/local/sbin/hdparm", X_OK) = -1 ENOENT (No such file or directory)
access("/usr/local/bin/hdparm", X_OK) = -1 ENOENT (No such file or directory)
access("/sbin/hdparm", X_OK) = 0
getuid() = 0
stat("/sbin/hdparm", {st_mode=S_IFREG|0755, st_size=137, ...}) = 0
stat("/sbin/hdparm", {st_mode=S_IFREG|0755, st_size=137, ...}) = 0
The Linux VFS is very fast but repeatedly doing this is wasteful.
Remember the result of searching the PATH for the hdparm command at
startup and refresh this when the [Rescan For Supported Actions] button
is pressed in the File System Support dialog. This is the same as
GParted already does for file system specific commands and their
capabilities.
Bug 751251 - Show serial number in device information
With a Kobo Touch eReader connected via USB, GParted would hang when
scanning the device with this error written to the terminal:
$ sudo src/gpartedbin
======================
libparted : 2.3
======================
(gpartedbin:10261): glibmm-CRITICAL **:
unhandled exception (type Glib::Error) in signal handler:
domain: g_convert_error
code : 1
what : Invalid byte sequence in conversion input
The hdparm command was printing binary data as the serial number.
Fragment of the 'hdparm -I /dev/sdf' output:
# hdparm -I /dev/sdf
/dev/sdf:
SG_IO: bad/missing sense data, sb[]: 70 00 05 00 00 00 00 0a 00 00 00 00 24 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
ATAPI Optical card reader/writer, with non-removable media
Model Number: {BINARY_DATA}
Serial Number: {BINARY_DATA}
Firmware Revision: {BINARY_DATA}
GParted reads command output using the Glib::IOChannel class. However
by default an IOChannel performs character set conversion on the data it
reads, so when it came across an invalid byte sequence in the binary
data the above exception was raised and the IOChannel::read() method
never returned. Hence GParted became stuck reportedly scanning the
same device forever. Code fragment:
src/PipeCapture.cc
49 bool PipeCapture::OnReadable( Glib::IOCondition condition )
50 {
...
58 Glib::ustring str;
>> 59 Glib::IOStatus status = channel->read( str, 512 );
60 if (status == Glib::IO_STATUS_NORMAL)
61 {
62 for( Glib::ustring::iterator s = str.begin(); s != str.end(); s++ )
Quote from the IOChannel class reference:
https://developer.gnome.org/glibmm/stable/classGlib_1_1IOChannel.html
Note that IOChannels implement an automatic implicit character set
conversion to the data stream, and usually will not pass by default
binary data unchanged. To set the encoding of the channel, use
e.g. set_encoding("ISO-8859-15"). To set the channel to no encoding,
use set_encoding() without any arguments.
Fix by disabling the automatic character set conversion in the IOChannel
used to read output from executed commands.
Bug 751251 - Show serial number in device information
Run "hdparm -I /dev/DISK" to get the hard drive serial number of
every device which has one and display it in the Device Information.
The displayed value can either be the actual serial number, "none" or
blank. "none" means the device doesn't have a hard drive serial number,
such as for Linux software RAID arrays, BIOS fake RAID arrays or USB
flash drives. Blank means something went wrong getting the serial
number. Either it couldn't be found in the hdparm output or the hdparm
command wasn't installed.
Example real hard drive:
# hdparm -I /dev/sda
...
ATA device, with non-removable media
Model Number: SAMSUNG HM500JI
Serial Number: S1WFJDSZ123732
...
Example Linux software RAID array:
# hdparm -I /dev/md127
/dev/md127:
HDIO_DRIVE_CMD(identify) failed: Inappropriate ioctl for device
On my desktop with 4 internal hard drives 2 Linux software RAID arrays
on those hard drives, 2 USB flash drives and 1 USB hard drive attached,
running hdparm 9 times added 0.07 seconds to the device refresh time.
Bug 751251 - Show serial number in device information
Btrfs-progs 4.1, released June 2015, includes support for changing the
UUID of a btrfs file system using the btrfstune command. Check for
availability by looking for the -u option in the btrfstune help output.
Use btrfstune like this:
# umount /dev/sdb1
# btrfstune -f -u /dev/sdb1
Current fsid: e7ad5dba-d721-4f99-990b-1ba2901c8ad2
New fsid: 231563d9-e173-410d-b1da-d34c4319a423
Set superblock flag CHANGING_FSID
Change fsid in extents
Change fsid on devices
Clear superblock flag CHANGING_FSID
Fsid change finished
# echo $?
0
Bug 751337 - btrfstune in btrfs-progs 4.1 supports changing the file
system UUID
Explain why the implementation uses a clear_cache() call, rather than a
straight load_cache() call. This commit from 2014-02-17 implemented
incremental loading of the btrfs device cache:
76e64f2905
Detect busy status of multi-device btrfs file systems (#723842)
The lvm query commands were always run and the cache loaded even if
GParted, actually blkid, didn't identify any LVM2 PVs. (GParted uses
libparted and blkid to identify partition content and the lvm commands
to provide the needed configuration details).
Now implement complete lazy initialization of the cache. Never force
loading of the cache. The cache is only loaded when the first value is
accessed from it. When there are no LVM2 PVs, the cache is never
queried, so never loaded. All the needed infrastructure for delayed
loading was previously added by this commit from 2011-12-11:
ff8ad04120
Lazy initialize the cache from querying LVM2 PVs (#160787)
Every public member function which access values from the cache already
calls initialize_if_required(). Just need to replace force loading of
the cache with a function which just clears the cache.
On my desktop, only when there are no LVM2 PVs, not loading the cache
and therefore not executing these external commands in
load_lvm2_pv_info_cache() saves 1.0 seconds of the 3.7 seconds it takes
to perform the a refresh in GParted:
lvm vgscan
lvm pvs ... -o pv_name,...
lvm pvs ... -o vg_name,...
Bug 750582 - Refactor the LVM2_PV_Info module object interface and
internal cache representation
GParted used to cache the results of the "lvm pvs" commands used to query
the state of the Logical Volume Manager as a series of lines of text.
Then every time a particular value was queried GParted would split all
the lines of text into fields until the required value was found.
Stop this repeat splitting of cached lines of text. Instead parse the
lines of text into separate fields and store in structures of values of
the correct type in the cache.
Bug 750582 - Refactor the LVM2_PV_Info module object interface and
internal cache representation
The LVM2_PV_Info cache had a pretend multi-object interface, yet all the
data is static. An LVM2_PV_Info object doesn't contain any member
variables, yet was needed just to call the member functions.
Make all the member functions static removing the need to use any
LVM2_PV_Info objects.
Bug 750582 - Refactor the LVM2_PV_Info module object interface and
internal cache representation
An LVM2_PV_Info object contains no member variables as all the data is
static (exists once in the program and accessed by all objects). The
constructor did nothing, except when passed true to load the cache.
Provide a separate load_cache() member function and remove the
constructors and destructor which do nothing. The C++ compiler will
provide a default constructor and destructor, which don't do anything as
there are no member variables to initialise and finalise.
This makes the interface a little easier to understand. Mostly a step
along the way of refactoring how the LVM2_PV_Info cache module works.
Bug 750582 - Refactor the LVM2_PV_Info module object interface and
internal cache representation
The copy, resize/move and new dialog classes (Dialog_Partition_Copy,
Dialog_Partition_Resize_Move and Dialog_Partition_New respectively) had
to be used like this:
construct dialog object passing some parameters
call Set_Data() to pass more parameters
run() dialog
call Get_New_Partition()
There is nothing in the classes which forces Set_Data() to be called,
but it must be called for the dialogs to work and prevent GParted from
crashing.
Make these class APIs safer by making it impossible to program
incorrectly in this regard. Move all the additional parameters from
each Set_Data() method to each constructor. The constructors just call
the now private set_data() methods.
The member variable was named selected_partition. It is assigned from
Win_GParted::selected_partition_ptr (which is a pointer to a const
partition object so is never updated). This gives connotations that it
won't be modified.
However it is updated freely as the new resultant partition object is
prepared before being returned from the dialog, most notable in the
Get_New_Partition() methods.
Therefore rename from selected_partition to new_partition.
The code goes like this:
Dialog_Partition_Copy::Get_New_Partition()
call Dialog_Base_Partition::Get_New_Partition()
Update this->selected_partition with results from running
the dialog.
return this->selected_partition by value.
Save value back to this->selected_partition.
Update this->selected_partition some more.
return this->selected_partition by value.
So there is an unnecessary copy of the partition object returned from
the base class Get_New_Partition() function back to the same variable in
the derived copy class Get_New_Partition() function.
Need to keep the base class Get_New_Partition() function as derived
class Dialog_Partition_Resize_Move uses that implementation as it
doesn't override it, and it's part of the interface.
Avoid this unnecessary copy by moving base class Get_New_Partition()
code into a new private function, called prepare_new_partition(), which
doesn't return anything. Then have Get_New_Partition() in both classes
just return the required partition object. Like this:
Dialog_Base_Partition::Get_New_Partition()
call prepare_new_partition()
return this->selected_partition by value.
Dialog_Partition_Copy::Get_New_Partition()
call Dialog_Base_Partition::prepare_new_partition()
Update this->selected_partition some more.
return this->selected_partition by value.
Bug 750168 - Reduce the amount of copying of partition objects
When Operation objects are created they take a copy of the Device object
to which the operation is to be applied. The Device object includes a
vector of all the contained Partition objects currently on the device,
so these get copied too.
These additional deep copied Partition objects in the Operation object
are never accessed. Therefore don't copy the contained Partition
objects when copying the Device object into the Operation object.
Bug 750168 - Reduce the amount of copying of partition objects
When opening the Manage Flags dialog, creation of the dialog object was
creating a copy of the selected partition object. If this was an
extended partition it also included recursively constructing the
contained logical partitions too.
Instead, replace the partition object in the DialogManageFlags class
with a reference to it.
Bug 750168 - Reduce the amount of copying of partition objects
When opening the Partition Information dialog, creation of the dialog
object was creating a copy of the partition object to be displayed. If
this was an extended partition it also included recursively constructing
the contained logical partitions too.
Instead, replace the partition object in the Dialog_Partition_Info class
with a reference to it.
NOTE:
In C++ a reference is really just a pointer under the hood. As such,
dereferences of a pointer to an object in the context of needing a
reference to the object doesn't copy the object. It merely initialises
the reference from the pointer.
Specifically, with this prototype:
Dialog_Partition_Info( const Partition & partition );
and the dialog object being constructed in Win_GParted::activate_info():
Dialog_Partition_Info dialog( *selected_partition_ptr );
the partition object is not copy constructed. A reference (pointer) to
it is merely passed to the dialog constructor.
Bug 750168 - Reduce the amount of copying of partition objects
The first actions of Win_GParted::activate_resize() were to create a
copy of the vector of partitions for the currently displayed device and
visually apply any pending operations. Exactly this has already been
done in Win_GParted::Refresh_Visual() with the result now available in
the member variable display_partitions. Stop this unnecessary partition
object copying and processing by just using display_partitions member
variable instead.
Bug 750168 - Reduce the amount of copying of partition objects
Document how GParted displays partitions in the GUI and manages the
lifetime and ownership of that data.
Bug 750168 - Reduce the amount of copying of partition objects
Mike Fleetwood