GParted_Core and Operation classes both have an insert_unallocated()
method which do the same thing with very nearly identical code. Both
methods insert unallocated partitions into the vector of partitions
within the specified range of sectors to fill in any gaps larger than
1 MiB. The only difference was how the two methods got the device path;
the GParted_Core class method got it via a parameter and the Operation
class method got it by calling get_path() on its device member variable.
The GParted_Core insert_unallocated() method gets called during device
scanning and the Operation one gets called when constructing the visual
for a pending operation.
Consolidate down to a single insert_unallocated() implementation by
making the Operation class method call the GParted_Core class method.
Make the GParted_Core class method static and public so that it can be
called using the class name from outside the class.
Bug 759726 - Implement Partition object polymorphism
The current code uses push_back() and insert() to copy Partition objects
into the vector of pointers. This has a few issues:
1) Unnecessary copying of Partition objects;
2) Hides the nature of the PartitionVector class as a manager of
pointers to Partition objects by providing copy semantics to add
items. It is generally better to be explicit;
3) C++ doesn't provide polymorphic copy construction directly, but this
is easily worked around by following the Virtual Constructor idiom
[1], which would allow PartitionLUKS derived class objects to be
copied into the vector.
Add push_back_adopt() and insert_adopt() methods which add a pointer to
a Partition object into the PartitionVector adopting ownership.
[1] Wikibooks: More C++ Idioms / Virtual Constructor
https://en.wikibooks.org/wiki/More_C%2B%2B_Idioms/Virtual_Constructor
Bug 759726 - Implement Partition object polymorphism
The PartitionVector class is now internally using pointers to Partition
objects and taking on management of their lifetimes. It therefore has
to implement the Big 3: destructor, copy constructor and copy assignment
operator [1][2]. This is because the implicitly-defined copy
constructor and assignment operator perform memberwise "shallow copying"
and the destructor does nothing. This not correct for classes which
contain non-class types such as raw pointers.
The semantics of the interface still copies each Partition object into
the PartitionVector when they are added with push_back() and insert().
Note that a PartitionVector object is explicitly copy assigned in
Win_GParted::Refresh_Visual(). They are also implicitly copied when
(1) the implementing vector is resized larger to allow it to hold more
pointers to Partition objects than it previously had capacity for; and
(2) a Partition object is copied including the logicals PartitionVector
member.
[1] The rule of three/five/zero
http://en.cppreference.com/w/cpp/language/rule_of_three
[2] Rule of Three
https://en.wikipedia.org/wiki/Rule_of_three_%28C%2B%2B_programming%29
Bug 759726 - Implement Partition object polymorphism
Lots of files which use the Partition class relied on the declaration
being included via other header files. This is bad practice.
Add #include "Partition.h" into every file which uses the Partition
class which doesn't already include it. Header file #include guards are
specifically to allow this.
get_partitions() method was returning a vector of partitions. However
the calling code only needed to know whether any partitions were found
or not. Replace with found_partitions() method reporting the needed
boolean.
Now use of std::vector<Partition> partitions is hidden within the
Dialog_Rescue_Data class implementation.
Bug 759726 - Implement Partition object polymorphism
Just creates PartitionVector class and includes it in partition.h so
that it is built and validated by the compiler. Not used anywhere yet.
Implementation strategy is to create a PartitionLUKS class derived from
the Partition class. This implies polymorphism of Partition objects,
which in C++ requires using pointers and references to objects, and not
using objects directly. (See C++ object slicing). Later this
PartitionVector class will be modified to use pointers to Partition
objects and act as the owner of the pointed to Partition objects.
Bug 759726 - Implement Partition object polymorphism
Debian has deprecated the use of bzip2 for tarballs since dpkg-1.17.7,
released 2014-04-21. See:
https://lintian.debian.org/tags/uses-deprecated-compression-for-data-tarball.html
The choices going forward were to use gzip for maximum compatibility and
speed, or to use xz for maximum compression.
Since we strive for backwards compatibility, gzip was chosen.
Bug 760099 - Revert tarball compression to gzip
GParted was also displaying the GPT partition names as the file system
labels for some type of file systems.
Create 2 empty partitions on a GPT partitioned disk and name them like
this. Then create a hfsplus file system with an empty label. (Actually
single space character but blkid treats it as empty. Can't use GParted
for this because when no label is specified mkfs.hfsplus sets the label
to "untitled").
# sgdisk -p /dev/sdb
/dev/sdb: 4194304 sectors, 2.0 GiB
Logical sector size: 512 bytes
...
Number Start (sector) End (sector) Size Code Name
1 2048 526335 256.0 MiB 8300 empty partition
2 526336 1050623 256.0 MiB 8300 hfsplus partition
# mkfs.hfsplus -v " " /dev/sdb2
Even though only the GPT partition names are set to "SOMETHING
partition", GParted will display them as the file system labels too.
Also in the Information dialog for the empty partition a file system
UUID is displayed as well even though none exists.
Blkid output looks like this:
# blkid -V
blkid from util-linux 2.27.1 (libblkid 2.27.0, 02-Nov-2015)
# blkid | grep /dev/sdb
/dev/sdb1: PARTLABEL="empty partition" PARTUUID="bf3d2085-65b7-4ae6-97da-5ff968ad7d2c"
/dev/sdb2: UUID="2c893037-ff76-38f2-9158-2352ef5dc8de" TYPE="hfsplus" PARTLABEL="hfsplus partition" PARTUUID="457e9c2b-e4f2-4235-833b-28208592aaac"
With blkid from util-linux >= 2.22, released September 2012, it is
including additional PARTLABEL and PARTUUID name and value pairs for
GPT partitions [1]. The FS_Info module regular expressions used to
match the file system LABEL and UUID names also happen to match these
new PARTLABEL and PARTUUID names too. Hence this bug when GParted has
to fall back to using the FS_Info module to read the file system label,
when there is no working file system specific method. Effects: exfat,
f2fs, hfs, hfsplus, ufs, unknown.
Fix by requiring all the regular expressions used to search the blkid
output to also match the space character in front of the name.
[1] Util-linux 2.22 Release Notes
https://git.kernel.org/cgit/utils/util-linux/util-linux.git/tree/Documentation/releases/v2.22-ReleaseNotes?h=v2.22
Bug 759972 - GParted displays partition names also as file system labels
with new blkid for some file systems
In the UI new partitions were being named "unallocated" instead of
"New Partition #1", etc. Also deleting a new partition not yet created
deletes all new partitions from the operation list because it matches
by name only and they were all named "unallocated". Broken by this
recent commit:
762cd1094a
Return class member from Dialog_Partition_New::Get_New_Partition() (#757671)
Prior to this commit in the create new partition dialog code did these
relevant steps:
Dialog_Partition_New::Get_New_Partition()
Partition part_temp;
...
part_temp.Set(..., "New Partition #%1", ...);
Create local Partition object using default constructor which calls
Partition::Reset() and clears the paths vector. It then calls Set()
which appends the new name making the vector:
paths = ["New Partition #%1"]
After the above commit the code did these steps:
Dialog_Partition_New::Dialog_Partition_New(..., selected_partition, ...)
set_data(..., selected_partition, ...);
new_partition = selected_partition;
Dialog_Partition_New::Get_New_Partition(...)
new_partition.Set(..., "New Partition #%1", ...);
New_partition is copied from the selected unallocated partition in which
the new partition will be created. So new_partition is now named
"unallocated". Then the Set() call appends the new name making the
vector:
paths = ["unallocated", "New Partition #%1"]
As get_path() returns the first name in the paths vector the path name
changed from "New Partition #%1" to "unallocated" causing this bug.
Fix by resetting the new_partition object to clear all vestiges of it
being a copy of the selected unallocated partition, Reset() call, before
then calling Set(). This then appends the new name to an empty vector
making it contain just the required new name:
paths = ["New Partition #%1"]
Bug 759488 - Pending create partitions are all getting named
"unallocated"
Capture and parse the progress reports of ntfsresize and resize2fs and
update the dialog progress bar.
Bug 467925 - gparted: add progress bar during operation
The Canadian English translation po file incorrectly changed the
following string with a parameter from %1 to %2:
msgid "old end: %1"
msgstr "old end: %2"
The error resulted in a missing "old end:" numerical value in the
gparted details log.
This error was introduced back in 2011-09-06 with the following
commit:
06eeaafc8c
Updated Canadian English translation.
Bug 756878 - GParted - Fix missing "old end" value in detail log of
en_CA translation
Return newly constructed partition object by reference rather than by
copy from the Copy, Resize/Move and New dialog classes. This is another
case of stopping copying partition objects in preparation for using
polymorphic Partition objects. In C++ polymorphism has to use pass by
pointer and reference and not pass by value, copying, to avoid object
slicing.
The returned reference to the partition is only valid until the dialog
object containing the new_partition member is destroyed. This is okay
because in all three cases the returned referenced partition is copied
into a context with new lifetime expectations before the dialog object
is destroyed.
Case 1: GParted_Core::activate_paste()
Referenced new_partition is copied in the OperationCopy constructor
before the dialog object goes out of scope.
Operation * operation = new OperationCopy( ...,
dialog.Get_New_Partition( ... ),
... );
Case 2: GParted_Core::activate_new()
Referenced new_partition is copied in the OperationCreate
constructor before the dialog object goes out of scope.
Operation * operation = new OperationCreate( ...,
dialog.Get_New_Partition( ... ) );
Case 3: GParted_Core::activate_resize()
Temporary partition object is copied from the referenced
new_partition before the dialog object goes out of scope.
Partition part_temp = dialog.Get_New_Partition( ... );
Bug 757671 - Rework Dialog_Partition_New::Get_New_Partition() a bit
Replace the use of local Partition variable with class member in
preparation for Dialog_Partition_New::Get_New_Partition() being changed
to return the new partition object by reference instead of by value.
part_temp -> new_partition
Bug 757671 - Rework Dialog_Partition_New::Get_New_Partition() a bit
When creating a new extended partition, the unallocated space within it
was being created before adjusting the partition boundaries for
alignment reasons. This must be wrong. Move creation of the
unallocated space to after adjusting the partition boundaries for
alignment reasons. First introduced by this commit:
a30f991ca5
Fix size reduced by 1 MiB when created after cylinder aligned partition
Also added a big FIXME comment explaining how further adjustments are
still made by snap_to_alignment() to the partition boundaries including
a test case where this too late adjustment causes overlapping boundaries
and apply to fail.
Bug 757671 - Rework Dialog_Partition_New::Get_New_Partition() a bit
This is just to make the parameter name in the Dialog_Partition_New
constructor and set_data() method match the name of the equivalent
parameter in the Dialog_Partition_Copy and Dialog_Partition_Resize_Move
classes. (All three classes inherit from Dialog_Base_Partition and have
similar interfaces).
Bug 757671 - Rework Dialog_Partition_New::Get_New_Partition() a bit
Blkid recognises many more file system types and RAID member signatures
than libparted. GParted already uses blkid detection instead of or
before libparted for whole disk devices [1] and for ext4 detection [2]
(only required with libparted < 1.9.0). Also GParted could only use
blkid detection on non-512 byte sector devices [3] before libparted was
fixed in version 3.2 [4]. Blkid was documented as a mandatory
requirement from GParted 0.24.0 [5].
Util-linux package, of which blkid command is a part, is a core piece of
Linux software which is very actively maintained and used by lots of
other packages. Parted package is much less active and has added
detection of fewer file systems and doesn't recognise any RAID members.
In cases of multiple signatures within a partition blkid and libparted
can report different results leading to confusion and issues for
GParted. This was the primary reason for bug 688882 "Improve clearing
of file system signatures" and a number of other changes to GParted.
Also as the mount command links with libblkid it uses it's detection
when telling the kernel the type of a file system to be mounted.
There aren't any current issues with GParted's file system detection but
given the above argument, switch to using blkid before libparted for
file system detection. Only falling back to libparted when blkid
doesn't report a result, notably for extended partitions. Order of
information sources for detection is now:
1) mdadm (for SWRaid members)
2) blkid
3) libparted
4) GParted internal code
References:
[1] f8faee6377
Avoid whole disk FAT being detected as MSDOS partition table (#743181)
[2] 533eb1bc03
Added support for ext4 file systems
[3] 9e5e9f5627
Enhance file system detection to use FS_Info method - blkid
[4] http://git.savannah.gnu.org/cgit/parted.git/commit/?id=80678bdd957cf49a9ccfc8b88ba3fb8b4c63fc12
Fix filesystem detection on non 512 byte sectors
[5] 749a249571
Document blkid command as a mandatory requirement (#753436)
Bug 757781 - Always use blkid file system detection before libparted
The GParted_Core code only interacts with derived Operation* objects
through the Operation base class interface and in one case the
OperationCopy class. Therefore include Operation.h and OperationCopy.h
headers and no others.
ZFS labels were not cleared by GParted when clearing old file system
signatures.
# wget https://git.kernel.org/cgit/utils/util-linux/util-linux.git/plain/tests/ts/blkid/images-fs/zfs.img.bz2
# bzip2 -dc zfs.img.bz2 > /dev/sdb1
[In GParted format to cleared /dev/sdb1]
# blkid /dev/sdb1
/dev/sdb1: LABEL="tank" UUID="1782036546311300980" UUID_SUB="13179280127379850514" TYPE="zfs_member"
Update to also zero all 4 ZFS labels.
NOTE:
GParted now writes a little over 1 MiB when clearing old file system
signatures. As this is performed in the main thread the UI is not able
to respond during this action. Testing this on a range of USB flash
keys and hard drives found the slowest normal time to write this was
0.25 seconds, with an occasional outlier up to 2.8 seconds from a USB
flash key. This is considered acceptable.
Following the previous commit "Add erasing of SWRaid metadata 0.90 and
1.0 super blocks (#756829)" signature zeroing specified to write 4 KiB
of zeros at position end - 64 KiB, aligned to 64 KiB. Example operation
details from formatting a 1 GiB partition to cleared:
Format /dev/sdb8 as cleared
+ calibrate /dev/sdb8
+ clear old file system signatures in /dev/sdb8
+ write 68.00 KiB of zeros as byte offset 0
+ wite 4.00 KiB of zeros at byte offset 67108864
+ wite 64.00 KiB of zeros at byte offset 1073676288
+ write 8.00 KiB of zeros at byte offset 1073733632
+ flush operating system cache of /dev/sdb
However it actually wrote 64 KiB. This is because the rounding /
alignment was also applied to the zeroing length. Before this commit
rounding / alignment was always less than or equal to the length so this
wasn't seen before. Instead just apply device sector size rounding up
to the length.
Bug 756829 - SWRaid member detection enhancements
The super blocks for Linux Software RAID arrays using metadata types
0.90 and 1.0 are stored at the end of the partition and not currently
cleared by GParted.
Create a SWRaid array, stop it and format it to cleared using GParted.
The signature remains.
# mdadm --create /dev/md1 --level=linear --raid-devices=1 --force --metadata=1.0 /dev/sdb1
# mdadm --stop /dev/md1
[In GParted format to cleared /dev/sdb1]
# blkid /dev/sdb1
/dev/sdb1: UUID="8ac947a7-063f-2266-5f2a-e5d178198139" UUID_SUB="49bd51d4-4c54-fb16-a45e-bd795f783f59" LABEL="rockover:1" TYPE="linux_raid_member"
As fixed in other cases before [1][2] it is necessary to clear all
signatures before formatting as a new file system to prevent recognition
issues. For example now format the partition as a FAT32 file system.
Now there are two signatures and libparted reports one type and blkid
reports another.
# mkdosfs -F 32 /dev/sdb1
# blkid /dev/sdb1
/dev/sdb1: UUID="8ac947a7-063f-2266-5f2a-e5d178198139" UUID_SUB="49bd51d4-4c54-fb16-a45e-bd795f783f59" LABEL="rockover:1" TYPE="linux_raid_member"
# parted /dev/sdb print
Model: ATA SAMSUNG SSD UM41 (scsi)
Disk /dev/sdb: 8012MB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number Start End Size Type File system Flags
1 1049kB 538MB 537MB primary fat32
(Deliberately avoided btrfs, ext2/3/4 and xfs as recent versions of
their mkfs tools clear other signatures first for the same reason).
[1] Bug 688882 - Improve clearing of file system signatures
[2] 3c75f3f5b1
Use wipefs to clear old signatures before creating new file systems (#688882)
Update erase_filesystem_signatures() to also zero the necessary sectors
to clear SWRaid metadata 0.90 and 1.0 super blocks.
Bug 756829 - SWRaid member detection enhancements
If the installation is unusual / broken such that the mdadm command is
not found but there are active SWRaid arrays, provide a fallback with
some of the information. In this case populate the cache with details
only available from /proc/mdstat. Information will be limited to active
arrays only and their members. No UUIDs or labels. There will be no
information about inactive arrays and GParted will use it's normal
libparted and blkid identification for those devices and partitions.
As mdadm has gained the capability to manage Fake/BIOS RAID arrays they
also appear in /proc/mdstat when mdadm is used to start them. Enhance
the parser of /proc/mdstat to only extract information for SWRaid arrays
with recognised metadata versions.
Bug 756829 - SWRaid member detection enhancements
Automatically load the cache of SWRaid information for the first time if
any of the querying methods are called before the first explicit
load_cache() call. Means we can't accidentally use the class and
incorrectly find no SWRaid members when they do exist.
Bug 756829 - SWRaid member detection enhancements
Busy file systems are accessed via a mount point, LVM Physical Volumes
are activated via the Volume Group name and busy SWRaid members are
accessed via the array device, /dev entry. Therefore choose to show the
array device in the mount point field for busy SWRaid members.
The kernel device name for an SWRaid array (without leading "/dev/") is
the same as used in /proc/mdstat and /proc/partitions. Therefore the
array device (with leading "/dev/") displayed in GParted will match
between the mount point for busy SWRaid members and the array itself as
used in the device combo box.
# cat /proc/mdstat
Personalities : [raid1]
md1 : active raid1 sda1[2] sdb1[3]
524224 blocks super 1.0 [2/2] [UU]
...
# cat /proc/partitions
major minor #blocks name
8 0 33554432 sda
8 1 524288 sda1
...
8 16 33554432 sdb
8 17 524288 sdb1
...
9 1 524224 md1
...
Bug 756829 - SWRaid member detection enhancements
In cases where blkid wrongly reports a file system instead of an SWRaid
member (sometimes confused by metadata 0.90/1.0 mirror array or old
version not recognising SWRaid members), the UUID and label are
obviously wrong too. Therefore have to use the UUID and label returned
by the mdadm query command and never anything reported by blkid or any
file system specific command.
Example of blkid reporting the wrong type, UUID and label for /dev/sda1
and the correct values for /dev/sdb1:
# blkid | egrep 'sd[ab]1'
/dev/sda1: UUID="10ab5f7d-7d8a-4171-8b6a-5e973b402501" TYPE="ext4" LABEL="chimney-boot"
/dev/sdb1: UUID="15224a42-c25b-bcd9-15db-60004e5fe53a" UUID_SUB="0a095e45-9360-1b17-0ad1-1fe369e22b98" LABEL="chimney:1" TYPE="linux_raid_member"
# mdadm -E -s -v
ARRAY /dev/md/1 level=raid1 metadata=1.0 num-devices=2 UUID=15224a42:c25bbcd9:15db6000:4e5fe53a name=chimney:1
devices=/dev/sda1,/dev/sdb1
...
ARRAY /dev/md127 level=raid1 num-devices=2 UUID=8dc7483c:d74ee0a8:b6a8dc3c:a57e43f8
devices=/dev/sdb6,/dev/sda6
...
NOTES:
* In mdadm terminology the label is called the array name, hence name=
parameter for array md/1 in the above output.
* Metadata 0.90 arrays don't support naming, hence the missing name=
parameter for array md127 in the above output.
Bug 756829 - SWRaid member detection enhancements
Add active attribute to the cache of SWRaid members. Move parsing of
/proc/mdstat to discover busy SWRaid members into the cache loading
code. New parsing code is a little different because it is finding all
members of active arrays rather than determining if a specific member is
active.
Bug 756829 - SWRaid member detection enhancements
Detection of Linux SWRaid members currently fails in a number of cases:
1) Arrays which use metadata type 0.90 or 1.0 store the super block at
the end of the partition. So file system signatures in at least
linear and mirrored arrays occur at the same offsets in the
underlying partitions. As libparted only recognises file systems
this is what is detected, rather than an SWRaid member.
# mdadm -E -s -v
ARRAY /dev/md/1 level=raid1 metadata=1.0 num-devices=2 UUID=15224a42:c25bbcd9:15db6000:4e5fe53a name=chimney:1
devices=/dev/sda1,/dev/sdb1
...
# wipefs /dev/sda1
offset type
----------------------------------------------------------------
0x438 ext4 [filesystem]
LABEL: chimney-boot
UUID: 10ab5f7d-7d8a-4171-8b6a-5e973b402501
0x1fffe000 linux_raid_member [raid]
LABEL: chimney:1
UUID: 15224a42-c25b-bcd9-15db-60004e5fe53a
# parted /dev/sda print
Model: ATA VBOX HARDDISK (scsi)
Disk /dev/sda: 34.4GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number Start End Size Type File system Flags
1 1049kB 538MB 537MB primary ext4 boot, raid
...
2) Again with metadata type 0.90 or 1.0 arrays blkid may report the
contained file system instead of an SWRaid member. Have a single
example of this configuration with a mirrored array containing the
/boot file system. Blkid reports one member as ext4 and the other as
SWRaid!
# blkid | egrep 'sd[ab]1'
/dev/sda1: UUID="10ab5f7d-7d8a-4171-8b6a-5e973b402501" TYPE="ext4" LABEL="chimney-boot"
/dev/sdb1: UUID="15224a42-c25b-bcd9-15db-60004e5fe53a" UUID_SUB="0a095e45-9360-1b17-0ad1-1fe369e22b98" LABEL="chimney:1" TYPE="linux_raid_member"
Bypassing the blkid cache gets the correct result.
# blkid -c /dev/null /dev/sda1
/dev/sda1: UUID="15224a42-c25b-bcd9-15db-60004e5fe53a" UUID_SUB="d0460f90-d11a-e80a-ee1c-3d104dae7e5d" LABEL="chimney:1" TYPE="linux_raid_member"
However this can't be used because if a user has a floppy configured
in the BIOS but no floppy attached, GParted will wait for minutes as
the kernel tries to access non-existent hardware on behalf of the
blkid query. See commit:
18f863151c
Fix long scan problem when BIOS floppy setting incorrect
3) Old versions of blkid don't recognise SWRaid members at all so always
report the file system when found. Occurs with blkid v1.0 on
RedHat / CentOS 5.
The only way I can see how to fix all these cases is to use the mdadm
command to query the configured arrays. Then use this information for
first choice when detecting partition content, making the order: SWRaid
members, libparted, blkid and internal.
GParted shell wrapper already creates temporary blank udev rules to
prevent Linux Software RAID arrays being automatically started when
GParted refreshes its device information[1]. However an administrator
could manually stop or start arrays or change their configuration
between refreshes so GParted must load this information every refresh.
On my desktop with 4 internal hard drives and 3 testing Linux Software
RAID arrays, running mdadm adds between 0.20 and 0.30 seconds to the
device refresh time.
[1] a255abf343
Prevent GParted starting stopped Linux Software RAID arrays (#709640)
Bug 756829 - SWRaid member detection enhancements
Mike Fleetwood