PATCH SET OVERVIEW:
Currently the supported actions of each file system and their size
limits are stored in struct FS objects. These are created by calling
file system specific derived implementations of
FileSystem::get_filesystem_support(). This happens when GParted is
started or when a when a rescan for supported actions is performed. The
file system size limits are expressed as a fixed number of bytes.
The maximum UDF file system size is specified in terms of file system
block size units. Also the file system block size must match the sector
size of the underlying device. Typically 2K for optical media and 512
bytes or 4K for hard drives.
Therefore GParted can't properly express the true UDF file system size
limits because they depend on the block size of an existing UDF file
system or the sector size of the device for new UDF file systems. In
fact other file systems such as EXT2/3/4 and XFS actually express their
maximum file system size in terms of numbers of file system blocks but
these tend to always be 4K and don't have to match the sector size of
the underlying device, so fixed byte values tend to suffice.
To update GParted for this, first separate file system size limits from
struct FS into struct FS_Limits and provide new
FileSystem::get_filesystem_limits() method to allow the limits to be
queried independently of the calls to get_filesystem_support().
Second, pass Partition objects and allow derived get_filesystem_limits()
implementations.
THIS PATCH:
Just creates a separate structure storing fixed value file system
minimum and maximum size limits along with getter method
get_filesystem_limits().
Bug 787204 - Minimum and maximum size of the UDF partition/disk
Those external tools were introduced in version 2.0 of udftools package
and can show or change UDF label, UDF uuid and can provide information
needed for counting total/free sectors.
Bug 792052 - Add support for changing UDF label/uuid and show disk usage
The general rule is that:
1) For a partition change step BEFORE a file system change step,
rollback on failure;
2) For a partition change step AFTER a file system change step, don't
rollback on failure.
Examining every case where resize_move_partition() is called and whether
rollback on failure is wanted or not:
* In resize_move()
Resize / move extended partition. No associated file system change.
NO ROLLBACK
Just to keep possibly applied operation.
* #1 in move()
Making all encompassing partition before moving file system.
ROLLBACK
To restore partition boundaries back to those of the file system.
* #2 in move()
Recreating original partition boundaries after file system move
failed or was cancelled and has been rolled back.
NO ROLLBACK
To keep updated partition boundaries to match restored file system
data.
* #3 in move()
Replacing all encompassing partition with final partition after
successful file system move.
NO ROLLBACK
Keep new partition boundaries to match moved file system.
* #1 in resize_encryption()
Making the partition larger before growing closed LUKS encrypted
data.
ROLLBACK
Restore partition boundaries back to those of the closed LUKS
encrypted data.
* #2 in resize_encryption()
Shrinking the partition after open LUKS mapping has been shrunk, but
before swap is re-created (smaller).
NO ROLLBACK
Difficult case because the partition shrink is in the middle of a
LUKS shrink and a swap shrink (re-create). If swap was actually
shrunk like other types of file system, rather than re-created, then
the operation sequence would be (1) shrink swap, (2) shrink LUKS
encryption, (3) shrink partition. In this hypothetical case and the
actual case no rollback is preferred to try to keep the new
partition boundaries match the shrunk open LUKS encryption mapping.
* #3 in resize_encryption()
Grow the partition before growing open LUKS mapping and re-creating
swap larger.
ROLLBACK
Restore partition boundaries back to those of the smaller open LUKS
encryption mapping.
* #4 in resize_encryption()
Shrink the partition after shrinking the file system and open LUKS
encryption mapping.
NO ROLLBACK
Keep new smaller partition boundaries to match shrunk encrypted file
system.
* #5 in resize_encryption()
Grow the partition before growing the open LUKS encryption mapping
and file system.
ROLLBACK
Restore partition boundaries back to those of the not yet grown
encrypted file system.
* #1 in resize_plain()
Resize partition before re-creating swap a different size.
ROLLBACK
Restore partition boundaries back to those of the not yet resized
(re-created) swap space.
* #2 in resize_plain()
Shrink partition after shrinking the file system.
NO ROLLBACK
Keep new smaller partition boundaries to match shrunk file system.
* #3 in resize_plain()
Grow partition before growing the file system.
ROLLBACK
Restore partition boundaries back to those of the not yet grown
file system.
Removes the default value from the rollback_on_fail parameter so
rollback or not has to be explicitly specified for every call of
resize_move_partition().
Bug 791875 - Rollback specific failed partition change steps
Even after implementing a fix for bug 790418 "Unable to inform the
kernel of the change message may lead to corrupted partition table"
GParted/libparted can still encounter errors informing the kernel of the
new partition layout. This has been seen with GParted on CentOS 7 with
libparted 3.1.
In such a case the partition has been successfully written to the disk
but just informing the kernel failed. This is a problem because when a
partition is being moved in advance of a file system move step, failure
to inform the kernel leaves the partition boundaries not matching the on
disk limits of the file system. For a move to the left this leaves the
partition reported as unknown, apparently losing the user's data.
For example start with a 512 MiB partition containing an XFS file
system. This is recognised by blkid and parted, hence also by GParted.
# blkid /dev/sdb1
/dev/sdb1: UUID=... TYPE="xfs" PARTUUID="37965980-01"
# parted /dev/sdb unit s print
Model: ATA VBOX HARDDISK (scsi)
Disk /dev/sdb: 16777216s
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:
Number Start End Size Type File system Flags
1 1048576s 2097151s 1048576s primary xfs
Now move the partition 100 MiB to the left and have it fail to inform
the kernel after the first partition change step. Operation details:
Move /dev/sdb1 to the left (ERROR)
* calibrate /dev/sdb1 (SUCCESS)
* check file system on /dev/sdb1 for errors and (if poss...(SUCCESS)
* grow partition from 512.00 MiB to 612.00 MiB (ERROR)
old start: 1048576
old end: 2097151
old size: 1048576 (512.00 MiB)
requested start: 843776
requested end: 2097151
requested size: 1253376 (612.00 MiB)
* libparted messages (ERROR)
Error informing the kernel about modifications to partition
/dev/sdb1 -- Device or resource busy. This means Linux won't
know about any changes you made to /dev/sdb1 until you reboot
-- so you shouldn't mount it or use it in any way before
rebooting. Failed to add partition 1 (resource temporarily
unavailable)
Now because the start of the partition is 100 MiB before the start of
the file system, the file system is no longer recognised, and apparently
the user's data has been lost.
# blkid /dev/sdb1
/dev/sdb1: PARTUUID="37965980-01"
# parted /dev/sdb unit s print
...
Number Start End Size Type File system Flags
1 843776s 2097151s 1253376s primary
It doesn't matter why updating the partition failed, even if it was
because of an error writing to the disk. Rollback of the change to the
partition should be attempted. The worst case scenario is that rollback
of the change fails, which is the equivalent to how the code worked
before this patch set.
However in other cases where the partition boundaries are being updated
after a file system move or shrink step then the partition should be
updated to match the new location of the file system itself. And no
rollback is wanted. If the failure was only informing the kernel then
in fact the partition has actually been updated on disk after all.
So each partition resize/move step needs examining on a case by case
basis to decide if rolling back the change to the partition is wanted or
not.
This patch only adds partition change rollback into
resize_move_partition(). Rollback remains disabled until all cases are
examined in the following patch.
Bug 791875 - Rollback specific failed partition change steps
Extract common code which updates a DMRaid device mapper entry into a
sub-function. This will also be needed when adding rollback of a
partition change on failure.
Bug 791875 - Rollback specific failed partition change steps
Extract the code which actually implements the partition change into a
sub-function ready for adding rollback of the change on failure.
Bug 791875 - Rollback specific failed partition change steps
All libparted messages were reported as informational, even for a step
which failed. Instead identify libparted messages as either
informational or errors depending on whether this step was successful
or not respectively.
Bug 790842 - Report libparted messages into operation details at the
point at which they occur
Replace the explicit adding of libparted exception messages with a
callback to do it instead, and fire the callback just once per operation
by only changing the very top-level OperationDetail to use the new
set_success_and_capture_errors(). Therefore this still produces exactly
the same operation details with libparted messages at the end of each
operation.
Bug 790842 - Report libparted messages into operation details at the
point at which they occur
All code implementing a step of an operation follows this pattern:
od.add_child(OperationDetail("Step heading"));
od.get_last_child().add_child(OperationDetail("More details"));
// Do step
success = ...
od.get_last_child().set_status(success ? STATUS_SUCCESS
: STATUS_ERROR);
At this point any libparted messages reported via exceptions need to be
added into the OperationDetail tree. Also adding further children into
the tree after collecting those errors needs to be prohibited (as much
as the previous patch prohibited it).
Add a new method which will replace the final set_status() call above
like this which set the status, captures the errors and flags that
further children shouldn't be added:
...
od.get_last_child().set_success_and_capture_errors(status);
It emits a callback to capture the errors to provide flexibility and so
that the OperationDetail class doesn't have to get into the details of
how GParted_Core saves libparted exception messages.
Bug 790842 - Report libparted messages into operation details at the
point at which they occur
Want functionality to prevent further child details being added to an
OperationDetail. This is so that the captured libparted error messages
are always the last child in the list, and more details (at that point
in the tree) can't be added.
For example we want GParted to report like this:
Move /dev/sdb3 to the right and shrink it from 1.14 GiB to...(SUCCESS)
...
* shrink partition from 1.14 GiB to 1.00 GiB (SUCCESS)
* old start: 4464640
old end: 6856703
old size: 2392064 (1.14 GiB)
* new start: 4464640
new end: 6561791
new size: 2097152 (1.00 GiB)
* libparted messages (INFO)
* DEBUG: GParted generated synthetic libparted excepti...
and not like this:
Move /dev/sdb3 to the right and shrink it from 1.14 GiB to...(SUCCESS)
...
* shrink partition from 1.14 GiB to 1.00 GiB (SUCCESS)
* old start: 4464640
old end: 6856703
old size: 2392064 (1.14 GiB)
* libparted messages (INFO)
* DEBUG: GParted generated synthetic libparted excepti...
* new start: 4464640
new end: 6561791
new size: 2097152 (1.00 GiB)
So actually preventing the addition of more child details would stop
users seeing information they should see. So instead just report a bug
message into the operation details. This doesn't stop anything, but the
bug message will be seen and allow us to fix GParted.
So far nothing is enforced. This patch just adds the mechanism to
report a bug when a new child detail is added when prohibited.
Bug 790842 - Report libparted messages into operation details at the
point at which they occur
Remove whole_device flag and replace with new partition type
TYPE_UNPARTITIONED. Minimally adapt the remaining code to compile and
run.
Bug 788308 - Remove whole_device partition flag
PATCHSET OVERVIEW:
When unpartitioned drive read-write support was added this commit added
a whole_device flag:
5098744f9a
Add whole_device flag to the partition object (#743181)
Using a whole_device flags now seems not the correct way to model
unpartitioned drives. GParted models an uninitialised drive as:
.path = _("uninitialized")
.type = TYPE_UNALLOCATED
.whole_device = true
.filesystem = FS_UNALLOCATED
and a whole drive file system, using ext4 for example, as:
.path = "/dev/sdb"
.type = TYPE_PRIMARY
.whole_device = true
.filesystem = FS_EXT4
No partitioning changed yet the type of the partition in the model
changed between TYPE_UNALLOCATED and TYPE_PRIMARY depending on whether
the whole drive contains a recognised file system or not.
The partition object describing a file system within a LUKS encryption
mapping is another case of the model not matching reality.
.path = /dev/mapper/crypt_sdb1_crypt
.type = TYPE_PRIMARY
.whole_device = true
.filesystem = FS_EXT4
There is no partition table within the encryption mapping, the file
system fills it, but GParted records it as a primary partition.
Make TYPE_UNALLOCATED and TYPE_PRIMARY be reserved for representing
unallocated space and primary partitions within a partitioned disk drive
and introduce new TYPE_UNPARTITIONED for all cases of an unpartitioned
whole disk drive.
The GParted UI does differentiate between an unallocated whole disk
device and anything else by requiring a partition table to be created
first, even if that is just the loop partition table. That
determination can simply look for the partition object containing file
system type FS_UNALLOCATED instead.
THIS PATCH:
Create set_unpartitioned() helper method to set a partition object to
represent a whole disk drive and use everywhere such an object is
modelled. This matches what existing methods Set_Unallocated() and
indeed Set() do for unallocated space and any type of partition
respectively.
For now the partition type is still set to either TYPE_UNALLOCATED or
TYPE_PRIMARY so the rest of the code base remains the same.
TYPE_UNPARTITIONED will be introduced later.
Bug 788308 - Remove whole_device partition flag
Naming a file system image file on the command line is shown by GParted
as unknown.
$ truncate -s 100M /tmp/fat.img
$ mkfs.vfat /tmp/fat.img
$ sudo ./gpartedbin /tmp/fat.img
Currently the FS_Info cache is loaded for all devices reported by
blkid (plus all whole disk devices identified from /proc/partitions even
if blkid reports nothing). However file system images named on the
command line are not queried so GParted can't identify them.
Fix by ensuring that the FS_Info blkid cache is loaded for all named
devices, including named file system image files.
Note that Mount_Info::load_cache() depends on the contents of the
FS_Info cache to lookup UUID= and LABEL= device names from /etc/fstab.
However only file systems in block devices can be mounted like this, and
never file system image files, so the fact that the cache may be
extended afterwards by FS_Info::load_cache_for_paths() does not matter.
History
Prior to version 0.22.0, when unpartitioned drive support was added,
GParted could recognise some file system image files using loop
partition handling in libparted. However libparted before version 3.2
reported the loop partition name as the whole disk device name appended
with "1" so all the query commands were provided a non-existent name to
use. Therefore no file system usage or the label was displayed.
Bug 787181 - Fix detection of file system images
So far GParted is still loading the default non-reversible encoded
labels from blkid in the initial loading of the FS_Info module cache.
This encoded label is used to match LABEL=<label> when reading
/etc/fstab, via the get_path_by_label() call, so works for ASCII only
labels. This prevents GParted enabling the "mount on >" partition menu
item when non-ASCII labels are used.
To fix this:
1) Stop reading the labels the wrong way.
Via the blkid command used to initially load the FS_Info module cache
and is subject to default non-reversible encoding of non-printable
ASCII bytes.
2) Read all the labels the right way, but only when needed.
Only when /etc/fstab file contains LABEL=<label> and
get_path_by_label() is called, read all the labels from blkid without
encoding them via run_blkid_update_cache_one_label().
3) Return label from the cache.
get_label() returns the cached label, loading it into the cache first
if needed with run_blkid_update_cache_one_label().
In the worst case scenario of having a LABEL=<label> in /etc/fstab blkid
will be run for every partition containing a recognised file system to
read the label. On my desktop with 5 hard drives, 4 SWRaid arrays and
31 recognised file systems running 'blkid -o value -s LABEL ...' 31
times took 0.074 seconds of a total scan time of 9.072 seconds. Less
that 1% of the total scanning time. When LABEL=<label> is not used in
/etc/fstab individual blkid executions are only used to read labels for
file systems where there is no file system specific tool available
reducing the impact further. Blkid itself caches the data in it's
blkid.tab cache file rather than reading all file systems on each
invocation. Also the Linux file system cache will already contain the
blkid executable file, needed libraries files and the blkid.tab cache
file itself. Hence why repeated execution of blkid is so fast.
Further to the updated comment in set_partition_label_and_uuid().
Matching LABEL=<label> from /etc/fstab uses the label obtained from
blkid run in the C locale so this kind of assumes it returns the label
correctly and it does for my limited testing on Unicode enabled
desktops. Just not sure if it would be true for all cases in all
locales compared to the FS specific command run in the users default
locale.
Bug 786502 - Support reading Unicode labels when file system specific
tools aren't available
Move the code which reads the Unicode label from FS_Info::get_label()
into new function run_blkid_update_cache_one_label() which also replaces
the non-reversibly encoded copy loaded during the initial cache load.
This is mainly a bit of code refactoring ready for the following change.
It deliberately keeps the initial loaded labels so that reading
/etc/fstab and decoding LABEL=<label> to block special device names via
FS_Info::get_path_by_label() continues to works, at least for ASCII only
labels.
Bug 786502 - Support reading Unicode labels when file system specific
tools aren't available
Make especially the Volume Identifier length limit code simpler to
understand and therefore easier to maintain.
Bug 784533 - Add support for UDF file system
UDF label is stored in the Logical Volume Identifier which has space for
either 126 Latin1 or 63 UCS-2 characters. For compatibility reasons
with older versions of blkid, the possibly truncated UDF label is also
stored in the Volume Identifier which only has space for 30 Latin1 or 15
UCS-2 characters.
Because versions of mkudffs prior to 1.1 damage the label if it contains
non-ASCII characters, make sure GParted does not call such versions of
mkudffs with a non-ASCII character label.
Bug 784533 - Add support for UDF file system
Add support for detecting UDF file systems and formatting hard disks
with revision 2.01 UDF file systems using udftools. Formatting optical
disks or any other media types is not supported yet. Changing label or
UUID after formatting is not supported as the tools do not yet exist.
Bug 784533 - Add support for UDF file system
For large output a lot of time is used copying capturebuf to callerbuf
to provide a Glib::ustring copy of the buffer for the update callback.
However update callbacks are only used when commands are run to apply
operations by FileSystem::execute_command() and their output is
incrementally displayed in the UI. Whereas update callbacks are never
used when commands are used to query information via
Utils::execute_command().
Stop performing interim copying of capturebuf to callerbuf when there
are no update callbacks registered as it is unnecessary.
Time to read portions of the recorded fsck.fat output via
fat16::set_used_sectors() and intermediate copies aren't required:
1 MiB 10 MiB 122 MiB
old code : 0.074 sec 1.41 sec 210 sec [3:30]
new code : 0.063 sec 0.56 sec 6.57 sec
Bug 777973 - Segmentation fault on bad disk
If PipeCapture reads a NUL byte in the middle of what is expected to be
a multi-byte UTF-8 character then PipeCapture either returns the
captured characters to the previous update or loops forever depending on
whether the end of the stream is encountered before the read buffer is
full or not. This is equivalent to saying whether the NUL byte occurs
within the last 512 bytes of the output or not.
This is caused by a bug in g_utf8_get_char_validated() reporting that a
partial UTF-8 character has been found when the NUL byte is encountered
in the middle of a multi-byte character even though more bytes are
available in the length specified buffer. g_utf8_get_char_validated()
is always stopping at the NUL byte assuming it is working with a NUL
terminated string.
Workaround this by checking for g_utf8_get_char_validated() claiming a
partial UTF-8 character has been found when in fact there are at least
enough bytes in the read buffer to instead determine that it is really
an invalid UTF-8 character.
Reference:
Bug 780095 - g_utf8_get_char_validated() stopping at nul byte even
for length specified buffers
https://bugzilla.gnome.org/show_bug.cgi?id=780095
Bug 777973 - Segmentation fault on bad disk
A user had a very corrupted FAT file system such that fsck.fat produced
122 MiB of output. GParted has to read this output to get the file
system usage information. However GParted takes more than 48 hours to
read the 122 MiB of output, while using 100% CPU time and is
unresponsive for the duration.
Modified fsck.fat to output just the first 1 MiB of output and used perf
to capture performance data from GParted reading that output:
# perf -g -F 1999 -- ./gpartedbin
# perf report --stdio
67.84% Glib::ustring::replace [4.23s]
17.67% g_utf8_pointer_to_offset [1.10s]
8.48% g_utf8_offset_to_pointer [0.53s]
[ 6.01% (everything else) ] [0.38s]
[100.00% TOTAL ] [6.24s]
And to read the first 10 MiB of output the performance figures are:
92.95% Glib::ustring::replace [257.44s]
4.35% g_utf8_pointer_to_offset [ 12.05s]
2.13% g_utf8_offset_to_pointer [ 5.90s]
[ 0.58% (everything else) ] [ 1.61s]
[100.00% TOTAL ] [277.00s]
See how the total time is increasing non-linearly, 44 times longer for
only 10 times as much data. This is because of the exponential increase
in time spent in Glib::ustring::replace.
After a lot of experimentation I came to the conclusion that
Glib::ustrings are not appropriate for storing and editing large buffers
of data, sizes megabytes and above. The issues are that iterators are
invalid after the content changes and replacing UTF-8 characters by
index gets exponentially slower as the size of the string increases.
Hence the > 48 hours of 100% CPU time to read and apply the line
discipline to the 122 MiB of fsck.fat output. See code comment for a
more detailed description of the issues found.
Rewrote OnReadable() to use Glib::ustrings as little as possible.
Instead using buffers and vectors of fixed width data types allowing for
fast access using pointers and indexes (converted to pointers by the
compiler with simple arithmetic). Again see code comment for a more
detailed description of the implementation.
Repeating the performance capture with the new code for the first 1 MiB
of fsck.fat output:
63.34% memcpy [0.35s]
[ 36.66% (everything else) ] [0.21s]
[100.00% TOTAL ] [0.56s]
And for the first 10 MiB of fsck.fat output:
96.66% memcpy [63.60s]
[ 3.34% (everything else) ] [ 2.20s]
[100.00% TOTAL ] [65.80s]
Simple timings taken to read portions of the fsck.fat output (when not
using perf):
1 MiB 10 MiB 122 MiB
old code : 6.2 sec 277 sec > 48 hours
(4:37)
new code : 0.6 sec 66 sec 17262 sec
(1:06) (4:47:42)
Performance of the code is still non-linear because of the assignment
of the ever growing capturebuf to callerbuf for every block of input
read. This is required to generate a consistent Glib::ustring copy of
the input for the update callback. However this is much faster than
before, and I have a plan for further improvements.
Bug 777973 - Segmentation fault on bad disk
As the source code is managed in GIT and there is a .gitignore file in
the top level directory specifying file names to exclude from version
control, then the old per-directory .cvsignore files for CVS are
redundant.
Add the only missing and applicable entry from src/.cvsignore of '.libs'
to .gitignore and remove all the .cvsignore files.
Using the default MiB alignment, creating an MSDOS logical partition
between two other existing logical partitions fails with this error
dialog:
(-) <b>An error occurred while applying the operations</b>
See the details for more information.
<b>IMPORTANT</b>
If you want support, you need to provide the saved details!
See http://gparted.org/save-details.htm for more information.
[ OK ]
and these operation details:
+ libparted messages
- Unable to satisfy all constraints on the partition.
This bug was introduced by this commit included in GParted 0.23.0:
90e3ed68fc
Shallow copy Device object into Operation object (#750168)
The commit message claimed that the deep copied Partition objects inside
the Device inside the Operation object are never accessed. This turned
out not to be true. Win_GParted::Add_Operation() uses them as part of
snap_to_alignment() which updates requested partition boundaries to
account for alignment requirements and the space needed for EBR
(Extended Boot Record) preceding logical partitions.
In this case the new logical partition was trying to be created over the
top of the EBR for the following logical partition because
snap_to_alignment() wasn't aware of its existence.
Fix by making Add_Operation() and snap_to_alignment() refer to the
current device, as displayed in the UI, rather than the shallow copy
included in the Operation object. Hopefully now it is true that the
not copied vector of Partition objects in the Device object in each
Operation object are never accessed.
Bug 779339 - enforce at least 1 MiB "free space following"
The Operation class already provided find_index_extended() method and
was used in the Operation and derived classes where required. It
returns the index to the extended partition in the PartitionVector
object, or -1 when no extended partition exists.
There were several cases of the same functionality being open coded in
GParted_Core and Win_GParted. Therefore move the implementation to
find_extended_partition() in PartitionVector compilation unit and use
this implementation everywhere.
The clear_mountpoints parameter has never been used since
add_mountpoint*() were first added [1][2]. clear_mountpoints() method
[3] is available to provide this functionality and used. Therefore
removed unused parameter and code.
[1] add_mountpoints() added 2006-03-15
9532c3cad1
Made Partition::mountpoints private
[2] add_mountpoint() added 2011-12-16
208083f11d84dbd4f186271a3cdbf5170db259f8b8
Display LVM2 VGNAME as the PV's mount point (#160787)
[3] clear_mountpoint() added 2006-03-19
ad9f2126e7
fixed issues with copying (see also #335004) cleanups + added FIXME added
Now that resizing of encrypted file systems is implemented add growing
of the open LUKS mapping as part of the check repair operation.
Resizing an encrypted file system requires the LUKS mapping to be open
to access the file system within; therefore it also requires libparted
and kernel support for online partition resizing. This limits resizing
to the latest distributions with libparted >= 3.2 and kernel >= 3.6.
However growing an open LUKS mapping as part of a check repair operation
doesn't require resizing the partition. Therefore route via offline
grow of LUKS to avoid those extra, unnecessary requirement. This does
mean that offline LUKS grow artificially requires cryptsetup, but that is
not really significant as even opening LUKS requires cryptsetup.
So now checking an encrypted file system on even the oldest
distributions does:
1) runs FSCK on the encrypted file system;
2) grows the encryption volume to fill the partition;
3) grows the file system to fill the encryption mapping.
Bug 774818 - Implement LUKS read-write actions NOT requiring a
passphrase
Moving of closed LUKS is simply enabled by luks .move capability being
set and requires no further coding.
Resizing of encrypted file systems requires both the LUKS mapping and
encrypted file system within to be resized in the right order for both
shrinking and growing. To keep the code simple split resizing of plain
and encrypted into separate functions.
Bug 774818 - Implement LUKS read-write actions NOT requiring a
passphrase
Changing the Resize/Move dialog code to also handle PartitionLUKS
objects was considered too complicated. Instead create an unencrypted
equivalent using clone_as_plain(), pass that to the Resize/Move dialog
and finally apply the change back using Partition*::resize().
Bug 774818 - Implement LUKS read-write actions NOT requiring a
passphrase
Add a resize() method to both Partition and PartitionLUKS classes. They
take a reference Partition object, and update the position, size and
file system usage of *this Partition to match. This is ready for taking
a partition returned from Resize/Move dialog and applying the change.
Bug 774818 - Implement LUKS read-write actions NOT requiring a
passphrase
Implement a specialist PartitionLUKS clone method. Creates a new
Partition object which has the same space usage as the source encrypted
file system, but is a plain file system. Namely, the overhead of the
LUKS header has been added to the file system usage. This is ready for
feeding this representation of the partition to the Resize/Move dialog.
Bug 774818 - Implement LUKS read-write actions NOT requiring a
passphrase
Already have:
Utils::get_filesystem_string(FS_EXT2) -> "ext2"
virtual Partition::get_filesystem_string() -> "ext2"
virtual PartitionLUKS::get_filesystem_string() -> "[Encrypted] ext2"
Add these:
Utils::get_encrypted_string() -> "[Encrypted]"
Utils::get_filesystem_string(false, FS_EXT2) -> "ext2"
Utils::get_filesystem_string(true, FS_EXT2) -> "[Encrypted] ext2"
This is ready for use of Utils::get_filesystem_string(true, FS_EXT2)
when composing the preview of a format of an encrypted file system by
Win_GParted::activate_format().
Bug 774818 - Implement LUKS read-write actions NOT requiring a
passphrase
When composing, describing and implementing the operation just need the
code to query and set the Partition object directly containing the file
system, instead of the enclosing encryption mapping to make it work.
The operation details for setting a new UUID on an encrypted ext4 file
system become:
Set a new random UUID on [Encrypted] ext4 file system on /dev/sdb4
+ calibrate /dev/sdb4
+ Set UUID on /dev/mapper/sdb4_crypt to a new, random value
+ tune2fs -U random /dev/mapper/sdb4_crypt
tune2fs 1.41.12 (17-May-2010)
Also note the now documented rule in apply_operation_to_disk() which
says each operation must leave the status of the encryption mapping and
file system as it found it.
Bug 774818 - Implement LUKS read-write actions NOT requiring a
passphrase
Provide and use a single interface for getting the file system string
for display, regardless of whether the partition is encrypted or the
encryption mapping is active or not.
Example return values for get_filesystem_string() for different types
and states of Partition objects:
1) Plain ext4 file system: -> "ext4"
2) Closed encrypted: -> "[Encrypted]"
3) Open encrypted ext4 file system: -> "[Encrypted] ext4"
This simplifies the code in TreeView_Detail::create_row() which sets the
file system type displayed in the main window. The same method will
then also be used when setting the operation description as each
operation is updated to handle encrypted file systems.
Bug 774818 - Implement LUKS read-write actions NOT requiring a
passphrase
The previous commit changed how the code behind the main window
retrieved the file system label for display. This is the relevant
changes in TreeView_Detail::create_row():
+ const Partition & filesystem_ptn = partition.get_filesystem_partition();
...
- Glib::ustring temp_filesystem_label = partition.get_filesystem_label();
+ Glib::ustring temp_filesystem_label = filesystem_ptn.get_filesystem_label();
treerow[treeview_detail_columns.label] = temp_filesystem_label;
In the case of an encrypted file system get_filesystem_label() is now
called on the Partition object directly rather than on the outer
Partition object containing the LUKS encryption.
The code behind the Information dialog always obtained and used the
Partition object directly containing the file system to call
get_filesystem_label() since read-only LUKS support was added.
Therefore the virtualised PartitionLUKS::get_filesystem_label() is no
longer needed, so remove it.
Bug 774818 - Implement LUKS read-write actions NOT requiring a
passphrase
There are multiple cases of code wanting to work with the Partition
object directly containing the file system, regardless of whether it is
within a PartitionLUKS object or not. The code had to do something
similar to this to access it:
const Partition * filesystem_ptn = &partition;
if ( partition.filesystem == FS_LUKS && partition.busy )
filesystem_ptn = &dynamic_cast<const PartitionLUKS *>( &partition )->get_encrypted();
...
// Access Partition object directly containing the file system
filesystem_ptn-> ...
Implement and use virtual accessor get_filesystem_partition() which
allows the code to be simplified like this:
const Partition & filesystem_ptn = partition.get_filesystem_partition();
...
// Access Partition object directly containing the file system
filesystem_ptn. ...
Bug 774818 - Implement LUKS read-write actions NOT requiring a
passphrase
More recent versions of blkid don't report an ISO9660 file system on the
whole disk device if partitions can be reports for embedded partitions.
However when querying the whole disk device directly then the expected
ISO9660 file system is reported. For example on CentOS 7 with the
previous ISO images:
# wget http://git.kernel.org/cgit/utils/util-linux/util-linux.git/plain/tests/ts/isosize/sample.iso.gz
# dd if=/dev/zero bs=1M of=/dev/sdc
# zcat sample.iso.gz | dd of=/dev/sdc
# blkid -v
blkid from util-linux 2.23.2 (libblkid 2.23.0, 25-Apr-2013)
# blkid | fgrep /dev/sdc
/dev/sdc1: UUID="2013-01-04-22-05-45-00" LABEL="ARCH_201301" TYPE="iso9660" PTTYPE="dos"
# blkid /dev/sdc
/dev/sdc: UUID="2013-01-04-22-05-45-00" LABEL="ARCH_201301" TYPE="iso9660" PTTYPE="dos"
# wget http://cdimage.debian.org/debian-cd/8.6.0/amd64/iso-cd/debian-8.6.0-amd64-netinst.iso
# dd if=/dev/zero bs=1M of=/dev/sdc
# dd if=debian-8.6.0-amd64-netinst.iso bs=1M of=/dev/sdc
# blkid | fgrep /dev/sdc
/dev/sdc1: UUID="2016-09-17-14-23-48-00" LABEL="Debian 8.6.0 amd64 1" TYPE="iso9660" PTTYPE="dos"
/dev/sdc2: SEC_TYPE="msdos" UUID="17F3-1162" TYPE="vfat"
# blkid /dev/sdc
/dev/sdc: UUID="2016-09-17-14-23-48-00" LABEL="Debian 8.6.0 amd64 1" TYPE="iso9660" PTTYPE="dos"
This behavioural difference with blkid is probably as a result of newer
versions of udev informing the kernel of the partitions embedded within
the ISO9660 image, and not directly as a result of a change in blkid
itself. Older distributions don't have partition entries for the above
ISO images, but CentOS 7 (with udev 219) and later distributions do have
partition entries:
# fgrep sdc /proc/partitions
8 16 8388608 sdc
8 17 252928 sdc1
8 18 416 sdc2
Fix by ensuring that the blkid FS_Info cache has entries for whole disk
devices, even if each entry has all empty attributes because there is a
partition table and not a recognised file system.
Calling blkid on whole disk devices containing partition tables produces
output like this, with newer versions of blkid:
# blkid /dev/sda
/dev/sda: PTTYPE="dos"
# blkid /dev/sdb
/dev/sdb: PTTYPE="gpt"
This will be loaded into the FS_Info cache as a blank entry for the
device by run_blkid_load_cache(). There will be a path name but all the
other attributes will be blank because there are no TYPE, SEC_TYPE, UUID
or LABEL name value pairs. With older versions of blkid no output is
produced at all. In that case load_fs_info_cache_extra_for_path() will
create the same blank entry with just the path name defined.
Bug 771244 - gparted does not recognize the iso9660 file system in
cloned Ubuntu USB boot drives
Move code from GParted_Core::set_devices_thread() performing top level
population of each Device object during the scan of the drives into new
set_device_from_disk() method.
Bug 771244 - gparted does not recognize the iso9660 file system in
cloned Ubuntu USB boot drives
Requires blkid.
Note that FS_LUKS was also moved to more closely match the order in
include/Utils.h
Bug 771244 - gparted does not recognize the iso9660 file system in
cloned Ubuntu USB boot drives
FileSystem::resize() (and derived) is only ever called from a single
location and always with the fill_partition argument supplied [1].
Therefore providing a default for the argument is unnecessary. So
remove it.
[1] Since these two commits from 2006-06-17, before GParted 0.3.0, which
added resize_filesystem() and maximize_filesystem() methods to wrap
calls to p_filesystem->resize() into a single location:
08245cd08c
cleanups in the core and the fs'es (resize)
2d7fb5700b
more cleanups in the core and the fs'es (these changelogs are getting
It made the code look a little messy, is easily resolved in the build
system and made the dependencies more complicated than needed. Each
GParted header was tracked via multiple different names (different
numbers of "../include/" prefixes). For example just looking at how
DialogFeatures.o depends on Utils.h:
$ cd src
$ make DialogFeatures.o
$ egrep ' [^ ]*Utils.h' .deps/DialogFeatures.Po
../include/DialogFeatures.h ../include/../include/Utils.h \
../include/../include/../include/../include/../include/../include/Utils.h \
../include/../include/../include/Utils.h \
After removing "../include/" from the GParted header #includes, just
need to add "-I../include" to the compile command via the AM_CPPFLAGS in
src/Makefile.am. Now the dependencies on GParted header files are
tracked under a single name (with a single "../include/" prefix). Now
DialogFeatures.o only depends on a single name to Utils.h:
$ make DialogFeatures.o
$ egrep ' [^ ]*Utils.h' .deps/DialogFeatures.Po
../include/DialogFeatures.h ../include/Utils.h ../include/i18n.h \
Linux-swap is recreated as part of copy, resize and move operations and
the code was special cased to implement that by calling the linux-swap
specific resize method. However the displayed text always said "growing
file system" and then proceeded to recreate linux swap. Example
operation:
Copy /dev/sdb1 to /dev/sdb2
...
+ copy file system from /dev/sdb1 to /dev/sdb2
Partition copy action skipped because linux-swap file system does not contain data
+ grow file system to fill the partition
+ create new linux-swap file system
+ mkswap -L"" -U "77d939ef-54d6-427a-a2bf-a053da7eed4c" /dev/sdb2
Setting up swapspace version 1, size = 262140 KiB
LABEL=, UUID=77d939ef-54d6-427a-a2bf-a053da7eed4c
Fix by writing recreate_linux_swap_filesystem() method with better
messaging and use everywhere maximise_filesystem() was previously used
to recreate linux-swap. Also as this is a create step, erase the
partition first to prevent the possibility of any other file system
signatures being found afterwards. Now the operation steps are more
reflective of what is actually being performed.
Copy /dev/sdb1 to /dev/sdb2
...
+ copy file system from /dev/sdb1 to /dev/sdb2
Partition copy action skipped because linux-swap file system does not contain data
+ clear old file system signatures in /dev/sdb2
+ create new linux-swap file system
+ mkswap -L"" -U "77d939ef-54d6-427a-a2bf-a053da7eed4c" /dev/sdb2
Setting up swapspace version 1, size = 262140 KiB
LABEL=, UUID=77d939ef-54d6-427a-a2bf-a053da7eed4c
Bug 775932 - Refactor mostly applying of operations
resize_filesystem() was meeting two different needs:
1) when called with fill_partition = false it generated operation
details;
2) when called from maximize_filesystem() with fill_partition = true it
skipped generating any operation details;
then ran the switch statement to select the resize implementation. So
extract the common switch statement into new method
resize_filesystem_implement().
Then observe that the only time resize_filesystem() was called to grow
the file system was when re-creating linux-swap. Therefore change that
call to use maximize_filesystem() and rename to shrink_filesystem() and
modify the operation detail messages to match.
Bug 775932 - Refactor mostly applying of operations
Make the methods called below apply_operation_to_disk() follow a
standard naming convention:
* Contains "_partition"
Uses libparted to query or change the partition in the disk label
(partition table).
E.g.:
calibrate_partition()
create_partition()
delete_partition()
name_partition()
resize_move_partition()
set_partition_type()
* Contains "_filesystem"
Manipulates the file system within the partition, mostly using the
FileSystem and derived class methods.
E.g.:
create_filesystem()
remove_filesystem()
label_filesystem()
copy_filesystem()
erase_filesystem_signatures()
check_repair_filesystem()
resize_filesystem()
maximize_filesystem()
* Other
Compound method calling multiple partition and file system related
apply methods.
E.g.:
create()
format()
copy()
resize_move()
resize()
move()
Rename:
Delete() -> delete_partition()
change_uuid() -> change_filesystem_uuid()
Bug 775932 - Refactor mostly applying of operations
Until now an encryption mapping has been modelled as a Partition object
similar to a partition like this:
.encrypted.device_path = "/dev/sdb1"
.encrypted.path = "/dev/mapper/sdb1_crypt"
.encrypted.whole_device = false
.encrypted.sector_start = // start of the mapping in the partition
.encrypted.sector_end = // end of the mapping in the partition
However accessing device_path in the start to end sector range is not
equivalent to accessing the partition path as it doesn't provide access
to the encrypted data. Therefore existing functions which read and
write partition data (GParted file system copying and signature erasure)
via libparted using the device_path won't work and will in fact destroy
the encrypted data. This could be coded around with an extra case in
the device opening code, however it is not necessary.
An encrypted block special device /dev/mapper/CRYPTNAME looks just like
a whole disk device because it doesn't contain a partition and the file
system it contains starts at sector 0 and goes to the end. Therefore
model an encryption mapping in the same way a whole disk device is
modelled as a Partition object like this:
.encrypted.device_path = "/dev/mapper/sdb1_crypt"
.encrypted.path = "/dev/mapper/sdb1_crypt"
.encrypted.whole_device = true
.encrypted.sector_start = 0
.encrypted.sector_end = // size of the encryption mapping - 1
Now GParted file system copy and erasure will just work without any
change. Just need to additionally store the LUKS header size, which was
previous stored in the sector_start, for use in the
get_sectors_{used,unused,unallocated}() calculations.
Bug 775932 - Refactor mostly applying of operations
Split out the switch statement selecting the copy implementation and
associated copy file system operation detail message into a separate
copy_filesystem() method, matching how a number of other operations are
coded. This is why the previous copy_filesystem() methods needed
renaming.
Re-write the remaining copy() into if-operation-fails-return-false style
to simplify it. Re-write final complicated conditional check repair and
maximise file system into separate positive if conditions for swap and
larger partition to make it understandable.
The min_size parameter to copy() was queried from the partition_src
parameter also passed to copy(). Drop the parameter and query inside
copy() instead.
Bug 775932 - Refactor mostly applying of operations
Mike Fleetwood