Older versions of blkid don't correctly distinguish between FAT16 and
FAT32 file systems when overwriting one with the other. This effects
GParted too with these file systems on whole disk devices where only
blkid is used to recognise the contents. See previous fix for why only
blkid is used in this case:
Avoid whole disk FAT being detected as MSDOS partition table
(#743181)
Example:
# blkid -v
blkid from util-linux 2.20.1 (liblkid 2.20.0, 19-Oct-2011)
# mkdosfs -F16 -I /dev/md1
# blkid | fgrep md1
/dev/md1: SEC_TYPE="msdos" UUID="7C23-95D9" TYPE="vfat"
# mkdosfs -F32 -I /dev/md1
# blkid | fgrep md1
/dev/md1: SEC_TYPE="msdos" UUID="7F93-98F4" TYPE="vfat"
So blkid recognised the UUID changed but didn't remove the SEC_TYPE for
the FAT32 file system. See FS_Info::get_fs_type() as it uses this to
distinguish between FAT16 and FAT32. This is a caching update bug in
blkid, because telling blkid not to use the cache gets the right
results:
# blkid -c /dev/null | fgrep md1
/dev/md1: UUID="7F93-98F4" TYPE="vfat"
With testing determined that blkid from util-linux 2.23 and later are
not affected and earlier versions are affected. Mostly recently known
affected distribution is Ubuntu 14.04 LTS with util-linux 2.20.1.
The straight forward fix would be to instruct blkid to not use its cache
with 'blkid -c /dev/null'. But using blkid's cache is needed to prevent
blkid hanging for minutes when trying to access a non-existent floppy
drive when the BIOS is set incorrectly. See commit:
18f863151c
Fix long scan problem when BIOS floppy setting incorrect
Instead, when using an older affected version of blkid and when blkid
cache reports a vfat file system, run blkid again bypassing the cache.
The device is known to exist and contain a vfat file system, just not
whether it is a FAT16 or FAT32 file system, so can't be a non-existent
floppy device and won't hang.
Bug 743181 - Add unpartitioned drive read-write support
Libparted 1.9.0 to 2.3 inclusive, recognises whole disk device FAT file
systems as MSDOS partition tables. This causes GParted to do the same.
# dd if=/dev/zero bs=1M of=/dev/md4
# mkdosfs -F32 -v -I /dev/md4
# blkid /dev/md4
/dev/md4: UUID="53FE-31F2" TYPE="vfat"
# parted /dev/md4
GNU Parted 2.1
Using /dev/md4
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) print
Model: Unknown (unknown)
Disk /dev/md4: 536MB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number Start End Size Type File system Flags
(parted) quit
# /tmp/parted24/bin/parted /dev/md4
GNU Parted 2.4
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) print
Model: Linux Software RAID Array (md)
Disk /dev/md4: 536MB
Sector size (logical/physical): 512B/512B
Partition Table: loop
Number Start End Size File system Flags
1 0.00B 536MB 536MB fat32
(parted) quit
This was fixed in libparted 2.4 by commit:
http://git.savannah.gnu.org/cgit/parted.git/commit/?id=616a2a1659d89ff90f9834016a451da8722df509
libparted: avoid regression when processing a whole-disk FAT partition
Make GParted immune to this bug by moving blkid performed whole disk
device file system detection before libparted partition detection. Also
have to always erase old file system signatures on whole disk devices
when creating new partition tables to ensure that blkid doesn't detect
those old signatures before libparted has a chance to detect the new
partition table.
Bug 743181 - Add unpartitioned drive read-write support
When writing "loop" partition table over the top of some whole disk
device file system types GParted continued to show those whole disk
device file systems rather than the virtual unknown partition from the
"loop" partition table.
This affected btrfs, jfs, reiser4 and reiserfs. It occurred because of
several factors:
1) Libparted only zeroed the first and last 9.5 KiB (assuming 512 byte
sectors) of the device before writing a new partition table. See
ped_disk_clobber().
2) These file systems have their super blocks and therefore signatures
after the first 9.5 KiB.
3) Whole disk device file system detection is performed using blkid
before checking for a libparted "loop" partition table. See
GParted_Core::set_devices_thread().
Ref:
libparted 3.2: disk.c:ped_disk_clobber()
http://git.savannah.gnu.org/cgit/parted.git/tree/libparted/disk.c?id=v3.2#n302
Fix by always erasing any possible file system signatures on the device
before creating a new "loop" partition table.
NOTE:
This is typically taking up to 0.5 seconds in my testing on a 5400 RPM
hard drive, during which time the GParted UI is hung and the create
partition table dialog shows the apply button pressed but no other
progress indication.
Bug 743181 - Add unpartitioned drive read-write support
Creating a new partition table was getting libparted to read any
existing partition table before creating a new partition table on the
device. This is an unnecessary step, and if the device didn't already
contain a partition table also printed this error from libparted:
/dev/sdb: unrecognised disk label
Since get_device_and_disk() has been split into two, just call
get_device() instead to just populate the PedDevice object representing
the disk device. Removes a small unnecessary step.
Bug 743181 - Add unpartitioned drive read-write support
The preview of clearing a whole disk device file system was previewing
the same as formatting to all other file system types; as a cleared file
system spanning the whole disk device. However when implemented this
removes all signatures on the disk so it actually becomes an unallocated
and unpartitioned device. Make the preview match what happens in when
implemented.
GParted previously used mydevice.max_prims = -1 to represent an
unpartitioned device. It is now represented as:
mydevice.max_prims = 1
mydevice.disktype = _("unrecognized")
mydevice.partitions[0].type = TYPE_UNALLOCATED
mydevice.partitions[0].whole_device = true
mydevice.partitions[0].filesystem = FS_UNALLOCATED
and the check for an unpartitioned device in Win_GParted.cc becomes:
partitions[0].type == TYPE_UNALLOCATED && partitions[0].whole_device
Bug 743181 - Add unpartitioned drive read-write support
Previously GParted displayed a device containing the parted "loop"
partition table signature "GNU Parted Loopback 0" and nothing else, as
an unrecognised device.
Now make GParted display this as a virtual whole disk device partition
with unknown contents, complete with the unable to detect a file system
warning. This change then allows a whole disk device file system to be
created with the following two steps:
1) Create "loop" partition table on a device;
2) Format to required file system.
GParted represents a whole disk device file system as:
mydevice.max_prims = 1
mydevice.disktype = "none"
mydevice.partitions[0].type = TYPE_PRIMARY
mydevice.partitions[0].whole_device = true
mydevice.partitions[0].filesystem = FS_EXT4 (example)
Now represents just Parted's "loop" signature as:
mydevice.max_prims = 1
mydevice.disktype = "loop"
mydevice.partitions[0].type = TYPE_PRIMARY
mydevice.partitions[0].whole_device = true
mydevice.partitions[0].filesystem = FS_UNKNOWN
And as before, an unpartitioned device as:
mydevice.max_prims = -1
mydevice.disktype = _("unrecognized")
mydevice.partitions[0].type = TYPE_UNALLOCATED
mydevice.partitions[0].whole_device = true
mydevice.partitions[0].filesystem = FS_UNALLOCATED
Bug 743181 - Add unpartitioned drive read-write support
Only allow resizing, not moving of a whole disk device file system.
There is no actual partition to move and moving a file system away from
the start of a disk only makes it unrecognisable.
Also don't perform the partition resize step as there's no actual
partition to be resized. Only the file system is being resized.
(Libparted actually allows the virtual partition spanning a whole disk
device to be resized, implementing it as a no-operation, but only for
recognised file systems. For unrecognised file systems it fails with
"unrecognised disk label").
Note that the existing resize dialog was designed for resizing partition
boundaries, and their contained file systems, not for resizing file
systems within a fixed boundary. The difference is noticeable when
there is unallocated space because the file system doesn't fill the
whole disk device. The dialog starts resizing a virtual partition the
size of the whole disk device, not the actual size of the file system.
Leave addressing this for a possible future update.
Bug 743181 - Add unpartitioned drive read-write support
First, copying into a whole disk device fails on the set partition type
step. Fails with either libparted error "The flag 'lvm' is not
available for loop disk labels" or "unrecognised disk label" depending
whether libparted recognised the content and created a virtual partition
or not. (This is with libparted 2.4).
Fix by just skipping setting the partition type on whole disk devices.
Second, if any file system specific tools are used during the copy, they
will fail because they are passed the device name as "copy of /dev/SRC"
instead of "/dev/DST". Occurs when either the destination whole disk
device is not an identical size to the source so the file system check
and grow steps are added, or when file system specific tools are used to
copy the file system as with XFS or recent EXT2/3/4 tools.
Fix by re-adding the real partition path from libparted for whole disk
devices, as is already done for partitioned device names in
GParted_Core::calibrate_partition().
Bug 743181 - Add unpartitioned drive read-write support
Creation of reiserfs file system fails in GParted with the this error.
# mkreiserfs -f --label "" /dev/sdb < /dev/null
mkreiserfs 3.6.24
/dev/sdb is entire device, not just one partition!
Continue (y/n):
# echo $?
1
Add second force flag, -f, to the mkreiserfs command to make it work.
Bug 743181 - Add unpartitioned drive read-write support
Creation of ext2/3/4 and ntfs file systems fails in GParted on whole
disk devices with these errors.
# mkfs.ext4 -L "" /dev/sdb < /dev/null
mke2fs 1.42.9 (4-Feb-2014)
/dev/sdb is entire device, not just one partition!
Proceed anyway? (y,n)
# echo $?
1
# mkntfs -Q -v -L "" /dev/sdc
/dev/sdc is entire device, not just one partition.
Refusing to make a filesystem here!
# echo $?
1
Add force flag, -F, to both mkfs commands to make them work.
Bug 683643 - Doesn't properly support partitionless drives.
Formatting a whole disk device fails on the set partition type step with
libparted error "unrecognised disk label". This is because the previous
step just cleared the old file system signatures leaving libparted with
nothing to recognise. Therefore libparted doesn't present a virtual
"loop" partition table.
As there is no partition table, there's no partition and no partition
type. Just skip setting the partition type on whole disk devices.
Bug 743181 - Add unpartitioned drive read-write support
This enables Format to Cleared operation to succeed on whole disk device
file systems even when libparted doesn't recognise the file system.
(Turns out that making calibrate work in the previous commit happened to
make Format to Cleared operation succeed, but only if libparted
recognised the file system on the whole disk device).
Bug 743181 - Add unpartitioned drive read-write support
In the operational results of the calibrate step include the type of the
path GParted is working with, either partition or whole disk device. Do
the same for the create empty partition step too for consistency, even
though it only ever creates partitions. Looks like:
create empty partition
path: /dev/sdb3 (partition)
start: 2099200
end: 4196351
size: 2097152 (1.00 GiB)
calibrate /dev/sdc
path: /dev/sdc (device)
start: 0
end: 1953525167
size: 1953525168 (931.51 GiB)
Makes it explicit to the users what GParted has detected. Helps the
developers when looking at saved results to understand what decisions
were made and why specific steps were performed or not.
Bug 743181 - Add unpartitioned drive read-write support
This enables the Check, Label and New UUID operations to succeed on
whole disk device file systems even when libparted doesn't recognise the
file system.
This benefits reiser4 and lvm2 pv file systems with all versions of
libparted, current version is 3.2, and for nilfs2 with libparted < 2.4.
Bug 743181 - Add unpartitioned drive read-write support
get_device_and_disk() basically calls libparted to get a PedDevice
object representing a disk device and a PedDisk object representing a
partition table. Re-implement get_device_and_disk() using two separate
functions, get_device() and get_disk(), to get one of these objects
each.
No functionality changes with this commit. It enables future commits to
incrementally add support for whole disk devices into GParted without
needing libparted to recognise the contents and create a virtual "loop"
partition table.
Bug 743181 - Add unpartitioned drive read-write support
Enable operations on whole disk devices containing any recognised file
system.
The new partition operation on an empty whole disk device continues to
display the "No partition table found on device /dev/DEVICE" information
dialog.
Specifically unsupported operations:
* Delete -
Deletion of a partition only involves removal of the entry in the
partition table leaving the file system intact on the disk. However
this doesn't work for a whole disk device file system. Instead the
file system signatures would have to be erased which is much more
destructive and virtually impossible to undo. Therefore don't
implement whole disk device file system deletion. Alternatives are
to format the file system to cleared or create a partition table on
the device. Both of these imply overwriting the existing data and
set the expectation that undo is not possible.
* Manage flags -
There's no partition table, so there's no partition, so there's no
flags.
Resize/Move operation is being supported so that a whole disk device
file system can be resized to handle devices which can be resize, such
as those from SANs or Linux Software RAID arrays. The start of the file
system must remain fixed so move won't be allowed.
So far only simple operations work if they don't need libparted support
at all [1], or only need libparted support for the calibrate step AND
the file system on the whole disk device is recognised by libparted [2].
(Needs libparted to provide a "loop" partition, hence the recognition
requirement, so that the calibrate step can successfully read the
virtual "loop" partition table. Doesn't matter whether it's an old
version of libparted and it gets the name of the device wrong as GParted
is already using the whole disk device name anyway).
[1] Operations not needing any libparted support:
Mount on, Unmount, Swapon, Swapoff, Activate and Deactivate
[2] Operations only needing libparted support for the calibrate step:
Check, Label, New UUID
Bug 743181 - Add unpartitioned drive read-write support
Need to be able to take different actions in the GParted_Core partition
manipulation methods and in Win_GParted UI methods to deal with
libparted supported partitions or whole disk devices without a partition
table. Add boolean whole_device to the partition object and set
appropriately to allow for this.
Bug 743181 - Add unpartitioned drive read-write support
As was done with a failed mount operation, include the failed activate/
deactivate command in the error dialog. Two example error dialogs now
look like this:
(-) Could not deactivate swap
# swapoff -v /dev/sdb7
swapoff: /dev/sdb7: swapoff failed: Invalid argument
[ OK ]
(-) Could not unmount /dev/sdb6
# umount -v "/mnt/6"
umount: /mnt/6: not mounted
[ OK ]
On RHEL/CentOS 6, GParted fails to mount nilfs2 file system like this:
# mkfs.nilfs2 /dev/sdb1
# mount /dev/sdb1 /mnt/1
mount: you must specify the filesystem type
This fails because mount internally uses libblkid to determine the file
system type when it is not specified on the command line. However on
RHEL/CentOS 6 libblkid is too old to recognise nilfs2.
GParted used libparted recognition first and blkid second. Mount only
uses libblkid. When there are multiple signatures on a partition
GParted may report a different result to blkid alone. Therefore fix by
first trying to mount the file system without specifying the type, as is
already done, and if that fails, trying specifying the file system type.
This allows GParted to mount nilfs2 file systems.
# mount -t nilfs2 /dev/sdb1 /mnt/1
# mount | fgrep sdb1
/dev/sdb1 on /mnt/1 type nilfs2 (rw,gcpid=30946)
And for unsupported file systems the error dialog from the failed mount
command shows both commands like this:
(-) Could not mount /dev/sdb3 on /mnt/3
# mount -v /dev/sdb3 "/mnt/3"
mount: unknown filesystem type 'reiser4'
# mount -v -t reiser4 /dev/sdb3 "/mnt/3"
mount: unknown filesystem type 'reiser4'
[ OK ]
Bug 742741 - Nilfs2 file system is unusable on RHEL/CentOS 6
Function Utils::get_filesystem_kernel_name() returns the name of the
file system as needed for use in the mount command:
mount -t TYPE DEVICE DIR
Needed because the kernel / mount name is 'hfsplus' where as libparted /
GParted, as reported by Utils::get_filesystem_string(), calls it 'hfs+'.
So far just added debugging when mounting a file system to test the
function works.
# ./gartedbin
======================
libparted : 2.1
======================
DEBUG: (hfsplus) # mount -v /dev/sdb5 "/mnt/5"
DEBUG: (nilfs2) # mount -v /dev/sdb1 "/mnt/1"
Bug 742741 - Nilfs2 file system is unusable on RHEL/CentOS 6
On RHEL/CentOS 6 GParted fails to mount nilfs2 file system. Include the
failing file system mount command in the error dialog so the user knows
what command failed. The error dialog now looks like:
(-) Could not mount /dev/sdc1 on /mnt/1
# mount -v /dev/sdc1 "/mnt/1"
mount: you must specify the filesystem type
[ OK ]
Also stop telling the dialog that the secondary text contains pango
markup as the command line and error message certainly isn't pango
markup text.
Bug 742741 - Nilfs2 file system is unusable on RHEL/CentOS 6
GParted can create a nilfs2 file system on RHEL/CentOS 6, yet both
libparted and blkid are too old to recognise it when created. Add
GParted internal detection of nilfs2 file systems.
Easy way to get the required signature needed to recognise nilfs2 is
using wipefs to report the signature bytes it erases, using a more up to
date distribution. So using CentOS 7:
# mkfs.nilfs2 /dev/sdb1
# wipefs -a /dev/sdb1
/dev/sdb1: 2 bytes were erased at offset 0x00000406 (nilfs2): 34 34
Also from util-linux source libblkid/src/superblocks/nilfs2.c
#define NILFS_SB_MAGIC 0x3434
Bug 742741 - Nilfs2 file system is unusable on RHEL/CentOS 6
For libparted recognised file systems covering the whole disk device,
libparted reports the partition table as "loop". With libparted 3.1 and
earlier, the partition device was reported wrongly as /dev/sdb1, instead
of /dev/sdb, by the ped_partition_get_path() call. This causes GParted
to fail to report file system usage, label and UUID, and busy status.
Also multiple file system tool command errors from using the wrong
device are displayed.
# mkfs.ext4 -L whole-ext4 /dev/sdb
# ./gpartedbin
Example partition errors:
e2label: No such file or directory while trying to open /dev/sdb1
Couldn't find valid filesystem superblock.
tune2fs 1.42.8 (20-Jun-2013)
tune2fs: No such file or directory while trying to open /dev/sdb1
Couldn't find valid filesystem superblock.
dumpe2fs 1.42.8 (20-Jun-2013)
dumpe2fs: No such file or directory while trying to open /dev/sdb1
Couldn't find valid filesystem superblock.
Unable to read the contents of this file system!
Because of this some operations may be unavailable.
This cause might be a missing software package.
The following list of software packages is required for ext4 file
system support: e2fsprogs v1.41+.
This also breaks most of the manipulation operations, again because it
is using the wrong device name.
Fix by ignoring libparted's "loop" partition table and just let it get
displayed via the "none" partition table mechanism.
Doing this renders the direction taken in bug 683643 to make GParted
work with libparted "loop" partition tables mute, as GParted no longer
works with such partition tables. Instead they are replaced by "none"
partition tables within GParted.
Bug 683643 - Doesn't properly support partitionless drives
https://bugzilla.gnome.org/show_bug.cgi?id=683643
Bug 741430 - GParted cannot recognise LVM signature on unpartitioned
drive
So that, even though the menu item is greyed out, it is correctly named
Activate or Deactivate for an LVM2 Physical Volume on a whole disk
device, rather than the default Mount or Unmount.
Bug 741430 - GParted cannot recognise LVM signature on unpartitioned
drive
At this time we are providing read-only support for file systems on
whole disk devices. Also all manipulation operations fail at the
calibrate step with libparted error "unrecognised disk label", and
manage flags is for real partitions only.
Bug 741430 - GParted cannot recognise LVM signature on unpartitioned
drive
For file systems which libparted recognises, when found on the whole
disk device, it reports with partition table "loop" and a partition
covering the whole disk. GParted duly displays this to the user.
For file systems which libparted doesn't recognise it reports
"unrecognised disk label". As of the latest libparted 3.2, these file
system aren't recognised and can't currently be shown when on the whole
disk device:
BitLocker, Crypt LUKS, exFAT, F2FS, LVM2 Physical Volume,
Linux Software RAID, ReFS, Reiser 4
So only when libparted doesn't recognise a file system on the whole disk
device and GParted does, either via blkid or it's internal code, display
this with partition table "none".
Bug 741430 - GParted cannot recognise LVM signature on unpartitioned
drive
Move code which queries the file system label and UUID of a partition
into a separate helper function.
Bug 741430 - GParted cannot recognise LVM signature on unpartitioned
drive
Make GParted_Core::get_filesystem() capable of recognising file system
signatures on whole disk devices. Pass NULL for lp_partition to make it
read signatures from the whole disk rather than a specific partition.
Add debugging to test it works.
# fgrep sdb /proc/partitions
8 16 8388608 sdb
# lvm pvcreate /dev/sdb
Writing physical volume data to disk "/dev/sdb"
Physical volume "/dev/sdb" successfully created
# ./gpartedbin
======================
libparted : 1.8.1
======================
Unable to open /dev/sdb - unrecognised disk label.
DEBUG: /dev/sdb (lvm2 pv)
Bug 741430 - GParted cannot recognise LVM signature on unpartitioned
drive
Used 'fstype' as the name of local variable storing the FILESYSTEM
enumeration type.
'fs_type' was the name of the local variable storing the string with the
name of the file system type. Rename it to 'fsname' to distinguish it
from the aforementioned variable.
Bug 741430 - GParted cannot recognise LVM signature on unpartitioned
drive
Refactor GParted internal file system signature detection to remove code
duplication. There were 5 separate copies of code to: allocate a
buffer, open, read and close the device, free the buffer and compare the
signature.
Bug 741430 - GParted cannot recognise LVM signature on unpartitioned
drive
(1) Add recognition of LUKS signatures detected by blkid command, via
the FS_Info cache.
(2) Relocate internal GParted detection of LUKS signature to back after
libparted and blkid identification. This reverses previous commit:
7695593d21
Improve crypt-luks detection (#578300)
Bug 578300 must have been caused by either: (1) reading stale data from
the whole disk device, or (2) formatting as LUKS over the top of ext3
leaving old signatures behind. Case (1) has been resolved by commit:
3bea067596
Flush devices when scanning to prevent reading stale signatures
(#723842)
Re-tested case (2) and find that no old signatures are left behind:
# mkfs.ext3 /dev/sda8
# wipefs /dev/sda8
offset type
----------------------------------------------------------------
0x438 ext3 [filesystem]
UUID: 756dcd5f-73c5-4881-81ee-dc88e8b18f42
# cryptsetup luksFormat /dev/sda8
# wipefs /dev/sda8
offset type
----------------------------------------------------------------
0x0 crypto_LUKS [crypto]
UUID: de15ed1a-f461-46b7-803b-d0fd3b4bd680
Bug 741430 - GParted cannot recognise LVM signature on unpartitioned
drive
Trying to create an XFS file system causes the following error and
the create new xfs file system step to never complete.
# ./gpartedbin
======================
libparted : 2.4
======================
(gpartedbin:15572): glibmm-CRITICAL **:
unhandled exception (type Glib::Error) in signal handler:
domain: g-shell-error-quark
code : 0
what : Text ended before matching quote was found for ". (The text was 'mkfs.xfs -f -L "mylabel" " /dev/sdb8')
Creation of XFS file systems was broken by the extra stray double quote
added to the command before the device name, by commit:
63aeb150ac
Rename member variables and methods in Partition class (#741424)
Bug 744108 - Creating an XFS file system never completes
Encountered this dialog:
Libparted Bug Found!
(-) Invalid argument during seek for read on /dev/md1
[ Retry ] [ Cancel ] [ Ignore ]
After investigation found that GParted was asking libparted to read from
sector -1. Libparted was actually reporting an error but GParted was
incorrectly displaying this as a bug. This was because setting the
dialog title was falling through from the error case to the bug case.
Fix this.
Also provide unique titles for all libparted exception types so that
they can all be distinguished.
* List of exceptions from libparted: enum _PedExceptionType
http://www.gnu.org/software/parted/api/group__PedException.html
Bug 743825 - gparted displaying dialog: libparted bug found!
Mike Fleetwood