So far the read file system usage figures, read via the file system
interface classes using file system specific tools, have been checked to
the exact sector for:
0 <= used <= size
0 <= unused <= size
unallocated = 0
used + unused = size
However for JFS and NTFS this fails like this:
# ./test_SupportedFileSystems --gtest_filter='*ReadUsage/*' | fgrep ' ms'
[ OK ] My/SupportedFileSystemsTest.CreateAndReadUsage/btrfs (335 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndReadUsage/exfat (0 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndReadUsage/ext2 (38 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndReadUsage/ext3 (131 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndReadUsage/ext4 (32 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndReadUsage/f2fs (47 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndReadUsage/fat16 (19 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndReadUsage/fat32 (48 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndReadUsage/hfs (0 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndReadUsage/hfsplus (0 ms)
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadUsage/jfs, where GetParam() = 17 (73 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndReadUsage/linuxswap (20 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndReadUsage/luks (0 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndReadUsage/lvm2pv (410 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndReadUsage/minix (0 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndReadUsage/nilfs2 (226 ms)
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadUsage/ntfs, where GetParam() = 23 (56 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndReadUsage/reiser4 (49 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndReadUsage/reiserfs (139 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndReadUsage/udf (34 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndReadUsage/xfs (67 ms)
[----------] 21 tests from My/SupportedFileSystemsTest (1726 ms total)
[==========] 21 tests from 1 test case ran. (1726 ms total)
# ./test_SupportedFileSystems --gtest_filter='*ReadUsage/jfs:*ReadUsage/ntfs'
Running main() from test_SupportedFileSystems.cc
Note: Google Test filter = *ReadUsage/jfs:*ReadUsage/ntfs
[==========] Running 2 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 2 tests from My/SupportedFileSystemsTest
[ RUN ] My/SupportedFileSystemsTest.CreateAndReadUsage/jfs
test_SupportedFileSystems.cc:465: Failure
Expected equality of these values:
m_partition.sectors_unallocated
Which is: 2472
0
test_SupportedFileSystems.cc:517: Failure
Expected equality of these values:
m_partition.sectors_used + m_partition.sectors_unused
Which is: 521816
m_partition.get_sector_length()
Which is: 524288
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadUsage/jfs, where GetParam() = 17 (36 ms)
[ RUN ] My/SupportedFileSystemsTest.CreateAndReadUsage/ntfs
test_SupportedFileSystems.cc:465: Failure
Expected equality of these values:
m_partition.sectors_unallocated
Which is: 8
0
test_SupportedFileSystems.cc:517: Failure
Expected equality of these values:
m_partition.sectors_used + m_partition.sectors_unused
Which is: 524280
m_partition.get_sector_length()
Which is: 524288
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadUsage/ntfs, where GetParam() = 23 (35 ms)
[----------] 2 tests from My/SupportedFileSystemsTest (71 ms total)
[----------] Global test environment tear-down
[==========] 2 tests from 1 test case ran. (72 ms total)
[ PASSED ] 0 tests.
[ FAILED ] 2 tests, listed below:
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadUsage/jfs, where GetParam() = 17
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadUsage/ntfs, where GetParam() = 23
2 FAILED TESTS
So JFS is reporting 2472 unallocated sectors in a size of 524288 sectors
and NTFS is reporting 8 unallocated sectors in the same size. This
exact issue is already solved for GParted so that it doesn't show a
small amount of unallocated space by commits [1][2] from Bug 499202 [3].
Fix the same way, use the accessors to the file system usage figures
which don't show unallocated space when it is below the significant
threshold.
[1] b5c80f18a9
Enhance calculation of significant unallocated space (#499202)
[2] 7ebedc4bb3
Don't show intrinsic unallocated space (#499202)
[3] Bug 499202 - gparted does not see the difference if partition size
differs from filesystem size
https://bugzilla.gnome.org/show_bug.cgi?id=499202Closes!49 - Add file system interface tests
Checking a MINIX V3 file system fails like this:
$ ./test_SupportedFileSystems --gtest_filter='*Check/minix'
...
[ RUN ] My/SupportedFileSystemsTest.CreateAndCheck/minix
test_SupportedFileSystems.cc:554: Failure
Value of: m_fs_object->check_repair(m_partition, m_operation_detail)
Actual: false
Expected: true
Operation details:
mkfs.minix -3 '/home/centos/programming/c/gparted/tests/test_SupportedFileSystems.img' 00:00:00 (SUCCESS)
87392 inodes
262144 blocks
Firstdatazone=5507 (5507)
Zonesize=1024
Maxsize=2147483647
fsck.minix '/home/centos/programming/c/gparted/tests/test_SupportedFileSystems.img' 00:00:00 (ERROR)
fsck.minix from util-linux 2.23.2
bad magic number in super-block
[ FAILED ] My/SupportedFileSystemsTest.CreateAndCheck/minix, where GetParam() = 21 (182 ms)
fsck.minix doesn't support checking MINIX V3 file systems until this
commit, first included in util-linux 2.27 released 2015-09-07.
https://git.kernel.org/pub/scm/utils/util-linux/util-linux.git/commit/?id=86a9f3dad58addb50eca9daa9d233827a005dad7
fsck.minix: add minix v3 support
CentOS 7 only includes util-linux 2.23.2 so is affected by this, however
Ubuntu 18.04 LTS includes util-linux 2.31.1 so is not affected.
Just always skip this test for now. Plan to re-enable later when the
oldest supported distributions and GitLab CI images include the needed
util-linux release.
Closes!49 - Add file system interface tests
The tests were failing like this:
$ ./test_SupportedFileSystems --gtest_filter='*CreateAndReadUUID/fat16'
....
[ RUN ] My/SupportedFileSystemsTest.CreateAndReadUUID/fat16
test_SupportedFileSystems.cc:552: Failure
Expected equality of these values:
m_partition.uuid.size()
Which is: 9
36U
Which is: 36
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadUUID/fat16, where GetParam() = 13 (45 ms)
This is because the test was expecting a full 36 character UUID as used
by Linux file systems. Also accept shorter 9 character "UUID"s as used
by FAT16/32 file systems.
Closes!49 - Add file system interface tests
For NILFS2 the read and write tests which use nilfs-tune all fail using
an image file, even when run as root, however the other tests succeed.
Selected output from the test program:
# ./test_SupportedFileSystems --gtest_filter='*/nilfs2' | fgrep ' ms'
[ OK ] My/SupportedFileSystemsTest.Create/nilfs2 (22 ms)
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadUsage/nilfs2, where GetParam() = 22 (31 ms)
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadLabel/nilfs2, where GetParam() = 22 (30 ms)
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadUUID/nilfs2, where GetParam() = 22 (30 ms)
[ FAILED ] My/SupportedFileSystemsTest.CreateAndWriteLabel/nilfs2, where GetParam() = 22 (37 ms)
[ FAILED ] My/SupportedFileSystemsTest.CreateAndWriteUUID/nilfs2, where GetParam() = 22 (39 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndCheck/nilfs2 (0 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndRemove/nilfs2 (0 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndGrow/nilfs2 (386 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndShrink/nilfs2 (345 ms)
[----------] 10 tests from My/SupportedFileSystemsTest (920 ms total)
[==========] 10 tests from 1 test case ran. (920 ms total)
nilfs-tune fails like this when given an image file:
# truncate -s 256M test.img
# mkfs.nilfs2 test.img
mkfs.nilfs2 (nilfs-utils 2.2.7)
Start writing file system initial data to the device
Blocksize:4096 Device:test.img Device Size:268435456
File system initialization succeeded !!
# nilfs-tune -l test.img
nilfs-tune 2.2.7
nilfs-tune: test.img: cannot open NILFS
# echo $?
1
However using nilfs-tune via a loop device works:
# losetup --show --find /dev/loop0
/dev/loop0
# nilfs-tune -l /dev/loop0
nilfs-tune 2.2.7
Filesystem volume name: (none)
Filesystem UUID: fc49912c-4d39-4672-8610-1e1185d0db5f
Filesystem magic number: 0x3434
Filesystem revision #: 2.0
Filesystem features: (none)
Filesystem state: valid
Filesystem OS type: Linux
Block size: 4096
...
So nilfs-tune only works with block devices. Fix by making these tests
require a loop device and therefore make them root only. Now these
tests are skipped as non-root user and pass as root.
Closes!49 - Add file system interface tests
File systems BTRFS, JFS, NILFS2 and XFS can only be resized while
mounted, but only root can mount file systems. Therefore these tests
fail. Also BTRFS resize uses 'btrfs filesystem show' to discover the
devid, which also fails as described in the previous commit message.
Note that root can mount a file system image directly, but that it
implicitly creates loop device:
# truncate -s 256M test.img
# mkfs.xfs test.img
# mount test.img /mnt/1
# fgrep /mnt/1 /proc/mounts
/dev/loop0 /mnt/1 xfs rw,seclabel,relatime,attr2,inode64,noquota 0 0
# losetup -a
/dev/loop0: [64768]:35826659 (/root/test.img)
Therefore make these tests root only and require an explicit loop
device. Now these file system resize tests succeed as root and are
skipped as non-root.
Closes!49 - Add file system interface tests
For BTRFS the read (and resize) tests fail when using an image file,
however the create, write and check tests pass. Selected output from
the test program:
$ ./test_SupportedFileSystems --gtest_filter='*/btrfs' | fgrep ' ms'
[ OK ] My/SupportedFileSystemsTest.Create/btrfs (43 ms)
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadUsage/btrfs, where GetParam() = 7 (95 ms)
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadLabel/btrfs, where GetParam() = 7 (158 ms)
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadUUID/btrfs, where GetParam() = 7 (164 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndWriteLabel/btrfs (164 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndWriteUUID/btrfs (132 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndCheck/btrfs (129 ms)
[ OK ] My/SupportedFileSystemsTest.CreateAndRemove/btrfs (0 ms)
[ FAILED ] My/SupportedFileSystemsTest.CreateAndGrow/btrfs, where GetParam() = 7 (155 ms)
[ FAILED ] My/SupportedFileSystemsTest.CreateAndShrink/btrfs, where GetParam() = 7 (97 ms)
[----------] 10 tests from My/SupportedFileSystemsTest (1137 ms total)
[==========] 10 tests from 1 test case ran. (1137 ms total)
The read operations fail because 'btrfs filesystem show' doesn't work on
am image file:
$ truncate -s 256M test.img
$ mkfs.btrfs test.img
btrfs-progs v4.9.1
See http://btrfs.wiki.kernel.org for more information.
Label: (null)
UUID: de1624ae-39bb-4796-aee4-7ee1fa24c06a
Node side: 16384
Sector size: 4096
Filesystem size: 256.00MiB
Block group profiles:
Data: single
Metadata: DUP
System: DUP
SSD detected: no
Incompat features: extref, skinny-metadata
Number of devices: 1
Devices:
ID SIZE PATH
1 256.00MiB test.img
$ btrfs filesystem show test.img
ERROR: not a valid btrfs filesystem: /home/centos/programming/c/gparted/tests/test.img
$ echo $1
1
Querying a BTRFS image file also fails as root:
$ su
Password:
# btrfs filesystem show test.img
ERROR: not a valid btrfs filesystem: /home/centos/programming/c/gparted/tests/test.img
# echo $1
1
However querying the BTRFS via a loop device succeeds:
# losetup --show --find test.img
/dev/loop0
# btrfs filesystem show /dev/loop0
Label: none uuid: de1624ae-39bb-4796-aee4-7ee1fa24c06a
Total devices 1 FS bytes used 112.00KiB
devid 1 size 256.00MiB used 88.00MiB path /root/test.img
There must be some kernel level BTRFS file system device discovery
happening because now after creating a loop device for the image file,
the BTRFS can be shown via the image file directly:
# btrfs filesystem show test.img
Label: none uuid: de1624ae-39bb-4796-aee4-7ee1fa24c06a
Total devices 1 FS bytes used 112.00KiB
devid 1 size 256.00MiB used 88.00MiB path /root/test.img
Anyway for the BTRFS reading tests make them required a loop device and
therefore root only. Now these tests are skipped as non-root user and
pass as root.
Addressing BTRFS resizing test failures will be handled in a following
commit.
Closes!49 - Add file system interface tests
Creating an LVM2 PV as a non-root user on an image file fails like this:
$ truncate -s 256M test.img
$ lvm pvcreate `pwd`/test.img
WARNING: Running as a non-root user. Functionality may be unavailable.
/run/lvm/lvmetad.socket: access failed: Permission denied
WARNING: Failed to connect to lvmetad. Falling back to device scanning.
/run/lock/lvm/P_orphans:aux: open failed: Permission denied
Can't get lock for orphan PVs.
$ echo $?
5
Trying the same as root also fails:
# truncate -s 256M test.img
# lvm pvcreate `pwd`/test.img
Device /root/test.img not found.
# echo $?
5
LVM seems strongly predicated on only using block devices [1]. LVM can
use loop devices though, but loop devices can only be created by root.
# truncate -s 256M test.img
# losetup -f --show `pwd`/test.img
/dev/loop0
# lvm pvcreate /dev/loop0
Physical volume "/dev/loop0" successfully created.
# echo $?
0
Make the LVM2 PV tests require user root and use loop device over the
test image. Tests for the other file system types still directly uses
the image file. This makes the LVM2 PV tests pass when run as root, or
successfully skipped when run as non-root.
[1] lvmconfig --typeconfig default --withcomments --withspace | less
From the "devices" section of the commented default configuration,
LVM uses block devices found below /dev, devices provided by udev
and/or found in sysfs.
Closes!49 - Add file system interface tests
With the previous commit, execution of test_SupportedFileSystems is
failing in the GitLab CI Ubuntu image. Fragment from file
tests/test-suite.log:
FAIL: test_SupportedFileSystems
===============================
Terminate called after throwing an instance of 'Glib::ConvertError'
Aborted (core dumped)
FAIL test_SupportedFileSystems (exit status: 134)
This core dump can be re-created locally by (1) removing modprobe from
the PATH, and (2) executing the test program in the C locale.
$ LC_ALL=C ./test_SupportedFileSystems
Running main() from test_SupportedFileSystems.cc
terminate called after throwing an instance of 'Glib::ConvertError'
Aborted
$ echo $?
134
Backtrace from gdb:
(gdb) backtrace
#0 0x00007f4f93002337 in __GI_raise (sig=sig@entry=6)
at ../nptl/sysdeps/unix/sysv/linux/raise.c:55
#1 0x00007f4f93003a28 in __GI_abort () at abort.c:90
#2 0x00007f4f93b2e7d5 in __gnu_cxx::__verbose_terminate_handler() ()
at ../../../../libstdc++-v3/libsupc++/vterminate.cc:95
#3 0x00007f4f93b2c746 in __cxxabiv1::__terminate(void (*)()) (handler=<optimized out>)
at ../../../../libstdc++-v3/libsupc++/eh_terminate.cc:38
#4 0x00007f4f93b2c773 in std::terminate() ()
at ../../../../libstdc++-v3/libsupc++/eh_terminate.cc:48
#5 0x00007f4f93b2c993 in __cxxabiv1::__cxa_throw(void*, std::type_info*, void (*)(void*))
(obj=0x260d4b0, tinfo=0x7f4f966c1930 <typeinfo for Glib::ConvertError>, dest=0x7f4f96486fa0 <Glib::ConvertError::~ConvertError()>)
at ../../../../libstdc++-v3/libsupc++/eh_throw.cc:87
#6 0x00007f4f96486e27 in Glib::ConvertError::throw_func(_GError*) (gobject=0x260bf90) at convert.cc:329
#7 0x00007f4f9649b5d7 in Glib::Error::throw_exception(_GError*) (gobject=0x260bf90) at error.cc:175
#8 0x00007f4f964a7155 in Glib::operator<<(std::ostream&, Glib::ustring const&)
(os=warning: RTTI symbol not found for class 'std::ostream' ..., utf8_string=...) at ustring.cc:1430
#9 0x000000000044d66f in GParted::Utils::execute_command(Glib::ustring const&, char const*, Glib::ustring&, Glib::ustring&, bool)
(command=..., input=input@entry=0x0, output=..., error=..., use_C_locale=use_C_locale@entry=true)
at ../src/Utils.cc:688
#10 0x000000000044dae9 in GParted::Utils::kernel_supports_fs(Glib::ustring const&)
(use_C_locale=true, error=..., output=..., command=...)
at ../src/Utils.cc:659
#11 0x000000000044dae9 in GParted::Utils::kernel_supports_fs(Glib::ustring const&) (fs=...)
at ../src/Utils.cc:480
#12 0x0000000000460008 in GParted::jfs::get_filesystem_support() (this=0x25e8e60)
at ../src/jfs.cc:59
#13 0x00000000004464f9 in GParted::SupportedFileSystems::find_supported_filesystems() (this=0x25e8690)
at ../src/SupportedFileSystems.cc:120
#14 0x0000000000412360 in GParted::SupportedFileSystemsTest::setup_supported_filesystems() ()
at test_SupportedFileSystems.cc:278
#15 0x00000000004151b0 in GParted::SupportedFileSystemsTest::get_supported_fstypes() ()
at test_SupportedFileSystems.cc:256
#16 0x00000000004152c0 in GParted::gtest_MySupportedFileSystemsTest_EvalGenerator_() ()
at test_SupportedFileSystems.cc:495
#17 0x000000000041c7d6 in testing::internal::ParameterizedTestCaseInfo<GParted::SupportedFileSystemsTest>::RegisterTests()
(this=0x2528ac0) at ../lib/gtest/include/gtest/internal/gtest-param-util.h:549
#18 0x0000000000479fb5 in testing::internal::UnitTestImpl::RegisterParameterizedTests() (this=0x25288d0)
at ./include/gtest/internal/gtest-param-util.h:709
#19 0x0000000000479fb5 in testing::internal::UnitTestImpl::RegisterParameterizedTests()
(this=this@entry=0x2528800) at ./src/gtest.cc:2658
#20 0x000000000048a001 in testing::internal::UnitTestImpl::PostFlagParsingInit() (this=0x2528800)
at ./src/gtest.cc:4980
#21 0x000000000049e399 in testing::internal::InitGoogleTestImpl<char>(int*, char**)
(argc=argc@entry=0x7ffe9d208a3c, argv=argv@entry=0x7ffe9d208b38) at ./src/gtest.cc:5934
#22 0x000000000048d285 in testing::InitGoogleTest(int*, char**)
(argc=argc@entry=0x7ffe9d208a3c, argv=argv@entry=0x7ffe9d208b38) at ./src/gtest.cc:5952
#23 0x0000000000410404 in main(int, char**) (argc=1, argv=0x7ffe9d208b38)
at test_SupportedFileSystems.cc:557
The test program runs when executed in my locale and produces these
messages:
$ ./test_SupportedFileSystems
Running main() from test_SupportedFileSystems.cc
Failed to execute child process “modprobe” (No such file or directory)
Failed to execute child process “modprobe” (No such file or directory)
[==========] Running 210 tests from 1 test case.
...
So the test program is aborting when trying to print the failed to
execute child process message, but only in the C locale.
This doesn't affect the CentOS GitLab CI image because that installs the
kmod package with modprobe by default, however the Ubuntu image doesn't
have the kmod package.
Fix this by explicitly installing the kmod package into both the CentOS
and Ubuntu GitLab CI images.
Closes!49 - Add file system interface tests
Extend testing to all fully supported file systems, those with an
implemented FileSystem derived class.
Note that in main() GParted threading needs to now be initialised before
InitGoogleTest() because it calls INSTANTIATE_TEST_CASE_P() which in
turn calls get_supported_fstypes() which eventually constructs all the
individual file system interface objects and discovers available
support, some of which use execute_command(). Example call chain:
InitGoogleTest()
INSTANTIATE_TEST_CASE_P()
get_supported_fstypes()
setup_supported_filesystems()
{SupportedFileSystems}->find_supported_filesystems()
{btrfs}->get_filesystem_support()
Utils::execute_command()
In the CentOS 7 GitLab CI image the EPEL (Extra Packages for Enterprise
Linux) repository is added to provide f2fs-tools and ntfsprogs.
23 of 210 tests fail on CentOS 7 and 22 on Ubuntu 18.04 LTS. The
following commits will resolve these test failures.
$ ./test_SupportedFileSystems
Running main() from test_SupportedFileSystems.cc
[==========] Running 210 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 210 tests from My/SupportedFileSystemsTest
...
[----------] 210 tests from My/SupportedFileSystemsTest (11066 ms total)
[----------] Global test environment tear-down
[==========] 210 tests from 1 test case ran. (11067 ms total)
[ PASSED ] 187 tests.
[ FAILED ] 23 tests, listed below:
[ FAILED ] My/SupportedFileSystemsTest.Create/lvm2pv, where GetParam() = 20
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadUsage/btrfs, where GetParam() = 7
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadUsage/jfs, where GetParam() = 17
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadUsage/lvm2pv, where GetParam() = 20
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadUsage/nilfs2, where GetParam() = 22
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadUsage/ntfs, where GetParam() = 23
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadLabel/btrfs, where GetParam() = 7
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadLabel/nilfs2, where GetParam() = 22
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadUUID/btrfs, where GetParam() = 7
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadUUID/fat16, where GetParam() = 13
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadUUID/fat32, where GetParam() = 14
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadUUID/jfs, where GetParam() = 17
[ FAILED ] My/SupportedFileSystemsTest.CreateAndReadUUID/nilfs2, where GetParam() = 22
[ FAILED ] My/SupportedFileSystemsTest.CreateAndWriteLabel/nilfs2, where GetParam() = 22
[ FAILED ] My/SupportedFileSystemsTest.CreateAndWriteUUID/nilfs2, where GetParam() = 22
[ FAILED ] My/SupportedFileSystemsTest.CreateAndCheck/lvm2pv, where GetParam() = 20
[ FAILED ] My/SupportedFileSystemsTest.CreateAndCheck/minix, where GetParam() = 21
[ FAILED ] My/SupportedFileSystemsTest.CreateAndRemove/lvm2pv, where GetParam() = 20
[ FAILED ] My/SupportedFileSystemsTest.CreateAndGrow/btrfs, where GetParam() = 7
[ FAILED ] My/SupportedFileSystemsTest.CreateAndGrow/lvm2pv, where GetParam() = 20
[ FAILED ] My/SupportedFileSystemsTest.CreateAndGrow/xfs, where GetParam() = 27
[ FAILED ] My/SupportedFileSystemsTest.CreateAndShrink/btrfs, where GetParam() = 7
[ FAILED ] My/SupportedFileSystemsTest.CreateAndShrink/lvm2pv, where GetParam() = 20
23 FAILED TESTS
Closes!49 - Add file system interface tests
Until now the parameterised test values are printed as part of the test
names as just 0, 1, etc. like this:
$ ./test_SupportedFileSystems
Running main() from test_SupportedFileSystems.cc
[==========] Running 20 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 20 tests from My/SupportedFileSystemsTest
[ RUN ] My/SupportedFileSystemsTest.Create/0
[ OK ] My/SupportedFileSystemsTest.Create/0 (48 ms)
[ RUN ] My/SupportedFileSystemsTest.Create/1
[ OK ] My/SupportedFileSystemsTest.Create/1 (11 ms)
Provide the file system types as the names for the parameterised test
values [1]. Now the test names are printed like this:
$ ./test_SupportedFileSystems
Running main() from test_SupportedFileSystems.cc
[==========] Running 20 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 20 tests from My/SupportedFileSystemsTest
[ RUN ] My/SupportedFileSystemsTest.Create/ext2
[ OK ] My/SupportedFileSystemsTest.Create/ext2 (51 ms)
[ RUN ] My/SupportedFileSystemsTest.Create/linuxswap
[ OK ] My/SupportedFileSystemsTest.Create/linuxswap (11 ms)
Also use these Google Test name friendly ASCII alphanumeric only names
everywhere the file system type needs to be reported in this test
program.
[1] Specifying Names for Value-Parameterized Test Parameters
https://github.com/google/googletest/blob/v1.8.x/googletest/docs/advanced.md#specifying-names-for-value-parameterized-test-parametersCloses!49 - Add file system interface tests
Use Google Test Value-Parameterised to call every test for both ext2
and linux-swap.
https://github.com/google/googletest/blob/v1.8.x/googletest/docs/advanced.md#value-parameterized-tests
Running the test now looks like this:
$ ./test_SupportedFileSystems
Running main() from test_SupportedFileSystems.cc
[==========] Running 20 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 20 tests from My/SupportedFileSystemsTest
[ RUN ] My/SupportedFileSystemsTest.Create/0
[ OK ] My/SupportedFileSystemsTest.Create/0 (97 ms)
[ RUN ] My/SupportedFileSystemsTest.Create/1
[ OK ] My/SupportedFileSystemsTest.Create/1 (15 ms)
[ RUN ] My/SupportedFileSystemsTest.CreateAndReadUsage/0
[ OK ] My/SupportedFileSystemsTest.CreateAndReadUsage/0 (106 ms)
[ RUN ] My/SupportedFileSystemsTest.CreateAndReadUsage/1
[ OK ] My/SupportedFileSystemsTest.CreateAndReadUsage/1 (14 ms)
[ RUN ] My/SupportedFileSystemsTest.CreateAndReadLabel/0
[ OK ] My/SupportedFileSystemsTest.CreateAndReadLabel/0 (95 ms)
[ RUN ] My/SupportedFileSystemsTest.CreateAndReadLabel/1
[ OK ] My/SupportedFileSystemsTest.CreateAndReadLabel/1 (23 ms)
[ RUN ] My/SupportedFileSystemsTest.CreateAndReadUUID/0
[ OK ] My/SupportedFileSystemsTest.CreateAndReadUUID/0 (99 ms)
[ RUN ] My/SupportedFileSystemsTest.CreateAndReadUUID/1
[ OK ] My/SupportedFileSystemsTest.CreateAndReadUUID/1 (22 ms)
[ RUN ] My/SupportedFileSystemsTest.CreateAndWriteLabel/0
[ OK ] My/SupportedFileSystemsTest.CreateAndWriteLabel/0 (102 ms)
[ RUN ] My/SupportedFileSystemsTest.CreateAndWriteLabel/1
[ OK ] My/SupportedFileSystemsTest.CreateAndWriteLabel/1 (22 ms)
[ RUN ] My/SupportedFileSystemsTest.CreateAndWriteUUID/0
[ OK ] My/SupportedFileSystemsTest.CreateAndWriteUUID/0 (101 ms)
[ RUN ] My/SupportedFileSystemsTest.CreateAndWriteUUID/1
[ OK ] My/SupportedFileSystemsTest.CreateAndWriteUUID/1 (21 ms)
[ RUN ] My/SupportedFileSystemsTest.CreateAndCheck/0
[ OK ] My/SupportedFileSystemsTest.CreateAndCheck/0 (153 ms)
[ RUN ] My/SupportedFileSystemsTest.CreateAndCheck/1
test_SupportedFileSystems.cc:424: Skip test. check not supported or support not found
[ OK ] My/SupportedFileSystemsTest.CreateAndCheck/1 (0 ms)
[ RUN ] My/SupportedFileSystemsTest.CreateAndRemove/0
test_SupportedFileSystems.cc:437: Skip test. remove not supported or support not found
[ OK ] My/SupportedFileSystemsTest.CreateAndRemove/0 (0 ms)
[ RUN ] My/SupportedFileSystemsTest.CreateAndRemove/1
test_SupportedFileSystems.cc:437: Skip test. remove not supported or support not found
[ OK ] My/SupportedFileSystemsTest.CreateAndRemove/1 (0 ms)
[ RUN ] My/SupportedFileSystemsTest.CreateAndGrow/0
[ OK ] My/SupportedFileSystemsTest.CreateAndGrow/0 (266 ms)
[ RUN ] My/SupportedFileSystemsTest.CreateAndGrow/1
[ OK ] My/SupportedFileSystemsTest.CreateAndGrow/1 (32 ms)
[ RUN ] My/SupportedFileSystemsTest.CreateAndShrink/0
[ OK ] My/SupportedFileSystemsTest.CreateAndShrink/0 (111 ms)
[ RUN ] My/SupportedFileSystemsTest.CreateAndShrink/1
[ OK ] My/SupportedFileSystemsTest.CreateAndShrink/1 (28 ms)
[----------] 20 tests from My/SupportedFileSystemsTest (1311 ms total)
[----------] Global test environment tear-down
[==========] 20 tests from 1 test case ran. (1342 ms total)
[ PASSED ] 20 tests.
Closes!49 - Add file system interface tests
Replace directly using ext2 derived FileSystem interface class with
using the SupportedFileSystems class. This is a step in getting ready
for testing all the GParted file system interface classes in one go.
Closes!49 - Add file system interface tests
GParted_Core::FILESYSTEMS and ::FILESYSTEM_MAP and the methods that
query and manipulate them are self-contained. Therefore move them into
a separate SupportedFileSystems module.
Also having a single class maintaining all FileSystem interface objects
will make testing all the file system types much easier as there will be
no need to duplicate this functionality in the test.
Closes!49 - Add file system interface tests
Here are the errors reported in the deliberately broken
CreateAndReadLabel test from the previous commit message:
[ RUN ] ext2Test.CreateAndReadLabel
test_ext2.cc:311: Failure
Value of: m_partition.get_messages().empty()
Actual: false
Expected: true
Partition messages:
e2label: No such file or directory while trying to open /does_not_exist/test_ext2.img
Couldn't find valid filesystem superblock.
[ FAILED ] ext2Test.CreateAndReadLabel (77 ms)
Even though the test was deliberately broken by setting the wrong path
for the file system image and the e2label command failed, apparently
testing for the expected label still passed. What happened was that the
desired "TEST_LABEL" has to be in the Partition object and then the file
system was created. Then reading the file system label failed, however
"TEST_LABEL" was already set in the Partition object so it matched.
Reading the label is unique among the read actions of usage, label and
UUID as the others don't need to be set before the file system is
created. GParted doesn't encounter this issue because when refreshing
devices it creates new blank Partition objects and then performs the
read actions to populate them.
Fix by resetting the Partition object back to only containing basic
information before all the reading file system information tests, even
though it is only needed in the read label case. This also better
reflects how GParted works.
Now with the same deliberate brokenness the test also reports the label
does not match it's expected value:
$ ./test_ext2 --gtest_filter='ext2Test.CreateAndReadLabel'
Running main() from test_ext2.cc
Note: Google Test filter = ext2Test.CreateAndReadLabel
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from ext2Test
[ RUN ] ext2Test.CreateAndReadLabel
test_ext2.cc:322: Failure
Expected equality of these values:
fs_label
Which is: "TEST_LABEL"
m_partition.get_filesystem_label().c_str()
Which is: ""
test_ext2.cc:272: Failure
Value of: m_partition.get_messages().empty()
Actual: false
Expected: true
Partition messages:
e2label: No such file or directory while trying to open /does_not_exist/test_ext2.img
Couldn't find valid filesystem superblock.
[ FAILED ] ext2Test.CreateAndReadLabel (70 ms)
[----------] 1 test from ext2Test (70 ms total)
[----------] Global test environment tear-down
[==========] 1 test from 1 test case ran. (75 ms total)
[ PASSED ] 0 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] ext2Test.CreateAndReadLabel
1 FAILED TEST
Closes!49 - Add file system interface tests
The file system reading methods report errors into Partition messages
because they are used as part of the GParted refresh, rather than
reporting errors into OperationDetails used when applying operations.
Therefore test for no messages for success and print the messages on
failure.
For example, temporarily breaking the read label test code by setting
the wrong file system image name produces this:
$ ./test_ext2 --gtest_filter='ext2Test.CreateAndReadLabel'
Running main() from test_ext2.cc
Note: Google Test filter = ext2Test.CreateAndReadLabel
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from ext2Test
[ RUN ] ext2Test.CreateAndReadLabel
test_ext2.cc:311: Failure
Value of: m_partition.get_messages().empty()
Actual: false
Expected: true
Partition messages:
e2label: No such file or directory while trying to open /does_not_exist/test_ext2.img
Couldn't find valid filesystem superblock.
[ FAILED ] ext2Test.CreateAndReadLabel (77 ms)
[----------] 1 test from ext2Test (77 ms total)
[----------] Global test environment tear-down
[==========] 1 test from 1 test case ran. (85 ms total)
[ PASSED ] 0 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] ext2Test.CreateAndReadLabel
1 FAILED TEST
$ echo $?
1
Closes!49 - Add file system interface tests
As seen in the first commit message, operation detail text is XML
encoded. This makes it harder to read, especially commands which often
have single quotes encoded as '. For example:
<b><i>mkfs.ext2 -F -L '' '/home/centos/programming/c/gparted/tests/test_ext2.img'</i></b>
Strip this encoding when printing the operation details. Now the same
example looks like:
mkfs.ext2 -F -L '' '/home/centos/programming/c/gparted/tests/test_ext2.img'
Closes!49 - Add file system interface tests
Running test_ext2 in GitLab Continuous Integration environment fails like
this:
(test_ext2:6338): Gtk-WARNING **: 09:06:17.576: cannot open display:
Running main() from test_ext2.cc
Obviously the GitLab CI environment doesn't have an X11 display, but
unfortunately this test case code requires one.
Utils::execute_command() calls Gtk::Main::run() so requires a Gtk::Main
object constructing and therefore an X11 display, even though this
program never displays anything graphical. The call chain is:
main() test_ext2.cc
Gtk::Main::Main() gtkmm/gtk/src/main.ccg
Gtk::Main::init() [1]
gtk_init() gtk/gtk/gtkmain.c [2]
which exits with a non-zero exit status when the DISPLAY environment
variable is unset.
Looked at deriving from Gtk::Main class and writing a replacement init()
method which calls gtk_init_check() instead of gtk_init() but
Gtk::Main::instance_ is a private member so not accessible in a derived
class.
Tried using Glib::MainLoop instead of Gtk::Main, but that doesn't
initialise everything that Gtk::Main(), so the program crashes.
Therefore use xvfb-run [3][4] to run this test program against a virtual
X11 display when a real display isn't available. Coded execution of
xvfb-run into this test program so that it can simply be executed on the
command line like the other test programs, without having to remember to
run "xvfb-run ./test_ext2 ...".
[1] Gtk::Main::init()
https://gitlab.gnome.org/GNOME/gtkmm/blob/3.10.1/gtk/src/main.ccg#L287
[2] gtk_init()
https://gitlab.gnome.org/GNOME/gtk/blob/3.10.9/gtk/gtkmain.c#L1000
[3] how to run gtk app on linux without an x server
https://superuser.com/questions/624918/how-to-run-gtk-app-on-linux-without-an-x-server
[4] Using GTK without DISPLAY
https://stackoverflow.com/questions/11694278/using-gtk-without-displayCloses!49 - Add file system interface tests
This is the first step of adding testing of the derived FileSystem
interface classes which call the file system specific executables.
Rather than mocking command execution and returned output the tests run
the real commands, effectively making this integration testing.
Test case setup determines the file system supported actions using
get_filesystem_support() and individual tests are skipped if a feature
is not supported, just as GParted does for it's actions.
Each test creates it's own sparse image file and a fresh file system,
performs a test on one FileSystem interface call and deletes the image
file. This makes each test independent and allows them to run as a
non-root user, provided the file system command itself doesn't require
root. Errors reported for a failed interface call will include the
GParted OperationDetails, which in turn includes the file system
specific command used and stdout and stderr from it's execution.
For example, temporarily breaking the test code to create a 10 KiB image
file instead of 256 MiB one produces this:
$ ./test_ext2
Running main() from test_ext2.cc
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from ext2Test
[ RUN ] ext2Test.Create
test_ext2.cc:199: Failure
Value of: s_ext2_obj->create(m_partition, m_operation_detail)
Actual: false
Expected: true
Operation details:
<b><i>mkfs.ext2 -F -L '' '/home/centos/programming/c/gparted/tests/test_ext2.img'</i></b> 00:00:00 (ERROR)
<i></i>
<i>mke2fs 1.42.9 (28-Dec-2013)
/home/centos/programming/c/gparted/tests/test_ext2.img: Not enough space to build proposed filesystem while setting up superblock
</i>
[ FAILED ] ext2Test.Create (25 ms)
[----------] 1 test from ext2Test (25 ms total)
[----------] Global test environment tear-down
[==========] 1 test from 1 test case ran. (30 ms total)
[ PASSED ] 0 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] ext2Test.Create
1 FAILED TEST
$ echo $?
1
Also as Utils:: and FileSystem::execute_command() are needed, this
requires linking with GParted_Core for GParted_Core::mainthread and
therefore with most of the non-UI classes in gpartedbin.
Closes!49 - Add file system interface tests
To match the renaming performed as part of Bug 741424 "Add support for
GPT partition names". In particular this is most closely similar to:
d480800600
Rename enum to OPERATION_LABEL_FILESYSTEM (#741424)
When GParted GitLab Continuous Integration was setup, CentOS 7 image was
used for a slow moving distribution and Ubuntu as a different, faster
moving distribution.
CentOS 8 has recently been released [1]. To avoid automatically
switching to it when an official CentOS 8 docker image is made available
[2], explicitly specify the CentOS 7 image.
[1] Release for CentOS Linux 8 and CentOS Streams (2019-09-24)
https://lists.centos.org/pipermail/centos-announce/2019-September/023449.html
[2] Docker Official Images, The official build of CentOS
https://hub.docker.com/_/centos/Closes!48 - Remain with CentOS 7 for GitLab CI
Was attempting to erase the wrong key "key-long" for the test. It
should be erasing "key-too-long". Fix this. Given that that line in
the test was to erase a non-existent key and confirm failure; the test
passed before with the wrong non-existent key, and still passes with the
right non-existent key. Clearly a cut and paste error as a result of
copying the previous LongPassword test when initially added in:
c6657aab9e
Add unit tests for PasswordRAMStore module (#795617)
Separate udevsettle command was merged into udevadm back in udev 117
(released 2007-11-13) by:
225cb03bd8
udevadm: merge all udev tools into a single binary
The oldest supported distributions have these much newer versions of
udev:
Distro EOL udevadm -V
Debian 8 2020-Apr 215 (combined systemd and udev)
RHEL / CentOS 7 2024-Jun 219 (combined systemd and udev)
Ubuntu 16.04 LTS 2021-Apr 229 (combined systemd and udev)
Along with the previous patch "Avoid reading the same sector multiple
times in a row (!46)(#16)" internal detection of file system signatures
now reads the minimum number of sectors in increasing order.
As the code still considers the sector size, it now also reads less
sectors from devices with larger sectors to get all the signatures. So
on a 512 byte sector device it maximally seeks to and reads from these
byte offsets:
0, 512, 1024, 65536
and on a 4096 byte sector device, only from these byte offsets:
0, 65536
Closes!46 - Whole device FAT32 file system reports device busy warning
from mlabel
Closes#16 - "invalid argument for seek()" error on very small (<=40KiB)
drives
GParted internal file system detection is reading the same sector
multiple times in a row. Add an optimisation to only read a sector if
it is different to the previously read sector.
Closes!46 - Whole device FAT32 file system reports device busy warning
from mlabel
Closes#16 - "invalid argument for seek()" error on very small (<=40KiB)
drives
After the previous commit the lp_device structure pointer parameter is
only used to provide sector_size. Just pass that instead.
Closes!46 - Whole device FAT32 file system reports device busy warning
from mlabel
Closes#16 - "invalid argument for seek()" error on very small (<=40KiB)
drives
GParted_Core::useable_device() doesn't change the pointed to PedDevice
structure. Pass it as const so the compiler enforces this.
Closes!46 - Whole device FAT32 file system reports device busy warning
from mlabel
For every device that GParted scans, it determines if it can be used by
attempting to read the first sector. This is the sequence of events,
as reported in the previous two commit messages:
gparted |set_devices_thread(pdevices) pdevices->["/dev/sdb"]
...
gparted | useable_device(lp_device) lp_device->path="/dev/sdb"
gparted | ped_device_open()
libparted | open("/dev/sdb", O_RDWR) = 8
gparted | ped_device_read()
gparted | ped_device_close()
libparted | close(8)
udev(async)| KERNEL change /devices/.../sdb (block)
...
udev(async)| UDEV change /devices/.../sdb (block)
Note that these udev block device change events occur before the ones
waited for in the first commit "Wait for udev change on /dev/DISK when
querying whole device FS (!46)" so these were already waited for.
So because libparted only opens devices read-write this triggers a set
of udev change events for the whole block device, and if the device is
partitioned, also remove and re-add events for all the partitions.
Switch to using POSIX open(), read() and close() calls so read-only
access can be specified to avoid the unnecessary udev change events.
Closes!46 - Whole device FAT32 file system reports device busy warning
from mlabel
After the previous fix, on my CentOS 7 VM, GParted is now often
reporting a warning that the /dev/PTN entry is missing when reading the
label of the file system in the first partition. This happens
regardless of the file system type.
Example error for EXT4:
e2label: No such file or directory while trying to open /dev/sdb1
Couldn't find valid file system superblock.
Example error for XFS:
/dev/sdb1: No such file or directory
fatal error -- couldn't initialize XFS library
As with the case in the previous commit, GParted gets the label via
blkid instead.
Again with debugging and sleeping in GParted, stracing the GParted
thread GParted_Core::set_devices_thread() and using 'udevadm monitor' to
watch udev events the following sequence of events is observed:
gparted |set_devices_thread(pdevices) pdevices->["/dev/sdb"]
gparted | ped_device_get("/dev/sdb")
libparted | open("/dev/sdb", O_RDONLY) = 8
libparted | close(8)
gparted | useable_device(lp_device) lp_device->path="/dev/sdb"
gparted | ped_device_open()
libparted | open("/dev/sdb", O_RDWR) = 8
gparted | ped_device_read()
gparted | ped_device_close()
libparted | close(8)
udev(async)| KERNEL remove /devices/.../sdb/sdb1 (block)
udev(async)| KERNEL remove /devices/.../sdb/sdb2 (block)
udev(async)| KERNEL change /devices/.../sdb (block)
udev(async)| KERNEL add /devices/.../sdb/sdb1 (block)
udev(async)| KERNEL add /devices/.../sdb/sdb2 (block)
udev(async)| UDEV remove /devices/.../sdb/sdb1 (block)
udev(async)| UDEV remove /devices/.../sdb/sdb2 (block)
udev(async)| UDEV change /devices/.../sdb (block)
udev(async)| UDEV add /devices/.../sdb/sdb1 (block)
udev(async)| UDEV add /devices/.../sdb/sdb2 (block)
gparted | set_device_from_disk(device, device_path="/dev/sdb")
gparted | get_device(device_path="/dev/sdb", ..., flush=true)
gparted | ped_device_get()
gparted | flush_device(lp_device) lp_device->path="/dev/sdb"
gparted | ped_device_open()
libparted | open("/dev/sdb", O_RDWR) = 8
gparted | ped_device_sync()
gparted | ped_device_close()
libparted | close(8)
udev(async)| KERNEL remove /devices/.../sdb/sdb1 (block)
udev(async)| KERNEL remove /devices/.../sdb/sdb2 (block)
udev(async)| KERNEL change /devices/.../sdb (block)
udev(async)| KERNEL add /devices/.../sdb/sdb1 (block)
udev(async)| KERNEL add /devices/.../sdb/sdb2 (block)
udev(async)| UDEV remove /devices/.../sdb/sdb1 (block)
udev(async)| UDEV remove /devices/.../sdb/sdb2 (block)
udev(async)| UDEV change /devices/.../sdb (block)
udev(async)| UDEV add /devices/.../sdb/sdb1 (block)
udev(async)| UDEV add /devices/.../sdb/sdb2 (block)
gparted | settle_device()
gparted | Utils::execute_command("udevadm settle --timeout=1")
gparted | detect_filesystem(lp_device, NULL, ...) lp_device->path="/dev/sdb"
gparted | detect_filesystem_internal(lp_device, NULL) lp_device->path="/dev/sdb"
gparted | ped_device_open()
libparted | open("/dev/sdb", O_RDWR) = 8
gparted | ped_device_read()
gparted | ped_device_read()
gparted | ped_device_read()
gparted | ped_device_read()
gparted | ped_device_read()
gparted | ped_device_read()
gparted | ped_device_read()
gparted | ped_device_read()
gparted | ped_device_read()
gparted | ped_device_read()
gparted | ped_device_read()
gparted | ped_device_read()
gparted | ped_device_read()
gparted | ped_device_close()
libparted | close(8)
udev(async)| KERNEL remove /devices/.../sdb/sdb1 (block)
udev(async)| KERNEL remove /devices/.../sdb/sdb2 (block)
udev(async)| KERNEL change /devices/.../sdb (block)
udev(async)| KERNEL add /devices/.../sdb/sdb1 (block)
udev(async)| KERNEL add /devices/.../sdb/sdb2 (block)
udev(async)| UDEV remove /devices/.../sdb/sdb1 (block)
udev(async)| UDEV remove /devices/.../sdb/sdb2 (block)
udev(async)| UDEV change /devices/.../sdb (block)
udev(async)| UDEV add /devices/.../sdb/sdb1 (block)
udev(async)| UDEV add /devices/.../sdb/sdb2 (block)
gparted | get_disk(lp_device, lp_disk, strict=false)
gparted | ped_disk_new(lp_device) lp_device->path="/dev/sdb"
libparted | open("/dev/sdb", O_RDWR) = 8
libparted | close(8)
udev(async)| KERNEL remove /devices/.../sdb/sdb1 (block)
udev(async)| KERNEL remove /devices/.../sdb/sdb2 (block)
udev(async)| KERNEL change /devices/.../sdb (block)
udev(async)| KERNEL add /devices/.../sdb/sdb1 (block)
udev(async)| KERNEL add /devices/.../sdb/sdb2 (block)
udev(async)| UDEV remove /devices/.../sdb/sdb1 (block)
udev(async)| UDEV remove /devices/.../sdb/sdb2 (block)
udev(async)| UDEV change /devices/.../sdb (block)
udev(async)| UDEV add /devices/.../sdb/sdb1 (block)
udev(async)| UDEV add /devices/.../sdb/sdb2 (block)
gparted | set_device_partitions()
gparted | detect_filesystem()
gparted | set_partition_label_and_uuid(partition) partition.get_path()="/dev/sdb1"
gparted | read_label()
gparted | ext2::read_label()
gparted | Utils::execute_command("e2label /dev/sdb1")
So in this case for a partitioned drive, libparted ped_disk_new() is
opening and closing the device read-write and triggering the udev remove
and add partition block special entries exactly when the label is trying
to be read from the first partition, causing the device missing error.
The call chain is:
set_devices_thread()
set_device_from_disk()
get_disk()
ped_disk_new()
Looking at the source for ped_disk_new() [1] it is calling
ped_device_open() and ped_device_close(), hence why it is opening the
device read-write and triggering the udev events.
Looking back at commit "Wait for udev to recreate /dev/PTN entries when
calibrating (#762941)" [2] and re-testing it, the udev events were also
caused by ped_disk_new() with this call chain:
apply_operation_to_disk()
calibrate_partition()
get_disk()
ped_disk_new()
Fix this probe case and keep the fix for the previous calibrate case by
moving the waiting for udev events from calibrate_partition() into
get_disk(), right after ped_disk_new(). The maximum wait time is
shortened to the shorter 1 second probing timeout to avoid the longer
10 second apply timeout in this probing case. The calibrate case commit
message said "The longest I have seen udev take to do this is 0.80
seconds in a VM".
[1] parted/libparted/disk.c:ped_disk_new()
http://git.savannah.gnu.org/cgit/parted.git/tree/libparted/disk.c?id=v3.2#n169
[2] fd9013d5f6
Wait for udev to recreate /dev/PTN entries when calibrating
(#762941)
Closes!46 - Whole device FAT32 file system reports device busy warning
from mlabel
GParted nearly always shows this warning against a whole disk device
FAT32 file system on my development VMs:
plain floppy: device "/dev/sdb" busy (Resource temporarily
unavailable): Cannot initialize '::'
mlabel: Cannot initialize drive
However GParted does get the label via blkid instead. Occasionally
everything works correctly and there is no warning.
Never found a similar error for any other file system on a whole disk
device. The timing never seems to clash.
Remember that when a writable file descriptor of a disk device is
closed, udev events are triggered which update the kernel and /dev
entries for any partition changes. (This is in addition to coding which
has always existed in the partitioning tools, including fdisk and
parted, to inform the kernel, but not create /dev entries, of partition
changes).
With debugging and sleeping in GParted, stracing the GParted thread
GParted_Core::set_devices_thread() and using 'udevadm monitor' to watch
udev events the following sequence of events is observed:
gparted |set_devices_thread(pdevices) pdevices->["/dev/sdb"]
gparted | ped_device_get("/dev/sdb")
libparted | open("/dev/sdb", O_RDONLY) = 8
libparted | close(8)
gparted | useable_device(lp_device) lp_device->path="/dev/sdb"
gparted | ped_device_open()
libparted | open("/dev/sdb", O_RDWR) = 8
gparted | ped_device_read()
gparted | ped_device_close()
libparted | close(8)
udev(async)| KERNEL change /devices/.../sdb (block)
udev(async)| KERNEL change /devices/.../sdb (block)
udev(async)| UDEV change /devices/.../sdb (block)
udev(async)| UDEV change /devices/.../sdb (block)
gparted | set_device_from_disk(device, device_path="/dev/sdb")
gparted | get_device(device_path="/dev/sdb", ..., flush=true)
gparted | ped_device_get()
gparted | flush_device(lp_device) lp_device->path="/dev/sdb"
gparted | ped_device_open()
libparted | open("/dev/sdb", O_RDWR) = 8
gparted | ped_device_sync()
gparted | ped_device_close()
libparted | close(8)
udev(async)| KERNEL change /devices/.../sdb (block)
udev(async)| KERNEL change /devices/.../sdb (block)
udev(async)| UDEV change /devices/.../sdb (block)
udev(async)| UDEV change /devices/.../sdb (block)
gparted | set_device_one_partition()
gparted | set_partition_label_and_uuid()
gparted | read_label()
gparted | fat16::read_label()
gparted | Utils::execute_command("mlabel -s :: -i /dev/sdb")
This sequence of events only shows which close()s by libparted trigger
udev events. It does not show that processing those events are
asynchronous and overlap with GParted executing mlabel, causing the
device busy error.
As libparted's ped_device_open() doesn't offer a way to specify that a
device will only be used for reading, it always opens it read-write
[1][2]. Hence this sequence in GParted_Core::flush_device():
ped_device_open()
ped_device_sync()
ped_device_close()
always triggers udev change events on the whole disk device. Fix by
waiting for those udev events to complete.
[1] libparted documentation, PedDevice, ped_device_open()
https://www.gnu.org/software/parted/api/group__PedDevice.html#ga7c87bd06e76553c8ff3262b5e6ca256
[2] parted/libparted/arch/linux.c:linux_open()
http://git.savannah.gnu.org/cgit/parted.git/tree/libparted/arch/linux.c?id=v3.2#n1628Closes!46 - Whole device FAT32 file system reports device busy warning
from mlabel
The last distribution to include the separate 'udevinfo' command was
RHEL / CentOS 5 with udev 095. All supported distributions have much
newer versions of udev and instead have the 'udevadm' command.
A user reported that GParted was slow to refresh on FAT32 file systems:
"can take very long, up to several minutes; can be reproduced by running
dosfsck manually". This is because the file system was large and almost
full, and GParted performs a file system check just to to report the
file system usage.
Created a 4 GiB FAT32 file system and almost filled it with 4 KiB files,
just over 970,000 files.
# df -k /mnt/2
Filesystem 1K-blocks Used Available Used% Mounted on
/dev/sdb2 4186108 39155384 270724 94% /mnt/2
# df -i /mnt/2
Filesystem Inodes IUsed IFree IUse% Mounted on
/dev/sdb2 0 0 0 - /mnt/2
# find /mnt/2 -type f -print | wc -l
971059
# find /mnt/2 -type d -print | wc -l
1949
Testing performance of the current fsck.fat:
# time fsck.fat -n -v /dev/sdb2 | \
> egrep 'bytes per logical sector|bytes per cluster|sectors total|clusters$'
512 bytes per logical sector
4096 bytes per cluster
8388608 sectors total
/dev/sdb2: 973008 files, 978846/1046527 clusters
real 0m11.552s
user 0m2.183s
sys 0m7.547s
Free sectors in the file system according to fsck.fat:
(1046527 - 978846) * 4096 / 512 = 541448 sectors
Repeating this test while also using 'blktrace /dev/sdb2' and Ctrl-C
around the test in a separate terminal, reports these numbers of I/Os
being performed:
Read requests Read bytes
15,563 165 MiB
Prior to this commit [1] from 0.0.9, GParted used libparted's
ped_file_system_get_resize_constraint() to report the minimum size to
which a FAT* file system can be resized. Use this test program [2] to
performance test this method:
# time ./fscons /dev/sdb2
dev=/dev/sdb2
sector_size=512
min_size=7909522
max_size=8388608
real 0m2.673s
user 0m0.070s
sys 0m1.834s
Free sectors in the file system according to libparted
ped_file_system_get_resize_constraint():
8388608 - 7909522 = 479086 sectors
blktrace reports these numbers of I/Os being performed:
Read requests Read bytes
7,821 71 MiB
So using libparted resize constraint is a bit faster but is still
reading too much data and is really too slow. Also when testing GParted
using this libparted method against a corrupted FAT32 file system, on
every refresh, one popup dialog is displayed for each error libparted
detects with the file system, each of which needs acknowledgement.
Example popup:
Libparted Error
\DIRNAME\FILENAME.EXT is 107724k, but is has 1920
clusters (122880k).
[ Cancel ][ Ignore ]
There could be a huge number of such errors in a corrupted file system.
Not really suitable for use by GParted.
Test the performance of mtools' minfo command to report the file system
figures:
# time minfo -i /dev/sdb2 :: | \
> egrep 'sector size:|cluster size:|small size:|big size:|free clusters='
sector size: 512 bytes
cluster size: 8 sectors
small size: 0 sectors
big size: 8388608 sectors
free clusters=67681
real 0m0.013s
user 0m0.004s
sys 0m0.019s
Free sectors in the file system according to minfo:
67681 * 8 = 541448 sectors
blktrace reports these numbers of I/Os being performed by minfo:
Read requests Read bytes
1 16 KiB
This matches with minfo just reading information from the BPB (BIOS
Parameter Block) [3] from sector 0 and the FS Information Sector [4]
usually in sector 1. Note that the free cluster figure reported by
minfo comes from the FS Information Sector because it only reports it
for FAT32 file systems, not for FAT16 file systems. Scanning the File
Allocation Table (FAT) [5] to count free clusters is exactly what mdir,
without the '-f' (fast) flag, does. Test the performance of mdir:
# export MTOOLS_SKIP_CHECK=1
# time mdir -i /dev/sdb2 ::/ | fgrep 'bytes free'
277 221 376 bytes free
real 0m0.023s
user 0m0.011s
sys 0m0.023s
Free sectors in the file system according to mdir:
277221376 / 512 = 541448 sectors
blktrace reports these number of I/Os being performed by mdir:
Read requests Read bytes
5 448 KiB
So minfo and mdir together provide the needed information and are 2 to 3
orders of magnitude faster because they only read the needed BPB and FAT
data from the drive. Use these together to read the file system usage.
[1] 61cd0ce778
lots of stuff and cleanups, including fixing getting used/unused
space of hfs/hfs+/fat16/fat32
[2] fscons.c
/* FILE: fscons.c
* SYNOPSIS: Report libparted's FS resize limits.
* BUILD: gcc -o fscons fscons.c -lparted -lparted-fs-resize
*/
int main(int argc, const char *argv[])
{
PedDevice* dev = NULL;
PedDisk* tab = NULL;
PedPartition* ptn = NULL;
PedFileSystem* fs = NULL;
PedConstraint* cons = NULL;
if (argc != 2)
{
fprintf(stderr, "Usage: fscons BLOCKDEV\n");
exit(1);
}
dev = ped_device_get(argv[1]);
if (dev == NULL)
{
fprintf(stderr, "ped_device_get(\"%s\") failed\n", argv[1]);
exit(1);
}
printf("dev=%s\n", dev->path);
printf("sector_size=%ld\n", dev->sector_size);
tab = ped_disk_new(dev);
if (tab == NULL)
{
fprintf(stderr, "ped_disk_new(dev) failed\n");
exit(1);
}
ptn = ped_disk_get_partition_by_sector(tab, 0);
if (ptn == NULL)
{
fprintf(stderr, "ped_disk_get_partition(tab, 0) failed\n");
exit(1);
}
fs = ped_file_system_open(&ptn->geom);
if (fs == NULL)
{
fprintf(stderr, "ped_file_system_open(&ptn->geom) failed\n");
exit(1);
}
cons = ped_file_system_get_resize_constraint(fs);
if (cons == NULL)
{
fprintf(stderr, "ped_file_system_get_resize_constraint(fs) failed\n");
exit(1);
}
printf("min_size=%ld\n", cons->min_size);
printf("max_size=%ld\n", cons->max_size);
ped_constraint_destroy(cons);
ped_file_system_close(fs);
ped_disk_destroy(tab);
ped_device_destroy(dev);
return 0;
}
[3] Design of the FAT file system, BIOS Parameter Block
https://en.wikipedia.org/wiki/Design_of_the_FAT_file_system#BIOS_Parameter_Block
[4] Design of the FAT file system, FS Information Sector
https://en.wikipedia.org/wiki/Design_of_the_FAT_file_system#FS_Information_Sector
[5] Design of the FAT file system, File Allocation Table
https://en.wikipedia.org/wiki/Design_of_the_FAT_file_system#File_Allocation_Table
Bug 569921 - dosfsck -n delays device scan
When GParted is configured with `--disable-doc` the help documentation
is neither built, nor installed.
With this configuration the Help -> Contents menu displays a message
dialog indicating the "Documentation is not available".
On GNOME 3 no title is shown; however, on some other graphic toolkits
/ window managers an unspecified title is shown.
For example on GParted Live with Fluxbox the following is displayed:
Unnamed
-------
Documentation is not available
This build of gparted is configured without documentation.
Documentation is available at the project web site.
https://gparted.org
OK
To address the unspecified title on non-GNOME 3 graphic toolkits, add
a title that works with and without GNOME 3.
Closes!45 - Add missing window title to Help Contents dialog
Mike Fleetwood