Increase PipeCapture maximum read size to 64K (#777973)

For large output PipeCapture spends all it's time copying the capturebuf
to callerbuf to provide a consistent view for any registered update
callbacks.  This overhead is dependant of the size of the ever growing
captured output and the number of times OnReadable() is called.
Therefore increase the maximum read size to exponentially reduce this
overhead.

Time taken to read varying amounts of fsck.fat output with various
read buffer sizes:

                 1 MiB   10 MiB            122 MiB
    512b  :   0.60 sec   65 sec [1:05]   17262 sec [4:47:42]
    4096b :   0.19 sec   13 sec           2157 sec [  35:57]
    64K   :   0.07 sec    1.4 sec          210 sec [   3:30]

Note that this is only increasing the maximum size that can be read from
the output of the external command.  If the command produces it's output
slowly, such as the with progress reporting commands like mkfs.ext4,
then only the available number of bytes is read reporting the next
progress increment.  However if the command produces it's output
quickly, such as when testing this bug using a modified fsck.fat
concatenating the 122 MiB of pre-recorded output, then full buffer reads
are performed.

To ensure that a single call to OnReadable() couldn't block the UI too
long, the time taken for OnReadable() to process a full buffer of
various sizes was recorded as:

    512b  :   0.031 milliseconds
    4096b :   0.188 milliseconds
    64K   :   3.576 milliseconds

Adding this amount of processing time in the UI under normal
circumstances is not a problem.

As the captured output increases, the time taken by OnReadable() becomes
dominated by the time taken to copy the ever increasing capture buffer
to handle it's expansion and to copy it to the caller buffer for the
update callback.  At the end of the 122M captured fsck.fat output
OnReadable() takes 350 milliseconds per call.  This is not a problem
because this is an extreme case in which GParted is already hung and
increasing the buffer size is reducing the overall hang time from over
4 hours to a few minutes.

Bug 777973 - Segmentation fault on bad disk
This commit is contained in:
Mike Fleetwood 2017-03-28 07:27:29 +01:00 committed by Curtis Gedak
parent 8dbbb47ce2
commit 6f49f3049d
1 changed files with 2 additions and 1 deletions

View File

@ -15,6 +15,7 @@
*/
#include "PipeCapture.h"
#include "Utils.h"
#include <iostream>
#include <string>
@ -27,7 +28,7 @@
namespace GParted {
const size_t READBUF_SIZE = 512;
const size_t READBUF_SIZE = 64*KIBIBYTE;
PipeCapture::PipeCapture( int fd, Glib::ustring &buffer ) : fill_offset( 0 ),
cursor( 0 ),