From 6f49f3049dc8fb72ef771074db73b60a79925962 Mon Sep 17 00:00:00 2001 From: Mike Fleetwood Date: Tue, 28 Mar 2017 07:27:29 +0100 Subject: [PATCH] Increase PipeCapture maximum read size to 64K (#777973) For large output PipeCapture spends all it's time copying the capturebuf to callerbuf to provide a consistent view for any registered update callbacks. This overhead is dependant of the size of the ever growing captured output and the number of times OnReadable() is called. Therefore increase the maximum read size to exponentially reduce this overhead. Time taken to read varying amounts of fsck.fat output with various read buffer sizes: 1 MiB 10 MiB 122 MiB 512b : 0.60 sec 65 sec [1:05] 17262 sec [4:47:42] 4096b : 0.19 sec 13 sec 2157 sec [ 35:57] 64K : 0.07 sec 1.4 sec 210 sec [ 3:30] Note that this is only increasing the maximum size that can be read from the output of the external command. If the command produces it's output slowly, such as the with progress reporting commands like mkfs.ext4, then only the available number of bytes is read reporting the next progress increment. However if the command produces it's output quickly, such as when testing this bug using a modified fsck.fat concatenating the 122 MiB of pre-recorded output, then full buffer reads are performed. To ensure that a single call to OnReadable() couldn't block the UI too long, the time taken for OnReadable() to process a full buffer of various sizes was recorded as: 512b : 0.031 milliseconds 4096b : 0.188 milliseconds 64K : 3.576 milliseconds Adding this amount of processing time in the UI under normal circumstances is not a problem. As the captured output increases, the time taken by OnReadable() becomes dominated by the time taken to copy the ever increasing capture buffer to handle it's expansion and to copy it to the caller buffer for the update callback. At the end of the 122M captured fsck.fat output OnReadable() takes 350 milliseconds per call. This is not a problem because this is an extreme case in which GParted is already hung and increasing the buffer size is reducing the overall hang time from over 4 hours to a few minutes. Bug 777973 - Segmentation fault on bad disk --- src/PipeCapture.cc | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/PipeCapture.cc b/src/PipeCapture.cc index 3579b2fe..053c3b2b 100644 --- a/src/PipeCapture.cc +++ b/src/PipeCapture.cc @@ -15,6 +15,7 @@ */ #include "PipeCapture.h" +#include "Utils.h" #include #include @@ -27,7 +28,7 @@ namespace GParted { -const size_t READBUF_SIZE = 512; +const size_t READBUF_SIZE = 64*KIBIBYTE; PipeCapture::PipeCapture( int fd, Glib::ustring &buffer ) : fill_offset( 0 ), cursor( 0 ),