MyFavMirrors/minio
1
0
Fork 0
mirror of https://github.com/minio/minio.git synced 2024-12-24 22:25:54 -05:00
Code Issues Packages Projects Releases Wiki Activity

Remove read-ahead for small files (#8522)

Browse Source 
We should only read ahead if we are reading big files. We enable it for files >= 16MB.

Benchmark on 64KB objects.

Before:

```
Operation: GET
Errors: 0
Average: 59.976s, 87.13 MB/s, 1394.07 ops ended/s.
Fastest: 1s, 90.99 MB/s, 1455.00 ops ended/s.
50% Median: 1s, 87.53 MB/s, 1401.00 ops ended/s.
Slowest: 1s, 81.39 MB/s, 1301.00 ops ended/s.
```

After:

```
Operation: GET
Errors: 0
Average: 59.992s, 207.99 MB/s, 3327.85 ops ended/s.
Fastest: 1s, 219.20 MB/s, 3507.00 ops ended/s.
50% Median: 1s, 210.54 MB/s, 3368.00 ops ended/s.
Slowest: 1s, 179.14 MB/s, 2865.00 ops ended/s.
```

The 64KB buffer is actually a small disadvantage for this case, but I believe it will be better in general than no buffer.
This commit is contained in:
Klaus Post 2019-11-14 23:58:41 +03:00 committed by Harshavardhana
parent e3273bc5bf
commit 1dd38750f7
1 changed files with 14 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

15
cmd/posix.go
View File

@ -17,6 +17,7 @@
package cmd package cmd
import ( import (
"bufio"
"context" "context"
"errors" "errors"
"io" "io"
@ -49,6 +50,13 @@ const (
diskMinTotalSpace = diskMinFreeSpace // Min 900MiB total space. diskMinTotalSpace = diskMinFreeSpace // Min 900MiB total space.
maxAllowedIOError = 5 maxAllowedIOError = 5
readBlockSize = 4 * humanize.MiByte // Default read block size 4MiB. readBlockSize = 4 * humanize.MiByte // Default read block size 4MiB.
// On regular files bigger than this;
readAheadSize = 16 << 20
// Read this many buffers ahead.
readAheadBuffers = 4
// Size of each buffer.
readAheadBufSize = 1 << 20
) )
// isValidVolname verifies a volname name in accordance with object // isValidVolname verifies a volname name in accordance with object
@ -1112,8 +1120,13 @@ func (s *posix) ReadFileStream(volume, path string, offset, length int64) (io.Re
io.Reader io.Reader
io.Closer io.Closer
}{Reader: io.LimitReader(file, length), Closer: file} }{Reader: io.LimitReader(file, length), Closer: file}
if length >= readAheadSize {
return readahead.NewReadCloserSize(r, readAheadBuffers, readAheadBufSize)
}
return readahead.NewReadCloser(r), nil // Just add a small 64k buffer.
r.Reader = bufio.NewReaderSize(r.Reader, 64<<10)
return r, nil
} }
// CreateFile - creates the file. // CreateFile - creates the file.