MyFavMirrors/minio
1
0
Fork 0
mirror of https://github.com/minio/minio.git synced 2025-11-07 04:42:56 -05:00
Code Issues Packages Projects Releases Wiki Activity

Reduce WriteAll allocs (#10810)

Browse Source 
WriteAll saw 127GB allocs in a 5 minute timeframe for 4MiB buffers 
used by `io.CopyBuffer` even if they are pooled.

Since all writers appear to write byte buffers, just send those 
instead and write directly. The files are opened through the `os` 
package so they have no special properties anyway.

This removes the alloc and copy for each operation.

REST sends content length so a precise alloc can be made.
This commit is contained in:
Klaus Post
2020-11-02 16:14:31 -08:00
committed by GitHub
parent 8527f22df1
commit 86e0d272f3
9 changed files with 33 additions and 26 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
cmd/storage-rest-client.go
View File

@@ -315,11 +315,11 @@ func (client *storageRESTClient) DeleteVersion(ctx context.Context, volume, path
}
// WriteAll - write all data to a file.
func (client *storageRESTClient) WriteAll(ctx context.Context, volume string, path string, reader io.Reader) error {
func (client *storageRESTClient) WriteAll(ctx context.Context, volume string, path string, b []byte) error {
values := make(url.Values)
values.Set(storageRESTVolume, volume)
values.Set(storageRESTFilePath, path)
respBody, err := client.call(ctx, storageRESTMethodWriteAll, values, reader, -1)
respBody, err := client.call(ctx, storageRESTMethodWriteAll, values, bytes.NewBuffer(b), int64(len(b)))
defer http.DrainBody(respBody)
return err
}