Avoid synchronizing usage writes (#11560)

If the periodic `case <-t.C:` save gets held up for a long time it will end up 
synchronize all disk writes for saving the caches.

We add jitter to per set writes so they don't sync up and don't hold a 
lock for the write, since it isn't needed anyway.

If an outage prevents writes for a long while we also add individual 
waits for each disk in case there was a queue.

Furthermore limit the number of buffers kept to 2GiB, since this could get 
huge in large clusters. This will not act as a hard limit but should be enough 
for normal operation.
This commit is contained in:
Klaus Post
2021-02-18 00:38:37 -08:00
committed by GitHub
parent 8e8a792d9d
commit 8a6b13c239
3 changed files with 14 additions and 3 deletions

View File

@@ -522,7 +522,7 @@ func (d *dataUsageCache) save(ctx context.Context, store objectIO, name string)
dataUsageBucket,
name,
NewPutObjReader(r),
ObjectOptions{})
ObjectOptions{NoLock: true})
if isErrBucketNotFound(err) {
return nil
}