fetch bucket replication stats across peers in single call (#14956)

current implementation relied on recursively calling one bucket
at a time across all peers, this would be very slow and chatty
when there are 100's of buckets which would mean 100*peerCount
amount of network operations.

This PR attempts to reduce this entire call into `peerCount`
amount of network calls only. This functionality addresses also a
concern where the Prometheus metrics would significantly slow
down when one of the peers is offline.
This commit is contained in:
Harshavardhana
2022-05-23 09:15:30 -07:00
committed by GitHub
parent 90a52a29c5
commit f8650a3493
10 changed files with 411 additions and 6 deletions

View File

@@ -1604,8 +1604,9 @@ func getBucketUsageMetrics() *MetricsGroup {
Value: float64(time.Since(dataUsageInfo.LastUpdate)),
})
bucketReplStats := getAllLatestReplicationStats(dataUsageInfo.BucketsUsage)
for bucket, usage := range dataUsageInfo.BucketsUsage {
stats := getLatestReplicationStats(bucket, usage)
stats := bucketReplStats[bucket]
quota, _ := globalBucketQuotaSys.Get(ctx, bucket)