Harshavardhana c606c76323 fix: prioritized latest buckets for crawler to finish the scans faster (#11115) ...

crawler should only ListBuckets once not for each serverPool,
buckets are same across all pools, across sets and ListBuckets
always returns an unified view, once list buckets returns
sort it by create time to scan the latest buckets earlier
with the assumption that latest buckets would have lesser
content than older buckets allowing them to be scanned faster
and also to be able to provide more closer to latest view.

2020-12-15 17:34:54 -08:00

26 KiB

Raw Blame History

View Raw