mirror of
https://github.com/minio/minio.git
synced 2025-11-20 18:06:10 -05:00
add data update tracking using bloom filter (#9208)
By monitoring PUT/DELETE and heal operations it is possible to track changed paths and keep a bloom filter for this data. This can help prioritize paths to scan. The bloom filter can identify paths that have not changed, and the few collisions will only result in a marginal extra workload. This can be implemented on either a bucket+(1 prefix level) with reasonable performance. The bloom filter is set to have a false positive rate at 1% at 1M entries. A bloom table of this size is about ~2500 bytes when serialized. To not force a full scan of all paths that have changed cycle bloom filters would need to be kept, so we guarantee that dirty paths have been scanned within cycle runs. Until cycle bloom filters have been collected all paths are considered dirty.
This commit is contained in:
@@ -491,7 +491,6 @@ func serverMain(ctx *cli.Context) {
|
||||
// Initialize object layer with the supplied disks, objectLayer is nil upon any error.
|
||||
func newObjectLayer(ctx context.Context, endpointZones EndpointZones) (newObject ObjectLayer, err error) {
|
||||
// For FS only, directly use the disk.
|
||||
|
||||
if endpointZones.NEndpoints() == 1 {
|
||||
// Initialize new FS object layer.
|
||||
return NewFSObjectLayer(endpointZones[0].Endpoints[0].Path)
|
||||
|
||||
Reference in New Issue
Block a user