remove double reads delete versions (#13544)

deleting collection of versions belonging
to same object, we can avoid re-reading
the xl.meta from the disk instead purge
all the requested versions in-memory,

the tradeoff is to allocate a map to de-dup
the versions, allow disks to be read only
once per object.

additionally reduce the data transfer between
nodes by shortening msgp data values.
This commit is contained in:
Harshavardhana
2021-11-01 10:50:07 -07:00
committed by GitHub
parent 15dcacc1fc
commit bb639d9f29
12 changed files with 310 additions and 259 deletions

4
go.mod
View File

@@ -63,7 +63,7 @@ require (
github.com/nats-io/stan.go v0.8.3
github.com/ncw/directio v1.0.5
github.com/nsqio/go-nsq v1.0.8
github.com/philhofer/fwd v1.1.1
github.com/philhofer/fwd v1.1.2-0.20210722190033-5c56ac6d0bb9
github.com/pierrec/lz4 v2.6.0+incompatible
github.com/pkg/errors v0.9.1
github.com/prometheus/client_golang v1.11.0
@@ -74,7 +74,7 @@ require (
github.com/secure-io/sio-go v0.3.1
github.com/shirou/gopsutil/v3 v3.21.9
github.com/streadway/amqp v1.0.0
github.com/tinylib/msgp v1.1.6
github.com/tinylib/msgp v1.1.7-0.20211026165309-e818a1881b0e
github.com/valyala/bytebufferpool v1.0.0
github.com/xdg/scram v0.0.0-20180814205039-7eeb5667e42c
github.com/yargevad/filepathx v1.0.0