2019-03-19 16:50:59 -04:00
|
|
|
module github.com/minio/minio
|
|
|
|
|
[feat]: change erasure coding default block size from 10MiB to 1MiB (#11721)
major performance improvements in range GETs to avoid large
read amplification when ranges are tiny and random
```
-------------------
Operation: GET
Operations: 142014 -> 339421
Duration: 4m50s -> 4m56s
* Average: +139.41% (+1177.3 MiB/s) throughput, +139.11% (+658.4) obj/s
* Fastest: +125.24% (+1207.4 MiB/s) throughput, +132.32% (+612.9) obj/s
* 50% Median: +139.06% (+1175.7 MiB/s) throughput, +133.46% (+660.9) obj/s
* Slowest: +203.40% (+1267.9 MiB/s) throughput, +198.59% (+753.5) obj/s
```
TTFB from 10MiB BlockSize
```
* First Access TTFB: Avg: 81ms, Median: 61ms, Best: 20ms, Worst: 2.056s
```
TTFB from 1MiB BlockSize
```
* First Access TTFB: Avg: 22ms, Median: 21ms, Best: 8ms, Worst: 91ms
```
Full object reads however do see a slight change which won't be
noticeable in real world, so not doing any comparisons
TTFB still had improvements with full object reads with 1MiB
```
* First Access TTFB: Avg: 68ms, Median: 35ms, Best: 11ms, Worst: 1.16s
```
v/s
TTFB with 10MiB
```
* First Access TTFB: Avg: 388ms, Median: 98ms, Best: 20ms, Worst: 4.156s
```
This change should affect all new uploads, previous uploads should
continue to work with business as usual. But dramatic improvements can
be seen with these changes.
2021-03-06 17:09:34 -05:00
|
|
|
go 1.16
|
2019-04-02 21:28:39 -04:00
|
|
|
|
2019-03-19 16:50:59 -04:00
|
|
|
require (
|
2021-05-10 21:15:11 -04:00
|
|
|
cloud.google.com/go/storage v1.10.0
|
2020-08-31 13:19:25 -04:00
|
|
|
github.com/Azure/azure-pipeline-go v0.2.2
|
|
|
|
github.com/Azure/azure-storage-blob-go v0.10.0
|
2020-12-12 23:31:28 -05:00
|
|
|
github.com/Shopify/sarama v1.27.2
|
2021-03-16 23:06:57 -04:00
|
|
|
github.com/VividCortex/ewma v1.1.1
|
2019-04-02 21:28:39 -04:00
|
|
|
github.com/alecthomas/participle v0.2.1
|
2020-11-17 12:13:28 -05:00
|
|
|
github.com/bcicen/jstream v1.0.1
|
2020-10-14 02:59:20 -04:00
|
|
|
github.com/beevik/ntp v0.3.0
|
2021-05-27 11:02:39 -04:00
|
|
|
github.com/bits-and-blooms/bloom/v3 v3.0.1
|
2020-03-18 19:19:29 -04:00
|
|
|
github.com/cespare/xxhash/v2 v2.1.1
|
2020-11-19 14:50:22 -05:00
|
|
|
github.com/cheggaaa/pb v1.0.29
|
2021-02-02 18:37:30 -05:00
|
|
|
github.com/colinmarc/hdfs/v2 v2.2.0
|
2019-04-02 21:28:39 -04:00
|
|
|
github.com/coredns/coredns v1.4.0
|
2021-04-19 13:30:42 -04:00
|
|
|
github.com/coreos/go-semver v0.3.0 // indirect
|
|
|
|
github.com/coreos/go-systemd v0.0.0-20190321100706-95778dfbb74e // indirect
|
|
|
|
github.com/coreos/pkg v0.0.0-20180928190104-399ea9e2e55f // indirect
|
2020-06-12 23:04:01 -04:00
|
|
|
github.com/dchest/siphash v1.2.1
|
2019-04-02 21:28:39 -04:00
|
|
|
github.com/dgrijalva/jwt-go v3.2.0+incompatible
|
|
|
|
github.com/djherbis/atime v1.0.0
|
2020-10-04 13:18:46 -04:00
|
|
|
github.com/dswarbrick/smart v0.0.0-20190505152634-909a45200d6d
|
2019-04-02 21:28:39 -04:00
|
|
|
github.com/dustin/go-humanize v1.0.0
|
2020-12-12 23:31:28 -05:00
|
|
|
github.com/eclipse/paho.mqtt.golang v1.3.0
|
2021-05-10 18:06:58 -04:00
|
|
|
github.com/elastic/go-elasticsearch/v7 v7.12.0
|
2020-12-12 23:31:28 -05:00
|
|
|
github.com/fatih/color v1.10.0
|
2019-03-19 16:50:59 -04:00
|
|
|
github.com/fatih/structs v1.1.0
|
2021-02-04 16:49:52 -05:00
|
|
|
github.com/go-ldap/ldap/v3 v3.2.4
|
2020-06-04 19:11:42 -04:00
|
|
|
github.com/go-sql-driver/mysql v1.5.0
|
2021-04-19 13:30:42 -04:00
|
|
|
github.com/gomodule/redigo v2.0.0+incompatible
|
2020-12-12 23:31:28 -05:00
|
|
|
github.com/google/uuid v1.1.2
|
2021-04-29 22:01:43 -04:00
|
|
|
github.com/gorilla/handlers v1.5.1
|
2020-08-23 02:14:02 -04:00
|
|
|
github.com/gorilla/mux v1.8.0
|
2021-04-19 13:30:42 -04:00
|
|
|
github.com/hashicorp/golang-lru v0.5.3 // indirect
|
2021-02-02 18:37:30 -05:00
|
|
|
github.com/jcmturner/gokrb5/v8 v8.4.2
|
2021-04-29 22:01:43 -04:00
|
|
|
github.com/json-iterator/go v1.1.11
|
|
|
|
github.com/klauspost/compress v1.12.2
|
2021-03-10 12:41:29 -05:00
|
|
|
github.com/klauspost/cpuid/v2 v2.0.4
|
2020-11-27 13:10:15 -05:00
|
|
|
github.com/klauspost/pgzip v1.2.5
|
2019-11-05 17:20:37 -05:00
|
|
|
github.com/klauspost/readahead v1.3.1
|
2021-02-04 16:49:52 -05:00
|
|
|
github.com/klauspost/reedsolomon v1.9.11
|
2021-03-29 11:07:23 -04:00
|
|
|
github.com/lib/pq v1.9.0
|
2020-12-12 23:31:28 -05:00
|
|
|
github.com/mattn/go-colorable v0.1.8
|
|
|
|
github.com/mattn/go-isatty v0.0.12
|
2020-12-10 16:23:06 -05:00
|
|
|
github.com/miekg/dns v1.1.35
|
2019-10-23 01:59:13 -04:00
|
|
|
github.com/minio/cli v1.22.0
|
2021-04-29 22:01:43 -04:00
|
|
|
github.com/minio/csvparser v1.0.0
|
2021-03-25 14:44:58 -04:00
|
|
|
github.com/minio/highwayhash v1.0.2
|
2021-05-10 21:15:11 -04:00
|
|
|
github.com/minio/kes v0.14.0
|
2021-05-06 15:55:54 -04:00
|
|
|
github.com/minio/madmin-go v1.0.2
|
2021-03-03 14:13:31 -05:00
|
|
|
github.com/minio/minio-go/v7 v7.0.11-0.20210302210017-6ae69c73ce78
|
2021-05-03 11:51:43 -04:00
|
|
|
github.com/minio/parquet-go v1.0.0
|
2021-04-29 22:01:43 -04:00
|
|
|
github.com/minio/rpc v1.0.0
|
2020-08-01 02:58:22 -04:00
|
|
|
github.com/minio/selfupdate v0.3.1
|
2021-02-22 16:19:53 -05:00
|
|
|
github.com/minio/sha256-simd v1.0.0
|
2021-02-04 16:49:52 -05:00
|
|
|
github.com/minio/simdjson-go v0.2.1
|
2021-05-07 17:41:58 -04:00
|
|
|
github.com/minio/sio v0.3.0
|
2019-03-19 16:50:59 -04:00
|
|
|
github.com/mitchellh/go-homedir v1.1.0
|
2020-03-27 00:07:39 -04:00
|
|
|
github.com/montanaflynn/stats v0.5.0
|
2020-12-09 17:30:35 -05:00
|
|
|
github.com/nats-io/nats-server/v2 v2.1.9
|
2020-07-07 22:04:29 -04:00
|
|
|
github.com/nats-io/nats.go v1.10.0
|
2021-03-29 11:07:23 -04:00
|
|
|
github.com/nats-io/stan.go v0.8.3
|
2019-04-24 00:25:06 -04:00
|
|
|
github.com/ncw/directio v1.0.5
|
2020-12-12 23:31:28 -05:00
|
|
|
github.com/nsqio/go-nsq v1.0.8
|
2020-12-10 16:23:06 -05:00
|
|
|
github.com/olivere/elastic/v7 v7.0.22
|
2021-01-07 22:27:31 -05:00
|
|
|
github.com/philhofer/fwd v1.1.1
|
2021-05-03 11:51:43 -04:00
|
|
|
github.com/pierrec/lz4 v2.6.0+incompatible
|
2020-08-23 12:43:48 -04:00
|
|
|
github.com/pkg/errors v0.9.1
|
2020-12-12 23:31:28 -05:00
|
|
|
github.com/prometheus/client_golang v1.8.0
|
2021-01-18 23:35:38 -05:00
|
|
|
github.com/prometheus/client_model v0.2.0
|
2021-03-29 11:07:23 -04:00
|
|
|
github.com/prometheus/procfs v0.6.0
|
2019-03-19 16:50:59 -04:00
|
|
|
github.com/rjeczalik/notify v0.9.2
|
2020-06-04 17:58:34 -04:00
|
|
|
github.com/rs/cors v1.7.0
|
2021-02-04 16:49:52 -05:00
|
|
|
github.com/secure-io/sio-go v0.3.1
|
2021-05-06 11:52:02 -04:00
|
|
|
github.com/shirou/gopsutil/v3 v3.21.3
|
2021-04-19 13:30:42 -04:00
|
|
|
github.com/spf13/pflag v1.0.5 // indirect
|
2020-12-12 23:31:28 -05:00
|
|
|
github.com/streadway/amqp v1.0.0
|
2021-05-21 14:41:25 -04:00
|
|
|
github.com/tinylib/msgp v1.1.6-0.20210521143832-0becd170c402
|
2021-05-20 12:00:11 -04:00
|
|
|
github.com/valyala/bytebufferpool v1.0.0
|
2019-03-19 16:50:59 -04:00
|
|
|
github.com/valyala/tcplisten v0.0.0-20161114210144-ceec8f93295a
|
2020-03-20 14:10:27 -04:00
|
|
|
github.com/xdg/scram v0.0.0-20180814205039-7eeb5667e42c
|
2020-12-03 14:35:18 -05:00
|
|
|
go.etcd.io/etcd v0.0.0-20201125193152-8a03d2e9614b
|
2021-04-19 13:30:42 -04:00
|
|
|
go.uber.org/zap v1.14.1
|
2021-05-06 11:52:02 -04:00
|
|
|
golang.org/x/crypto v0.0.0-20210421170649-83a5a9bb288b
|
2021-05-11 20:16:44 -04:00
|
|
|
golang.org/x/sys v0.0.0-20210510120138-977fb7262007
|
2021-04-19 13:30:42 -04:00
|
|
|
golang.org/x/time v0.0.0-20200416051211-89c76fbcd5d1 // indirect
|
2021-05-11 20:16:44 -04:00
|
|
|
golang.org/x/tools v0.1.1 // indirect
|
2021-05-10 21:15:11 -04:00
|
|
|
google.golang.org/api v0.31.0
|
2020-12-12 23:31:28 -05:00
|
|
|
gopkg.in/yaml.v2 v2.3.0
|
2021-04-19 13:30:42 -04:00
|
|
|
honnef.co/go/tools v0.0.1-2020.1.6 // indirect
|
|
|
|
sigs.k8s.io/yaml v1.2.0 // indirect
|
2021-01-29 19:45:11 -05:00
|
|
|
)
|
2021-05-10 21:15:11 -04:00
|
|
|
|
|
|
|
replace google.golang.org/grpc => google.golang.org/grpc v1.29.1
|