2019-03-19 16:50:59 -04:00
|
|
|
module github.com/minio/minio
|
|
|
|
|
[feat]: change erasure coding default block size from 10MiB to 1MiB (#11721)
major performance improvements in range GETs to avoid large
read amplification when ranges are tiny and random
```
-------------------
Operation: GET
Operations: 142014 -> 339421
Duration: 4m50s -> 4m56s
* Average: +139.41% (+1177.3 MiB/s) throughput, +139.11% (+658.4) obj/s
* Fastest: +125.24% (+1207.4 MiB/s) throughput, +132.32% (+612.9) obj/s
* 50% Median: +139.06% (+1175.7 MiB/s) throughput, +133.46% (+660.9) obj/s
* Slowest: +203.40% (+1267.9 MiB/s) throughput, +198.59% (+753.5) obj/s
```
TTFB from 10MiB BlockSize
```
* First Access TTFB: Avg: 81ms, Median: 61ms, Best: 20ms, Worst: 2.056s
```
TTFB from 1MiB BlockSize
```
* First Access TTFB: Avg: 22ms, Median: 21ms, Best: 8ms, Worst: 91ms
```
Full object reads however do see a slight change which won't be
noticeable in real world, so not doing any comparisons
TTFB still had improvements with full object reads with 1MiB
```
* First Access TTFB: Avg: 68ms, Median: 35ms, Best: 11ms, Worst: 1.16s
```
v/s
TTFB with 10MiB
```
* First Access TTFB: Avg: 388ms, Median: 98ms, Best: 20ms, Worst: 4.156s
```
This change should affect all new uploads, previous uploads should
continue to work with business as usual. But dramatic improvements can
be seen with these changes.
2021-03-06 17:09:34 -05:00
|
|
|
go 1.16
|
2019-04-02 21:28:39 -04:00
|
|
|
|
2019-03-19 16:50:59 -04:00
|
|
|
require (
|
2021-05-10 21:15:11 -04:00
|
|
|
cloud.google.com/go/storage v1.10.0
|
2020-08-31 13:19:25 -04:00
|
|
|
github.com/Azure/azure-pipeline-go v0.2.2
|
|
|
|
github.com/Azure/azure-storage-blob-go v0.10.0
|
2020-12-12 23:31:28 -05:00
|
|
|
github.com/Shopify/sarama v1.27.2
|
2021-03-16 23:06:57 -04:00
|
|
|
github.com/VividCortex/ewma v1.1.1
|
2019-04-02 21:28:39 -04:00
|
|
|
github.com/alecthomas/participle v0.2.1
|
2020-11-17 12:13:28 -05:00
|
|
|
github.com/bcicen/jstream v1.0.1
|
2020-10-14 02:59:20 -04:00
|
|
|
github.com/beevik/ntp v0.3.0
|
2021-05-27 11:02:39 -04:00
|
|
|
github.com/bits-and-blooms/bloom/v3 v3.0.1
|
2021-10-28 00:14:02 -04:00
|
|
|
github.com/cespare/xxhash/v2 v2.1.2
|
2020-11-19 14:50:22 -05:00
|
|
|
github.com/cheggaaa/pb v1.0.29
|
2021-02-02 18:37:30 -05:00
|
|
|
github.com/colinmarc/hdfs/v2 v2.2.0
|
2019-04-02 21:28:39 -04:00
|
|
|
github.com/coredns/coredns v1.4.0
|
2021-10-18 11:44:36 -04:00
|
|
|
github.com/cosnicolaou/pbzip2 v1.0.1
|
2020-06-12 23:04:01 -04:00
|
|
|
github.com/dchest/siphash v1.2.1
|
2019-04-02 21:28:39 -04:00
|
|
|
github.com/djherbis/atime v1.0.0
|
2020-10-04 13:18:46 -04:00
|
|
|
github.com/dswarbrick/smart v0.0.0-20190505152634-909a45200d6d
|
2019-04-02 21:28:39 -04:00
|
|
|
github.com/dustin/go-humanize v1.0.0
|
2020-12-12 23:31:28 -05:00
|
|
|
github.com/eclipse/paho.mqtt.golang v1.3.0
|
2021-05-10 18:06:58 -04:00
|
|
|
github.com/elastic/go-elasticsearch/v7 v7.12.0
|
2021-10-28 00:14:02 -04:00
|
|
|
github.com/fatih/color v1.13.0
|
2021-02-04 16:49:52 -05:00
|
|
|
github.com/go-ldap/ldap/v3 v3.2.4
|
2021-10-23 21:38:33 -04:00
|
|
|
github.com/go-ole/go-ole v1.2.6 // indirect
|
2021-06-17 23:27:04 -04:00
|
|
|
github.com/go-openapi/loads v0.20.2
|
2020-06-04 19:11:42 -04:00
|
|
|
github.com/go-sql-driver/mysql v1.5.0
|
2021-10-28 00:14:02 -04:00
|
|
|
github.com/golang-jwt/jwt v3.2.2+incompatible
|
2021-04-19 13:30:42 -04:00
|
|
|
github.com/gomodule/redigo v2.0.0+incompatible
|
2021-10-28 00:14:02 -04:00
|
|
|
github.com/google/uuid v1.3.0
|
2020-08-23 02:14:02 -04:00
|
|
|
github.com/gorilla/mux v1.8.0
|
2021-02-02 18:37:30 -05:00
|
|
|
github.com/jcmturner/gokrb5/v8 v8.4.2
|
2021-10-23 21:38:33 -04:00
|
|
|
github.com/json-iterator/go v1.1.12
|
2021-10-28 00:14:02 -04:00
|
|
|
github.com/klauspost/compress v1.13.6
|
2021-10-23 21:38:33 -04:00
|
|
|
github.com/klauspost/cpuid/v2 v2.0.9
|
2020-11-27 13:10:15 -05:00
|
|
|
github.com/klauspost/pgzip v1.2.5
|
2019-11-05 17:20:37 -05:00
|
|
|
github.com/klauspost/readahead v1.3.1
|
2021-09-23 13:46:24 -04:00
|
|
|
github.com/klauspost/reedsolomon v1.9.13
|
2021-03-29 11:07:23 -04:00
|
|
|
github.com/lib/pq v1.9.0
|
2020-12-10 16:23:06 -05:00
|
|
|
github.com/miekg/dns v1.1.35
|
2019-10-23 01:59:13 -04:00
|
|
|
github.com/minio/cli v1.22.0
|
2021-10-28 00:14:02 -04:00
|
|
|
github.com/minio/console v0.11.1-0.20211028015028-610c8a46533c
|
2021-04-29 22:01:43 -04:00
|
|
|
github.com/minio/csvparser v1.0.0
|
2021-03-25 14:44:58 -04:00
|
|
|
github.com/minio/highwayhash v1.0.2
|
2021-05-10 21:15:11 -04:00
|
|
|
github.com/minio/kes v0.14.0
|
2021-11-02 18:27:03 -04:00
|
|
|
github.com/minio/madmin-go v1.1.11-0.20211102182201-e51fd3d6b104
|
2021-10-18 22:13:29 -04:00
|
|
|
github.com/minio/minio-go/v7 v7.0.15
|
2021-05-03 11:51:43 -04:00
|
|
|
github.com/minio/parquet-go v1.0.0
|
2021-10-12 20:22:53 -04:00
|
|
|
github.com/minio/pkg v1.1.5
|
2020-08-01 02:58:22 -04:00
|
|
|
github.com/minio/selfupdate v0.3.1
|
2021-02-22 16:19:53 -05:00
|
|
|
github.com/minio/sha256-simd v1.0.0
|
2021-02-04 16:49:52 -05:00
|
|
|
github.com/minio/simdjson-go v0.2.1
|
2021-05-07 17:41:58 -04:00
|
|
|
github.com/minio/sio v0.3.0
|
2021-10-14 15:37:30 -04:00
|
|
|
github.com/minio/zipindex v0.2.1
|
2019-03-19 16:50:59 -04:00
|
|
|
github.com/mitchellh/go-homedir v1.1.0
|
2021-06-14 17:54:37 -04:00
|
|
|
github.com/montanaflynn/stats v0.6.6
|
2021-07-23 21:33:31 -04:00
|
|
|
github.com/nats-io/nats-server/v2 v2.3.2
|
2021-06-01 11:55:49 -04:00
|
|
|
github.com/nats-io/nats-streaming-server v0.21.2 // indirect
|
2021-07-23 21:33:31 -04:00
|
|
|
github.com/nats-io/nats.go v1.11.1-0.20210623165838-4b75fc59ae30
|
2021-03-29 11:07:23 -04:00
|
|
|
github.com/nats-io/stan.go v0.8.3
|
2019-04-24 00:25:06 -04:00
|
|
|
github.com/ncw/directio v1.0.5
|
2020-12-12 23:31:28 -05:00
|
|
|
github.com/nsqio/go-nsq v1.0.8
|
2021-11-01 13:50:07 -04:00
|
|
|
github.com/philhofer/fwd v1.1.2-0.20210722190033-5c56ac6d0bb9
|
2021-05-03 11:51:43 -04:00
|
|
|
github.com/pierrec/lz4 v2.6.0+incompatible
|
2020-08-23 12:43:48 -04:00
|
|
|
github.com/pkg/errors v0.9.1
|
2021-10-28 00:14:02 -04:00
|
|
|
github.com/prometheus/client_golang v1.11.0
|
2021-01-18 23:35:38 -05:00
|
|
|
github.com/prometheus/client_model v0.2.0
|
2021-09-17 18:02:21 -04:00
|
|
|
github.com/prometheus/procfs v0.7.3
|
2020-06-04 17:58:34 -04:00
|
|
|
github.com/rs/cors v1.7.0
|
2021-10-05 13:13:04 -04:00
|
|
|
github.com/rs/dnscache v0.0.0-20210201191234-295bba877686
|
2021-02-04 16:49:52 -05:00
|
|
|
github.com/secure-io/sio-go v0.3.1
|
2021-10-23 21:38:33 -04:00
|
|
|
github.com/shirou/gopsutil/v3 v3.21.9
|
2020-12-12 23:31:28 -05:00
|
|
|
github.com/streadway/amqp v1.0.0
|
2021-11-01 13:50:07 -04:00
|
|
|
github.com/tinylib/msgp v1.1.7-0.20211026165309-e818a1881b0e
|
2021-05-20 12:00:11 -04:00
|
|
|
github.com/valyala/bytebufferpool v1.0.0
|
2020-03-20 14:10:27 -04:00
|
|
|
github.com/xdg/scram v0.0.0-20180814205039-7eeb5667e42c
|
2021-10-01 14:50:00 -04:00
|
|
|
github.com/yargevad/filepathx v1.0.0
|
2021-10-28 00:14:02 -04:00
|
|
|
go.etcd.io/etcd/api/v3 v3.5.0
|
|
|
|
go.etcd.io/etcd/client/v3 v3.5.0
|
2021-09-21 15:57:10 -04:00
|
|
|
go.opencensus.io v0.22.5 // indirect
|
2021-10-28 00:14:02 -04:00
|
|
|
go.uber.org/atomic v1.9.0
|
|
|
|
go.uber.org/zap v1.19.1
|
2021-10-23 21:38:33 -04:00
|
|
|
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519
|
|
|
|
golang.org/x/net v0.0.0-20211020060615-d418f374d309 // indirect
|
|
|
|
golang.org/x/sys v0.0.0-20211020174200-9d6173849985
|
2021-06-24 21:29:30 -04:00
|
|
|
golang.org/x/time v0.0.0-20210220033141-f8bda1e9f3ba
|
2021-05-10 21:15:11 -04:00
|
|
|
google.golang.org/api v0.31.0
|
2021-05-28 18:17:01 -04:00
|
|
|
gopkg.in/yaml.v2 v2.4.0
|
2021-01-29 19:45:11 -05:00
|
|
|
)
|