MyFavMirrors/minio
1
0
Fork 0
mirror of https://github.com/minio/minio.git synced 2025-11-13 07:11:44 -05:00
Code Issues Packages Projects Releases Wiki Activity

Automatically set goroutines based on shardSize (#5346)

Browse Source 
Update reedsolomon library to enable feature to automatically
set number of go-routines based on the input shard size,
since shard size is sort of a constant in Minio for
objects > 10MiB (default blocksize)

klauspost reported around 15-20% improvement in performance
numbers on older systems such as AVX and SSE3

```
name                  old speed      new speed      delta
Encode10x2x10000-8    5.45GB/s ± 1%  6.22GB/s ± 1%  +14.20%    (p=0.000 n=9+9)
Encode100x20x10000-8  1.44GB/s ± 1%  1.64GB/s ± 1%  +13.77%  (p=0.000 n=10+10)
Encode17x3x1M-8       10.0GB/s ± 5%  12.0GB/s ± 1%  +19.88%  (p=0.000 n=10+10)
Encode10x4x16M-8      7.81GB/s ± 5%  8.56GB/s ± 5%   +9.58%   (p=0.000 n=10+9)
Encode5x2x1M-8        15.3GB/s ± 2%  19.6GB/s ± 2%  +28.57%   (p=0.000 n=9+10)
Encode10x2x1M-8       12.2GB/s ± 5%  15.0GB/s ± 5%  +22.45%  (p=0.000 n=10+10)
Encode10x4x1M-8       7.84GB/s ± 1%  9.03GB/s ± 1%  +15.19%    (p=0.000 n=9+9)
Encode50x20x1M-8      1.73GB/s ± 4%  2.09GB/s ± 4%  +20.59%   (p=0.000 n=10+9)
Encode17x3x16M-8      10.6GB/s ± 1%  11.7GB/s ± 4%  +10.12%   (p=0.000 n=8+10)
```
This commit is contained in:
Harshavardhana
2018-01-03 13:47:22 -08:00
committed by kannappanr
parent b1fb550d5c
commit c0721164be
13 changed files with 151 additions and 47 deletions
Download Patch File Download Diff File Expand all files Collapse all files

60
vendor/vendor.json vendored
View File

@@ -268,30 +268,6 @@
"revisionTime": "2016-01-19T13:13:26-08:00"
},
{
"checksumSHA1": "NYs0qvjZwsMZAXMtg2HRiED2cb4=",
"path": "github.com/joyent/triton-go",
"revision": "8365851ee7afcbb4cc1c7ba2e414b242ce0574f1",
"revisionTime": "2017-12-15T19:09:06Z"
},
{
"checksumSHA1": "Cth7NCLH/HaeKh9ZMRpQtudTEQQ=",
"path": "github.com/joyent/triton-go/authentication",
"revision": "8365851ee7afcbb4cc1c7ba2e414b242ce0574f1",
"revisionTime": "2017-12-15T19:09:06Z"
},
{
"checksumSHA1": "3ju04DVaxotpCKBF3Q/0vCSOlec=",
"path": "github.com/joyent/triton-go/client",
"revision": "8365851ee7afcbb4cc1c7ba2e414b242ce0574f1",
"revisionTime": "2017-12-15T19:09:06Z"
},
{
"checksumSHA1": "/WtyDZMgstGbBYtQ0f+ZfKMS4v8=",
"path": "github.com/joyent/triton-go/storage",
"revision": "8365851ee7afcbb4cc1c7ba2e414b242ce0574f1",
"revisionTime": "2017-12-15T19:09:06Z"
},
{
"checksumSHA1": "K6exl2ouL7d8cR2i378EzZOdRVI=",
"path": "github.com/howeyc/gopass",
"revision": "bf9dde6d0d2c004a008c27aaee91170c786f6db8",
@@ -315,6 +291,30 @@
"revision": "8152e7eb6ccf8679a64582a66b78519688d156ad",
"revisionTime": "2016-01-12T19:33:35Z"
},
{
"checksumSHA1": "NYs0qvjZwsMZAXMtg2HRiED2cb4=",
"path": "github.com/joyent/triton-go",
"revision": "8365851ee7afcbb4cc1c7ba2e414b242ce0574f1",
"revisionTime": "2017-12-15T19:09:06Z"
},
{
"checksumSHA1": "Cth7NCLH/HaeKh9ZMRpQtudTEQQ=",
"path": "github.com/joyent/triton-go/authentication",
"revision": "8365851ee7afcbb4cc1c7ba2e414b242ce0574f1",
"revisionTime": "2017-12-15T19:09:06Z"
},
{
"checksumSHA1": "3ju04DVaxotpCKBF3Q/0vCSOlec=",
"path": "github.com/joyent/triton-go/client",
"revision": "8365851ee7afcbb4cc1c7ba2e414b242ce0574f1",
"revisionTime": "2017-12-15T19:09:06Z"
},
{
"checksumSHA1": "/WtyDZMgstGbBYtQ0f+ZfKMS4v8=",
"path": "github.com/joyent/triton-go/storage",
"revision": "8365851ee7afcbb4cc1c7ba2e414b242ce0574f1",
"revisionTime": "2017-12-15T19:09:06Z"
},
{
"path": "github.com/klauspost/cpuid",
"revision": "349c675778172472f5e8f3a3e0fe187e302e5a10",
@@ -327,10 +327,10 @@
"revisionTime": "2016-10-16T15:41:25Z"
},
{
"checksumSHA1": "Nlhq327ZGeEh1gkPEBNPJKFlwds=",
"checksumSHA1": "ehsrWipiGIWqa4To8TmelIx06vI=",
"path": "github.com/klauspost/reedsolomon",
"revision": "e52c150f961e65ab9538bf1276b33bf469f919d8",
"revisionTime": "2017-11-18T15:17:31Z"
"revision": "0b30fa71cc8e4e9010c9aba6d0320e2e5b163b29",
"revisionTime": "2017-12-19T13:34:37Z"
},
{
"checksumSHA1": "dNYxHiBLalTqluak2/Z8c3RsSEM=",
@@ -522,9 +522,9 @@
"checksumSHA1": "iqUXcP3VA+G1/gVLRpQpBUt/BuA=",
"path": "github.com/satori/uuid",
"revision": "5bf94b69c6b68ee1b541973bb8e1144db23a194b",
"revisionTime": "2017-03-21T23:07:31Z"
},
{
"revisionTime": "2017-03-21T23:07:31Z"
},
{
"checksumSHA1": "llmzhtIUy63V3Pl65RuEn18ck5g=",
"path": "github.com/segmentio/go-prompt",
"revision": "f0d19b6901ade831d5a3204edc0d6a7d6457fbb2",