118270d76f
Vendor the upstream changes with Avx512 ( #7225 )
...
Thanks to @fwessels we have Avx512 support with 4x improvement
2019-02-12 17:32:27 +05:30
def04f01cf
Update reedsolomon/highwayhash to start using ppc64le support ( #7003 )
...
Thanks to @fwessels for the upstream work on reedsolomon and
highwayhash which has resulted in 10x performance improvement
on ppc64 architecture.
2018-12-20 23:17:05 +05:30
c0721164be
Automatically set goroutines based on shardSize ( #5346 )
...
Update reedsolomon library to enable feature to automatically
set number of go-routines based on the input shard size,
since shard size is sort of a constant in Minio for
objects > 10MiB (default blocksize)
klauspost reported around 15-20% improvement in performance
numbers on older systems such as AVX and SSE3
```
name old speed new speed delta
Encode10x2x10000-8 5.45GB/s ± 1% 6.22GB/s ± 1% +14.20% (p=0.000 n=9+9)
Encode100x20x10000-8 1.44GB/s ± 1% 1.64GB/s ± 1% +13.77% (p=0.000 n=10+10)
Encode17x3x1M-8 10.0GB/s ± 5% 12.0GB/s ± 1% +19.88% (p=0.000 n=10+10)
Encode10x4x16M-8 7.81GB/s ± 5% 8.56GB/s ± 5% +9.58% (p=0.000 n=10+9)
Encode5x2x1M-8 15.3GB/s ± 2% 19.6GB/s ± 2% +28.57% (p=0.000 n=9+10)
Encode10x2x1M-8 12.2GB/s ± 5% 15.0GB/s ± 5% +22.45% (p=0.000 n=10+10)
Encode10x4x1M-8 7.84GB/s ± 1% 9.03GB/s ± 1% +15.19% (p=0.000 n=9+9)
Encode50x20x1M-8 1.73GB/s ± 4% 2.09GB/s ± 4% +20.59% (p=0.000 n=10+9)
Encode17x3x16M-8 10.6GB/s ± 1% 11.7GB/s ± 4% +10.12% (p=0.000 n=8+10)
```
2018-01-03 13:47:22 -08:00
6e6aeb6a9e
Updated version of klauspost/reedsolomon using proper AVX2 instructions as well a providing support for Cauchy matrices. ( #5215 )
2017-11-24 14:23:20 -08:00
93f126364e
Updated version of klauspost/reedsolomon with NEON support for ARM ( #4865 )
2017-08-30 09:49:00 -07:00
85fcee1919
erasure: simplify XL backend operations ( #4649 ) ( #4758 )
...
This change provides new implementations of the XL backend operations:
- create file
- read file
- heal file
Further this change adds table based tests for all three operations.
This affects also the bitrot algorithm integration. Algorithms are now
integrated in an idiomatic way (like crypto.Hash).
Fixes #4696
Fixes #4649
Fixes #4359
2017-08-14 18:08:42 -07:00
fffe4ac7e6
Prevent unnecessary verification of parity blocks while reading ( #4683 )
...
* Prevent unnecessary verification of parity blocks while reading erasure
coded file.
* Update klauspost/reedsolomon and just only reconstruct data blocks while
reading (prevent unnecessary parity block reconstruction)
* Remove Verification of (all) reconstructed Data and Parity blocks since
in our case we are protected by bit rot protection. And even if the
verification would fail (essentially impossible) there is no way to
definitively say whether the data is still correct or not, so this call
make no sense for our use case.
2017-08-11 18:25:46 -07:00
faa6b1e925
vendorize deps for snappy, blake2b and sha256 ( #3476 )
...
Bring in new optimization and portability changes.
Fixes https://github.com/minio/minio-go/issues/578
2016-12-19 19:32:55 -08:00
35d438e0ff
vendorize: update all vendorized packages. ( #2206 )
...
Bring in new changes from upstream for all the packages.
Important ones include
- gorilla/mux
- logrus
- jwt
2016-07-14 14:59:20 -07:00
a98a7fb1ad
Implement XL layer - preliminary work.
2016-04-25 12:47:31 -07:00
Harshavardhana
Frank Wessels
Andreas Auernhammer
Harshavardhana