MyFavMirrors/minio
1
0
Fork 0
mirror of https://github.com/minio/minio.git synced 2025-12-03 06:22:32 -05:00
Code Issues Packages Projects Releases Wiki Activity

Compress better on amd64 (#12974)

Browse Source 
Since S2 has amd64 assembly, it now operates at a reasonable 
speed to use by default.

Here are some examples of stream compression speed, 16 cores:
```
nyc-taxi-data-10M.csv	s2	1	3325605752	-> 1095998837	312ms	10139.07MB/s		67.04% reduction
nyc-taxi-data-10M.csv	s2	2	3325605752	-> 917905514	428ms	7393.74MB/s		72.40%

github-june-2days-2019.json	s2	1	6273951764	-> 1043196283	391ms	15301.99 MB/s		83.37%
github-june-2days-2019.json	s2	2	6273951764	-> 955924506	519ms	11510.81MB/s		84.76%

github-ranks-backup.bin	s2	1	1862623243	-> 623911363	146ms	12133MB/s		66.50%
github-ranks-backup.bin	s2	2	1862623243	-> 563752759	230ms	7705.26MB/s		69.73%
```

We keep non-assembly platforms on the faster, but less efficient mode.
This commit is contained in:
Klaus Post
2021-08-16 20:55:07 +02:00
committed by GitHub
parent 47dfc1b1b0
commit 92bb2928e4
3 changed files with 14 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
go.mod
View File

@@ -32,7 +32,7 @@ require (
github.com/gorilla/mux v1.8.0
github.com/jcmturner/gokrb5/v8 v8.4.2
github.com/json-iterator/go v1.1.11
github.com/klauspost/compress v1.13.3
github.com/klauspost/compress v1.13.4
github.com/klauspost/cpuid/v2 v2.0.4
github.com/klauspost/pgzip v1.2.5
github.com/klauspost/readahead v1.3.1