minio/docs
Klaus Post ddea0bdf11 Concurrent CSV parsing and reduce S3 select allocations (#8200)
```
CSV parsing, BEFORE:
BenchmarkReaderBasic-12         	    2842	    407533 ns/op	  397860 B/op	     957 allocs/op
BenchmarkReaderReplace-12       	    2718	    429914 ns/op	  397844 B/op	     957 allocs/op
BenchmarkReaderReplaceTwo-12    	    2718	    435556 ns/op	  397855 B/op	     957 allocs/op
BenchmarkAggregateCount_100K-12    	     171	   6798974 ns/op	16667102 B/op	  308077 allocs/op
BenchmarkAggregateCount_1M-12    	      19	  65657411 ns/op	168057743 B/op	 3146610 allocs/op
BenchmarkSelectAll_10M-12    	       1	20882119900 ns/op	2758799896 B/op	41978762 allocs/op

CSV parsing, AFTER:
BenchmarkReaderBasic-12         	    3721	    312549 ns/op	  101920 B/op	     338 allocs/op
BenchmarkReaderReplace-12       	    3776	    318810 ns/op	  101993 B/op	     340 allocs/op
BenchmarkReaderReplaceTwo-12    	    3610	    330967 ns/op	  102012 B/op	     341 allocs/op
BenchmarkAggregateCount_100K-12    	     295	   4149588 ns/op	 3553623 B/op	  103261 allocs/op
BenchmarkAggregateCount_1M-12    	      30	  37746503 ns/op	33827931 B/op	 1049435 allocs/op
BenchmarkSelectAll_10M-12    	       1	17608495800 ns/op	1416504040 B/op	21007082 allocs/op

~ benchcmp old.txt new.txt
benchmark                           old ns/op       new ns/op       delta
BenchmarkReaderBasic-12             407533          312549          -23.31%
BenchmarkReaderReplace-12           429914          318810          -25.84%
BenchmarkReaderReplaceTwo-12        435556          330967          -24.01%
BenchmarkAggregateCount_100K-12     6798974         4149588         -38.97%
BenchmarkAggregateCount_1M-12       65657411        37746503        -42.51%
BenchmarkSelectAll_10M-12           20882119900     17608495800     -15.68%

benchmark                           old allocs     new allocs     delta
BenchmarkReaderBasic-12             957            338            -64.68%
BenchmarkReaderReplace-12           957            340            -64.47%
BenchmarkReaderReplaceTwo-12        957            341            -64.37%
BenchmarkAggregateCount_100K-12     308077         103261         -66.48%
BenchmarkAggregateCount_1M-12       3146610        1049435        -66.65%
BenchmarkSelectAll_10M-12           41978762       21007082       -49.96%

benchmark                           old bytes      new bytes      delta
BenchmarkReaderBasic-12             397860         101920         -74.38%
BenchmarkReaderReplace-12           397844         101993         -74.36%
BenchmarkReaderReplaceTwo-12        397855         102012         -74.36%
BenchmarkAggregateCount_100K-12     16667102       3553623        -78.68%
BenchmarkAggregateCount_1M-12       168057743      33827931       -79.87%
BenchmarkSelectAll_10M-12           2758799896     1416504040     -48.66%
```

```
BenchmarkReaderHuge/97K-12         	    2200	    540840 ns/op	 184.32 MB/s	 1604450 B/op	     687 allocs/op
BenchmarkReaderHuge/194K-12        	    1522	    752257 ns/op	 265.04 MB/s	 2143135 B/op	    1335 allocs/op
BenchmarkReaderHuge/389K-12        	    1190	    947858 ns/op	 420.69 MB/s	 3221831 B/op	    2630 allocs/op
BenchmarkReaderHuge/778K-12        	     806	   1472486 ns/op	 541.61 MB/s	 5201856 B/op	    5187 allocs/op
BenchmarkReaderHuge/1557K-12       	     426	   2575269 ns/op	 619.36 MB/s	 9101330 B/op	   10233 allocs/op
BenchmarkReaderHuge/3115K-12       	     286	   4034656 ns/op	 790.66 MB/s	12397968 B/op	   16099 allocs/op
BenchmarkReaderHuge/6230K-12       	     172	   6830563 ns/op	 934.05 MB/s	16008416 B/op	   26844 allocs/op
BenchmarkReaderHuge/12461K-12      	     100	  11409467 ns/op	1118.39 MB/s	22655163 B/op	   48107 allocs/op
BenchmarkReaderHuge/24922K-12      	      66	  19780395 ns/op	1290.19 MB/s	35158559 B/op	   90216 allocs/op
BenchmarkReaderHuge/49844K-12      	      34	  37282559 ns/op	1369.03 MB/s	60528624 B/op	  174497 allocs/op
```
2019-09-13 14:18:35 -07:00
..
bigdata Update docs with new settings (#7803) 2019-06-18 17:52:27 -07:00
bucket/notifications Enable event persistence in AMQP (#7565) 2019-07-25 11:20:24 -07:00
chroot Removes the incorrect coverage badge from the docs (#7651) 2019-05-16 12:11:49 +05:30
compression Update compression README.md (#7823) 2019-06-23 13:52:58 +05:30
config Remove deprecated ENVs 2019-08-07 18:57:38 -07:00
deployment/kernel-tuning Removes the incorrect coverage badge from the docs (#7651) 2019-05-16 12:11:49 +05:30
disk-caching Allow cached content to be encrypted (#8001) 2019-08-23 10:13:22 -07:00
distributed Add some design docs for distributed setup (#7950) 2019-07-23 07:48:10 +05:30
docker docs: Use --user to start container in non-root (#7966) 2019-07-24 17:35:52 -07:00
erasure docs: Fix dead link of HighwayHash (#7847) 2019-07-03 14:32:58 -07:00
federation/lookup Migrate minio etcd config to backend config (#7751) 2019-06-15 03:07:54 -07:00
gateway Doc: Fix PutObject API known limitations text (#7704) 2019-08-13 08:50:43 -07:00
kms add UpdateKey method to KMS interface (#7974) 2019-08-01 15:47:47 -07:00
lifecycle Add bucket lifecycle expiry feature (#7834) 2019-08-09 10:02:41 -07:00
logging Enhance audit logging to capture responseTimes (#8067) 2019-08-12 20:32:34 -07:00
metrics Bump readiness check upto 10000 go-routines (#8057) 2019-08-10 18:13:14 +05:30
multi-tenancy Removes the incorrect coverage badge from the docs (#7651) 2019-05-16 12:11:49 +05:30
multi-user Fix multi-user doc to reflect the latest change 2019-08-14 14:12:23 -07:00
orchestration Update yaml files to latest version RELEASE.2019-09-11T19-53-16Z 2019-09-11 19:59:05 +00:00
screenshots doc: Merge large bucket with distributed docs (#7761) 2019-06-11 13:44:33 -07:00
security Replace Minio refs in docs with MinIO and links (#7494) 2019-04-09 11:39:42 -07:00
select Concurrent CSV parsing and reduce S3 select allocations (#8200) 2019-09-13 14:18:35 -07:00
shared-backend Removes the incorrect coverage badge from the docs (#7651) 2019-05-16 12:11:49 +05:30
sts LDAP STS API (#8091) 2019-09-10 04:42:29 +05:30
tls remove encryption key section for certtool docs (#7820) 2019-06-21 10:39:02 -07:00
zh_CN docs: Fix dead link of HighwayHash (#7847) 2019-07-03 14:32:58 -07:00
minio-limits.md Update doc with Object Tagging not supported (#8116) 2019-08-21 16:10:59 -07:00