mirror of
https://github.com/minio/minio.git
synced 2025-01-13 16:03:21 -05:00
7e1661f4fa
This improves the performance of certain queries dramatically, such as 'count(*)' etc. Without this PR ``` ~ time mc select --query "select count(*) from S3Object" myminio/sjm-airlines/star2000.csv.gz 2173762 real 0m42.464s user 0m0.071s sys 0m0.010s ``` With this PR ``` ~ time mc select --query "select count(*) from S3Object" myminio/sjm-airlines/star2000.csv.gz 2173762 real 0m17.603s user 0m0.093s sys 0m0.008s ``` Almost a 250% improvement in performance. This PR avoids a lot of type conversions and instead relies on raw sequences of data and interprets them lazily. ``` benchcmp old new benchmark old ns/op new ns/op delta BenchmarkSQLAggregate_100K-4 551213 259782 -52.87% BenchmarkSQLAggregate_1M-4 6981901985 2432413729 -65.16% BenchmarkSQLAggregate_2M-4 13511978488 4536903552 -66.42% BenchmarkSQLAggregate_10M-4 68427084908 23266283336 -66.00% benchmark old allocs new allocs delta BenchmarkSQLAggregate_100K-4 2366 485 -79.50% BenchmarkSQLAggregate_1M-4 47455492 21462860 -54.77% BenchmarkSQLAggregate_2M-4 95163637 43110771 -54.70% BenchmarkSQLAggregate_10M-4 476959550 216906510 -54.52% benchmark old bytes new bytes delta BenchmarkSQLAggregate_100K-4 1233079 1086024 -11.93% BenchmarkSQLAggregate_1M-4 2607984120 557038536 -78.64% BenchmarkSQLAggregate_2M-4 5254103616 1128149168 -78.53% BenchmarkSQLAggregate_10M-4 26443524872 5722715992 -78.36% ``` |
||
|---|---|---|
| .. | ||
| aliyun/aliyun-oss-go-sdk/oss | ||
| Azure | ||
| beorn7/perks | ||
| cheggaaa/pb | ||
| coredns/coredns | ||
| coreos | ||
| davecgh/go-spew | ||
| dgrijalva/jwt-go | ||
| djherbis/atime | ||
| dustin/go-humanize | ||
| eapache | ||
| eclipse/paho.mqtt.golang | ||
| elazarl/go-bindata-assetfs | ||
| fatih | ||
| garyburd/redigo | ||
| go-ini/ini | ||
| go-sql-driver/mysql | ||
| gogo/protobuf | ||
| golang | ||
| googleapis/gax-go | ||
| gorilla | ||
| grpc-ecosystem/go-grpc-middleware | ||
| hashicorp | ||
| howeyc/gopass | ||
| inconshreveable/go-update | ||
| joyent/triton-go | ||
| klauspost | ||
| lib/pq | ||
| mattn | ||
| matttproud/golang_protobuf_extensions | ||
| miekg/dns | ||
| minio | ||
| mitchellh | ||
| nats-io | ||
| nsqio/go-nsq | ||
| pkg | ||
| prometheus | ||
| rjeczalik/notify | ||
| rs | ||
| ryanuber/go-glob | ||
| satori/uuid | ||
| segmentio/go-prompt | ||
| skyrings/skyring-common | ||
| streadway/amqp | ||
| tidwall | ||
| valyala/tcplisten | ||
| xwb1989/sqlparser | ||