MyFavMirrors/minio
1
0
Fork 0
mirror of https://github.com/minio/minio.git synced 2024-12-26 23:25:54 -05:00
Code Issues Packages Projects Releases Wiki Activity
5d954ea228
minio/go.mod

103 lines
4.0 KiB
Modula-2
Raw Normal View History

add go modules file and start running go 1.11 style builds (#7354)
2019-03-19 16:50:59 -04:00
module github.com/minio/minio
[feat]: change erasure coding default block size from 10MiB to 1MiB (#11721) major performance improvements in range GETs to avoid large read amplification when ranges are tiny and random ``` ------------------- Operation: GET Operations: 142014 -> 339421 Duration: 4m50s -> 4m56s * Average: +139.41% (+1177.3 MiB/s) throughput, +139.11% (+658.4) obj/s * Fastest: +125.24% (+1207.4 MiB/s) throughput, +132.32% (+612.9) obj/s * 50% Median: +139.06% (+1175.7 MiB/s) throughput, +133.46% (+660.9) obj/s * Slowest: +203.40% (+1267.9 MiB/s) throughput, +198.59% (+753.5) obj/s ``` TTFB from 10MiB BlockSize ``` * First Access TTFB: Avg: 81ms, Median: 61ms, Best: 20ms, Worst: 2.056s ``` TTFB from 1MiB BlockSize ``` * First Access TTFB: Avg: 22ms, Median: 21ms, Best: 8ms, Worst: 91ms ``` Full object reads however do see a slight change which won't be noticeable in real world, so not doing any comparisons TTFB still had improvements with full object reads with 1MiB ``` * First Access TTFB: Avg: 68ms, Median: 35ms, Best: 11ms, Worst: 1.16s ``` v/s TTFB with 10MiB ``` * First Access TTFB: Avg: 388ms, Median: 98ms, Best: 20ms, Worst: 4.156s ``` This change should affect all new uploads, previous uploads should continue to work with business as usual. But dramatic improvements can be seen with these changes.
2021-03-06 17:09:34 -05:00
go 1.16
Migrate to go1.12 to simplify our cmd/http package (#7302) Simplify the cmd/http package overall by removing custom plain text v/s tls connection detection, by migrating to go1.12 and choose minimum version to be go1.12 Also remove all the vendored deps, since they are not useful anymore.
2019-04-02 21:28:39 -04:00
add go modules file and start running go 1.11 style builds (#7354)
2019-03-19 16:50:59 -04:00
require (
Support for remote tier management (#12090) With this change, MinIO's ILM supports transitioning objects to a remote tier. This change includes support for Azure Blob Storage, AWS S3 compatible object storage incl. MinIO and Google Cloud Storage as remote tier storage backends. Some new additions include: - Admin APIs remote tier configuration management - Simple journal to track remote objects to be 'collected' This is used by object API handlers which 'mutate' object versions by overwriting/replacing content (Put/CopyObject) or removing the version itself (e.g DeleteObjectVersion). - Rework of previous ILM transition to fit the new model In the new model, a storage class (a.k.a remote tier) is defined by the 'remote' object storage type (one of s3, azure, GCS), bucket name and a prefix. * Fixed bugs, review comments, and more unit-tests - Leverage inline small object feature - Migrate legacy objects to the latest object format before transitioning - Fix restore to particular version if specified - Extend SharedDataDirCount to handle transitioned and restored objects - Restore-object should accept version-id for version-suspended bucket (#12091) - Check if remote tier creds have sufficient permissions - Bonus minor fixes to existing error messages Co-authored-by: Poorna Krishnamoorthy <poorna@minio.io> Co-authored-by: Krishna Srinivas <krishna@minio.io> Signed-off-by: Harshavardhana <harsha@minio.io>
2021-04-19 13:30:42 -04:00
cloud.google.com/go/storage v1.8.0
Add archived parquet as int. package (#9912) Since github.com/minio/parquet-go is archived add it as internal package.
2020-06-25 10:31:16 -04:00
git.apache.org/thrift.git v0.13.0
update for smartos build (solaris too) (#10378)
2020-08-31 13:19:25 -04:00
github.com/Azure/azure-pipeline-go v0.2.2
github.com/Azure/azure-storage-blob-go v0.10.0
fix: web handlers to enforce replication (#10249) This PR also preserves source ETag for replication
2020-08-12 20:32:24 -04:00
github.com/Azure/go-autorest/autorest/adal v0.9.1 // indirect
upgrade event notification dependencies (#11095)
2020-12-12 23:31:28 -05:00
github.com/Shopify/sarama v1.27.2
add additional metrics per disk API latency, API call counts #11250) ``` mc admin info --json ``` provides these details, for now, we shall eventually expose this at Prometheus level eventually. Co-authored-by: Harshavardhana <harsha@minio.io>
2021-03-16 23:06:57 -04:00
github.com/VividCortex/ewma v1.1.1
Migrate to go1.12 to simplify our cmd/http package (#7302) Simplify the cmd/http package overall by removing custom plain text v/s tls connection detection, by migrating to go1.12 and choose minimum version to be go1.12 Also remove all the vendored deps, since they are not useful anymore.
2019-04-02 21:28:39 -04:00
github.com/alecthomas/participle v0.2.1
update jstream to latest release v1.0.1 (#10909)
2020-11-17 12:13:28 -05:00
github.com/bcicen/jstream v1.0.1
update NTP package for accurate time resolution fixes (#10670)
2020-10-14 02:59:20 -04:00
github.com/beevik/ntp v0.3.0
re-implement data usage crawler to be more efficient (#9075) Implementation overview: https://gist.github.com/klauspost/1801c858d5e0df391114436fdad6987b
2020-03-18 19:19:29 -04:00
github.com/cespare/xxhash/v2 v2.1.1
Add support for replication of object tags, retention metadata (#10880)
2020-11-19 14:50:22 -05:00
github.com/cheggaaa/pb v1.0.29
Update colinmarc/hdfs (#11417) Updates needed dependency as well. Fixes #11416
2021-02-02 18:37:30 -05:00
github.com/colinmarc/hdfs/v2 v2.2.0
Migrate to go1.12 to simplify our cmd/http package (#7302) Simplify the cmd/http package overall by removing custom plain text v/s tls connection detection, by migrating to go1.12 and choose minimum version to be go1.12 Also remove all the vendored deps, since they are not useful anymore.
2019-04-02 21:28:39 -04:00
github.com/coredns/coredns v1.4.0
Support for remote tier management (#12090) With this change, MinIO's ILM supports transitioning objects to a remote tier. This change includes support for Azure Blob Storage, AWS S3 compatible object storage incl. MinIO and Google Cloud Storage as remote tier storage backends. Some new additions include: - Admin APIs remote tier configuration management - Simple journal to track remote objects to be 'collected' This is used by object API handlers which 'mutate' object versions by overwriting/replacing content (Put/CopyObject) or removing the version itself (e.g DeleteObjectVersion). - Rework of previous ILM transition to fit the new model In the new model, a storage class (a.k.a remote tier) is defined by the 'remote' object storage type (one of s3, azure, GCS), bucket name and a prefix. * Fixed bugs, review comments, and more unit-tests - Leverage inline small object feature - Migrate legacy objects to the latest object format before transitioning - Fix restore to particular version if specified - Extend SharedDataDirCount to handle transitioned and restored objects - Restore-object should accept version-id for version-suspended bucket (#12091) - Check if remote tier creds have sufficient permissions - Bonus minor fixes to existing error messages Co-authored-by: Poorna Krishnamoorthy <poorna@minio.io> Co-authored-by: Krishna Srinivas <krishna@minio.io> Signed-off-by: Harshavardhana <harsha@minio.io>
2021-04-19 13:30:42 -04:00
github.com/coreos/go-semver v0.3.0 // indirect
github.com/coreos/go-systemd v0.0.0-20190321100706-95778dfbb74e // indirect
github.com/coreos/pkg v0.0.0-20180928190104-399ea9e2e55f // indirect
Support bucket versioning (#9377) - Implement a new xl.json 2.0.0 format to support, this moves the entire marshaling logic to POSIX layer, top layer always consumes a common FileInfo construct which simplifies the metadata reads. - Implement list object versions - Migrate to siphash from crchash for new deployments for object placements. Fixes #2111
2020-06-12 23:04:01 -04:00
github.com/dchest/siphash v1.2.1
Migrate to go1.12 to simplify our cmd/http package (#7302) Simplify the cmd/http package overall by removing custom plain text v/s tls connection detection, by migrating to go1.12 and choose minimum version to be go1.12 Also remove all the vendored deps, since they are not useful anymore.
2019-04-02 21:28:39 -04:00
github.com/dgrijalva/jwt-go v3.2.0+incompatible
github.com/djherbis/atime v1.0.0
add NVMe drive info [model num, serial num, drive temp. etc.] (#10613) * add NVMe drive info [model num, serial num, drive temp. etc.] * Ignore fuse partitions * Add the nvme logic only for linux * Move smart/nvme structs to a separate file Co-authored-by: wlan0 <sidharthamn@gmail.com>
2020-10-04 13:18:46 -04:00
github.com/dswarbrick/smart v0.0.0-20190505152634-909a45200d6d
Migrate to go1.12 to simplify our cmd/http package (#7302) Simplify the cmd/http package overall by removing custom plain text v/s tls connection detection, by migrating to go1.12 and choose minimum version to be go1.12 Also remove all the vendored deps, since they are not useful anymore.
2019-04-02 21:28:39 -04:00
github.com/dustin/go-humanize v1.0.0
upgrade event notification dependencies (#11095)
2020-12-12 23:31:28 -05:00
github.com/eclipse/paho.mqtt.golang v1.3.0
github.com/fatih/color v1.10.0
add go modules file and start running go 1.11 style builds (#7354)
2019-03-19 16:50:59 -04:00
github.com/fatih/structs v1.1.0
update dependenices for relevant projects (#11445) - minio-go -> v7.0.8 - ldap/v3 -> v3.2.4 - reedsolomon -> v1.9.11 - sio-go -> v0.3.1 - msgp -> v1.1.5 - simdjson-go, md5-simd, highwayhash
2021-02-04 16:49:52 -05:00
github.com/go-ldap/ldap/v3 v3.2.4
Update go-sql-driver pkg to support `checkConnLiveness` (#9766) Fixes #9679
2020-06-04 19:11:42 -04:00
github.com/go-sql-driver/mysql v1.5.0
Support for remote tier management (#12090) With this change, MinIO's ILM supports transitioning objects to a remote tier. This change includes support for Azure Blob Storage, AWS S3 compatible object storage incl. MinIO and Google Cloud Storage as remote tier storage backends. Some new additions include: - Admin APIs remote tier configuration management - Simple journal to track remote objects to be 'collected' This is used by object API handlers which 'mutate' object versions by overwriting/replacing content (Put/CopyObject) or removing the version itself (e.g DeleteObjectVersion). - Rework of previous ILM transition to fit the new model In the new model, a storage class (a.k.a remote tier) is defined by the 'remote' object storage type (one of s3, azure, GCS), bucket name and a prefix. * Fixed bugs, review comments, and more unit-tests - Leverage inline small object feature - Migrate legacy objects to the latest object format before transitioning - Fix restore to particular version if specified - Extend SharedDataDirCount to handle transitioned and restored objects - Restore-object should accept version-id for version-suspended bucket (#12091) - Check if remote tier creds have sufficient permissions - Bonus minor fixes to existing error messages Co-authored-by: Poorna Krishnamoorthy <poorna@minio.io> Co-authored-by: Krishna Srinivas <krishna@minio.io> Signed-off-by: Harshavardhana <harsha@minio.io>
2021-04-19 13:30:42 -04:00
github.com/gomodule/redigo v2.0.0+incompatible
github.com/google/martian v2.1.1-0.20190517191504-25dcb96d9e51+incompatible // indirect
upgrade event notification dependencies (#11095)
2020-12-12 23:31:28 -05:00
github.com/google/uuid v1.1.2
fix: remove gorilla/rpc/ deps fully after our fork (#11108)
2020-12-15 14:18:06 -05:00
github.com/gorilla/handlers v1.5.1
move to gorilla mux 1.8.0 (#10328)
2020-08-23 02:14:02 -04:00
github.com/gorilla/mux v1.8.0
Support for remote tier management (#12090) With this change, MinIO's ILM supports transitioning objects to a remote tier. This change includes support for Azure Blob Storage, AWS S3 compatible object storage incl. MinIO and Google Cloud Storage as remote tier storage backends. Some new additions include: - Admin APIs remote tier configuration management - Simple journal to track remote objects to be 'collected' This is used by object API handlers which 'mutate' object versions by overwriting/replacing content (Put/CopyObject) or removing the version itself (e.g DeleteObjectVersion). - Rework of previous ILM transition to fit the new model In the new model, a storage class (a.k.a remote tier) is defined by the 'remote' object storage type (one of s3, azure, GCS), bucket name and a prefix. * Fixed bugs, review comments, and more unit-tests - Leverage inline small object feature - Migrate legacy objects to the latest object format before transitioning - Fix restore to particular version if specified - Extend SharedDataDirCount to handle transitioned and restored objects - Restore-object should accept version-id for version-suspended bucket (#12091) - Check if remote tier creds have sufficient permissions - Bonus minor fixes to existing error messages Co-authored-by: Poorna Krishnamoorthy <poorna@minio.io> Co-authored-by: Krishna Srinivas <krishna@minio.io> Signed-off-by: Harshavardhana <harsha@minio.io>
2021-04-19 13:30:42 -04:00
github.com/hashicorp/golang-lru v0.5.3 // indirect
Update colinmarc/hdfs (#11417) Updates needed dependency as well. Fixes #11416
2021-02-02 18:37:30 -05:00
github.com/jcmturner/gokrb5/v8 v8.4.2
Update minio-go library to latest (#9813)
2020-06-12 13:18:42 -04:00
github.com/json-iterator/go v1.1.10
Update S2 compression (#11753) Relevant updates: * Less allocations on decode: https://github.com/klauspost/compress/pull/322 * Fixed rare out-of-bounds write on amd64. * ARM64 decompression assembly. Around 2x output speed. https://github.com/klauspost/compress/pull/324 * Speed up decompression on non-assembly platforms. https://github.com/klauspost/compress/pull/328 Upgrade cpuid to match simdjson.
2021-03-10 12:41:29 -05:00
github.com/klauspost/compress v1.11.12
github.com/klauspost/cpuid/v2 v2.0.4
Upgrade compress and pgzip package (#10992) Should provide faster decompression for s3 select and other places where it is used.
2020-11-27 13:10:15 -05:00
github.com/klauspost/pgzip v1.2.5
Fix JSON Close data race. (#8486) The JSON stream library has no safe way of aborting while Since we cannot expect the called to safely handle "Read" and "Close" calls we must handle this. Also any Read error returned from upstream will crash the server. We preserve the errors and instead always return io.EOF upstream, but send the error on Close. `readahead v1.3.1` handles Read after Close better. Updates to `progressReader` is mostly to ensure safety. Fixes #8481
2019-11-05 17:20:37 -05:00
github.com/klauspost/readahead v1.3.1
update dependenices for relevant projects (#11445) - minio-go -> v7.0.8 - ldap/v3 -> v3.2.4 - reedsolomon -> v1.9.11 - sio-go -> v0.3.1 - msgp -> v1.1.5 - simdjson-go, md5-simd, highwayhash
2021-02-04 16:49:52 -05:00
github.com/klauspost/reedsolomon v1.9.11
use Access() instead of Lstat() for frequent use (#11911) using Lstat() is causing tiny memory allocations, that are usually wasted and never used, instead we can simply uses Access() call that does 0 memory allocations.
2021-03-29 11:07:23 -04:00
github.com/lib/pq v1.9.0
upgrade event notification dependencies (#11095)
2020-12-12 23:31:28 -05:00
github.com/mattn/go-colorable v0.1.8
fix: loading users regression additionally also move to latest gorilla/mux master to fix the DNS style bucket routing regression resolves #10022 resolves #10023
2020-07-11 17:02:07 -04:00
github.com/mattn/go-ieproxy v0.0.1 // indirect
upgrade event notification dependencies (#11095)
2020-12-12 23:31:28 -05:00
github.com/mattn/go-isatty v0.0.12
update deps to latest for some vulnerable deps (#11080) - github.com/miekg/dns - update elasticsearch library deps to circumvent some aws-sdk-go deps
2020-12-10 16:23:06 -05:00
github.com/miekg/dns v1.1.35
Migrate config to KV data format (#8392) - adding oauth support to MinIO browser (#8400) by @kanagaraj - supports multi-line get/set/del for all config fields - add support for comments, allow toggle - add extensive validation of config before saving - support MinIO browser to support proper claims, using STS tokens - env support for all config parameters, legacy envs are also supported with all documentation now pointing to latest ENVs - preserve accessKey/secretKey from FS mode setups - add history support implements three APIs - ClearHistory - RestoreHistory - ListHistory - add help command support for each config parameters - all the bug fixes after migration to KV, and other bug fixes encountered during testing.
2019-10-23 01:59:13 -04:00
github.com/minio/cli v1.22.0
highwayhash: update to latest version containing an arm64 fix (#11901) This commit updates the highwayhash version to `v1.0.2` that fixes a critical issue on arm64.
2021-03-25 14:44:58 -04:00
github.com/minio/highwayhash v1.0.2
update dependenices for relevant projects (#11445) - minio-go -> v7.0.8 - ldap/v3 -> v3.2.4 - reedsolomon -> v1.9.11 - sio-go -> v0.3.1 - msgp -> v1.1.5 - simdjson-go, md5-simd, highwayhash
2021-02-04 16:49:52 -05:00
github.com/minio/md5-simd v1.1.1 // indirect
Avoid notification event for replicas (#11683) Creating notification events for replica creation is not particularly useful to send as the notification event generated at source already includes replication completion events. For applications using replica cluster as failover, avoiding duplicate notifications for replica event will allow seamless failover.
2021-03-03 14:13:31 -05:00
github.com/minio/minio-go/v7 v7.0.11-0.20210302210017-6ae69c73ce78
update CREDITS with minio/selfupdate project
2020-08-01 02:58:22 -04:00
github.com/minio/selfupdate v0.3.1
update sha256-simd to v1.0.0 (#11607)
2021-02-22 16:19:53 -05:00
github.com/minio/sha256-simd v1.0.0
update dependenices for relevant projects (#11445) - minio-go -> v7.0.8 - ldap/v3 -> v3.2.4 - reedsolomon -> v1.9.11 - sio-go -> v0.3.1 - msgp -> v1.1.5 - simdjson-go, md5-simd, highwayhash
2021-02-04 16:49:52 -05:00
github.com/minio/simdjson-go v0.2.1
Optimize decryptObjectInfo (#10726) `decryptObjectInfo` is a significant bottleneck when listing objects. Reduce the allocations for a significant speedup. https://github.com/minio/sio/pull/40 ``` λ benchcmp before.txt after.txt benchmark old ns/op new ns/op delta Benchmark_decryptObjectInfo-32 24260928 808656 -96.67% benchmark old MB/s new MB/s speedup Benchmark_decryptObjectInfo-32 0.04 1.24 31.00x benchmark old allocs new allocs delta Benchmark_decryptObjectInfo-32 75112 48996 -34.77% benchmark old bytes new bytes delta Benchmark_decryptObjectInfo-32 287694772 4228076 -98.53% ```
2020-10-29 12:34:20 -04:00
github.com/minio/sio v0.2.1
add go modules file and start running go 1.11 style builds (#7354)
2019-03-19 16:50:59 -04:00
github.com/mitchellh/go-homedir v1.1.0
Implement oboard diagnostics admin API (#9024) - Implement a graph algorithm to test network bandwidth from every node to every other node - Saturate any network bandwidth adaptively, accounting for slow and fast network capacity - Implement parallel drive OBD tests - Implement a paging mechanism for OBD test to provide periodic updates to client - Implement Sys, Process, Host, Mem OBD Infos
2020-03-27 00:07:39 -04:00
github.com/montanaflynn/stats v0.5.0
update nats.io/jwt dependency (#11066) primarily for a vulnerability report https://nvd.nist.gov/vuln/detail/CVE-2020-26521
2020-12-09 17:30:35 -05:00
github.com/nats-io/nats-server/v2 v2.1.9
use Access() instead of Lstat() for frequent use (#11911) using Lstat() is causing tiny memory allocations, that are usually wasted and never used, instead we can simply uses Access() call that does 0 memory allocations.
2021-03-29 11:07:23 -04:00
github.com/nats-io/nats-streaming-server v0.21.1 // indirect
fix: migrate to go.etcd.io import path (#9987) with the merge of https://github.com/etcd-io/etcd/pull/11823 etcd v3.5.0 will now have a properly imported versioned path this fixes our pending migration to newer repo
2020-07-07 22:04:29 -04:00
github.com/nats-io/nats.go v1.10.0
update nats.io/jwt dependency (#11066) primarily for a vulnerability report https://nvd.nist.gov/vuln/detail/CVE-2020-26521
2020-12-09 17:30:35 -05:00
github.com/nats-io/nkeys v0.2.0 // indirect
use Access() instead of Lstat() for frequent use (#11911) using Lstat() is causing tiny memory allocations, that are usually wasted and never used, instead we can simply uses Access() call that does 0 memory allocations.
2021-03-29 11:07:23 -04:00
github.com/nats-io/stan.go v0.8.3
Use O_DIRECT while writing to disk (#7479) - Use O_DIRECT while writing to disk - Remove MINIO_DRIVE_SYNC option
2019-04-24 00:25:06 -04:00
github.com/ncw/directio v1.0.5
upgrade event notification dependencies (#11095)
2020-12-12 23:31:28 -05:00
github.com/nsqio/go-nsq v1.0.8
update deps to latest for some vulnerable deps (#11080) - github.com/miekg/dns - update elasticsearch library deps to circumvent some aws-sdk-go deps
2020-12-10 16:23:06 -05:00
github.com/olivere/elastic/v7 v7.0.22
fix: readData in bulk call using messagepack byte wrappers (#11228) This PR refactors the way we use buffers for O_DIRECT and to re-use those buffers for messagepack reader writer. After some extensive benchmarking found that not all objects have this benefit, and only objects smaller than 64KiB see this benefit overall. Benefits are seen from almost all objects from 1KiB - 32KiB Beyond this no objects see benefit with bulk call approach as the latency of bytes sent over the wire v/s streaming content directly from disk negate each other with no remarkable benefits. All other optimizations include reuse of msgp.Reader, msgp.Writer using sync.Pool's for all internode calls.
2021-01-07 22:27:31 -05:00
github.com/philhofer/fwd v1.1.1
upgrade event notification dependencies (#11095)
2020-12-12 23:31:28 -05:00
github.com/pierrec/lz4 v2.5.2+incompatible
Support https and basic-auth for elasticsearch notification target (#10332)
2020-08-23 12:43:48 -04:00
github.com/pkg/errors v0.9.1
upgrade event notification dependencies (#11095)
2020-12-12 23:31:28 -05:00
github.com/prometheus/client_golang v1.8.0
Updated Prometheus metrics (#11141) * Add metrics for nodes online and offline * Add cluster capacity metrics * Introduce v2 metrics
2021-01-18 23:35:38 -05:00
github.com/prometheus/client_model v0.2.0
use Access() instead of Lstat() for frequent use (#11911) using Lstat() is causing tiny memory allocations, that are usually wasted and never used, instead we can simply uses Access() call that does 0 memory allocations.
2021-03-29 11:07:23 -04:00
github.com/prometheus/procfs v0.6.0
add go modules file and start running go 1.11 style builds (#7354)
2019-03-19 16:50:59 -04:00
github.com/rjeczalik/notify v0.9.2
simplify context timeout for readiness (#9772) additionally also add CORS support to restrict for specific origin, adds a new config and updated the documentation as well
2020-06-04 17:58:34 -04:00
github.com/rs/cors v1.7.0
update dependenices for relevant projects (#11445) - minio-go -> v7.0.8 - ldap/v3 -> v3.2.4 - reedsolomon -> v1.9.11 - sio-go -> v0.3.1 - msgp -> v1.1.5 - simdjson-go, md5-simd, highwayhash
2021-02-04 16:49:52 -05:00
github.com/secure-io/sio-go v0.3.1
update gopsutil to use the v3 API (#11638)
2021-03-01 03:15:46 -05:00
github.com/shirou/gopsutil/v3 v3.21.1
add data update tracking using bloom filter (#9208) By monitoring PUT/DELETE and heal operations it is possible to track changed paths and keep a bloom filter for this data. This can help prioritize paths to scan. The bloom filter can identify paths that have not changed, and the few collisions will only result in a marginal extra workload. This can be implemented on either a bucket+(1 prefix level) with reasonable performance. The bloom filter is set to have a false positive rate at 1% at 1M entries. A bloom table of this size is about ~2500 bytes when serialized. To not force a full scan of all paths that have changed cycle bloom filters would need to be kept, so we guarantee that dirty paths have been scanned within cycle runs. Until cycle bloom filters have been collected all paths are considered dirty.
2020-04-27 13:06:21 -04:00
github.com/spaolacci/murmur3 v1.1.0 // indirect
Support for remote tier management (#12090) With this change, MinIO's ILM supports transitioning objects to a remote tier. This change includes support for Azure Blob Storage, AWS S3 compatible object storage incl. MinIO and Google Cloud Storage as remote tier storage backends. Some new additions include: - Admin APIs remote tier configuration management - Simple journal to track remote objects to be 'collected' This is used by object API handlers which 'mutate' object versions by overwriting/replacing content (Put/CopyObject) or removing the version itself (e.g DeleteObjectVersion). - Rework of previous ILM transition to fit the new model In the new model, a storage class (a.k.a remote tier) is defined by the 'remote' object storage type (one of s3, azure, GCS), bucket name and a prefix. * Fixed bugs, review comments, and more unit-tests - Leverage inline small object feature - Migrate legacy objects to the latest object format before transitioning - Fix restore to particular version if specified - Extend SharedDataDirCount to handle transitioned and restored objects - Restore-object should accept version-id for version-suspended bucket (#12091) - Check if remote tier creds have sufficient permissions - Bonus minor fixes to existing error messages Co-authored-by: Poorna Krishnamoorthy <poorna@minio.io> Co-authored-by: Krishna Srinivas <krishna@minio.io> Signed-off-by: Harshavardhana <harsha@minio.io>
2021-04-19 13:30:42 -04:00
github.com/spf13/pflag v1.0.5 // indirect
upgrade event notification dependencies (#11095)
2020-12-12 23:31:28 -05:00
github.com/streadway/amqp v1.0.0
fix: merge duplicate keys in post policy (#11843) some SDKs might incorrectly send duplicate entries for keys such as "conditions", Go stdlib unmarshal for JSON does not support duplicate keys - instead skips the first duplicate and only preserves the last entry. This can lead to issues where a policy JSON while being valid might not properly apply the required conditions, allowing situations where POST policy JSON would end up allowing uploads to unauthorized buckets and paths. This PR fixes this properly.
2021-03-21 01:16:30 -04:00
github.com/tidwall/gjson v1.6.8
Add archived parquet as int. package (#9912) Since github.com/minio/parquet-go is archived add it as internal package.
2020-06-25 10:31:16 -04:00
github.com/tidwall/sjson v1.0.4
Update yaml files to latest version RELEASE.2021-02-07T01-31-02Z
2021-02-06 20:53:27 -05:00
github.com/tinylib/msgp v1.1.3
add go modules file and start running go 1.11 style builds (#7354)
2019-03-19 16:50:59 -04:00
github.com/valyala/tcplisten v0.0.0-20161114210144-ceec8f93295a
fix: web handlers to enforce replication (#10249) This PR also preserves source ETag for replication
2020-08-12 20:32:24 -04:00
github.com/willf/bitset v1.1.11 // indirect
add data update tracking using bloom filter (#9208) By monitoring PUT/DELETE and heal operations it is possible to track changed paths and keep a bloom filter for this data. This can help prioritize paths to scan. The bloom filter can identify paths that have not changed, and the few collisions will only result in a marginal extra workload. This can be implemented on either a bucket+(1 prefix level) with reasonable performance. The bloom filter is set to have a false positive rate at 1% at 1M entries. A bloom table of this size is about ~2500 bytes when serialized. To not force a full scan of all paths that have changed cycle bloom filters would need to be kept, so we guarantee that dirty paths have been scanned within cycle runs. Until cycle bloom filters have been collected all paths are considered dirty.
2020-04-27 13:06:21 -04:00
github.com/willf/bloom v2.0.3+incompatible
added support for SASL/SCRAM on Kafka bucket notifications. (#9168) fixes #9167
2020-03-20 14:10:27 -04:00
github.com/xdg/scram v0.0.0-20180814205039-7eeb5667e42c
fix: etcd import paths again depend on v3.4.14 release (#11020) Due to botched upstream renames of project repositories and incomplete migration to go.mod support, our current dependency version of `go.mod` had bugs i.e it was using commits from master branch which didn't have the required fixes present in release-3.4 branches which leads to some rare bugs https://github.com/etcd-io/etcd/pull/11477 provides a workaround for now and we should migrate to this. release-3.5 eventually claims to fix all of this properly until then we cannot use /v3 import right now
2020-12-03 14:35:18 -05:00
go.etcd.io/etcd v0.0.0-20201125193152-8a03d2e9614b
Support for remote tier management (#12090) With this change, MinIO's ILM supports transitioning objects to a remote tier. This change includes support for Azure Blob Storage, AWS S3 compatible object storage incl. MinIO and Google Cloud Storage as remote tier storage backends. Some new additions include: - Admin APIs remote tier configuration management - Simple journal to track remote objects to be 'collected' This is used by object API handlers which 'mutate' object versions by overwriting/replacing content (Put/CopyObject) or removing the version itself (e.g DeleteObjectVersion). - Rework of previous ILM transition to fit the new model In the new model, a storage class (a.k.a remote tier) is defined by the 'remote' object storage type (one of s3, azure, GCS), bucket name and a prefix. * Fixed bugs, review comments, and more unit-tests - Leverage inline small object feature - Migrate legacy objects to the latest object format before transitioning - Fix restore to particular version if specified - Extend SharedDataDirCount to handle transitioned and restored objects - Restore-object should accept version-id for version-suspended bucket (#12091) - Check if remote tier creds have sufficient permissions - Bonus minor fixes to existing error messages Co-authored-by: Poorna Krishnamoorthy <poorna@minio.io> Co-authored-by: Krishna Srinivas <krishna@minio.io> Signed-off-by: Harshavardhana <harsha@minio.io>
2021-04-19 13:30:42 -04:00
go.uber.org/zap v1.14.1
kms: encrypt IAM/config data with the KMS (#12041) This commit changes the config/IAM encryption process. Instead of encrypting config data (users, policies etc.) with the root credentials MinIO now encrypts this data with a KMS - if configured. Therefore, this PR moves the MinIO-KMS configuration (via env. variables) to a "top-level" configuration. The KMS configuration cannot be stored in the config file since it is used to decrypt the config file in the first place. As a consequence, this commit also removes support for Hashicorp Vault - which has been deprecated anyway. Signed-off-by: Andreas Auernhammer <aead@mail.de>
2021-04-22 11:45:30 -04:00
golang.org/x/crypto v0.0.0-20210415154028-4f45737414dc
Support for remote tier management (#12090) With this change, MinIO's ILM supports transitioning objects to a remote tier. This change includes support for Azure Blob Storage, AWS S3 compatible object storage incl. MinIO and Google Cloud Storage as remote tier storage backends. Some new additions include: - Admin APIs remote tier configuration management - Simple journal to track remote objects to be 'collected' This is used by object API handlers which 'mutate' object versions by overwriting/replacing content (Put/CopyObject) or removing the version itself (e.g DeleteObjectVersion). - Rework of previous ILM transition to fit the new model In the new model, a storage class (a.k.a remote tier) is defined by the 'remote' object storage type (one of s3, azure, GCS), bucket name and a prefix. * Fixed bugs, review comments, and more unit-tests - Leverage inline small object feature - Migrate legacy objects to the latest object format before transitioning - Fix restore to particular version if specified - Extend SharedDataDirCount to handle transitioned and restored objects - Restore-object should accept version-id for version-suspended bucket (#12091) - Check if remote tier creds have sufficient permissions - Bonus minor fixes to existing error messages Co-authored-by: Poorna Krishnamoorthy <poorna@minio.io> Co-authored-by: Krishna Srinivas <krishna@minio.io> Signed-off-by: Harshavardhana <harsha@minio.io>
2021-04-19 13:30:42 -04:00
golang.org/x/mod v0.4.0 // indirect
kms: encrypt IAM/config data with the KMS (#12041) This commit changes the config/IAM encryption process. Instead of encrypting config data (users, policies etc.) with the root credentials MinIO now encrypts this data with a KMS - if configured. Therefore, this PR moves the MinIO-KMS configuration (via env. variables) to a "top-level" configuration. The KMS configuration cannot be stored in the config file since it is used to decrypt the config file in the first place. As a consequence, this commit also removes support for Hashicorp Vault - which has been deprecated anyway. Signed-off-by: Andreas Auernhammer <aead@mail.de>
2021-04-22 11:45:30 -04:00
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110
use Access() instead of Lstat() for frequent use (#11911) using Lstat() is causing tiny memory allocations, that are usually wasted and never used, instead we can simply uses Access() call that does 0 memory allocations.
2021-03-29 11:07:23 -04:00
golang.org/x/sys v0.0.0-20210225134936-a50acf3fe073
Support for remote tier management (#12090) With this change, MinIO's ILM supports transitioning objects to a remote tier. This change includes support for Azure Blob Storage, AWS S3 compatible object storage incl. MinIO and Google Cloud Storage as remote tier storage backends. Some new additions include: - Admin APIs remote tier configuration management - Simple journal to track remote objects to be 'collected' This is used by object API handlers which 'mutate' object versions by overwriting/replacing content (Put/CopyObject) or removing the version itself (e.g DeleteObjectVersion). - Rework of previous ILM transition to fit the new model In the new model, a storage class (a.k.a remote tier) is defined by the 'remote' object storage type (one of s3, azure, GCS), bucket name and a prefix. * Fixed bugs, review comments, and more unit-tests - Leverage inline small object feature - Migrate legacy objects to the latest object format before transitioning - Fix restore to particular version if specified - Extend SharedDataDirCount to handle transitioned and restored objects - Restore-object should accept version-id for version-suspended bucket (#12091) - Check if remote tier creds have sufficient permissions - Bonus minor fixes to existing error messages Co-authored-by: Poorna Krishnamoorthy <poorna@minio.io> Co-authored-by: Krishna Srinivas <krishna@minio.io> Signed-off-by: Harshavardhana <harsha@minio.io>
2021-04-19 13:30:42 -04:00
golang.org/x/time v0.0.0-20200416051211-89c76fbcd5d1 // indirect
golang.org/x/tools v0.1.1-0.20210201201750-4d4ee958a9b7 // indirect
google.golang.org/api v0.25.0
google.golang.org/genproto v0.0.0-20200527145253-8367513e4ece // indirect
upgrade event notification dependencies (#11095)
2020-12-12 23:31:28 -05:00
gopkg.in/yaml.v2 v2.3.0
Support for remote tier management (#12090) With this change, MinIO's ILM supports transitioning objects to a remote tier. This change includes support for Azure Blob Storage, AWS S3 compatible object storage incl. MinIO and Google Cloud Storage as remote tier storage backends. Some new additions include: - Admin APIs remote tier configuration management - Simple journal to track remote objects to be 'collected' This is used by object API handlers which 'mutate' object versions by overwriting/replacing content (Put/CopyObject) or removing the version itself (e.g DeleteObjectVersion). - Rework of previous ILM transition to fit the new model In the new model, a storage class (a.k.a remote tier) is defined by the 'remote' object storage type (one of s3, azure, GCS), bucket name and a prefix. * Fixed bugs, review comments, and more unit-tests - Leverage inline small object feature - Migrate legacy objects to the latest object format before transitioning - Fix restore to particular version if specified - Extend SharedDataDirCount to handle transitioned and restored objects - Restore-object should accept version-id for version-suspended bucket (#12091) - Check if remote tier creds have sufficient permissions - Bonus minor fixes to existing error messages Co-authored-by: Poorna Krishnamoorthy <poorna@minio.io> Co-authored-by: Krishna Srinivas <krishna@minio.io> Signed-off-by: Harshavardhana <harsha@minio.io>
2021-04-19 13:30:42 -04:00
honnef.co/go/tools v0.0.1-2020.1.6 // indirect
sigs.k8s.io/yaml v1.2.0 // indirect
Update yaml files to latest version RELEASE.2021-01-30T00-20-58Z
2021-01-29 19:45:11 -05:00
)
Reference in New Issue Copy Permalink