Commit Graph

7696 Commits

Author SHA1 Message Date
Harshavardhana ce93b2681b
fix: re-use er.getDisks() properly in certain calls (#11043) 2020-12-07 10:04:07 -08:00
Harshavardhana 8d036ed6d8
fix: allow sub-admin to modify password for other users (#11039)
fixes #11037
2020-12-06 20:36:34 -08:00
Harshavardhana 9c53cc1b83
fix: heal multiple buckets in bulk (#11029)
makes server startup, orders of magnitude
faster with large number of buckets
2020-12-05 13:00:44 -08:00
Harshavardhana 3514e89eb3
support envs as well for new crawler sub-system (#11033) 2020-12-04 21:54:24 -08:00
Nitish Tiwari 6ff12f5f01
Add the dashboard json file (#11028)
This will allow users to contribute to the dashboard as needed.
2020-12-04 16:27:41 -08:00
Harshavardhana ee2a436a5b
fix: release locks if the client timedout (#11030)
situations where client indeed timedout there was
a potential to falsely think that lock is still
active.
2020-12-04 11:33:56 -08:00
Klaus Post a896125490
Add crawler delay config + dynamic config values (#11018) 2020-12-04 09:32:35 -08:00
Harshavardhana e083471ec4
use argon2 with sync.Pool for better memory management (#11019) 2020-12-03 19:23:19 -08:00
Nitish Tiwari de9b64834e
fix: update grafana dashboard docs (#11023)
Refer to the official Grafana dashboard
2020-12-03 15:56:15 -08:00
Harshavardhana 919441d9c4 update gocredits with new updated dependencies 2020-12-03 11:39:10 -08:00
Harshavardhana 80d31113e5
fix: etcd import paths again depend on v3.4.14 release (#11020)
Due to botched upstream renames of project repositories
and incomplete migration to go.mod support, our current
dependency version of `go.mod` had bugs i.e it was
using commits from master branch which didn't have
the required fixes present in release-3.4 branches

which leads to some rare bugs

https://github.com/etcd-io/etcd/pull/11477 provides
a workaround for now and we should migrate to this.

release-3.5 eventually claims to fix all of this
properly until then we cannot use /v3 import right now
2020-12-03 11:35:18 -08:00
Ritesh H Shukla 7e2b79984e
Stream bucket bandwidth measurements (#11014) 2020-12-03 11:34:42 -08:00
Minio Trusted d54cf77356 Update yaml files to latest version RELEASE.2020-12-03T05-49-24Z 2020-12-03 06:05:56 +00:00
Harshavardhana 951b6b203b skip metacache entries healing to speed up startup 2020-12-02 21:30:54 -08:00
Harshavardhana 44e23b7f4f fix: startup being slow - wait only if IOCount > 0 2020-12-02 21:06:17 -08:00
Harshavardhana c22a387695 fix: building docker images 2020-12-02 17:34:42 -08:00
Minio Trusted 1ab4d6a6aa Update yaml files to latest version RELEASE.2020-12-03T00-03-10Z 2020-12-03 00:24:16 +00:00
Harshavardhana 96c0ce1f0c
add support for tuning healing to make healing more aggressive (#11003)
supports `mc admin config set <alias> heal sleep=100ms` to
enable more aggressive healing under certain times.

also optimize some areas that were doing extra checks than
necessary when bitrotscan was enabled, avoid double sleeps
make healing more predictable.

fixes #10497
2020-12-02 11:12:00 -08:00
Anis Elleuch fe11e9047d
deprecate CommonName from TLS docs (#11017)
CommonName is not supported anymore in Go 1.15

fix the TLS documentation to use subjAltNames
2020-12-02 10:18:39 -08:00
Harshavardhana ce0e17b62b
fix: load certs on windows from registry (#11016)
fixes #11007
2020-12-02 02:23:51 -08:00
ebozduman 303be1866d
Adds "x-amz-usr-agent" and "x-id" params to be used in authentication of presignedURL (#10792) 2020-12-02 02:02:49 -08:00
Harshavardhana a6113b2315
update minio-go version v7.0.6 (#11012) 2020-12-01 19:15:38 -08:00
Sudarshan (Sid) 3ca046b408
Added set keyword in the command set to enable encryption on buckets (#11010) 2020-12-01 16:00:49 -08:00
Harshavardhana 4ec45753e6 rename server sets to server pools 2020-12-01 13:50:33 -08:00
Klaus Post e6ea5c2703
crawler: Missing folder heal check per set (#10876) 2020-12-01 12:07:39 -08:00
Harshavardhana 790833f3b2 Revert "Support variable server sets (#10314)"
This reverts commit aabf053d2f.
2020-12-01 12:02:29 -08:00
Klaus Post 02aecb2fc1
select: Check if CSV is valid utf8 (#10991)
Check if first block of data is valid utf8.

Fixes #10970
2020-12-01 09:09:06 -08:00
Harshavardhana 7cbca43eb1
fix: allow admins to create users (#11005)
PR #10978 introduced a regression, root
credential should be allowed to create users
2020-11-30 21:53:23 -08:00
Poorna Krishnamoorthy 2f564437ae
Disallow writeback caching with cache_after (#11002)
fixes #10974
2020-11-30 20:53:27 -08:00
Harshavardhana ae4ded7fd1
fix: s3select tests with new minio-py SDK (#10995) 2020-11-29 13:05:37 -08:00
Harshavardhana bdd094bc39
fix: avoid sending errors on missing objects on locked buckets (#10994)
make sure multi-object delete returned errors that are AWS S3 compatible
2020-11-28 21:15:45 -08:00
Harshavardhana e6fa410778
fix: allow accountInfo, addUser and getUserInfo implicit (#10978)
- accountInfo API that returns information about
  user, access to buckets and the size per bucket
- addUser - user is allowed to change their secretKey
- getUserInfo - returns user info if the incoming
  is the same user requesting their information
2020-11-27 17:23:57 -08:00
Harshavardhana 350c5ff8f8
fix: update RPM spec using rpmbuild (#10979)
To test this PR

> rpmbuild --undefine=_disable_source_fetch -ba minio.spec
2020-11-27 13:36:02 -08:00
Klaus Post f139a19238
Upgrade compress and pgzip package (#10992)
Should provide faster decompression for s3 select and other places where it is used.
2020-11-27 10:10:15 -08:00
Harshavardhana e90efd73a2 add docker ubi buildx script 2020-11-26 21:41:05 -08:00
Harshavardhana 81c907b4bf
fix: docker buildx support for multiplatform build (#10983) 2020-11-26 09:47:30 -08:00
Nitish Tiwari ab49471f33
Add Dockerfile based on Red Hat UBI (#10958)
See https://connect.redhat.com/zones/containers/container-certification-policy-guide
for details

Co-authored-by: Harshavardhana <harsha@minio.io>
2020-11-26 15:52:22 +05:30
Harshavardhana aabf053d2f
Support variable server sets (#10314) 2020-11-25 16:28:47 -08:00
Minio Trusted f839bb5a0a Update yaml files to latest version RELEASE.2020-11-25T22-36-25Z 2020-11-25 22:53:02 +00:00
Anis Elleuch 91130e884b
Avoid sending errors in gob in storage requests (#10977) 2020-11-25 12:42:48 -08:00
Poorna Krishnamoorthy 2ff655a745
Refactor replication, ILM handling in DELETE API (#10945) 2020-11-25 11:24:50 -08:00
Klaus Post 0422eda6a2
metacache: Always close block writer (#10973)
In some cases a writer could be left behind unclosed, leaking compression blocks.

Always close and set compression concurrency to 2 which should be fine to keep up.
2020-11-25 09:37:30 -08:00
Harshavardhana 31e6f60847
fix: improve error handling in metacache (#10965) 2020-11-25 01:11:22 -08:00
Poorna Krishnamoorthy 7742238495
fix: marshaling stack overflow in noncurrentversion lifecycle config (#10971) 2020-11-24 20:43:11 -08:00
Poorna Krishnamoorthy 3ad41fe89d
Add admin API to edit remote bucket target credentials (#10848) 2020-11-24 19:09:05 -08:00
Harshavardhana f96ed3769f run go mod tidy 2020-11-24 12:04:15 -08:00
Klaus Post a75fafdbe2
Remove msgp workaround (#10964)
The error in `github.com/philhofer/fwd` was quickly fixed through 
https://github.com/philhofer/fwd/pull/22 - update the dependency 
and remove the workaround.
2020-11-24 11:58:10 -08:00
Klaus Post a58b7874ef
Temporary workaround for msgp skipping (#10960)
Due to https://github.com/philhofer/fwd/issues/20 when skipping a metadata entry that is >2048 bytes and the buffer is full (2048 bytes) the skip will fail with `io.ErrNoProgress`.

Enlarge the buffer so we temporarily make this much more unlikely.

If it still happens we will have to rewrite the skips to reads.

Fixes #10959
2020-11-23 18:51:59 -08:00
Harshavardhana 6990de9c94
fix: dangling object delete shall return object doesn't exist (#10961)
dangling object when deleted means object doesn't exist
anymore, so we should return appropriate errors, this
allows crawler heal to ensure that it removes the tracker
for dangling objects.
2020-11-23 18:50:53 -08:00
Anis Elleuch 75a8e81f8f
azure: Specify different Azure storage in the shell env (#10943)
AZURE_STORAGE_ACCOUNT and AZURE_STORAGE_KEY are used in 
azure CLI to specify the azure blob storage access & secret keys. With this commit, 
it is possible to set them if you want the gateway's own credentials to be
different from the Azure blob credentials.

Co-authored-by: Harshavardhana <harsha@minio.io>
2020-11-23 16:45:56 -08:00