Commit Graph

6295 Commits

Author SHA1 Message Date
Harshavardhana
686d4656de
fix: set appropriate defaults when new keys added (#8795)
A new key was added in identity_openid recently
required explicitly for client to set the optional
value without that it would be empty, handle this
appropriately.

Fixes #8787
2020-01-10 16:57:18 -08:00
Harshavardhana
5aa5dcdc6d
lock: improve locker initialization at init (#8776)
Use reference format to initialize lockers
during startup, also handle `nil` for NetLocker
in dsync and remove *errorLocker* implementation

Add further tuning parameters such as

 - DialTimeout is now 15 seconds from 30 seconds
 - KeepAliveTimeout is not 20 seconds, 5 seconds
   more than default 15 seconds
 - ResponseHeaderTimeout to 10 seconds
 - ExpectContinueTimeout is reduced to 3 seconds
 - DualStack is enabled by default remove setting
   it to `true`
 - Reduce IdleConnTimeout to 30 seconds from
   1 minute to avoid idleConn build up

Fixes #8773
2020-01-10 02:35:06 -08:00
Harshavardhana
0a70bc24ac
Disallow only policy statements which are exactly same (#8785) 2020-01-09 19:29:57 -08:00
Kevin Humphreys
656146b699 doc: Prometheus metrics name fix (#8774)
changed docs to reflect proper Prometheus metrics
2020-01-09 18:36:58 -08:00
Harshavardhana
5e40b9a563
fix: docs for live/ready check implementation details 2020-01-09 18:29:24 -08:00
Joe Adams
89d1221217 Fix typo in prometheus monitoring docs (#8780) 2020-01-09 09:08:41 -08:00
Praveen raj Mani
4cd1bbb50a This PR fixes two things (#8772)
- Stop spawning store replay routines when testing the notification targets
- Properly honor the target.Close() to clean the resources used

Fixes #8707

Co-authored-by: Harshavardhana <harsha@minio.io>
2020-01-09 19:45:44 +05:30
Harshavardhana
c2cde6beb5
policy: Allow duplicate statements with different effects (#8775)
This allows "Allow" and "Deny" conflicting statements,
where we evaluate to implicit "Deny".
2020-01-08 23:00:54 -08:00
Harshavardhana
abc1c1070a Add custom policy claim name (#8764)
In certain organizations policy claim names
can be not just 'policy' but also things like
'roles', the value of this field might also
be *string* or *[]string* support this as well

In this PR we are still not supporting multiple
policies per STS account which will require a
more comprehensive change.
2020-01-08 17:21:58 -08:00
poornas
fd56aa42a6 Fix error message wording for PutObjectLockConfig (#8759)
Co-Authored-By: kannappanr <30541348+kannappanr@users.noreply.github.com>
Co-authored-by: Harshavardhana <harsha@minio.io>
2020-01-08 15:36:23 -08:00
Klaus Post
3d318bae76 init: Use constant time retries (#8769)
Exponential backoff does not seem like a good fit for
this function since we can expect a few roundtrips on
initial startup.

This retry loop get slow pretty quickly with initial
wait being 1 second and each try being double the
wait until 30 seconds is reached.

Instead simply try 2 times per second.
2020-01-08 13:37:34 -08:00
Harshavardhana
aa2e89bfe3 Use jsoniter whenever applicable instead of encoding/json (#8766)
This PR adds jsoniter package to replace encoding/json
in places where faster json unmarshal is necessary
whenever input JSON is large enough.

Some benchmarking comparison between jsoniter and enconding/json

benchmark                            old MB/s     new MB/s     speedup
BenchmarkParseUnmarshal/N10-4        110.02       331.17       3.01x
BenchmarkParseUnmarshal/N100-4       125.74       524.09       4.17x
BenchmarkParseUnmarshal/N500-4       131.68       542.60       4.12x
BenchmarkParseUnmarshal/N1000-4      133.93       514.88       3.84x
BenchmarkParseUnmarshal/N5000-4      122.10       415.36       3.40x
BenchmarkParseUnmarshal/N10000-4     132.13       403.90       3.06x
2020-01-08 17:01:42 +05:30
Harshavardhana
60813bef29
Allow proper setCount SLAs across zones (#8752)
Fixes scenario where zones are appropriately
handled, along with supporting overriding set
count. The new fix also ensures that we handle
the various setup types properly.

Update documentation to properly indicate the
behavior.

Fixes #8750

Co-authored-by: Nitish Tiwari <nitish@minio.io>
2020-01-07 09:13:44 -08:00
Harshavardhana
b123be5612 fix: browser should listBuckets from etcd in global federation (#8760) 2020-01-07 09:03:00 +05:30
Harshavardhana
933c60bc3a Add crypto context errors (#8740)
Currently when connections to vault fail, client
perpetually retries this leads to assumptions that
the server has issues and masks the problem.

Re-purpose *crypto.Error* type to send appropriate
errors back to the client.
2020-01-06 16:15:22 -08:00
ebozduman
796cca4166 Creates zipped files with correct mod times for objects (#8693) 2020-01-06 12:43:00 -08:00
Klaus Post
fe379f9428 Copy metadata on update (#8755)
Fixes #8706

Co-authored-by: Harshavardhana <harsha@minio.io>
2020-01-06 10:15:44 -08:00
Harshavardhana
ae0b165431 fix: --anonymous flag shouldn't print any keys (#8753)
Fixes #8744
2020-01-06 22:12:47 +05:30
Harshavardhana
7a06e158f1
Add docs for sysctl tuning parameters (#8749) 2020-01-06 00:30:04 -08:00
Harshavardhana
5eab3db344 Fix doc reference for prometheus (#8748) 2020-01-05 13:44:39 -08:00
Brian Candler
9f44fcd540 Clarify behaviour of erasure coding sets (#8745) 2020-01-05 13:00:11 -08:00
Harshavardhana
b9b2b37366 Fix SECURITY.md format and newlines 2020-01-03 17:49:34 -08:00
George Xie
7f31d933a8 fixes some typos, for CREDITS change (#8743) 2020-01-03 17:49:01 -08:00
Harshavardhana
6695fd6a61
Add more context aware error for policy parsing errors (#8726)
In existing functionality we simply return a generic
error such as "MalformedPolicy" which indicates just
a generic string "invalid resource" which is not very
meaningful when there might be multiple types of errors
during policy parsing. This PR ensures that we send
these errors back to client to indicate the actual
error, brings in two concrete types such as

 - iampolicy.Error
 - policy.Error

Refer #8202
2020-01-03 11:28:52 -08:00
Minio Trusted
84e55e2e6f Update yaml files to latest version RELEASE.2020-01-03T19-12-21Z 2020-01-03 19:20:28 +00:00
Harshavardhana
b00cda8ad4 Avoid running lock maintenance from all nodes (#8737)
Co-Authored-By: Krishnan Parthasarathi <krisis@users.noreply.github.com>
2020-01-03 23:11:07 +05:30
Anis Elleuch
d861edfc00 xl: Print the correct err msg when access to the backend is forbidden (#8735)
minio server /data{1..4} shows an error about inability to bind a port, though
the real problem is /data{1..4} cannot be created because of the lack of
permissions.

This commit fix the behavior.
2020-01-03 21:15:26 +05:30
Harshavardhana
dd311623df Update design doc with zone implementation details (#8738)
Fixes #8719
2020-01-02 16:46:16 -08:00
Harshavardhana
cb935980a5 Fix version to be release-tag (#8730) 2020-01-02 20:18:32 +05:30
Praveen raj Mani
157721f694 Fix readiness to return 200 for read-only mode (#8728)
- We should declare a cluster ready even if read quorum is achieved (atleast n/2 disks are online).
- Such that, all the zones should have enough read quorum. Thus making the cluster ready for reads.
2020-01-02 05:05:01 -08:00
Nitish Tiwari
97d799b9f0 doc: Update Kubernetes examples to use Readiness Checks (#8727) 2020-01-01 08:25:09 -08:00
Harshavardhana
0b7bd024fb Fix dependencies graph for minio source compilation (#8717)
We had messy cyclical dependency problem with `mc`
due to dependencies in pkg/console, moved the pkg/console
to minio for more control and also to avoid any further
cyclical dependencies of `mc` clobbering up the
dependencies on server.

Fixes #8659
2019-12-31 09:36:13 +05:30
Harshavardhana
3af70b36fd Disallow creating buckets even with different domains (#8716)
If two distinct clusters are started with different domains
along with single common domain, this situation was leading
to conflicting buckets getting created on different clusters

To avoid this do not prematurely error out if the key has no
entries, let the caller decide on which entry matches and
which entry is valid. This allows support for MINIO_DOMAIN
with one common domain, but each cluster may have their own
domains.

Fixes #8705
2019-12-30 17:11:47 -08:00
Minio Trusted
8eba97da74 Update yaml files to latest version RELEASE.2019-12-30T05-45-39Z 2019-12-30 05:56:46 +00:00
Harshavardhana
669c9da85d Disable federated buckets when etcd is namespaced (#8709)
This is to ensure that when we have multiple tenants
deployed all sharing the same etcd for global bucket
should avoid listing each others buckets, this leads
to information leak which should be avoided unless
etcd is not namespaced for IAM assets in which case
it can be assumed that its a federated setup.

Federated setup and namespaced IAM assets on etcd
is not supported since namespacing is only useful
when you wish to separate the tenants as isolated
instances of MinIO.

This PR allows a new type of behavior, primarily
driven by the usecase of m3(mkube) multi-tenant
deployments with global bucket support.
2019-12-29 08:56:45 -08:00
Praveen raj Mani
5d09233115 Fix Readiness check (#8681)
- Remove goroutine-check in Readiness check
- Bring in quorum check for readiness

Fixes #8385

Co-authored-by: Harshavardhana <harsha@minio.io>
2019-12-28 22:24:43 +05:30
Haines Chan
b2044dd22f Fix typo in comments (#8704) 2019-12-27 15:51:32 +05:30
Anis Elleuch
c31e67dcce Better error when the server is unable to write in the backend (#8697) 2019-12-25 22:05:54 -08:00
kannappanr
cd59a945d8
Fix: Policy action name for Get/Put BucketLifeCycle (#8698)
S3:GetBucketLifeCycle should be changed to S3:GetLifeCycleConfiguration
S3:PutBucketLifeCycle should be changed to S3:PutLifeCycleConfiguration
2019-12-25 22:04:46 -08:00
Harshavardhana
d4a390028a node 6.x is EOL'ed upgrade to latest stable (#8702) 2019-12-26 08:27:35 +05:30
Bruce Wang
c476b27a65 Comment typo "index max" to "index map" (#8700) 2019-12-24 21:57:43 -08:00
kannappanr
eaa838a63f mint: Change aws cli version (#8699) 2019-12-24 19:58:19 -08:00
Minio Trusted
4364102363 Update yaml files to latest version RELEASE.2019-12-24T23-04-45Z 2019-12-24 23:15:44 +00:00
Harshavardhana
99ad445260
Avoid double for loops in notification init (#8691) 2019-12-24 13:49:48 -08:00
Harshavardhana
54431b3953 Change replica set detection for localhost on single endpoint (#8692) 2019-12-24 11:31:32 -08:00
Harshavardhana
f68a7005c0 Improve disk formatting stage for large disk sets (#8690) 2019-12-23 16:31:03 -08:00
Harshavardhana
725172e13b
fix: Do not need safe-mode for unreachable targets upon restart (#8686) 2019-12-21 22:35:50 -08:00
Harshavardhana
0f092c2be8 Add env values and description for config sub-systems (#8684)
The output is directly borrowed from `mc` command output
2019-12-21 22:52:16 +05:30
Harshavardhana
a3c8ef79a4 fix: remove extra newline from GetConfig() output (#8678) 2019-12-20 14:47:14 -08:00
Aditya Manthramurthy
01468d5a75 Fix user and policy deletion IAM commands (#8683) 2019-12-20 14:42:08 -08:00