Commit Graph

6298 Commits

Author SHA1 Message Date
Etienne 5f34b5e6a5 Update README.md (#8788) 2020-01-10 21:08:09 -08:00
poornas 9199033db7 Set X-Cache and X-Cache-Lookup headers for cache (#8794)
X-Cache sets cache status of HIT if object is
served from the disk cache, or MISS otherwise.
X-Cache-Lookup is set to HIT if object was found
in the cache even if not served (for e.g. if cache
 entry was invalidated by ETag verification)
2020-01-10 20:21:13 -08:00
Klaus Post 2bf6cf0e15 Enable multiple concurrent profile types (#8792) 2020-01-10 17:19:58 -08:00
Harshavardhana 686d4656de
fix: set appropriate defaults when new keys added (#8795)
A new key was added in identity_openid recently
required explicitly for client to set the optional
value without that it would be empty, handle this
appropriately.

Fixes #8787
2020-01-10 16:57:18 -08:00
Harshavardhana 5aa5dcdc6d
lock: improve locker initialization at init (#8776)
Use reference format to initialize lockers
during startup, also handle `nil` for NetLocker
in dsync and remove *errorLocker* implementation

Add further tuning parameters such as

 - DialTimeout is now 15 seconds from 30 seconds
 - KeepAliveTimeout is not 20 seconds, 5 seconds
   more than default 15 seconds
 - ResponseHeaderTimeout to 10 seconds
 - ExpectContinueTimeout is reduced to 3 seconds
 - DualStack is enabled by default remove setting
   it to `true`
 - Reduce IdleConnTimeout to 30 seconds from
   1 minute to avoid idleConn build up

Fixes #8773
2020-01-10 02:35:06 -08:00
Harshavardhana 0a70bc24ac
Disallow only policy statements which are exactly same (#8785) 2020-01-09 19:29:57 -08:00
Kevin Humphreys 656146b699 doc: Prometheus metrics name fix (#8774)
changed docs to reflect proper Prometheus metrics
2020-01-09 18:36:58 -08:00
Harshavardhana 5e40b9a563
fix: docs for live/ready check implementation details 2020-01-09 18:29:24 -08:00
Joe Adams 89d1221217 Fix typo in prometheus monitoring docs (#8780) 2020-01-09 09:08:41 -08:00
Praveen raj Mani 4cd1bbb50a This PR fixes two things (#8772)
- Stop spawning store replay routines when testing the notification targets
- Properly honor the target.Close() to clean the resources used

Fixes #8707

Co-authored-by: Harshavardhana <harsha@minio.io>
2020-01-09 19:45:44 +05:30
Harshavardhana c2cde6beb5
policy: Allow duplicate statements with different effects (#8775)
This allows "Allow" and "Deny" conflicting statements,
where we evaluate to implicit "Deny".
2020-01-08 23:00:54 -08:00
Harshavardhana abc1c1070a Add custom policy claim name (#8764)
In certain organizations policy claim names
can be not just 'policy' but also things like
'roles', the value of this field might also
be *string* or *[]string* support this as well

In this PR we are still not supporting multiple
policies per STS account which will require a
more comprehensive change.
2020-01-08 17:21:58 -08:00
poornas fd56aa42a6 Fix error message wording for PutObjectLockConfig (#8759)
Co-Authored-By: kannappanr <30541348+kannappanr@users.noreply.github.com>
Co-authored-by: Harshavardhana <harsha@minio.io>
2020-01-08 15:36:23 -08:00
Klaus Post 3d318bae76 init: Use constant time retries (#8769)
Exponential backoff does not seem like a good fit for
this function since we can expect a few roundtrips on
initial startup.

This retry loop get slow pretty quickly with initial
wait being 1 second and each try being double the
wait until 30 seconds is reached.

Instead simply try 2 times per second.
2020-01-08 13:37:34 -08:00
Harshavardhana aa2e89bfe3 Use jsoniter whenever applicable instead of encoding/json (#8766)
This PR adds jsoniter package to replace encoding/json
in places where faster json unmarshal is necessary
whenever input JSON is large enough.

Some benchmarking comparison between jsoniter and enconding/json

benchmark                            old MB/s     new MB/s     speedup
BenchmarkParseUnmarshal/N10-4        110.02       331.17       3.01x
BenchmarkParseUnmarshal/N100-4       125.74       524.09       4.17x
BenchmarkParseUnmarshal/N500-4       131.68       542.60       4.12x
BenchmarkParseUnmarshal/N1000-4      133.93       514.88       3.84x
BenchmarkParseUnmarshal/N5000-4      122.10       415.36       3.40x
BenchmarkParseUnmarshal/N10000-4     132.13       403.90       3.06x
2020-01-08 17:01:42 +05:30
Harshavardhana 60813bef29
Allow proper setCount SLAs across zones (#8752)
Fixes scenario where zones are appropriately
handled, along with supporting overriding set
count. The new fix also ensures that we handle
the various setup types properly.

Update documentation to properly indicate the
behavior.

Fixes #8750

Co-authored-by: Nitish Tiwari <nitish@minio.io>
2020-01-07 09:13:44 -08:00
Harshavardhana b123be5612 fix: browser should listBuckets from etcd in global federation (#8760) 2020-01-07 09:03:00 +05:30
Harshavardhana 933c60bc3a Add crypto context errors (#8740)
Currently when connections to vault fail, client
perpetually retries this leads to assumptions that
the server has issues and masks the problem.

Re-purpose *crypto.Error* type to send appropriate
errors back to the client.
2020-01-06 16:15:22 -08:00
ebozduman 796cca4166 Creates zipped files with correct mod times for objects (#8693) 2020-01-06 12:43:00 -08:00
Klaus Post fe379f9428 Copy metadata on update (#8755)
Fixes #8706

Co-authored-by: Harshavardhana <harsha@minio.io>
2020-01-06 10:15:44 -08:00
Harshavardhana ae0b165431 fix: --anonymous flag shouldn't print any keys (#8753)
Fixes #8744
2020-01-06 22:12:47 +05:30
Harshavardhana 7a06e158f1
Add docs for sysctl tuning parameters (#8749) 2020-01-06 00:30:04 -08:00
Harshavardhana 5eab3db344 Fix doc reference for prometheus (#8748) 2020-01-05 13:44:39 -08:00
Brian Candler 9f44fcd540 Clarify behaviour of erasure coding sets (#8745) 2020-01-05 13:00:11 -08:00
Harshavardhana b9b2b37366 Fix SECURITY.md format and newlines 2020-01-03 17:49:34 -08:00
George Xie 7f31d933a8 fixes some typos, for CREDITS change (#8743) 2020-01-03 17:49:01 -08:00
Harshavardhana 6695fd6a61
Add more context aware error for policy parsing errors (#8726)
In existing functionality we simply return a generic
error such as "MalformedPolicy" which indicates just
a generic string "invalid resource" which is not very
meaningful when there might be multiple types of errors
during policy parsing. This PR ensures that we send
these errors back to client to indicate the actual
error, brings in two concrete types such as

 - iampolicy.Error
 - policy.Error

Refer #8202
2020-01-03 11:28:52 -08:00
Minio Trusted 84e55e2e6f Update yaml files to latest version RELEASE.2020-01-03T19-12-21Z 2020-01-03 19:20:28 +00:00
Harshavardhana b00cda8ad4 Avoid running lock maintenance from all nodes (#8737)
Co-Authored-By: Krishnan Parthasarathi <krisis@users.noreply.github.com>
2020-01-03 23:11:07 +05:30
Anis Elleuch d861edfc00 xl: Print the correct err msg when access to the backend is forbidden (#8735)
minio server /data{1..4} shows an error about inability to bind a port, though
the real problem is /data{1..4} cannot be created because of the lack of
permissions.

This commit fix the behavior.
2020-01-03 21:15:26 +05:30
Harshavardhana dd311623df Update design doc with zone implementation details (#8738)
Fixes #8719
2020-01-02 16:46:16 -08:00
Harshavardhana cb935980a5 Fix version to be release-tag (#8730) 2020-01-02 20:18:32 +05:30
Praveen raj Mani 157721f694 Fix readiness to return 200 for read-only mode (#8728)
- We should declare a cluster ready even if read quorum is achieved (atleast n/2 disks are online).
- Such that, all the zones should have enough read quorum. Thus making the cluster ready for reads.
2020-01-02 05:05:01 -08:00
Nitish Tiwari 97d799b9f0 doc: Update Kubernetes examples to use Readiness Checks (#8727) 2020-01-01 08:25:09 -08:00
Harshavardhana 0b7bd024fb Fix dependencies graph for minio source compilation (#8717)
We had messy cyclical dependency problem with `mc`
due to dependencies in pkg/console, moved the pkg/console
to minio for more control and also to avoid any further
cyclical dependencies of `mc` clobbering up the
dependencies on server.

Fixes #8659
2019-12-31 09:36:13 +05:30
Harshavardhana 3af70b36fd Disallow creating buckets even with different domains (#8716)
If two distinct clusters are started with different domains
along with single common domain, this situation was leading
to conflicting buckets getting created on different clusters

To avoid this do not prematurely error out if the key has no
entries, let the caller decide on which entry matches and
which entry is valid. This allows support for MINIO_DOMAIN
with one common domain, but each cluster may have their own
domains.

Fixes #8705
2019-12-30 17:11:47 -08:00
Minio Trusted 8eba97da74 Update yaml files to latest version RELEASE.2019-12-30T05-45-39Z 2019-12-30 05:56:46 +00:00
Harshavardhana 669c9da85d Disable federated buckets when etcd is namespaced (#8709)
This is to ensure that when we have multiple tenants
deployed all sharing the same etcd for global bucket
should avoid listing each others buckets, this leads
to information leak which should be avoided unless
etcd is not namespaced for IAM assets in which case
it can be assumed that its a federated setup.

Federated setup and namespaced IAM assets on etcd
is not supported since namespacing is only useful
when you wish to separate the tenants as isolated
instances of MinIO.

This PR allows a new type of behavior, primarily
driven by the usecase of m3(mkube) multi-tenant
deployments with global bucket support.
2019-12-29 08:56:45 -08:00
Praveen raj Mani 5d09233115 Fix Readiness check (#8681)
- Remove goroutine-check in Readiness check
- Bring in quorum check for readiness

Fixes #8385

Co-authored-by: Harshavardhana <harsha@minio.io>
2019-12-28 22:24:43 +05:30
Haines Chan b2044dd22f Fix typo in comments (#8704) 2019-12-27 15:51:32 +05:30
Anis Elleuch c31e67dcce Better error when the server is unable to write in the backend (#8697) 2019-12-25 22:05:54 -08:00
kannappanr cd59a945d8
Fix: Policy action name for Get/Put BucketLifeCycle (#8698)
S3:GetBucketLifeCycle should be changed to S3:GetLifeCycleConfiguration
S3:PutBucketLifeCycle should be changed to S3:PutLifeCycleConfiguration
2019-12-25 22:04:46 -08:00
Harshavardhana d4a390028a node 6.x is EOL'ed upgrade to latest stable (#8702) 2019-12-26 08:27:35 +05:30
Bruce Wang c476b27a65 Comment typo "index max" to "index map" (#8700) 2019-12-24 21:57:43 -08:00
kannappanr eaa838a63f mint: Change aws cli version (#8699) 2019-12-24 19:58:19 -08:00
Minio Trusted 4364102363 Update yaml files to latest version RELEASE.2019-12-24T23-04-45Z 2019-12-24 23:15:44 +00:00
Harshavardhana 99ad445260
Avoid double for loops in notification init (#8691) 2019-12-24 13:49:48 -08:00
Harshavardhana 54431b3953 Change replica set detection for localhost on single endpoint (#8692) 2019-12-24 11:31:32 -08:00
Harshavardhana f68a7005c0 Improve disk formatting stage for large disk sets (#8690) 2019-12-23 16:31:03 -08:00
Harshavardhana 725172e13b
fix: Do not need safe-mode for unreachable targets upon restart (#8686) 2019-12-21 22:35:50 -08:00