Commit Graph

6291 Commits

Author SHA1 Message Date
Harshavardhana 5e40b9a563
fix: docs for live/ready check implementation details 2020-01-09 18:29:24 -08:00
Joe Adams 89d1221217 Fix typo in prometheus monitoring docs (#8780) 2020-01-09 09:08:41 -08:00
Praveen raj Mani 4cd1bbb50a This PR fixes two things (#8772)
- Stop spawning store replay routines when testing the notification targets
- Properly honor the target.Close() to clean the resources used

Fixes #8707

Co-authored-by: Harshavardhana <harsha@minio.io>
2020-01-09 19:45:44 +05:30
Harshavardhana c2cde6beb5
policy: Allow duplicate statements with different effects (#8775)
This allows "Allow" and "Deny" conflicting statements,
where we evaluate to implicit "Deny".
2020-01-08 23:00:54 -08:00
Harshavardhana abc1c1070a Add custom policy claim name (#8764)
In certain organizations policy claim names
can be not just 'policy' but also things like
'roles', the value of this field might also
be *string* or *[]string* support this as well

In this PR we are still not supporting multiple
policies per STS account which will require a
more comprehensive change.
2020-01-08 17:21:58 -08:00
poornas fd56aa42a6 Fix error message wording for PutObjectLockConfig (#8759)
Co-Authored-By: kannappanr <30541348+kannappanr@users.noreply.github.com>
Co-authored-by: Harshavardhana <harsha@minio.io>
2020-01-08 15:36:23 -08:00
Klaus Post 3d318bae76 init: Use constant time retries (#8769)
Exponential backoff does not seem like a good fit for
this function since we can expect a few roundtrips on
initial startup.

This retry loop get slow pretty quickly with initial
wait being 1 second and each try being double the
wait until 30 seconds is reached.

Instead simply try 2 times per second.
2020-01-08 13:37:34 -08:00
Harshavardhana aa2e89bfe3 Use jsoniter whenever applicable instead of encoding/json (#8766)
This PR adds jsoniter package to replace encoding/json
in places where faster json unmarshal is necessary
whenever input JSON is large enough.

Some benchmarking comparison between jsoniter and enconding/json

benchmark                            old MB/s     new MB/s     speedup
BenchmarkParseUnmarshal/N10-4        110.02       331.17       3.01x
BenchmarkParseUnmarshal/N100-4       125.74       524.09       4.17x
BenchmarkParseUnmarshal/N500-4       131.68       542.60       4.12x
BenchmarkParseUnmarshal/N1000-4      133.93       514.88       3.84x
BenchmarkParseUnmarshal/N5000-4      122.10       415.36       3.40x
BenchmarkParseUnmarshal/N10000-4     132.13       403.90       3.06x
2020-01-08 17:01:42 +05:30
Harshavardhana 60813bef29
Allow proper setCount SLAs across zones (#8752)
Fixes scenario where zones are appropriately
handled, along with supporting overriding set
count. The new fix also ensures that we handle
the various setup types properly.

Update documentation to properly indicate the
behavior.

Fixes #8750

Co-authored-by: Nitish Tiwari <nitish@minio.io>
2020-01-07 09:13:44 -08:00
Harshavardhana b123be5612 fix: browser should listBuckets from etcd in global federation (#8760) 2020-01-07 09:03:00 +05:30
Harshavardhana 933c60bc3a Add crypto context errors (#8740)
Currently when connections to vault fail, client
perpetually retries this leads to assumptions that
the server has issues and masks the problem.

Re-purpose *crypto.Error* type to send appropriate
errors back to the client.
2020-01-06 16:15:22 -08:00
ebozduman 796cca4166 Creates zipped files with correct mod times for objects (#8693) 2020-01-06 12:43:00 -08:00
Klaus Post fe379f9428 Copy metadata on update (#8755)
Fixes #8706

Co-authored-by: Harshavardhana <harsha@minio.io>
2020-01-06 10:15:44 -08:00
Harshavardhana ae0b165431 fix: --anonymous flag shouldn't print any keys (#8753)
Fixes #8744
2020-01-06 22:12:47 +05:30
Harshavardhana 7a06e158f1
Add docs for sysctl tuning parameters (#8749) 2020-01-06 00:30:04 -08:00
Harshavardhana 5eab3db344 Fix doc reference for prometheus (#8748) 2020-01-05 13:44:39 -08:00
Brian Candler 9f44fcd540 Clarify behaviour of erasure coding sets (#8745) 2020-01-05 13:00:11 -08:00
Harshavardhana b9b2b37366 Fix SECURITY.md format and newlines 2020-01-03 17:49:34 -08:00
George Xie 7f31d933a8 fixes some typos, for CREDITS change (#8743) 2020-01-03 17:49:01 -08:00
Harshavardhana 6695fd6a61
Add more context aware error for policy parsing errors (#8726)
In existing functionality we simply return a generic
error such as "MalformedPolicy" which indicates just
a generic string "invalid resource" which is not very
meaningful when there might be multiple types of errors
during policy parsing. This PR ensures that we send
these errors back to client to indicate the actual
error, brings in two concrete types such as

 - iampolicy.Error
 - policy.Error

Refer #8202
2020-01-03 11:28:52 -08:00
Minio Trusted 84e55e2e6f Update yaml files to latest version RELEASE.2020-01-03T19-12-21Z 2020-01-03 19:20:28 +00:00
Harshavardhana b00cda8ad4 Avoid running lock maintenance from all nodes (#8737)
Co-Authored-By: Krishnan Parthasarathi <krisis@users.noreply.github.com>
2020-01-03 23:11:07 +05:30
Anis Elleuch d861edfc00 xl: Print the correct err msg when access to the backend is forbidden (#8735)
minio server /data{1..4} shows an error about inability to bind a port, though
the real problem is /data{1..4} cannot be created because of the lack of
permissions.

This commit fix the behavior.
2020-01-03 21:15:26 +05:30
Harshavardhana dd311623df Update design doc with zone implementation details (#8738)
Fixes #8719
2020-01-02 16:46:16 -08:00
Harshavardhana cb935980a5 Fix version to be release-tag (#8730) 2020-01-02 20:18:32 +05:30
Praveen raj Mani 157721f694 Fix readiness to return 200 for read-only mode (#8728)
- We should declare a cluster ready even if read quorum is achieved (atleast n/2 disks are online).
- Such that, all the zones should have enough read quorum. Thus making the cluster ready for reads.
2020-01-02 05:05:01 -08:00
Nitish Tiwari 97d799b9f0 doc: Update Kubernetes examples to use Readiness Checks (#8727) 2020-01-01 08:25:09 -08:00
Harshavardhana 0b7bd024fb Fix dependencies graph for minio source compilation (#8717)
We had messy cyclical dependency problem with `mc`
due to dependencies in pkg/console, moved the pkg/console
to minio for more control and also to avoid any further
cyclical dependencies of `mc` clobbering up the
dependencies on server.

Fixes #8659
2019-12-31 09:36:13 +05:30
Harshavardhana 3af70b36fd Disallow creating buckets even with different domains (#8716)
If two distinct clusters are started with different domains
along with single common domain, this situation was leading
to conflicting buckets getting created on different clusters

To avoid this do not prematurely error out if the key has no
entries, let the caller decide on which entry matches and
which entry is valid. This allows support for MINIO_DOMAIN
with one common domain, but each cluster may have their own
domains.

Fixes #8705
2019-12-30 17:11:47 -08:00
Minio Trusted 8eba97da74 Update yaml files to latest version RELEASE.2019-12-30T05-45-39Z 2019-12-30 05:56:46 +00:00
Harshavardhana 669c9da85d Disable federated buckets when etcd is namespaced (#8709)
This is to ensure that when we have multiple tenants
deployed all sharing the same etcd for global bucket
should avoid listing each others buckets, this leads
to information leak which should be avoided unless
etcd is not namespaced for IAM assets in which case
it can be assumed that its a federated setup.

Federated setup and namespaced IAM assets on etcd
is not supported since namespacing is only useful
when you wish to separate the tenants as isolated
instances of MinIO.

This PR allows a new type of behavior, primarily
driven by the usecase of m3(mkube) multi-tenant
deployments with global bucket support.
2019-12-29 08:56:45 -08:00
Praveen raj Mani 5d09233115 Fix Readiness check (#8681)
- Remove goroutine-check in Readiness check
- Bring in quorum check for readiness

Fixes #8385

Co-authored-by: Harshavardhana <harsha@minio.io>
2019-12-28 22:24:43 +05:30
Haines Chan b2044dd22f Fix typo in comments (#8704) 2019-12-27 15:51:32 +05:30
Anis Elleuch c31e67dcce Better error when the server is unable to write in the backend (#8697) 2019-12-25 22:05:54 -08:00
kannappanr cd59a945d8
Fix: Policy action name for Get/Put BucketLifeCycle (#8698)
S3:GetBucketLifeCycle should be changed to S3:GetLifeCycleConfiguration
S3:PutBucketLifeCycle should be changed to S3:PutLifeCycleConfiguration
2019-12-25 22:04:46 -08:00
Harshavardhana d4a390028a node 6.x is EOL'ed upgrade to latest stable (#8702) 2019-12-26 08:27:35 +05:30
Bruce Wang c476b27a65 Comment typo "index max" to "index map" (#8700) 2019-12-24 21:57:43 -08:00
kannappanr eaa838a63f mint: Change aws cli version (#8699) 2019-12-24 19:58:19 -08:00
Minio Trusted 4364102363 Update yaml files to latest version RELEASE.2019-12-24T23-04-45Z 2019-12-24 23:15:44 +00:00
Harshavardhana 99ad445260
Avoid double for loops in notification init (#8691) 2019-12-24 13:49:48 -08:00
Harshavardhana 54431b3953 Change replica set detection for localhost on single endpoint (#8692) 2019-12-24 11:31:32 -08:00
Harshavardhana f68a7005c0 Improve disk formatting stage for large disk sets (#8690) 2019-12-23 16:31:03 -08:00
Harshavardhana 725172e13b
fix: Do not need safe-mode for unreachable targets upon restart (#8686) 2019-12-21 22:35:50 -08:00
Harshavardhana 0f092c2be8 Add env values and description for config sub-systems (#8684)
The output is directly borrowed from `mc` command output
2019-12-21 22:52:16 +05:30
Harshavardhana a3c8ef79a4 fix: remove extra newline from GetConfig() output (#8678) 2019-12-20 14:47:14 -08:00
Aditya Manthramurthy 01468d5a75 Fix user and policy deletion IAM commands (#8683) 2019-12-20 14:42:08 -08:00
Harshavardhana 8f1243986e
fix: listenBucket should filter events based on bucket (#8677)
Currently all bucket events are sent to all watchers
with matching prefix and event names, this becomes
problematic and prone to performance issues, fix this
situation by filtering based on buckets as well.
2019-12-20 11:45:03 -08:00
Minio Trusted dc1eb57f4e Update yaml files to latest version RELEASE.2019-12-19T22-52-26Z 2019-12-19 22:58:19 +00:00
Harshavardhana 586614c73f fix: temp credentials shouldn't allow policy/group changes (#8675)
This PR fixes the issue where we might allow policy changes
for temporary credentials out of band, this situation allows
privilege escalation for those temporary credentials. We
should disallow any external actions on temporary creds
as a practice and we should clearly differentiate which
are static and which are temporary credentials.

Refer #8667
2019-12-19 14:21:21 -08:00
Harshavardhana d140074773 fix: replica set deployment for multi tenants (#8673)
Changes in IP underneath are dynamic in replica sets
with multiple tenants, so deploying in that fashion
will not work until we wait for atleast one participatory
server to be local.

This PR also ensures that multi-tenant zone expansion also
works in replica set k8s deployments.

Introduces a new ENV `KUBERNETES_REPLICA_SET` check to call
appropriate code paths.
2019-12-19 13:45:56 -08:00