Commit Graph

448 Commits

Author SHA1 Message Date
Andreas Auernhammer 21d8c0fd13 refactor vault configuration and add master-key KMS (#6488)
This refactors the vault configuration by moving the
vault-related environment variables to `environment.go`
(Other ENV should follow in the future to have a central
place for adding / handling ENV instead of magic constants
and handling across different files)

Further this commit adds master-key SSE-S3 support.
The operator can specify a SSE-S3 master key using
`MINIO_SSE_MASTER_KEY` which will be used as master key
to derive and encrypt per-object keys for SSE-S3
requests.

This commit is also a pre-condition for SSE-S3
auto-encyption support.

Fixes #6329
2018-12-12 12:20:29 +05:30
Harshavardhana b9b353db4b Add env to support synchronous ops for all calls (#6877) 2018-12-11 16:22:56 -08:00
James Neiman, President 313ba74b09 Update to Minio GCS Gateway (#6887) 2018-12-06 10:09:37 -08:00
Minio Trusted 78a0fd951e Update yaml files to latest version RELEASE.2018-12-06T01-27-43Z 2018-12-06 01:35:43 +00:00
James Neiman, President 950b4ad9af Update to How to secure access to Minio server with TLS (#6845) 2018-12-04 17:30:39 -08:00
Andreas Auernhammer 5549a44566 rename vault namespace env variable to be more idiomatic (#6905)
This commit renames the env variable for vault namespaces
such that it begins with `MINIO_SSE_`. This is the prefix
for all Minio SSE related env. variables (like KMS).
2018-12-01 05:28:49 -08:00
Praveen raj Mani e7af31c2ff Removed `clientID` from NATS-Streaming Config (#6391)
clientID must be a unique `UUID` for each connections. Now, the
server generates it, rather considering the config.

Removing it as it is non-beneficial right now.

Fixes #6364
2018-11-30 10:46:17 +05:30
Minio Trusted e7971b1d55 Update yaml files to latest version RELEASE.2018-11-30T03-56-59Z 2018-11-30 04:02:10 +00:00
poornas 6a8ccc5925 update README.md (#6893) 2018-11-29 15:50:57 -08:00
James Neiman, President b50a245208 Update to Minio Multi-Tenant Deployment Guide (#6871)
Initial edits.
2018-11-27 18:03:07 -08:00
poornas 45bb11e020 Set namespace on vault client if VAULT_NAMESPACE env is set (#6867) 2018-11-27 14:42:32 -08:00
jingsam b65cf281fd Update azure.md (#6834) 2018-11-27 14:05:27 -08:00
Xie Yanbo f781548b0c fix typo (#6812) 2018-11-27 14:04:50 -08:00
jingsam 25ee8e74f7 Update README.md (#6832) 2018-11-27 14:04:11 -08:00
jingsam c975d2cc7e Update README.md (#6833) 2018-11-27 14:03:08 -08:00
jingsam ea66528739 Update gcs.md (#6835) 2018-11-27 14:02:23 -08:00
Nitish Tiwari dd8c2aa5c6 Cleanup Kubernetes documentation (#6861)
Also add details on why Readiness checks are not recommended for Minio
StatefulSets.
2018-11-25 13:34:20 -08:00
Harshavardhana 9e3fce441e Audit log claims from token (#6847) 2018-11-22 09:33:24 +05:30
Minio Trusted 2fc024e880 Update yaml files to latest version RELEASE.2018-11-22T02-51-56Z 2018-11-22 02:57:28 +00:00
Minio Trusted 32d837cf88 Update yaml files to latest version RELEASE.2018-11-17T01-23-48Z 2018-11-17 01:30:29 +00:00
Harshavardhana 2929c1832d Add sample STS request/response output (#6794) 2018-11-12 07:53:55 -08:00
Harshavardhana ca7c3a3278 Add 'mc config host add' command in multi-user doc (#6777) 2018-11-08 09:42:47 -08:00
Matthias Schneider 71c66464c1 feature: added nsq as broker for events (#6740) 2018-11-07 10:23:13 -08:00
Eco 88959ce600 Format correction in server limits doc (#6773) 2018-11-06 14:50:11 -08:00
Minio Trusted bdea19b583 Update yaml files to latest version RELEASE.2018-11-06T01-01-02Z 2018-11-06 01:05:53 +00:00
poornas eb1f9c9916 Update KMS readme with vault quick start guide (#6747) 2018-11-05 13:01:18 -08:00
Harshavardhana bef0318c36 Support audit logs with additional fields (#6738)
This PR adds support

- Request query params
- Request headers
- Response headers

AuditLogEntry is exported and versioned as well
starting with this PR.
2018-11-02 18:40:08 -07:00
Harshavardhana 6491dfbbd6 Fix etcd TLS handling (#6748)
etcd fails to connect if TLS config is set, make TLS
conditional to input arguments instead
2018-11-01 21:41:11 -07:00
Harshavardhana 9fe51e392b Support etcd TLS certficates (#6719)
This PR supports two models for etcd certs

- Client-to-server transport security with HTTPS
- Client-to-server authentication with HTTPS client certificates
2018-10-29 11:14:12 -07:00
Harshavardhana 7e879a45d5 Add policy claim support for JWT (#6660)
This way temporary credentials can use canned
policies on the server without configuring OPA.
2018-10-29 11:08:59 -07:00
Harshavardhana bab4c90c45 Fix broken links in docs (#6700) 2018-10-25 11:39:31 +05:30
Minio Trusted a2fc0b14d6 Update yaml files to latest version RELEASE.2018-10-25T01-27-03Z 2018-10-25 01:31:57 +00:00
Harshavardhana fde8c38638 Add default canned policies (#6690) 2018-10-24 17:14:27 -07:00
Kaan Kabalak e6252dee5a Fix links not working on Docs site (#6692)
The relative link paths that weren't working have been changed to
direct links to the corresponding Github pages.
2018-10-24 17:00:26 -07:00
Nitish Tiwari 7b7be66fa1
Cleanup Kubernetes documentation (#6678) 2018-10-23 18:22:43 +05:30
Nitish Tiwari 32bd1b31e9
Fix images for 8 node distributed deployment (#6685)
fixes #6633
2018-10-23 10:50:38 +05:30
Eco f287b15e71 docs/minio-limits.md formatting (#6683)
Formatted docs to show missing "\" character, added "/" to list of unsupported chars and made note of the fact that list is not exhaustive.
2018-10-22 21:00:46 -07:00
Guido García 06ef8248c3 docs: add link to s3 gateway (#6666)
Minor change: Add a link to S3 gateway to make it easier to find that info.
2018-10-22 11:47:13 -07:00
Harshavardhana 0edfb32621 Fix multi-user doc (#6662) 2018-10-19 12:35:44 +05:30
Minio Trusted 44cf9ac62f Update yaml files to latest version RELEASE.2018-10-18T00-28-58Z 2018-10-18 00:34:26 +00:00
Eco 3457e504cf Spelling changes and fixed link (#6596) 2018-10-17 10:55:55 -07:00
Harshavardhana 26b4b466df
Fix a typo in multi-user doc (#6643) 2018-10-16 20:39:44 -07:00
Nitish Tiwari ef585037a0 Update config documentation (#6634) 2018-10-16 16:45:04 -07:00
Harshavardhana 1e7e5e297c
Add canned policy support (#6637)
This PR adds an additional API where we can create
a new set of canned policies which can be used with one
or many users.
2018-10-16 12:48:19 -07:00
poornas 557f382477 cache: remove cache space constraint (#6635)
relax cache constraint of requiring 100 times size of object
being cached for better cache utilization.
2018-10-16 11:06:42 +05:30
Harshavardhana 143e7fe300 Add etcd support to support STS on gateway mode (#6531) 2018-10-12 11:32:18 -07:00
Mariska Hoogenboom fae284d6b9 Docs fix for restart issue with orchestrated minio stack (#6606) (#6613) 2018-10-11 14:41:19 +05:30
Harshavardhana 54ae364def Introduce STS client grants API and OPA policy integration (#6168)
This PR introduces two new features

- AWS STS compatible STS API named AssumeRoleWithClientGrants

```
POST /?Action=AssumeRoleWithClientGrants&Token=<jwt>
```

This API endpoint returns temporary access credentials, access
tokens signature types supported by this API

  - RSA keys
  - ECDSA keys

Fetches the required public key from the JWKS endpoints, provides
them as rsa or ecdsa public keys.

- External policy engine support, in this case OPA policy engine

- Credentials are stored on disks
2018-10-09 14:00:01 -07:00
Minio Trusted d8a2975a68 Update yaml files to latest version RELEASE.2018-10-06T00-15-16Z 2018-10-06 00:19:47 +00:00
Eco 2af0f11731 Update readme.md (#6568) 2018-10-05 16:25:22 -07:00
Minio Trusted b92c324254 Update yaml files to latest version RELEASE.2018-10-05T01-03-03Z 2018-10-05 01:08:39 +00:00
Harshavardhana f163bed40d
Add Vault support for custom CAs directory (#6527) 2018-10-01 13:49:10 -07:00
Praveen raj Mani ce9d36d954 Add object compression support (#6292)
Add support for streaming (golang/LZ77/snappy) compression.
2018-09-28 09:06:17 +05:30
Minio Trusted 6c26227081 Update yaml files to latest version RELEASE.2018-09-25T21-34-43Z 2018-09-25 21:39:03 +00:00
Harshavardhana 7d0645fb3a Deprecate domain, browser as config entries (#6498) 2018-09-20 14:56:32 -07:00
Jay Mundrawala 052a7b8eec Allow minio s3 gateway to use different AWS auth mechanisms (#6422)
Allow minio s3 gateway to use aws environment credentials,
IAM instance credentials, or AWS file credentials.

If AWS_ACCESS_KEY_ID, AWS_SECRET_ACCSES_KEY are set, 
or minio is running on an ec2 instance with IAM instance credentials, 
or there is a file $HOME/.aws/credentials, minio running as an S3
gateway will authenticate with AWS S3 using those one of credentials.

The lookup order:
1. AWS environment varaibles
2. IAM instance credentials
3. $HOME/.aws/credentials
4. minio environment variables

To authenticate with the minio gateway, you will always use the
minio environment variables MINIO_ACCESS_KEY MINIO_SECRET_KEY.
2018-09-19 18:05:30 +05:30
Minio Trusted 63c03758e6 Update yaml files to latest version RELEASE.2018-09-12T18-49-56Z 2018-09-12 18:55:31 +00:00
Minio Trusted 985fd7d4e7 Update yaml files to latest version RELEASE.2018-09-11T01-39-21Z 2018-09-11 01:43:45 +00:00
ebozduman fb4186f6b9 Adds missing info to docs for credentials and domain env. vars. (#6447)
* Adds missing information to documentation for credentials and domain environment variables for distributed minio server startup.
2018-09-10 17:14:40 -07:00
Praveen raj Mani e7a4512a90 Redis documentation Fix (#6378)
Fixed few typos and missing `format` field in the example config provided.

Fixes #6356
2018-09-07 07:12:01 -07:00
Annanay Agarwal 7cb87f863e Kafka (sarama) authentication with user/pass (#6291) 2018-09-07 00:01:58 -07:00
Nitish Tiwari 67d8396af4
Fix Manta gateway client creation flow (#6425)
This commit fixes the Manta gateway client creation flow. We now affix
the endpoint scheme with endpoint URL while creating the Manta client
for gateway.

Also add steps in Manta gateway docs on how to run with custom Manta
endpoint.

Fixes #6408
2018-09-07 08:41:42 +05:30
Janko Marohnić 8b0cc376f4 Remove "List Object Parts" from Azure limitations (#6427)
Since https://github.com/minio/minio/pull/5198 has been implemented,
this is not a limitation anymore.
2018-09-06 17:19:51 -07:00
Harshavardhana 9e32cc283f Fix distributed docs to mention homogenous envs (#6405)
Also deprecate old syntax use only ellipses
2018-09-05 08:54:04 +05:30
Barnaby Keene d13bd5b9b5 Remove double backtick that was breaking docs (#6410)
On the documentation site, the double backtick with nothing in between was breaking the page render and making the text itself look quite awkward!
2018-09-04 12:06:57 -07:00
Minio Trusted c8c70a3750 Update yaml files to latest version RELEASE.2018-09-01T00-38-25Z 2018-09-01 00:43:06 +00:00
Anis Elleuch d524924b80 Fix gateway s3 doc to run custom S3 endpoint (#6369)
To pass a custom S3 endpoint in S3 gateway, the user needs
to specify it as an argument after 'minio gateway s3' and not
as '--address' option since this latter specifies the address
to which the gateway should listen.
2018-08-28 10:09:07 +05:30
Minio Trusted 1d6ce115da Update yaml files to latest version RELEASE.2018-08-25T01-56-38Z 2018-08-25 02:04:10 +00:00
Nitish Tiwari 0aee722e3f
Fix Minio browser screenshots in docs (#6334)
Fixes #6308
2018-08-22 13:15:36 +05:30
Minio Trusted 68b9e9e7e7 Update yaml files to latest version RELEASE.2018-08-21T00-37-20Z 2018-08-21 00:44:41 +00:00
poornas 8b2801bd46 Update documentation to show how to update minio config fields (#6301)
- with recent commit 1fb2e9ef95, config
can no longer be updated by editing config.json. This is because config
has been migrated inside the minio backend. Update documentation on
how to set/get configuration using mc admin config command.
2018-08-20 13:37:10 -07:00
Nitish Tiwari bf14e5ce1b Fix distributed doc as total storage capacity is not displayed anymore (#6309) 2018-08-18 10:01:24 -07:00
Minio Trusted 6c0d53a1c5 Update yaml files to latest version RELEASE.2018-08-18T03-49-57Z 2018-08-18 03:54:06 +00:00
Harshavardhana 5a4a57700b Add select docs and fix return values for Select API (#6300) 2018-08-17 17:11:39 -07:00
Kaan Kabalak 50dec08002 Correct link paths in Chinese documentation (#6299) 2018-08-17 13:16:17 -07:00
poornas e71ef905f9 Add support for SSE-S3 server side encryption with vault (#6192)
Add support for sse-s3 encryption with vault as KMS.

Also refactoring code to make use of headers and functions defined in
crypto package and clean up duplicated code.
2018-08-17 12:52:14 -07:00
junpeng liu 3d197c1449 Modify several translation errors (#6038) 2018-08-17 12:04:09 +05:30
Harshavardhana f5df3b4795 Remove select docs (#6287)
Select API is sufficiently documented, this doc is also incomplete.

- https://aws.amazon.com/blogs/aws/s3-glacier-select/
- https://aws.amazon.com/blogs/developer/introducing-support-for-amazon-s3-select-in-the-aws-sdk-for-ruby/
- https://aws.amazon.com/blogs/developer/introducing-support-for-amazon-s3-select-in-the-aws-sdk-for-javascript/
- https://aws.amazon.com/blogs/developer/category/storage/s3-select/
2018-08-15 19:47:22 -07:00
Arjun Mishra 7c14cdb60e S3 Select API Support for CSV (#6127)
Add support for trivial where clause cases
2018-08-15 03:30:19 -07:00
Minio Trusted a1a426e523 Update yaml files to latest version RELEASE.2018-08-02T23-11-36Z 2018-08-02 23:17:30 +00:00
Harshavardhana a091b1a3ee Fix admin API doc formatting (#6235) 2018-08-02 14:21:38 -07:00
Harshavardhana 556a51120c Deprecate ListLocks and ClearLocks (#6233)
No locks are ever left in memory, we also
have a periodic interval of clearing stale locks
anyways. The lock instrumentation was not complete
and was seldom used.

Deprecate this for now and bring it back later if
it is really needed. This also in-turn seems to improve
performance slightly.
2018-08-02 23:09:42 +05:30
Nitish Tiwari b16e33bcf5 Fix Kubernetes TLS doc to avoid creating CAs dir on read only mount (#6214) 2018-07-31 10:58:34 -07:00
Minio Trusted df88421087 Update yaml files to latest version RELEASE.2018-07-31T02-11-47Z 2018-07-31 02:17:27 +00:00
Minio Trusted 869018ad14 Update yaml files to latest version RELEASE.2018-07-23T18-34-49Z 2018-07-23 18:39:08 +00:00
Nitish Tiwari b8f4f26cf6 Add S3 gateway documentation (#6165)
Fixes #4830
2018-07-19 11:54:38 -07:00
Rob Girard 2a12e694f3 Changed command line examples (#6149)
Order for server:disk originally provided wouldn't stripe 
wide and may lead to availability issues.

Also added Short-form examples using {1...n} and a 
warning about {1..2} vs {1...3}
2018-07-14 20:48:38 +05:30
Minio Trusted 7c4a41b933 Update yaml files to latest version RELEASE.2018-07-13T00-09-07Z 2018-07-13 00:43:51 +00:00
Nitish Tiwari 2aa18cafc6 Update federation target to etcd/clientv3 (#6119)
With CoreDNS now supporting etcdv3 as the DNS backend, we
can update our federation target to etcdv3. Users will now be
able to use etcdv3 server as the federation backbone.

Minio will update bucket data to etcdv3 and CoreDNS can pick
that data up and serve it as bucket style DNS path.
2018-07-12 14:12:40 -07:00
Nitish Tiwari c9bc7e47b9
Update distributed docs (#6123)
We need to clarify that distributed Minio doesn't strictly need a
fresh drive, instead it just needs a fresh directory on the drive.
2018-07-10 07:32:24 +05:30
Minio Trusted 42c5b64e4e Update yaml files to latest version RELEASE.2018-07-10T01-42-11Z 2018-07-10 01:46:03 +00:00
Mike Scarlett c310cbbe89 Update comments regarding GCS component count (#6131) 2018-07-06 17:07:11 -07:00
poornas 1da362538b cache: allow ellipse style entries for MINIO_CACHE_DRIVES (#6088)
Fixes #5863
2018-07-03 16:54:10 -07:00
Harshavardhana a5453c307f Fix kernel tuning script to ignore write failures (#6107)
Certain SCSI drivers do not allow certain tuning parameters
like nr_requests, max_sectors_kb to be changed, ignore these
errors silently as this script is simply a best effort.

Fixes #6103
2018-06-30 14:55:21 -07:00
Minio Trusted f53d511798 Update yaml files to latest version RELEASE.2018-06-29T02-11-29Z 2018-06-29 02:14:58 +00:00
Harshavardhana e5e522fc61
docs: fix all Chinese doc links for the new docs site (#6097)
Additionally fix typos, default to US locale words
2018-06-28 16:02:02 -07:00
Annanay Agarwal 78abe5234e Add functionality to make cache limit configurable (#5991) 2018-06-25 10:24:12 -07:00
Minio Trusted f46ee54194 Update yaml files to latest version RELEASE.2018-06-22T23-48-46Z 2018-06-22 23:52:10 +00:00
Harshavardhana 6005dbf01f Fix broken doc links (#6068) 2018-06-22 09:14:41 +05:30
Dee Koder c91abe6c4b Update with absolute path for images (#6060) 2018-06-21 08:29:12 +05:30