Commit Graph

6412 Commits

Author SHA1 Message Date
Harshavardhana 4f31a9a33b Reload users upon AddUser on peers (#6975)
Also migrate ReloadFormat to notification subsystem,
remove GetConfig() we do not use this API anymore
2018-12-18 14:39:21 -08:00
Nitish Tiwari 65ddff8899 Fix NAS Gateway Docker command example (#6967)
Fixes #6965
2018-12-18 14:37:17 -08:00
Harshavardhana e7c902bbbc
Return proper errors when admin API is not initialized (#6988)
Especially in gateway IAM admin APIs are not enabled
if etcd is not enabled, we should enable admin API though
but only enable IAM and Config APIs with etcd configured.
2018-12-18 13:03:26 -08:00
Andreas Auernhammer 5a5895203b add howto generate a master key and add master key disclaimer (#6992)
This commit adds a section to the master key documentation
describing how to generate a random 256 bit master key.

Further this commit adds a warning that master keys are not
recommended for production systems because it's (currently)
not possible to replace a master key (e.g. in case of compromise).
2018-12-18 13:00:32 -08:00
poornas 7da0336ac8 Make sure env are loaded before gateway layer initialization (#6989) 2018-12-18 10:42:09 -08:00
Harshavardhana 3be616de3f Send deployment ID in notification event response elements (#6991) 2018-12-18 10:05:26 -08:00
Harshavardhana c5bf22fd90 Turn off printing IPv6 endpoints when listening on all interfaces (#6986)
By default when we listen on all interfaces, we print all the
endpoints that at local to all interfaces including IPv6
addresses. Remove IPv6 addresses in endpoint list to be
printed in endpoints unless explicitly specified with '--address'
2018-12-18 21:56:30 +05:30
poornas 7c9f934875 Disallow SSE requests when object layer has encryption disabled (#6981) 2018-12-14 21:39:59 -08:00
Eco b6f9b24b30 Small corrections and example for auto-encryption (#6982) 2018-12-14 16:21:41 -08:00
poornas 13cb814a0e update KMS README.md to set approle env (#6978) 2018-12-14 14:03:16 -08:00
Andreas Auernhammer d264d2c899 add auto-encryption feature (#6523)
This commit adds an auto-encryption feature which allows
the Minio operator to ensure that uploaded objects are
always encrypted.

This change adds the `autoEncryption` configuration option
as part of the KMS conifguration and the ENV. variable
`MINIO_SSE_AUTO_ENCRYPTION:{on,off}`.

It also updates the KMS documentation according to the
changes.

Fixes #6502
2018-12-14 13:35:48 -08:00
Harshavardhana bebaff269c Support IPv6 in minio command line (#6947)
Fixes #6946
2018-12-14 13:07:46 +05:30
Harshavardhana 52b159b1db Allow versionId to be null for Delete,CopyObjectPart (#6972) 2018-12-14 11:34:37 +05:30
Nitish Tiwari 324834e4da Remove duplicate switch case (#6966)
Fixes #6948
2018-12-13 21:58:48 -08:00
Harshavardhana c2ed1347d9 Do not list objects unless specified in policy (#6970)
Currently we use GetObject to check if we are allowed to list,
this might be a security problem since there are many users now
who actively disable a publicly readable listing, anyone who
can guess the browser URL can list the objects.

This PR turns off this behavior and provides a more expected way
based on the policies.

This PR also additionally improves the Download() object
implementation to use a more streamlined code.

These are precursor changes to facilitate federation and web
identity support in browser.
2018-12-14 09:45:09 +05:30
Anis Elleuch 50f6f9fe58 S3 api: Ignore encoding in xml body (#6953)
One user reported having discovered the following error:

API: SYSTEM()
Time: 20:06:17 UTC 12/06/2018
Error: xml: encoding "US-ASCII" declared but Decoder.CharsetReader is nil
1: cmd/handler-utils.go:43:cmd.parseLocationConstraint()
2: cmd/auth-handler.go:250:cmd.checkRequestAuthType()
3: cmd/bucket-handlers.go:411:cmd.objectAPIHandlers.PutBucketHandler()
4: cmd/api-router.go100cmd.(objectAPIHandlers).PutBucketHandler-fm()
5: net/http/server.go:1947:http.HandlerFunc.ServeHTTP()

Hence, adding support of different xml encoding. Although there
is no clear specification about it, even setting "GARBAGE" as an xml
encoding won't change the behavior of AWS, hence the encoding seems
to be ignored.

This commit will follow that behavior and will ignore encoding field
and consider all xml as utf8 encoded.
2018-12-13 12:09:50 -08:00
Minio Trusted 48cb0ea34b Update yaml files to latest version RELEASE.2018-12-13T02-04-19Z 2018-12-13 02:09:53 +00:00
Harshavardhana 6f7c99a333 Allow versionId to be null for Copy,Get,Head API calls (#6942)
Fixes #6935
2018-12-12 11:43:44 -08:00
Harshavardhana 3498f5b0ec List exact DNS entries for a requested bucketName (#6936)
Currently we would end up considering common prefix
buckets to be part of the same DNS service record,
which leads to Minio server wrongly forwarding the
records to incorrect IPs.
2018-12-12 10:47:03 -08:00
Andreas Auernhammer 21d8c0fd13 refactor vault configuration and add master-key KMS (#6488)
This refactors the vault configuration by moving the
vault-related environment variables to `environment.go`
(Other ENV should follow in the future to have a central
place for adding / handling ENV instead of magic constants
and handling across different files)

Further this commit adds master-key SSE-S3 support.
The operator can specify a SSE-S3 master key using
`MINIO_SSE_MASTER_KEY` which will be used as master key
to derive and encrypt per-object keys for SSE-S3
requests.

This commit is also a pre-condition for SSE-S3
auto-encyption support.

Fixes #6329
2018-12-12 12:20:29 +05:30
Kale Blankenship 79b9a9ce46 Provide actual size in events instead of compressed size. (#6950)
Previous behavior did not check if the object was compressed and
incorrectly reported the stored size rather than the actual object
size.
2018-12-11 17:30:15 -08:00
Harshavardhana b9b353db4b Add env to support synchronous ops for all calls (#6877) 2018-12-11 16:22:56 -08:00
poornas 11a9b317a3 Disable ListenBucket notifications for NAS gateway (#6954) 2018-12-11 16:16:09 -08:00
Praveen raj Mani 9af7d627ac Preserve the compression headers while copying (#6952)
Fixes #6951
2018-12-11 12:05:41 -08:00
Harshavardhana 76d9d54603 Filter listing buckets based on user level access (#6940)
Fixes #6701
2018-12-10 22:57:22 +05:30
Harshavardhana 4c7c571875 Support JSON to CSV and CSV to JSON output format conversion (#6910)
This PR implements one of the pending items in issue #6286
in S3 API a user can request CSV output for a JSON document
and a JSON output for a CSV document. This PR refactors
the code a little bit to bring this feature.
2018-12-07 14:55:32 -08:00
James Neiman, President 313ba74b09 Update to Minio GCS Gateway (#6887) 2018-12-06 10:09:37 -08:00
Harshavardhana 3e124315c8 Increase the keep alive timeout to 30 secs (#6924)
Go by default uses a 3 * minute, we should
atleast use 30 secs as 10 secs is too aggressive.
2018-12-06 22:56:16 +05:30
Minio Trusted 78a0fd951e Update yaml files to latest version RELEASE.2018-12-06T01-27-43Z 2018-12-06 01:35:43 +00:00
Anis Elleuch 40852801ea fix: Better check of RPC type requests (#6927)
guessIsRPCReq() considers all POST requests as RPC but doesn't
check if this is an object operation API or not, which is actually
confusing bucket forwarder handler when it receives a new multipart
upload API which is a POST http request.

Due to this bug, users having a federated setup are not able to
upload a multipart object using an endpoint which doesn't actually
contain the specified bucket that will store the object.

Hence this commit will fix the described issue.
2018-12-05 14:28:48 -08:00
poornas f6980c4630 fix ConfigSys and NotificationSys initialization for NAS (#6920) 2018-12-05 14:03:42 -08:00
Harshavardhana 8fcc787cba Register notFound handler only once per root router (#6926)
registering notFound handler more than once causes
gorilla mux to return error for all registered paths
greater than > 8. This might be a bug in the gorilla/mux
but we shouldn't be using it this way. NotFound handler
should be only registered once per root router.

Fixes #6915
2018-12-05 11:54:12 -08:00
Praveen raj Mani 4e6d3c093f Errors in notification config should not crash the server (#6881)
Fixes #6870
2018-12-04 18:27:12 -08:00
Anis Elleuch 61145361fd Fallback to non-loopback IF addresses for Domain IPs (#6918)
When MINIO_PUBLIC_IPS is not specified and no endpoints are passed
as arguments, fallback to the address of non loop-back interfaces.

This is useful so users can avoid setting MINIO_PUBLIC_IPS in docker
or orchestration scripts, ince users naturally setup an internal
network that connects all instances.
2018-12-04 17:35:22 -08:00
James Neiman, President 950b4ad9af Update to How to secure access to Minio server with TLS (#6845) 2018-12-04 17:30:39 -08:00
Harshavardhana 6add646130 Fix logging of initialization errors in distributed mode (#6914) 2018-12-04 10:25:56 -08:00
Harshavardhana 20e61fb362 Redirect browser requests only if browser is enabled (#6909)
This PR fixes an issue introduced in PR #6848, when
browser is disabled we shouldn't re-direct the requests
returning AccessDenied.

Fixes #6907
2018-12-04 13:08:24 +05:30
Bala FA 18ced1102c handle post policy only if it is set. (#6852)
Previously policy in post form is assumed to be set always.  This is
fixed by doing the check when policy is set.
2018-12-03 12:01:28 -08:00
Harshavardhana d6af3c1237 Add bucket notification support for NAS gateway (#6908)
Fixes #6885
2018-12-03 14:02:14 +05:30
Andreas Auernhammer 5549a44566 rename vault namespace env variable to be more idiomatic (#6905)
This commit renames the env variable for vault namespaces
such that it begins with `MINIO_SSE_`. This is the prefix
for all Minio SSE related env. variables (like KMS).
2018-12-01 05:28:49 -08:00
Praveen raj Mani e7af31c2ff Removed `clientID` from NATS-Streaming Config (#6391)
clientID must be a unique `UUID` for each connections. Now, the
server generates it, rather considering the config.

Removing it as it is non-beneficial right now.

Fixes #6364
2018-11-30 10:46:17 +05:30
Minio Trusted e7971b1d55 Update yaml files to latest version RELEASE.2018-11-30T03-56-59Z 2018-11-30 04:02:10 +00:00
Harshavardhana 26120d7838 Ignore permission errors on config-dir (#6894) 2018-11-29 18:14:05 -08:00
Harshavardhana bef7c01c58 Choose right users in federation mode for CopyObject (#6895) 2018-11-29 17:35:11 -08:00
poornas 6a8ccc5925 update README.md (#6893) 2018-11-29 15:50:57 -08:00
kannappanr d85199e9de
Vendorize minio-go (#6883)
Fixes #6873
2018-11-29 11:13:03 -08:00
Harshavardhana 8608a84c23 Ignore hidden directory .snapshot for NetApp volumes (#6889) 2018-11-29 11:39:21 +05:30
Harshavardhana b7226f4c82 Add transaction lock when migrating configs (#6878)
When migrating configs it happens often that some
servers fail to start due to version mismatch etc.

Hold a transaction lock such that all servers get
serialized.
2018-11-28 17:38:23 -08:00
Pontus Leitzler f930ffe9e2 Fix gcs context (#6869) 2018-11-28 13:49:51 +05:30
James Neiman, President b50a245208 Update to Minio Multi-Tenant Deployment Guide (#6871)
Initial edits.
2018-11-27 18:03:07 -08:00