Commit Graph

514 Commits

Author SHA1 Message Date
Harshavardhana
8ce424bacd Enhance audit logging to capture responseTimes (#8067)
Audit logging requires to have

- timeToFirstByte
- timeToResponse

timing information
2019-08-12 20:32:34 -07:00
Cary FitzHugh
a0715f3b59 Adding Notes on Swarm Network Aliases (#8054) 2019-08-10 21:25:01 +05:30
Harshavardhana
5a28ef0d47 Bump readiness check upto 10000 go-routines (#8057)
Most of our current workloads reach this value
regularly, it doesn't make sense to keep 1000
go-routine limit.
2019-08-10 18:13:14 +05:30
poornas
3385bf3da8 Rewrite cache implementation to cache only on GET (#7694)
Fixes #7458
Fixes #7573 
Fixes #7938 
Fixes #6934
Fixes #6265 
Fixes #6630 

This will allow the cache to consistently work for
server and gateways. Range GET requests will
be cached in the background after the request
is served from the backend.

- All cached content is automatically bitrot protected.

- Avoid ETag verification if a cache-control header
is set and the cached content is still valid.

- This PR changes the cache backend format, and all existing
content will be migrated to the new format. Until the data is
migrated completely, all content will be served from the backend.
2019-08-09 17:09:08 -07:00
Anis Elleuch
1ce8d2c476 Add bucket lifecycle expiry feature (#7834) 2019-08-09 10:02:41 -07:00
Harshavardhana
d759a7ce99
Fix time formatting of Expiration field in STS (#8043)
Without explicit conversion to UTC() from Unix
time the zone information is lost, this leads
to XML marshallers marshaling the time into
a wrong format.

This PR fixes the compatibility issue with AWS STS
API by keeping Expiration format close to ISO8601
or RFC3339

Fixes #8041
2019-08-08 15:44:57 -07:00
Harshavardhana
9ee619ea14 Remove deprecated ENVs 2019-08-07 18:57:38 -07:00
Minio Trusted
55b385beee Update yaml files to latest version RELEASE.2019-08-07T01-59-21Z 2019-08-07 02:05:41 +00:00
Andreas Auernhammer
a6f4cf61f2 add UpdateKey method to KMS interface (#7974)
This commit adds a new method `UpdateKey` to the KMS
interface.

The purpose of `UpdateKey` is to re-wrap an encrypted
data key (the key generated & encrypted with a master key by e.g.
Vault).
For example, consider Vault with a master key ID: `master-key-1`
and an encrypted data key `E(dk)` for a particular object. The
data key `dk` has been generated randomly when the object was created.
Now, the KMS operator may "rotate" the master key `master-key-1`.
However, the KMS cannot forget the "old" value of that master key
since there is still an object that requires `dk`, and therefore,
the `D(E(dk))`.
With the `UpdateKey` method call MinIO can ask the KMS to decrypt
`E(dk)` with the old key (internally) and re-encrypted `dk` with
the new master key value: `E'(dk)`.

However, this operation only works for the same master key ID.
When rotating the data key (replacing it with a new one) then
we perform a `UnsealKey` operation with the 1st master key ID
and then a `GenerateKey` operation with the 2nd master key ID.

This commit also updates the KMS documentation and removes
the `encrypt` policy entry (we don't use `encrypt`) and
add a policy entry for `rewarp`.
2019-08-01 15:47:47 -07:00
Minio Trusted
dfa8835720 Update yaml files to latest version RELEASE.2019-08-01T22-18-54Z 2019-08-01 22:27:41 +00:00
Minio Trusted
c301f5882d Update yaml files to latest version RELEASE.2019-07-31T18-57-56Z 2019-07-31 19:06:20 +00:00
Praveen raj Mani
b0cea1c0f3 Enable event persistence in AMQP (#7565) 2019-07-25 11:20:24 -07:00
Harshavardhana
6f2b4675fa
Add krb5 support for HDFS gateway (#7933) 2019-07-24 18:05:48 -07:00
Harshavardhana
a4ce1daf99 docs: Use --user to start container in non-root (#7966) 2019-07-24 17:35:52 -07:00
Praveen raj Mani
55d4eee6f1 Enable event persistence in MySQL and PostgreSQL (#7629) 2019-07-24 10:18:29 -07:00
Minio Trusted
5b71c21330 Update yaml files to latest version RELEASE.2019-07-24T02-02-23Z 2019-07-24 02:09:19 +00:00
Praveen raj Mani
c9349747ca Enable event-persistence in NATS and NATS-Streaming (#7612) 2019-07-23 10:37:25 -07:00
Praveen raj Mani
2b9b907f9c Enable event persistence in Redis (#7601) 2019-07-23 10:22:08 -07:00
Harshavardhana
87e6533cf3 Add some design docs for distributed setup (#7950) 2019-07-23 07:48:10 +05:30
Minio Trusted
b058e32348 Update yaml files to latest version RELEASE.2019-07-17T22-54-12Z 2019-07-17 22:59:33 +00:00
Lucas
ea66a52ed1 Add KMS master key from Docker secret (#7825) 2019-07-17 20:55:26 +01:00
Harshavardhana
55dd017e62 Deprecate auto detection of container user (#7930)
There is no reliable way to handle fallbacks for
MinIO deployments, due to various command line
options and multiple locations which require
access inside container.

Parsing command line options is tricky to figure
out which is the backend disk etc, we did try
to fix this in implementations of check-user.go
but it wasn't complete and introduced more bugs.

This PR simplifies the entire approach to rather
than running Docker container as non-root by default
always, it allows users to opt-in. Such that they
are aware that that is what they are planning to do.

In-fact there are other ways docker containers can
be run as regular users, without modifying our
internal behavior and adding more complexities.
2019-07-17 19:20:55 +01:00
Harshavardhana
a57c747667 Document vault in prod mode instead of dev mode (#7928) 2019-07-16 01:32:15 +01:00
Praveen raj Mani
bf278ca36f Enable event persistence in NSQ (#7579) 2019-07-12 10:41:57 +01:00
Praveen raj Mani
bba562235b Enable persistent event store in elasticsearch (#7564) 2019-07-12 08:23:20 +05:30
Minio Trusted
2d96745156 Update yaml files to latest version RELEASE.2019-07-10T00-34-56Z 2019-07-10 00:39:45 +00:00
Minio Trusted
22bc15d89b Update yaml files to latest version RELEASE.2019-07-05T21-20-21Z 2019-07-05 21:24:43 +00:00
Praveen raj Mani
bb871a7c31 Enable event persistence in webhook (#7614) 2019-07-05 15:21:41 +05:30
iliul
a39e810965 docs: Fix dead link of HighwayHash (#7847)
Signed-off-by: Lei Liu <liul.stone@gmail.com>
2019-07-03 14:32:58 -07:00
mizuno-keyence
09103991ea [Bugfix] duplicating flag registration (#7853) 2019-07-03 14:31:19 -07:00
Matthew Wegner
0bcd8abc5c doc: "admin user policy" command typo (#7865)
Under "change user policy", the `mc admin set-policy` command is wrong.  It should be `mc admin user set-policy`.
2019-07-02 11:48:26 -07:00
Minio Trusted
5db60a6c59 Update yaml files to latest version RELEASE.2019-06-27T21-13-50Z 2019-06-27 21:18:52 +00:00
Klaus Post
61229b38f7 Update compression README.md (#7823)
- Snappy is not and RLE compressor, it is LZ77 based.
- Add `xz` as a common file type.
- Add most common media container types.
- Never heard of `application/x-spoon`. Google turns up a blank as well.
- Change link to minio blog post on compression & encryption.
2019-06-23 13:52:58 +05:30
Andreas Auernhammer
219d841496 remove encryption key section for certtool docs (#7820)
This commit removes the encryption key section from
the certool.exe docs because:
 - MinIO does not support any TLS cipher that encrypts
   something with the private key. We only support PFS
   ciphers.
 - The doc comment is not really accurate anyway.
2019-06-21 10:39:02 -07:00
poornas
680fdf6f90 Update doc link (#7814) 2019-06-20 16:49:21 -07:00
Harshavardhana
1af6e8cb72
Add support for session policies in STS APIs (#7747)
This PR adds support for adding session policies
for further restrictions on STS credentials, useful
in situations when applications want to generate
creds for multiple interested parties with different
set of policy restrictions.

This session policy is not mandatory, but optional.

Fixes #7732
2019-06-20 15:28:33 -07:00
Harshavardhana
35c38e4bd8 Add docs regarding trace (#7808) 2019-06-19 14:28:03 -07:00
Minio Trusted
e43d3a075c Update yaml files to latest version RELEASE.2019-06-19T18-24-42Z 2019-06-19 18:31:37 +00:00
Harshavardhana
43e0ef4248 Update docs with new settings (#7803) 2019-06-18 17:52:27 -07:00
Minio Trusted
59f7266081 Update yaml files to latest version RELEASE.2019-06-15T23-07-18Z 2019-06-15 23:17:28 +00:00
Harshavardhana
4a4048fe27 Migrate minio etcd config to backend config (#7751)
etcd when used in federated setups, currently
mandates that all clusters should have same
config.json, which is too restrictive and makes
federation a restrictive environment.

This change makes it apparent that each cluster
needs to be independently managed if necessary
from `mc admin info` command line.

Each cluster with in federation can have their
own root credentials and as well as separate
regions. This way buckets get further restrictions
and allows for root creds to be not common
across clusters/data centers.

Existing data in etcd gets migrated to backend
on each clusters, upon start. Once done
users can change their config entries
independently.
2019-06-15 03:07:54 -07:00
Minio Trusted
da2887f914 Update yaml files to latest version RELEASE.2019-06-13T01-41-13Z 2019-06-13 01:46:00 +00:00
Harshavardhana
a075015293 doc: Merge large bucket with distributed docs (#7761) 2019-06-11 13:44:33 -07:00
Minio Trusted
d3a2efbf91 Update yaml files to latest version RELEASE.2019-06-11T00-44-33Z 2019-06-11 00:50:31 +00:00
Harshavardhana
002a205c9c Fix OPA result response handling (#7763)
Also update the document with updated rego policy
and updated OPA agent REST API.

This PR is to fix a regression caused by PR #7637
2019-06-10 17:06:32 -07:00
Harshavardhana
91ceae23d0 Add support for customizable user (#7569) 2019-06-10 20:27:42 +05:30
Harshavardhana
cb1566c6e6 S3 Gateway: Handle restricted access credentials (#7757) 2019-06-07 15:49:13 -07:00
Waldemar Quevedo
8b0c86298d Update import for NATS clients (#7743)
Signed-off-by: Waldemar Quevedo <wally@synadia.com>
2019-06-04 11:16:07 +05:30
Minio Trusted
df19546cc8 Update yaml files to latest version RELEASE.2019-06-04T01-15-58Z 2019-06-04 01:22:32 +00:00
Minio Trusted
48ffb7731a Update yaml files to latest version RELEASE.2019-06-01T03-46-14Z 2019-06-01 03:52:47 +00:00