Commit Graph

4910 Commits

Author SHA1 Message Date
Harshavardhana
e6ec645035 Implement support for calculating disk usage per tenant (#5969)
Fixes #5961
2018-05-23 15:41:29 +05:30
Acid Chicken (硫酸鶏)
483fe4bed5 Fix typo (#5960) 2018-05-22 08:09:30 +05:30
Harshavardhana
71c4ff9d10 Fix tabs/indents in help message (#5962) 2018-05-21 11:11:57 -07:00
Harshavardhana
ac58283001 Support endpoints through env (#5952) 2018-05-18 17:51:03 -07:00
Andreas Auernhammer
9c8b7306f5 security: fix write-to-RAM DoS vulnerability (#5957)
This commit fixes a DoS vulnerability for certain APIs using
signature V4 by verifying the content-md5 and/or content-sha56 of
the request body in a streaming mode.

The issue was caused by reading the entire body of the request into
memory to verify the content-md5 or content-sha56 checksum if present.

The vulnerability could be exploited by either replaying a V4 request
(in the 15 min time frame) or sending a V4 presigned request with a
large body.
2018-05-18 11:27:25 -07:00
rawipfel
1cf381f1b0 handle Kubernetes read-only secrets (#5951) 2018-05-18 10:31:11 +05:30
kannappanr
ffa6b45d00 Remove treewalk abort log message (#5953)
Log doesn't add any value and can be removed.

Fixes #5943
2018-05-17 17:26:16 -07:00
Bala FA
4eb788df79 rename checkPathValid() to getValidPath() (#5949) 2018-05-17 07:27:07 -07:00
Minio Trusted
f9e8ac429e Update yaml files to latest version RELEASE.2018-05-16T23-35-33Z 2018-05-16 23:39:52 +00:00
Harshavardhana
64288d6eb5 fix: export policy.Resource for remote RPC policy updates (#5941)
This PR fixes a regression where the remote bucket policies
were not updated due to policy.Resource not having exported
fields, gob fails with unexported fields while marshalling.
2018-05-16 16:17:00 -07:00
Nitish Tiwari
41496e1406 Fix broken healthcheck link (#5935) 2018-05-16 14:43:25 -07:00
Nitish Tiwari
e6ebcc4cb6 Remove redundant prometheus data points (#5934)
Removed field minio_http_requests_total as it was redundant with
minio_http_requests_duration_seconds_count

Also removed field minio_server_start_time_seconds as it was
redundant with process_start_time_seconds
2018-05-15 12:23:43 -07:00
Nitish Tiwari
9cab0f25e0 Add top level metrics document to summarize monitoring endpoints (#5923)
Minio server supports healthcheck and prometheus related
unauthenticated endpoints. This document summarizes this
information in a single place and add links for more detailed
documentation if needed.
2018-05-15 12:23:21 -07:00
Minio Trusted
5c21e89559 Update yaml files to latest version RELEASE.2018-05-11T00-29-24Z 2018-05-11 00:34:18 +00:00
Harshavardhana
c872c30ea3 fix: introduce isLeafDir in healing to fix the crash (#5920)
This PR also supports healing directories.

Fixes #5917
2018-05-10 16:53:42 -07:00
Harshavardhana
5b74f918d4 Implement a dummy GetBucketACL for legacy reasons (#5906)
GetBucketACL call returns empty for all GET in ACL requests, 
the primary purpose of this PR is to provide legacy API support 
for legacy applications.

Fixes #5706
2018-05-10 09:32:26 +05:30
Minio Trusted
518f856900 Update yaml files to latest version RELEASE.2018-05-10T00-00-42Z 2018-05-10 00:03:30 +00:00
Krishna Srinivas
cc8178cdc4 Log errors only once for event notification errors (#5905) 2018-05-09 15:59:45 -07:00
Harshavardhana
4886bfbc72 fix: Avoid more crashes due to concurrent map usage (#5912)
This PR fixes another situation where a crash occurs
thanks to @krishnasrinivas for reproducing this

Fixes #5897
2018-05-09 15:11:51 -07:00
Anis Elleuch
c525424179 Return detailed msg for notification errors (#5913) 2018-05-09 15:11:24 -07:00
Anis Elleuch
c1f1fa3cc8 More fixes for error messages (#5911) 2018-05-09 13:29:37 -07:00
Harshavardhana
98f81ced86 fix: Avoid concurrent map writes in go-routines (#5898)
Fixes #5897
2018-05-09 11:25:38 -07:00
Anis Elleuch
2c0020e9ee Fix typo in prometheus scrape url (#5909) 2018-05-09 11:20:03 -07:00
Praveen raj Mani
3971483e18 Remove trailing/leading spaces in Certificates (#5885)
Fixes:  #5632
2018-05-09 14:26:38 +05:30
Ashish Kumar Sinha
deb685c5b5 Enhancements in Minio Prometheus exporter (#5848)
Standardized Minio collectors based on Prometheus 
recommendations.
2018-05-09 01:38:27 -07:00
Krishna Srinivas
bb34bd91f1 Fix unnecessary log messages to avoid flooding the logs (#5900) 2018-05-09 01:38:27 -07:00
Krishna Srinivas
9de8fefa90 Fix azure list-objects not to list minio.sys.tmp/ (#5895) 2018-05-09 01:38:27 -07:00
Anis Elleuch
6d5f2a4391 Better support of empty directories (#5890)
Better support of HEAD and listing of zero sized objects with trailing
slash (a.k.a empty directory). For that, isLeafDir function is added
to indicate if the specified object is an empty directory or not. Each
backend (xl, fs) has the responsibility to store that information.
Currently, in both of XL & FS, an empty directory is represented by
an empty directory in the backend.

isLeafDir() checks if the given path is an empty directory or not,
since dir listing is costly if the latter contains too many objects,
readDirN() is added in this PR to list only N number of entries.
In isLeadDir(), we will only list one entry to check if a directory
is empty or not.
2018-05-09 01:38:21 -07:00
Anis Elleuch
32700fca52 Enhance fatal errors printing of common issues seen by users (#5878) 2018-05-08 19:04:36 -07:00
Harshavardhana
54cd29b51c Add new S3 APIs which are not implemented (#5899)
Fixes #5896
2018-05-08 21:45:10 +05:30
Harshavardhana
c98d8cb1c7 fs: fix a regression allow reading of existing files (#5889) 2018-05-07 17:00:44 -07:00
Minio Trusted
2ecc976950 Update yaml files to latest version RELEASE.2018-05-04T23-13-12Z 2018-05-04 23:16:52 +00:00
Andreas Auernhammer
c5a00e513c fix DoS vulnerability in request authentication (#5887)
This commit fixes a DoS vulnerability in the
request authentication. The root cause is an 'unlimited'
read-into-RAM from the request body.

Since this read happens before the request authentication
is verified the vulnerability can be exploit without any
access privileges.

This commit limits the size of the request body to 3 MB.
This is about the same size as AWS. The limit seems to be
between 1.6 and 3.2 MB - depending on the AWS machine which
is handling the request.
2018-05-04 11:16:14 -07:00
Anis Elleuch
9439dfef64 Use defer style to stop tickers to avoid current/possible misuse (#5883)
This commit ensures that all tickers are stopped using defer ticker.Stop()
style. This will also fix one bug seen when a client starts to listen to
event notifications and that case will result a leak in tickers.
2018-05-04 10:43:20 -07:00
Krishna Srinivas
0f746a14a3 Do not use crypto.SHA3_256 as placeholder for HighwayHash256 (#5847) 2018-05-04 10:42:22 -07:00
Harshavardhana
d69ba7d085 fs: fix logging by not logging common scenarios (#5882)
Fixes #5881
2018-05-03 08:29:38 +05:30
Krishna Srinivas
d6df9b16ac Return NoSuchKey for anonReqs with s3:ListBucket policy (#5876) 2018-05-02 12:13:27 +05:30
Krishna Srinivas
0ad0969d61 Remove s3:ListObjects policy action to be in sync with AWS-S3 (#5875) 2018-05-02 12:13:17 +05:30
wd256
7f207156ec Copy an object without user-defined metadata for Azure gateway (#5872) 2018-05-02 10:20:00 +05:30
Andreas Auernhammer
ca8520fb31 vendor x/sys/cpu and update x/crypto/blake2b (#5870)
This change updates the vendor'd x/crypto/blake2b package and
adds x/sys/cpu.

This change fixes an issue for Go1.11 affecting x/crypto/blake2b
since G1.11 removes the runtime-internal functions `suports_avx()` and
`supports_avx2()`.
2018-05-01 11:16:56 +05:30
Harshavardhana
5f9041571f Heal only when atleast one of the disk is unformatted (#5866)
Current healing has an issue when disks are healed
even when they are offline without knowing if disk
is unformatted. This can lead to issues of pre-maturely
removing the disk from the set just because it was
temporarily offline.

There is an increasing number of `mc admin heal` usage
on a cron or regular basis. It is possible that if healing
code saw disk is offline it might prematurely take it down,
this causes availability issues.

Fixes #5826
2018-05-01 09:07:39 +05:30
cbenien
e43cd0d5c8 Increased MaxIdleConnsPerHost to prevent excessive re-connections and TIME_WAIT when more than 100 clients are using minio (#5860) 2018-04-30 11:59:41 -07:00
Egor Panfilov
4f68e0109c Fix output of AMQP example (#5867) 2018-04-30 08:39:07 +05:30
Minio Trusted
97c0c1abfd Update yaml files to latest version RELEASE.2018-04-27T23-33-52Z 2018-04-27 23:37:02 +00:00
Krishna Srinivas
9816264eed Support for ListObjectsV1 style marker for Azure gateway (#5856)
fixes #4948
2018-04-27 16:08:46 -07:00
Krishna Srinivas
11b8e292a7 Disable logging during unit tests (#5861) 2018-04-27 15:03:19 -07:00
Harshavardhana
b6ca39ea48 Support migrating inconsistent bucket policies (#5855)
Previously we used allow bucket policies without
`Version` field to be set to any given value, but
this behavior is inconsistent with AWS S3.

PR #5790 addressed this by making bucket policies
stricter and cleaner, but this causes a breaking
change causing any existing policies perhaps without
`Version` field or the field to be empty to fail upon
server startup.

This PR brings a code to migrate under these scenarios
as a one time operation.
2018-04-27 15:02:54 -07:00
Elson Rodriguez
1bd7eb979c Add example on how to deploy minio without a backing store. (#5810)
Many on-premise clusters do not have a PV abstraction, this example 
shows a way to deploy minio effectively in such environments.
2018-04-27 20:58:23 +05:30
poornas
0dc3d7ac18 fix error checks when cache is offline/missing. (#5850) 2018-04-26 10:39:05 +05:30
Krishna Srinivas
9aace6d36d Continue healing other objects even if objects without quorum exist (#5851)
fixes #5815
2018-04-25 11:56:39 -07:00