Commit Graph

4955 Commits

Author SHA1 Message Date
kannappanr
001d9a4ae7
Remove unwanted volume access denied log (#6042)
Fixes #6039
2018-06-18 11:43:41 -07:00
Harshavardhana
c3a5146422 Presign content-sha256 should look for query and header (#6046) 2018-06-15 14:21:17 -07:00
Nitish Tiwari
36c39d04da Update vendorized minio-go to support start-after param (#6043)
Fixes #6032
2018-06-14 15:08:02 -07:00
Harshavardhana
28d526bc68 Change CriticalIf to FatalIf for proper error message (#6040)
During startup until the object layer is initialized
logger is disabled to provide for a cleaner UI error
message. CriticalIf is disabled, use FatalIf instead.

Also never call os.Exit(1) on running servers where
you can return error to client in handlers.
2018-06-14 10:17:07 -07:00
Harshavardhana
05f96f3956 Avoid allocating larger buffer if the incoming body is small (#6035) 2018-06-13 11:55:12 -07:00
Harshavardhana
cb9ee1584a Fix TestHealStartNStatusHandler sporadic failure (#6015)
Fixes #5818
2018-06-12 16:36:31 -07:00
Andreas Auernhammer
9f4c120731 limit memory allocations during multiple object deletion (#6033)
This commit limits the amount of memory allocated by the
S3 Multi-Object-Delete-API. The server used to allocate as
many bytes as provided by the client using Content-Length.

S3 specifies that the S3 Multi-Object-Delete-API can delete
at most 1000 objects using a single request.
(See: https://docs.aws.amazon.com/AmazonS3/latest/API/multiobjectdeleteapi.html)
Since the maximum S3 object name is limited to 1024 bytes the
XML body sent by the client can only contain up to 1000 * 1024
bytes (excluding XML format overhead).

This commit limits the size of the parsed XML for the S3
Multi-Object-Delete-API to 2 MB. This fixes a DoS
vulnerability since (auth.) clients, MitM-adversaries
(without TLS) and un-auth. users accessing buckets allowing
multi-delete by policy can kill the server.

This behavior is similar to the AWS-S3 implementation.
2018-06-11 10:51:38 -07:00
Harshavardhana
12a916091e Convert federation doc as quickstart guide (#6030) 2018-06-09 14:00:11 +05:30
Minio Trusted
842092f8de Update yaml files to latest version RELEASE.2018-06-09T03-43-35Z 2018-06-09 03:47:19 +00:00
Harshavardhana
371349787f Remove region requirement for Healing (#6031) 2018-06-08 17:54:57 -07:00
Nitish Tiwari
3dc13323e5 Use random host from among multiple hosts to create requests
Also use hosts passed to Minio startup command to populate IP
addresses if MINIO_PUBLIC_IPS is not set.
2018-06-08 10:22:01 -07:00
Nitish Tiwari
6ce7265c8c Add support for CopyObject across regions and multiple Minio IPs
This PR adds CopyObject support for objects residing in buckets
in different Minio instances (where Minio instances are part of
a federated setup).

Also, added support for multiple Minio domain IPs. This is required
for distributed deployments, where one deployment may have multiple
nodes, each with a different public IP.
2018-06-08 10:22:01 -07:00
Nitish Tiwari
f30c95a301 Add etcd handling for web-handler methods 2018-06-08 10:22:01 -07:00
Harshavardhana
481390d51a Converge etcd functionality as part of quick.Config 2018-06-08 10:22:01 -07:00
Nitish
6df1e4a529 Add functionality to add old buckets to etcd on startup
Buckets already present on a Minio server before it joins a
bucket federated deployment will now be added to etcd during
startup. In case of a bucket name collision, admin is informed
via Minio server console message.

Added configuration migration for configuration stored in etcd
backend.

Also, environment variables are updated and ListBucket path style
request is no longer forwarded.
2018-06-08 10:22:01 -07:00
Harshavardhana
853ea371ce Bring etcd support for bucket DNS federation
- Supports centralized `config.json`
- Supports centralized `bucket` service records
  for client lookups
- implement a new proxy forwarder
2018-06-08 10:22:01 -07:00
Minio Trusted
7872c192ec Update yaml files to latest version RELEASE.2018-06-08T03-49-38Z 2018-06-08 03:52:34 +00:00
Kaan Kabalak
39919708d6 Fix regression issue where users couldn't click into folders (#6029)
This commit fixes the issue introduced in #6023

The clicks on folder names direct the users inside the folders; while
the clicks on file names don't download the files, and will open file
previews on a separate modal in the future. Additionally, when a file
has been selected using the checkbox, it can now be downloaded without
being inside a zip file.

Fixes #6026
2018-06-07 20:43:51 -07:00
Minio Trusted
7e12c3e8b9 Update yaml files to latest version RELEASE.2018-06-07T19-10-07Z 2018-06-07 19:13:18 +00:00
Kaan Kabalak
94ec6f374e NewUX: On filename click, select file instead of downloading (#6023)
This commit has been done according to @abperiasamy's feedback as we
are going to reserve the click on file icon to open the preview modal
in the future.

Also, when the user now selects a single file, the file itself is
downloaded instead of a .zip file containing the file.

Fixes #6019
2018-06-07 12:02:02 -07:00
Harshavardhana
617a6d8e47 Add request tracing support for other REST APIs (#6013) 2018-06-07 10:41:13 -07:00
Praveen raj Mani
c0cfe21c00 Ignore region in the case of admin API (#5919)
Admin API is not an S3 API and hence it is not required
to honor server region while validating admin API calls.

Fixes #2411
2018-06-07 10:37:31 -07:00
Bala FA
6a8bfcef1c remove separate file for posix utils. (#5948) 2018-06-07 12:31:40 +05:30
Harshavardhana
6138cae8e7 Persist MINIO_WORM as part of config.json (#6022) 2018-06-06 18:10:51 -07:00
Krishna Srinivas
113570b514 Refresh in-memory bucket policy cache every 5 minutes (#6007) 2018-06-06 12:52:56 -07:00
Bala FA
6a53dd1701 Implement HTTP POST based RPC (#5840)
Added support for new RPC support using HTTP POST.  RPC's 
arguments and reply are Gob encoded and sent as HTTP 
request/response body.

This patch also removes Go RPC based implementation.
2018-06-06 14:21:56 +05:30
Harshavardhana
9d41051e91 Relax runtime requirement do not fail, print warning (#6017) 2018-06-06 09:25:12 +05:30
Harshavardhana
3143454982 Deprecate and remove configurable disk usage check (#6016) 2018-06-05 18:53:44 -07:00
Harshavardhana
eafc15cd47 Fix presigned URL for access key with special characters (#6012)
Fixes #6011
2018-06-05 10:48:51 -07:00
Harshavardhana
6fb0604502 Allow usage check to be configurable (#6006) 2018-06-04 18:35:41 -07:00
Harshavardhana
df1b33013f Fix byte pool usage, use only one pool for all sets. (#5990) 2018-06-01 16:41:23 -07:00
Harshavardhana
537fd8c821 Bring nicer ui error if creds are missing in Gateway (#6003) 2018-06-01 15:45:11 -07:00
Krishna Srinivas
69b9d6fbee Add user-agent for azure blob requests (#6002) 2018-06-01 15:44:48 -07:00
Harshavardhana
c22b9d5d4d
Fix s3cmd issue with ACL handling (#5999)
With the implementation of dummy GET ACL handlers,
tools like s3cmd perform few operations which causes
the ACL call to be invoked. Make sure that in our
router configuration GET?acl comes before actual
GET call to facilitate this dummy call.
2018-05-31 19:43:50 -07:00
Harshavardhana
c247e603d2 terminate disk-usage routine in tests (#5998)
tests were written in the manner by editing internal
variables of fsObjects to mimic certain behavior from
APIs, but this is racy when an active go-routine is
reading from the same variable.

Make sure to terminate the go-routine if possible for
these tests.
2018-05-31 14:03:46 -07:00
Harshavardhana
74328c3061 Implement auto cert reloading (#5963) 2018-05-31 12:30:15 -07:00
Harshavardhana
487ecedc51 fix admin info peer to point to first endpoint (#5996)
The current problem is that when you invoke

```
mc admin info myminio | head -1
●  localhost:9000
```

This output is incorrect as the expected output should be
```
mc admin info myminio | head -1
●  192.168.1.17:9000
```
2018-05-31 08:52:21 +05:30
Andreas Auernhammer
9fb94e6aa8 ensure authenticated request bodies for Admin-API (#5984)
This commit adds a check to the server's admin-API such that it only
accepts Admin-API requests with authenticated bodies. Further this
commit updates the `madmin` package to always add the
`X-Amz-Content-Sha256` header.

This change improves the Admin-API security since the server does not
accept unauthenticated request bodies anymore.

After this commit `mc` must be updated to the new `madmin` api because
requests over TLS connections will fail.
2018-05-30 14:49:03 -07:00
Harshavardhana
5282639f3c Add prometheus endpoint to support total Used storageInfo (#5988)
Since we deprecated Total/Free we don't need to update
prometheus with those metrics. This PR also adds support
for caching implementation.
2018-05-30 11:30:14 -07:00
Arjun Mishra
dd0db526d9 Remove quotes for XSS Protection Header (#5992) 2018-05-30 14:59:27 +05:30
Yaroslav Skopets
a50cc7e937 Add Prometheus metrics for Minio gateway (#5987) 2018-05-30 10:13:46 +05:30
Praveen raj Mani
7ac0fccb6e updated pika API changes in rabbitMQ notification doc (#5980)
The pika python package used for listening on rabbitMQ events
has breaking change. 'type' param changed to 'exchange_type'
in newer releases.
2018-05-29 00:35:28 -07:00
Minio Trusted
3cdf601cf7 Update yaml files to latest version RELEASE.2018-05-25T19-49-13Z 2018-05-25 19:53:08 +00:00
Nitish Tiwari
5afd856355 Update Kubernetes TLS doc with info for distributed setups (#5971)
Also, add details on how to create wildcard self-signed certificates
using openssl
2018-05-23 20:41:25 -07:00
Harshavardhana
000e360196 Deprecate showing drive capacity and total free (#5976)
This addresses a situation that we shouldn't be
displaying Total/Free anymore, instead we should simply
show the total usage.
2018-05-23 17:30:25 -07:00
Harshavardhana
e6ec645035 Implement support for calculating disk usage per tenant (#5969)
Fixes #5961
2018-05-23 15:41:29 +05:30
Acid Chicken (硫酸鶏)
483fe4bed5 Fix typo (#5960) 2018-05-22 08:09:30 +05:30
Harshavardhana
71c4ff9d10 Fix tabs/indents in help message (#5962) 2018-05-21 11:11:57 -07:00
Harshavardhana
ac58283001 Support endpoints through env (#5952) 2018-05-18 17:51:03 -07:00
Andreas Auernhammer
9c8b7306f5 security: fix write-to-RAM DoS vulnerability (#5957)
This commit fixes a DoS vulnerability for certain APIs using
signature V4 by verifying the content-md5 and/or content-sha56 of
the request body in a streaming mode.

The issue was caused by reading the entire body of the request into
memory to verify the content-md5 or content-sha56 checksum if present.

The vulnerability could be exploited by either replaying a V4 request
(in the 15 min time frame) or sending a V4 presigned request with a
large body.
2018-05-18 11:27:25 -07:00