Commit Graph

6216 Commits

Author SHA1 Message Date
Anis Elleuch b3c19e2d4b storage: Expect empty param in REST requests (#8167)
Empty parameter was forgotten to be added to restQueries() function,
scanning with deep parameter wasn't working properly for distributed
setup.
2019-08-31 13:51:25 +05:30
Yao Zongyou ec9bfd3aef speed up the performance of s3select on csv (#7945) 2019-08-31 00:07:40 -07:00
Bala FA fa3546bb03 Add NetPerfInfo() API in madmin (#8112) 2019-08-31 08:27:53 +05:30
Harshavardhana 42e716a094
formatsToDrivesInfo should return drives with correct order (#8157)
This is a defensive change to avoid any future issues,
from this part of the code. New change also ensures
to populate UUID if present for the right disk.
2019-08-30 14:11:18 -07:00
Andreas Auernhammer 6b2ed0fc47 fix `DownloadZIP` for encrypted objects (#8159)
This commit fixes the web ZIP download handler for
encrypted objects. The decryption logic has moved into
`getObjectNInfo`. So trying to decrypt the (already decrypted)
content again in the ZIP handler obviously causes an error.

This commit fixes this by removing the decryption logic from the
the handler.

Fixes #7965
2019-08-30 10:46:09 -07:00
Harshavardhana 0cd0f6c255
Avoid error modification during IAM migration (#8156)
The underlying errors are important, for IAM
requirements and should wait appropriately at
the caller level, this allows for distributed
setups to run properly and not fail prematurely
during startup.

Also additionally fix the onlineDisk counting
2019-08-30 10:41:02 -07:00
poornas 4553db3872 mint: turn off progress for awscli sync test (#8158)
Bumping up to awscli 1.11.177 which supports --no-progress flag
2019-08-30 10:26:48 -07:00
Aditya Manthramurthy 847a3ea0a2 Add unit tests and refactor to improve coverage (#7617) 2019-08-29 13:53:27 -07:00
Aditya Manthramurthy 1f3d270de8 Fix delete policy routing (#8145) 2019-08-29 07:07:43 +05:30
Minio Trusted daa57e8e5a Update yaml files to latest version RELEASE.2019-08-29T00-25-01Z 2019-08-29 00:30:50 +00:00
Aditya Manthramurthy eb18c82976 Remove policy query param from being rejected for objects (#8144) 2019-08-28 16:58:40 -07:00
Krishna Srinivas 2ab0681c0c Do not ignore Lock()'s return value (#8142) 2019-08-28 16:12:57 -07:00
Harshavardhana 83d4c5763c
Decouple ServiceUpdate to ServerUpdate to be more native (#8138)
The change now is to ensure that we take custom URL as
well for updating the deployment, this is required for
hotfix deliveries for certain deployments - other than
the community release.

This commit changes the previous work d65a2c6725
with newer set of requirements.

Also deprecates PeerUptime()
2019-08-28 15:04:43 -07:00
Ashish Kumar Sinha c6f86d35d3 Rectify Examples and README.MD (#8141) 2019-08-28 10:56:02 -07:00
Harshavardhana d65a2c6725
Implement cluster-wide in-place updates (#8070)
This PR is a breaking change and also deprecates
`minio update` command, from this release onwards
all users are advised to just use `mc admin update`
2019-08-27 11:37:47 -07:00
Harshavardhana 70136fb55b
Look for network errors appropriately for RemoteStorageAPI (#8128)
net.Error is very unreliable in providing better error
handling, we need to ensure that we always have a fallback
option in case of network failures.

This fixes an important issue in our distributed server
setups when one of the servers is down, all deployments
out there are recommended to upgrade after this fix is
merged to ensure that availability is not lost.

Fixes #8127
Fixes #8016
Fixes #7964
2019-08-25 13:32:49 -07:00
Harshavardhana d6dd98e597
Avoid data-race in getDisksInfo call (#8126) 2019-08-23 17:03:15 -07:00
Krishna Srinivas c38ada1a26 write() to disk in 4MB blocks for better performance (#7888) 2019-08-23 15:36:46 -07:00
poornas 48bc3f1d53 Allow cached content to be encrypted (#8001)
If MINIO_CACHE_ENCRYPTION_MASTER_KEY is set,
automatically encrypt all cached content on disk.
2019-08-23 10:13:22 -07:00
Praveen raj Mani e211f6f52e Parallelize the DiskInfo calls in xl.StorageInfo() (#8115) 2019-08-22 20:02:40 -07:00
Harshavardhana f13f421e84
Allow CopyObject in pathStyle across federated instances (#8064)
Fixes #7976
2019-08-21 22:02:39 -10:00
Aditya Manthramurthy cd03bfb3cf Fix ignoring claims in list buckets call (#8118) 2019-08-21 19:20:11 -10:00
Harshavardhana add1b6cb6b
Remove go mod cache due to buggy travis caching (#8113)
There are sporadic failures due to go-mod build
caches on travis, deprecate it since we moved to
using GOPROXY.
2019-08-21 13:38:52 -10:00
Dee Koder 4f3c9d7677
Update doc with Object Tagging not supported (#8116) 2019-08-21 16:10:59 -07:00
poornas 2e19619e79 browser: Avoid logging BucketNotEmpty error (#8110) 2019-08-21 10:01:46 -10:00
Minio Trusted de736f2864 Update yaml files to latest version RELEASE.2019-08-21T19-40-07Z 2019-08-21 19:46:01 +00:00
Harshavardhana 2fa98b1d6a Convert errAuthentication as AccessDenied appropriately (#8105)
Fixes #8062
2019-08-21 09:13:15 +05:30
kannappanr 99a4298938 Use a non-strict invalid bucket name check in Get and Delete object (#8073) 2019-08-20 17:40:52 -10:00
Harshavardhana 069badc7e9
Allow CopyObjectPart to work in federated setups (#8066)
Fixes #8065
2019-08-20 07:19:22 -10:00
Harshavardhana c601cb2f1e
Add listBucketObjectsVersions implementation (#8093)
This API implementation simply behaves like listObjects()
but returns back single version for each object, this
implementation should be considered dummy it is only
meant for some applications which rely on this.
2019-08-19 11:02:54 -10:00
Harshavardhana 9ca7470ccc
Avoid using jsoniter, move to fastjson (#8063)
This is to avoid using unsafe.Pointer type
code dependency for MinIO, this causes
crashes on ARM64 platforms

Refer #8005 collection of runtime crashes due
to unsafe.Pointer usage incorrectly. We have
seen issues like this before when using
jsoniter library in the past.

This PR hopes to fix this using fastjson
2019-08-19 08:35:52 -10:00
Harshavardhana b3ca304c01
Avoid excessive listing attempts in the daily sweep (#8081)
Add better dynamic timeouts for locks, also
add jitters before launching daily sweep to ensure
that not all the servers in distributed setup
are not trying to hold locks to begin the sweep
round.

Also, add enough delay for incoming requests based
on totalSetCount*totalDriveCount.

A possible fix for #8071
2019-08-19 08:22:32 -10:00
Bala FA 60f52f461f add network read performance collection support. (#8038)
ReST API on /minio/admin/v1/performance?perfType=net[?size=N] 
returns

```
{
  "PEER-1": [
             {
	       "addr": ADDR,
	       "readPerf": DURATION,
	       "error": ERROR,
	     },
	     ...
	   ],
  ...
  ...
  "PEER-N": [
             {
	       "addr": ADDR,
	       "readPerf": DURATION,
	       "error": ERROR,
	     },
	     ...
	   ]
}
```
2019-08-19 08:26:32 +05:30
Nitish Tiwari e5fb6294a7 Remove healthcheck script for Docker image (#8095)
There are multiple possibilities for running MinIO within
a container e.g. configurable address, non-root user etc.
This makes it difficult to identify actual IP / Port to
use to check healthcheck status from within a container.

It is simpler to use external healthcheck mechanisms
like healthcheck command in docker-compose to check
for MinIO health status. This is similar to how checks
work in Kubernetes as well.

This PR removes the healthcheck script used inside
Docker container and ad documentation on how to
use docker-compose based healthcheck mechanism.
2019-08-17 12:44:04 -07:00
Harshavardhana a15bb19d37
Allow audit logging to work while tracing (#8077)
It is observed that when `mc admin trace` is being
used due to ResponseWriter wrapper, we loose information
about statusCode,statusText for audit logging.

This PR fixes this behavior
2019-08-15 16:17:46 -07:00
poornas 7bf1caa0fe Fix broken link to go install docs (#8090) 2019-08-15 16:00:50 -07:00
Harshavardhana 6e7962bf35
Return if paths are empty in DeleteFileBulk (#8085)
This avoids a network call, also fixes an issue
when empty paths are passed the underlying call
fails with "405 Method Not Allowed".

This is reproducible when you are deleting a
non-existent object.

Fixes #8083
2019-08-15 13:15:49 -07:00
Aditya Manthramurthy 825e29f301 Check if user or group is disabled when evaluating policy (#8078) 2019-08-14 16:59:16 -07:00
Harshavardhana cebeca3075 Fix multi-user doc to reflect the latest change 2019-08-14 14:12:23 -07:00
Minio Trusted 26640033b0 Update yaml files to latest version RELEASE.2019-08-14T20-37-41Z 2019-08-14 20:42:16 +00:00
Krishnan Parthasarathi bbb56739bd Add User-Agent header with MinIO release details in http logs (#7843)
This would allow http log target server to distinguish between log
messages across different versions of MinIO deployments.
2019-08-14 11:43:43 -07:00
Nitish Tiwari 1cd801b2e9 Fix DeleteObjects() to remove renamed objects inside (#8072) 2019-08-14 11:15:25 -07:00
Aditya Manthramurthy bf9b619d86 Set the policy mapping for a user or group (#8036)
Add API to set policy mapping for a user or group

Contains a breaking Admin APIs change.

- Also enforce all applicable policies
- Removes the previous /set-user-policy API

 Bump up peerRESTVersion

Add get user info API to show groups of a user
2019-08-13 13:41:06 -07:00
kannappanr bc79b435a2 Doc: Fix PutObject API known limitations text (#7704) 2019-08-13 08:50:43 -07:00
maihde 0ed6daab59 fix: #8051 so that stale DNS entries are cleaned-up (#8053) 2019-08-13 08:49:26 -07:00
Praveen raj Mani e96f19c867 Use default ports for the schemes when dialing (#7960)
Fixes #7946
2019-08-13 16:43:11 +05:30
Harshavardhana aaef18b1a3 Fail for disks which overflow upon usage calculation (#8056)
Fixes #8035
2019-08-13 02:58:43 -07:00
Kaan Kabalak a48a034e5a Make directory path in the header editable (#8018)
This change will allow users to navigate to their desired locations,
including buckets and directories that haven't been "created" yet

Fixes #7883

Add tests

Change tooltip wording

Migrate to Font Awesome 5 to use path icon

Fix sidebar not closing on mobile
2019-08-12 22:36:19 -07:00
Harshavardhana bf8ec8ad73
Cleanup ui-errors and print proper error messages (#8068)
* Cleanup ui-errors and print proper error messages

Change HELP to HINT instead, handle more error
cases when starting up MinIO. One such is related
to #8048

* Apply suggestions from code review
2019-08-12 21:25:34 -07:00
Harshavardhana 8ce424bacd Enhance audit logging to capture responseTimes (#8067)
Audit logging requires to have

- timeToFirstByte
- timeToResponse

timing information
2019-08-12 20:32:34 -07:00