Commit Graph

5758 Commits

Author SHA1 Message Date
poornas 97090aa16c Add admin API to send trace notifications to registered (#7128)
Remove current functionality to log trace to file
using MINIO_HTTP_TRACE env, and replace it with
mc admin trace command on mc client.
2019-06-08 15:54:41 -07:00
dependabot[bot] fb531235de Bump nwmatcher from 1.4.3 to 1.4.4 in /browser (#7760)
Bumps [nwmatcher](https://github.com/dperini/nwmatcher) from 1.4.3 to 1.4.4.
- [Release notes](https://github.com/dperini/nwmatcher/releases)
- [Commits](https://github.com/dperini/nwmatcher/commits)

Signed-off-by: dependabot[bot] <support@github.com>
2019-06-07 23:41:41 -07:00
Harshavardhana cb1566c6e6 S3 Gateway: Handle restricted access credentials (#7757) 2019-06-07 15:49:13 -07:00
Harshavardhana 6d89435356 Reload a specific user or policy on peers (#7705)
Fixes #7587
2019-06-06 17:46:22 -07:00
Andreas Auernhammer 975237cbf8 Create SECURITY.md (#7692) 2019-06-06 16:56:43 -07:00
dependabot[bot] 501af06001 Bump handlebars from 4.0.11 to 4.1.2 in /browser (#7755)
Bumps [handlebars](https://github.com/wycats/handlebars.js) from 4.0.11 to 4.1.2.
- [Release notes](https://github.com/wycats/handlebars.js/releases)
- [Changelog](https://github.com/wycats/handlebars.js/blob/master/release-notes.md)
- [Commits](https://github.com/wycats/handlebars.js/compare/v4.0.11...v4.1.2)

Signed-off-by: dependabot[bot] <support@github.com>
2019-06-06 11:16:09 -07:00
Harshavardhana a69f74533c Add region as part of error XML (#7752) 2019-06-05 16:28:21 -07:00
Joe Stevens a19cf063b5 Fixes for multiplatform dev and testing from forks (#7734)
Add support for correct dependency URLs on all platforms

only build mountinfo.go on linux

make testfile path relative to support fork work
2019-06-04 00:59:40 -07:00
Waldemar Quevedo 8b0c86298d Update import for NATS clients (#7743)
Signed-off-by: Waldemar Quevedo <wally@synadia.com>
2019-06-04 11:16:07 +05:30
Minio Trusted df19546cc8 Update yaml files to latest version RELEASE.2019-06-04T01-15-58Z 2019-06-04 01:22:32 +00:00
Harshavardhana ff7799452b Fix build failure due to nats project migration 2019-06-03 17:49:49 -07:00
Harshavardhana 97be455f63 Fix build failure in web-handlers.go 2019-06-03 16:44:09 -07:00
Harshavardhana d16a409943
Ignore srvRecords from domain level entries (#7742)
Fixes #7741
2019-06-03 16:16:05 -07:00
Krishnan Parthasarathi 74efbb4153 Add deploymentID to web handler logs (#7712) 2019-06-03 15:40:04 -07:00
Harshavardhana 0cfd5a21ba
[gateway] Remove policy reload, instead read policy from backend (#7727)
Inconsistencies can arise after applying bucket policies in
gateway mode, since all gateway instances do not share a
common shared state. This is by design to keep gateway as
shared nothing architecture.

This PR fixes such inconsistencies by reloading policy
if any from the backend.

Fixes #7723
2019-06-03 11:06:13 -07:00
dependabot[bot] 1ce2d29bbb Bump fstream from 1.0.11 to 1.0.12 in /browser (#7730)
Bumps [fstream](https://github.com/npm/fstream) from 1.0.11 to 1.0.12.
- [Release notes](https://github.com/npm/fstream/releases)
- [Commits](https://github.com/npm/fstream/compare/v1.0.11...v1.0.12)
2019-05-31 22:11:38 -07:00
Minio Trusted 48ffb7731a Update yaml files to latest version RELEASE.2019-06-01T03-46-14Z 2019-06-01 03:52:47 +00:00
Harshavardhana 1cfd4a48d9 Add specific headers in CORS, along with wildcard (#7726)
Fixes #7492
2019-05-31 09:23:55 -07:00
Harshavardhana 993a79d9c6
Disable http2 until we have upstream bugs fixed (#7711)
We should revert this PR in future once we
have upstream bugs fixed regarding http2 behavior
2019-05-30 19:49:33 -07:00
Harshavardhana 0c16b1c9a7 Fix docs.min.io linking issues (#7710) 2019-05-30 17:19:25 -07:00
Kanagaraj M 900cc27b51 validate keys before updating for IAM user (#7720)
New secretkey should be validated before updating
it on the config.

Fixes #7715
2019-05-30 05:14:35 -07:00
Kanagaraj M 8528017ad3 add min length validation to access key and secret key (#7721)
While changing the credentials through MinIO browser,
Update button will be disabled if keys are lesser than
minimum length.

Fixes #7713
2019-05-30 13:58:22 +05:30
Praveen raj Mani a73da7755e Remove senstive encryption entries from event data (#7719)
Fixes #7716
2019-05-29 22:29:37 -07:00
Harshavardhana 2c0b3cadfc Update go mod with sem versions of our libraries (#7687) 2019-05-29 16:35:12 -07:00
Praveen raj Mani 763fce909b Enable event persistence in kafka (#7633) 2019-05-29 13:19:48 -07:00
Kanagaraj M da8214845a allow users to change password through browser (#7683)
Allow IAM users to change the password using
browser UI.
2019-05-29 13:18:46 -07:00
Krishna Srinivas 74e2fe0879 Return "SlowDown" to S3 clients for network related errors (#7610)
Consider errors returned by httpClient.Do() as network errors. This is because
the http clients returns different types of errors and it is hard to catch
all the error types.
2019-05-29 10:21:47 -07:00
Harshavardhana cb7f9ba286 Add disaggregated architecture document (#7691)
This document describes how to configure Spark,
Hive to use MinIO as replacement for HDFS.
2019-05-29 08:04:51 +05:30
Harshavardhana 7906f464fd
Skip non-bucket dns entry (#7699)
Fixes #7697
2019-05-28 19:13:49 -07:00
Harshavardhana 7e4c9a9e1e Properly watch for users, policies, temp users (#7701)
Users were not reloaded properly when etcd was
configured in gateway, server modes.

This PR fixes this issue.
2019-05-28 11:18:53 +05:30
Scott Edlund f6fd407e47 Add limitation to b2.md (#7680)
The B2 gateway does not implement reporting md5sum as an etag response from PutObject.  Add to Known Limitations.
2019-05-27 21:04:43 -07:00
Nitish Tiwari 46ced81f41
Fix Gateway startup sequence to populate etcd (if set) with bucket info (#7686) 2019-05-24 08:41:52 +05:30
Minio Trusted 7a02faab72 Update yaml files to latest version RELEASE.2019-05-23T00-29-34Z 2019-05-23 00:35:13 +00:00
Dee Koder e252114f06 Revert "cache: Rewrite to cache only on download (#7575)" (#7684)
This reverts commit a13b58f630.
2019-05-22 14:54:15 -07:00
Harshavardhana 39b3e4f9b3 Avoid using io.ReadFull() for WriteAll and CreateFile (#7676)
With these changes we are now able to peak performances
for all Write() operations across disks HDD and NVMe.

Also adds readahead for disk reads, which also increases
performance for reads by 3x.
2019-05-22 13:47:15 -07:00
Anis Elleuch 158b8c2e86 sets: Correctly set IsTruncated in listing (#7675)
IsTruncated should not be set to true if there is no further
possible entries beyond maxKeys.

This commit will also move wide testing on object API from xl
to xl sets.
2019-05-22 13:36:16 -07:00
Praveen raj Mani c4c79f61ce Notification: Changes to persistent event store (#7658)
This patch includes the following changes in event store interface
- Removes memory store. We will not persist events in memory anymore, if `queueDir` is not set.
- Orders the events before replaying to the broker.
2019-05-22 13:34:48 -07:00
Harshavardhana 59e1d94770 Remove stale entry spurious logging (#7663)
The problem in current code was we were removing
an entry from a lock lockerMap without considering
the fact that different entry for same resource is
a possibility due the nature of locks that can be
acquired in parallel before we decide if the lock
is considered stale

A sequence of events is as follows

 - Lock("resource")
 - lockMaintenance(finds a long lived lock in this "resource")
 - Owner node rebooted which now retruns Expired() as true for
   this "resource"
 - Unlock("resource") which succeeded in quorum
 - Now by this time application retried and acquired a new
   Lock() on the same "resource"
 - Now that we have Expired() true from the previous call,
   we proceed to purge the entry from the local lockMap()
   local lockMap reports a different entry for the expired
   UID which results in a spurious log entry.

This PR removes this logging as this situation is an
expected scenario.
2019-05-22 12:21:36 -07:00
Andrei Mikhalenia 59e847aebe Signature v4: Allow signed headers from GET parameters 2019-05-21 21:00:02 -07:00
poornas a13b58f630 cache: Rewrite to cache only on download (#7575)
This will allow cache to consistently work for
server and gateways. Range GET requests will
be cached in the background after the request
is served from the backend.

Fixes: #7458, #7573, #6265, #6630
2019-05-22 08:30:27 +05:30
Kaan Kabalak 9e7a19d6b9 Fix object share link expiry always being 7 days (#7669)
The expiry value wasn't being sent to the server on modal value change

Fixes #7668
2019-05-19 17:45:28 -07:00
Harshavardhana 16c648b109 Remove "Connection" close instead reduce MaxConns per host (#7654)
This is necessary to avoid connection build up between servers
unexpectedly for example in a situation where 16 servers are
talking to each other and one server now allows a maximum of

15*4096 = 61440 idle connections

Will be kept in pool. Such a large pool is perhaps inefficient for
many reasons and also affects overall system resources.

This PR also reduces idleConnection timeout from 120 secs to 60 secs.
2019-05-17 12:52:25 +05:30
Krishnan Parthasarathi c871456269 File must be sync'd before closing (#7657)
- group sync and close action into a single defer statement to avoid
  evaluation order related bugs in future.
2019-05-16 18:30:51 -07:00
Harshavardhana 55aa20595f
Remove an empty entry being added into XML marshal (#7656) 2019-05-16 13:09:21 -07:00
poornas 707ed2b302 gcs: use MD5Sum as ETag if present in object attrs (#7643)
Fixes: 7642
2019-05-16 12:00:12 -07:00
ebozduman 78be3f8947 Removes the incorrect coverage badge from the docs (#7651) 2019-05-16 12:11:49 +05:30
ebozduman 67d508b214 Adjusts help content dynamically according to OS (#7646) 2019-05-15 14:02:44 +05:30
Harshavardhana 0022c9d210 Add connection close proactively for Walk() http/rpc (#7645) 2019-05-14 16:10:51 -07:00
Anis Elleuch 9b4a81ee60 xl: Avoid possible race during bulk Multi Delete (#7644)
errs was passed to many goroutines but they are all allowed
to update errs if any error happens during deletion, which
can cause a data race.

This commit will avoid issuing bulk delete operations in parallel
to avoid the warning race.
2019-05-14 14:43:22 -07:00
Harshavardhana b3f22eac56 Offload listing to posix layer (#7611)
This PR adds one API WalkCh which
sorts and sends list over the network

Each disk walks independently in a sorted manner.
2019-05-14 13:49:10 -07:00