Commit Graph

4802 Commits

Author SHA1 Message Date
Remco Verhoef 6508da5fde Add usage for GCS gateway, closes #4280 2017-06-16 14:53:36 -07:00
Remco Verhoef 3379f005a5 Initial implementation of Google Cloud Storage 2017-06-16 14:47:02 -07:00
Remco Verhoef 4be609eb82 Added AllAccessDisabled error 2017-06-16 14:47:02 -07:00
Remco Verhoef dd5b975001 Add comment, gateway should validate object name 2017-06-16 14:47:02 -07:00
Remco Verhoef 9ac3538141 Move anonymous error to object translation from Azure specific to gateway 2017-06-16 14:47:02 -07:00
Remco ace4f9fd15 Implement gateway support Google Cloud Storage 2017-06-16 14:47:02 -07:00
Nitish Tiwari b283a2c21f Bump docs references to latest Minio release RELEASE.2017-06-13T19-01-01Z (#4546) 2017-06-15 14:20:36 -07:00
Anand Babu (AB) Periasamy ed37bf3703 Reduce macOS instructions 2017-06-14 23:26:33 -07:00
Rushan d0f18dc1b5 browser: Use custom input number step-up/down function (#4524) 2017-06-14 17:34:11 -07:00
splinter98 8293f546af Add support for MQTT server as a notification target (#4474)
This implementation is similar to AMQP notifications:

* Notifications are published on a single topic as a JSON feed
* Topic is configurable, as is the QoS. Uses the paho.mqtt.golang
  library for the mqtt connection, and supports connections over tcp
  and websockets, with optional secure tls support.
* Additionally the minio server configuration has been bumped up
  so mqtt configuration can be added.
* Configuration migration code is added with tests.

MQTT is an ISO standard M2M/IoT messaging protocol and was
originally designed for applications for limited bandwidth
networks. Today it's use is growing in the IoT space.
2017-06-14 17:27:49 -07:00
Anis Elleuch af8071c86a xl: Fix rare freeze after many disk/network errors (#4438)
xl.storageDisks is sometimes passed to some low-level XL functions. Some disks in
xl.storageDisks are set to nil when they encounter some errors. This means all
elements in xl.storageDisks will be nil after some time which lead to an unusable XL.
2017-06-14 17:14:27 -07:00
Daniel Lind dce76d9307 Fix xl.diskWithAllParts to proper checksum algorithm (#4509) 2017-06-14 17:13:02 -07:00
Harshavardhana 11c4223f2c Add docker release files. (#4473)
We used to release by building directly on the docker
hub auto build process, which is sufficient for edge
but it is not a good idea to do it for stable releases.

Do not build docker release binaries again, but instead
use the released binaries themselves which are signed
and validated.
2017-06-14 02:08:35 -07:00
Nitish Tiwari 26903da8c2 Add steps to remove Minio volumes in the swarm (#4536) 2017-06-13 16:10:11 -07:00
Harshavardhana 353f2d3a6e fs: Hold `format.json` readLock ref to avoid GC. (#4532)
Looks like if we follow pattern such as

```
_ = rlk
```

Go can potentially kick in GC and close the fd when
the reference is lost, only speculation is that
the cause here is `SetFinalizer` which is set on
`os.close()` internally in `os` stdlib.

This is unexpected and unsual endeavour for Go, but
we have to make sure the reference is never lost
and always dies with the server.

Fixes #4530
2017-06-13 08:29:07 -07:00
Nitish Tiwari c8947af227 Update Kubernetes-yaml deployment example and Helm deployment doc with Minio image update steps (#4515) 2017-06-13 01:37:14 -07:00
Harshavardhana 075b8903d7 fs: Add safe locking semantics for `format.json` (#4523)
This patch also reverts previous changes which were
merged for migration to the newer disk format. We will
be bringing these changes in subsequent releases. But
we wish to add protection in this release such that
future release migrations are protected.

Revert "fs: Migration should handle bucketConfigs as regular objects. (#4482)"
This reverts commit 976870a391.

Revert "fs: Migrate object metadata to objects directory. (#4195)"
This reverts commit 76f4f20609.
2017-06-12 17:40:28 -07:00
Harshavardhana b8463a738c Add support for DCOS host detection, improve Docker detection. (#4525)
isDocker was currently reading from `/proc/cgroup` file. But
this file alone is rather not conclusive evidence. Docker
internally has `.dockerenv` as a special file which we should
use instead.

Fixes #4456
2017-06-13 00:33:21 +00:00
Frank Wessels 6f4862659f Investigate issue #4461 (#4521)
* Code to investigate issue #4461 (rare test failure in TestListenAndServeTLS)

* Use UTCNow() instead of time.Now().UTC()
2017-06-13 00:20:29 +00:00
Nitish Tiwari f59d7a04b4 Removed references to docker-machine in Swarm guide (#4502) 2017-06-10 21:44:20 -07:00
Dee Koder b28d5fa633 Clarify macOS instructions on brew paths. Deleted homebrew upgrade instructions. (#4501) 2017-06-09 14:24:32 -07:00
Harshavardhana 48dbd49980 Add support for kubernetes host detection (#4514)
Additionally improve what we print for `docker pull`
such that its precisely the relevant release tag.

Fixes #4456
2017-06-09 02:42:12 -07:00
Bala FA 3dfe254a11 gateway: make each backend as subcommands. (#4506)
Fixes #4450
2017-06-08 23:28:45 -07:00
Krishna Srinivas ec2920e981 Allow "minio server ." to start minio in fs mode (#4513) 2017-06-08 18:58:51 -07:00
Rushan 0f5483f497 browser: Disable usage/free stats for browser-gateway (#4497) 2017-06-08 15:09:50 -07:00
Krishnan Parthasarathi 8a6b0cc0cd TestInitListeners: Use port 0 pick available port (#4508) 2017-06-08 12:08:21 -07:00
Krishna Srinivas 2c56788f8d Validate gateway arguments (#4376)
Fixes #4355
2017-06-08 11:20:56 -07:00
Frank Wessels 145328ac9f tests: Run select statement in separate goroutine (#4499)
Instead of after the wg.Wait() so as to make sure that the 'earliest'
of the two select case that becomes active is selected..
2017-06-08 07:39:50 -07:00
poornas 45a568dd85 Give more specific error message on browser for nested policies (#4488) 2017-06-07 19:31:23 -07:00
Frank Wessels 7dcc1e92b4 Prevent unnecessary (superfluous) initialization of return variable (#4490) 2017-06-08 00:24:46 +00:00
poornas 999ae1cb96 Fix browser download returning zero bytes for s3 (#4483) 2017-06-06 18:19:35 -07:00
poornas 6651c2fc5f disable settings change on browser in gateway mode (#4472) 2017-06-06 14:56:41 -07:00
Harshavardhana 976870a391 fs: Migration should handle bucketConfigs as regular objects. (#4482)
Current code failed to anticipate the existence of files
which could have been created to corrupt the namespace such
as `policy.json` file created at the bucket top level.

In the current release creating such as file conflicts
with the namespace for future bucket policy operations.
We implemented migration of backend format to avoid situations
such as these.

This PR handles this situation, makes sure that the
erroneous files should have been moved properly.

Fixes #4478
2017-06-06 12:15:35 -07:00
Harshavardhana f99987e47c Generate sha1sum as well for release for backward compatibility. (#4475)
Additionally remove support for arm6vl in release, since
go 1.8 the support for armv6 has been dropped and we do
not see high usage events from this platform.
2017-06-06 11:25:06 -07:00
Harshavardhana 1c3f244fc5 creds: Secretkey should be generated upto 40 characters in length. (#4471)
Current code allowed it wrongly to generate secret key upto 100
we should only use 100 as a value to validate but for generating
it should be 40.

Fixes #4470
2017-06-05 15:18:03 -07:00
Aditya Manthramurthy 986aa8fabf Bypass network in lock requests to local server (#4465)
This makes lock RPCs similar to other RPCs where requests to the local
server bypass the network. Requests to the local lock-subsystem may
bypass the network layer and directly access the locking
data-structures.

This incidentally fixes #4451.
2017-06-05 12:25:04 -07:00
poornas 2559614bfd fix: Set UIversion in reply for policy API (#4469) 2017-06-05 08:11:54 -07:00
Harshavardhana a4d1ef1b62 browser: update ui-assets with new changes. (#4467)
Fixes #4269
2017-06-02 15:11:47 -07:00
Harshavardhana 432bf7d99e Fail if formatting is wrong in our CI tests. (#4459)
We didn't fail before, we should helps in avoiding
formatting issues to creep into the codebase.
2017-06-02 14:05:51 -07:00
poornas 18c4e5d357 Enable browser support for gateway (#4425) 2017-06-01 09:43:20 -07:00
Aditya Manthramurthy 64f4dbc272 Disable redirect of HTTP request to a HTTPS Minio server (#4454)
Fixes #4452
2017-05-31 20:33:13 -07:00
Frank Wessels 9ba57a8df0 Add errCorruptedFormat to list of ignored errors for metadata operations. (#4447)
Fixes listing of objects where xl.json is empty or corrupted to skip to the next disk/server (issue 4354).
2017-05-31 20:03:32 -07:00
Dee Koder 5621e6a494 Refactor service stop signal message. (#4428) 2017-05-31 11:53:04 -07:00
Frank Wessels 0f0758aece Load IO error count for posix atomically (#4448)
* Load error count atomically in order to check for maximum allowed number of IO errors.

* Remove unused (previously atomic) network IO error count
2017-05-31 09:22:53 -07:00
Aditya Manthramurthy a0e02f43e1 Fix and cleanup update message and improve related tests (#4361)
Fixes #4232
2017-05-31 09:22:00 -07:00
Harshavardhana 28352f3f5d log: Startup banner should strip standard ports. (#4443)
APIEndpoints list should strip off standard ports
to avoid confusion with clients.
2017-05-31 09:21:28 -07:00
Harshavardhana 975972d57e server: Redirection should use globalMinioPort with host without port. (#4445)
Currently redirection doesn't work in following scenarios

 - server started with port ":80" and TLS is configured
   client requested insecure request on port "80"
   gets redirected to port 443 and fails.
2017-05-31 09:21:02 -07:00
Harshavardhana 458f22f37c log: Fix printing of signature error request headers. (#4444)
The following commit f44f2e341c
fix was incomplete and we still had presigned URLs printing
in query strings in wrong fashion.

This PR fixes this properly. Avoid double encoding
percent encoded strings such as

`s3%!!(MISSING)A(MISSING)`

Print properly as json encoded.

`s3%3AObjectCreated%3A%2A`
2017-05-31 00:11:06 -07:00
Krishna Srinivas 0bba3cc8e3 gateway-azure: Convert S3 metadata to azure metadata (#4384)
fixes #4292
2017-05-30 20:05:41 -07:00
Cesar Alvernaz bac5303b10 Some minor fixes (#4441)
Word terminations
2017-05-30 11:03:25 -07:00