Commit Graph

3659 Commits

Author SHA1 Message Date
Anis Elleuch ffbee70e04 Avoid removing 'tmp' directory inside '.minio.sys' (#3294) 2016-11-20 14:25:43 -08:00
Harshavardhana 2c3a2241e7 update: Change update notifier for new style banner. (#3289)
For binary releases and operating systems it would be

All operating systems.
```
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Minio is 25 days 12 hours 30 minutes old                           ┃
┃ Update: https://dl.minio.io/server/minio/release/linux-amd64/minio ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛
```

On docker.
```
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Minio is 25 days 12 hours 32 minutes old ┃
┃ Update: docker pull minio/minio          ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛
```
2016-11-19 23:20:13 -08:00
Harshavardhana 1c47365445 xl/bootup: Upon bootup handle errors loading bucket and event configs. (#3287)
In a situation when we have lots of buckets the bootup time
might have slowed down a bit but during this situation the
servers quickly going up and down would be an in-transit state.

Certain calls which do not use quorum like `readXLMetaStat`
might return an error saying `errDiskNotFound` this is returned
in place of expected `errFileNotFound` which leads to an issue
where server doesn't start.

To avoid this situation we need to ignore them as safe values
to be ignored, for the most part these are network related errors.

Fixes #3275
2016-11-19 17:37:57 -08:00
Krishnan Parthasarathi 81eb7c0301 Have separate directory names for docker-compose and Dockerfile (#3292) 2016-11-19 10:42:22 -08:00
Anis Elleuch 4a926d19a7 Add GnuTLS config documentation for Windows (#3285) 2016-11-18 12:03:23 -08:00
Bala FA 05dc52a206 fix: use constants for access/secret key min/max length (#3271) 2016-11-16 17:33:55 -08:00
Harshavardhana c91d3791f9 heal: Add healing support for bucket, bucket metadata files. (#3252)
This patch implements healing in general but it is only used
as part of quickHeal().

Fixes #3237
2016-11-16 16:42:23 -08:00
Bala FA df8153859c docs: Add comments of using canonical import paths. (#3269) 2016-11-16 16:23:32 -08:00
Aditya Manthramurthy 2f43709f85 Prevent gorilla mux from normalizing path (Fixes #3256) (#3268)
Also fix test to not use a bucket name with a leading slash - this
causes the bucket name to become empty and go to an unintended API
call (listbuckets).
2016-11-16 16:23:22 -08:00
Bala FA 61d67a061c tests: add unit test for DeleteMultipleObjectsHandler. (#3267)
Fixes #3058
2016-11-16 09:46:09 -08:00
Harshavardhana 1b85302161 Fix spelling and golint errors. (#3266)
Fixes #3263
2016-11-15 18:14:23 -08:00
Anis Elleuch 6512d9978e Add support of user self signed certificates
Additionally add documentation about how to configure TLS with Minio
2016-11-15 16:15:23 -08:00
Aditya Manthramurthy e216201901 Remove control command from minio binary (Fixes #3264) (#3265) 2016-11-15 13:39:02 -08:00
Krishnan Parthasarathi 7abcededf2 Add tests for storage rpc handlers (#3262) 2016-11-15 12:12:06 -08:00
Harshavardhana 398421b9f5 xl/bootup: Server bootup shouldn't return for missing buckets. (#3255)
Ref #3196
2016-11-14 15:45:00 -08:00
Anis Elleuch b8f0d9352f signature-v2: encode path and query strings when calculating signature (#3253) 2016-11-14 10:23:21 -08:00
Harshavardhana f234c35020 lock: slice length of lock clients should be precisely urls. (#3254)
This patch fixes a possible bug, reproduced rarely only seen
once.

```
panic: runtime error: index out of range

goroutine 136 [running]:
panic(0xac1a40, 0xc4200120b0)
    /usr/local/go/src/runtime/panic.go:500 +0x1a1
github.com/minio/minio/vendor/github.com/minio/dsync.lock.func1(0xc4203d2240, 0x4, 0xc420474080, 0x4, 0x4, 0xc4202abb60, 0x0, 0xa86d01, 0xefcfc0, 0xc420417a80)
    /go/src/github.com/minio/minio/vendor/github.com/minio/dsync/drwmutex.go:170 +0x69b
created by github.com/minio/minio/vendor/github.com/minio/dsync.lock
    /go/src/github.com/minio/minio/vendor/github.com/minio/dsync/drwmutex.go:191 +0xf4
```
2016-11-14 10:18:56 -08:00
koolhead17 3bcf7b7593 Docs: Modified Docker instruction adding distributed image tag. (#3250) 2016-11-13 13:40:56 -08:00
Anis Elleuch 0c042a622a Return objects content types in Web List Objects handler (#3249) 2016-11-13 12:26:40 -08:00
Harshavardhana c57a358c9d Validate date header only for Signed{,V2} and StreamingSigned. (#3248)
Fixes #2941
2016-11-13 12:08:24 -08:00
Anis Elleuch 380d6c6435 Use getObjectInfo() in both FS and XL ListObjects() to simplify and to return complete object information (#3247) 2016-11-13 11:48:02 -08:00
Harshavardhana 716316f711 Reduce number of envs and options from command line. (#3230)
Ref #3229

After review with @abperiasamy we decided to remove all the unnecessary options

- MINIO_BROWSER (Implemented as a security feature but now deemed obsolete
  since even if blocking access to MINIO_BROWSER, s3 API port is open)
- MINIO_CACHE_EXPIRY (Defaults to 72h)
- MINIO_MAXCONN (No one used this option and we don't test this)
- MINIO_ENABLE_FSMETA (Enable FSMETA all the time)

Remove --ignore-disks option - this option was implemented when XL layer
 would initialize the backend disks and heal them automatically to disallow
 XL accidentally using the root partition itself this option was introduced.

This behavior has been changed XL no longer automatically initializes
`format.json`  a HEAL is controlled activity, so ignore-disks is not
useful anymore. This change also addresses the problems of our documentation
going forward and keeps things simple. This patch brings in reduction of
options and defaulting them to a valid known inputs.  This patch also
serves as a guideline of limiting many ways to do the same thing.
2016-11-11 16:40:55 -08:00
Anis Elleuch 98e79b4b50 Use 307 StatusTemporaryRedirect to redirect clients from http to https with forcing them to keep HTTP Verb (#3246) 2016-11-11 15:04:51 -08:00
Anis Elleuch e2216a0936 exponentialBackoffWait returns zero after some retries, limit attempt number inside (#3245) 2016-11-11 09:49:30 -08:00
Harshavardhana e8d9d710d0 rpc: Protect racy access of internal auth states. (#3238)
Fixes #3232
2016-11-11 00:14:32 -08:00
Harshavardhana a8ab02a73a v4/presign: Fix presign requests when there are more signed headers. (#3222)
This fix removes a wrong logic which fails for requests which
have more signed headers in a presign request.

Fixes #3217
2016-11-10 21:57:15 -08:00
Aditya Manthramurthy cf022de4d5 Add tests for s3PeerAPIHandlers (Fixes #3067) (#3242) 2016-11-10 16:43:04 -08:00
Bala FA 3995e21c5b fix: Ignore object not found error in RemoveObject() in web-handler. (#3228)
Fixes #3181
2016-11-10 15:02:03 -08:00
Harshavardhana 2f7fb78692 rpc: Our rpcClient should make an attempt to reconnect. (#3221)
rpcClient should attempt a reconnect if the call fails
with 'rpc.ErrShutdown' this is needed since at times when
the servers are taken down and brought back up.

The hijacked connection from net.Dial is usually closed.

So upon first attempt rpcClient might falsely indicate that
disk to be down, to avoid this state make another dial attempt
to really fail.

Fixes #3206
Fixes #3205
2016-11-10 07:44:41 -08:00
Bala FA cf2fb30ac7 event: Add event notification for object deletion in web browser. (#3226)
Fixes #3201
2016-11-10 07:42:55 -08:00
Harshavardhana 51d1e6f75b Fix missing CompleteMultipartUpload Etag. (#3227)
Fixes #3224
2016-11-10 07:41:02 -08:00
Harshavardhana 2f373684f5 Fix the server startup messages and help text. (#3211) 2016-11-09 23:37:12 -08:00
Anis Elleuch 5741a53d46 More portable way to list files to be spellchecked and include docs/ directory (#3220) 2016-11-09 15:17:47 -08:00
Aditya Manthramurthy dd0698d14c Improve namespace lock API: (#3203)
- abstract out instrumentation information.
- use separate lockInstance type that encapsulates the nsMutex, volume,
  path and opsID as the frontend or top-level lock object.
2016-11-09 10:58:41 -08:00
Harshavardhana 3e67bfcc88 heal: Print heal command appropriately without export path. (#3208)
Fixes #3204
2016-11-09 10:50:14 -08:00
Anis Elleuch ea579f5b69 Avoid shutdown fs multiple times and create a new fs in each loop (#3213) 2016-11-09 10:10:14 -08:00
Anis Elleuch daf6f3a5c0 Fix path comparing in checkgopath (#3215) 2016-11-09 10:09:42 -08:00
Aditya Manthramurthy d44e9d6da9 Prevent weird messages from rpc lib on start (Fixes #3209): (#3212)
This is done by not making the methods of the BucketMetaState interface
as methods (via type nesting) on the type implementing
RPCs (s3PeerAPIHandlers).
2016-11-08 23:47:44 -08:00
Bala FA 9c2cfb5cb6 tests: Add missing unit test cases for AbortMultipartUploadHandler(). (#3200)
Fixes #3070
2016-11-08 16:25:00 -08:00
Bala FA 91a0ade908 tests: add unit test for HeadObjectHandler (#3197)
Fixes #3068
2016-11-07 16:02:27 -08:00
Bala FA 5eb4002bf7 cleanup build scripts. (#3192) 2016-11-07 13:28:41 -08:00
Aditya Manthramurthy 85a5c358d8 Add bucket metadata state client/handler (Fixes #3022) (#3152)
- Adds an interface to update in-memory bucket metadata state called
  BucketMetaState - this interface has functions to:
     - update bucket notification configuration,
     - bucket listener configuration,
     - bucket policy configuration, and
     - send bucket event

- This interface is implemented by `localBMS` a type for manipulating
  local node in-memory bucket metadata, and by `remoteBMS` a type for
  manipulating remote node in-memory bucket metadata.

- The remote node interface, makes an RPC call, but the local node
  interface does not - it updates in-memory bucket state directly.

- Rename mkPeersFromEndpoints to makeS3Peers and refactored it.

- Use arrayslice instead of map in s3Peers struct

- `s3Peers.SendUpdate` now receives an arrayslice of peer indexes to
  send the request to, with a special nil value slice indicating that
  all peers should be sent the update.

- `s3Peers.SendUpdate` now returns an arrayslice of errors, representing
  errors from peers when sending an update. The array positions
  correspond to peer array s3Peers.peers

Improve globalS3Peers:

- Make isDistXL a global `globalIsDistXL` and remove from s3Peers

- Make globalS3Peers an array of (address, bucket-meta-state) pairs.

- Fix code and tests.
2016-11-07 12:09:24 -08:00
Harshavardhana 33c771bb3e tests: Add tests for browser peer rpc. Fixes #3062 (#3189) 2016-11-07 11:43:35 -08:00
Karthic Rao 286a8924fd Add leak detection to object-handler tests. (#3195) 2016-11-06 21:53:50 -08:00
Karthic Rao efca29b00e Fix typos and comments in leak_detect_test.go. (#3193) 2016-11-06 21:02:29 -08:00
Harshavardhana 9161016962 tests: Improve coverage on signature v4 tests. (#3188)
Fixes #3065
2016-11-06 11:47:16 -08:00
Anis Elleuch 5ff30777e1 Rewrite connection muxer peek process to avoid server blocking by silent clients (#3187) 2016-11-06 11:41:01 -08:00
Anis Elleuch 754c0770d6 Merge ListenAndServe and ListenAndServeTLS for simplification purpose (#3186) 2016-11-05 20:32:13 -07:00
Harshavardhana 1105508453 connection muxer should use bufio.Reader to be simpler. (#3177) 2016-11-05 12:57:31 -07:00
Harshavardhana 1ba497950c Fix net.Listener to fully close the underlying socket. (#3171)
Leads to races and accepting connections. This patch implements
a way to reject accepting new connections.
2016-11-05 09:43:28 -07:00