Commit Graph

3690 Commits

Author SHA1 Message Date
Anis Elleuch c667d20dfc config-migrate: Fix buggy continuous re-migration of v9 to v10 config (#3338) 2016-11-23 15:53:55 -08:00
Harshavardhana 6efee2072d objectLayer: Check for `format.json` in a wrapped disk. (#3311)
This is needed to validate if the `format.json` indeed exists
when a fresh node is brought online.

This wrapped implementation also connects to the remote node
by attempting a re-login. Subsequently after a successful
connect `format.json` is validated as well.

Fixes #3207
2016-11-23 15:48:10 -08:00
Harshavardhana 7a5bbf7a2e Revert "Vendor update for dsync, fixing major go routine leak issue. (#3308)"
This reverts commit 273228fafa.
2016-11-23 15:17:41 -08:00
Anis Elleuch 14cb3645a3 config/logger: remove syslogger and upgrade to config v10 which eliminates syslog config (#3336) 2016-11-23 15:00:53 -08:00
Krishna Srinivas f4f512fedd FS/multipart: Bug fix related to part path. Hold lock on part while appending. (#3335) 2016-11-23 12:50:09 -08:00
Nitish Tiwari 1983925dcf Added distributed Minio architecture files. (#3330) 2016-11-23 11:36:52 -08:00
Anis Elleuch 22c98d3fa2 logger: Disassociate shared log config between console, file and syslog (#3333)
logurs is not helping us to set different log formats (json/text) to
different loggers. Now, we create different logurs instances and call
them in errorIf and fatalIf
2016-11-23 11:35:04 -08:00
Krishna Srinivas 01ae5bb39c FS/multipart: Fix append-parts to use minioMetaTmpBucket. (#3304) 2016-11-23 03:04:04 -08:00
koolhead17 11faf3f16d docs: Modified FreeBSD guide with official how to install golang link, (#3293) 2016-11-23 02:35:19 -08:00
Remco Verhoef d3df6c711a How to run Minio as a service on Windows (#3327) 2016-11-23 02:27:54 -08:00
Bala FA ed6e781679 globals: make read only variables as constants. (#3326) 2016-11-22 20:13:20 -08:00
Bala FA baf1c1638d server: set maximum allowed request body. (#3324)
This patch sets the value as 5GiB + 64MiB.  5GiB is the maximum
allowed object size and 64MiB is for form data fields and headers.
2016-11-22 19:58:51 -08:00
Bala FA 825000bc34 Use humanize constants for KiB, MiB and GiB units. (#3322) 2016-11-22 18:18:22 -08:00
Bala FA c1ebcbcda2 Remove usused code. (#3321) 2016-11-22 17:44:18 -08:00
Anis Elleuch 41a3a9e402 server: forbid zero port in address flag since it confuses clients and (#3318) 2016-11-22 17:01:15 -08:00
Bala FA 1d4ac4b084 Rename getUUID() into mustGetUUID() (#3320)
In case of UUID generation failure mustGetUUID() will panic than
infinitely trying in for loop.
2016-11-22 16:52:37 -08:00
Bala FA 71b357e4f2 Remove uploadIDChange structure. (#3309)
addUploadID() and removeUploadID() are wrappers to updateUploadJSON()
which is called with respective arguments.
2016-11-22 15:29:39 -08:00
Anis Elleuch 339c9019b9 Protect multipart directory from removing when it is empty (#3315) 2016-11-22 13:15:06 -08:00
Harshavardhana dd93f808c8 web: Add more data for jsonrpc responses. (#3296)
This change adds more richer error response
for JSON-RPC by interpreting object layer
errors to corresponding meaningful errors
for the web browser.

```go
&json2.Error{
   Message: "Bucket Name Invalid, Only lowercase letters, full stops, and numbers are allowed.",
}
```

Additionally this patch also allows PresignedGetObject()
to take expiry parameter to have variable expiry.
2016-11-22 11:12:38 -08:00
Anis Elleuch 4098025c11 Remove XL multipart tmp files when the latter is canceled (#3214)
XL multipart fails to remove tmp files when an error occurs during upload, this case covers the scenario where an upload is canceled manually by the client in the middle of job.
2016-11-21 16:34:57 -08:00
Bala FA bef0a50bc1 Cleanup and fixes (#3273)
* newRequestID() (previously generateUploadID()) returns string than byte array.
* Remove unclear comments and added appropriate comments.
* SHA-256, MD5 Hash functions return Hex/Base64 encoded string than byte array.
* Remove duplicate MD5 hasher functions.
* Rename listObjectsValidateArgs() into validateListObjectsArgs()
* Remove repeated auth check code in all bucket request handlers.
* Remove abbreviated names in bucket-metadata
* Avoid nested if in bucketPolicyMatchStatement()
* Use ioutil.ReadFile() instead of os.Open() and ioutil.ReadAll()
* Set crossDomainXML as constant.
2016-11-21 13:51:05 -08:00
Anis Elleuch 71ada9d6f8 commitXLMetadata() expects src and dst buckets in its arguments (#3307) 2016-11-21 12:45:02 -08:00
tibbes 33c022fcec Fix checkdeps.sh on Mac (#3306)
Update the check_minimum_version function to use numeric comparison (not
string comparison) on components of version numbers. Fixes the following
output:

```
$ make
Checking deps:
ERROR
OSX version '10.11.6' not supported.
Minimum supported version: 10.8
make: *** [checks] Error 1
```
2016-11-21 12:25:46 -08:00
Karthic Rao 273228fafa Vendor update for dsync, fixing major go routine leak issue. (#3308) 2016-11-21 10:47:48 -08:00
Harshavardhana aa98702908 api: Handle content-length-range policy properly. (#3297)
content-length-range policy in postPolicy API was
not working properly handle it. The reflection
strategy used has changed in recent version of Go.
Any free form interface{} of any integer is treated
as `float64` this caused a bug where content-length-range
parsing failed to provide any value.

Fixes #3295
2016-11-21 04:15:26 -08:00
Harshavardhana 5197649081 utils: reduceErrs returns and validates quorum errors. (#3300)
This is needed as explained by @krisis

Lets say we have following errors.

```
[]error{nil, errFileNotFound, errDiskAccessDenied, errDiskAccesDenied}
```

Since the last two errors are filtered, the maximum is nil,
depending on map order.

Let's say we get nil from reduceErr. Clearly at this point
we don't have quorum nodes agreeing about the data and since
GetObject only requires N/2 (Read quorum) and isDiskQuorum
would have returned true. This is problematic and can lead to
undersiable consequences.

Fixes #3298
2016-11-21 01:47:26 -08:00
Harshavardhana 066f64d34a bootup: MetaVolume init should use isErrIngored helper. (#3303) 2016-11-21 01:46:55 -08:00
Krishna Srinivas afa4c7c3ef fs/multipart: Append multipart parts in a proper Go routine in background. (#3282) 2016-11-20 23:42:53 -08:00
Krishna Srinivas 38537c7df2 Print line numbers to give more info on the failed tests in ExecObjectLayerAPIAnonTest() (#3302) 2016-11-20 23:41:39 -08:00
Krishnan Parthasarathi eed9ab0464 XL: pickValidXLMeta should return error instead of panic'ing (#3277) 2016-11-20 20:56:44 -08:00
Harshavardhana 0b9f0d14a1 auth/rpc: Take remote disk offline after maximum allowed attempts. (#3288)
Disks when are offline for a long period of time, we should
ignore the disk after trying Login upto 5 times.

This is to reduce the network chattiness, this also reduces
the overall time spent on `net.Dial`.

Fixes #3286
2016-11-20 16:57:12 -08:00
Anis Elleuch ffbee70e04 Avoid removing 'tmp' directory inside '.minio.sys' (#3294) 2016-11-20 14:25:43 -08:00
Harshavardhana 2c3a2241e7 update: Change update notifier for new style banner. (#3289)
For binary releases and operating systems it would be

All operating systems.
```
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Minio is 25 days 12 hours 30 minutes old                           ┃
┃ Update: https://dl.minio.io/server/minio/release/linux-amd64/minio ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛
```

On docker.
```
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Minio is 25 days 12 hours 32 minutes old ┃
┃ Update: docker pull minio/minio          ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛
```
2016-11-19 23:20:13 -08:00
Harshavardhana 1c47365445 xl/bootup: Upon bootup handle errors loading bucket and event configs. (#3287)
In a situation when we have lots of buckets the bootup time
might have slowed down a bit but during this situation the
servers quickly going up and down would be an in-transit state.

Certain calls which do not use quorum like `readXLMetaStat`
might return an error saying `errDiskNotFound` this is returned
in place of expected `errFileNotFound` which leads to an issue
where server doesn't start.

To avoid this situation we need to ignore them as safe values
to be ignored, for the most part these are network related errors.

Fixes #3275
2016-11-19 17:37:57 -08:00
Krishnan Parthasarathi 81eb7c0301 Have separate directory names for docker-compose and Dockerfile (#3292) 2016-11-19 10:42:22 -08:00
Anis Elleuch 4a926d19a7 Add GnuTLS config documentation for Windows (#3285) 2016-11-18 12:03:23 -08:00
Bala FA 05dc52a206 fix: use constants for access/secret key min/max length (#3271) 2016-11-16 17:33:55 -08:00
Harshavardhana c91d3791f9 heal: Add healing support for bucket, bucket metadata files. (#3252)
This patch implements healing in general but it is only used
as part of quickHeal().

Fixes #3237
2016-11-16 16:42:23 -08:00
Bala FA df8153859c docs: Add comments of using canonical import paths. (#3269) 2016-11-16 16:23:32 -08:00
Aditya Manthramurthy 2f43709f85 Prevent gorilla mux from normalizing path (Fixes #3256) (#3268)
Also fix test to not use a bucket name with a leading slash - this
causes the bucket name to become empty and go to an unintended API
call (listbuckets).
2016-11-16 16:23:22 -08:00
Bala FA 61d67a061c tests: add unit test for DeleteMultipleObjectsHandler. (#3267)
Fixes #3058
2016-11-16 09:46:09 -08:00
Harshavardhana 1b85302161 Fix spelling and golint errors. (#3266)
Fixes #3263
2016-11-15 18:14:23 -08:00
Anis Elleuch 6512d9978e Add support of user self signed certificates
Additionally add documentation about how to configure TLS with Minio
2016-11-15 16:15:23 -08:00
Aditya Manthramurthy e216201901 Remove control command from minio binary (Fixes #3264) (#3265) 2016-11-15 13:39:02 -08:00
Krishnan Parthasarathi 7abcededf2 Add tests for storage rpc handlers (#3262) 2016-11-15 12:12:06 -08:00
Harshavardhana 398421b9f5 xl/bootup: Server bootup shouldn't return for missing buckets. (#3255)
Ref #3196
2016-11-14 15:45:00 -08:00
Anis Elleuch b8f0d9352f signature-v2: encode path and query strings when calculating signature (#3253) 2016-11-14 10:23:21 -08:00
Harshavardhana f234c35020 lock: slice length of lock clients should be precisely urls. (#3254)
This patch fixes a possible bug, reproduced rarely only seen
once.

```
panic: runtime error: index out of range

goroutine 136 [running]:
panic(0xac1a40, 0xc4200120b0)
    /usr/local/go/src/runtime/panic.go:500 +0x1a1
github.com/minio/minio/vendor/github.com/minio/dsync.lock.func1(0xc4203d2240, 0x4, 0xc420474080, 0x4, 0x4, 0xc4202abb60, 0x0, 0xa86d01, 0xefcfc0, 0xc420417a80)
    /go/src/github.com/minio/minio/vendor/github.com/minio/dsync/drwmutex.go:170 +0x69b
created by github.com/minio/minio/vendor/github.com/minio/dsync.lock
    /go/src/github.com/minio/minio/vendor/github.com/minio/dsync/drwmutex.go:191 +0xf4
```
2016-11-14 10:18:56 -08:00
koolhead17 3bcf7b7593 Docs: Modified Docker instruction adding distributed image tag. (#3250) 2016-11-13 13:40:56 -08:00
Anis Elleuch 0c042a622a Return objects content types in Web List Objects handler (#3249) 2016-11-13 12:26:40 -08:00