Commit Graph

4592 Commits

Author SHA1 Message Date
Harshavardhana
f25ec31565 Set maxResources appropriately for gateway like server (#5321) 2017-12-24 20:09:30 +05:30
Matthieu Paret
374feda237 add HTTPStats to madmin (#5299) 2017-12-22 17:47:30 -08:00
A. Elleuch
6ef0161835 fix: Restore empty files when healing (#5257)
HealFile() does not process the case when an empty file is lost in
some disks. Since, Reedsolomon erasure doesn't handle restoring empty
data, HealFile will create empty files similarly to CreateFile().
2017-12-22 14:57:57 -08:00
Nitish Tiwari
1a3dbbc9dd
Add x-amz-storage-class support (#5295)
This adds configurable data and parity options on a per object
basis. To use variable parity

- Users can set environment variables to cofigure variable
parity

- Then add header x-amz-storage-class to putobject requests
with relevant storage class values

Fixes #4997
2017-12-22 16:58:13 +05:30
Aditya Manthramurthy
f1355da72e Add base64 encoded MD5 output for Hash Reader (#5315)
- Use it to send the Content-MD5 header correctly encoded to S3
  Gateway

- Fixes a bug in PutObject (including anonymous PutObject) and
  PutObjectPart with S3 Gateway found when testing with Mint.
2017-12-21 17:27:33 -08:00
Krishnan Parthasarathi
bbe521ffec ReInitDisk RPC handler should use retryStorage (#5310) 2017-12-21 12:28:01 +05:30
Paul Stack
7d75d61621 Add Support for Manta Object Storage as a Gateway (#5025)
Manta is an Object Storage by [Joyent](https://www.joyent.com/)

This PR adds initial support for Manta. It is intended as non-production 
ready so that feedback can be obtained.
2017-12-20 13:37:56 +05:30
Harshavardhana
1f77708a30 Limit number of connections upto system maxlimit (#5109) 2017-12-20 13:30:14 +05:30
Timon Wong
84fc78d60f Implement Alibaba Cloud OSS gateway support (#5103) 2017-12-19 13:55:17 +05:30
poornas
a182fe8c15 update steps to make changes to config.json (#5292) 2017-12-17 21:00:12 -08:00
Harshavardhana
819d1e80c6 Add more delays on distributed startup for slow network (#5240)
Refer #5237
2017-12-16 08:25:29 -08:00
Kaan Kabalak
ffdf115bf2 Fix mobile issue where checkbox was seen over sidebar on list item hover (#5288)
Fixes #5287
2017-12-15 12:36:05 -08:00
Harshavardhana
eb7c690ea9 Support in-place upgrades of new minio binary and releases. (#4961)
This PR allows 'minio update' to not only shows update banner
but also allows for in-place upgrades.

Updates are done safely by validating the downloaded
sha256 of the binary.

Fixes #4781
2017-12-15 12:33:42 -08:00
Nitish Tiwari
8c08571cd9
Update Kubernetes example yaml files (#5278)
Removed the non production ready Kubernetes constructs that are not needed
for standard Minio deployment. General cleanup of the documents.
2017-12-12 10:29:00 +05:30
kannappanr
2853fa1882 Remove logger field info from docs (#5281)
Logger field is removed from the docs,
as it has been removed from the config file.
2017-12-11 13:20:05 -08:00
Harshavardhana
1672c73988 Fix snap docs with instructions to run (#5266) 2017-12-07 08:38:58 +05:30
Harshavardhana
5476415016 Remove unused deps for cloudresourcemanager (#5274) 2017-12-06 16:41:56 -08:00
Kaan Kabalak
67ac74471d Fix error message on browser window resize when user has no buckets (#5275)
This commit handles the case where the list of buckets is empty on the
listObjects function of actions.js

Fixes #5267
2017-12-06 15:52:23 -08:00
kannappanr
a1c1a18dc5 Remove "logger" field from config.json (#5268)
File logging removed as part of improvement to server logging.

config.json format updated to version 21.

Fixes #5176
2017-12-06 12:48:29 +05:30
Harshavardhana
eb2894233c Convert gateways into respective packages (#5200)
- Make azure gateway a package
- Make b2 gateway a package
- Make gcs gateway a package
- Make s3 gateway a package
- Make sia gateway a package
2017-12-05 17:58:09 -08:00
Nitish Tiwari
52e382b697 Update go version in snap file (#5256) 2017-12-04 15:49:31 -08:00
techknowlogick
0d435e11b1 Change container name in b2 docs (#5259) 2017-12-04 22:34:00 +05:30
Harshavardhana
2755a0b763 Check if SSL is configured to validate input arguments (#5252)
This PR handles following situations

- secure endpoints provided, server should fail to start
  if TLS is not configured

- insecure endpoints provided, server starts ignoring
  if TLS is configured or not.

Fixes #5251
2017-12-04 12:17:12 +05:30
Aditya Manthramurthy
043e030a4a Add CopyObjectPart support to gateway S3 (#5213)
- Adds a metadata argument to the CopyObjectPart API to facilitate
  implementing encryption for copying APIs too.

- Update vendored minio-go - this version implements the
  CopyObjectPart client API for use with the S3 gateway.

Fixes #4885
2017-12-02 08:33:59 +05:30
Harshavardhana
490c30f853
erasure: Support cleaning up of stale multipart objects (#5250)
Just like our single directory/disk setup, this PR brings
the functionality to cleanup stale multipart objects
older > 2 weeks.
2017-11-30 18:11:42 -08:00
Harshavardhana
59749a2b85
erasure: Remove prefix based listing support on ListMultipartUploads (#5248)
Previously we have removed this support under FS on #4996,
deprecate this on erasure coded backend as well to simplify
our multipart support.
2017-11-30 15:58:46 -08:00
Michael Lynch
fc3cf97b81 Removing isValidObjectName from Sia gateway (#5243)
This check incorrectly rejects most valid filenames. The only filenames Sia
forbids are leading forward slashes and path traversal characters, but it's
better to simply allow Sia to reject invalid names on its own rather than try
to anticipate errors from Sia:

https://github.com/NebulousLabs/Sia/blob/master/doc/api/Renter.md#path-parameters-4
2017-11-30 14:43:21 -08:00
Harshavardhana
d45a8784fc
Fix hash order to generate more even distribution (#5247)
The problem in existing code was the following line

```
start := int(keyCrc%uint32(cardinality)) | 1
```

A given a value of N cardinality the ending result
because of the the bitwise '|' would lead to always
higher affinity to odd sequences.

As can be seen from the test cases that this can
lead to many objects being allocated the same set
of disks or atleast the first disk is an odd disk
always.  This introduces a performance problem
for majority of the objects under concurrent load.

Remove `| 1` to provide a more cleaner distribution
and the new code will be.
```
start := int(keyCrc % uint32(cardinality))
```

Thanks to Krishna Srinivas for pointing out the bitwise
situation here.
2017-11-30 12:57:03 -08:00
Nitish Tiwari
6d7319380c Add Transparent Hugepage information (#5246)
Fixes #5242
2017-11-30 12:18:00 -08:00
Krishna Srinivas
14e6c5ec08 Simplify the steps to make changes to config.json (#5186)
This change introduces following simplified steps to follow 
during config migration.

```
 // Steps to move from version N to version N+1
 // 1. Add new struct serverConfigVN+1 in config-versions.go
 // 2. Set configCurrentVersion to "N+1"
 // 3. Set serverConfigCurrent to serverConfigVN+1
 // 4. Add new migration function (ex. func migrateVNToVN+1()) in config-migrate.go
 // 5. Call migrateVNToVN+1() from migrateConfig() in config-migrate.go
 // 6. Make changes in config-current_test.go for any test change
```
2017-11-29 13:12:47 -08:00
A. Elleuch
98d07210e7 fix: Ignore logging some tcp routine errors (#5097) 2017-11-28 13:51:17 -08:00
Nitish Tiwari
6923630389
Update bucket notification docs to mention events supported (#5235)
Fixes #4898
2017-11-28 15:41:26 +05:30
Nitish Tiwari
0c73c81919
Cleanup TLS setup document (#5231)
Fixes #4959
2017-11-28 15:15:50 +05:30
Harshavardhana
a46b640da3 gateway/sia: Support proper {make,get}Bucket operations (#5229)
Current implementation we faked the makeBucket operations
to allow for s3 clients to behave properly. But instead
we can create a placeholder zero byte file instead, which
is a hexadecimal representation of the bucket name itself.
2017-11-28 13:40:44 +05:30
Krishna Srinivas
71f9d2beff Increase maximum size of PUT request to 5TB (#5241)
fixes #5148
2017-11-28 12:59:02 +05:30
Michael Lynch
cf414a6053 Fixing Sia file uploads (#5233)
The Sia gateway had a bug with uploading that prevented the user's uploads
from reaching the Sia backend. The PutObject function called fsRemoveFile at
the end of the function, which didn't give the Sia backend enough time to
upload the file to the Sia network.

This adds a goroutine that watches the file upload progress and doesn't delete
the file until the upload reaches 100% complete.

Note that this solution has the limitation where if the minio process dies in
the middle of upload, it will leave orphaned files in the SIA_TEMP directory
that the user will need to remove manually.
2017-11-28 12:25:15 +05:30
Harshavardhana
05b395e81d
Add more unit tests for azure/gcs/b2 gateway (#5236)
Also adds a blazer SDK update exposing
error response headers.
2017-11-27 18:29:22 -08:00
Paul Nicholls
6a2d7ae808 gateway/azure: ListParts return an empty list if no parts uploaded yet (#5230)
This makes azure ListParts implementation behave more like S3 
by returning an empty list rather than an error when no parts have
been uploaded yet.
2017-11-27 17:42:27 -08:00
Eduardo Reyes
685afb6749 Fixes last item in the list menu being blocked. (#5234)
This commit fixes an issue where the last item's menu on a list of files
that scrolls gets blocked by the floating add button.

The fix is simply add the same padding that we use for the responsive
view, since it works just fine in responsive.
2017-11-27 14:02:57 -08:00
Harshavardhana
8efa82126b
Convert errors tracer into a separate package (#5221) 2017-11-25 11:58:29 -08:00
Frank Wessels
6e6aeb6a9e Updated version of klauspost/reedsolomon using proper AVX2 instructions as well a providing support for Cauchy matrices. (#5215) 2017-11-24 14:23:20 -08:00
Arnaud de Mouhy
c9e00ae0a5 Add support for dynamic address in docker healthcheck (#5228) 2017-11-24 14:11:26 +05:30
Nitish Tiwari
08e0698b7e
Update docs to latest Minio server release (#5227) 2017-11-23 10:19:53 +05:30
Harshavardhana
135a6a7bb4 Add chinese translation docs. (#5224) 2017-11-22 15:15:40 -08:00
Dee Koder
8b4d7048f8 fixed typo variable under domain heading. (#5223) 2017-11-22 12:37:00 -08:00
David G
f4d4ea5c36 Implement Sia Gateway (#5114) 2017-11-22 12:12:10 -08:00
Aditya Manthramurthy
d1a6c32d80 Improve make and make install messages (#5207) 2017-11-21 16:22:01 -08:00
Krishna Srinivas
bbd05a8f1c gateway-gcs: Close the writer with error in case of any errors. (#5217)
fixes #5216
2017-11-21 14:45:37 -08:00
Krishna Srinivas
4393afb7e2 Remove checkGCSProjectID() as it needs extra permission setting (#5210)
fixes #5209
2017-11-21 10:43:39 -08:00
Krishna Srinivas
1a53734477 Rename UserDefined to UserMetadata for events (#5206)
fixes #5165
2017-11-20 15:32:25 -08:00