Commit Graph

5028 Commits

Author SHA1 Message Date
kannappanr
2853fa1882 Remove logger field info from docs (#5281)
Logger field is removed from the docs,
as it has been removed from the config file.
2017-12-11 13:20:05 -08:00
Harshavardhana
1672c73988 Fix snap docs with instructions to run (#5266) 2017-12-07 08:38:58 +05:30
Harshavardhana
5476415016 Remove unused deps for cloudresourcemanager (#5274) 2017-12-06 16:41:56 -08:00
Kaan Kabalak
67ac74471d Fix error message on browser window resize when user has no buckets (#5275)
This commit handles the case where the list of buckets is empty on the
listObjects function of actions.js

Fixes #5267
2017-12-06 15:52:23 -08:00
kannappanr
a1c1a18dc5 Remove "logger" field from config.json (#5268)
File logging removed as part of improvement to server logging.

config.json format updated to version 21.

Fixes #5176
2017-12-06 12:48:29 +05:30
Harshavardhana
eb2894233c Convert gateways into respective packages (#5200)
- Make azure gateway a package
- Make b2 gateway a package
- Make gcs gateway a package
- Make s3 gateway a package
- Make sia gateway a package
2017-12-05 17:58:09 -08:00
Nitish Tiwari
52e382b697 Update go version in snap file (#5256) 2017-12-04 15:49:31 -08:00
techknowlogick
0d435e11b1 Change container name in b2 docs (#5259) 2017-12-04 22:34:00 +05:30
Harshavardhana
2755a0b763 Check if SSL is configured to validate input arguments (#5252)
This PR handles following situations

- secure endpoints provided, server should fail to start
  if TLS is not configured

- insecure endpoints provided, server starts ignoring
  if TLS is configured or not.

Fixes #5251
2017-12-04 12:17:12 +05:30
Aditya Manthramurthy
043e030a4a Add CopyObjectPart support to gateway S3 (#5213)
- Adds a metadata argument to the CopyObjectPart API to facilitate
  implementing encryption for copying APIs too.

- Update vendored minio-go - this version implements the
  CopyObjectPart client API for use with the S3 gateway.

Fixes #4885
2017-12-02 08:33:59 +05:30
Harshavardhana
490c30f853
erasure: Support cleaning up of stale multipart objects (#5250)
Just like our single directory/disk setup, this PR brings
the functionality to cleanup stale multipart objects
older > 2 weeks.
2017-11-30 18:11:42 -08:00
Harshavardhana
59749a2b85
erasure: Remove prefix based listing support on ListMultipartUploads (#5248)
Previously we have removed this support under FS on #4996,
deprecate this on erasure coded backend as well to simplify
our multipart support.
2017-11-30 15:58:46 -08:00
Michael Lynch
fc3cf97b81 Removing isValidObjectName from Sia gateway (#5243)
This check incorrectly rejects most valid filenames. The only filenames Sia
forbids are leading forward slashes and path traversal characters, but it's
better to simply allow Sia to reject invalid names on its own rather than try
to anticipate errors from Sia:

https://github.com/NebulousLabs/Sia/blob/master/doc/api/Renter.md#path-parameters-4
2017-11-30 14:43:21 -08:00
Harshavardhana
d45a8784fc
Fix hash order to generate more even distribution (#5247)
The problem in existing code was the following line

```
start := int(keyCrc%uint32(cardinality)) | 1
```

A given a value of N cardinality the ending result
because of the the bitwise '|' would lead to always
higher affinity to odd sequences.

As can be seen from the test cases that this can
lead to many objects being allocated the same set
of disks or atleast the first disk is an odd disk
always.  This introduces a performance problem
for majority of the objects under concurrent load.

Remove `| 1` to provide a more cleaner distribution
and the new code will be.
```
start := int(keyCrc % uint32(cardinality))
```

Thanks to Krishna Srinivas for pointing out the bitwise
situation here.
2017-11-30 12:57:03 -08:00
Nitish Tiwari
6d7319380c Add Transparent Hugepage information (#5246)
Fixes #5242
2017-11-30 12:18:00 -08:00
Krishna Srinivas
14e6c5ec08 Simplify the steps to make changes to config.json (#5186)
This change introduces following simplified steps to follow 
during config migration.

```
 // Steps to move from version N to version N+1
 // 1. Add new struct serverConfigVN+1 in config-versions.go
 // 2. Set configCurrentVersion to "N+1"
 // 3. Set serverConfigCurrent to serverConfigVN+1
 // 4. Add new migration function (ex. func migrateVNToVN+1()) in config-migrate.go
 // 5. Call migrateVNToVN+1() from migrateConfig() in config-migrate.go
 // 6. Make changes in config-current_test.go for any test change
```
2017-11-29 13:12:47 -08:00
A. Elleuch
98d07210e7 fix: Ignore logging some tcp routine errors (#5097) 2017-11-28 13:51:17 -08:00
Nitish Tiwari
6923630389
Update bucket notification docs to mention events supported (#5235)
Fixes #4898
2017-11-28 15:41:26 +05:30
Nitish Tiwari
0c73c81919
Cleanup TLS setup document (#5231)
Fixes #4959
2017-11-28 15:15:50 +05:30
Harshavardhana
a46b640da3 gateway/sia: Support proper {make,get}Bucket operations (#5229)
Current implementation we faked the makeBucket operations
to allow for s3 clients to behave properly. But instead
we can create a placeholder zero byte file instead, which
is a hexadecimal representation of the bucket name itself.
2017-11-28 13:40:44 +05:30
Krishna Srinivas
71f9d2beff Increase maximum size of PUT request to 5TB (#5241)
fixes #5148
2017-11-28 12:59:02 +05:30
Michael Lynch
cf414a6053 Fixing Sia file uploads (#5233)
The Sia gateway had a bug with uploading that prevented the user's uploads
from reaching the Sia backend. The PutObject function called fsRemoveFile at
the end of the function, which didn't give the Sia backend enough time to
upload the file to the Sia network.

This adds a goroutine that watches the file upload progress and doesn't delete
the file until the upload reaches 100% complete.

Note that this solution has the limitation where if the minio process dies in
the middle of upload, it will leave orphaned files in the SIA_TEMP directory
that the user will need to remove manually.
2017-11-28 12:25:15 +05:30
Harshavardhana
05b395e81d
Add more unit tests for azure/gcs/b2 gateway (#5236)
Also adds a blazer SDK update exposing
error response headers.
2017-11-27 18:29:22 -08:00
Paul Nicholls
6a2d7ae808 gateway/azure: ListParts return an empty list if no parts uploaded yet (#5230)
This makes azure ListParts implementation behave more like S3 
by returning an empty list rather than an error when no parts have
been uploaded yet.
2017-11-27 17:42:27 -08:00
Eduardo Reyes
685afb6749 Fixes last item in the list menu being blocked. (#5234)
This commit fixes an issue where the last item's menu on a list of files
that scrolls gets blocked by the floating add button.

The fix is simply add the same padding that we use for the responsive
view, since it works just fine in responsive.
2017-11-27 14:02:57 -08:00
Harshavardhana
8efa82126b
Convert errors tracer into a separate package (#5221) 2017-11-25 11:58:29 -08:00
Frank Wessels
6e6aeb6a9e Updated version of klauspost/reedsolomon using proper AVX2 instructions as well a providing support for Cauchy matrices. (#5215) 2017-11-24 14:23:20 -08:00
Arnaud de Mouhy
c9e00ae0a5 Add support for dynamic address in docker healthcheck (#5228) 2017-11-24 14:11:26 +05:30
Nitish Tiwari
08e0698b7e
Update docs to latest Minio server release (#5227) 2017-11-23 10:19:53 +05:30
Harshavardhana
135a6a7bb4 Add chinese translation docs. (#5224) 2017-11-22 15:15:40 -08:00
Dee Koder
8b4d7048f8 fixed typo variable under domain heading. (#5223) 2017-11-22 12:37:00 -08:00
David G
f4d4ea5c36 Implement Sia Gateway (#5114) 2017-11-22 12:12:10 -08:00
Aditya Manthramurthy
d1a6c32d80 Improve make and make install messages (#5207) 2017-11-21 16:22:01 -08:00
Krishna Srinivas
bbd05a8f1c gateway-gcs: Close the writer with error in case of any errors. (#5217)
fixes #5216
2017-11-21 14:45:37 -08:00
Krishna Srinivas
4393afb7e2 Remove checkGCSProjectID() as it needs extra permission setting (#5210)
fixes #5209
2017-11-21 10:43:39 -08:00
Krishna Srinivas
1a53734477 Rename UserDefined to UserMetadata for events (#5206)
fixes #5165
2017-11-20 15:32:25 -08:00
Andreas Auernhammer
e95c0bb913 return AWS compliant error if SSE-C key is wrong (#5203)
This PR changes the behavior of DecryptRequest.
Instead of returning `object-tampered` if the client provided
key is wrong DecryptRequest will return `access-denied`.

This is AWS S3 behavior.

Fixes #5202
2017-11-20 14:04:10 -08:00
Krishna Srinivas
fce556b8a0 Support for ListObjectParts in azure-gateway (#5198)
fixes #5169
2017-11-20 14:03:20 -08:00
Andreas Auernhammer
b97f99766f add benchmarks for erasure backend (#5084)
This change adds benchmarks for erasure read/write in different setups.
2017-11-17 14:57:04 -08:00
Nitish Tiwari
f7b6f7b22f Update getObjectInfo to stat for objects with trailing / (#5179)
Apache Spark sends getObject requests with trailing "/".
This PR updates the getObjectInfo to stat for files
even if they are sent with trailing "/".

Fixes #2965
2017-11-16 16:00:27 -08:00
Krishnan Parthasarathi
2a0a62b78d Return ErrContentSHA256Mismatch when sha256sum is invalid (#5188) 2017-11-16 11:13:04 -08:00
Krishnan Parthasarathi
67f66c40c1 Fix ListenBucketNotification deadlock (#5028)
Previously ListenBucketNotificationHandler could deadlock with
PutObjectHandler's eventNotify call when a client closes its
connection. This change removes the cyclic dependency between the
channel and map of ARN to channels by using a separate done channel to
signal that the client has quit.
2017-11-16 10:56:06 -08:00
Krishna Srinivas
5a2bdf6959 Handle Path validation inside the PostPolicy handler (#5192) 2017-11-15 14:10:45 -08:00
silenceshell
51e78a3e20 fix a typo (#5187) 2017-11-15 15:12:14 +05:30
Harshavardhana
9eb52ec7c7 Remove release scripts for minio. (#5181)
Use `GOOS=<osname> go build`  to build minio for any platform of choice.
2017-11-14 17:05:40 -08:00
Harshavardhana
0827a2747b api: CopyObject should return NotImplemented for now (#5183)
Commit ca6b4773ed introduces SSE-C
support for HEAD, GET, PUT operations but since we do not
implement CopyObject() we should return NotImplemented.
2017-11-14 16:57:19 -08:00
Krishna Srinivas
e7a724de0d Virtual host style S3 requests (#5095) 2017-11-14 16:56:24 -08:00
Krishna Srinivas
d57d57ddf5 Update ui-assets.go (#5185) 2017-11-14 16:22:35 -08:00
Harshavardhana
a4d6195244 Add public data-types for easier external loading (#5170)
This change brings public data-types such that
we can ask projects to implement gateway projects
externally than maintaining in our repo.

All publicly exported structs are maintained in object-api-datatypes.go

completePart --> CompletePart
uploadMetadata --> MultipartInfo

All other exported errors are at object-api-errors.go
2017-11-14 13:55:10 +05:30
Krishna Srinivas
7d3eaf79ff Set Minio user-agent for GCS calls (#5154) 2017-11-13 19:06:51 -08:00