Commit Graph

5959 Commits

Author SHA1 Message Date
kannappanr 1de3bd6911
Save http trace to a file (#5300)
Save http trace to a file instead of displaying it onto the console.
the environment variable MINIO_HTTP_TRACE will be a filepath instead
of a boolean.

This to handle the scenario where both json and http tracing are
turned on. In that case, both http trace and json output are displayed
on the screen making the json not parsable. Loging this trace onto
a file helps us avoid that scenario.

Fixes #5263
2018-01-05 11:24:31 -08:00
Kaan Kabalak de2ce5acb4 Correct color code of Excel file icon (#5361)
Fixes #5352
2018-01-05 16:19:41 +05:30
Paul Stack a1a98617ca gateway/manta: Add support for RBAC (#5332)
Manta has the ability to allow users to authenticate with a 
username other than the main account. We want to expose 
this functionality to minio manta gateway.
2018-01-05 13:30:29 +05:30
Andreas Auernhammer b85c75996d add support for encrypted TLS private keys (#5308)
This change adds support for password-protected private keys.
If the private key is encrypted the server tries to decrypt
the key with the password provided by the env variable 
MINIO_CERT_PASSWD.

Fixes #5302
2018-01-05 13:18:08 +05:30
Harshavardhana cc2497f52f Exitcode with '1' when update is available (#5354)
--quiet should simply update the binary without any prompt.

Fixes #5347
2018-01-04 21:26:43 +05:30
Nitish Tiwari 1e5fb4b79a
Fix storage class related issues (#5338)
- Update startup banner to print storage class in capitals. This
makes it easier to identify different storage classes available.

- Update response metadata to not send STANDARD storage class.
This is in accordance with AWS S3 behaviour.

- Update minio-go library to bring in storage class related
changes. This is needed to make transparent translation of
storage class headers for Minio S3 Gateway.
2018-01-04 11:44:45 +05:30
kannappanr 6f7c6fc560 Honor browser enabled config value in startup message (#5313)
Currently, browser access information is displayed without checking
if browser enabled flag is turned off in config.json. Fixing it to
hide the information if the flag is turned off.

Fixes #5312
2018-01-04 11:00:52 +05:30
Harshavardhana c0721164be Automatically set goroutines based on shardSize (#5346)
Update reedsolomon library to enable feature to automatically
set number of go-routines based on the input shard size,
since shard size is sort of a constant in Minio for
objects > 10MiB (default blocksize)

klauspost reported around 15-20% improvement in performance
numbers on older systems such as AVX and SSE3

```
name                  old speed      new speed      delta
Encode10x2x10000-8    5.45GB/s ± 1%  6.22GB/s ± 1%  +14.20%    (p=0.000 n=9+9)
Encode100x20x10000-8  1.44GB/s ± 1%  1.64GB/s ± 1%  +13.77%  (p=0.000 n=10+10)
Encode17x3x1M-8       10.0GB/s ± 5%  12.0GB/s ± 1%  +19.88%  (p=0.000 n=10+10)
Encode10x4x16M-8      7.81GB/s ± 5%  8.56GB/s ± 5%   +9.58%   (p=0.000 n=10+9)
Encode5x2x1M-8        15.3GB/s ± 2%  19.6GB/s ± 2%  +28.57%   (p=0.000 n=9+10)
Encode10x2x1M-8       12.2GB/s ± 5%  15.0GB/s ± 5%  +22.45%  (p=0.000 n=10+10)
Encode10x4x1M-8       7.84GB/s ± 1%  9.03GB/s ± 1%  +15.19%    (p=0.000 n=9+9)
Encode50x20x1M-8      1.73GB/s ± 4%  2.09GB/s ± 4%  +20.59%   (p=0.000 n=10+9)
Encode17x3x16M-8      10.6GB/s ± 1%  11.7GB/s ± 4%  +10.12%   (p=0.000 n=8+10)
```
2018-01-03 13:47:22 -08:00
Minio Trusted b1fb550d5c Update yaml files to latest version RELEASE.2018-01-02T23-07-00Z 2018-01-02 23:11:15 +00:00
Andreas Auernhammer a6318dbdaf fix timing oracle attack against signature V2/V4 verification (#5335)
This change replaces the non-constant time comparison of
request signatures with a constant time implementation. This
prevents a timing attack which can be used to learn a valid 
signature for a request without knowing the secret key.

Fixes #5334
2018-01-02 12:00:02 +05:30
Harshavardhana e39d7ddb0f Fix PostPolicy form tests without hardcoded dates (#5337)
Fixes #5336
2018-01-01 07:28:10 +05:30
Kaan Kabalak 659f724f4c Integrate existing remove bucket functionality from newux to current UI (#5289)
This commit takes the existing remove bucket functionality written by
brendanashworth, integrates it to the current UI with a dropdown for
each bucket, and fixes small issues that were present, like the dropdown
not disappearing after the user clicks on 'Delete' for certain buckets.
This feature only deletes a bucket that is empty (that has no objects).

Fixes #4166
2017-12-29 18:45:44 +05:30
Nitish Tiwari baaf67d82e
Update config.json guide with details of version 22 (#5328)
Fixes #5296
2017-12-28 23:04:44 +05:30
A. Elleuch 2244adff07 fix: Better printing of XL config init error (#5284) 2017-12-28 23:02:48 +05:30
Nitish Tiwari e3d841ffd1
Fix config.json parsing to fetch correct storage class (#5327) 2017-12-28 14:19:45 +05:30
Minio Trusted 751632d79e Update yaml files to lastest version RELEASE.2017-12-28T01-21-00Z 2017-12-28 01:26:09 +00:00
Nitish Tiwari 545a9e4a82 Fix storage class related issues (#5322)
- Add storage class metadata validation for request header
- Change storage class header values to be consistent with AWS S3
- Refactor internal method to take only the reqd argument
2017-12-27 10:06:16 +05:30
Harshavardhana f25ec31565 Set maxResources appropriately for gateway like server (#5321) 2017-12-24 20:09:30 +05:30
Matthieu Paret 374feda237 add HTTPStats to madmin (#5299) 2017-12-22 17:47:30 -08:00
A. Elleuch 6ef0161835 fix: Restore empty files when healing (#5257)
HealFile() does not process the case when an empty file is lost in
some disks. Since, Reedsolomon erasure doesn't handle restoring empty
data, HealFile will create empty files similarly to CreateFile().
2017-12-22 14:57:57 -08:00
Nitish Tiwari 1a3dbbc9dd
Add x-amz-storage-class support (#5295)
This adds configurable data and parity options on a per object
basis. To use variable parity

- Users can set environment variables to cofigure variable
parity

- Then add header x-amz-storage-class to putobject requests
with relevant storage class values

Fixes #4997
2017-12-22 16:58:13 +05:30
Aditya Manthramurthy f1355da72e Add base64 encoded MD5 output for Hash Reader (#5315)
- Use it to send the Content-MD5 header correctly encoded to S3
  Gateway

- Fixes a bug in PutObject (including anonymous PutObject) and
  PutObjectPart with S3 Gateway found when testing with Mint.
2017-12-21 17:27:33 -08:00
Krishnan Parthasarathi bbe521ffec ReInitDisk RPC handler should use retryStorage (#5310) 2017-12-21 12:28:01 +05:30
Paul Stack 7d75d61621 Add Support for Manta Object Storage as a Gateway (#5025)
Manta is an Object Storage by [Joyent](https://www.joyent.com/)

This PR adds initial support for Manta. It is intended as non-production 
ready so that feedback can be obtained.
2017-12-20 13:37:56 +05:30
Harshavardhana 1f77708a30 Limit number of connections upto system maxlimit (#5109) 2017-12-20 13:30:14 +05:30
Timon Wong 84fc78d60f Implement Alibaba Cloud OSS gateway support (#5103) 2017-12-19 13:55:17 +05:30
poornas a182fe8c15 update steps to make changes to config.json (#5292) 2017-12-17 21:00:12 -08:00
Harshavardhana 819d1e80c6 Add more delays on distributed startup for slow network (#5240)
Refer #5237
2017-12-16 08:25:29 -08:00
Kaan Kabalak ffdf115bf2 Fix mobile issue where checkbox was seen over sidebar on list item hover (#5288)
Fixes #5287
2017-12-15 12:36:05 -08:00
Harshavardhana eb7c690ea9 Support in-place upgrades of new minio binary and releases. (#4961)
This PR allows 'minio update' to not only shows update banner
but also allows for in-place upgrades.

Updates are done safely by validating the downloaded
sha256 of the binary.

Fixes #4781
2017-12-15 12:33:42 -08:00
Nitish Tiwari 8c08571cd9
Update Kubernetes example yaml files (#5278)
Removed the non production ready Kubernetes constructs that are not needed
for standard Minio deployment. General cleanup of the documents.
2017-12-12 10:29:00 +05:30
kannappanr 2853fa1882 Remove logger field info from docs (#5281)
Logger field is removed from the docs,
as it has been removed from the config file.
2017-12-11 13:20:05 -08:00
Harshavardhana 1672c73988 Fix snap docs with instructions to run (#5266) 2017-12-07 08:38:58 +05:30
Harshavardhana 5476415016 Remove unused deps for cloudresourcemanager (#5274) 2017-12-06 16:41:56 -08:00
Kaan Kabalak 67ac74471d Fix error message on browser window resize when user has no buckets (#5275)
This commit handles the case where the list of buckets is empty on the
listObjects function of actions.js

Fixes #5267
2017-12-06 15:52:23 -08:00
kannappanr a1c1a18dc5 Remove "logger" field from config.json (#5268)
File logging removed as part of improvement to server logging.

config.json format updated to version 21.

Fixes #5176
2017-12-06 12:48:29 +05:30
Harshavardhana eb2894233c Convert gateways into respective packages (#5200)
- Make azure gateway a package
- Make b2 gateway a package
- Make gcs gateway a package
- Make s3 gateway a package
- Make sia gateway a package
2017-12-05 17:58:09 -08:00
Nitish Tiwari 52e382b697 Update go version in snap file (#5256) 2017-12-04 15:49:31 -08:00
techknowlogick 0d435e11b1 Change container name in b2 docs (#5259) 2017-12-04 22:34:00 +05:30
Harshavardhana 2755a0b763 Check if SSL is configured to validate input arguments (#5252)
This PR handles following situations

- secure endpoints provided, server should fail to start
  if TLS is not configured

- insecure endpoints provided, server starts ignoring
  if TLS is configured or not.

Fixes #5251
2017-12-04 12:17:12 +05:30
Aditya Manthramurthy 043e030a4a Add CopyObjectPart support to gateway S3 (#5213)
- Adds a metadata argument to the CopyObjectPart API to facilitate
  implementing encryption for copying APIs too.

- Update vendored minio-go - this version implements the
  CopyObjectPart client API for use with the S3 gateway.

Fixes #4885
2017-12-02 08:33:59 +05:30
Harshavardhana 490c30f853
erasure: Support cleaning up of stale multipart objects (#5250)
Just like our single directory/disk setup, this PR brings
the functionality to cleanup stale multipart objects
older > 2 weeks.
2017-11-30 18:11:42 -08:00
Harshavardhana 59749a2b85
erasure: Remove prefix based listing support on ListMultipartUploads (#5248)
Previously we have removed this support under FS on #4996,
deprecate this on erasure coded backend as well to simplify
our multipart support.
2017-11-30 15:58:46 -08:00
Michael Lynch fc3cf97b81 Removing isValidObjectName from Sia gateway (#5243)
This check incorrectly rejects most valid filenames. The only filenames Sia
forbids are leading forward slashes and path traversal characters, but it's
better to simply allow Sia to reject invalid names on its own rather than try
to anticipate errors from Sia:

https://github.com/NebulousLabs/Sia/blob/master/doc/api/Renter.md#path-parameters-4
2017-11-30 14:43:21 -08:00
Harshavardhana d45a8784fc
Fix hash order to generate more even distribution (#5247)
The problem in existing code was the following line

```
start := int(keyCrc%uint32(cardinality)) | 1
```

A given a value of N cardinality the ending result
because of the the bitwise '|' would lead to always
higher affinity to odd sequences.

As can be seen from the test cases that this can
lead to many objects being allocated the same set
of disks or atleast the first disk is an odd disk
always.  This introduces a performance problem
for majority of the objects under concurrent load.

Remove `| 1` to provide a more cleaner distribution
and the new code will be.
```
start := int(keyCrc % uint32(cardinality))
```

Thanks to Krishna Srinivas for pointing out the bitwise
situation here.
2017-11-30 12:57:03 -08:00
Nitish Tiwari 6d7319380c Add Transparent Hugepage information (#5246)
Fixes #5242
2017-11-30 12:18:00 -08:00
Krishna Srinivas 14e6c5ec08 Simplify the steps to make changes to config.json (#5186)
This change introduces following simplified steps to follow 
during config migration.

```
 // Steps to move from version N to version N+1
 // 1. Add new struct serverConfigVN+1 in config-versions.go
 // 2. Set configCurrentVersion to "N+1"
 // 3. Set serverConfigCurrent to serverConfigVN+1
 // 4. Add new migration function (ex. func migrateVNToVN+1()) in config-migrate.go
 // 5. Call migrateVNToVN+1() from migrateConfig() in config-migrate.go
 // 6. Make changes in config-current_test.go for any test change
```
2017-11-29 13:12:47 -08:00
A. Elleuch 98d07210e7 fix: Ignore logging some tcp routine errors (#5097) 2017-11-28 13:51:17 -08:00
Nitish Tiwari 6923630389
Update bucket notification docs to mention events supported (#5235)
Fixes #4898
2017-11-28 15:41:26 +05:30
Nitish Tiwari 0c73c81919
Cleanup TLS setup document (#5231)
Fixes #4959
2017-11-28 15:15:50 +05:30