Commit Graph

5320 Commits

Author SHA1 Message Date
Praveen raj Mani
5399d91965 Add version for appveyor to use go1.10.4 (#6384) 2018-08-30 14:09:31 +05:30
Harshavardhana
53a0bbeb5b
Encrypted reader wrapped in NewGetObjectReader should be closed (#6383) 2018-08-29 19:18:00 -07:00
Harshavardhana
384a862940
Return quorum error based on disks in abortMultipartUpload (#6362)
Fixes #4980
2018-08-29 13:36:19 -07:00
Harshavardhana
029f52880b
With no read quorum config should be treated as notFound (#6374)
This will allow the config subsystem to initialize properly
in situations where many servers are coming up in a rolling
fashion.
2018-08-28 14:23:22 -07:00
Harshavardhana
5b05df215a
Change SelectAPI to use new GetObjectNInfo API (#6373)
This PR also removes some double checks
2018-08-28 13:08:30 -07:00
Harshavardhana
a13cd7b7c4 Separate build functional tests from Makefile (#6351)
Recently travis seems to have issues with builds after
merge, so this PR is an experiment to fix this.

Failed: https://travis-ci.org/minio/minio/builds/419769285#L2310

Succeeded: https://travis-ci.org/minio/minio/builds/419565606#L2322

This PR was tested to be working properly on my fork to build
on travis, would need to do the same in minio/minio and see
how it goes through.
2018-08-28 13:57:01 +05:30
Anis Elleuch
d524924b80 Fix gateway s3 doc to run custom S3 endpoint (#6369)
To pass a custom S3 endpoint in S3 gateway, the user needs
to specify it as an argument after 'minio gateway s3' and not
as '--address' option since this latter specifies the address
to which the gateway should listen.
2018-08-28 10:09:07 +05:30
Aditya Manthramurthy
e6d740ce09 Implement GetObjectNInfo object layer call (#6290)
This combines calling GetObjectInfo and GetObject while returning a
io.ReadCloser for the object's body. This allows the two operations to
be under a single lock, fixing a race between getting object info and
reading the object body.
2018-08-27 15:28:23 +05:30
Harshavardhana
cea4f82586
Revert "Remove curl after the usage (#6347)" (#6366)
This reverts commit dafa5073cb.

This commit revert also fixes #6365
2018-08-26 23:40:21 -07:00
Minio Trusted
1d6ce115da Update yaml files to latest version RELEASE.2018-08-25T01-56-38Z 2018-08-25 02:04:10 +00:00
Anis Elleuch
06d2dfa31c Fix WORM and BROWSER status calculation (#6360)
One typo introduced in a recent commit miscalculates if worm and browser
are enabled or not. A simple test is also added to detect this issue
in the future if it ever happens again.
2018-08-24 14:36:14 -07:00
poornas
d547873b17 webhandler - display encryption errors properly (#6339)
For encrypted objects, download errors need to be
displayed in web response format instead of xml format.

Fixes #6327
2018-08-24 07:56:24 -07:00
Praveen raj Mani
01721a840a Build fix for go1.11rc1 (#6354)
Vendorized the following packages

- "github.com/rjeczalik/notify"
- "github.com/minio/highwayhash"

Fixes #6315
2018-08-24 13:29:17 +05:30
Krishna Srinivas
52f6d5aafc Rename of structs and methods (#6230)
Rename of ErasureStorage to Erasure (and rename of related variables and methods)
2018-08-23 23:35:37 -07:00
Harshavardhana
2211a5f1b8 Avoid ListenBucket targets to be listed in ServerInfo (#6340)
In current master when you do `mc watch` you can see a
dynamic ARN being listed which exposes the remote IP as well

```
mc watch play/airlines
```

On another terminal
```
mc admin info play
●  play.minio.io:9000
   Uptime : online since 11 hours ago
  Version : 2018-08-22T07:50:45Z
   Region :
 SQS ARNs : arn:minio:sqs::httpclient+51c39c3f-131d-42d9-b212-c5eb1450b9ee+73.222.245.195:33408
    Stats : Incoming 30GiB, Outgoing 7.6GiB
  Storage : Used 7.7GiB
```

SQS ARNs listed as part of ServerInfo should be only external targets,
since listing an ARN here is not useful and it cannot be re-purposed in
any manner.

This PR fixes this issue by filtering out httpclient from the ARN list.

This is a regression introduced in #5294 0e4431725c
2018-08-23 23:31:14 -07:00
Harshavardhana
1ffa6adcd4 Ignore io.EOF returned by ReadFrom for zero byte fs.json (#6346)
Fixes #6256
2018-08-24 11:34:21 +05:30
kannappanr
add57a6938
Add content-length as part of event notification structure (#6341)
Fixes #6321
2018-08-23 14:40:54 -07:00
Manjunath A Kumatagi
dafa5073cb Remove curl after the usage (#6347) 2018-08-23 10:48:29 -07:00
Praveen raj Mani
65e05a06fb Remove notifications Fix (#6082)
Remove all the notifications for an empty rulesMap

Fixes #6053
2018-08-23 22:53:18 +05:30
Dee Koder
5c9354894b Update Issue & PR Templates to have regression information when available (#6342) 2018-08-23 01:41:22 -07:00
Harshavardhana
b01e69e08f
Initialize global object layer after all subsystems have initialized (#6333)
This is to ensure that object API operations are not performed
on a server on which subsystems are yet to be initialized.
2018-08-22 23:11:17 -07:00
Raphael Randschau
8601f29d95 select: fix int overflow of math.MaxInt64 on ARM (#6317) 2018-08-22 16:16:04 +05:30
Nitish Tiwari
0aee722e3f
Fix Minio browser screenshots in docs (#6334)
Fixes #6308
2018-08-22 13:15:36 +05:30
Harshavardhana
beb6d40ce6
Avoid crash when policy subsystem is not initialized (#6326)
Fixes #6324
2018-08-21 15:38:51 -07:00
poornas
19db921555 CopyObject: fix regression in key rotation (#6331)
After key rotation, metadata was not being replaced with new sealed key.
Regression introduced in commit e71ef905f9
2018-08-21 15:12:00 -07:00
Minio Trusted
68b9e9e7e7 Update yaml files to latest version RELEASE.2018-08-21T00-37-20Z 2018-08-21 00:44:41 +00:00
kannappanr
2d84b02bc4 Check for absence of checksum field and attributes. (#6298)
Fixes #6295
2018-08-20 16:58:47 -07:00
poornas
8b2801bd46 Update documentation to show how to update minio config fields (#6301)
- with recent commit 1fb2e9ef95, config
can no longer be updated by editing config.json. This is because config
has been migrated inside the minio backend. Update documentation on
how to set/get configuration using mc admin config command.
2018-08-20 13:37:10 -07:00
Harshavardhana
7d7e21aebb Merge initConfig logic to ConfigSys (#6312) 2018-08-19 13:57:18 -07:00
Nitish Tiwari
bf14e5ce1b Fix distributed doc as total storage capacity is not displayed anymore (#6309) 2018-08-18 10:01:24 -07:00
Andreas Auernhammer
d531080b7e add SSE-KMS not-implemented error handling (#6234)
This commit adds error handling for SSE-KMS requests to
HEAD, GET, PUT and COPY operations. The server responds
with `not implemented` if a client sends a SSE-KMS
request.
2018-08-17 21:07:19 -07:00
Harshavardhana
a6b8a5487a Fail gateway properly with an error on port conflicts (#6303) 2018-08-17 21:06:36 -07:00
Minio Trusted
6c0d53a1c5 Update yaml files to latest version RELEASE.2018-08-18T03-49-57Z 2018-08-18 03:54:06 +00:00
Harshavardhana
9f14433cbd Ensure that setConfig uses latest functionality (#6302) 2018-08-17 18:51:34 -07:00
Harshavardhana
50a817e3d3 Use new listener which implements enhanced tcp features (#6289)
This package provide customizable TCP net.Listener with various
performance-related options:

 * SO_REUSEPORT. This option allows linear scaling server performance
   on multi-CPU servers.
   See https://www.nginx.com/blog/socket-sharding-nginx-release-1-9-1/ for details.
 * TCP_DEFER_ACCEPT. This option expects the server reads from the accepted
   connection before writing to them.
 * TCP_FASTOPEN. See https://lwn.net/Articles/508865/ for details.
2018-08-17 18:44:02 -07:00
Harshavardhana
5a4a57700b Add select docs and fix return values for Select API (#6300) 2018-08-17 17:11:39 -07:00
Harshavardhana
3de5a3157f Enhance picking valid xlMeta based on quorum (#6297)
This PR borrows the idea from getFormatXLQuorum()
2018-08-17 14:42:04 -07:00
Kaan Kabalak
50dec08002 Correct link paths in Chinese documentation (#6299) 2018-08-17 13:16:17 -07:00
poornas
e71ef905f9 Add support for SSE-S3 server side encryption with vault (#6192)
Add support for sse-s3 encryption with vault as KMS.

Also refactoring code to make use of headers and functions defined in
crypto package and clean up duplicated code.
2018-08-17 12:52:14 -07:00
junpeng liu
3d197c1449 Modify several translation errors (#6038) 2018-08-17 12:04:09 +05:30
Harshavardhana
65de2d68c0 Allow for proper garbage collection pooling bytes.Buffer (#6266) 2018-08-16 18:37:43 -07:00
Harshavardhana
1103ad2d08 Watch for symlinked certs in container envs (#6282)
Fixes #6278
2018-08-16 18:37:21 -07:00
Harshavardhana
eab947cf42 Make sure to update modTime in erasure metadata (#6296)
This is to ensure that when we update xl.json with new
parts have the latest modtime, in-turn avoids consistency
issues when the disk is offline.
2018-08-16 17:55:01 -07:00
Harshavardhana
0fe9e95250 Validate prefixes on all sets (#6294)
This PR fixes a regression introduced in 8eb838bf91
where hashing technique was used on prefixes to get the right set
to perform the operation, this is not correct since prefixes and
their corresponding keys might hash to a different value depending
on the key length.

For prefixes/directories we should look everywhere to support proper
quorum based listing.

Fixes #6293
2018-08-16 16:49:38 -07:00
kannappanr
c7946ab9ab
Remove unnecessary error log messages (#6186) 2018-08-16 12:57:49 -07:00
Harshavardhana
f5df3b4795 Remove select docs (#6287)
Select API is sufficiently documented, this doc is also incomplete.

- https://aws.amazon.com/blogs/aws/s3-glacier-select/
- https://aws.amazon.com/blogs/developer/introducing-support-for-amazon-s3-select-in-the-aws-sdk-for-ruby/
- https://aws.amazon.com/blogs/developer/introducing-support-for-amazon-s3-select-in-the-aws-sdk-for-javascript/
- https://aws.amazon.com/blogs/developer/category/storage/s3-select/
2018-08-15 19:47:22 -07:00
Harshavardhana
f26325c988 Support supplying custom drives per set count (#6261) 2018-08-15 16:35:21 -07:00
Arjun Mishra
7c14cdb60e S3 Select API Support for CSV (#6127)
Add support for trivial where clause cases
2018-08-15 03:30:19 -07:00
Harshavardhana
0e02328c98 Migrate config.json from config-dir to backend (#6195)
This PR is the first set of changes to move the config
to the backend, the changes use the existing `config.json`
allows it to be migrated such that we can save it in on
backend disks.

In future releases, we will slowly migrate out of the
current architecture.

Fixes #6182
2018-08-15 10:11:47 +05:30
Harshavardhana
380524ae27
Unlock read lock on uploadID upon errors (#6283) 2018-08-14 18:35:30 -07:00