Commit Graph

5794 Commits

Author SHA1 Message Date
Harshavardhana 285c09fe6b Support buckets with '.' with etcd+coreDNS (#7353)
Fixes #7340
2019-03-12 17:57:08 -07:00
kannappanr ce4563370c
Distributed: Allow healing if all disks are on root partitions (#7358)
If all the disks are on root partitions in distributed mode, consider it
to be a test setup and allow healing to proceed.

Fixes #7346
2019-03-12 16:47:06 -07:00
Aditya Manthramurthy e463386921 Add JSON Path expression evaluation support (#7315)
- Includes support for FROM clause JSON path
2019-03-09 08:13:37 -08:00
Aditya Manthramurthy b296b3cf8b Skip non-bucket dns entry in federated bucket list (#7339) 2019-03-08 14:22:58 -08:00
Harshavardhana 0b96ad4fdc
http2 throws custom error Content-Length shorter handle it (#7334)
We should internally handle when http2 input stream has smaller
content than its content-length header

Upstream issue reported https://github.com/golang/go/issues/30648

This a change which we need to handle internally until Go fixes it
correctly, till now our code doesn't expect a custom error to be returned.
2019-03-07 16:11:28 -08:00
Aditya Manthramurthy f4879ed96d Use jstream to serialize records to JSON format in S3Select (#7318)
- Also, switch to jstream to generate internal record representation
  from CSV/JSON readers

- This fixes a bug in which JSON output objects have their keys
  reversed from the order they are specified in the Select columns.

- Also includes a fix for tests.
2019-03-07 00:20:10 -08:00
Aditya Manthramurthy f97a33a63f Downgrade node temporarily (#7333)
To fix jest related build failure as recommended at https://github.com/facebook/jest/issues/8069#issuecomment-470307590
2019-03-06 16:05:49 -08:00
Minio Trusted 6387407e79 Update yaml files to latest version RELEASE.2019-03-06T22-47-10Z 2019-03-06 22:56:46 +00:00
Anis Elleuch b05825ffe8 s3: Fix precondition failed in CopyObjectPart when src is encrypted (#7276)
CopyObject precondition checks into GetObjectReader
in order to perform SSE-C pre-condition checks using the
last 32 bytes of encrypted ETag rather than the decrypted
ETag

This also necessitates moving precondition checks for
gateways to gateway layer rather than object handler check
2019-03-06 12:38:41 -08:00
kannappanr 39ddb78c75
CORS: Expose all headers on response (#7331)
Fixes #7289
2019-03-06 11:58:53 -08:00
Harshavardhana 12eb71828b Fix posix tests for SimpleCI (#7328) 2019-03-05 19:53:01 -08:00
Praveen raj Mani c0a1369b73 Construct dynamic XML error responses for postpolicyform validation (#7321)
Fixes #7314
2019-03-05 12:10:47 -08:00
kannappanr c57159a0fe
fs mode: List already existing buckets with capital letters (#7244)
if a bucket with `Captialized letters` is created, `InvalidBucketName` error
will be returned. 
In the case of pre-existing buckets, it will be listed.

Fixes #6938
2019-03-05 10:42:32 -08:00
Kale Blankenship ef132c5714 Replace snappy.Writer/io.Pipe with snappyCompressReader. (#7316)
Prevents deferred close functions from being called while still
  attempting to copy reader to snappyWriter.
 Reduces code duplication when compressing objects.
2019-03-05 08:35:37 -08:00
Aditya Manthramurthy c54b0c0ca1 Fix a race in tests (#7326) 2019-03-05 21:34:17 +05:30
Aditya Manthramurthy e8e9cd3e74 Close GlobalServiceDoneCh when quitting (#7322)
This change allows indefinitely running go-routines to cleanup
gracefully.

This channel is now closed at the beginning of each test so that
long-running go-routines quit and a new one is assigned.
2019-03-04 14:33:14 -08:00
Sidhartha Mani f7611bcd48 update simple-ci scripts (#7309) 2019-03-03 21:59:14 +05:30
Aditya Manthramurthy 19c10cb4d0 Update vendored sjson (includes a bug fix) (#7317) 2019-03-01 15:04:38 -08:00
poornas 6b4c6f69af Remove minio-go validator from vendor dir (#7312) 2019-02-28 12:16:26 -08:00
poornas 2564147ab4 Filter Expires header from user metadata (#7269)
Instead save it as a struct field in ObjectInfo as it is
a standard HTTP header - Fixes minio/mc#2690
2019-02-28 11:01:25 -08:00
Harshavardhana c3ca954684 Implement AssumeRole API for Minio users (#7267)
For actual API reference read here

https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html

Documentation is added and updated as well at docs/sts/assume-role.md

Fixes #6381
2019-02-27 17:46:55 -08:00
Harshavardhana ce588d1489 Improve ListObjects performance by listing in parallel (#7270)
The side affect of this change memory
increase, but this is a trade-off between
performance and actual memory usage.

For all practical scenarios this should be
an adequate change.
2019-02-27 14:39:22 -08:00
Sidhartha Mani b983da957d run gateway mint test in full mode (#7296) 2019-02-27 10:03:23 -08:00
Harshavardhana 5e69a107d8 Support detecting mountpoints correctly (#7288)
Currently windows support was relying on Symlink as
a way to detect a drive, this doesn't work in latest
Windows 2016, fix this to use a proper mechanism by
using win32 APIs.

Additionally also add support for detecting bind mounts
on Linux.
2019-02-26 18:04:53 -08:00
Dee Koder 5085bef397 Share button icon is changed (#7300) 2019-02-26 16:34:29 -08:00
poornas 8022a6efd9 Return ETag for 0-byte object prefixes (#7291)
Fixes: #7290
2019-02-26 15:09:14 -08:00
Minio Trusted d3125857a8 Update yaml files to latest version RELEASE.2019-02-26T19-51-46Z 2019-02-26 19:58:09 +00:00
Praveen raj Mani 78d116c487 Event persistence for MQTT (#7268)
- The events will be persisted in queueStore if `queueDir` is set.
- Else, if queueDir is not set events persist in memory.

The events are replayed back when the mqtt broker is back online.
2019-02-25 18:01:13 -08:00
dcharbonnier 2fc341394d fix json syntax (#7285) 2019-02-25 13:02:59 +05:30
Anis Elleuch 6584c7ea2b s3: Encode StartAfter when encoding type is passed (#7281)
In ObjectList V2, StartAfter needs to be encoded when encoding-type
is specified.
2019-02-24 18:50:28 -08:00
Harshavardhana 2520e535a0
Allow lazyQuotes for certain types of CSV (#7278)
Set lazyQuotes to true, to allow a quote to appear
in an unquote field and a non-doubled quote may
appear in a quoted field.
2019-02-24 06:51:02 -08:00
Anis Elleuch 5efbe8a1b3 s3: Add support of encodingType parameter (#7265)
This commit honors encoding-type parameter in object listing,
parts listing and multipart uploads listing.
2019-02-24 11:44:24 +05:30
Nitish Tiwari dab314900d Fix healthcheck script to wait for netstat command output (#7275)
Fixes #7272
2019-02-23 09:32:53 -08:00
Harshavardhana 7923b83953 Support multiple-domains in MINIO_DOMAIN (#7274)
Fixes #7173
2019-02-23 08:48:01 +05:30
Aditya Manthramurthy 80a351633f Update vendorized bcicen/jstream (#7257)
- Includes an error handling fix that is waiting to be merged upstream
- Uses order-preserving (un)marshalling for JSON objects.
2019-02-20 23:59:23 -08:00
Harshavardhana bedcb7442a Write xml.Header first instead of spaces to handle XML parsers (#7253)
Clients like AWS SDK Java and AWS cli XML parsers are
unable to handle on `\r\n` characters to avoid these
errors send XML header first and write white space characters
instead.

Also handle cases to avoid double WriteHeader calls
2019-02-21 11:50:15 +05:30
Harshavardhana 2232b0b55f Fix mac build failure for healthcheck binary (#7263) 2019-02-21 08:11:16 +05:30
Harshavardhana 91576d416d Fix GetLocalPeer usage in perf handlers (#7249)
GetLocalPeer usage should be fixed and used only
once per call for not all local endpoints.
2019-02-20 16:04:55 -08:00
Minio Trusted 3aabe45fd9 Update yaml files to latest version RELEASE.2019-02-20T22-44-29Z 2019-02-20 22:49:40 +00:00
Krishna Srinivas 6dd26b8231 Detect change in underlying mounted disks (#7229) 2019-02-20 13:32:29 -08:00
poornas e098852a80 Revert PR #7241 to fix vault renewal (#7259)
- Current implementation was spawning renewer goroutines
without waiting for the lease duration to end. Remove vault renewer
and call vault.RenewToken directly and manage reauthentication if
lease expired.
2019-02-20 12:23:59 -08:00
Nitish Tiwari 1e82c4a7c4
Implement Docker healthcheck script in Go (#7105)
Go script makes it easy to read/maintain. Also updated the timeout
in Dockerfiles from 5s to default 30s and test interval to 1m

Higher timeout makes sense as server may sometimes respond slowly
if under high load as reported in #6974

Fixes #6974
2019-02-20 21:42:03 +05:30
Krishna Srinivas ce960565b1 Validate and reject unusual requests (#7258) 2019-02-19 21:02:41 -08:00
poornas 755e675d5c Fix: send decrypted size to notification event (#7248) 2019-02-19 14:14:26 +05:30
Harshavardhana b6c00405ec Do not pro-actively return false in isObjectDir() (#7246)
We should change the logic for both isObject()
and isObjectDir() leaf detection to be done
with quorum, due to how our directory navigation
works - this allows for properly deleting all
the dangling directories or objects if any.
2019-02-15 16:21:19 -08:00
Harshavardhana 8f62935448 Add proper requestID for STS errors (#7245) 2019-02-14 17:54:33 -08:00
Harshavardhana 396d78352d Support HTTP/2.0 (#7204)
Fixes #6704
2019-02-14 17:53:46 -08:00
Aditya Manthramurthy 8a405cab2f COUNT() function in select should return an int (#7243) 2019-02-13 16:32:59 -08:00
Minio Trusted 6d778a883f Update yaml files to latest version RELEASE.2019-02-14T00-21-45Z 2019-02-14 00:27:52 +00:00
Harshavardhana a51781e5cf Use context to fill in more details about error XML (#7232) 2019-02-13 16:07:21 -08:00