Commit Graph

4998 Commits

Author SHA1 Message Date
Krishna Srinivas 13ab8e17e2 gateway-gcs: use minio.sys.temp/multipart/v1 as url base (#4562) 2017-06-21 10:27:44 -07:00
Nitish Tiwari 9d3d64df2d Added HEALTHCHECK in release Dockerfiles (#4550)
A new script `healthcheck.sh` checks for http status 200/404
2017-06-20 19:24:45 -07:00
Krishnan Parthasarathi 146bc3e638 Add MINIO_REGION to server help message (#4558)
* Add e.g for setting MINIO_REGION env variable
* Add MINIO_REGION to region table
2017-06-20 15:02:18 -07:00
Harshavardhana 0543d45fb3 fix: OwnerID in response should be 64 character in length. (#4554)
Rather than sending a custom "minio" string, we can
change this to `sha256('arn:aws:iam::minio:user/admin')`.

Fixes #4553
2017-06-20 15:01:13 -07:00
Krishnan Parthasarathi fe426944ea Fix GCS help message (#4570) 2017-06-20 14:25:16 -07:00
Aditya Manthramurthy c1a6ca0c33 Fix spelling of function name to `startLockMaintenance` (#4561) 2017-06-20 12:10:02 -07:00
Harshavardhana 5a78266821 gateway/gcs: Complete minio browser support for gcs. (#4552)
Fixes #4460
2017-06-19 19:45:13 -07:00
Harshavardhana f5b4b0765a Update minio-go dependency (#4551)
This updates dependency for

 - AWS S3 backend.
 - pkg/madmin

```
- Relax isValidBucketName to allow reading existing buckets. (#708) (3 minutes ago) <Harshavardhana>
- For GCS the size limit of S3 is not useful. (#711) (3 days ago) <Harshavardhana>
- s3utils: Support AWS S3 US GovCloud endpoint. (#701) (3 days ago) <Harshavardhana>
- api: Always strip 80/443 port from host (#709) (3 days ago) <Anis Elleuch>
- Redact signature strings properly. (#706) (4 days ago) <Harshavardhana>
- api: Single putObject can use temporary file always. (#703) (6 days ago) <Harshavardhana>
- Spelling fix (#704) (7 days ago) <Jacob Taylor>
- api/encrypt: Get() on encrypted object should be a reader. (#699) (2 weeks ago) <Harshavardhana>
- get: Fix reading an object if its size is unknown (#694) (3 weeks ago) <Anis Elleuch>
- fixes #696 by updating the examples for put-encrypted-object and get-encrypted-object (#697) (3 weeks ago) <Tejay Cardon>
- fix InvalidAccessKeyId error according to amazon documentation (#692) (4 weeks ago) <samkevich>
- Add AWS S3 SSE-C example. (#689) (4 weeks ago) <Harshavardhana>
- According to RFC7232 Etag should be in quotes for If-Match. (#688) (5 weeks ago) <Harshavardhana>
- api: getReaderSize() should honor seeked file descriptors. (#681) (5 weeks ago) <Harshavardhana>
- tests: Use bytes.Repeat() when generating big data (#683) (5 weeks ago) <Anis Elleuch>
- api: Failed call retry with region only when http.StatusBadRequest. (#678) (5 weeks ago) <Harshavardhana>
- api: Add NewWithCredentials() (#646) (5 weeks ago) <Harshavardhana>
```
2017-06-19 16:02:35 -07:00
Krishna Srinivas 3928c1e14c gateway/gcs: Change in multipart backend format (#4455) 2017-06-17 16:00:41 -07:00
Krishnan Parthasarathi 94241cd153 Update Minio on DC/OS doc for latest Minio package (#4549) 2017-06-17 12:21:37 -07:00
Harshavardhana a86c2e2ce1 xl/fs: Return InvalidPart{} error for part ETag mismatch. (#4541)
Fixes #4539
2017-06-17 11:20:39 -07:00
Harshavardhana e99244be02 xl: prepare storage should Abort properly. (#4542)
Current state-machine didn't honor a situation
which can arise when there is a combination of

 - formatted
 - unformatted
 - corrupted

disks - this combination invariably goes into a
mode where all servers are waiting perpetually
forever thinking we will get quorum in future.

At this point there is a distant possibility of
ever getting a quorum since we don't even have
quorum number of disks offline.

We should exit and print a proper message per disk
to indicate what went wrong and what was detected
by the server.

Refer #4477
2017-06-17 11:20:12 -07:00
Nitish Tiwari 58833711e0 Added ListObjectsV2 and ListObjectsV2 Anon support to Gateway S3 and Azure. (#4547) 2017-06-16 22:17:00 -07:00
Harshavardhana f99f218999 Add support for reading and saving config on Gateway. (#4463)
This is also a first step towards supporting bucket
notification for gateway.
2017-06-16 16:01:41 -07:00
Krishnan Parthasarathi 4fb5fc72d7 GCS gateway allows apps to supply their own marker (#4495)
Most s3 compatible apps use object keys returned in listing as
marker. This change allows this behaviour with gateway-gcs too.
2017-06-16 15:02:07 -07:00
Remco Verhoef d86973dcca Allow bucket creation in different regions, closes #4287 and #4241
* I needed to remove the region check from PutBucketHandler
2017-06-16 15:02:07 -07:00
Krishnan Parthasarathi 8085ba4494 Filter out internal object prefix during listing (#4435)
We use ZZZZ-Minio/ prefix internally in our GCS gateway which should be
filtered out in the response to ListObjects.
2017-06-16 15:02:07 -07:00
poornas 9bd0eb1a9e Set default ETag value if vendor returns empty md5 string (#4409)
The ETag is constructed from md5 atttribute of object attributes
returned by the vendor's Composer. The md5 attribute comes back
as nil for large uploads. Instead the CRC32C should be used.

Refer to https://cloud.google.com/storage/docs/hashes-etags

Fixes #4397
2017-06-16 15:02:07 -07:00
Anis Elleuch e4e0abfc05 fix: Check project id before starting gateway (#4412) 2017-06-16 15:02:07 -07:00
poornas 12b2fc894b Remove profile option for gcs from gateway help message (#4421) 2017-06-16 15:01:34 -07:00
Krishna Srinivas 2aa76e7407 Change md5Sum to etag (#4399) 2017-06-16 14:58:49 -07:00
Remco Verhoef 0dab038858 Cleanup and update the PR with the master branch. 2017-06-16 14:55:32 -07:00
Remco Verhoef a76556ec1b Map only default region us-east-1 to gcs us region 2017-06-16 14:54:37 -07:00
Harshavardhana 91c7bb65c5 gateway/gcs: send proper error responses for Get/SetBucket policies. (#4338)
Fixes #4323
2017-06-16 14:54:37 -07:00
Anis Elleuch 5d602034ea gateway: Use default params when no args provided (#4315)
For S3 & Azure, use default parameters when no arguments (endpoint) are
provided. This also avoids a crash.
2017-06-16 14:54:37 -07:00
Nitish Tiwari b829ec4a6b Fixes https://github.com/minio/minio/issues/4320 (#4332)
- Add description for error ErrBucketAlreadyExists
2017-06-16 14:54:37 -07:00
Remco Verhoef 9c50a9f567 Fix ListObjectParts to list properly all parts - closes #4322 2017-06-16 14:54:37 -07:00
Remco Verhoef 52122c0309 Fix uploadIDMarker handling. 2017-06-16 14:54:37 -07:00
Remco Verhoef 3b9d313c87 Fix issue with AbortMultipartUpload, closes #4322 2017-06-16 14:54:37 -07:00
Remco Verhoef bfff251e2a Fix issue with UNSIGNED payloads.
Additionally also fixes escaping slashes in
temporary multipart names
2017-06-16 14:54:37 -07:00
Remco Verhoef 52b500cce9 Verify multipart etag during complete, closes #4288 2017-06-16 14:54:37 -07:00
Remco Verhoef c63cdca11f Support iterating through ListObjectParts using NextPartNumberMarker, closes #4284 2017-06-16 14:54:37 -07:00
Remco Verhoef 4430085981 Add region to gcs gateway example 2017-06-16 14:54:37 -07:00
Remco Verhoef 5c78415b31 Verify md5 content hash, closes #4285 2017-06-16 14:54:37 -07:00
Remco Verhoef bd67117756 Use maxKeys for iterator 2017-06-16 14:54:37 -07:00
Remco Verhoef f3e5e9fb29 Support marker, closes #4286 2017-06-16 14:54:37 -07:00
Remco Verhoef 2de1921fe8 Use MINIO_REGION environment variable for region configuration, closes #4287 2017-06-16 14:54:37 -07:00
Remco Verhoef dd7e47f264 Add access and secret key to example, needed to access Minio Gateway 2017-06-16 14:54:37 -07:00
Remco Verhoef fe9d826bef Implement bucket policies 2017-06-16 14:53:36 -07:00
Remco Verhoef 6dbc5aba09 Return correct error when PutObject fails 2017-06-16 14:53:36 -07:00
Remco Verhoef de5374f74c Map S3 regions to Google (multi)regions 2017-06-16 14:53:36 -07:00
Remco Verhoef bf55591c64 Make every backend responsible for parsing its own arguments, fixes #4293 2017-06-16 14:53:36 -07:00
Remco Verhoef 2d814e340f Return BucketAlreadyExists when bucket exists with another user 2017-06-16 14:53:36 -07:00
Remco Verhoef 0a8cf1a6b0 Allow bucket creation in different regions, closes #4287 and #4241
* I needed to remove the region check from PutBucketHandler
2017-06-16 14:53:36 -07:00
Remco Verhoef 07949f68d8 Translate gcs errors to S3 compatible errors, fixes #4278 2017-06-16 14:53:36 -07:00
Remco Verhoef 909a89647b Use default endpoint when not supplied 2017-06-16 14:53:36 -07:00
Remco Verhoef 6508da5fde Add usage for GCS gateway, closes #4280 2017-06-16 14:53:36 -07:00
Remco Verhoef 3379f005a5 Initial implementation of Google Cloud Storage 2017-06-16 14:47:02 -07:00
Remco Verhoef 4be609eb82 Added AllAccessDisabled error 2017-06-16 14:47:02 -07:00
Remco Verhoef dd5b975001 Add comment, gateway should validate object name 2017-06-16 14:47:02 -07:00