Commit Graph

1189 Commits

Author SHA1 Message Date
Harshavardhana 2f9354b17e Use uuid.New() for uploadID in azure gateway (#5721) 2018-03-28 11:17:25 -07:00
Harshavardhana 6e9c853312 After healing re-load disks with the new format (#5718)
This PR also fixes correct calculation of drive states
before and after healing of objects.

Fixes #5700
Fixes #5708
2018-03-28 06:41:39 +05:30
poornas 76d1e8bbcd change fs.json format to include checksum fields (#5685) 2018-03-27 17:23:10 -07:00
Bala FA 3ebe61abdf Quick support to server level WORM (#5602)
This is a trival fix to support server level WORM.  The feature comes
with an environment variable `MINIO_WORM`.

Usage:
```
$ export MINIO_WORM=on
$ minio server endpoint
```
2018-03-27 16:44:45 -07:00
Nitish Tiwari 58291acb8d Fix object deletion where quorum is not available (#5709)
Object deletion should not be possible if quorum is not
available. This PR updates deleteObject() to check for
quorum errors before proceeding with object deletion.

Fixes #5535
2018-03-26 16:39:28 -07:00
Harshavardhana de44be86d0 Use readQuorum instead of writeQuorum to check bucket exists (#5715)
Fixes #5708
Fixes #5700
2018-03-26 16:36:57 -07:00
Harshavardhana 35e64573fa Add missing env doc for MINIO_DOMAIN (#5705) 2018-03-26 09:11:39 -07:00
kannappanr 7c747a9643
Return complete Location URL in CompleteMultipartUpload (#5692)
Remove getLocation function.
Fixes #5687
2018-03-23 13:46:57 -07:00
Nitish Tiwari cf67f1838e Add unit tests for endpoint-ellipses.go (#5681) 2018-03-22 16:23:54 -07:00
Kaan Kabalak a6adef0bdf Refactor bucket delete and bucket policy (#5580)
This commit adds the bucket delete and bucket policy functionalities
to the browser.

Part of rewriting the browser code to follow best practices and
guidelines of React (issues #5409 and #5410)

The backend code has been modified by @krishnasrinivas to prevent
issue #4498 from occuring. The relevant changes have been made to the
code according to the latest commit and the unit tests in the backend.
This commit also addresses issue #5449.
2018-03-21 11:38:56 -07:00
Andreas Auernhammer a4d2e2b428 [refactor] simplify en/decrypted size computation (#5658)
This commit replaces the en/decrypted size computation
with functions from the `sio` package.

Fixes #5657
2018-03-19 22:18:12 +05:30
Harshavardhana 2938e332ba Fix format migration regression (#5668)
Migration regression got introduced in 9083bc152e
adding more unit tests to catch this scenario, we need to fix this by
re-writing the formats after the migration to 'V3'.

This bug only happens when a user is migrating directly from V1 to V3,
not from V1 to V2 and V2 to V3.

Added additional unit tests to cover these situations as well.

Fixes #5667
2018-03-19 21:43:00 +05:30
Harshavardhana f23944aed7 Fix heal bucket deadlock after replacing disks (#5661)
Fixes #5659
2018-03-16 15:09:31 -07:00
Harshavardhana 3145462ad2 Return InvalidDigest when md5 sent by client is invalid (#5654)
This is to ensure proper compatibility with AWS S3, handle
special cases where

- Content-Md5 is set to empty
- Content-Md5 is set to invalid
2018-03-16 11:22:34 -07:00
Krishna Srinivas 9ede179a21 Use context.Background() instead of nil
Rename Context[Get|Set] -> [Get|Set]Context
2018-03-15 16:28:25 -07:00
Ersan 33fe42df8a Adds context support for nas, oss and s3 2018-03-15 16:28:25 -07:00
kannappanr bdb1a90720 Add context support for gateway b2, manta, gcs 2018-03-15 16:28:25 -07:00
Krishna Srinivas e452377b24 Add context to the object-interface methods.
Make necessary changes to xl fs azure sia
2018-03-15 16:28:25 -07:00
Krishna Srinivas 9083bc152e Flat multipart backend implementation for Erasure backend (#5447) 2018-03-15 13:55:23 -07:00
Bala FA 0e4431725c make notification as separate package (#5294)
* Remove old notification files

* Add net package

* Add event package

* Modify minio to take new notification system
2018-03-15 13:03:41 -07:00
Nitish Tiwari abffa00b76
Update healthcheck related examples and add head support (#5650)
- Add head method for healthcheck endpoint. Some platforms/users
may use the HTTP Head method to check for health status.

- Add liveness and readiness probe examples in Kubernetes yaml
example docs. Note that readiness probe not added to StatefulSet
example due to https://github.com/kubernetes/kubernetes/issues/27114
2018-03-15 09:55:02 +05:30
Harshavardhana 4af89543cf Update minio-go dependencies to latest 5.0.0 release (#5640)
With following changes

- Add SSE and refactor encryption API (#942) <Andreas Auernhammer>
- add copyObject test changing metadata and preserving etag (#944) <Harshavardhana>
- Add SSE-C tests for multipart, copy, get range operations (#941) <Harshavardhana>
- Removing conditional check for notificationInfoCh in api-notication (#940) <Matthew Magaldi>
- Honor prefix parameter in ListBucketPolicies API (#929) <kannappanr>
- test for empty objects uploaded with SSE-C headers (#927) <kannappanr>
- Encryption headers should also be set during initMultipart (#930) <Harshavardhana>
- Add support for Content-Language metadata header (#928) <kannappanr>
- Fix check for duplicate notification configuration entries (#917) <kannappanr>
- allow OS to cleanup sockets in TIME_WAIT (#925) <Harshavardhana>
- Sign V2: Fix signature calculation in virtual host style (#921) <A. Elleuch>
- bucket policy: Support json string in Principal field (#919) <A. Elleuch>
- Fix copyobject failure for empty files (#918) <kannappanr>
- Add new constructor NewWithOptions to SDK (#915) <poornas>
- Support redirect headers to sign again with new Host header. (#829) <Harshavardhana>
- Fail in PutObject if invalid user metadata is passed <Harshavadhana>
- PutObjectOptions Header: Don't include invalid header <Isaac Hess>
- increase max retry count to 10 (#913) <poornas>
- Add new regions for Paris and China west. (#905) <Harshavardhana>
- fix s3signer to use req.Host header (#899) <Bartłomiej Nogaś>
2018-03-14 19:38:29 +05:30
Harshavardhana 5dc5e4928c Support content-language header (#5648) 2018-03-14 15:27:32 +05:30
Yueh-Ting Chen e8d8360d00 fix spelling (#5643) 2018-03-13 19:23:37 +05:30
Harshavardhana 29ef7d29e4 Fix deadlock in in-place CopyObject decryption/encryption (#5637)
In-place decryption/encryption already holds write
locks on them, attempting to acquire a read lock would
fail.
2018-03-12 13:52:38 -07:00
Nitish Tiwari 10b01ac836
Add healthcheck endpoints (#5543)
This PR adds readiness and liveness endpoints to probe Minio server
instance health. Endpoints can only be accessed without authentication
and the paths are /minio/health/live and /minio/health/ready for
liveness and readiness respectively.

The new healthcheck liveness endpoint is used for Docker healthcheck
now.

Fixes #5357
Fixes #5514
2018-03-12 11:46:53 +05:30
Harshavardhana d90985b6d8 Return authHeaderMalformed for an incorrect region in signature (#5618) 2018-03-09 18:18:57 -08:00
Harshavardhana 7aaf01eb74 Save ETag when updating metadata (#5626)
Fixes #5622
2018-03-09 10:50:39 -08:00
kannappanr 380e0ddb57
Remove unwanted errorIf calls (#5621)
Remove errorIf call with a errSignatureMismatch error
2018-03-09 00:51:05 -08:00
poornas 247c1bb5ef Pass location in MakeBucketWithLocation call (#5605)
fixes regression for gateways
2018-03-08 12:56:20 +05:30
Harshavardhana 27258b9c54 Ensure to load only regular files for CAs (#5612)
In kubernetes statefulset like environments when secrets
are mounted to pods they have sub-directories, we should
ideally be only looking for regular files here and skip
all others.
2018-03-07 22:16:28 +05:30
Harshavardhana b325593b47 SSE-C CopyObject key-rotation doesn't need metadata REPLACE value (#5611)
Fix a compatibility issue with AWS S3 where to do key rotation
we need to replace an existing object's metadata. In such a
scenario "REPLACE" metadata directive is not necessary.
2018-03-06 16:04:48 -08:00
Anis Elleuch cac10bcbf7 SSE-C: Add support in Bucket Post Policy (#5607)
* SSE-C: Add support in Bucket Post Policy

* Rename isSSECustomerRequest & isSSECopyCustomerRequest to hasSSECustomerHeader hasSSECopyCustomerHeader
2018-03-05 08:02:56 -08:00
Aditya Manthramurthy ea8973b7d7 Return bit-rot verified data instead of re-reading from disk (#5568)
- Data from disk was being read after bitrot verification to return
  data for GetObject. Strictly speaking this does not guarantee bitrot
  protection, as disks may return bad data even temporarily.

- This fix reads data from disk, verifies data for bitrot and then
  returns data to the client directly.
2018-03-04 14:16:45 -08:00
Harshavardhana 52eea7b9c1
Support SSE-C multipart source objects in CopyObject (#5603)
Current code didn't implement the logic to support
decrypting encrypted multiple parts, this PR fixes
by supporting copying encrypted multipart objects.
2018-03-02 17:24:02 -08:00
Harshavardhana e4f6877c8b Handle incoming proxy requests ip, scheme (#5591)
This PR implements functions to get the right ip, scheme
from the incoming proxied requests.
2018-03-02 15:23:04 -08:00
Harshavardhana d71b1d25f8
Make sure to filter out internal metadata (#5601)
Currently we reply back `X-Minio-Internal` values
back to the client for an encrypted object, we should
filter these out and only reply AWS compatible headers.
2018-03-01 16:15:53 -08:00
Anis Elleuch 5f37988db5 Use toAPIErrorCode in HeadObject handler when decrypting request fails (#5600) 2018-03-01 16:01:56 -08:00
Harshavardhana 1b7b8f14c9 Set appropriate encryption headers in HEAD object response (#5596)
Currently we don't set two SSE-C specific headers fix it
for AWS S3 compatibility.
2018-03-01 14:16:40 -08:00
Anis Elleuch 120b061966 Add multipart support in SSE-C encryption (#5576)
*) Add Put/Get support of multipart in encryption
*) Add GET Range support for encryption
*) Add CopyPart encrypted support
*) Support decrypting of large single PUT object
2018-03-01 11:37:57 -08:00
kannappanr d32f90fe95 Honor global flags irrespective of the position. (#5486)
Flags like `json, config-dir, quiet` are now honored even if they are
between minio and gateway in the cli, like, `minio --json gateway s3`.

Fixes #5403
2018-02-28 20:13:33 -08:00
Harshavardhana 6faa1ef11a
Fix shadowing issue reported by go vet (#5590) 2018-02-28 14:30:00 -08:00
Harshavardhana 9af254a82f Remove stable sort usage when not needed (#5586)
Stable sort is needed when we are sorting based on two or more
distinct elements. When equal elements are indistinguishable,
such as with integers, or more generally, any data where the
entire element is the key like `PartNumber`, stability is not
an issue.
2018-02-28 08:33:00 +05:30
Harshavardhana 6b3db7556a Fix gofmt issues reported for simplification (#5581)
added travis to catch this issue
2018-02-26 23:52:03 +05:30
Harshavardhana 5204a754db
Move rpc version to 2.0.0 to align with backend migration (#5575)
Fixes #5574
2018-02-23 16:58:37 -08:00
Harshavardhana 7cc678c653 Support encryption for CopyObject, GET-Range requests (#5544)
- Implement CopyObject encryption support
- Handle Range GETs for encrypted objects

Fixes #5193
2018-02-23 15:07:21 -08:00
Krishnan Parthasarathi e5e3d17216 Do not close *lock.LockedFile on failure (#5565) 2018-02-21 11:28:24 -08:00
Harshavardhana 0ea54c9858 Change CopyObject{Part} to single srcInfo argument (#5553)
Refactor such that metadata and etag are
combined to a single argument `srcInfo`.

This is a precursor change for #5544 making
it easier for us to provide encryption/decryption
functions.
2018-02-21 14:18:47 +05:30
Krishna Srinivas a00e052606 Provide more descriptive error during erasure init (#5282)
fixes #5239
2018-02-20 18:42:09 -08:00
Anis Elleuch d2d49f6c6c xl: Avoid removing directory content in Delete API (#5548)
Delete & Multi Delete API should not try to remove the directory content.
The only permitted case is with zero size object with a trailing slash
in its name.
2018-02-20 15:33:26 -08:00