Commit Graph

5088 Commits

Author SHA1 Message Date
Ersan
33fe42df8a Adds context support for nas, oss and s3 2018-03-15 16:28:25 -07:00
kannappanr
bdb1a90720 Add context support for gateway b2, manta, gcs 2018-03-15 16:28:25 -07:00
Krishna Srinivas
e452377b24 Add context to the object-interface methods.
Make necessary changes to xl fs azure sia
2018-03-15 16:28:25 -07:00
Krishna Srinivas
9083bc152e Flat multipart backend implementation for Erasure backend (#5447) 2018-03-15 13:55:23 -07:00
Bala FA
0e4431725c make notification as separate package (#5294)
* Remove old notification files

* Add net package

* Add event package

* Modify minio to take new notification system
2018-03-15 13:03:41 -07:00
Nitish Tiwari
abffa00b76
Update healthcheck related examples and add head support (#5650)
- Add head method for healthcheck endpoint. Some platforms/users
may use the HTTP Head method to check for health status.

- Add liveness and readiness probe examples in Kubernetes yaml
example docs. Note that readiness probe not added to StatefulSet
example due to https://github.com/kubernetes/kubernetes/issues/27114
2018-03-15 09:55:02 +05:30
Nitish Tiwari
1c0c3369c9 Cleanup large bucket support documents (#5641) 2018-03-14 15:22:17 -07:00
Harshavardhana
4af89543cf Update minio-go dependencies to latest 5.0.0 release (#5640)
With following changes

- Add SSE and refactor encryption API (#942) <Andreas Auernhammer>
- add copyObject test changing metadata and preserving etag (#944) <Harshavardhana>
- Add SSE-C tests for multipart, copy, get range operations (#941) <Harshavardhana>
- Removing conditional check for notificationInfoCh in api-notication (#940) <Matthew Magaldi>
- Honor prefix parameter in ListBucketPolicies API (#929) <kannappanr>
- test for empty objects uploaded with SSE-C headers (#927) <kannappanr>
- Encryption headers should also be set during initMultipart (#930) <Harshavardhana>
- Add support for Content-Language metadata header (#928) <kannappanr>
- Fix check for duplicate notification configuration entries (#917) <kannappanr>
- allow OS to cleanup sockets in TIME_WAIT (#925) <Harshavardhana>
- Sign V2: Fix signature calculation in virtual host style (#921) <A. Elleuch>
- bucket policy: Support json string in Principal field (#919) <A. Elleuch>
- Fix copyobject failure for empty files (#918) <kannappanr>
- Add new constructor NewWithOptions to SDK (#915) <poornas>
- Support redirect headers to sign again with new Host header. (#829) <Harshavardhana>
- Fail in PutObject if invalid user metadata is passed <Harshavadhana>
- PutObjectOptions Header: Don't include invalid header <Isaac Hess>
- increase max retry count to 10 (#913) <poornas>
- Add new regions for Paris and China west. (#905) <Harshavardhana>
- fix s3signer to use req.Host header (#899) <Bartłomiej Nogaś>
2018-03-14 19:38:29 +05:30
Harshavardhana
5dc5e4928c Support content-language header (#5648) 2018-03-14 15:27:32 +05:30
tparker00
bbd7f8a4af Updated healthcheck file (#5647)
Healthcheck is no longer checking for a file so the -I in the curl check will fail.
2018-03-14 13:21:56 +05:30
Nitish Tiwari
6882467db8
Update volume in Dockerfiles (#5646)
All sample commands in Minio docs use the volume path as /data.
Updated the dockerfiles to follow the same pattern.

Fixes #5635
2018-03-14 06:59:27 +05:30
Yueh-Ting Chen
e8d8360d00 fix spelling (#5643) 2018-03-13 19:23:37 +05:30
Alexander Stuart-Kregor
5053f96274 Swap max_hw_sectors_kb for max_sectors_kb. Fixes #5633. (#5638) 2018-03-13 18:47:49 +05:30
Minio Trusted
16000255f8 Update yaml files to latest version RELEASE.2018-03-12T21-25-28Z 2018-03-12 21:28:21 +00:00
Nitish Tiwari
4a4d1d1b82 Add Minio TLS configuration doc for Kubernetes deployment (#5617) 2018-03-12 14:22:23 -07:00
Harshavardhana
29ef7d29e4 Fix deadlock in in-place CopyObject decryption/encryption (#5637)
In-place decryption/encryption already holds write
locks on them, attempting to acquire a read lock would
fail.
2018-03-12 13:52:38 -07:00
Nitish Tiwari
574b667c56 Remove madmin docs from top level docs directory (#5636)
madmin package is well documented in its source directory here
https://github.com/minio/minio/tree/master/pkg/madmin.

Hence, keeping another copy is not required as it makes it difficult
to maintain.
2018-03-12 11:51:58 -07:00
Nitish Tiwari
10b01ac836
Add healthcheck endpoints (#5543)
This PR adds readiness and liveness endpoints to probe Minio server
instance health. Endpoints can only be accessed without authentication
and the paths are /minio/health/live and /minio/health/ready for
liveness and readiness respectively.

The new healthcheck liveness endpoint is used for Docker healthcheck
now.

Fixes #5357
Fixes #5514
2018-03-12 11:46:53 +05:30
Harshavardhana
d90985b6d8 Return authHeaderMalformed for an incorrect region in signature (#5618) 2018-03-09 18:18:57 -08:00
Harshavardhana
7aaf01eb74 Save ETag when updating metadata (#5626)
Fixes #5622
2018-03-09 10:50:39 -08:00
Nitish Tiwari
ba0c7544ea
Cleanup orchestration documents (#5623)
- Remove hostPort from Kubernetes deployment example docs. Initially
hostPort was added to ensure Minio pods are allocated to separate
machines, but as per latest Kubernetes documents this is not
recommended approach (ref: https://kubernetes.io/docs/concepts/
configuration/overview/#services). To define pod allocations,
Affinity and Anti-Affinity concepts are the recommended approach.
(ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node)

- Add Minio release tag to Docker-Compose example file.
2018-03-09 15:21:41 +05:30
kannappanr
380e0ddb57
Remove unwanted errorIf calls (#5621)
Remove errorIf call with a errSignatureMismatch error
2018-03-09 00:51:05 -08:00
Andreas Auernhammer
889dd387f1 [doc] fix openssl command for ECDSA key generation (#5616)
This change fixes the command for generating ECDSA private keys.
The current command produces private key files which cannot be parsed
by the server.

Fixes #5614
2018-03-08 15:06:42 -08:00
Dee Koder
973ff2fabd Fix mqtt example py code which was not working (#5619) 2018-03-08 16:46:52 +05:30
poornas
247c1bb5ef Pass location in MakeBucketWithLocation call (#5605)
fixes regression for gateways
2018-03-08 12:56:20 +05:30
Harshavardhana
27258b9c54 Ensure to load only regular files for CAs (#5612)
In kubernetes statefulset like environments when secrets
are mounted to pods they have sub-directories, we should
ideally be only looking for regular files here and skip
all others.
2018-03-07 22:16:28 +05:30
Harshavardhana
b325593b47 SSE-C CopyObject key-rotation doesn't need metadata REPLACE value (#5611)
Fix a compatibility issue with AWS S3 where to do key rotation
we need to replace an existing object's metadata. In such a
scenario "REPLACE" metadata directive is not necessary.
2018-03-06 16:04:48 -08:00
Dee Koder
a34901af77 Add missing wget command for easy cut copy paste. (#5609)
We should keep this for easy one click copy paste on remote servers 
rather than 2 things for users to deal with.
2018-03-06 13:36:48 +05:30
Anis Elleuch
cac10bcbf7 SSE-C: Add support in Bucket Post Policy (#5607)
* SSE-C: Add support in Bucket Post Policy

* Rename isSSECustomerRequest & isSSECopyCustomerRequest to hasSSECustomerHeader hasSSECopyCustomerHeader
2018-03-05 08:02:56 -08:00
Aditya Manthramurthy
ea8973b7d7 Return bit-rot verified data instead of re-reading from disk (#5568)
- Data from disk was being read after bitrot verification to return
  data for GetObject. Strictly speaking this does not guarantee bitrot
  protection, as disks may return bad data even temporarily.

- This fix reads data from disk, verifies data for bitrot and then
  returns data to the client directly.
2018-03-04 14:16:45 -08:00
Harshavardhana
52eea7b9c1
Support SSE-C multipart source objects in CopyObject (#5603)
Current code didn't implement the logic to support
decrypting encrypted multiple parts, this PR fixes
by supporting copying encrypted multipart objects.
2018-03-02 17:24:02 -08:00
Harshavardhana
e4f6877c8b Handle incoming proxy requests ip, scheme (#5591)
This PR implements functions to get the right ip, scheme
from the incoming proxied requests.
2018-03-02 15:23:04 -08:00
Harshavardhana
d71b1d25f8
Make sure to filter out internal metadata (#5601)
Currently we reply back `X-Minio-Internal` values
back to the client for an encrypted object, we should
filter these out and only reply AWS compatible headers.
2018-03-01 16:15:53 -08:00
Anis Elleuch
5f37988db5 Use toAPIErrorCode in HeadObject handler when decrypting request fails (#5600) 2018-03-01 16:01:56 -08:00
Andreas Auernhammer
4e5237b02a vendor update github.com/minio/sio (#5599)
This change updates the sio library and adds DARE 2.0 support
to the server.
2018-03-01 15:28:34 -08:00
Harshavardhana
1b7b8f14c9 Set appropriate encryption headers in HEAD object response (#5596)
Currently we don't set two SSE-C specific headers fix it
for AWS S3 compatibility.
2018-03-01 14:16:40 -08:00
Anis Elleuch
120b061966 Add multipart support in SSE-C encryption (#5576)
*) Add Put/Get support of multipart in encryption
*) Add GET Range support for encryption
*) Add CopyPart encrypted support
*) Support decrypting of large single PUT object
2018-03-01 11:37:57 -08:00
kannappanr
d32f90fe95 Honor global flags irrespective of the position. (#5486)
Flags like `json, config-dir, quiet` are now honored even if they are
between minio and gateway in the cli, like, `minio --json gateway s3`.

Fixes #5403
2018-02-28 20:13:33 -08:00
Harshavardhana
6faa1ef11a
Fix shadowing issue reported by go vet (#5590) 2018-02-28 14:30:00 -08:00
Harshavardhana
9af254a82f Remove stable sort usage when not needed (#5586)
Stable sort is needed when we are sorting based on two or more
distinct elements. When equal elements are indistinguishable,
such as with integers, or more generally, any data where the
entire element is the key like `PartNumber`, stability is not
an issue.
2018-02-28 08:33:00 +05:30
Harshavardhana
6b3db7556a Fix gofmt issues reported for simplification (#5581)
added travis to catch this issue
2018-02-26 23:52:03 +05:30
Harshavardhana
5204a754db
Move rpc version to 2.0.0 to align with backend migration (#5575)
Fixes #5574
2018-02-23 16:58:37 -08:00
Harshavardhana
7cc678c653 Support encryption for CopyObject, GET-Range requests (#5544)
- Implement CopyObject encryption support
- Handle Range GETs for encrypted objects

Fixes #5193
2018-02-23 15:07:21 -08:00
poornas
b7536570f8 update docs for NAS gateway (#5569) 2018-02-22 09:15:19 +05:30
Harshavardhana
e09d97abaf Fix docs in admin API (#5559) 2018-02-21 12:00:46 -08:00
Krishnan Parthasarathi
e5e3d17216 Do not close *lock.LockedFile on failure (#5565) 2018-02-21 11:28:24 -08:00
Harshavardhana
0ea54c9858 Change CopyObject{Part} to single srcInfo argument (#5553)
Refactor such that metadata and etag are
combined to a single argument `srcInfo`.

This is a precursor change for #5544 making
it easier for us to provide encryption/decryption
functions.
2018-02-21 14:18:47 +05:30
Krishna Srinivas
a00e052606 Provide more descriptive error during erasure init (#5282)
fixes #5239
2018-02-20 18:42:09 -08:00
Anis Elleuch
d2d49f6c6c xl: Avoid removing directory content in Delete API (#5548)
Delete & Multi Delete API should not try to remove the directory content.
The only permitted case is with zero size object with a trailing slash
in its name.
2018-02-20 15:33:26 -08:00
Harshavardhana
db9e83de62 Avoid significant connections in TIME_WAIT (#5555)
MaxIdleConns limits the total number of connections
kept in the pool for re-use. In addition, MaxIdleConnsPerHost
limits the number for a single host. Since minio gateways
usually connect to the same host, setting `MaxIdleConns = 100`
won't really have much of an impact since the idle connection
pool is limited to 2 anyway.

Now, with the pool set to a limit of 2, and when using
the client heavily from 2+ goroutines, the `http.Transport`
will open a connection, use it, then try to return it to
the idle-pool which often fails since there's a limit of 2.
So it's going to close the connection and new ones will be
opened on demand again, many of which get closed soon after
being used. Since those connections/sockets don't disappear
from the OS immediately, use `MaxIdleConnsPerHost = 100`
which fixes this problem.
2018-02-20 12:23:37 -08:00