Commit Graph

7008 Commits

Author SHA1 Message Date
Harshavardhana
60d91234b9
fix: avoid broken link when preview image (#10021)
closes #9589
2020-07-12 12:09:22 -07:00
Harshavardhana
37c14207d6
fix: cors handling again for not just OPTIONS request (#10025)
CORS is notorious requires specific headers to be
handled appropriately in request and response,
using cors package as part of handlerFunc() for
options method lacks the necessary control this
package needs to add headers.
2020-07-12 10:56:57 -07:00
Harshavardhana
3b9fbf80ad
fix: make sure to use new restClient for healthcheck (#10026)
Without instantiating a new rest client we can
have a recursive error which can lead to
healthcheck returning always offline, this can
prematurely take the servers offline.
2020-07-11 22:19:38 -07:00
Minio Trusted
c2fdf73491 Update yaml files to latest version RELEASE.2020-07-11T21-14-23Z 2020-07-11 21:30:53 +00:00
Harshavardhana
143f9371c6 fix: loading users regression
additionally also move to latest gorilla/mux master
to fix the DNS style bucket routing regression

resolves #10022
resolves #10023
2020-07-11 14:03:27 -07:00
Harshavardhana
3f1902face
fix: cors should be available on all paths (#10020) 2020-07-11 13:49:24 -07:00
Harshavardhana
c0adb52213
sync to disk only upon successful legacy metadata rename (#10018) 2020-07-11 09:37:34 -07:00
Harshavardhana
3520e946a2 fix: versioning docs add more examples 2020-07-11 00:57:46 -07:00
Harshavardhana
f38adc1865 cleanup security overview guide 2020-07-11 00:34:56 -07:00
Harshavardhana
d5ff1c8e3b fix docs image urls to be absolute path 2020-07-11 00:27:30 -07:00
Minio Trusted
ad7417bc50 Update yaml files to latest version RELEASE.2020-07-11T06-07-16Z 2020-07-11 06:28:08 +00:00
Harshavardhana
2d17c16d93
fix: make sure to honor versioning from browser UI deletes (#10016) 2020-07-10 22:21:04 -07:00
Harshavardhana
aded0bc81a Update dockerfiles for the release 2020-07-10 18:43:35 -07:00
Harshavardhana
36d36fab0b
fix: add virtual host style workaround for gorilla/mux issue (#10010)
gorilla/mux broke their recent release 1.7.4 which we
upgraded to, we need the current workaround to ensure
that our regex matches appropriately.

An upstream PR is sent, we should remove the
workaround once we have a new release.
2020-07-10 15:21:32 -07:00
Harshavardhana
ba756cf366
fix: extract array type for policy claim if present (#10014) 2020-07-10 14:48:44 -07:00
Benjamin Sodenkamp
c00d410e61
Added bucket name param to ToJSONError call (#9961)
when called with InvalidBucketName error.
The user is shown a more specific error
when the param is present.
2020-07-10 12:10:39 -07:00
Klaus Post
968342c732
Remove usage of go-ieproxy for windows (#10009)
There is a potential for deadlock on Windows 10
refer https://github.com/mattn/go-ieproxy/issues/17 

remove this dependency for now.
2020-07-10 12:08:14 -07:00
Harshavardhana
5c15656c55
support bootstrap client to use healthcheck restClient (#10004)
- reduce locker timeout for early transaction lock
  for more eagerness to timeout
- reduce leader lock timeout to range from 30sec to 1minute
- add additional log message during bootstrap phase
2020-07-10 09:26:21 -07:00
Harshavardhana
2e8fc6ebfe
cleanup STS docs (#10003) 2020-07-10 09:07:12 -07:00
kannappanr
efe9fe6124
azure: Return success when deleting non-existent object (#9981) 2020-07-10 08:30:23 -07:00
Nitish Tiwari
30c251efd3
Add Grafana dashboard (#10000) 2020-07-09 12:01:58 -07:00
Klaus Post
c850905e43
fix: threadwalk lockup under high load (#9992)
Main issue is that `t.pool[params]` should be `t.pool[oldest]`.

We add a bit more safety features for the code.

* Make writes to the endTimerCh non-blocking in all cases
   so multiple releases cannot lock up.
* Double check expectations.
* Shift down deletes with copy instead of truncating slice.
* Actually delete the oldest if we are above total limit.
* Actually delete the oldest found and not the current.
* Unexport the mutex so nobody from the outside can meddle with it.
2020-07-09 07:02:18 -07:00
Andreas Auernhammer
a317a2531c
admin: new API for creating KMS master keys (#9982)
This commit adds a new admin API for creating master keys.
An admin client can send a POST request to:
```
/minio/admin/v3/kms/key/create?key-id=<keyID>
```

The name / ID of the new key is specified as request
query parameter `key-id=<ID>`.

Creating new master keys requires KES - it does not work with
the native Vault KMS (deprecated) nor with a static master key
(deprecated).

Further, this commit removes the `UpdateKey` method from the `KMS`
interface. This method is not needed and not used anymore.
2020-07-08 18:50:43 -07:00
Ravind Kumar
ee20ebe07a Remove dead link related to DC/OS Deployment Guide (#9996) 2020-07-08 17:38:20 -07:00
Harshavardhana
2743d4ca87
fix: Add support for preserving mtime for replication (#9995)
This PR is needed for bucket replication support
2020-07-08 17:36:56 -07:00
Harshavardhana
6136a963c8
fix: bump the response header timeout for forwarder as well (#9994)
continuation of #9986, add more place where the lower timeout
comes into effect.
2020-07-08 10:55:24 -07:00
Harshavardhana
60417950c7
fix: the versioning/object lock documentation appropriately (#9988)
- Move the bucket level features into `docs/bucket` directory
- fix issue template and simplify some of them
2020-07-08 08:44:43 -07:00
Anis Elleuch
fa211f6a10
heal: Fix healing delete markers (#9989) 2020-07-07 20:54:09 -07:00
Harshavardhana
72e0745e2f
fix: migrate to go.etcd.io import path (#9987)
with the merge of https://github.com/etcd-io/etcd/pull/11823
etcd v3.5.0 will now have a properly imported versioned path

this fixes our pending migration to newer repo
2020-07-07 19:04:29 -07:00
Klaus Post
aa4d1021eb
Remove timeout from putobject and listobjects (#9986)
Use a separate client for these calls that can take a long time.

Add request context to these so they are canceled when the client 
disconnects instead except for ListObject which doesn't have any equivalent.
2020-07-07 12:19:57 -07:00
Harshavardhana
93e7e4a0e5
fix: cors handling after gorilla mux update (#9980)
fixes #9979
2020-07-06 20:55:19 -07:00
Anis Elleuch
c2f7cd1104
Consider errFileVersionNotFound during healing assessment (#9977)
Healing an object which has multiple versions was not working because
the healing code forgot to consider errFileVersionNotFound error as a
use case that needs healing
2020-07-06 08:09:48 -07:00
Harshavardhana
38eef5ce4c
fix: documentation fixes for docker ENV settings (#9975)
- update CREDITS file
- fix markdown links
- talk a bit more about upgrades
2020-07-06 06:42:34 -07:00
Anis Elleuch
4cf80f96ad
fix: lifecycle XML parsing errors with Versioning (#9974) 2020-07-05 09:08:42 -07:00
Anis Elleuch
d4af132fc4
lifecycle: Expiry should not delete versions (#9972)
Currently, lifecycle expiry is deleting all object versions which is not
correct, unless noncurrent versions field is specified.

Also, only delete the delete marker if it is the only version of the
given object.
2020-07-04 20:56:02 -07:00
Harshavardhana
c087a05b43
fix: simplify data structure before release (#9968)
- additionally upgrade to msgp@v1.1.2
- change StatModTime,StatSize fields as
  simple Size/ModTime
- reduce 50000 entries per List batch to 10000
  as client needs to wait too long to see the
  first batch some times which is not desired
  and it is worth we write the data as soon
  as we have it.
2020-07-04 12:25:53 -07:00
Harshavardhana
cdb0e6ffed
support proper values for listMultipartUploads/listParts (#9970)
object KMS is configured with auto-encryption,
there were issues when using docker registry -
this has been left unnoticed for a while.

This PR fixes an issue with compatibility.

Additionally also fix the continuation-token implementation
infinite loop issue which was missed as part of #9939

Also fix the heal token to be generated as a client
facing value instead of what is remembered by the
server, this allows for the server to be stateless 
regarding the token's behavior.
2020-07-03 19:27:13 -07:00
Harshavardhana
03b84091fc
auto enable versioning with object locking (#9967)
this is to preserve versioning for object-locked
buckets from current release code.
2020-07-03 15:30:06 -07:00
Anis Elleuch
2be20588bf
Reroute requests based token heal/listing (#9939)
When manual healing is triggered, one node in a cluster will 
become the authority to heal. mc regularly sends new requests 
to fetch the status of the ongoing healing process, but a load 
balancer could land the healing request to a node that is not 
doing the healing request.

This PR will redirect a request to the node based on the node 
index found described as part of the client token. A similar
technique is also used to proxy ListObjectsV2 requests
by encoding this information in continuation-token
2020-07-03 11:53:03 -07:00
Harshavardhana
e59ee14f40
Tune tcp keep-alives with new kernel timeout options (#9963)
For more deeper understanding https://blog.cloudflare.com/when-tcp-sockets-refuse-to-die/
2020-07-03 10:03:41 -07:00
Anis Elleuch
21a37e3393
fix: ListObjectVersions should return ordered Version & DeleteMarker (#9959)
The S3 specification says that versions are ordered in the response of
list object versions.

mc snapshot needs this to know which version comes first especially when
two versions have the same exact last-modified field.
2020-07-03 09:15:44 -07:00
Harshavardhana
810a4f0723
fix: return proper errors Get/HeadObject for deleteMarkers (#9957) 2020-07-02 16:17:27 -07:00
Krishna Srinivas
4c266df863
fix: proxy ListObjects request to one of the server based on hash(bucket) (#9881) 2020-07-02 10:56:22 -07:00
Klaus Post
abd999f64a
fix: list object versions in distributed setup (#9958)
Remove calls to `WalkVersions` was calling the wrong endpoint, 
so unless quorum could be reached with local disks no results 
would ever be returned.
2020-07-02 10:29:50 -07:00
Minio Trusted
04de19c870 Update yaml files to latest version RELEASE.2020-07-02T00-15-09Z 2020-07-01 17:36:52 -07:00
Benjamin Sodenkamp
648cb13e02
Added 'close' to results channel in Walk() (#9956) 2020-07-01 14:29:45 -07:00
Harshavardhana
174f428571
add additional fdatasync before close() on writes (#9947) 2020-07-01 10:57:23 -07:00
Harshavardhana
5388ae4acb
make sure to delete data-usage cache upon bucket deletes (#9952) 2020-07-01 10:55:28 -07:00
Klaus Post
2e338e84cb
fix owhanging crashes in parquet S3 select (#9921) 2020-07-01 08:15:41 -07:00
kannappanr
5089a7167d
Handle empty retention in get/put object retention (#9948)
Fixes #9943
2020-06-30 16:44:24 -07:00