Commit Graph

7868 Commits

Author SHA1 Message Date
Harshavardhana
d348ec0f6c avoid double listObjectParts calls improves performance (#9606)
this PR is to avoid double calls across multiple calls
in APIs

- CopyObjectPart
- PutObjectPart
2020-05-15 08:06:45 -07:00
Harshavardhana
b730bd1396 fix: possible race in FS local lockMap (#9598) 2020-05-14 23:59:07 -07:00
Klaus Post
56e0c6adf8 Track if bloom filter is dirty (#9601)
Only save bloom filter on cycles and updates.

Fixes #9600
2020-05-14 21:46:36 -07:00
Anis Elleuch
f44a960dcd tests: Fix one multi-delete test failure in Windows CI (#9602)
There is a disparency of behavior under Linux & Windows about
the returned error when trying to rename a non existant path.

err := os.Rename("/path/does/not/exist", "/tmp/copy")

Linux:
  isSysErrNotDir(err) = false
  os.IsNotExist(err) = true

Windows:
  isSysErrNotDir(err) = true
  os.IsNotExist(err) = true

ENOTDIR in Linux is returned when the destination path
of the rename call contains a file in one of the middle
segments of the path (e.g. /tmp/file/dst, where /tmp/file
is an actual file not a directory)

However, as shown above, Windows has more scenarios when
it returns ENOTDIR. For example, when the source path contains
an inexistant directory in its path.

In that case, we want errFileNotFound returned and not
errFileAccessDenied, so this commit will add a further check to close
the disparency between Windows & Linux.
2020-05-14 18:09:30 -07:00
kannappanr
6c1bbf918d do not add quotes around etag, if already present (#9603) 2020-05-14 17:43:54 -07:00
Anis Elleuch
48e614b167 honor lifecycle expiration with tag rule (#9604) 2020-05-14 16:21:03 -07:00
poornas
fe8d33452b Allow writes for bucket exceeding FIFO quota (#9575)
the quota will be enforced while
deleting oldest entries in FIFO manner.
2020-05-14 15:18:24 -07:00
Andreas Auernhammer
c19ece6921 update KMS guide to reflect latest KES changes (#9591)
This commit updates the two client env. variables:
```
KES_CLIENT_TLS_KEY_FILE
KES_CLIENT_TLS_CERT_FILE
```

The KES CLI client expects the client key and certificate
as `KES_CLIENT_KEY` resp. `KES_CLIENT_CERT`.
2020-05-14 14:02:43 -07:00
Klaus Post
216fa57b88 merge nested hash readers (#9582)
The `ioutil.NopCloser(reader)` was hiding nested hash readers.

We make it an `io.Closer` so it can be attached without wrapping 
and allows for nesting, by merging the requests.
2020-05-14 14:01:31 -07:00
Dzmitry Pasiukevich
a9558ae248 Simplify cast of string to rune slice in wildcard matching (#9577) 2020-05-14 08:20:13 -07:00
Klaus Post
ee9077db7d fix: windows tests for all cases (#9594)
Replaces #9299
2020-05-13 23:55:38 -07:00
Harshavardhana
9c85928740 add formatting message for zones in ordinals (#9596)
Unlike the message
> Formatting 2 zone, 1 set(s), 6 drives per set.

It is more readable as ordinal
> Formatting 2nd zone, 1 set(s), 6 drives per set.
2020-05-13 20:25:29 -07:00
kannappanr
af0309371e mint tests s3cmd remove copy object test (#9595)
test is failing in Azure gateway as a mapping from 
x-amz-storage-class to Azure storage tier is failing 
and there is no way to not send storage class in s3cmd.
2020-05-13 19:55:34 -07:00
ebozduman
ead3c186a6 Fixes broken image preview for anon user (#9584) 2020-05-13 13:26:12 -07:00
Harshavardhana
6ac48a65cb fix: use unused cacheMetrics code in prometheus (#9588)
remove all other unusued/deadcode
2020-05-13 08:15:26 -07:00
poornas
2ecf5ba1de fix panic when checking if es/nats event target is active (#9587) 2020-05-13 06:37:22 -07:00
Krishna Srinivas
94f1a1dea3 add option for O_SYNC writes for standalone FS backend (#9581) 2020-05-12 19:24:59 -07:00
Anis Elleuch
c045ae15e7 fix: avoid undoing bucket creation and return the first err instead (#9578) 2020-05-12 15:20:42 -07:00
Harshavardhana
1756b7c6ff fix: LDAP derivative accounts parentUser validation is not needed (#9573)
* fix: LDAP derivative accounts parentUser validation is not needed

fixes #9435

* Update cmd/iam.go

Co-authored-by: Lenin Alevski <alevsk.8772@gmail.com>

Co-authored-by: Lenin Alevski <alevsk.8772@gmail.com>
2020-05-12 09:21:08 -07:00
Klaus Post
e25ace2151 Forward RPC errors from crawler (#9569)
The `keepHTTPResponseAlive` would cause errors to be 
returned with status OK.

- Add '32' as a filler byte until a response is ready
- '0' to indicate the response is ready to be consumed
- '1' to indicate response has an error which needs
to be returned to the caller

Clear out 'file not found' errors from dir walker, since it may be 
in a folder that has been deleted since it was scanned.
2020-05-11 20:41:38 -07:00
poornas
a8e5a86fa0 Remove brittle tests for cache (#9570) 2020-05-11 15:41:10 -07:00
Harshavardhana
f8edc233ab support multiple policies for temporary users (#9550) 2020-05-11 13:04:11 -07:00
Harshavardhana
337c2a7cb4 add audit logging for all admin calls (#9568)
- add ServiceRestart/ServiceStop actions
- audit log appropriately in all admin handlers

fixes #9522
2020-05-11 10:34:08 -07:00
Harshavardhana
2d735144b9 fix: distributed docs image path 2020-05-11 09:33:55 -07:00
Harshavardhana
0113035237 update distributed setup guide (#9566) 2020-05-11 09:19:10 -07:00
Anis Elleuch
52a1d248b2 policy: Do not return an error for invalid value during parsing (#9442)
s3:HardwareInfo was removed recently. Users having that admin action
stored in the backend will have an issue starting the server.

To fix this, we need to avoid returning an error in Marshal/Unmarshal
when they encounter an invalid action and validate only in specific
location.

Currently the validation is done and in ParseConfig().
2020-05-10 10:55:28 -07:00
Harshavardhana
b5ed42c845 ignore policy/group missing errors appropriately (#9559) 2020-05-09 13:59:12 -07:00
Klaus Post
d9e7cadacf Update reed+solomon (#9562)
Only create encoder when strictly needed.
2020-05-09 09:54:20 -07:00
Harshavardhana
36e88cbd50 fix mint tests in awscli to ignore NotImplemented properly (#9561) 2020-05-08 22:32:10 -07:00
Anis Elleuch
6d76efb9bb Add support of TCP fast open in internode calls (#9486) 2020-05-08 14:33:23 -07:00
Harshavardhana
a1de9cec58 cleanup object-lock/bucket tagging for gateways (#9548)
This PR is to ensure that we call the relevant object
layer APIs for necessary S3 API level functionalities
allowing gateway implementations to return proper
errors as NotImplemented{}

This allows for all our tests in mint to behave
appropriately and can be handled appropriately as
well.
2020-05-08 13:44:44 -07:00
Anis Elleuch
6885c72f32 disable check for DirectIO in standalone FS mode (#9558) 2020-05-08 12:07:51 -07:00
poornas
0f1389e992 Fix azure gateway handling of ETag for CopyObject (#9544)
fixes #9428
2020-05-08 11:30:35 -07:00
Minio Trusted
5bf3eeaa77 Update yaml files to latest version RELEASE.2020-05-08T02-40-49Z 2020-05-08 02:49:15 +00:00
Harshavardhana
9dda1fd624 Remove B2 gateway implementation (#9547)
S3 is now natively supported by B2 cloud storage provider
there is no reason to use specialized gateway for B2 anymore,
our current S3 gateway with caching would work with B2.

Resolves #8584
RELEASE.2020-05-08T02-40-49Z
2020-05-07 19:00:30 -07:00
Harshavardhana
2dc46cb153 Report correct error when O_DIRECT is not supported (#9545)
fixes #9537
2020-05-07 16:12:16 -07:00
remche
0674c0075e add LDAP StartTLS support (#9472) 2020-05-07 15:08:33 -07:00
Harshavardhana
518ef670da update browser assets with new changes (#9543)
- f7c91eff5 - Share button for public objects (#9162) (5 days ago) <Egor Rudinsky>
- 60d415bb8 - deprecate/remove global WORM mode (#9436) (13 days ago) <Harshavardhana>
2020-05-07 13:21:05 -07:00
Harshavardhana
0dd626ec67 fix: requests without bucket should route to the original router (#9541)
requests in federated setups for STS type calls which are
performed at '/' resource should be routed by the muxer,
the assumption is simply such that requests without a bucket
in a federated setup cannot be proxied, so serve them at
current server.
2020-05-07 11:49:04 -07:00
Harshavardhana
53f4c0fdc0 fix: deprecate old settings and elaborate on tuning (#9542)
fixes #9538
2020-05-07 11:08:57 -07:00
P R
7e3ea77fdf Checking for access denied in web browser request. (#9523)
Fixes #9485
2020-05-06 21:31:44 -07:00
Harshavardhana
7290d23b26 Apply partNumber checks only on multipart objects (#9528) 2020-05-06 16:58:09 -07:00
Minio Trusted
24f20eb1bd Update yaml files to latest version RELEASE.2020-05-06T23-23-25Z 2020-05-06 23:32:41 +00:00
Harshavardhana
4c9de098b0 heal buckets during init and make sure to wait on quorum (#9526)
heal buckets properly during expansion, and make sure
to wait for the quorum properly such that healing can
be retried.
RELEASE.2020-05-06T23-23-25Z
2020-05-06 14:25:05 -07:00
Harshavardhana
a2ccba69e5 add kes retries upto two times with jitter backoff (#9527)
KES calls are not retried and under certain situations
when KES is under high load, the request should be
retried automatically.
2020-05-06 11:44:06 -07:00
Harshavardhana
8eb99d3a87 fix: complete multipart upload respond with ETag quoted (#9525)
Fixes #9517
2020-05-05 17:47:54 -07:00
Bala FA
3773874cd3 add bucket tagging support (#9389)
This patch also simplifies object tagging support
2020-05-05 14:18:13 -07:00
Harshavardhana
6c62b1a2ea fix broken retry tests 2020-05-04 22:01:39 -07:00
Harshavardhana
b768645fde fix: unexpected logging with bucket metadata conversions (#9519) 2020-05-04 20:04:06 -07:00
Harshavardhana
7b58dcb28c fix: return context error from context reader (#9507) 2020-05-04 14:33:49 -07:00