Commit Graph

7119 Commits

Author SHA1 Message Date
Klaus Post
8e6787a302
Fix TestDataUpdateTracker hanging (#10302)
Keep dataUpdateTracker while goroutine is starting.

This will ensure the object is updated one `start` returns

Tested with

```
λ go test -cpu=1,2,4,8 -test.run TestDataUpdateTracker -count=1000
PASS
ok      github.com/minio/minio/cmd      8.913s
```

Fixes #10295
2020-08-20 13:17:42 -07:00
Harshavardhana
59352d0ac2
load all blocking metadata in background (#10298)
most of this metadata already has fallbacks
and there is no good reason to load them
in blocking fashion
2020-08-20 10:38:53 -07:00
Harshavardhana
75d44b3bae
add disk for more context in bitrot errors (#10296) 2020-08-20 09:41:15 -07:00
Klaus Post
95ae6c4b49
Fix missing unlock in *healSequence.hasEnded() (#10305)
The background healing sequence would always hang when this function is called.
2020-08-20 08:48:09 -07:00
Harshavardhana
98ca770f81
mint fixes for go1.14 (#10294)
add go.mod to ensure that mint can be
built with go1.14.x
2020-08-19 20:07:37 -07:00
Krishnan Parthasarathi
ccd967e3be
Add ExpiresAt to LicenseInfo (#10293) 2020-08-19 19:21:04 -07:00
KevinSmile
0ebb73ee2e
use const instead of literals (#10292) 2020-08-19 16:43:52 -07:00
Harshavardhana
c8b84a0e9e
Add nancy vulnerability scanner (#10289) 2020-08-19 14:25:21 -07:00
Ritesh H Shukla
3acb5cff45
Update code comment (#10287) 2020-08-19 14:24:58 -07:00
Harshavardhana
ab801ad3d4
build edge containers with go1.14 (#10279) 2020-08-19 12:08:11 +05:30
Harshavardhana
74116204ce
handle fresh setup with mixed drives (#10273)
fresh drive setups when one of the drive is
a root drive, we should ignore such a root
drive and not proceed to format.

This PR handles this properly by marking
the disks which are root disk and they are
taken offline.
2020-08-18 14:37:26 -07:00
飞雪无情
2eb5f934d8
add bucket versioning zh_CN document (#10281) 2020-08-18 13:10:26 -07:00
Minio Trusted
b43d376a87 Update yaml files to latest version RELEASE.2020-08-18T19-41-00Z 2020-08-18 20:04:22 +00:00
Harshavardhana
e4a44f6224
fix: commonPrefixes behavior in ListObjectVersions (#10286)
```
$ aws s3api --profile minio --endpoint-url http://localhost:9003 \
    list-object-versions --bucket testbucket \
    --delimiter / --prefix Veeam/Archive/

{
    "CommonPrefixes": [
        {
            "Prefix": "Veeam/Archive/003/"
        }
    ]
}
```

Also add coverage tests similar to ListObjects to
catch errors in future, skip these tests in FS mode
2020-08-18 12:19:44 -07:00
poornas
0272973175
Fix regression in web ui for retention (#10285)
Fixes: #10283 regression from PR #9259
2020-08-18 12:09:42 -07:00
Klaus Post
adca28801d
feat: disable Parquet by default (breaking change) (#9920)
I have built a fuzz test and it crashes heavily in seconds and will OOM shortly after.
It seems like supporting Parquet is basically a completely open way to crash the 
server if you can upload a file and run s3 select on it.

Until Parquet is more hardened it is DISABLED by default since hostile 
crafted input can easily crash the server.

If you are in a controlled environment where it is safe to assume no hostile
content can be uploaded to your cluster you can safely enable Parquet.

To enable Parquet set the environment variable `MINIO_API_SELECT_PARQUET=on`
while starting the MinIO server.

Furthermore, we guard parquet by recover functions.
2020-08-18 10:23:28 -07:00
Harshavardhana
d2a3f92452
fix: health handler for lockers (#10280) 2020-08-18 07:27:41 -07:00
Harshavardhana
ede86845e5
docs: Add policy variables for resource and conditions (#10278)
Bonus fix adds LDAP policy variable and clarifies the
usage of policy variables for temporary credentials.

fixes #10197
2020-08-17 17:39:55 -07:00
Harshavardhana
e57c742674
use single dynamic timeout for most locked API/heal ops (#10275)
newDynamicTimeout should be allocated once, in-case
of temporary locks in config and IAM we should
have allocated timeout once before the `for loop`

This PR doesn't fix any issue as such, but provides
enough dynamism for the timeout as per expectation.
2020-08-17 11:29:58 -07:00
Klaus Post
bb5976d727
healbucket: Send object version ID (#10263)
Based on our previous conversations I assume we should send the version
 id when healing an object.

Maybe we should even list object versions and heal all?
2020-08-17 08:25:44 -07:00
Minio Trusted
670724184c Update yaml files to latest version RELEASE.2020-08-16T18-39-38Z 2020-08-16 18:56:41 +00:00
Harshavardhana
f7c1a59de1
add validation logs for configured Logger/Audit HTTP targets (#10274)
extra logs in-case of misconfiguration of audit/logger targets
2020-08-16 10:25:00 -07:00
飞雪无情
01a2ccc52f
add bucket retention zh_CN document (#10264) 2020-08-14 22:12:03 -07:00
Anis Elleuch
51ba1dac49
listing: Fix result when prefix is an object with a slash (#10267)
In a non recursive mode, issuing a list request where prefix
is an existing object with a slash and delimiter is a slash will
return entries in the object directory (data dir IDs)

```
$ aws s3api --profile minioadmin --endpoint-url http://localhost:9000 \
        list-objects-v2 --bucket testbucket --prefix code_of_conduct.md/ --delimiter '/'
{
    "CommonPrefixes": [
        {
            "Prefix":
"code_of_conduct.md/ec750fe0-ea7e-4b87-bbec-1e32407e5e47/"
        }
    ]
}
```

This commit adds a fast exit track in Walk() in this specific case.
2020-08-14 20:13:24 -07:00
Harshavardhana
a4463dd40f
fix: storageClass shouldn't set the value upon failure (#10271) 2020-08-14 19:48:04 -07:00
Harshavardhana
83a82d818e
allow lock tolerance to match storage-class drive tolerance (#10270) 2020-08-14 18:17:14 -07:00
Harshavardhana
1d1c4430b2
decrypt ETags in parallel around 500 at a time (#10261)
Listing speed-up gained from 10secs for
just 400 entries to 2secs for 400 entries
2020-08-14 11:56:35 -07:00
Krishnan Parthasarathi
4e00b47b52
licverifier: fail verify if accountId is missing in license metadata (#10258) 2020-08-13 17:05:24 -07:00
Harshavardhana
43e6d1ce2d
fix: missing proxy request by bucket for ListVersions (#10260) 2020-08-13 16:31:58 -07:00
Harshavardhana
30da442a85
rootDisk on containers can have different device Id (#10259)
use `/etc/hosts` instead of `/` to check for common
device id, if the device is same for `/etc/hosts`
and the --bind mount to detect root disks.

Bonus enhance healthcheck logging by adding maintenance
tags, for all messages.
2020-08-13 15:21:20 -07:00
Harshavardhana
038d91feaa
fix: add public certs automatically as part of global CAs (#10256) 2020-08-13 09:46:50 -07:00
Harshavardhana
e7ba78beee
use GlobalContext instead of context.Background when possible (#10254) 2020-08-13 09:16:01 -07:00
Krishnan Parthasarathi
ab43804efd
licverifier: Validate JWT token expiry (#10253)
With this change the expiry is validated for the license key JWT
2020-08-12 21:31:52 -07:00
Minio Trusted
1c865dd119 Update yaml files to latest version RELEASE.2020-08-13T02-39-50Z 2020-08-13 02:57:25 +00:00
Harshavardhana
b32d0a5b60 use the correct endpoints for offline drives 2020-08-12 19:17:49 -07:00
poornas
79e21601b0
fix: web handlers to enforce replication (#10249)
This PR also preserves source ETag for replication
2020-08-12 17:32:24 -07:00
Harshavardhana
34253aa595
feat: cache env value in-case network is not reachable (#10251) 2020-08-12 16:53:15 -07:00
Harshavardhana
79ed7ce451
fs: listObjects shouldn't take FS locks while listing (#10248) 2020-08-12 15:23:14 +05:30
Harshavardhana
900eebb9a4
use jwt instead of basicAuth for webEnv (#10246) 2020-08-11 16:09:34 -07:00
飞雪无情
6914b2c99d
Add bucket replication zh_CN document (#10243) 2020-08-11 11:33:01 -07:00
Harshavardhana
0dd3a08169
move the certPool loader function into pkg/certs (#10239) 2020-08-11 08:29:50 -07:00
Klaus Post
f8f290e848
security: Remove insecure custom headers (#10244)
Background: https://github.com/google/security-research/security/advisories/GHSA-76wf-9vgp-pj7w

Remove these custom headers from incoming and outgoing requests.
2020-08-11 08:29:29 -07:00
Harshavardhana
9179cdfc9d
update mint tests with new minio-py APIs (#10238) 2020-08-10 14:32:17 -07:00
Krishnan Parthasarathi
76b6dc0112
Add licverifier package (#10237)
license verification package implements a simple library to 
verify MinIO Subnet license keys.
2020-08-10 13:30:12 -07:00
Harshavardhana
ce303f5c7e
update node for mint 14.x (#10236) 2020-08-10 11:25:40 -07:00
飞雪无情
b4b7a18497
add bucket quota zh_CN document (#10232) 2020-08-10 10:55:34 -07:00
Harshavardhana
1e2ebc9945
feat: time to bring back http2.0 support (#10230)
Bonus move our CI/CD to go1.14
2020-08-10 09:02:29 -07:00
飞雪无情
a49e3647b6
add bucket lifecycle zh_CN document (#10231) 2020-08-09 02:29:25 -07:00
Minio Trusted
954e17c3d0 Update yaml files to latest version RELEASE.2020-08-08T04-50-06Z 2020-08-07 22:32:27 -07:00
Harshavardhana
2a9819aff8
fix: refactor background heal for cluster health (#10225) 2020-08-07 19:43:06 -07:00