Commit Graph

7456 Commits

Author SHA1 Message Date
Anis Elleuch 46ee8659b4
fix write quorum calculation for bucket operations (#10364)
When the number of disks is odd, the calculation of quorum 
for bucket operations were not correct, fix it.
2020-08-27 12:55:32 -07:00
poornas 73a6b4ea11
fix typo in replication docs (#10366) 2020-08-27 12:54:23 -07:00
Minio Trusted c1b88c17cc Update yaml files to latest version RELEASE.2020-08-27T05-16-20Z 2020-08-27 05:36:01 +00:00
Harshavardhana a359e36e35
tolerate listing with only readQuorum disks (#10357)
We can reduce this further in the future, but this is a good
value to keep around. With the advent of continuous healing,
we can be assured that namespace will eventually be
consistent so we are okay to avoid the necessity to
a list across all drives on all sets.

Bonus Pop()'s in parallel seem to have the potential to
wait too on large drive setups and cause more slowness
instead of gaining any performance remove it for now.

Also, implement load balanced reply for local disks,
ensuring that local disks have an affinity for

- cleanupStaleMultipartUploads()
2020-08-26 19:29:35 -07:00
Jorge Israel Peña 0a2e6d58a5
hdfs gateway handle listing single files (#10362) 2020-08-26 16:03:53 -07:00
Harshavardhana 7e80afdd7f remove nancy ignore list, as upstream issue is fixed 2020-08-26 15:45:42 -07:00
Klaus Post 1b119557c2
getDisksInfo: Attribute failed disks to correct endpoint (#10360)
If DiskInfo calls failed the information returned was used anyway 
resulting in no endpoint being set.

This would make the drive be attributed to the local system since 
`disk.Endpoint == disk.DrivePath` in that case.

Instead, if the call fails record the endpoint and the error only.
2020-08-26 10:11:26 -07:00
Harshavardhana 7778fef6bb
update continous heal metrics appropriately for scanned items (#10352)
bonus make sure to ignore objectNotFound, and versionNotFound
errors properly at all layers, since HealObjects() returns
objectNotFound error if the bucket or prefix is empty.
2020-08-26 08:53:33 -07:00
飞雪无情 ea1803417f
Use constants for gateway names to avoid bugs caused by spelling. (#10355) 2020-08-26 08:52:46 -07:00
飞雪无情 ea5094e842
Add missing go mod libraries for mint. (#10347)
The aws-sdk-go,healthcheck,security module in mint
lacks required dependency libraries,now add them in go.mod
2020-08-25 17:51:12 -07:00
Minio Trusted 5a974fb10c Update yaml files to latest version RELEASE.2020-08-26T00-00-49Z 2020-08-26 00:19:30 +00:00
Anis Elleuch 9acdeab73d
lifecycle: Accept document without expiration (#10348) 2020-08-25 12:38:59 -07:00
Harshavardhana d19b434ffc
fix: bring back delayed leaf detection in listing (#10346) 2020-08-25 12:26:48 -07:00
Klaus Post 17a1eda702
Disregard healing disks in crawling (#10349)
When crawling never use a disk we know is healing.

Most of the change involves keeping track of the original endpoint on xlStorage
and this also fixes DiskInfo.Endpoint never being populated.

Heal master will print `data-crawl: Disk "http://localhost:9001/data/mindev/data2/xl1" is 
Healing, skipping` once on a cycle (no more often than every 5m).
2020-08-25 10:55:15 -07:00
Minio Trusted 7d50a0cfea Update yaml files to latest version RELEASE.2020-08-25T00-21-20Z 2020-08-25 00:44:39 +00:00
Harshavardhana ceff7bcca5 fix: ruleguard gopath link 2020-08-24 16:25:00 -07:00
Daniel Valdivia 7d1734d033
indicate through HTTP header cluster healing in progress (#10342) 2020-08-24 15:20:50 -07:00
Harshavardhana 03ec6adfd0
fix: KES http2.0 communication support (#10341) 2020-08-24 14:37:53 -07:00
Harshavardhana 309b10f201 keep crawler cycle at 5 minutes 2020-08-24 14:05:16 -07:00
飞雪无情 2a8e40f19f
add chroot zh_CN document (#10337) 2020-08-24 13:58:19 -07:00
KevinSmile 5f7bd2b1da
fix: lifecycle-expiration validation bug (#10327) 2020-08-24 13:56:50 -07:00
Klaus Post c097ce9c32
continous healing based on crawler (#10103)
Design: https://gist.github.com/klauspost/792fe25c315caf1dd15c8e79df124914
2020-08-24 13:47:01 -07:00
Harshavardhana caad314faa
add ruleguard support, fix all the reported issues (#10335) 2020-08-24 12:11:20 -07:00
Klaus Post bc2ebe0021
Only enforce quota on success (#10339)
We should only enforce quotas if no error has been returned.

firstErr is safe to access since all goroutines have exited at this point.

If `firstErr` hasn't been set by something else return the context error if cancelled.
2020-08-24 10:15:46 -07:00
飞雪无情 21e8440423
add compression zh_CN document (#10330) 2020-08-23 22:07:36 -07:00
Harshavardhana 11aa393ba7
Allow region errors to be dynamic (#10323)
remove other FIXMEs as we are not planning to fix these, 
instead we will add dynamism case by case basis.

fixes #10250
2020-08-23 22:06:22 -07:00
Praveen raj Mani d0c910a6f3
Support https and basic-auth for elasticsearch notification target (#10332) 2020-08-23 09:43:48 -07:00
Harshavardhana 81c90ae430
move to gorilla mux 1.8.0 (#10328) 2020-08-22 23:14:02 -07:00
kannappanr d15a5ad4cc
S3 Gateway: Check for encryption headers properly (#10309) 2020-08-22 11:41:49 -07:00
飞雪无情 0ff246653b
CN bucket policy document that is no longer needed can be removed, (#10310)
which is consistent with the English document.
2020-08-22 11:05:21 -07:00
飞雪无情 113bcbdb78
update minio limit zh_CN document (#10320) 2020-08-22 11:04:31 -07:00
Harshavardhana 95411228db
add missing cleanupStaleMultipartUploads (#10325)
fixes #10319
2020-08-21 21:39:54 -07:00
ebozduman 23774353b7
get_object() returns NoSuchKey error when object is a prefix (#10315) 2020-08-21 13:08:01 -07:00
Tobias Nygren 052b5262ff
use statvfs(2) for disk.GetInfo on NetBSD (#10257) 2020-08-20 20:13:06 -07:00
poornas a2a5ec93d3
fix: use global context for filling cache in the background (#10308) 2020-08-20 14:23:24 -07:00
飞雪无情 331c517a5b
translate bucket versioning design document to zh_CN (#10300) 2020-08-20 13:18:51 -07:00
Harshavardhana 27a774cbe9
fix: FS mode should reject putBucketVersioning (#10307) 2020-08-20 13:18:06 -07:00
Klaus Post 8e6787a302
Fix TestDataUpdateTracker hanging (#10302)
Keep dataUpdateTracker while goroutine is starting.

This will ensure the object is updated one `start` returns

Tested with

```
λ go test -cpu=1,2,4,8 -test.run TestDataUpdateTracker -count=1000
PASS
ok      github.com/minio/minio/cmd      8.913s
```

Fixes #10295
2020-08-20 13:17:42 -07:00
Harshavardhana 59352d0ac2
load all blocking metadata in background (#10298)
most of this metadata already has fallbacks
and there is no good reason to load them
in blocking fashion
2020-08-20 10:38:53 -07:00
Harshavardhana 75d44b3bae
add disk for more context in bitrot errors (#10296) 2020-08-20 09:41:15 -07:00
Klaus Post 95ae6c4b49
Fix missing unlock in *healSequence.hasEnded() (#10305)
The background healing sequence would always hang when this function is called.
2020-08-20 08:48:09 -07:00
Harshavardhana 98ca770f81
mint fixes for go1.14 (#10294)
add go.mod to ensure that mint can be
built with go1.14.x
2020-08-19 20:07:37 -07:00
Krishnan Parthasarathi ccd967e3be
Add ExpiresAt to LicenseInfo (#10293) 2020-08-19 19:21:04 -07:00
KevinSmile 0ebb73ee2e
use const instead of literals (#10292) 2020-08-19 16:43:52 -07:00
Harshavardhana c8b84a0e9e
Add nancy vulnerability scanner (#10289) 2020-08-19 14:25:21 -07:00
Ritesh H Shukla 3acb5cff45
Update code comment (#10287) 2020-08-19 14:24:58 -07:00
Harshavardhana ab801ad3d4
build edge containers with go1.14 (#10279) 2020-08-19 12:08:11 +05:30
Harshavardhana 74116204ce
handle fresh setup with mixed drives (#10273)
fresh drive setups when one of the drive is
a root drive, we should ignore such a root
drive and not proceed to format.

This PR handles this properly by marking
the disks which are root disk and they are
taken offline.
2020-08-18 14:37:26 -07:00
飞雪无情 2eb5f934d8
add bucket versioning zh_CN document (#10281) 2020-08-18 13:10:26 -07:00
Minio Trusted b43d376a87 Update yaml files to latest version RELEASE.2020-08-18T19-41-00Z 2020-08-18 20:04:22 +00:00