Commit Graph

4892 Commits

Author SHA1 Message Date
Klaus Post
98cffbce03
s3zip: Limit over-read for single file (#16161) 2022-12-02 08:53:24 -08:00
Klaus Post
1cd875de1e
Persist updated metadata (#16160) 2022-12-02 08:35:04 -08:00
Harshavardhana
5a8df7efb3
re-implement StorageInfo to be a peer call (#16155) 2022-12-01 14:31:35 -08:00
Anis Elleuch
c84e2939e4
trace: Publish storage layer errors (#16153) 2022-12-01 12:10:54 -08:00
Anis Elleuch
641ab24aec
repl: resync orchestrator to use global shared lock (#16154) 2022-12-01 12:10:09 -08:00
Harshavardhana
71133105d7
re-order the top-level config keys for priority (#16150) 2022-12-01 07:50:08 -08:00
Aditya Manthramurthy
87cbd41265
feat: Allow at most one claim based OpenID IDP (#16145) 2022-11-29 15:40:49 -08:00
Klaus Post
cc1d8f0057
Check for abandoned data when healing (#16122) 2022-11-28 10:20:55 -08:00
Anis Elleuch
1f1dcdce65
move HTTP recorder to an internal library (#16128) 2022-11-28 10:20:27 -08:00
Shireesh Anjal
98a67a3776
Improvements in logger and audit webhooks (#16102) 2022-11-28 08:03:26 -08:00
Poorna
63fc6ba2cd
preserve replicated ETag properly on target (#16129) 2022-11-26 14:43:32 -08:00
jiuker
ce53d7f6c2
add disk.Close() in healFreshDisk to indicate idiomatic flow of code (#16124) 2022-11-26 00:26:15 -08:00
jiuker
fe8eed963e
fix: wrapped error will not equal in decommissioning (#16113) 2022-11-24 08:00:42 -08:00
Shireesh Anjal
59f877fc64
fix: Timestamp not added in diagnostics report (#16114) 2022-11-23 07:11:22 -08:00
Klaus Post
f96fe9773c
fix: duplicated shared prefix with custom delimiter when listing (#16111) 2022-11-22 08:51:04 -08:00
Anis Elleuch
04948b4d55
fix: checking for stale STS account under site replication (#16109) 2022-11-22 07:26:33 -08:00
Klaus Post
98ba622679
Reduce temporary file clean-up waits (#16110) 2022-11-22 07:23:36 -08:00
Harshavardhana
08103870a5
update single drive setup error message (#16098) 2022-11-18 14:47:38 -08:00
Anis Elleuch
993e586855
config: return XMinioConfigNotFound code for non existing config (#16065) 2022-11-18 10:28:14 -08:00
Harshavardhana
58ec835af0
fix: skip free version ID and marker in metadata equality (#16093) 2022-11-18 05:48:22 -08:00
Harshavardhana
6aea950d74
avoid partID lock validating uploadID exists prematurely (#16086) 2022-11-18 03:09:35 -08:00
Poorna
7198be5be9
bucket resync: persist reset id to bucket metadata (#16088) 2022-11-18 01:39:05 -08:00
Klaus Post
a22b4adf4c
distribute replication ops based on names (#16083) 2022-11-17 15:20:09 -08:00
Klaus Post
b7bb122be8
fix: replication auto-scaling deadlock (#16084) 2022-11-17 07:35:02 -08:00
Krishnan Parthasarathi
8441a3bf5f
fix: update metacache entry only once (#16072) 2022-11-16 11:25:00 -08:00
Harshavardhana
853c4de75a
allow changing endpoints in distributed setups (#16071) 2022-11-16 07:59:10 -08:00
jiuker
3597af789e
allow resultCh to be closed() after clusterMetaHealthInfo() (#16073) 2022-11-16 03:04:36 -08:00
Shireesh Anjal
5246e3be84
Send health diagnostics data as part of callhome (#16006) 2022-11-15 13:53:05 -08:00
Klaus Post
8a07000e58
fix: refactor getReplicationDiff for safe use (#16051) 2022-11-15 07:59:21 -08:00
Krishnan Parthasarathi
3bb82ef60d
top-locks: Include lock-held duration (#16061) 2022-11-15 07:57:52 -08:00
Harshavardhana
91f45c4aa6
avoid inconsistent versions healing when versions are large (#16066) 2022-11-14 18:35:26 -08:00
Poorna
d6bc141bd1
feat: Add support for site level resync (#15753) 2022-11-14 07:16:40 -08:00
jiuker
7ac64ad24a
fix: use errors.Is for wrapped returns (#16062) 2022-11-14 07:15:46 -08:00
Harshavardhana
6d76db9d6c
improve server startup error when pools are incorrect (#16056) 2022-11-11 19:40:45 -08:00
jiuker
bdcb485740
netPerfRX Reset() should use write Lock() (#16043) 2022-11-10 19:44:20 -08:00
Poorna
e32b948a49
fix: parsing multipart uploadID under site replicated setup (#16048)
continue the fix from #16034
2022-11-10 16:17:45 -08:00
Klaus Post
5b242f1d11
Add Audit target metrics (#16044) 2022-11-10 10:20:21 -08:00
Poorna
34d28dd79f
replication: Avoid blocking on mrf save (#16045) 2022-11-10 10:20:02 -08:00
Krishnan Parthasarathi
6eef9b4a23
lifecycle: simplify Eval and HasActiveRules (#16036) 2022-11-10 07:17:45 -08:00
Aditya Manthramurthy
5f1999cc71
fix: avoid URL unsafe chars in multipart upload ID (#16034) 2022-11-09 16:41:16 -08:00
Krishnan Parthasarathi
40a2c6b882
Return remote tier as StorageClass for transitioned objects (#16035) 2022-11-09 15:57:34 -08:00
jiuker
7b7356f04c
close the reader under disk cache bitrot verification (#16024) 2022-11-09 04:20:11 -08:00
Klaus Post
bbc312fce6
Add notification queue metrics (#16026) 2022-11-08 16:36:47 -08:00
Anis Elleuch
7260241511
Remove some logs caused by external apps (#16027) 2022-11-08 13:29:05 -08:00
Anis Elleuch
3b1a9b9fdf
Use the same lock for the scanner and site replication healing (#15985) 2022-11-08 08:55:55 -08:00
Harshavardhana
72afc2727a
rebalance status must return appropriate error initially (#16022) 2022-11-08 07:56:45 -08:00
Aditya Manthramurthy
76d822bf1e
Add LDAP policy entities API (#15908) 2022-11-07 14:35:09 -08:00
Klaus Post
ddeca9f12a
fix: filter rest errors and logs returned (#16019) 2022-11-07 10:38:08 -08:00
Harshavardhana
1f3db03bf0
allow changing argument for path for SNSD setup (#16013) 2022-11-07 00:11:58 -08:00
Harshavardhana
944c62daf4
skip flaky tests on windows OS (#16015) 2022-11-07 00:11:21 -08:00
Harshavardhana
9547b7d0e9
add deadlineConnections on remoteTransport (#16010) 2022-11-05 11:09:21 -07:00
Klaus Post
808ecfe0f2
merge versions across sets when listing (#16003) 2022-11-04 11:33:22 -07:00
Klaus Post
2894dd4d1a
fix: hold lock while serializing replication stats (#16007) 2022-11-04 09:59:14 -07:00
jiuker
fd8750e959
fix: http body must be drained in downloadBinary() (#16001) 2022-11-04 08:22:38 -07:00
Poorna
4f5d38a4b1
site replication edit: validate endpoint belongs to deployment (#16000) 2022-11-03 16:23:45 -07:00
Anis Elleuch
7e73fc2870
Implement inspect data API v2 (#15474)
Co-authored-by: Klaus Post <klauspost@gmail.com>
2022-11-02 13:36:38 -07:00
Harshavardhana
0d49b365ff
converge SNSD deployments into single code (#15988) 2022-11-01 16:41:01 -07:00
Anis Elleuch
7721595aa9
config: Deprecated delay/max_wait/scanner and introduce speed (#15941) 2022-11-01 08:04:07 -07:00
Harshavardhana
fd6f6fc8df
cleanup stale parent multipart directories (#15980) 2022-11-01 08:00:02 -07:00
Aditya Manthramurthy
4fb47cd568
fix: update admin IDP APIs to be more RESTful (#15896) 2022-10-31 14:52:26 -07:00
Klaus Post
ecc932d5dd
Clean entire tmp-old on restart (#15979) 2022-10-31 07:27:50 -07:00
Harshavardhana
b57fbff7c1
ignore background healInfo in single drive setup (#15968) 2022-10-31 07:26:10 -07:00
Poorna
d765b89a63
improve validation for replication resync API (#15964) 2022-10-28 23:21:33 -07:00
Harshavardhana
6e4acf0504
add a message of removal for gateway and hide the command (#15965) 2022-10-28 14:11:20 -07:00
Klaus Post
71954faa3a
mark pubsub type safe via generics (#15961) 2022-10-28 10:55:42 -07:00
Klaus Post
0f0e154315
fix: inconsistent replication delete marker timestamps (#15956) 2022-10-27 09:46:52 -07:00
Harshavardhana
136d41775f
remove numAvailableDisks check as it doesn't serve any purpose (#15954) 2022-10-27 09:05:24 -07:00
Harshavardhana
ec77d28e62
make subnet subsys dynamic and simplify callhome (#15927) 2022-10-27 00:20:01 -07:00
Klaus Post
86420a1f46
Store multipart checksums (#15953) 2022-10-26 18:14:58 -07:00
Poorna
7dd8b6c8ed
ensure ILM expiry creates non null deleteMarker for versioned bucket (#15947) 2022-10-26 16:09:27 -07:00
Anis Elleuch
533c9d4fe3
fix: lockName to disallow parallel same erasure set healing (#15951) 2022-10-26 12:43:54 -07:00
Anis Elleuch
a35ef155fc
return appropriate error status code in the lock handler (#15950) 2022-10-26 09:51:26 -07:00
Poorna
8dd3c41b2a
allow MakeBucket errors to be handled lazily (#15945)
remote error is not required to be passed back to the 
client - this is mostly because we have healing that should 
eventually, catch up on this and heal the bucket.
2022-10-25 23:32:37 -07:00
Krishnan Parthasarathi
4523da6543
feat: introduce pool-level rebalance (#15483) 2022-10-25 12:36:57 -07:00
Poorna
ce8456a1a9
proxy multipart to peers via multipart uploadID (#15926) 2022-10-25 10:52:29 -07:00
Poorna
9ce1884732
reject editing bucket replication config when site replication is enabled (#15937) 2022-10-24 20:24:32 -07:00
Harshavardhana
23b329b9df
remove gateway completely (#15929) 2022-10-24 17:44:15 -07:00
Krishnan Parthasarathi
0c34e51a75
Filter out tiering metadata during CopyObject (#15936) 2022-10-24 16:32:31 -07:00
Anis Elleuch
fc6c794972
Audit dangling object removal (#15933) 2022-10-24 11:35:07 -07:00
Klaus Post
86d543d0f6
Check for s3zip content offset (#15924) 2022-10-21 15:37:48 -07:00
Poorna
e4e90b53c1
fix: delete-marker replication check properly (#15923) 2022-10-21 14:45:06 -07:00
Anis Elleuch
58d776daa0
Set CONSOLE_MINIO_SERVER to 127.0.0.1 by default (#15887) 2022-10-21 14:42:28 -07:00
Krishnan Parthasarathi
f6b2e89109
Pass encrypted etag as is for immediate tiering (#15925) 2022-10-21 14:40:50 -07:00
Anis Elleuch
ac85c2af76
lifecycle: refactor rules filtering and tagging support (#15914) 2022-10-21 10:46:53 -07:00
Shireesh Anjal
5aba2aedb3
Do not freeze s3 traffic in healthinfo api (#15912) 2022-10-21 00:34:32 -07:00
Harshavardhana
a8332efa94
fix: Delete() of bucket metadata should not parse the config (#15904) 2022-10-19 17:55:09 -07:00
Aditya Manthramurthy
3dbef72dc7
fix: AccountInfo API for roleARN based accounts (#15907) 2022-10-19 17:54:41 -07:00
Aditya Manthramurthy
2d16e74f38
Add LDAP IDP Configuration APIs (#15840) 2022-10-19 11:00:10 -07:00
Anis Elleuch
de5070446d
Deprecate --listeners flag (#15900) 2022-10-19 08:45:50 -07:00
Harshavardhana
374abd1e7d
add filter support for tags and metadata in batch replication (#15885) 2022-10-18 21:22:21 -07:00
Anis Elleuch
0506d9e83d
storage: Return errDiskNotFound when a peer is during shutdown (#15868) 2022-10-18 13:50:46 -07:00
Klaus Post
bd3dfad8b9
Add concurrent Snowball extraction + options (#15836) 2022-10-18 13:50:21 -07:00
Harshavardhana
9fff315555
do not need to trace ignored objects (#15894) 2022-10-18 13:47:55 -07:00
Anis Elleuch
18fb86b7be
convert context.DeadlineExceed to offline disk in DiskInfo() (#15886) 2022-10-18 03:01:16 -07:00
Harshavardhana
58a8275e84
do not assume invalid buf to be non-xl.meta (#15843) 2022-10-17 09:39:21 -07:00
Aditya Manthramurthy
85fc7cea97
Pass role ARN for OIDC providers to console (#15862) 2022-10-15 12:57:03 -07:00
Harshavardhana
328d660106
support CRC32 Checksums on single drive setup (#15873) 2022-10-15 11:58:47 -07:00
Harshavardhana
c68910005b
validate bucket before attempting batch replication (#15861) 2022-10-15 11:58:31 -07:00
Harshavardhana
c79bcc8838 Revert "convert context.DeadlineExceed to offline disk in DiskInfo() (#15869)"
This reverts commit 0fe58dbb34.
2022-10-14 20:37:50 -07:00
Anis Elleuch
0fe58dbb34
convert context.DeadlineExceed to offline disk in DiskInfo() (#15869) 2022-10-14 19:32:13 -07:00