Commit Graph

5058 Commits

Author SHA1 Message Date
Harshavardhana 5996c8c4d5
feat: allow offline disks on a fresh start (#16541) 2023-02-06 09:26:09 -08:00
Harshavardhana 21885f9457
fix: liveness/readiness must return errors if KMS is unreachable (#16540) 2023-02-06 08:55:56 -08:00
Harshavardhana aa8b9572b9
remove double ENABLED help output (#16528) 2023-02-03 05:52:52 -08:00
Poorna 876e1a91b2
replication: Fix typo checking PreconditionFailed status code (#16517) 2023-02-02 19:22:02 +05:30
Anis Elleuch 2278fc8f47
Print original error when IAM load is failed in some places (#16511) 2023-02-01 17:32:22 +05:30
Harshavardhana a91f353621
support 'mc admin service restart' for windows (#16512) 2023-02-01 17:31:46 +05:30
Klaus Post cdb1b48ad9
Make localLocker lock attempts cancellable (#16510) 2023-01-31 09:41:17 -08:00
Kaan Kabalak 2d0f30f062
Fix typo in code comment (#16509) 2023-01-31 07:54:19 +05:30
Krishnan Parthasarathi cea2ca8c8e
Add restore-status header for multipart objects (#16508) 2023-01-31 07:53:45 +05:30
Klaus Post f713436dd0
Fix truncated list response on deleted replicated objects (#16504) 2023-01-30 09:13:53 -08:00
Klaus Post b923a62425
Check pool-index for invalid setups (#16501) 2023-01-30 18:33:07 +05:30
Harshavardhana 67fce4a5b3
fix: dangling delete() upon success should return 404 (#16494) 2023-01-27 12:43:45 -08:00
Poorna eaa65b7ade
fix replication healing on list to consider all versions (#16496) 2023-01-27 12:43:28 -08:00
Poorna 820d94447c
replication: fix target bucket passed on GET proxy (#16495) 2023-01-27 10:24:51 -08:00
Poorna ed20134a7b
replication: detect proxy header presence correctly (#16489) 2023-01-27 01:29:32 -08:00
Harshavardhana d19cbc81b5
fix: do not return IAM/Bucket metadata replication errors to client (#16486) 2023-01-26 11:11:54 -08:00
Anis Elleuch 1fd7946dce
Print golang http errors in MinIO log format (#16465) 2023-01-26 22:46:16 +05:30
Klaus Post 027ff0f3a8
fix: set modTime to current in snowball if archive shows empty (#16482) 2023-01-26 22:20:35 +05:30
Harshavardhana 54b561898f
fix: anonymize the x-amz-id-2 value from hostname (#16478) 2023-01-25 10:25:36 -08:00
Harshavardhana 65c104a589
add x-amz-id-2 to indicate the node that received the request (#16474) 2023-01-25 09:14:10 -08:00
Anis Elleuch 0a0416b6ea
Better error when setting up replication with a service account alias (#16472) 2023-01-25 21:50:12 +05:30
Anis Elleuch 441babdc41
Rename peer S3 prefix to avoid collision in the future (#16473) 2023-01-25 06:46:30 -08:00
Harshavardhana e64b9f6751
fix: disallow SSE-C encrypted objects on replicated buckets (#16467) 2023-01-24 15:46:33 -08:00
Florian Schwab d67a846ec4
allow restarting of decommissioning if completed, failed or canceld (#16464) 2023-01-24 07:07:59 -08:00
Poorna ca2a1c3f60
replication: clone metrics while loading metrics cache (#16462) 2023-01-24 02:10:32 -08:00
Poorna 93fbb228bf
Validate if parent user exists for service acct (#16443) 2023-01-24 08:17:18 +05:30
Anis Elleuch f37a5b6dae
Add CPU info in the check update user-agent (#16447) 2023-01-23 08:07:55 -08:00
Harshavardhana 31b0decd46
migrate to minio/mux from gorilla/mux (#16456) 2023-01-23 16:42:47 +05:30
Harshavardhana eb561e1c05
allow bootstrap platform checks to be pool specific (#16455) 2023-01-23 16:24:50 +05:30
Poorna ddad231921
replication: Avoid logging PreConditionFailed error (#16450) 2023-01-21 07:33:04 +05:30
Klaus Post 03b94f907f
fix: deleted object names for directory objects (#16448) 2023-01-20 21:16:06 +05:30
Shireesh Anjal 0f591d245d
fix: incorrect anonymization of drive endpoint (#16442) 2023-01-20 07:35:44 +05:30
Poorna 1b02e046c2
Fix bandwidth monitoring to be per remote target (#16360) 2023-01-19 18:52:16 +05:30
Harshavardhana d08e3cc895
add a way to avoid blocking queueHealTask() depending on caller (#16433) 2023-01-19 18:50:54 +05:30
Anis Elleuch d98116559b
Use async healing in PutObject call (#16431) 2023-01-19 00:54:22 -08:00
Krishnan Parthasarathi 71c95ad0d0
Signal stop-rebalance to all rebalancing pools (#16438) 2023-01-19 06:54:23 +05:30
Aditya Manthramurthy 698862ec5d
Fix transports/timeouts related regressions (#16427) 2023-01-18 10:06:38 +05:30
Harshavardhana b4ef5ff294
remove unnecessary code checking for supported features (#16423) 2023-01-17 19:37:47 +05:30
Harshavardhana 3db658e51e
use correct xml package for custom MarshalXML() (#16421) 2023-01-17 05:08:33 +05:30
Shireesh Anjal 5a9f7516d6
Add monthly license update job (#16391) 2023-01-17 05:08:15 +05:30
Anis Elleuch 3039fd4519
Optimize background heal status to use LocalStorageInfo (#16414) 2023-01-17 05:02:00 +05:30
Harshavardhana 095fc0561d
feat: allow decom of multiple pools (#16416) 2023-01-16 21:36:34 +05:30
Anis Elleuch beb1924437
Properly restart fresh disk healing when failed in some places (#16413) 2023-01-14 05:06:46 +05:30
jiuker c8e1154f1e
fix: reading from erasureDisks must be protected via read lock() (#16407) 2023-01-13 04:16:23 -08:00
Poorna b204c2dbec
fix: enforce deny on DeleteVersionAction (#16409) 2023-01-13 04:16:00 -08:00
Poorna b22b39de96
Avoid dangling deletes if disk not found (#16401) 2023-01-12 22:20:19 -08:00
Harshavardhana c242e6c391
fix: calculate common parity properly (#16406) 2023-01-13 03:28:16 +05:30
Anis Elleuch e05205756f
metrics: Add more logs when unable to read bucket usage (#16405) 2023-01-13 02:32:00 +05:30
Anis Elleuch 475a88b555
fix: error out if an object is found after a full decom (#16277) 2023-01-12 05:52:51 +05:30
Anis Elleuch 1ece3d1dfe
Add comment field to service accounts (#16380) 2023-01-10 21:57:52 +04:00
Anis Elleuch 2146ed4033
xl: Quit early when EC config is incorrect (#16390)
Co-authored-by: Anis Elleuch <anis@min.io>
2023-01-09 23:07:45 -08:00
Anis Elleuch ebd4388cca
s3: Return XMinioInvalidObjectName if the object contains null char (#16372) 2023-01-06 10:11:18 -08:00
Anis Elleuch 0333412148
fix: heal only once per disk per set among multiple disks (#16358) 2023-01-05 20:41:19 -08:00
Harshavardhana e0086c1be7
reduce startup delays on kubernetes (#16356) 2023-01-05 02:32:43 -08:00
Anis Elleuch 7883e55da2
Merge buckets list from different nodes in ListBuckets() call (#16357) 2023-01-04 08:53:58 -08:00
Harshavardhana a15a2556c3
converge listBuckets() as a peer call (#16346) 2023-01-03 23:39:40 -08:00
Harshavardhana f1bbb7fef5
vectorize cluster-wide calls such as bucket operations (#16313) 2023-01-03 08:16:39 -08:00
Harshavardhana 1cd8e1d8b6
remove the startup jitter before locks() (#16340) 2023-01-02 01:40:09 -08:00
jiuker 62cd918061
fix: close helmInfo file descriptor (#16319) 2023-01-01 23:26:59 -08:00
Klaus Post 6a04067514
fix: tweak read buffer size to reduce over-reading (#16338) 2023-01-01 08:14:20 -08:00
Taran Pelkey 49b3908635
fix: misplaced write response command in DetachPolicy() (#16333) 2022-12-30 20:04:03 -08:00
Harshavardhana f93183f66e
fix: a deadlock by refactoring listBuckets() under site replication (#16323) 2022-12-29 00:08:31 -08:00
Harshavardhana 2937711390
fix: DeleteObject() API with versionId under replication (#16325) 2022-12-28 22:48:33 -08:00
Anis Elleuch 27417459fb
metrics: Show healing info for all nodes (#16315) 2022-12-26 08:35:32 -08:00
Harshavardhana 5b8fe2e89a
allow locks with object affinity to spread across pools (#16312) 2022-12-23 20:55:45 -08:00
Anis Elleuch acc9c033ed
debug: Add X-Amz-Request-ID to lock/unlock calls (#16309) 2022-12-23 19:49:07 -08:00
Poorna 8528b265a9
Validate replication target update to avoid duplicate endpoints (#16311) 2022-12-23 15:44:48 -08:00
Harshavardhana b882310e2b
avoid locks for internal and invalid buckets in MakeBucket() (#16302) 2022-12-23 07:46:00 -08:00
Poorna de0b43de32
persist replication stats with leader lock (#16282) 2022-12-22 14:25:13 -08:00
jiuker 29dd7f1d68
tier verification leaks fd, that must be closed (#16296)
Co-authored-by: Harshavardhana <harsha@minio.io>
2022-12-22 10:35:54 -08:00
Poorna 6423e4c767
Remove site replication config if it succeeded locally (#16279) 2022-12-22 01:31:20 -08:00
Krishnan Parthasarathi 2fa35def2c
Fix DeleteObject when only free versions remain (#16289) 2022-12-21 16:24:07 -08:00
Anis Elleuch 34167c51d5
trace: Add bootstrap tracing events (#16286) 2022-12-21 15:52:29 -08:00
Harshavardhana a5f8af4efb
serialize replication stats() only when needed (#16280) 2022-12-20 00:07:53 -08:00
Harshavardhana 5a218f38a1
allow retries for transaction lock on startup (#16273) 2022-12-19 22:00:00 -08:00
Anis Elleuch e57e946206
Do not save credentials in config.json (#16275) 2022-12-19 12:27:06 -08:00
Klaus Post b4f71362e9
Avoid config migration on every startup (#16278) 2022-12-19 11:10:14 -08:00
Taran Pelkey ed37b7a9d5
Add API to fetch policy user/group associations (#16239) 2022-12-19 10:37:03 -08:00
Anis Elleuch 89db3fdb5d
Do not return an error when version disparity is detected (#16269) 2022-12-16 08:52:12 -08:00
Harshavardhana 80fc3a8a52
use newDynamicTimeoutWithOpts() when appropriate (#16266) 2022-12-15 13:11:37 -08:00
Klaus Post 988a2e8fed
Faster startup of large distributed systems with latency (#16259) 2022-12-15 08:31:21 -08:00
Harshavardhana 2433698372
fix: remove unnecessary logs for client conn errors (#16261) 2022-12-15 08:25:05 -08:00
Harshavardhana 5d7e8f79ed
fix: remove scanner healing with unnecessary logs (#16260) 2022-12-14 16:39:18 -08:00
Harshavardhana bad229e16e
fix: support event name s3:Restore:* (#16257) 2022-12-14 05:12:07 -08:00
Poorna d37e514733
Cleanup remote targets automatically on replication config removal. (#16221) 2022-12-14 03:24:06 -08:00
Harshavardhana c73ea27ed7
do not log checksum mismatch error, client received the error (#16246) 2022-12-14 01:57:40 -08:00
Krishnan Parthasarathi 0159b56717
fix: rebalance to account for object's on-disk size (#16240) 2022-12-14 00:15:14 -08:00
Aditya Manthramurthy 9e6cc847f8
Add HTTP2 config option for policy plugin (#16225) 2022-12-13 14:28:48 -08:00
Taran Pelkey 709eb283d9
Add endpoints for managing IAM policies (#15897)
Co-authored-by: Taran <taran@minio.io>
Co-authored-by: ¨taran-p¨ <¨taran@minio.io¨>
Co-authored-by: Aditya Manthramurthy <donatello@users.noreply.github.com>
2022-12-13 12:13:23 -08:00
Anis Elleuch 76dde82b41
Implement STS account info API (#16115) 2022-12-13 08:38:50 -08:00
Anis Elleuch 939c0100a6
log: Do not interpret verbs in object names in console output (#16233) 2022-12-13 08:27:40 -08:00
Aditya Manthramurthy 2d60bf8c50
Refactor HTTP transports (#16222) 2022-12-12 20:31:21 -08:00
Harshavardhana 37e20f6ef2
feat: allow listening specific addrs for API port (#16223) 2022-12-12 18:48:46 -08:00
Harshavardhana 2fc182d8e6
fix: iso8601TimeFormat padding issue for certain nanoseconds (#16207) 2022-12-12 10:28:30 -08:00
Shireesh Anjal a2cbeaa9e6
Use different subnet public key during dev/test (#16216) 2022-12-12 10:28:15 -08:00
Harshavardhana 444ff20bc5
do not rename multipart failed transactions back to tmp (#16204) 2022-12-12 01:40:29 -08:00
Harshavardhana 20ef5e7a6a
avoid double deletes() when no more versions (#16206) 2022-12-12 01:40:04 -08:00
Aditya Manthramurthy e06127566d
Add IAM API to attach/detach policies for LDAP (#16182) 2022-12-09 13:08:33 -08:00
Harshavardhana dfe73629a3
fix: delete marker discrepancies via DeleteObject() API (#16195) 2022-12-08 18:15:16 -08:00
Harshavardhana b03dd1af17
remove hard limit for number of buckets (#16194) 2022-12-08 12:24:03 -08:00
Harshavardhana 4bc367c490
fix: translate tier add errors properly (#16191) 2022-12-08 11:18:07 -08:00
Klaus Post 3eb2d086b2
Replace filepathx with fork (#16192) 2022-12-08 10:42:44 -08:00
Klaus Post 70986b6e6e
Add version id to healresult (#16193) 2022-12-08 07:49:10 -08:00
Klaus Post ebe395788b
feat: Encrypt s3zip file index (#16179) 2022-12-07 14:56:07 -08:00
Klaus Post 12fd6678ee
Encrypt checksums with KMS on CompleteMultipartUpload (#16177) 2022-12-07 10:18:18 -08:00
Harshavardhana 90d35b70b4
remove unnecessary logs for truncated XML inputs (#16184) 2022-12-07 08:30:52 -08:00
Javier Adriel 04ae9058ed
Populate end_session_endpoint (#16183) 2022-12-06 16:56:37 -08:00
Aditya Manthramurthy a30cfdd88f
Bump up madmin-go to v2 (#16162) 2022-12-06 13:46:50 -08:00
Anis Elleuch 1bae32dc96
xl: Delete older data-dir when replacing an existing version-id (#16176) 2022-12-06 13:43:18 -08:00
Anis Elleuch 932d2c3c62
Add X-Amz-Request-Id to internode calls (#16146) 2022-12-06 09:27:26 -08:00
jiuker 8d8d07ac5c
use readlock instead of writelock to get heal information (#16175) 2022-12-06 08:08:22 -08:00
Anis Elleuch 44735be38e
s3: Return correct error when Version is invalid in policy document (#16178) 2022-12-06 08:07:24 -08:00
Klaus Post 3fd9059b4e
opt: Only stream big data usage caches (#16168) 2022-12-05 13:01:11 -08:00
Klaus Post a713aee3d5
Run staticcheck on CI (#16170) 2022-12-05 11:18:50 -08:00
Andreas Auernhammer d882ba2cb4
kms: add support for KES enclaves (#16139)
Signed-off-by: Andreas Auernhammer <hi@aead.dev>
2022-12-04 02:34:24 -08:00
jiuker 6086f45d25
fix: in disk cache readCacheFileStream should closed upon return (#16138) 2022-12-04 02:28:10 -08:00
Klaus Post 98cffbce03
s3zip: Limit over-read for single file (#16161) 2022-12-02 08:53:24 -08:00
Klaus Post 1cd875de1e
Persist updated metadata (#16160) 2022-12-02 08:35:04 -08:00
Harshavardhana 5a8df7efb3
re-implement StorageInfo to be a peer call (#16155) 2022-12-01 14:31:35 -08:00
Anis Elleuch c84e2939e4
trace: Publish storage layer errors (#16153) 2022-12-01 12:10:54 -08:00
Anis Elleuch 641ab24aec
repl: resync orchestrator to use global shared lock (#16154) 2022-12-01 12:10:09 -08:00
Harshavardhana 71133105d7
re-order the top-level config keys for priority (#16150) 2022-12-01 07:50:08 -08:00
Aditya Manthramurthy 87cbd41265
feat: Allow at most one claim based OpenID IDP (#16145) 2022-11-29 15:40:49 -08:00
Klaus Post cc1d8f0057
Check for abandoned data when healing (#16122) 2022-11-28 10:20:55 -08:00
Anis Elleuch 1f1dcdce65
move HTTP recorder to an internal library (#16128) 2022-11-28 10:20:27 -08:00
Shireesh Anjal 98a67a3776
Improvements in logger and audit webhooks (#16102) 2022-11-28 08:03:26 -08:00
Poorna 63fc6ba2cd
preserve replicated ETag properly on target (#16129) 2022-11-26 14:43:32 -08:00
jiuker ce53d7f6c2
add disk.Close() in healFreshDisk to indicate idiomatic flow of code (#16124) 2022-11-26 00:26:15 -08:00
jiuker fe8eed963e
fix: wrapped error will not equal in decommissioning (#16113) 2022-11-24 08:00:42 -08:00
Shireesh Anjal 59f877fc64
fix: Timestamp not added in diagnostics report (#16114) 2022-11-23 07:11:22 -08:00
Klaus Post f96fe9773c
fix: duplicated shared prefix with custom delimiter when listing (#16111) 2022-11-22 08:51:04 -08:00
Anis Elleuch 04948b4d55
fix: checking for stale STS account under site replication (#16109) 2022-11-22 07:26:33 -08:00
Klaus Post 98ba622679
Reduce temporary file clean-up waits (#16110) 2022-11-22 07:23:36 -08:00
Harshavardhana 08103870a5
update single drive setup error message (#16098) 2022-11-18 14:47:38 -08:00
Anis Elleuch 993e586855
config: return XMinioConfigNotFound code for non existing config (#16065) 2022-11-18 10:28:14 -08:00
Harshavardhana 58ec835af0
fix: skip free version ID and marker in metadata equality (#16093) 2022-11-18 05:48:22 -08:00
Harshavardhana 6aea950d74
avoid partID lock validating uploadID exists prematurely (#16086) 2022-11-18 03:09:35 -08:00
Poorna 7198be5be9
bucket resync: persist reset id to bucket metadata (#16088) 2022-11-18 01:39:05 -08:00
Klaus Post a22b4adf4c
distribute replication ops based on names (#16083) 2022-11-17 15:20:09 -08:00
Klaus Post b7bb122be8
fix: replication auto-scaling deadlock (#16084) 2022-11-17 07:35:02 -08:00
Krishnan Parthasarathi 8441a3bf5f
fix: update metacache entry only once (#16072) 2022-11-16 11:25:00 -08:00
Harshavardhana 853c4de75a
allow changing endpoints in distributed setups (#16071) 2022-11-16 07:59:10 -08:00
jiuker 3597af789e
allow resultCh to be closed() after clusterMetaHealthInfo() (#16073) 2022-11-16 03:04:36 -08:00
Shireesh Anjal 5246e3be84
Send health diagnostics data as part of callhome (#16006) 2022-11-15 13:53:05 -08:00
Klaus Post 8a07000e58
fix: refactor getReplicationDiff for safe use (#16051) 2022-11-15 07:59:21 -08:00
Krishnan Parthasarathi 3bb82ef60d
top-locks: Include lock-held duration (#16061) 2022-11-15 07:57:52 -08:00
Harshavardhana 91f45c4aa6
avoid inconsistent versions healing when versions are large (#16066) 2022-11-14 18:35:26 -08:00
Poorna d6bc141bd1
feat: Add support for site level resync (#15753) 2022-11-14 07:16:40 -08:00
jiuker 7ac64ad24a
fix: use errors.Is for wrapped returns (#16062) 2022-11-14 07:15:46 -08:00
Harshavardhana 6d76db9d6c
improve server startup error when pools are incorrect (#16056) 2022-11-11 19:40:45 -08:00