Commit Graph

5184 Commits

Author SHA1 Message Date
Allan Roger Reid 483b226cc1
fix: avoid logging when object/version not found in replication (#16919) 2023-03-29 15:02:45 -07:00
Harshavardhana 8e02660a0d
update all our deps (#16899) 2023-03-28 03:45:24 -07:00
jiuker 16feef2a2c
fix: time.Parse RFC3339Nano (#16892) 2023-03-27 11:51:54 -07:00
jiuker 66ff17e452
fix: Avoid multiple write responses (#16894) 2023-03-27 09:15:23 -07:00
Harshavardhana 4c5edacae2
ignore operation timedout errors (#16891) 2023-03-26 03:16:51 -07:00
Anis Eleuch 8b4d0255b7
Set Console global Root CAs early to trust provided certs (#16890) 2023-03-25 09:58:38 -07:00
Poorna 74040b457b
Allow setting sync mode for site replication (#16876) 2023-03-24 14:41:23 -07:00
Klaus Post 5e3bfd2148
Add user tags when listing with metadata (#16883) 2023-03-23 10:27:19 -07:00
Klaus Post 8b0ab6ead6
Revert "Make localLocker lock attempts cancellable (#16510)" (#16884) 2023-03-23 10:26:21 -07:00
Shubhendu b1b0aadabf
Revert query parameter `src` from diag upload if callhome enabled (#16881) 2023-03-23 00:24:58 -07:00
Harshavardhana ac7d9c449a
add missing expiration information from 'sts info' (#16878) 2023-03-22 16:47:02 -07:00
Anis Eleuch 1346561b9d
return quorum error instead of insufficient storage error (#16874) 2023-03-22 16:22:37 -07:00
Minio Trusted 4bc52897b2 Update yaml files to latest version RELEASE.2023-03-22T06-36-24Z 2023-03-22 21:16:15 +00:00
Krishnan Parthasarathi 6017b63a06
top-locks: Group by lock request ID (#16860) 2023-03-21 18:35:29 -07:00
Harshavardhana 0448728228
fix: add deadline conns and dnsCache for remote transports (#16865) 2023-03-21 08:49:20 -07:00
Harshavardhana 12047702f5
fix: tweak the maintenance=true to satisfy baremetal first (#16864) 2023-03-21 08:48:38 -07:00
Harshavardhana fb1492f531
check for quorum errors for DeleteBucket() (#16859) 2023-03-20 23:38:06 -07:00
Aditya Manthramurthy 05444a0f6a
Use the official pub key to always verify binary (#16857) 2023-03-20 13:16:18 -07:00
Harshavardhana b3c54ec81e
reject object names with '\' on windows (#16856) 2023-03-20 13:16:00 -07:00
Harshavardhana 6c11dbffd5
add crash protection from backend modifications (#16846) 2023-03-20 09:08:42 -07:00
Harshavardhana 3b5dbf9046
allow bootstrapping to validate internode tokens (#16853) 2023-03-20 01:40:24 -07:00
Aditya Manthramurthy 09c733677a
Add test for fixed post policy exploit (#16855) 2023-03-20 01:06:45 -07:00
Harshavardhana 8d6558b236
fix: convert '\' to '/' on windows (#16852) 2023-03-20 00:35:25 -07:00
Aditya Manthramurthy 67f4ba154a
fix: post policy request security bypass (#16849) 2023-03-19 21:15:20 -07:00
Poorna 440ad20c1d
Add support for batch job cancellation (#16843) 2023-03-17 23:42:43 -07:00
Krishnan Parthasarathi 31fba6f434
Save bootstrap trace events in a circular buffer (#16823) 2023-03-17 16:01:03 -07:00
Harshavardhana 280442e533
reduce 250ms to 50ms retry looking for metacache block (#16795) 2023-03-17 14:44:01 -07:00
Shubhendu 850a945a18
Added query parameter `src` to diag upload if callhome enabled (#16837)
Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>
2023-03-17 08:30:16 -07:00
Harshavardhana 46f9049fb4
simplify error responses for KMS (#16793) 2023-03-16 11:59:42 -07:00
Aditya Manthramurthy 58266c9e2c
Add enable flag for LDAP IDP config (#16805) 2023-03-16 11:58:59 -07:00
Poorna d1e775313d
support decommissioning of tiered objects (#16751) 2023-03-16 07:48:05 -07:00
Harshavardhana e700be8cd6
fix: return appropriate Location header for MakeBucket() (#16820) 2023-03-15 13:40:40 -07:00
Harshavardhana e0f4dd6027
remove unncessary logs from WalkDir(), PutObject() (#16818) 2023-03-15 11:52:23 -07:00
Harshavardhana de02eca467
restore rotating root credentials properly (#16812) 2023-03-15 08:07:42 -07:00
Harshavardhana c7f7e67a10
Do not allow adding root user to IAM subsystem (#16803) 2023-03-13 12:46:17 -07:00
Klaus Post 628042e65e
tests: Protect globalLocalDrives against races (#16800) 2023-03-13 06:04:20 -07:00
Aditya Manthramurthy 6305b206e1
fix: site-repl should heal STS with virtual parent (#16792) 2023-03-10 16:21:51 -08:00
Klaus Post d85da9236e
Add Object Version count histogram (#16739) 2023-03-10 08:53:59 -08:00
Anis Elleuch 5c087bdcad
fix: a cosmetic error reporting with a lock timeout (#16788) 2023-03-09 15:16:13 -08:00
Klaus Post a547bf517d
Remove locks on usage cache (#16786) 2023-03-09 15:15:46 -08:00
Harshavardhana b984bf8d1a
allow expiration of all versions during Listing() (#16757) 2023-03-09 15:15:30 -08:00
Poorna fb6ab1cca2
fix: allow replication of 'null' delete markers (#16773) 2023-03-08 07:03:29 -08:00
Krishnan Parthasarathi 56c57e2c53
tier-stats: Avoid repeated logs (#16774) 2023-03-07 16:43:05 -08:00
Poorna a6057c35cc
Avoid peer notification when peer is offline, tune retries (#16737) 2023-03-07 08:13:28 -08:00
Harshavardhana 901887e6bf
feat: add lambda transformation functions target (#16507) 2023-03-07 08:12:41 -08:00
Poorna ee54643004
Avoid unnecessary replication heal attempts (#16769) 2023-03-07 07:43:38 -08:00
Harshavardhana 0a17acdb34
return error if policy changes on disabled groups (#16766) 2023-03-06 10:46:24 -08:00
Harshavardhana 72e5212842
fix: handle syscall.EROFS also for osIsPermission() (#16765) 2023-03-06 08:56:29 -08:00
ferhat elmas 714283fae2
cleanup ignored static analysis (#16767) 2023-03-06 08:56:10 -08:00
ferhat elmas 3423028713
cleanup Go linter settings (#16736) 2023-03-04 20:57:35 -08:00
jiuker 9d062b37d7
return underlying error with BackendDown{} error (#16738) 2023-03-03 23:56:53 -08:00
jiuker f350953a19
calculate disk cache usage percent accurately (#16740) 2023-03-02 20:32:22 -08:00
Daniel Valdivia 958bba5b42
Attach creds, owner and region to madmin calls (#16658)
Signed-off-by: Daniel Valdivia <18384552+dvaldivia@users.noreply.github.com>
2023-03-02 14:35:24 -08:00
Poorna 0f2b95b497
fix a data race in IAM loading (#16742) 2023-03-02 14:32:54 -08:00
Klaus Post d07089ceac
Fix scanner deadlock on lost global lock (#16726) 2023-02-28 21:34:45 -08:00
Harshavardhana 0ff931dc76
fix: allow CORS to work by default (#16713) 2023-02-27 10:10:45 -08:00
Praveen raj Mani 4d708cebe9
Support adding service accounts with expiration (#16430)
Co-authored-by: Harshavardhana <harsha@minio.io>
2023-02-27 10:10:22 -08:00
dorman 4d7c8e3bb8
Remove redundant log (#16710)
Co-authored-by: z30001483 <zekaifeng2@huawei.com>
2023-02-27 09:59:47 -08:00
Aditya Manthramurthy 8cde38404d
Add metrics for custom auth plugin (#16701) 2023-02-27 09:55:18 -08:00
Krishnan Parthasarathi fe7bf6cbbc
Support tier-add if tier backend not empty (#16715) 2023-02-27 09:26:26 -08:00
Shubhendu 8b4eb2304b
Set logger webhook proxy on subnet proxy change (#16665)
Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>
2023-02-27 08:35:36 -08:00
Harshavardhana ae029191a3
liveness returns "busy" if queued requests > available capacity (#16719) 2023-02-27 08:34:52 -08:00
Harshavardhana bfedea9bad
fix: disk healing should honor the right pool/set index (#16712) 2023-02-27 04:55:32 -08:00
Aditya Manthramurthy 7777d3b43a
Remove globalSTSTLSConfig (#16709) 2023-02-26 23:37:00 -08:00
Aditya Manthramurthy 9ed4fc9687
Remove globalOpenIDConfig (#16708) 2023-02-25 21:01:37 -08:00
jiuker b49b39e99d
fix: errNoSuchPolicy should use errors.Is (#16656) 2023-02-25 00:01:37 -08:00
jiuker cd3a2de5a3
add isValidLocation to common parseLocation (#16690) 2023-02-25 08:09:20 +05:30
jiuker 6e8960ccdd
fix: delete globalProfiler should lock (#16697) 2023-02-25 08:07:44 +05:30
Aditya Manthramurthy e05f3d5d84
Remove globalLDAPConfig (#16706) 2023-02-25 08:07:22 +05:30
Anis Elleuch 94c6cb1323
tests: Add test for S3 API error codes (#16705) 2023-02-25 08:06:29 +05:30
Klaus Post 9acf1024e4
Remove bloom filter (#16682)
Removes the bloom filter since it has so limited usability, often gets saturated anyway and adds a bunch of complexity to the scanner.

Also removes a tiny bit of CPU by each write operation.
2023-02-24 09:03:31 +05:30
Harshavardhana a0f06eac2a
add Veeam SOS API first implementation (#16688) 2023-02-22 19:54:57 +05:30
jiuker 83fe1a2732
log: add more info about BROWSER_URL (#16678) 2023-02-22 19:54:05 +05:30
Harshavardhana 5c98223c89
add correct HostId instead of deploymentId for error responses (#16686) 2023-02-22 15:41:09 +05:30
jiuker 663a0b7783
save correct bucketInfo on it's indexes (#16685) 2023-02-22 14:08:34 +05:30
Anis Elleuch 6efe4d1df6
perf: Only remove generated data when no bucket name specified (#16610) 2023-02-21 21:21:40 -08:00
Daniel Valdivia fb17f97cf3
move audit and logger message structure to minio/pkg (#16655)
Signed-off-by: Daniel Valdivia <18384552+dvaldivia@users.noreply.github.com>
2023-02-21 21:21:17 -08:00
Poorna 9202c6e26a
Reload bucket targets when replication config is refreshed (#16684) 2023-02-21 21:18:49 -08:00
Poorna 59a5456091
fix: STS error translation to API error (#16683) 2023-02-22 09:57:48 +05:30
Allan Roger Reid 8bfe972bab
Set meaningful message from minio with env variable KMS_SECRET_KEY (#16584) 2023-02-22 07:13:01 +05:30
Klaus Post fd6622458b
Add detailed scanner trace output and notifications (#16668) 2023-02-21 09:33:33 -08:00
Poorna 8a08861dd9
fix: healing of replication config for endpoint changes (#16648) 2023-02-20 16:06:13 +05:30
Harshavardhana b66d7dc708
add missing x-amz-id-2 to event notification date (#16646) 2023-02-20 15:41:47 +05:30
Harshavardhana b94733ab31
avoid locks when unnecessary in SiteReplicationMetaInfo() (#16650) 2023-02-18 05:35:22 -08:00
Klaus Post 84bb7d05a9
fix: healing deadlocks and ordering (#16643) 2023-02-17 23:22:43 +05:30
Harshavardhana 98a84d88e2
fix: trim and ignore './' directories (#16642) 2023-02-17 07:15:03 -08:00
jiuker e470268c7c
fix: a possible closer leak in SelectObjectHandler (#16598) 2023-02-17 01:44:40 -08:00
jiuker 3a6cd4f73d
fix: sync.pool won't Put reader back (#16638) 2023-02-17 01:42:43 -08:00
Harshavardhana 6ea150fd68
fix: avoid printing certain errors under few locations (#16631) 2023-02-17 01:40:31 -08:00
Anis Elleuch a7188bc9d0
fix: evaluate BypassGov policy action in deletion correctly (#16635) 2023-02-17 07:53:34 +05:30
Krishnan Parthasarathi a1dd08f2e6
Include tier name in MinIO/S3 target user-agent (#16630) 2023-02-15 22:09:46 -08:00
Krishnan Parthasarathi d136ac0596
Don't close transition task channel on server exit (#16627) 2023-02-15 22:09:25 -08:00
Poorna c33a237067
fix: under site replication disallow remote target modification (#16628) 2023-02-15 20:22:13 -08:00
Poorna eb7d3da994
Fix site replication status reporting of quota (#16626) 2023-02-16 08:08:35 +05:30
Harshavardhana 37134e42d4
ignore io.EOF, io.ErrUnexpectedEOF on xl.meta reads in WalkDir() (#16625) 2023-02-15 07:12:48 -08:00
Harshavardhana 0c1f8b4e0f
add user-agent for all minio.Client usage (#16619) 2023-02-14 13:19:30 -08:00
Anis Elleuch 857674c3a0
heal: Do not mark buckets as done when there is no online disks (#16621) 2023-02-14 12:50:13 -08:00
Harshavardhana 15a75bd79b
ignore preconditionFailed error in batch replication (#16615) 2023-02-14 07:22:08 -08:00
Andreas Auernhammer 74887c7372
kms: add support for KES API keys and switch to KES Go SDK (#16617)
Signed-off-by: Andreas Auernhammer <hi@aead.dev>
2023-02-14 07:19:20 -08:00
Harshavardhana 31188e9327
add parallel workers in batch replication (#16609) 2023-02-13 12:07:58 -08:00
Harshavardhana ee6d96eb46
Periodically remove stale buckets from in-memory (#16597) 2023-02-13 08:09:52 -08:00
Anis Elleuch d6d01067a0
fix: allow global leader lock context merge to be canceled (#16603)
Global leader lock was first designated to only acquired once
until the node is killed. However, currently, the code acquires
it repeatedly during the lifetime of the server, now there is a
goroutine leak.
2023-02-13 01:26:38 -08:00
jiuker a15b6f21b8
remove incorrect use of WaitGroup (#16596) 2023-02-12 20:59:45 -08:00
Anis Elleuch 689179bf18
ServerInfo: return per erasure set information (#16583) 2023-02-11 18:31:56 +05:30
Klaus Post d0f4cc89a5
Re-add Veeam Listing workaround (#16593) 2023-02-10 10:48:39 -08:00
Harshavardhana d65debb6bc
fix: comply with RFC6750 UserInfo endpoint requirements (#16592) 2023-02-10 22:20:25 +05:30
Harshavardhana 72daccd468
fix: scanner in healing cycle must use actual size (#16589) 2023-02-10 06:53:03 -08:00
Harshavardhana b363400587
fix: username replacements for aws:username must use parentUser (#16591) 2023-02-10 06:52:31 -08:00
jiuker b9f5f9ba3f
fix: aclHandlers convert XML parse error to relevant client error (#16587) 2023-02-10 02:59:10 -08:00
Anis Elleuch c8ffa59d28
Periodically refresh buckets metadata from the backend disks (#16561)
fixes #16553
2023-02-09 10:29:20 -08:00
Poorna 52aeebebea
Fix site replication meta info call to be non-blocking (#16526)
Co-authored-by: Harshavardhana <harsha@minio.io>
2023-02-08 21:16:53 -08:00
Harshavardhana 71f02adfca Revert "Print golang http errors in MinIO log format (#16465)"
This reverts commit 1fd7946dce.
2023-02-09 09:27:27 +05:30
Krishnan Parthasarathi 9de26531e4
tiering: UpdateWorkers may be called before Init (#16573) 2023-02-08 19:13:34 -08:00
Anis Elleuch b1d98febfd
New disk healing goes through the healing workers (#16568) 2023-02-08 09:25:29 -08:00
jiuker 1828fb212a
fix: avoid goroutine leak after timeouts in PeerMetrics (#16569) 2023-02-08 09:11:16 -08:00
jiuker be92046dfd
Anonymous pools in health info output are double counted (#16566) 2023-02-08 06:58:50 -08:00
Krishnan Parthasarathi 990fc415f7
Ensure safety of transitionState at startup (#16563) 2023-02-07 23:11:42 -08:00
Harshavardhana d8daabae9b
make site replication healing safer (#16560) 2023-02-07 21:44:42 -08:00
Harshavardhana 84fe4fd156
fix: multiObjectDelete by passing versionId for authorization (#16562) 2023-02-08 08:01:00 +05:30
Harshavardhana 747d475e76
initialize subsystems that are not dependent on buckets first (#16559) 2023-02-07 12:46:47 -08:00
Anis Elleuch 095b518802
Show a better error msg when internal data encryption key is incorrect (#16549) 2023-02-07 05:22:54 -08:00
Harshavardhana 0319ae756a
fix: pass proper username (simple) string as expected (#16555) 2023-02-07 03:43:08 -08:00
Harshavardhana 11c7ecb5cf
support if-match/if-none-match with s3 uploads (#16551) 2023-02-06 18:58:29 -08:00
Anis Elleuch ffd57fde90
Convert 'server closed idle connection' to errDiskNotFound (#16548) 2023-02-06 10:42:08 -08:00
jiuker a451d1cb8d
fix: close resp.body checking for kubernetes version (#16547) 2023-02-06 10:41:41 -08:00
Harshavardhana 14cf8f1b22
upgrade deps for minio/pkg v1.6.1 to include groups conditions (#16538) 2023-02-06 09:27:29 -08:00
Harshavardhana 5996c8c4d5
feat: allow offline disks on a fresh start (#16541) 2023-02-06 09:26:09 -08:00
Harshavardhana 21885f9457
fix: liveness/readiness must return errors if KMS is unreachable (#16540) 2023-02-06 08:55:56 -08:00
Harshavardhana aa8b9572b9
remove double ENABLED help output (#16528) 2023-02-03 05:52:52 -08:00
Poorna 876e1a91b2
replication: Fix typo checking PreconditionFailed status code (#16517) 2023-02-02 19:22:02 +05:30
Anis Elleuch 2278fc8f47
Print original error when IAM load is failed in some places (#16511) 2023-02-01 17:32:22 +05:30
Harshavardhana a91f353621
support 'mc admin service restart' for windows (#16512) 2023-02-01 17:31:46 +05:30
Klaus Post cdb1b48ad9
Make localLocker lock attempts cancellable (#16510) 2023-01-31 09:41:17 -08:00
Kaan Kabalak 2d0f30f062
Fix typo in code comment (#16509) 2023-01-31 07:54:19 +05:30
Krishnan Parthasarathi cea2ca8c8e
Add restore-status header for multipart objects (#16508) 2023-01-31 07:53:45 +05:30
Klaus Post f713436dd0
Fix truncated list response on deleted replicated objects (#16504) 2023-01-30 09:13:53 -08:00
Klaus Post b923a62425
Check pool-index for invalid setups (#16501) 2023-01-30 18:33:07 +05:30
Harshavardhana 67fce4a5b3
fix: dangling delete() upon success should return 404 (#16494) 2023-01-27 12:43:45 -08:00
Poorna eaa65b7ade
fix replication healing on list to consider all versions (#16496) 2023-01-27 12:43:28 -08:00
Poorna 820d94447c
replication: fix target bucket passed on GET proxy (#16495) 2023-01-27 10:24:51 -08:00
Poorna ed20134a7b
replication: detect proxy header presence correctly (#16489) 2023-01-27 01:29:32 -08:00
Harshavardhana d19cbc81b5
fix: do not return IAM/Bucket metadata replication errors to client (#16486) 2023-01-26 11:11:54 -08:00
Anis Elleuch 1fd7946dce
Print golang http errors in MinIO log format (#16465) 2023-01-26 22:46:16 +05:30
Klaus Post 027ff0f3a8
fix: set modTime to current in snowball if archive shows empty (#16482) 2023-01-26 22:20:35 +05:30
Harshavardhana 54b561898f
fix: anonymize the x-amz-id-2 value from hostname (#16478) 2023-01-25 10:25:36 -08:00
Harshavardhana 65c104a589
add x-amz-id-2 to indicate the node that received the request (#16474) 2023-01-25 09:14:10 -08:00
Anis Elleuch 0a0416b6ea
Better error when setting up replication with a service account alias (#16472) 2023-01-25 21:50:12 +05:30
Anis Elleuch 441babdc41
Rename peer S3 prefix to avoid collision in the future (#16473) 2023-01-25 06:46:30 -08:00
Harshavardhana e64b9f6751
fix: disallow SSE-C encrypted objects on replicated buckets (#16467) 2023-01-24 15:46:33 -08:00
Florian Schwab d67a846ec4
allow restarting of decommissioning if completed, failed or canceld (#16464) 2023-01-24 07:07:59 -08:00