Commit Graph

5449 Commits

Author SHA1 Message Date
Anis Eleuch
2448a9e047
grafana: Remove minio_s3_requests_errors_total metric (#17094) 2023-04-27 10:55:30 -07:00
Krishnan Parthasarathi
e7cac8acef
Add tags to auditLogLifecycle (#17081) 2023-04-26 17:49:00 -07:00
Harshavardhana
6105997299
remove unnecessary log when listing resume fails (#17086) 2023-04-26 14:53:25 -07:00
Harshavardhana
8c874884fc
fix: do not copy context in DiskInfo cache (#17085) 2023-04-26 12:13:54 -07:00
Aditya Manthramurthy
ebfe81e5fd
Fix put bucket policy error code (#17084) 2023-04-26 11:21:27 -07:00
Praveen raj Mani
72802a5972
Use 'minio/pkg/sync/errgroup' and 'minio/pkg/workers' (#17069) 2023-04-25 22:57:40 -07:00
Harshavardhana
b1f3935c5b
allow ListObjects() when a prefix is an object (#17074) 2023-04-25 22:41:54 -07:00
Harshavardhana
dbd53af369
fix: initialize reverse proxy forwarder with right public certs (#17080) 2023-04-25 15:50:32 -07:00
Harshavardhana
b09fe0e50e
fix: DeleteBucket for peers() must recreate bucket upon errors (#17079) 2023-04-25 14:16:35 -07:00
Krishnan Parthasarathi
fae9000304
heal: Pick maximally occuring modTime in quorum (#17071) 2023-04-25 10:13:57 -07:00
Harshavardhana
8fd07bcd51
simplify sort.Sort by using sort.Slice (#17066) 2023-04-24 13:28:18 -07:00
Anis Eleuch
6addc7a35d
server-info: Return initializing state properly (#17070) 2023-04-24 09:10:02 -07:00
Harshavardhana
477230c82e
avoid attempting to migrate old configs (#17004) 2023-04-21 13:56:08 -07:00
Harshavardhana
d1737199ed
fix: delete DNS upon success, update failure message (#17059) 2023-04-21 12:12:31 -07:00
Harshavardhana
84f31ed45d
simplify MRF, converge it to regular healing (#17026) 2023-04-19 07:47:42 -07:00
jiuker
8a81e317d6
verify maxPartID in object options helpers (#17015) 2023-04-18 22:34:30 -07:00
Anis Eleuch
224d9a752f
fix: the race in healing tracker code (#17048) 2023-04-18 14:49:56 -07:00
Anis Eleuch
0db34e4b85
Listen bucket events to send empty events with new line (#17037) 2023-04-18 08:11:30 -07:00
Klaus Post
f66625be67
Snowball: Extract headers for metadata (#17042) 2023-04-17 12:16:54 -07:00
Harshavardhana
6825bd7e75
fix: inlined objects don't need to honor long locks (#17039) 2023-04-17 12:16:37 -07:00
Klaus Post
839b9c9271
Reduce allocations in Walkdir (#17036) 2023-04-15 10:25:25 -07:00
Harshavardhana
dd9ed85e22
implement support for FTP/SFTP server (#16952) 2023-04-15 07:34:02 -07:00
jiuker
6c1410f7f5
fix: Type of rejection for FIFO quota input (#17016) 2023-04-15 01:22:18 -07:00
Krishnan Parthasarathi
f92450d8b3
commonParity should pick readable FileInfo (#17032) 2023-04-14 16:23:28 -07:00
Klaus Post
c133979b8e
Add part count to checksum (#17035) 2023-04-14 09:44:45 -07:00
Poorna
a9269cee29
heal: avoid logging version not found (#17031) 2023-04-13 19:45:52 -07:00
Klaus Post
958a480e53
fix: lambda function expiration when cred.Expiration is set (#17029) 2023-04-13 08:10:57 -07:00
Anis Eleuch
a42650c065
Add minio_bucket_usage_version_total metric to Prometheus (#17023) 2023-04-12 20:08:07 -07:00
Harshavardhana
bdad3730f7
fix: do not error out if the local bucket is missing (#17025) 2023-04-12 15:44:16 -07:00
Harshavardhana
a5835cecbf
fix: regression in counting total requests (#17024) 2023-04-12 14:37:19 -07:00
Poorna
cd6dec49c0
Add trace support for ilm activity (#16993) 2023-04-11 19:22:32 -07:00
Krishnan Parthasarathi
6877578bbc
Update minio_node_bucket_scans_finished metrics (#17006) 2023-04-11 19:21:34 -07:00
Harshavardhana
f3682b6149
allow writes to pools with inconsistent xl.meta (#17008) 2023-04-11 11:17:46 -07:00
ferhat elmas
056ca0c68e
refactor: reuse open file in storage interface (#16970) 2023-04-09 23:09:28 -07:00
Krishnan Parthasarathi
25f7a8e406
Indicate RenameData is called by healObject (#16997) 2023-04-09 10:25:37 -07:00
Anis Eleuch
1f1c267b6c
Add used inodes to disk info (#16994) 2023-04-07 07:52:14 -07:00
Anis Eleuch
fc94ea1ced
trace: Fix <unknown> func name of requests rejected by max clients (#16977) 2023-04-07 07:51:12 -07:00
jiuker
67d2cf8f30
add Err to Get getRemoteTargetClient. (#16982) 2023-04-07 07:50:46 -07:00
Harshavardhana
09a25ea7b7 lint: fix some lint issues on files 2023-04-06 22:42:10 -07:00
Klaus Post
260a63ca73
os/windows: do not return errFileNotFound in readDir's (#16988) 2023-04-06 22:28:58 -07:00
Shubhendu
d3f70ea340
Enable audit log for global handlers (#16964)
Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>
2023-04-06 21:03:39 -07:00
Anis Eleuch
91b6fe1af3
trace: Bootstrap to show the correct source line number (#16989) 2023-04-06 17:51:53 -07:00
Klaus Post
9803f68522
snowball: Restrict zstd window size (#16987) 2023-04-06 17:47:38 -07:00
Shubhendu
4c204707fd
Correct to remove null version while ILM rule application (#16971)
Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>
Co-authored-by: Harshavardhana <harsha@minio.io>
2023-04-06 14:10:01 -07:00
Harshavardhana
c06e0bfef9
set correct Host: value for replication event notification (#16984) 2023-04-06 10:20:53 -07:00
Klaus Post
8625a9dbb3
Make listing metadata permissions stricter (#16974) 2023-04-06 07:52:35 -07:00
jiuker
0320ac43cb
simplify bucketInfo return in GetBucketInfo peer call (#16983) 2023-04-06 01:30:50 -07:00
Anis Eleuch
111c7d4026
assumeRole return the correct http code for auth errors (#16967) 2023-04-05 22:19:31 -07:00
Klaus Post
62c3df0ca3
fix: directory listing on Go 1.20 windows (#16976) 2023-04-05 14:36:49 -07:00
Poorna
3158f2d12e
Add support for batch key rotation (#16844) 2023-04-04 10:56:54 -07:00
jiuker
6e359c586e
fix: close chan before return in scanner usage updates (#16960) 2023-04-04 10:51:05 -07:00
Poorna
699a24f7e5
batch: validate versioning on src/tgt buckets (#16955) 2023-04-04 10:50:11 -07:00
jiuker
5fa3665074
add isSysErrNotDir to openFileNoSync (#16958) 2023-04-04 08:00:08 -07:00
Harshavardhana
3b7781835e
add lock metrics per node (#16943) 2023-04-03 21:23:24 -07:00
Shubhendu
5fe1b46bfd
Enabled to send audit log while version deletion (#16954)
Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>
2023-04-03 11:58:04 -07:00
Poorna
407c9ddcbf
feat: add batch replicate from remote MinIO to local cluster (#16922) 2023-03-31 10:48:36 -07:00
Anis Eleuch
d90d0c8931
Use one http response recorder per external http call (#16938) 2023-03-31 09:37:29 -07:00
Harshavardhana
216a471bbb
on quorum DeleteObject() errors attempt an MRF (#16932) 2023-03-31 08:15:41 -07:00
jiuker
a7b7860e0e
fix: potential data conflicts save site-resync metadata (#16926) 2023-03-30 20:59:45 -07:00
Klaus Post
d703daa480
Add metadata extension to ListObjectVersions (#16930) 2023-03-30 12:20:42 -07:00
Poorna
dc8fdcb9c9
fix: error checking in DeleteBucket (#16929) 2023-03-30 11:54:08 -07:00
Harshavardhana
7a6c4e438e
allow more workers for ILM expiration (#16924) 2023-03-30 10:47:15 -07:00
jiuker
c468b4e2a8
fix: avoid out of slice index (#16925) 2023-03-30 08:03:48 -07:00
jiuker
b04956a676
fix: put *msgp.Reader back in pool (#16927) 2023-03-30 08:03:12 -07:00
Aditya Manthramurthy
518f6e4d39
fix: missing return after error response (#16920) 2023-03-29 16:21:13 -07:00
Allan Roger Reid
483b226cc1
fix: avoid logging when object/version not found in replication (#16919) 2023-03-29 15:02:45 -07:00
Harshavardhana
8e02660a0d
update all our deps (#16899) 2023-03-28 03:45:24 -07:00
jiuker
16feef2a2c
fix: time.Parse RFC3339Nano (#16892) 2023-03-27 11:51:54 -07:00
jiuker
66ff17e452
fix: Avoid multiple write responses (#16894) 2023-03-27 09:15:23 -07:00
Harshavardhana
4c5edacae2
ignore operation timedout errors (#16891) 2023-03-26 03:16:51 -07:00
Anis Eleuch
8b4d0255b7
Set Console global Root CAs early to trust provided certs (#16890) 2023-03-25 09:58:38 -07:00
Poorna
74040b457b
Allow setting sync mode for site replication (#16876) 2023-03-24 14:41:23 -07:00
Klaus Post
5e3bfd2148
Add user tags when listing with metadata (#16883) 2023-03-23 10:27:19 -07:00
Klaus Post
8b0ab6ead6
Revert "Make localLocker lock attempts cancellable (#16510)" (#16884) 2023-03-23 10:26:21 -07:00
Shubhendu
b1b0aadabf
Revert query parameter src from diag upload if callhome enabled (#16881) 2023-03-23 00:24:58 -07:00
Harshavardhana
ac7d9c449a
add missing expiration information from 'sts info' (#16878) 2023-03-22 16:47:02 -07:00
Anis Eleuch
1346561b9d
return quorum error instead of insufficient storage error (#16874) 2023-03-22 16:22:37 -07:00
Minio Trusted
4bc52897b2 Update yaml files to latest version RELEASE.2023-03-22T06-36-24Z 2023-03-22 21:16:15 +00:00
Krishnan Parthasarathi
6017b63a06
top-locks: Group by lock request ID (#16860) 2023-03-21 18:35:29 -07:00
Harshavardhana
0448728228
fix: add deadline conns and dnsCache for remote transports (#16865) 2023-03-21 08:49:20 -07:00
Harshavardhana
12047702f5
fix: tweak the maintenance=true to satisfy baremetal first (#16864) 2023-03-21 08:48:38 -07:00
Harshavardhana
fb1492f531
check for quorum errors for DeleteBucket() (#16859) 2023-03-20 23:38:06 -07:00
Aditya Manthramurthy
05444a0f6a
Use the official pub key to always verify binary (#16857) 2023-03-20 13:16:18 -07:00
Harshavardhana
b3c54ec81e
reject object names with '\' on windows (#16856) 2023-03-20 13:16:00 -07:00
Harshavardhana
6c11dbffd5
add crash protection from backend modifications (#16846) 2023-03-20 09:08:42 -07:00
Harshavardhana
3b5dbf9046
allow bootstrapping to validate internode tokens (#16853) 2023-03-20 01:40:24 -07:00
Aditya Manthramurthy
09c733677a
Add test for fixed post policy exploit (#16855) 2023-03-20 01:06:45 -07:00
Harshavardhana
8d6558b236
fix: convert '\' to '/' on windows (#16852) 2023-03-20 00:35:25 -07:00
Aditya Manthramurthy
67f4ba154a
fix: post policy request security bypass (#16849) 2023-03-19 21:15:20 -07:00
Poorna
440ad20c1d
Add support for batch job cancellation (#16843) 2023-03-17 23:42:43 -07:00
Krishnan Parthasarathi
31fba6f434
Save bootstrap trace events in a circular buffer (#16823) 2023-03-17 16:01:03 -07:00
Harshavardhana
280442e533
reduce 250ms to 50ms retry looking for metacache block (#16795) 2023-03-17 14:44:01 -07:00
Shubhendu
850a945a18
Added query parameter src to diag upload if callhome enabled (#16837)
Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>
2023-03-17 08:30:16 -07:00
Harshavardhana
46f9049fb4
simplify error responses for KMS (#16793) 2023-03-16 11:59:42 -07:00
Aditya Manthramurthy
58266c9e2c
Add enable flag for LDAP IDP config (#16805) 2023-03-16 11:58:59 -07:00
Poorna
d1e775313d
support decommissioning of tiered objects (#16751) 2023-03-16 07:48:05 -07:00
Harshavardhana
e700be8cd6
fix: return appropriate Location header for MakeBucket() (#16820) 2023-03-15 13:40:40 -07:00
Harshavardhana
e0f4dd6027
remove unncessary logs from WalkDir(), PutObject() (#16818) 2023-03-15 11:52:23 -07:00
Harshavardhana
de02eca467
restore rotating root credentials properly (#16812) 2023-03-15 08:07:42 -07:00
Harshavardhana
c7f7e67a10
Do not allow adding root user to IAM subsystem (#16803) 2023-03-13 12:46:17 -07:00
Klaus Post
628042e65e
tests: Protect globalLocalDrives against races (#16800) 2023-03-13 06:04:20 -07:00
Aditya Manthramurthy
6305b206e1
fix: site-repl should heal STS with virtual parent (#16792) 2023-03-10 16:21:51 -08:00
Klaus Post
d85da9236e
Add Object Version count histogram (#16739) 2023-03-10 08:53:59 -08:00
Anis Elleuch
5c087bdcad
fix: a cosmetic error reporting with a lock timeout (#16788) 2023-03-09 15:16:13 -08:00
Klaus Post
a547bf517d
Remove locks on usage cache (#16786) 2023-03-09 15:15:46 -08:00
Harshavardhana
b984bf8d1a
allow expiration of all versions during Listing() (#16757) 2023-03-09 15:15:30 -08:00
Poorna
fb6ab1cca2
fix: allow replication of 'null' delete markers (#16773) 2023-03-08 07:03:29 -08:00
Krishnan Parthasarathi
56c57e2c53
tier-stats: Avoid repeated logs (#16774) 2023-03-07 16:43:05 -08:00
Poorna
a6057c35cc
Avoid peer notification when peer is offline, tune retries (#16737) 2023-03-07 08:13:28 -08:00
Harshavardhana
901887e6bf
feat: add lambda transformation functions target (#16507) 2023-03-07 08:12:41 -08:00
Poorna
ee54643004
Avoid unnecessary replication heal attempts (#16769) 2023-03-07 07:43:38 -08:00
Harshavardhana
0a17acdb34
return error if policy changes on disabled groups (#16766) 2023-03-06 10:46:24 -08:00
Harshavardhana
72e5212842
fix: handle syscall.EROFS also for osIsPermission() (#16765) 2023-03-06 08:56:29 -08:00
ferhat elmas
714283fae2
cleanup ignored static analysis (#16767) 2023-03-06 08:56:10 -08:00
ferhat elmas
3423028713
cleanup Go linter settings (#16736) 2023-03-04 20:57:35 -08:00
jiuker
9d062b37d7
return underlying error with BackendDown{} error (#16738) 2023-03-03 23:56:53 -08:00
jiuker
f350953a19
calculate disk cache usage percent accurately (#16740) 2023-03-02 20:32:22 -08:00
Daniel Valdivia
958bba5b42
Attach creds, owner and region to madmin calls (#16658)
Signed-off-by: Daniel Valdivia <18384552+dvaldivia@users.noreply.github.com>
2023-03-02 14:35:24 -08:00
Poorna
0f2b95b497
fix a data race in IAM loading (#16742) 2023-03-02 14:32:54 -08:00
Klaus Post
d07089ceac
Fix scanner deadlock on lost global lock (#16726) 2023-02-28 21:34:45 -08:00
Harshavardhana
0ff931dc76
fix: allow CORS to work by default (#16713) 2023-02-27 10:10:45 -08:00
Praveen raj Mani
4d708cebe9
Support adding service accounts with expiration (#16430)
Co-authored-by: Harshavardhana <harsha@minio.io>
2023-02-27 10:10:22 -08:00
dorman
4d7c8e3bb8
Remove redundant log (#16710)
Co-authored-by: z30001483 <zekaifeng2@huawei.com>
2023-02-27 09:59:47 -08:00
Aditya Manthramurthy
8cde38404d
Add metrics for custom auth plugin (#16701) 2023-02-27 09:55:18 -08:00
Krishnan Parthasarathi
fe7bf6cbbc
Support tier-add if tier backend not empty (#16715) 2023-02-27 09:26:26 -08:00
Shubhendu
8b4eb2304b
Set logger webhook proxy on subnet proxy change (#16665)
Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>
2023-02-27 08:35:36 -08:00
Harshavardhana
ae029191a3
liveness returns "busy" if queued requests > available capacity (#16719) 2023-02-27 08:34:52 -08:00
Harshavardhana
bfedea9bad
fix: disk healing should honor the right pool/set index (#16712) 2023-02-27 04:55:32 -08:00
Aditya Manthramurthy
7777d3b43a
Remove globalSTSTLSConfig (#16709) 2023-02-26 23:37:00 -08:00
Aditya Manthramurthy
9ed4fc9687
Remove globalOpenIDConfig (#16708) 2023-02-25 21:01:37 -08:00
jiuker
b49b39e99d
fix: errNoSuchPolicy should use errors.Is (#16656) 2023-02-25 00:01:37 -08:00
jiuker
cd3a2de5a3
add isValidLocation to common parseLocation (#16690) 2023-02-25 08:09:20 +05:30
jiuker
6e8960ccdd
fix: delete globalProfiler should lock (#16697) 2023-02-25 08:07:44 +05:30
Aditya Manthramurthy
e05f3d5d84
Remove globalLDAPConfig (#16706) 2023-02-25 08:07:22 +05:30
Anis Elleuch
94c6cb1323
tests: Add test for S3 API error codes (#16705) 2023-02-25 08:06:29 +05:30
Klaus Post
9acf1024e4
Remove bloom filter (#16682)
Removes the bloom filter since it has so limited usability, often gets saturated anyway and adds a bunch of complexity to the scanner.

Also removes a tiny bit of CPU by each write operation.
2023-02-24 09:03:31 +05:30
Harshavardhana
a0f06eac2a
add Veeam SOS API first implementation (#16688) 2023-02-22 19:54:57 +05:30
jiuker
83fe1a2732
log: add more info about BROWSER_URL (#16678) 2023-02-22 19:54:05 +05:30
Harshavardhana
5c98223c89
add correct HostId instead of deploymentId for error responses (#16686) 2023-02-22 15:41:09 +05:30
jiuker
663a0b7783
save correct bucketInfo on it's indexes (#16685) 2023-02-22 14:08:34 +05:30
Anis Elleuch
6efe4d1df6
perf: Only remove generated data when no bucket name specified (#16610) 2023-02-21 21:21:40 -08:00
Daniel Valdivia
fb17f97cf3
move audit and logger message structure to minio/pkg (#16655)
Signed-off-by: Daniel Valdivia <18384552+dvaldivia@users.noreply.github.com>
2023-02-21 21:21:17 -08:00
Poorna
9202c6e26a
Reload bucket targets when replication config is refreshed (#16684) 2023-02-21 21:18:49 -08:00
Poorna
59a5456091
fix: STS error translation to API error (#16683) 2023-02-22 09:57:48 +05:30
Allan Roger Reid
8bfe972bab
Set meaningful message from minio with env variable KMS_SECRET_KEY (#16584) 2023-02-22 07:13:01 +05:30
Klaus Post
fd6622458b
Add detailed scanner trace output and notifications (#16668) 2023-02-21 09:33:33 -08:00
Poorna
8a08861dd9
fix: healing of replication config for endpoint changes (#16648) 2023-02-20 16:06:13 +05:30
Harshavardhana
b66d7dc708
add missing x-amz-id-2 to event notification date (#16646) 2023-02-20 15:41:47 +05:30
Harshavardhana
b94733ab31
avoid locks when unnecessary in SiteReplicationMetaInfo() (#16650) 2023-02-18 05:35:22 -08:00
Klaus Post
84bb7d05a9
fix: healing deadlocks and ordering (#16643) 2023-02-17 23:22:43 +05:30
Harshavardhana
98a84d88e2
fix: trim and ignore './' directories (#16642) 2023-02-17 07:15:03 -08:00
jiuker
e470268c7c
fix: a possible closer leak in SelectObjectHandler (#16598) 2023-02-17 01:44:40 -08:00
jiuker
3a6cd4f73d
fix: sync.pool won't Put reader back (#16638) 2023-02-17 01:42:43 -08:00
Harshavardhana
6ea150fd68
fix: avoid printing certain errors under few locations (#16631) 2023-02-17 01:40:31 -08:00
Anis Elleuch
a7188bc9d0
fix: evaluate BypassGov policy action in deletion correctly (#16635) 2023-02-17 07:53:34 +05:30
Krishnan Parthasarathi
a1dd08f2e6
Include tier name in MinIO/S3 target user-agent (#16630) 2023-02-15 22:09:46 -08:00
Krishnan Parthasarathi
d136ac0596
Don't close transition task channel on server exit (#16627) 2023-02-15 22:09:25 -08:00
Poorna
c33a237067
fix: under site replication disallow remote target modification (#16628) 2023-02-15 20:22:13 -08:00
Poorna
eb7d3da994
Fix site replication status reporting of quota (#16626) 2023-02-16 08:08:35 +05:30
Harshavardhana
37134e42d4
ignore io.EOF, io.ErrUnexpectedEOF on xl.meta reads in WalkDir() (#16625) 2023-02-15 07:12:48 -08:00
Harshavardhana
0c1f8b4e0f
add user-agent for all minio.Client usage (#16619) 2023-02-14 13:19:30 -08:00
Anis Elleuch
857674c3a0
heal: Do not mark buckets as done when there is no online disks (#16621) 2023-02-14 12:50:13 -08:00
Harshavardhana
15a75bd79b
ignore preconditionFailed error in batch replication (#16615) 2023-02-14 07:22:08 -08:00
Andreas Auernhammer
74887c7372
kms: add support for KES API keys and switch to KES Go SDK (#16617)
Signed-off-by: Andreas Auernhammer <hi@aead.dev>
2023-02-14 07:19:20 -08:00
Harshavardhana
31188e9327
add parallel workers in batch replication (#16609) 2023-02-13 12:07:58 -08:00
Harshavardhana
ee6d96eb46
Periodically remove stale buckets from in-memory (#16597) 2023-02-13 08:09:52 -08:00
Anis Elleuch
d6d01067a0
fix: allow global leader lock context merge to be canceled (#16603)
Global leader lock was first designated to only acquired once
until the node is killed. However, currently, the code acquires
it repeatedly during the lifetime of the server, now there is a
goroutine leak.
2023-02-13 01:26:38 -08:00
jiuker
a15b6f21b8
remove incorrect use of WaitGroup (#16596) 2023-02-12 20:59:45 -08:00
Anis Elleuch
689179bf18
ServerInfo: return per erasure set information (#16583) 2023-02-11 18:31:56 +05:30
Klaus Post
d0f4cc89a5
Re-add Veeam Listing workaround (#16593) 2023-02-10 10:48:39 -08:00
Harshavardhana
d65debb6bc
fix: comply with RFC6750 UserInfo endpoint requirements (#16592) 2023-02-10 22:20:25 +05:30
Harshavardhana
72daccd468
fix: scanner in healing cycle must use actual size (#16589) 2023-02-10 06:53:03 -08:00
Harshavardhana
b363400587
fix: username replacements for aws:username must use parentUser (#16591) 2023-02-10 06:52:31 -08:00
jiuker
b9f5f9ba3f
fix: aclHandlers convert XML parse error to relevant client error (#16587) 2023-02-10 02:59:10 -08:00
Anis Elleuch
c8ffa59d28
Periodically refresh buckets metadata from the backend disks (#16561)
fixes #16553
2023-02-09 10:29:20 -08:00
Poorna
52aeebebea
Fix site replication meta info call to be non-blocking (#16526)
Co-authored-by: Harshavardhana <harsha@minio.io>
2023-02-08 21:16:53 -08:00
Harshavardhana
71f02adfca Revert "Print golang http errors in MinIO log format (#16465)"
This reverts commit 1fd7946dce.
2023-02-09 09:27:27 +05:30
Krishnan Parthasarathi
9de26531e4
tiering: UpdateWorkers may be called before Init (#16573) 2023-02-08 19:13:34 -08:00
Anis Elleuch
b1d98febfd
New disk healing goes through the healing workers (#16568) 2023-02-08 09:25:29 -08:00
jiuker
1828fb212a
fix: avoid goroutine leak after timeouts in PeerMetrics (#16569) 2023-02-08 09:11:16 -08:00
jiuker
be92046dfd
Anonymous pools in health info output are double counted (#16566) 2023-02-08 06:58:50 -08:00
Krishnan Parthasarathi
990fc415f7
Ensure safety of transitionState at startup (#16563) 2023-02-07 23:11:42 -08:00
Harshavardhana
d8daabae9b
make site replication healing safer (#16560) 2023-02-07 21:44:42 -08:00
Harshavardhana
84fe4fd156
fix: multiObjectDelete by passing versionId for authorization (#16562) 2023-02-08 08:01:00 +05:30
Harshavardhana
747d475e76
initialize subsystems that are not dependent on buckets first (#16559) 2023-02-07 12:46:47 -08:00
Anis Elleuch
095b518802
Show a better error msg when internal data encryption key is incorrect (#16549) 2023-02-07 05:22:54 -08:00
Harshavardhana
0319ae756a
fix: pass proper username (simple) string as expected (#16555) 2023-02-07 03:43:08 -08:00
Harshavardhana
11c7ecb5cf
support if-match/if-none-match with s3 uploads (#16551) 2023-02-06 18:58:29 -08:00
Anis Elleuch
ffd57fde90
Convert 'server closed idle connection' to errDiskNotFound (#16548) 2023-02-06 10:42:08 -08:00
jiuker
a451d1cb8d
fix: close resp.body checking for kubernetes version (#16547) 2023-02-06 10:41:41 -08:00
Harshavardhana
14cf8f1b22
upgrade deps for minio/pkg v1.6.1 to include groups conditions (#16538) 2023-02-06 09:27:29 -08:00
Harshavardhana
5996c8c4d5
feat: allow offline disks on a fresh start (#16541) 2023-02-06 09:26:09 -08:00
Harshavardhana
21885f9457
fix: liveness/readiness must return errors if KMS is unreachable (#16540) 2023-02-06 08:55:56 -08:00
Harshavardhana
aa8b9572b9
remove double ENABLED help output (#16528) 2023-02-03 05:52:52 -08:00
Poorna
876e1a91b2
replication: Fix typo checking PreconditionFailed status code (#16517) 2023-02-02 19:22:02 +05:30
Anis Elleuch
2278fc8f47
Print original error when IAM load is failed in some places (#16511) 2023-02-01 17:32:22 +05:30
Harshavardhana
a91f353621
support 'mc admin service restart' for windows (#16512) 2023-02-01 17:31:46 +05:30
Klaus Post
cdb1b48ad9
Make localLocker lock attempts cancellable (#16510) 2023-01-31 09:41:17 -08:00
Kaan Kabalak
2d0f30f062
Fix typo in code comment (#16509) 2023-01-31 07:54:19 +05:30
Krishnan Parthasarathi
cea2ca8c8e
Add restore-status header for multipart objects (#16508) 2023-01-31 07:53:45 +05:30
Klaus Post
f713436dd0
Fix truncated list response on deleted replicated objects (#16504) 2023-01-30 09:13:53 -08:00
Klaus Post
b923a62425
Check pool-index for invalid setups (#16501) 2023-01-30 18:33:07 +05:30
Harshavardhana
67fce4a5b3
fix: dangling delete() upon success should return 404 (#16494) 2023-01-27 12:43:45 -08:00
Poorna
eaa65b7ade
fix replication healing on list to consider all versions (#16496) 2023-01-27 12:43:28 -08:00
Poorna
820d94447c
replication: fix target bucket passed on GET proxy (#16495) 2023-01-27 10:24:51 -08:00
Poorna
ed20134a7b
replication: detect proxy header presence correctly (#16489) 2023-01-27 01:29:32 -08:00
Harshavardhana
d19cbc81b5
fix: do not return IAM/Bucket metadata replication errors to client (#16486) 2023-01-26 11:11:54 -08:00
Anis Elleuch
1fd7946dce
Print golang http errors in MinIO log format (#16465) 2023-01-26 22:46:16 +05:30
Klaus Post
027ff0f3a8
fix: set modTime to current in snowball if archive shows empty (#16482) 2023-01-26 22:20:35 +05:30
Harshavardhana
54b561898f
fix: anonymize the x-amz-id-2 value from hostname (#16478) 2023-01-25 10:25:36 -08:00
Harshavardhana
65c104a589
add x-amz-id-2 to indicate the node that received the request (#16474) 2023-01-25 09:14:10 -08:00
Anis Elleuch
0a0416b6ea
Better error when setting up replication with a service account alias (#16472) 2023-01-25 21:50:12 +05:30
Anis Elleuch
441babdc41
Rename peer S3 prefix to avoid collision in the future (#16473) 2023-01-25 06:46:30 -08:00
Harshavardhana
e64b9f6751
fix: disallow SSE-C encrypted objects on replicated buckets (#16467) 2023-01-24 15:46:33 -08:00
Florian Schwab
d67a846ec4
allow restarting of decommissioning if completed, failed or canceld (#16464) 2023-01-24 07:07:59 -08:00
Poorna
ca2a1c3f60
replication: clone metrics while loading metrics cache (#16462) 2023-01-24 02:10:32 -08:00
Poorna
93fbb228bf
Validate if parent user exists for service acct (#16443) 2023-01-24 08:17:18 +05:30
Anis Elleuch
f37a5b6dae
Add CPU info in the check update user-agent (#16447) 2023-01-23 08:07:55 -08:00
Harshavardhana
31b0decd46
migrate to minio/mux from gorilla/mux (#16456) 2023-01-23 16:42:47 +05:30
Harshavardhana
eb561e1c05
allow bootstrap platform checks to be pool specific (#16455) 2023-01-23 16:24:50 +05:30
Poorna
ddad231921
replication: Avoid logging PreConditionFailed error (#16450) 2023-01-21 07:33:04 +05:30
Klaus Post
03b94f907f
fix: deleted object names for directory objects (#16448) 2023-01-20 21:16:06 +05:30
Shireesh Anjal
0f591d245d
fix: incorrect anonymization of drive endpoint (#16442) 2023-01-20 07:35:44 +05:30
Poorna
1b02e046c2
Fix bandwidth monitoring to be per remote target (#16360) 2023-01-19 18:52:16 +05:30
Harshavardhana
d08e3cc895
add a way to avoid blocking queueHealTask() depending on caller (#16433) 2023-01-19 18:50:54 +05:30
Anis Elleuch
d98116559b
Use async healing in PutObject call (#16431) 2023-01-19 00:54:22 -08:00
Krishnan Parthasarathi
71c95ad0d0
Signal stop-rebalance to all rebalancing pools (#16438) 2023-01-19 06:54:23 +05:30
Aditya Manthramurthy
698862ec5d
Fix transports/timeouts related regressions (#16427) 2023-01-18 10:06:38 +05:30
Harshavardhana
b4ef5ff294
remove unnecessary code checking for supported features (#16423) 2023-01-17 19:37:47 +05:30
Harshavardhana
3db658e51e
use correct xml package for custom MarshalXML() (#16421) 2023-01-17 05:08:33 +05:30
Shireesh Anjal
5a9f7516d6
Add monthly license update job (#16391) 2023-01-17 05:08:15 +05:30
Anis Elleuch
3039fd4519
Optimize background heal status to use LocalStorageInfo (#16414) 2023-01-17 05:02:00 +05:30
Harshavardhana
095fc0561d
feat: allow decom of multiple pools (#16416) 2023-01-16 21:36:34 +05:30
Anis Elleuch
beb1924437
Properly restart fresh disk healing when failed in some places (#16413) 2023-01-14 05:06:46 +05:30
jiuker
c8e1154f1e
fix: reading from erasureDisks must be protected via read lock() (#16407) 2023-01-13 04:16:23 -08:00
Poorna
b204c2dbec
fix: enforce deny on DeleteVersionAction (#16409) 2023-01-13 04:16:00 -08:00
Poorna
b22b39de96
Avoid dangling deletes if disk not found (#16401) 2023-01-12 22:20:19 -08:00
Harshavardhana
c242e6c391
fix: calculate common parity properly (#16406) 2023-01-13 03:28:16 +05:30
Anis Elleuch
e05205756f
metrics: Add more logs when unable to read bucket usage (#16405) 2023-01-13 02:32:00 +05:30
Anis Elleuch
475a88b555
fix: error out if an object is found after a full decom (#16277) 2023-01-12 05:52:51 +05:30
Anis Elleuch
1ece3d1dfe
Add comment field to service accounts (#16380) 2023-01-10 21:57:52 +04:00
Anis Elleuch
2146ed4033
xl: Quit early when EC config is incorrect (#16390)
Co-authored-by: Anis Elleuch <anis@min.io>
2023-01-09 23:07:45 -08:00
Anis Elleuch
ebd4388cca
s3: Return XMinioInvalidObjectName if the object contains null char (#16372) 2023-01-06 10:11:18 -08:00
Anis Elleuch
0333412148
fix: heal only once per disk per set among multiple disks (#16358) 2023-01-05 20:41:19 -08:00
Harshavardhana
e0086c1be7
reduce startup delays on kubernetes (#16356) 2023-01-05 02:32:43 -08:00
Anis Elleuch
7883e55da2
Merge buckets list from different nodes in ListBuckets() call (#16357) 2023-01-04 08:53:58 -08:00
Harshavardhana
a15a2556c3
converge listBuckets() as a peer call (#16346) 2023-01-03 23:39:40 -08:00
Harshavardhana
f1bbb7fef5
vectorize cluster-wide calls such as bucket operations (#16313) 2023-01-03 08:16:39 -08:00
Harshavardhana
1cd8e1d8b6
remove the startup jitter before locks() (#16340) 2023-01-02 01:40:09 -08:00
jiuker
62cd918061
fix: close helmInfo file descriptor (#16319) 2023-01-01 23:26:59 -08:00
Klaus Post
6a04067514
fix: tweak read buffer size to reduce over-reading (#16338) 2023-01-01 08:14:20 -08:00
Taran Pelkey
49b3908635
fix: misplaced write response command in DetachPolicy() (#16333) 2022-12-30 20:04:03 -08:00
Harshavardhana
f93183f66e
fix: a deadlock by refactoring listBuckets() under site replication (#16323) 2022-12-29 00:08:31 -08:00
Harshavardhana
2937711390
fix: DeleteObject() API with versionId under replication (#16325) 2022-12-28 22:48:33 -08:00
Anis Elleuch
27417459fb
metrics: Show healing info for all nodes (#16315) 2022-12-26 08:35:32 -08:00
Harshavardhana
5b8fe2e89a
allow locks with object affinity to spread across pools (#16312) 2022-12-23 20:55:45 -08:00
Anis Elleuch
acc9c033ed
debug: Add X-Amz-Request-ID to lock/unlock calls (#16309) 2022-12-23 19:49:07 -08:00
Poorna
8528b265a9
Validate replication target update to avoid duplicate endpoints (#16311) 2022-12-23 15:44:48 -08:00
Harshavardhana
b882310e2b
avoid locks for internal and invalid buckets in MakeBucket() (#16302) 2022-12-23 07:46:00 -08:00
Poorna
de0b43de32
persist replication stats with leader lock (#16282) 2022-12-22 14:25:13 -08:00
jiuker
29dd7f1d68
tier verification leaks fd, that must be closed (#16296)
Co-authored-by: Harshavardhana <harsha@minio.io>
2022-12-22 10:35:54 -08:00
Poorna
6423e4c767
Remove site replication config if it succeeded locally (#16279) 2022-12-22 01:31:20 -08:00
Krishnan Parthasarathi
2fa35def2c
Fix DeleteObject when only free versions remain (#16289) 2022-12-21 16:24:07 -08:00
Anis Elleuch
34167c51d5
trace: Add bootstrap tracing events (#16286) 2022-12-21 15:52:29 -08:00
Harshavardhana
a5f8af4efb
serialize replication stats() only when needed (#16280) 2022-12-20 00:07:53 -08:00
Harshavardhana
5a218f38a1
allow retries for transaction lock on startup (#16273) 2022-12-19 22:00:00 -08:00
Anis Elleuch
e57e946206
Do not save credentials in config.json (#16275) 2022-12-19 12:27:06 -08:00
Klaus Post
b4f71362e9
Avoid config migration on every startup (#16278) 2022-12-19 11:10:14 -08:00
Taran Pelkey
ed37b7a9d5
Add API to fetch policy user/group associations (#16239) 2022-12-19 10:37:03 -08:00
Anis Elleuch
89db3fdb5d
Do not return an error when version disparity is detected (#16269) 2022-12-16 08:52:12 -08:00
Harshavardhana
80fc3a8a52
use newDynamicTimeoutWithOpts() when appropriate (#16266) 2022-12-15 13:11:37 -08:00
Klaus Post
988a2e8fed
Faster startup of large distributed systems with latency (#16259) 2022-12-15 08:31:21 -08:00
Harshavardhana
2433698372
fix: remove unnecessary logs for client conn errors (#16261) 2022-12-15 08:25:05 -08:00
Harshavardhana
5d7e8f79ed
fix: remove scanner healing with unnecessary logs (#16260) 2022-12-14 16:39:18 -08:00
Harshavardhana
bad229e16e
fix: support event name s3:Restore:* (#16257) 2022-12-14 05:12:07 -08:00
Poorna
d37e514733
Cleanup remote targets automatically on replication config removal. (#16221) 2022-12-14 03:24:06 -08:00
Harshavardhana
c73ea27ed7
do not log checksum mismatch error, client received the error (#16246) 2022-12-14 01:57:40 -08:00
Krishnan Parthasarathi
0159b56717
fix: rebalance to account for object's on-disk size (#16240) 2022-12-14 00:15:14 -08:00
Aditya Manthramurthy
9e6cc847f8
Add HTTP2 config option for policy plugin (#16225) 2022-12-13 14:28:48 -08:00
Taran Pelkey
709eb283d9
Add endpoints for managing IAM policies (#15897)
Co-authored-by: Taran <taran@minio.io>
Co-authored-by: ¨taran-p¨ <¨taran@minio.io¨>
Co-authored-by: Aditya Manthramurthy <donatello@users.noreply.github.com>
2022-12-13 12:13:23 -08:00
Anis Elleuch
76dde82b41
Implement STS account info API (#16115) 2022-12-13 08:38:50 -08:00
Anis Elleuch
939c0100a6
log: Do not interpret verbs in object names in console output (#16233) 2022-12-13 08:27:40 -08:00
Aditya Manthramurthy
2d60bf8c50
Refactor HTTP transports (#16222) 2022-12-12 20:31:21 -08:00
Harshavardhana
37e20f6ef2
feat: allow listening specific addrs for API port (#16223) 2022-12-12 18:48:46 -08:00
Harshavardhana
2fc182d8e6
fix: iso8601TimeFormat padding issue for certain nanoseconds (#16207) 2022-12-12 10:28:30 -08:00
Shireesh Anjal
a2cbeaa9e6
Use different subnet public key during dev/test (#16216) 2022-12-12 10:28:15 -08:00
Harshavardhana
444ff20bc5
do not rename multipart failed transactions back to tmp (#16204) 2022-12-12 01:40:29 -08:00
Harshavardhana
20ef5e7a6a
avoid double deletes() when no more versions (#16206) 2022-12-12 01:40:04 -08:00
Aditya Manthramurthy
e06127566d
Add IAM API to attach/detach policies for LDAP (#16182) 2022-12-09 13:08:33 -08:00
Harshavardhana
dfe73629a3
fix: delete marker discrepancies via DeleteObject() API (#16195) 2022-12-08 18:15:16 -08:00
Harshavardhana
b03dd1af17
remove hard limit for number of buckets (#16194) 2022-12-08 12:24:03 -08:00
Harshavardhana
4bc367c490
fix: translate tier add errors properly (#16191) 2022-12-08 11:18:07 -08:00
Klaus Post
3eb2d086b2
Replace filepathx with fork (#16192) 2022-12-08 10:42:44 -08:00
Klaus Post
70986b6e6e
Add version id to healresult (#16193) 2022-12-08 07:49:10 -08:00
Klaus Post
ebe395788b
feat: Encrypt s3zip file index (#16179) 2022-12-07 14:56:07 -08:00
Klaus Post
12fd6678ee
Encrypt checksums with KMS on CompleteMultipartUpload (#16177) 2022-12-07 10:18:18 -08:00
Harshavardhana
90d35b70b4
remove unnecessary logs for truncated XML inputs (#16184) 2022-12-07 08:30:52 -08:00
Javier Adriel
04ae9058ed
Populate end_session_endpoint (#16183) 2022-12-06 16:56:37 -08:00
Aditya Manthramurthy
a30cfdd88f
Bump up madmin-go to v2 (#16162) 2022-12-06 13:46:50 -08:00
Anis Elleuch
1bae32dc96
xl: Delete older data-dir when replacing an existing version-id (#16176) 2022-12-06 13:43:18 -08:00
Anis Elleuch
932d2c3c62
Add X-Amz-Request-Id to internode calls (#16146) 2022-12-06 09:27:26 -08:00
jiuker
8d8d07ac5c
use readlock instead of writelock to get heal information (#16175) 2022-12-06 08:08:22 -08:00
Anis Elleuch
44735be38e
s3: Return correct error when Version is invalid in policy document (#16178) 2022-12-06 08:07:24 -08:00
Klaus Post
3fd9059b4e
opt: Only stream big data usage caches (#16168) 2022-12-05 13:01:11 -08:00
Klaus Post
a713aee3d5
Run staticcheck on CI (#16170) 2022-12-05 11:18:50 -08:00
Andreas Auernhammer
d882ba2cb4
kms: add support for KES enclaves (#16139)
Signed-off-by: Andreas Auernhammer <hi@aead.dev>
2022-12-04 02:34:24 -08:00
jiuker
6086f45d25
fix: in disk cache readCacheFileStream should closed upon return (#16138) 2022-12-04 02:28:10 -08:00
Klaus Post
98cffbce03
s3zip: Limit over-read for single file (#16161) 2022-12-02 08:53:24 -08:00
Klaus Post
1cd875de1e
Persist updated metadata (#16160) 2022-12-02 08:35:04 -08:00
Harshavardhana
5a8df7efb3
re-implement StorageInfo to be a peer call (#16155) 2022-12-01 14:31:35 -08:00
Anis Elleuch
c84e2939e4
trace: Publish storage layer errors (#16153) 2022-12-01 12:10:54 -08:00
Anis Elleuch
641ab24aec
repl: resync orchestrator to use global shared lock (#16154) 2022-12-01 12:10:09 -08:00
Harshavardhana
71133105d7
re-order the top-level config keys for priority (#16150) 2022-12-01 07:50:08 -08:00
Aditya Manthramurthy
87cbd41265
feat: Allow at most one claim based OpenID IDP (#16145) 2022-11-29 15:40:49 -08:00
Klaus Post
cc1d8f0057
Check for abandoned data when healing (#16122) 2022-11-28 10:20:55 -08:00
Anis Elleuch
1f1dcdce65
move HTTP recorder to an internal library (#16128) 2022-11-28 10:20:27 -08:00
Shireesh Anjal
98a67a3776
Improvements in logger and audit webhooks (#16102) 2022-11-28 08:03:26 -08:00
Poorna
63fc6ba2cd
preserve replicated ETag properly on target (#16129) 2022-11-26 14:43:32 -08:00
jiuker
ce53d7f6c2
add disk.Close() in healFreshDisk to indicate idiomatic flow of code (#16124) 2022-11-26 00:26:15 -08:00
jiuker
fe8eed963e
fix: wrapped error will not equal in decommissioning (#16113) 2022-11-24 08:00:42 -08:00
Shireesh Anjal
59f877fc64
fix: Timestamp not added in diagnostics report (#16114) 2022-11-23 07:11:22 -08:00
Klaus Post
f96fe9773c
fix: duplicated shared prefix with custom delimiter when listing (#16111) 2022-11-22 08:51:04 -08:00
Anis Elleuch
04948b4d55
fix: checking for stale STS account under site replication (#16109) 2022-11-22 07:26:33 -08:00
Klaus Post
98ba622679
Reduce temporary file clean-up waits (#16110) 2022-11-22 07:23:36 -08:00
Harshavardhana
08103870a5
update single drive setup error message (#16098) 2022-11-18 14:47:38 -08:00
Anis Elleuch
993e586855
config: return XMinioConfigNotFound code for non existing config (#16065) 2022-11-18 10:28:14 -08:00
Harshavardhana
58ec835af0
fix: skip free version ID and marker in metadata equality (#16093) 2022-11-18 05:48:22 -08:00
Harshavardhana
6aea950d74
avoid partID lock validating uploadID exists prematurely (#16086) 2022-11-18 03:09:35 -08:00
Poorna
7198be5be9
bucket resync: persist reset id to bucket metadata (#16088) 2022-11-18 01:39:05 -08:00
Klaus Post
a22b4adf4c
distribute replication ops based on names (#16083) 2022-11-17 15:20:09 -08:00
Klaus Post
b7bb122be8
fix: replication auto-scaling deadlock (#16084) 2022-11-17 07:35:02 -08:00
Krishnan Parthasarathi
8441a3bf5f
fix: update metacache entry only once (#16072) 2022-11-16 11:25:00 -08:00
Harshavardhana
853c4de75a
allow changing endpoints in distributed setups (#16071) 2022-11-16 07:59:10 -08:00
jiuker
3597af789e
allow resultCh to be closed() after clusterMetaHealthInfo() (#16073) 2022-11-16 03:04:36 -08:00
Shireesh Anjal
5246e3be84
Send health diagnostics data as part of callhome (#16006) 2022-11-15 13:53:05 -08:00
Klaus Post
8a07000e58
fix: refactor getReplicationDiff for safe use (#16051) 2022-11-15 07:59:21 -08:00
Krishnan Parthasarathi
3bb82ef60d
top-locks: Include lock-held duration (#16061) 2022-11-15 07:57:52 -08:00
Harshavardhana
91f45c4aa6
avoid inconsistent versions healing when versions are large (#16066) 2022-11-14 18:35:26 -08:00
Poorna
d6bc141bd1
feat: Add support for site level resync (#15753) 2022-11-14 07:16:40 -08:00
jiuker
7ac64ad24a
fix: use errors.Is for wrapped returns (#16062) 2022-11-14 07:15:46 -08:00
Harshavardhana
6d76db9d6c
improve server startup error when pools are incorrect (#16056) 2022-11-11 19:40:45 -08:00
jiuker
bdcb485740
netPerfRX Reset() should use write Lock() (#16043) 2022-11-10 19:44:20 -08:00
Poorna
e32b948a49
fix: parsing multipart uploadID under site replicated setup (#16048)
continue the fix from #16034
2022-11-10 16:17:45 -08:00
Klaus Post
5b242f1d11
Add Audit target metrics (#16044) 2022-11-10 10:20:21 -08:00
Poorna
34d28dd79f
replication: Avoid blocking on mrf save (#16045) 2022-11-10 10:20:02 -08:00
Krishnan Parthasarathi
6eef9b4a23
lifecycle: simplify Eval and HasActiveRules (#16036) 2022-11-10 07:17:45 -08:00
Aditya Manthramurthy
5f1999cc71
fix: avoid URL unsafe chars in multipart upload ID (#16034) 2022-11-09 16:41:16 -08:00
Krishnan Parthasarathi
40a2c6b882
Return remote tier as StorageClass for transitioned objects (#16035) 2022-11-09 15:57:34 -08:00
jiuker
7b7356f04c
close the reader under disk cache bitrot verification (#16024) 2022-11-09 04:20:11 -08:00
Klaus Post
bbc312fce6
Add notification queue metrics (#16026) 2022-11-08 16:36:47 -08:00