Commit Graph

5250 Commits

Author SHA1 Message Date
Poorna 98c26df53e
fix: allow past retention headers to be copied in batch replication (#17095) 2023-04-27 13:43:18 -07:00
Anis Eleuch 2448a9e047
grafana: Remove minio_s3_requests_errors_total metric (#17094) 2023-04-27 10:55:30 -07:00
Krishnan Parthasarathi e7cac8acef
Add tags to auditLogLifecycle (#17081) 2023-04-26 17:49:00 -07:00
Harshavardhana 6105997299
remove unnecessary log when listing resume fails (#17086) 2023-04-26 14:53:25 -07:00
Harshavardhana 8c874884fc
fix: do not copy context in DiskInfo cache (#17085) 2023-04-26 12:13:54 -07:00
Aditya Manthramurthy ebfe81e5fd
Fix put bucket policy error code (#17084) 2023-04-26 11:21:27 -07:00
Praveen raj Mani 72802a5972
Use 'minio/pkg/sync/errgroup' and 'minio/pkg/workers' (#17069) 2023-04-25 22:57:40 -07:00
Harshavardhana b1f3935c5b
allow ListObjects() when a prefix is an object (#17074) 2023-04-25 22:41:54 -07:00
Harshavardhana dbd53af369
fix: initialize reverse proxy forwarder with right public certs (#17080) 2023-04-25 15:50:32 -07:00
Harshavardhana b09fe0e50e
fix: DeleteBucket for peers() must recreate bucket upon errors (#17079) 2023-04-25 14:16:35 -07:00
Krishnan Parthasarathi fae9000304
heal: Pick maximally occuring modTime in quorum (#17071) 2023-04-25 10:13:57 -07:00
Harshavardhana 8fd07bcd51
simplify sort.Sort by using sort.Slice (#17066) 2023-04-24 13:28:18 -07:00
Anis Eleuch 6addc7a35d
server-info: Return initializing state properly (#17070) 2023-04-24 09:10:02 -07:00
Harshavardhana 477230c82e
avoid attempting to migrate old configs (#17004) 2023-04-21 13:56:08 -07:00
Harshavardhana d1737199ed
fix: delete DNS upon success, update failure message (#17059) 2023-04-21 12:12:31 -07:00
Harshavardhana 84f31ed45d
simplify MRF, converge it to regular healing (#17026) 2023-04-19 07:47:42 -07:00
jiuker 8a81e317d6
verify maxPartID in object options helpers (#17015) 2023-04-18 22:34:30 -07:00
Anis Eleuch 224d9a752f
fix: the race in healing tracker code (#17048) 2023-04-18 14:49:56 -07:00
Anis Eleuch 0db34e4b85
Listen bucket events to send empty events with new line (#17037) 2023-04-18 08:11:30 -07:00
Klaus Post f66625be67
Snowball: Extract headers for metadata (#17042) 2023-04-17 12:16:54 -07:00
Harshavardhana 6825bd7e75
fix: inlined objects don't need to honor long locks (#17039) 2023-04-17 12:16:37 -07:00
Klaus Post 839b9c9271
Reduce allocations in Walkdir (#17036) 2023-04-15 10:25:25 -07:00
Harshavardhana dd9ed85e22
implement support for FTP/SFTP server (#16952) 2023-04-15 07:34:02 -07:00
jiuker 6c1410f7f5
fix: Type of rejection for FIFO quota input (#17016) 2023-04-15 01:22:18 -07:00
Krishnan Parthasarathi f92450d8b3
commonParity should pick readable FileInfo (#17032) 2023-04-14 16:23:28 -07:00
Klaus Post c133979b8e
Add part count to checksum (#17035) 2023-04-14 09:44:45 -07:00
Poorna a9269cee29
heal: avoid logging version not found (#17031) 2023-04-13 19:45:52 -07:00
Klaus Post 958a480e53
fix: lambda function expiration when cred.Expiration is set (#17029) 2023-04-13 08:10:57 -07:00
Anis Eleuch a42650c065
Add minio_bucket_usage_version_total metric to Prometheus (#17023) 2023-04-12 20:08:07 -07:00
Harshavardhana bdad3730f7
fix: do not error out if the local bucket is missing (#17025) 2023-04-12 15:44:16 -07:00
Harshavardhana a5835cecbf
fix: regression in counting total requests (#17024) 2023-04-12 14:37:19 -07:00
Poorna cd6dec49c0
Add trace support for ilm activity (#16993) 2023-04-11 19:22:32 -07:00
Krishnan Parthasarathi 6877578bbc
Update minio_node_bucket_scans_finished metrics (#17006) 2023-04-11 19:21:34 -07:00
Harshavardhana f3682b6149
allow writes to pools with inconsistent xl.meta (#17008) 2023-04-11 11:17:46 -07:00
ferhat elmas 056ca0c68e
refactor: reuse open file in storage interface (#16970) 2023-04-09 23:09:28 -07:00
Krishnan Parthasarathi 25f7a8e406
Indicate RenameData is called by healObject (#16997) 2023-04-09 10:25:37 -07:00
Anis Eleuch 1f1c267b6c
Add used inodes to disk info (#16994) 2023-04-07 07:52:14 -07:00
Anis Eleuch fc94ea1ced
trace: Fix <unknown> func name of requests rejected by max clients (#16977) 2023-04-07 07:51:12 -07:00
jiuker 67d2cf8f30
add Err to Get getRemoteTargetClient. (#16982) 2023-04-07 07:50:46 -07:00
Harshavardhana 09a25ea7b7 lint: fix some lint issues on files 2023-04-06 22:42:10 -07:00
Klaus Post 260a63ca73
os/windows: do not return errFileNotFound in readDir's (#16988) 2023-04-06 22:28:58 -07:00
Shubhendu d3f70ea340
Enable audit log for global handlers (#16964)
Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>
2023-04-06 21:03:39 -07:00
Anis Eleuch 91b6fe1af3
trace: Bootstrap to show the correct source line number (#16989) 2023-04-06 17:51:53 -07:00
Klaus Post 9803f68522
snowball: Restrict zstd window size (#16987) 2023-04-06 17:47:38 -07:00
Shubhendu 4c204707fd
Correct to remove `null` version while ILM rule application (#16971)
Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>
Co-authored-by: Harshavardhana <harsha@minio.io>
2023-04-06 14:10:01 -07:00
Harshavardhana c06e0bfef9
set correct `Host:` value for replication event notification (#16984) 2023-04-06 10:20:53 -07:00
Klaus Post 8625a9dbb3
Make listing metadata permissions stricter (#16974) 2023-04-06 07:52:35 -07:00
jiuker 0320ac43cb
simplify bucketInfo return in GetBucketInfo peer call (#16983) 2023-04-06 01:30:50 -07:00
Anis Eleuch 111c7d4026
assumeRole return the correct http code for auth errors (#16967) 2023-04-05 22:19:31 -07:00
Klaus Post 62c3df0ca3
fix: directory listing on Go 1.20 windows (#16976) 2023-04-05 14:36:49 -07:00
Poorna 3158f2d12e
Add support for batch key rotation (#16844) 2023-04-04 10:56:54 -07:00
jiuker 6e359c586e
fix: close chan before return in scanner usage updates (#16960) 2023-04-04 10:51:05 -07:00
Poorna 699a24f7e5
batch: validate versioning on src/tgt buckets (#16955) 2023-04-04 10:50:11 -07:00
jiuker 5fa3665074
add isSysErrNotDir to openFileNoSync (#16958) 2023-04-04 08:00:08 -07:00
Harshavardhana 3b7781835e
add lock metrics per node (#16943) 2023-04-03 21:23:24 -07:00
Shubhendu 5fe1b46bfd
Enabled to send audit log while version deletion (#16954)
Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>
2023-04-03 11:58:04 -07:00
Poorna 407c9ddcbf
feat: add batch replicate from remote MinIO to local cluster (#16922) 2023-03-31 10:48:36 -07:00
Anis Eleuch d90d0c8931
Use one http response recorder per external http call (#16938) 2023-03-31 09:37:29 -07:00
Harshavardhana 216a471bbb
on quorum DeleteObject() errors attempt an MRF (#16932) 2023-03-31 08:15:41 -07:00
jiuker a7b7860e0e
fix: potential data conflicts save site-resync metadata (#16926) 2023-03-30 20:59:45 -07:00
Klaus Post d703daa480
Add metadata extension to ListObjectVersions (#16930) 2023-03-30 12:20:42 -07:00
Poorna dc8fdcb9c9
fix: error checking in DeleteBucket (#16929) 2023-03-30 11:54:08 -07:00
Harshavardhana 7a6c4e438e
allow more workers for ILM expiration (#16924) 2023-03-30 10:47:15 -07:00
jiuker c468b4e2a8
fix: avoid out of slice index (#16925) 2023-03-30 08:03:48 -07:00
jiuker b04956a676
fix: put *msgp.Reader back in pool (#16927) 2023-03-30 08:03:12 -07:00
Aditya Manthramurthy 518f6e4d39
fix: missing return after error response (#16920) 2023-03-29 16:21:13 -07:00
Allan Roger Reid 483b226cc1
fix: avoid logging when object/version not found in replication (#16919) 2023-03-29 15:02:45 -07:00
Harshavardhana 8e02660a0d
update all our deps (#16899) 2023-03-28 03:45:24 -07:00
jiuker 16feef2a2c
fix: time.Parse RFC3339Nano (#16892) 2023-03-27 11:51:54 -07:00
jiuker 66ff17e452
fix: Avoid multiple write responses (#16894) 2023-03-27 09:15:23 -07:00
Harshavardhana 4c5edacae2
ignore operation timedout errors (#16891) 2023-03-26 03:16:51 -07:00
Anis Eleuch 8b4d0255b7
Set Console global Root CAs early to trust provided certs (#16890) 2023-03-25 09:58:38 -07:00
Poorna 74040b457b
Allow setting sync mode for site replication (#16876) 2023-03-24 14:41:23 -07:00
Klaus Post 5e3bfd2148
Add user tags when listing with metadata (#16883) 2023-03-23 10:27:19 -07:00
Klaus Post 8b0ab6ead6
Revert "Make localLocker lock attempts cancellable (#16510)" (#16884) 2023-03-23 10:26:21 -07:00
Shubhendu b1b0aadabf
Revert query parameter `src` from diag upload if callhome enabled (#16881) 2023-03-23 00:24:58 -07:00
Harshavardhana ac7d9c449a
add missing expiration information from 'sts info' (#16878) 2023-03-22 16:47:02 -07:00
Anis Eleuch 1346561b9d
return quorum error instead of insufficient storage error (#16874) 2023-03-22 16:22:37 -07:00
Minio Trusted 4bc52897b2 Update yaml files to latest version RELEASE.2023-03-22T06-36-24Z 2023-03-22 21:16:15 +00:00
Krishnan Parthasarathi 6017b63a06
top-locks: Group by lock request ID (#16860) 2023-03-21 18:35:29 -07:00
Harshavardhana 0448728228
fix: add deadline conns and dnsCache for remote transports (#16865) 2023-03-21 08:49:20 -07:00
Harshavardhana 12047702f5
fix: tweak the maintenance=true to satisfy baremetal first (#16864) 2023-03-21 08:48:38 -07:00
Harshavardhana fb1492f531
check for quorum errors for DeleteBucket() (#16859) 2023-03-20 23:38:06 -07:00
Aditya Manthramurthy 05444a0f6a
Use the official pub key to always verify binary (#16857) 2023-03-20 13:16:18 -07:00
Harshavardhana b3c54ec81e
reject object names with '\' on windows (#16856) 2023-03-20 13:16:00 -07:00
Harshavardhana 6c11dbffd5
add crash protection from backend modifications (#16846) 2023-03-20 09:08:42 -07:00
Harshavardhana 3b5dbf9046
allow bootstrapping to validate internode tokens (#16853) 2023-03-20 01:40:24 -07:00
Aditya Manthramurthy 09c733677a
Add test for fixed post policy exploit (#16855) 2023-03-20 01:06:45 -07:00
Harshavardhana 8d6558b236
fix: convert '\' to '/' on windows (#16852) 2023-03-20 00:35:25 -07:00
Aditya Manthramurthy 67f4ba154a
fix: post policy request security bypass (#16849) 2023-03-19 21:15:20 -07:00
Poorna 440ad20c1d
Add support for batch job cancellation (#16843) 2023-03-17 23:42:43 -07:00
Krishnan Parthasarathi 31fba6f434
Save bootstrap trace events in a circular buffer (#16823) 2023-03-17 16:01:03 -07:00
Harshavardhana 280442e533
reduce 250ms to 50ms retry looking for metacache block (#16795) 2023-03-17 14:44:01 -07:00
Shubhendu 850a945a18
Added query parameter `src` to diag upload if callhome enabled (#16837)
Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>
2023-03-17 08:30:16 -07:00
Harshavardhana 46f9049fb4
simplify error responses for KMS (#16793) 2023-03-16 11:59:42 -07:00
Aditya Manthramurthy 58266c9e2c
Add enable flag for LDAP IDP config (#16805) 2023-03-16 11:58:59 -07:00
Poorna d1e775313d
support decommissioning of tiered objects (#16751) 2023-03-16 07:48:05 -07:00
Harshavardhana e700be8cd6
fix: return appropriate Location header for MakeBucket() (#16820) 2023-03-15 13:40:40 -07:00
Harshavardhana e0f4dd6027
remove unncessary logs from WalkDir(), PutObject() (#16818) 2023-03-15 11:52:23 -07:00
Harshavardhana de02eca467
restore rotating root credentials properly (#16812) 2023-03-15 08:07:42 -07:00
Harshavardhana c7f7e67a10
Do not allow adding root user to IAM subsystem (#16803) 2023-03-13 12:46:17 -07:00
Klaus Post 628042e65e
tests: Protect globalLocalDrives against races (#16800) 2023-03-13 06:04:20 -07:00
Aditya Manthramurthy 6305b206e1
fix: site-repl should heal STS with virtual parent (#16792) 2023-03-10 16:21:51 -08:00
Klaus Post d85da9236e
Add Object Version count histogram (#16739) 2023-03-10 08:53:59 -08:00
Anis Elleuch 5c087bdcad
fix: a cosmetic error reporting with a lock timeout (#16788) 2023-03-09 15:16:13 -08:00
Klaus Post a547bf517d
Remove locks on usage cache (#16786) 2023-03-09 15:15:46 -08:00
Harshavardhana b984bf8d1a
allow expiration of all versions during Listing() (#16757) 2023-03-09 15:15:30 -08:00
Poorna fb6ab1cca2
fix: allow replication of 'null' delete markers (#16773) 2023-03-08 07:03:29 -08:00
Krishnan Parthasarathi 56c57e2c53
tier-stats: Avoid repeated logs (#16774) 2023-03-07 16:43:05 -08:00
Poorna a6057c35cc
Avoid peer notification when peer is offline, tune retries (#16737) 2023-03-07 08:13:28 -08:00
Harshavardhana 901887e6bf
feat: add lambda transformation functions target (#16507) 2023-03-07 08:12:41 -08:00
Poorna ee54643004
Avoid unnecessary replication heal attempts (#16769) 2023-03-07 07:43:38 -08:00
Harshavardhana 0a17acdb34
return error if policy changes on disabled groups (#16766) 2023-03-06 10:46:24 -08:00
Harshavardhana 72e5212842
fix: handle syscall.EROFS also for osIsPermission() (#16765) 2023-03-06 08:56:29 -08:00
ferhat elmas 714283fae2
cleanup ignored static analysis (#16767) 2023-03-06 08:56:10 -08:00
ferhat elmas 3423028713
cleanup Go linter settings (#16736) 2023-03-04 20:57:35 -08:00
jiuker 9d062b37d7
return underlying error with BackendDown{} error (#16738) 2023-03-03 23:56:53 -08:00
jiuker f350953a19
calculate disk cache usage percent accurately (#16740) 2023-03-02 20:32:22 -08:00
Daniel Valdivia 958bba5b42
Attach creds, owner and region to madmin calls (#16658)
Signed-off-by: Daniel Valdivia <18384552+dvaldivia@users.noreply.github.com>
2023-03-02 14:35:24 -08:00
Poorna 0f2b95b497
fix a data race in IAM loading (#16742) 2023-03-02 14:32:54 -08:00
Klaus Post d07089ceac
Fix scanner deadlock on lost global lock (#16726) 2023-02-28 21:34:45 -08:00
Harshavardhana 0ff931dc76
fix: allow CORS to work by default (#16713) 2023-02-27 10:10:45 -08:00
Praveen raj Mani 4d708cebe9
Support adding service accounts with expiration (#16430)
Co-authored-by: Harshavardhana <harsha@minio.io>
2023-02-27 10:10:22 -08:00
dorman 4d7c8e3bb8
Remove redundant log (#16710)
Co-authored-by: z30001483 <zekaifeng2@huawei.com>
2023-02-27 09:59:47 -08:00
Aditya Manthramurthy 8cde38404d
Add metrics for custom auth plugin (#16701) 2023-02-27 09:55:18 -08:00
Krishnan Parthasarathi fe7bf6cbbc
Support tier-add if tier backend not empty (#16715) 2023-02-27 09:26:26 -08:00
Shubhendu 8b4eb2304b
Set logger webhook proxy on subnet proxy change (#16665)
Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>
2023-02-27 08:35:36 -08:00
Harshavardhana ae029191a3
liveness returns "busy" if queued requests > available capacity (#16719) 2023-02-27 08:34:52 -08:00
Harshavardhana bfedea9bad
fix: disk healing should honor the right pool/set index (#16712) 2023-02-27 04:55:32 -08:00
Aditya Manthramurthy 7777d3b43a
Remove globalSTSTLSConfig (#16709) 2023-02-26 23:37:00 -08:00
Aditya Manthramurthy 9ed4fc9687
Remove globalOpenIDConfig (#16708) 2023-02-25 21:01:37 -08:00
jiuker b49b39e99d
fix: errNoSuchPolicy should use errors.Is (#16656) 2023-02-25 00:01:37 -08:00
jiuker cd3a2de5a3
add isValidLocation to common parseLocation (#16690) 2023-02-25 08:09:20 +05:30
jiuker 6e8960ccdd
fix: delete globalProfiler should lock (#16697) 2023-02-25 08:07:44 +05:30
Aditya Manthramurthy e05f3d5d84
Remove globalLDAPConfig (#16706) 2023-02-25 08:07:22 +05:30
Anis Elleuch 94c6cb1323
tests: Add test for S3 API error codes (#16705) 2023-02-25 08:06:29 +05:30
Klaus Post 9acf1024e4
Remove bloom filter (#16682)
Removes the bloom filter since it has so limited usability, often gets saturated anyway and adds a bunch of complexity to the scanner.

Also removes a tiny bit of CPU by each write operation.
2023-02-24 09:03:31 +05:30
Harshavardhana a0f06eac2a
add Veeam SOS API first implementation (#16688) 2023-02-22 19:54:57 +05:30
jiuker 83fe1a2732
log: add more info about BROWSER_URL (#16678) 2023-02-22 19:54:05 +05:30
Harshavardhana 5c98223c89
add correct HostId instead of deploymentId for error responses (#16686) 2023-02-22 15:41:09 +05:30
jiuker 663a0b7783
save correct bucketInfo on it's indexes (#16685) 2023-02-22 14:08:34 +05:30
Anis Elleuch 6efe4d1df6
perf: Only remove generated data when no bucket name specified (#16610) 2023-02-21 21:21:40 -08:00
Daniel Valdivia fb17f97cf3
move audit and logger message structure to minio/pkg (#16655)
Signed-off-by: Daniel Valdivia <18384552+dvaldivia@users.noreply.github.com>
2023-02-21 21:21:17 -08:00
Poorna 9202c6e26a
Reload bucket targets when replication config is refreshed (#16684) 2023-02-21 21:18:49 -08:00
Poorna 59a5456091
fix: STS error translation to API error (#16683) 2023-02-22 09:57:48 +05:30
Allan Roger Reid 8bfe972bab
Set meaningful message from minio with env variable KMS_SECRET_KEY (#16584) 2023-02-22 07:13:01 +05:30
Klaus Post fd6622458b
Add detailed scanner trace output and notifications (#16668) 2023-02-21 09:33:33 -08:00
Poorna 8a08861dd9
fix: healing of replication config for endpoint changes (#16648) 2023-02-20 16:06:13 +05:30
Harshavardhana b66d7dc708
add missing x-amz-id-2 to event notification date (#16646) 2023-02-20 15:41:47 +05:30
Harshavardhana b94733ab31
avoid locks when unnecessary in SiteReplicationMetaInfo() (#16650) 2023-02-18 05:35:22 -08:00