Commit Graph

589 Commits

Author SHA1 Message Date
jiuker fb5ce3b87a
record err time when remote node is offline (#17262) 2023-05-30 10:07:26 -07:00
Klaus Post 6fe028b7c5
Revert s3 select simdjson reuse (#17310) 2023-05-30 10:02:22 -07:00
Anis Eleuch 54c5c88fe6
Add number of offline disks in quorum errors (#16822) 2023-05-25 09:39:06 -07:00
jiuker 443250d135
fix: for Target isActive use net.Dial instead (#17251) 2023-05-25 09:24:11 -07:00
jiuker d749aaab69
fix: ignore existing target status when adding new targets (#17250) 2023-05-24 22:57:37 -07:00
Krishnan Parthasarathi 62df731006
Add updatedAt for GetBucketLifecycleConfig (#17271) 2023-05-24 22:52:39 -07:00
Harshavardhana d0a0eb9738
support fan-out objects via PostUpload() (#17233) 2023-05-24 22:51:07 -07:00
Klaus Post 5677f73794
Add PostObject Checksum (#17244) 2023-05-23 07:58:33 -07:00
Krishnan Parthasarathi 55a3310446
logger-http: Don't retry after a succesful send (#17266) 2023-05-22 14:53:18 -07:00
Harshavardhana fc03be7891
simplify bucket metadata lookups for versioning/object locking (#17253) 2023-05-22 12:05:14 -07:00
jiuker b1b00a5055
fix: Avoid Income globalStats twice upon error (#17263) 2023-05-22 07:42:27 -07:00
jiuker 41fa8fa2d2
fix: increment counter when entry be skipped (#17237) 2023-05-19 08:36:52 -07:00
jiuker e94e6adf91
fix: return proper error if OIDC Discoverydoc fails to respond (#17242) 2023-05-19 02:13:33 -07:00
Klaus Post b06d7bf834
fix: leaking connections in JSON SQL with limited return (#17239) 2023-05-18 11:26:46 -07:00
Aditya Manthramurthy 9d96b18df0
Add "name" and "description" params to service acc (#17172) 2023-05-17 17:05:36 -07:00
Praveen raj Mani 85912985b6
Check for only network errors in audit webhook for reachability (#17228) 2023-05-17 11:10:33 -07:00
Klaus Post aaf1abc993
simplify HardLimitReader by using LimitReader for internal usage (#17218) 2023-05-16 13:14:37 -07:00
Harshavardhana ef2fc0f99e
fix: reduce using memory and temporary files. (#17206) 2023-05-15 14:08:54 -07:00
Anis Eleuch 684399433b
lock: Retry locking with an increasing random interval (#17200) 2023-05-13 08:42:21 -07:00
Poorna e07c2ab868
Use hash.NewLimitReader for internal multipart calls (#17191) 2023-05-12 11:19:08 -07:00
Klaus Post 99c4ffa34f
fix: avoid audit log race protection deadlocks (#17168) 2023-05-09 08:11:32 -07:00
Harshavardhana a7f266c907
allow JWT parsing on large session policy based tokens (#17167) 2023-05-09 00:53:08 -07:00
Praveen raj Mani 57acacd5a7
Support persistent queue store for loggers (#17121) 2023-05-08 21:20:31 -07:00
Klaus Post 76913a9fd5
Signed trailers for signature v4 (#16484) 2023-05-05 19:53:12 -07:00
Harshavardhana 5569acd95c
disallow EC:0 if not set during server startup (#17141) 2023-05-04 14:44:30 -07:00
Alex 6e24dff26a
Added MINIO_BROWSER_LOGIN_ANIMATION env support for WebUI console (#17123)
Signed-off-by: Benjamin Perez <benjamin@bexsoft.net>
2023-05-03 15:32:50 -07:00
Harshavardhana 9571b0825e
add configurable VRF interface and user-timeout (#17108) 2023-05-03 14:12:25 -07:00
Krishnan Parthasarathi 0ec722bc54
Add tags to NewerNoncurrentVersions audit event (#17110) 2023-05-02 12:56:33 -07:00
Anis Eleuch 4640b13c66
Use expontential backoff algo for internode reconnections (#17052) 2023-05-02 12:35:52 -07:00
Praveen raj Mani 1704abaf6b
fix: store notification events immediately for persistent queues (#17112) 2023-05-02 07:53:13 -07:00
Klaus Post e8c0a50862
optimization use small blocks up to 64KB (#17107) 2023-05-01 09:47:49 -07:00
Harshavardhana 7ae69accc0
allow root user to be disabled via config settings (#17089) 2023-04-28 12:24:14 -07:00
Klaus Post 7fad0c8b41
Remove checksums from HTTP range request, add part checksums (#17105) 2023-04-28 08:26:32 -07:00
jiuker 6e27264c6b
update cleanupRoutine comment (#17102) 2023-04-28 01:11:51 -07:00
Anis Eleuch 5c83c9724f
audit: Add request path and host to audit event (#17099) 2023-04-27 22:18:24 -07:00
jiuker b28d391a22
fix: add correct worker count before startHTTPLogger() (#17091) 2023-04-27 10:51:16 -07:00
jiuker c8b92f6067
protect wg.Done from being called twice (#17075) 2023-04-27 07:55:36 -07:00
Anis Eleuch 31b5acc245
tcp: Increase user timeout to 10 minutes (#17087) 2023-04-26 17:48:31 -07:00
Anis Eleuch 0b7ca094e4
Remove Expect 100-continue in internode communications (#17061) 2023-04-26 09:33:45 -07:00
Praveen raj Mani 72802a5972
Use 'minio/pkg/sync/errgroup' and 'minio/pkg/workers' (#17069) 2023-04-25 22:57:40 -07:00
Harshavardhana 8fd07bcd51
simplify sort.Sort by using sort.Slice (#17066) 2023-04-24 13:28:18 -07:00
Harshavardhana 8a9b9832fd
add Dial timeout for Kafka broker pings (#17044) 2023-04-17 15:45:01 -07:00
jiuker e96c88e914
fix: DeleteBucketThrottle must delete ARN (#17034) 2023-04-15 02:14:26 -07:00
Klaus Post c133979b8e
Add part count to checksum (#17035) 2023-04-14 09:44:45 -07:00
Harshavardhana a5835cecbf
fix: regression in counting total requests (#17024) 2023-04-12 14:37:19 -07:00
Poorna d350654aee
config: fix duplication of replication priority key (#17014) 2023-04-11 19:22:10 -07:00
Harshavardhana 09a25ea7b7 lint: fix some lint issues on files 2023-04-06 22:42:10 -07:00
Harshavardhana 47b7469a60
add buffer pool for proxy forwarder (#16942) 2023-04-06 15:54:12 -07:00
Praveen raj Mani 51f7f9aaa3
Generalize the event store using go generics (#16910) 2023-04-04 10:52:24 -07:00
Anis Eleuch d90d0c8931
Use one http response recorder per external http call (#16938) 2023-03-31 09:37:29 -07:00
Anis Eleuch c259a8ea38
Set tcp user timeout to clean sockets with data in the buffer (#16887) 2023-03-24 08:10:58 -07:00
mstmdev 2d51e42305
Remove the redundant conditional in the validateParity function (#16866) 2023-03-23 14:06:22 -07:00
Anis Eleuch 1346561b9d
return quorum error instead of insufficient storage error (#16874) 2023-03-22 16:22:37 -07:00
Klaus Post 11d04279c8
Add lazy init of audit logger (#16842) 2023-03-21 10:50:40 -07:00
Harshavardhana 3b5dbf9046
allow bootstrapping to validate internode tokens (#16853) 2023-03-20 01:40:24 -07:00
Aditya Manthramurthy 09c733677a
Add test for fixed post policy exploit (#16855) 2023-03-20 01:06:45 -07:00
Harshavardhana 46f9049fb4
simplify error responses for KMS (#16793) 2023-03-16 11:59:42 -07:00
Aditya Manthramurthy 58266c9e2c
Add enable flag for LDAP IDP config (#16805) 2023-03-16 11:58:59 -07:00
Nitish Tiwari 50dbd2cacc
Update audit log flow to use new headers with unit (#16797) 2023-03-13 22:50:19 -07:00
Harshavardhana b984bf8d1a
allow expiration of all versions during Listing() (#16757) 2023-03-09 15:15:30 -08:00
Harshavardhana 901887e6bf
feat: add lambda transformation functions target (#16507) 2023-03-07 08:12:41 -08:00
ferhat elmas 714283fae2
cleanup ignored static analysis (#16767) 2023-03-06 08:56:10 -08:00
ferhat elmas 3423028713
cleanup Go linter settings (#16736) 2023-03-04 20:57:35 -08:00
Harshavardhana 0ff931dc76
fix: allow CORS to work by default (#16713) 2023-02-27 10:10:45 -08:00
Praveen raj Mani 4d708cebe9
Support adding service accounts with expiration (#16430)
Co-authored-by: Harshavardhana <harsha@minio.io>
2023-02-27 10:10:22 -08:00
Aditya Manthramurthy 8cde38404d
Add metrics for custom auth plugin (#16701) 2023-02-27 09:55:18 -08:00
Shubhendu 8b4eb2304b
Set logger webhook proxy on subnet proxy change (#16665)
Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>
2023-02-27 08:35:36 -08:00
Anis Elleuch 8da0f4c5bb
Better error message when TLS certs do not have proper permissions (#16703) 2023-02-24 06:34:55 -08:00
Harshavardhana b21d3f9b82
event target registration failures must be returned (#16700) 2023-02-23 21:59:14 +05:30
Harshavardhana 5c98223c89
add correct HostId instead of deploymentId for error responses (#16686) 2023-02-22 15:41:09 +05:30
Daniel Valdivia fb17f97cf3
move audit and logger message structure to minio/pkg (#16655)
Signed-off-by: Daniel Valdivia <18384552+dvaldivia@users.noreply.github.com>
2023-02-21 21:21:17 -08:00
Shubhendu 6b65ba1551
Added attribute proxy for `mc admin config set ALIAS logger_webhook` (#16657)
Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>
2023-02-21 21:19:46 -08:00
Allan Roger Reid 8bfe972bab
Set meaningful message from minio with env variable KMS_SECRET_KEY (#16584) 2023-02-22 07:13:01 +05:30
Klaus Post fd6622458b
Add detailed scanner trace output and notifications (#16668) 2023-02-21 09:33:33 -08:00
jiuker e470268c7c
fix: a possible closer leak in SelectObjectHandler (#16598) 2023-02-17 01:44:40 -08:00
Harshavardhana 6ea150fd68
fix: avoid printing certain errors under few locations (#16631) 2023-02-17 01:40:31 -08:00
Harshavardhana e1e9ddd4a4
use kes.Status() for Status() call (#16629) 2023-02-16 22:12:24 +05:30
Andreas Auernhammer 74887c7372
kms: add support for KES API keys and switch to KES Go SDK (#16617)
Signed-off-by: Andreas Auernhammer <hi@aead.dev>
2023-02-14 07:19:20 -08:00
Harshavardhana 31188e9327
add parallel workers in batch replication (#16609) 2023-02-13 12:07:58 -08:00
Harshavardhana d65debb6bc
fix: comply with RFC6750 UserInfo endpoint requirements (#16592) 2023-02-10 22:20:25 +05:30
Anis Elleuch c8ffa59d28
Periodically refresh buckets metadata from the backend disks (#16561)
fixes #16553
2023-02-09 10:29:20 -08:00
Anis Elleuch fadc46b906
Add the access key and parent user in the audit log (#16572) 2023-02-08 11:05:26 -08:00
Anis Elleuch 095b518802
Show a better error msg when internal data encryption key is incorrect (#16549) 2023-02-07 05:22:54 -08:00
Harshavardhana 14cf8f1b22
upgrade deps for minio/pkg v1.6.1 to include groups conditions (#16538) 2023-02-06 09:27:29 -08:00
Harshavardhana 5996c8c4d5
feat: allow offline disks on a fresh start (#16541) 2023-02-06 09:26:09 -08:00
Harshavardhana aa8b9572b9
remove double ENABLED help output (#16528) 2023-02-03 05:52:52 -08:00
Klaus Post 0b7989aa4b
Fix Kafka initialization crash (#16523) 2023-02-02 19:21:19 +05:30
Harshavardhana d19cbc81b5
fix: do not return IAM/Bucket metadata replication errors to client (#16486) 2023-01-26 11:11:54 -08:00
Harshavardhana 65c104a589
add x-amz-id-2 to indicate the node that received the request (#16474) 2023-01-25 09:14:10 -08:00
Harshavardhana 3683673fb0
add missing gorilla/mux migration, update credits (#16461) 2023-01-23 08:46:37 -08:00
Poorna 1b02e046c2
Fix bandwidth monitoring to be per remote target (#16360) 2023-01-19 18:52:16 +05:30
Aditya Manthramurthy 698862ec5d
Fix transports/timeouts related regressions (#16427) 2023-01-18 10:06:38 +05:30
Harshavardhana 3db658e51e
use correct xml package for custom MarshalXML() (#16421) 2023-01-17 05:08:33 +05:30
Allan Roger Reid 9815dac48f
fix: allow bind on ipv6 loopback failures (#16388) 2023-01-11 08:47:39 +05:30
Anis Elleuch 1ece3d1dfe
Add comment field to service accounts (#16380) 2023-01-10 21:57:52 +04:00
Anis Elleuch 2146ed4033
xl: Quit early when EC config is incorrect (#16390)
Co-authored-by: Anis Elleuch <anis@min.io>
2023-01-09 23:07:45 -08:00
Anis Elleuch 1c85652cff
lint: Fix in darwin environment (#16368) 2023-01-05 10:12:01 -08:00
Klaus Post 6a04067514
fix: tweak read buffer size to reduce over-reading (#16338) 2023-01-01 08:14:20 -08:00
Harshavardhana 75faef888e
disable builds for go1.18 (#16332) 2022-12-30 11:37:07 -08:00
Harshavardhana b67d97b1ba
add missing fields in audit logs for non-compressed handlers (#16328) 2022-12-30 10:20:19 -08:00
Anis Elleuch acc9c033ed
debug: Add X-Amz-Request-ID to lock/unlock calls (#16309) 2022-12-23 19:49:07 -08:00
Anis Elleuch e57e946206
Do not save credentials in config.json (#16275) 2022-12-19 12:27:06 -08:00
Harshavardhana 80fc3a8a52
use newDynamicTimeoutWithOpts() when appropriate (#16266) 2022-12-15 13:11:37 -08:00
Klaus Post 988a2e8fed
Faster startup of large distributed systems with latency (#16259) 2022-12-15 08:31:21 -08:00
Harshavardhana bad229e16e
fix: support event name s3:Restore:* (#16257) 2022-12-14 05:12:07 -08:00
Harshavardhana c73ea27ed7
do not log checksum mismatch error, client received the error (#16246) 2022-12-14 01:57:40 -08:00
Aditya Manthramurthy 9e6cc847f8
Add HTTP2 config option for policy plugin (#16225) 2022-12-13 14:28:48 -08:00
Anis Elleuch 939c0100a6
log: Do not interpret verbs in object names in console output (#16233) 2022-12-13 08:27:40 -08:00
Aditya Manthramurthy 2d60bf8c50
Refactor HTTP transports (#16222) 2022-12-12 20:31:21 -08:00
Aditya Manthramurthy a469e6768d
Add LDAP DNS SRV record lookup support (#16201) 2022-12-12 11:27:27 -08:00
Harshavardhana 2fc182d8e6
fix: iso8601TimeFormat padding issue for certain nanoseconds (#16207) 2022-12-12 10:28:30 -08:00
Aditya Manthramurthy e06127566d
Add IAM API to attach/detach policies for LDAP (#16182) 2022-12-09 13:08:33 -08:00
jiuker 8edc2faaa9
reuse sha256 in config GetSettings (#16188) 2022-12-08 03:03:24 -08:00
Javier Adriel 04ae9058ed
Populate end_session_endpoint (#16183) 2022-12-06 16:56:37 -08:00
Aditya Manthramurthy a30cfdd88f
Bump up madmin-go to v2 (#16162) 2022-12-06 13:46:50 -08:00
Anis Elleuch 932d2c3c62
Add X-Amz-Request-Id to internode calls (#16146) 2022-12-06 09:27:26 -08:00
Harshavardhana 419f351df3
avoid logging gzipped body in trace output (#16172) 2022-12-05 13:21:27 -08:00
Klaus Post a713aee3d5
Run staticcheck on CI (#16170) 2022-12-05 11:18:50 -08:00
Andreas Auernhammer d882ba2cb4
kms: add support for KES enclaves (#16139)
Signed-off-by: Andreas Auernhammer <hi@aead.dev>
2022-12-04 02:34:24 -08:00
Harshavardhana 9b35c72349
fix: a crash in KMS cert reload function (#16158) 2022-12-02 11:19:05 -08:00
Aditya Manthramurthy 87cbd41265
feat: Allow at most one claim based OpenID IDP (#16145) 2022-11-29 15:40:49 -08:00
Harshavardhana be92cf5959
change dependency from amqp -> amqp091 (RabbitMQ) official (#16142) 2022-11-28 16:05:06 -08:00
Anis Elleuch 1f1dcdce65
move HTTP recorder to an internal library (#16128) 2022-11-28 10:20:27 -08:00
Shireesh Anjal 98a67a3776
Improvements in logger and audit webhooks (#16102) 2022-11-28 08:03:26 -08:00
Andreas Auernhammer 9b1e70e4f9
kms: fix possible deadlock due to nested RLock calls. (#16136)
Signed-off-by: Andreas Auernhammer <hi@aead.dev>
2022-11-28 07:31:07 -08:00
Harshavardhana 09d4f8cd0f
avoid serializing decryptKey() every 15mins (#16135)
if the certs are the same in an environment where the 
cert files are symlinks (e.g Kubernetes), then we resort
to reloading certs every 15mins - we can avoid reload
of the kes client instance. Ensure that the price to pay 
for contending with the lock must happen when necessary.
2022-11-28 01:14:33 -08:00
Anis Elleuch 97eb7dbf5f
notify: Return detailed err msg when connecting to target fails (#16118) 2022-11-24 07:59:19 -08:00
Harshavardhana 08103870a5
update single drive setup error message (#16098) 2022-11-18 14:47:38 -08:00
Anis Elleuch 993e586855
config: return XMinioConfigNotFound code for non existing config (#16065) 2022-11-18 10:28:14 -08:00
Shireesh Anjal 5246e3be84
Send health diagnostics data as part of callhome (#16006) 2022-11-15 13:53:05 -08:00
Harshavardhana 962d1f1a71
choose default values upon incorrect storage_class value (#16058) 2022-11-12 10:18:21 -08:00
Harshavardhana 6d76db9d6c
improve server startup error when pools are incorrect (#16056) 2022-11-11 19:40:45 -08:00
Ray 66239f30ce
configuring the nats target to reconnect forever (#16050) 2022-11-11 12:42:41 -08:00
jiuker bf89f79694
save deploymentID to avoid mutating request entry in Audit (#16053) 2022-11-11 12:42:15 -08:00
Klaus Post 5b242f1d11
Add Audit target metrics (#16044) 2022-11-10 10:20:21 -08:00
Krishnan Parthasarathi 6eef9b4a23
lifecycle: simplify Eval and HasActiveRules (#16036) 2022-11-10 07:17:45 -08:00
Krishnan Parthasarathi 7ba281728f
ilm: fix x-amz-expiration header evaluation (#16029) 2022-11-09 04:20:34 -08:00
Klaus Post bbc312fce6
Add notification queue metrics (#16026) 2022-11-08 16:36:47 -08:00
Harshavardhana 1b0dfb0f58
remove printing map() checksums (#16028) 2022-11-08 13:29:24 -08:00
Aditya Manthramurthy 76d822bf1e
Add LDAP policy entities API (#15908) 2022-11-07 14:35:09 -08:00
Klaus Post ddeca9f12a
fix: filter rest errors and logs returned (#16019) 2022-11-07 10:38:08 -08:00
Harshavardhana 21251d8c22
initialize streaming events without lazy initialization (#16016) 2022-11-07 08:01:24 -08:00
Harshavardhana 944c62daf4
skip flaky tests on windows OS (#16015) 2022-11-07 00:11:21 -08:00
Harshavardhana 9547b7d0e9
add deadlineConnections on remoteTransport (#16010) 2022-11-05 11:09:21 -07:00
Harshavardhana 76c4ea7682
force all internal MinIO operations to be under UTC (#16009) 2022-11-04 16:44:38 -07:00
Harshavardhana 7be65f66b8
support HS256 series of JWT signature for OpenID connect (#15993) 2022-11-03 16:41:53 -07:00
Anis Elleuch 7721595aa9
config: Deprecated delay/max_wait/scanner and introduce speed (#15941) 2022-11-01 08:04:07 -07:00
Harshavardhana 4892a766a8
do not panic if webhook returns an error (#15970) 2022-10-30 16:45:53 -07:00
Poorna d765b89a63
improve validation for replication resync API (#15964) 2022-10-28 23:21:33 -07:00
Klaus Post 71954faa3a
mark pubsub type safe via generics (#15961) 2022-10-28 10:55:42 -07:00