Harshavardhana
07b1281046
add queue_dir to help message for logger/audit targets
2023-08-29 16:07:35 -07:00
Harshavardhana
adb8be069e
tune-kafka targets to ensure timeout triggers on hung brokers ( #17898 )
...
hung brokers can cause slowness to the entire system
when many callers are hung, leading to large goroutine
build-up.
2023-08-22 20:26:35 -07:00
Harshavardhana
3a0125fa1f
remove unexpected logging from peer calls ( #17888 )
...
also make sure RequestID is set for system logs
2023-08-21 14:25:24 -07:00
Harshavardhana
11dfc817f3
do not log client canceled events ( #17838 )
2023-08-17 14:53:43 -07:00
Praveen raj Mani
0285df5a02
fix: prioritize audit_webhook and logger_webhook ENVs over the config KVS ( #17783 )
2023-08-03 02:47:07 -07:00
Anis Eleuch
9c0e8cd15b
logger: Avoid slow calls in http logger Send() function ( #17747 )
...
Send() is synchronous and can affect the latency of S3 requests when the
logger buffer is full.
Avoid checking if the HTTP target is online or not and increase the
workers anyway since the buffer is already full.
Also, avoid logs flooding when the audit target is down.
2023-07-29 12:49:18 -07:00
Aditya Manthramurthy
f3248a4b37
Redact all secrets from config viewing APIs ( #17380 )
...
This change adds a `Secret` property to `HelpKV` to identify secrets
like passwords and auth tokens that should not be revealed by the server
in its configuration fetching APIs. Configuration reporting APIs now do
not return secrets.
2023-06-23 07:45:27 -07:00
Aditya Manthramurthy
5a1612fe32
Bump up madmin-go and pkg deps ( #17469 )
2023-06-19 17:53:08 -07:00
Harshavardhana
dbd4c2425e
fix: kafka broker pings must not be greater than 1sec ( #17376 )
2023-06-07 11:47:00 -07:00
Krishnan Parthasarathi
55a3310446
logger-http: Don't retry after a succesful send ( #17266 )
2023-05-22 14:53:18 -07:00
jiuker
41fa8fa2d2
fix: increment counter when entry be skipped ( #17237 )
2023-05-19 08:36:52 -07:00
Praveen raj Mani
85912985b6
Check for only network errors in audit webhook for reachability ( #17228 )
2023-05-17 11:10:33 -07:00
Klaus Post
99c4ffa34f
fix: avoid audit log race protection deadlocks ( #17168 )
2023-05-09 08:11:32 -07:00
Praveen raj Mani
57acacd5a7
Support persistent queue store for loggers ( #17121 )
2023-05-08 21:20:31 -07:00
jiuker
6e27264c6b
update cleanupRoutine comment ( #17102 )
2023-04-28 01:11:51 -07:00
Anis Eleuch
5c83c9724f
audit: Add request path and host to audit event ( #17099 )
2023-04-27 22:18:24 -07:00
jiuker
b28d391a22
fix: add correct worker count before startHTTPLogger() ( #17091 )
2023-04-27 10:51:16 -07:00
Harshavardhana
8a9b9832fd
add Dial timeout for Kafka broker pings ( #17044 )
2023-04-17 15:45:01 -07:00
Harshavardhana
a5835cecbf
fix: regression in counting total requests ( #17024 )
2023-04-12 14:37:19 -07:00
Anis Eleuch
d90d0c8931
Use one http response recorder per external http call ( #16938 )
2023-03-31 09:37:29 -07:00
Klaus Post
11d04279c8
Add lazy init of audit logger ( #16842 )
2023-03-21 10:50:40 -07:00
Harshavardhana
3b5dbf9046
allow bootstrapping to validate internode tokens ( #16853 )
2023-03-20 01:40:24 -07:00
Harshavardhana
46f9049fb4
simplify error responses for KMS ( #16793 )
2023-03-16 11:59:42 -07:00
Nitish Tiwari
50dbd2cacc
Update audit log flow to use new headers with unit ( #16797 )
2023-03-13 22:50:19 -07:00
ferhat elmas
714283fae2
cleanup ignored static analysis ( #16767 )
2023-03-06 08:56:10 -08:00
Daniel Valdivia
fb17f97cf3
move audit and logger message structure to minio/pkg ( #16655 )
...
Signed-off-by: Daniel Valdivia <18384552+dvaldivia@users.noreply.github.com>
2023-02-21 21:21:17 -08:00
Shubhendu
6b65ba1551
Added attribute proxy for mc admin config set ALIAS logger_webhook ( #16657 )
...
Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>
2023-02-21 21:19:46 -08:00
Anis Elleuch
fadc46b906
Add the access key and parent user in the audit log ( #16572 )
2023-02-08 11:05:26 -08:00
Harshavardhana
aa8b9572b9
remove double ENABLED help output ( #16528 )
2023-02-03 05:52:52 -08:00
Harshavardhana
b67d97b1ba
add missing fields in audit logs for non-compressed handlers ( #16328 )
2022-12-30 10:20:19 -08:00
Anis Elleuch
939c0100a6
log: Do not interpret verbs in object names in console output ( #16233 )
2022-12-13 08:27:40 -08:00
Aditya Manthramurthy
a30cfdd88f
Bump up madmin-go to v2 ( #16162 )
2022-12-06 13:46:50 -08:00
Anis Elleuch
1f1dcdce65
move HTTP recorder to an internal library ( #16128 )
2022-11-28 10:20:27 -08:00
Shireesh Anjal
98a67a3776
Improvements in logger and audit webhooks ( #16102 )
2022-11-28 08:03:26 -08:00
jiuker
bf89f79694
save deploymentID to avoid mutating request entry in Audit ( #16053 )
2022-11-11 12:42:15 -08:00
Klaus Post
5b242f1d11
Add Audit target metrics ( #16044 )
2022-11-10 10:20:21 -08:00
Klaus Post
ddeca9f12a
fix: filter rest errors and logs returned ( #16019 )
2022-11-07 10:38:08 -08:00
Harshavardhana
23b329b9df
remove gateway completely ( #15929 )
2022-10-24 17:44:15 -07:00
Harshavardhana
f696a221af
allow tagging policy condition for GetObject ( #15777 )
2022-10-02 12:29:29 -07:00
Anis Elleuch
f69a98ce49
fix: loading Audit kafka configuration loading ( #15766 )
2022-09-29 08:35:08 -07:00
Anis Elleuch
20c89ebbb3
freeze before exit when _MINIO_DEBUG_NO_EXIT is defined ( #15709 )
...
this is to ensure keep k8s pods running, when they reach a "crashloop" stage
2022-09-22 11:57:27 -07:00
Krishnan Parthasarathi
5ce1448049
audit: account for response headers separately ( #15610 )
2022-09-01 12:51:04 -07:00
Anis Elleuch
1f28a3bb80
Avoid messages from go test output ( #15601 )
...
A lot of warning messages are printed in CI/CD failures generated by go
test. Avoid that by requiring at least Error level for logging when
doing go test.
2022-08-30 08:23:40 -07:00
Harshavardhana
433b6fa8fe
upgrade golang-lint to the latest ( #15600 )
2022-08-26 12:52:29 -07:00
ebozduman
b57e7321e7
Replaces 'disk'=>'drive' visible to end user ( #15464 )
2022-08-04 16:10:08 -07:00
Harshavardhana
026b87e39b
do not crash for unwrapErrs return nil ( #15456 )
...
fixes #15454
2022-08-02 15:10:11 -07:00
Harshavardhana
5e763b71dc
use logger.LogOnce to reduce printing disconnection logs ( #15408 )
...
fixes #15334
- re-use net/url parsed value for http.Request{}
- remove gosimple, structcheck and unusued due to https://github.com/golangci/golangci-lint/issues/2649
- unwrapErrs upto leafErr to ensure that we store exactly the correct errors
2022-07-27 09:44:59 -07:00
jiuker
6b4f833a12
convert repeated error checks into single function in logger ( #15387 )
2022-07-25 17:53:03 -07:00
Harshavardhana
0a8b78cb84
fix: simplify passing auditLog eventType ( #15278 )
...
Rename Trigger -> Event to be a more appropriate
name for the audit event.
Bonus: fixes a bug in AddMRFWorker() it did not
cancel the waitgroup, leading to waitgroup leaks.
2022-07-12 10:43:32 -07:00
Harshavardhana
32b2f6117e
fix: do not pass around sync.Map ( #15250 )
...
it is not safe to pass around sync.Map
through pointers, as it may be concurrently
updated by different callers.
this PR simplifies by avoiding sync.Map
altogether, we do not need sync.Map
to keep object->erasureMap association.
This PR fixes a crash when concurrently
using this value when audit logs are
configured.
```
fatal error: concurrent map iteration and map write
goroutine 247651580 [running]:
runtime.throw({0x277a6c1?, 0xc002381400?})
runtime/panic.go:992 +0x71 fp=0xc004d29b20 sp=0xc004d29af0 pc=0x438671
runtime.mapiternext(0xc0d6e87f18?)
runtime/map.go:871 +0x4eb fp=0xc004d29b90 sp=0xc004d29b20 pc=0x41002b
```
2022-07-07 17:04:25 -07:00
