Commit Graph

80 Commits

Author SHA1 Message Date
Aditya Manthramurthy
f3248a4b37 Redact all secrets from config viewing APIs (#17380)
This change adds a `Secret` property to `HelpKV` to identify secrets
like passwords and auth tokens that should not be revealed by the server
in its configuration fetching APIs. Configuration reporting APIs now do
not return secrets.
2023-06-23 07:45:27 -07:00
Aditya Manthramurthy
5a1612fe32 Bump up madmin-go and pkg deps (#17469) 2023-06-19 17:53:08 -07:00
Harshavardhana
dbd4c2425e fix: kafka broker pings must not be greater than 1sec (#17376) 2023-06-07 11:47:00 -07:00
Krishnan Parthasarathi
55a3310446 logger-http: Don't retry after a succesful send (#17266) 2023-05-22 14:53:18 -07:00
jiuker
41fa8fa2d2 fix: increment counter when entry be skipped (#17237) 2023-05-19 08:36:52 -07:00
Praveen raj Mani
85912985b6 Check for only network errors in audit webhook for reachability (#17228) 2023-05-17 11:10:33 -07:00
Klaus Post
99c4ffa34f fix: avoid audit log race protection deadlocks (#17168) 2023-05-09 08:11:32 -07:00
Praveen raj Mani
57acacd5a7 Support persistent queue store for loggers (#17121) 2023-05-08 21:20:31 -07:00
jiuker
6e27264c6b update cleanupRoutine comment (#17102) 2023-04-28 01:11:51 -07:00
Anis Eleuch
5c83c9724f audit: Add request path and host to audit event (#17099) 2023-04-27 22:18:24 -07:00
jiuker
b28d391a22 fix: add correct worker count before startHTTPLogger() (#17091) 2023-04-27 10:51:16 -07:00
Harshavardhana
8a9b9832fd add Dial timeout for Kafka broker pings (#17044) 2023-04-17 15:45:01 -07:00
Harshavardhana
a5835cecbf fix: regression in counting total requests (#17024) 2023-04-12 14:37:19 -07:00
Anis Eleuch
d90d0c8931 Use one http response recorder per external http call (#16938) 2023-03-31 09:37:29 -07:00
Klaus Post
11d04279c8 Add lazy init of audit logger (#16842) 2023-03-21 10:50:40 -07:00
Harshavardhana
3b5dbf9046 allow bootstrapping to validate internode tokens (#16853) 2023-03-20 01:40:24 -07:00
Harshavardhana
46f9049fb4 simplify error responses for KMS (#16793) 2023-03-16 11:59:42 -07:00
Nitish Tiwari
50dbd2cacc Update audit log flow to use new headers with unit (#16797) 2023-03-13 22:50:19 -07:00
ferhat elmas
714283fae2 cleanup ignored static analysis (#16767) 2023-03-06 08:56:10 -08:00
Daniel Valdivia
fb17f97cf3 move audit and logger message structure to minio/pkg (#16655)
Signed-off-by: Daniel Valdivia <18384552+dvaldivia@users.noreply.github.com>
2023-02-21 21:21:17 -08:00
Shubhendu
6b65ba1551 Added attribute proxy for mc admin config set ALIAS logger_webhook (#16657)
Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>
2023-02-21 21:19:46 -08:00
Anis Elleuch
fadc46b906 Add the access key and parent user in the audit log (#16572) 2023-02-08 11:05:26 -08:00
Harshavardhana
aa8b9572b9 remove double ENABLED help output (#16528) 2023-02-03 05:52:52 -08:00
Harshavardhana
b67d97b1ba add missing fields in audit logs for non-compressed handlers (#16328) 2022-12-30 10:20:19 -08:00
Anis Elleuch
939c0100a6 log: Do not interpret verbs in object names in console output (#16233) 2022-12-13 08:27:40 -08:00
Aditya Manthramurthy
a30cfdd88f Bump up madmin-go to v2 (#16162) 2022-12-06 13:46:50 -08:00
Anis Elleuch
1f1dcdce65 move HTTP recorder to an internal library (#16128) 2022-11-28 10:20:27 -08:00
Shireesh Anjal
98a67a3776 Improvements in logger and audit webhooks (#16102) 2022-11-28 08:03:26 -08:00
jiuker
bf89f79694 save deploymentID to avoid mutating request entry in Audit (#16053) 2022-11-11 12:42:15 -08:00
Klaus Post
5b242f1d11 Add Audit target metrics (#16044) 2022-11-10 10:20:21 -08:00
Klaus Post
ddeca9f12a fix: filter rest errors and logs returned (#16019) 2022-11-07 10:38:08 -08:00
Harshavardhana
23b329b9df remove gateway completely (#15929) 2022-10-24 17:44:15 -07:00
Harshavardhana
f696a221af allow tagging policy condition for GetObject (#15777) 2022-10-02 12:29:29 -07:00
Anis Elleuch
f69a98ce49 fix: loading Audit kafka configuration loading (#15766) 2022-09-29 08:35:08 -07:00
Anis Elleuch
20c89ebbb3 freeze before exit when _MINIO_DEBUG_NO_EXIT is defined (#15709)
this is to ensure keep k8s pods running, when they reach a "crashloop" stage
2022-09-22 11:57:27 -07:00
Krishnan Parthasarathi
5ce1448049 audit: account for response headers separately (#15610) 2022-09-01 12:51:04 -07:00
Anis Elleuch
1f28a3bb80 Avoid messages from go test output (#15601)
A lot of warning messages are printed in CI/CD failures generated by go
test. Avoid that by requiring at least Error level for logging when
doing go test.
2022-08-30 08:23:40 -07:00
Harshavardhana
433b6fa8fe upgrade golang-lint to the latest (#15600) 2022-08-26 12:52:29 -07:00
ebozduman
b57e7321e7 Replaces 'disk'=>'drive' visible to end user (#15464) 2022-08-04 16:10:08 -07:00
Harshavardhana
026b87e39b do not crash for unwrapErrs return nil (#15456)
fixes #15454
2022-08-02 15:10:11 -07:00
Harshavardhana
5e763b71dc use logger.LogOnce to reduce printing disconnection logs (#15408)
fixes #15334

- re-use net/url parsed value for http.Request{}
- remove gosimple, structcheck and unusued due to https://github.com/golangci/golangci-lint/issues/2649
- unwrapErrs upto leafErr to ensure that we store exactly the correct errors
2022-07-27 09:44:59 -07:00
jiuker
6b4f833a12 convert repeated error checks into single function in logger (#15387) 2022-07-25 17:53:03 -07:00
Harshavardhana
0a8b78cb84 fix: simplify passing auditLog eventType (#15278)
Rename Trigger -> Event to be a more appropriate
name for the audit event.

Bonus: fixes a bug in AddMRFWorker() it did not
cancel the waitgroup, leading to waitgroup leaks.
2022-07-12 10:43:32 -07:00
Harshavardhana
32b2f6117e fix: do not pass around sync.Map (#15250)
it is not safe to pass around sync.Map
through pointers, as it may be concurrently
updated by different callers.

this PR simplifies by avoiding sync.Map
altogether, we do not need sync.Map
to keep object->erasureMap association.

This PR fixes a crash when concurrently
using this value when audit logs are
configured.

```
fatal error: concurrent map iteration and map write

goroutine 247651580 [running]:
runtime.throw({0x277a6c1?, 0xc002381400?})
        runtime/panic.go:992 +0x71 fp=0xc004d29b20 sp=0xc004d29af0 pc=0x438671
runtime.mapiternext(0xc0d6e87f18?)
        runtime/map.go:871 +0x4eb fp=0xc004d29b90 sp=0xc004d29b20 pc=0x41002b
```
2022-07-07 17:04:25 -07:00
Klaus Post
ac055b09e9 Add detailed scanner metrics (#15161) 2022-07-05 14:45:49 -07:00
Klaus Post
9004d69c6f Make ReqInfo concurrency safe (#15204)
Some read/writes of ReqInfo did not get appropriate locks, leading to races.

Make sure reading and writing holds appropriate locks.
2022-06-30 10:48:50 -07:00
Harshavardhana
9d07cde385 use crypto/sha256 only for FIPS 140-2 compliance (#14983)
It would seem like the PR #11623 had chewed more
than it wanted to, non-fips build shouldn't really
be forced to use slower crypto/sha256 even for
presumed "non-performance" codepaths. In MinIO
there are really no "non-performance" codepaths.
This assumption seems to have had an adverse
effect in certain areas of CPU usage.

This PR ensures that we stick to sha256-simd
on all non-FIPS builds, our most common build
to ensure we get the best out of the CPU at
any given point in time.
2022-05-27 06:00:19 -07:00
Anis Elleuch
e952e2a691 audit/kafka: Fix quitting early after first logging (#14932)
A recent commit created some regressions:
- Kafka/Audit goroutines quit when the first log is sent
- Missing doneCh initialization in Kafka audit
2022-05-17 07:43:25 -07:00
Harshavardhana
040ac5cad8 fix: when logger queue is full exit quickly upon doneCh (#14928)
Additionally only reload requested sub-system not everything
2022-05-16 16:10:51 -07:00
Anis Elleuch
05685863e3 Cancel old logger/audit targets outside lock (#14927)
When configuring a new target, such as an audit target, the server waits
until all audit events are sent to the audit target before doing the
swap from the old to the new audit target. Therefore current S3 operations
can suffer from this since the audit swap lock will be held.

This behavior is unnecessary as the new audit target can enter in a
functional mode immediately and the old audit will just cancel itself
at its own pace.
2022-05-16 13:32:36 -07:00