Klaus Post
e8c0a50862
optimization use small blocks up to 64KB ( #17107 )
2023-05-01 09:47:49 -07:00
Harshavardhana
7ae69accc0
allow root user to be disabled via config settings ( #17089 )
2023-04-28 12:24:14 -07:00
Klaus Post
7fad0c8b41
Remove checksums from HTTP range request, add part checksums ( #17105 )
2023-04-28 08:26:32 -07:00
jiuker
6e27264c6b
update cleanupRoutine comment ( #17102 )
2023-04-28 01:11:51 -07:00
Anis Eleuch
5c83c9724f
audit: Add request path and host to audit event ( #17099 )
2023-04-27 22:18:24 -07:00
jiuker
b28d391a22
fix: add correct worker count before startHTTPLogger() ( #17091 )
2023-04-27 10:51:16 -07:00
jiuker
c8b92f6067
protect wg.Done from being called twice ( #17075 )
2023-04-27 07:55:36 -07:00
Anis Eleuch
31b5acc245
tcp: Increase user timeout to 10 minutes ( #17087 )
2023-04-26 17:48:31 -07:00
Anis Eleuch
0b7ca094e4
Remove Expect 100-continue in internode communications ( #17061 )
2023-04-26 09:33:45 -07:00
Praveen raj Mani
72802a5972
Use 'minio/pkg/sync/errgroup' and 'minio/pkg/workers' ( #17069 )
2023-04-25 22:57:40 -07:00
Harshavardhana
8fd07bcd51
simplify sort.Sort by using sort.Slice ( #17066 )
2023-04-24 13:28:18 -07:00
Harshavardhana
8a9b9832fd
add Dial timeout for Kafka broker pings ( #17044 )
2023-04-17 15:45:01 -07:00
jiuker
e96c88e914
fix: DeleteBucketThrottle must delete ARN ( #17034 )
2023-04-15 02:14:26 -07:00
Klaus Post
c133979b8e
Add part count to checksum ( #17035 )
2023-04-14 09:44:45 -07:00
Harshavardhana
a5835cecbf
fix: regression in counting total requests ( #17024 )
2023-04-12 14:37:19 -07:00
Poorna
d350654aee
config: fix duplication of replication priority key ( #17014 )
2023-04-11 19:22:10 -07:00
Harshavardhana
09a25ea7b7
lint: fix some lint issues on files
2023-04-06 22:42:10 -07:00
Harshavardhana
47b7469a60
add buffer pool for proxy forwarder ( #16942 )
2023-04-06 15:54:12 -07:00
Praveen raj Mani
51f7f9aaa3
Generalize the event store using go generics ( #16910 )
2023-04-04 10:52:24 -07:00
Anis Eleuch
d90d0c8931
Use one http response recorder per external http call ( #16938 )
2023-03-31 09:37:29 -07:00
Anis Eleuch
c259a8ea38
Set tcp user timeout to clean sockets with data in the buffer ( #16887 )
2023-03-24 08:10:58 -07:00
mstmdev
2d51e42305
Remove the redundant conditional in the validateParity function ( #16866 )
2023-03-23 14:06:22 -07:00
Anis Eleuch
1346561b9d
return quorum error instead of insufficient storage error ( #16874 )
2023-03-22 16:22:37 -07:00
Klaus Post
11d04279c8
Add lazy init of audit logger ( #16842 )
2023-03-21 10:50:40 -07:00
Harshavardhana
3b5dbf9046
allow bootstrapping to validate internode tokens ( #16853 )
2023-03-20 01:40:24 -07:00
Aditya Manthramurthy
09c733677a
Add test for fixed post policy exploit ( #16855 )
2023-03-20 01:06:45 -07:00
Harshavardhana
46f9049fb4
simplify error responses for KMS ( #16793 )
2023-03-16 11:59:42 -07:00
Aditya Manthramurthy
58266c9e2c
Add enable flag for LDAP IDP config ( #16805 )
2023-03-16 11:58:59 -07:00
Nitish Tiwari
50dbd2cacc
Update audit log flow to use new headers with unit ( #16797 )
2023-03-13 22:50:19 -07:00
Harshavardhana
b984bf8d1a
allow expiration of all versions during Listing() ( #16757 )
2023-03-09 15:15:30 -08:00
Harshavardhana
901887e6bf
feat: add lambda transformation functions target ( #16507 )
2023-03-07 08:12:41 -08:00
ferhat elmas
714283fae2
cleanup ignored static analysis ( #16767 )
2023-03-06 08:56:10 -08:00
ferhat elmas
3423028713
cleanup Go linter settings ( #16736 )
2023-03-04 20:57:35 -08:00
Harshavardhana
0ff931dc76
fix: allow CORS to work by default ( #16713 )
2023-02-27 10:10:45 -08:00
Praveen raj Mani
4d708cebe9
Support adding service accounts with expiration ( #16430 )
...
Co-authored-by: Harshavardhana <harsha@minio.io>
2023-02-27 10:10:22 -08:00
Aditya Manthramurthy
8cde38404d
Add metrics for custom auth plugin ( #16701 )
2023-02-27 09:55:18 -08:00
Shubhendu
8b4eb2304b
Set logger webhook proxy on subnet proxy change ( #16665 )
...
Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>
2023-02-27 08:35:36 -08:00
Anis Elleuch
8da0f4c5bb
Better error message when TLS certs do not have proper permissions ( #16703 )
2023-02-24 06:34:55 -08:00
Harshavardhana
b21d3f9b82
event target registration failures must be returned ( #16700 )
2023-02-23 21:59:14 +05:30
Harshavardhana
5c98223c89
add correct HostId instead of deploymentId for error responses ( #16686 )
2023-02-22 15:41:09 +05:30
Daniel Valdivia
fb17f97cf3
move audit and logger message structure to minio/pkg ( #16655 )
...
Signed-off-by: Daniel Valdivia <18384552+dvaldivia@users.noreply.github.com>
2023-02-21 21:21:17 -08:00
Shubhendu
6b65ba1551
Added attribute proxy for mc admin config set ALIAS logger_webhook ( #16657 )
...
Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>
2023-02-21 21:19:46 -08:00
Allan Roger Reid
8bfe972bab
Set meaningful message from minio with env variable KMS_SECRET_KEY ( #16584 )
2023-02-22 07:13:01 +05:30
Klaus Post
fd6622458b
Add detailed scanner trace output and notifications ( #16668 )
2023-02-21 09:33:33 -08:00
jiuker
e470268c7c
fix: a possible closer leak in SelectObjectHandler ( #16598 )
2023-02-17 01:44:40 -08:00
Harshavardhana
6ea150fd68
fix: avoid printing certain errors under few locations ( #16631 )
2023-02-17 01:40:31 -08:00
Harshavardhana
e1e9ddd4a4
use kes.Status() for Status() call ( #16629 )
2023-02-16 22:12:24 +05:30
Andreas Auernhammer
74887c7372
kms: add support for KES API keys and switch to KES Go SDK ( #16617 )
...
Signed-off-by: Andreas Auernhammer <hi@aead.dev>
2023-02-14 07:19:20 -08:00
Harshavardhana
31188e9327
add parallel workers in batch replication ( #16609 )
2023-02-13 12:07:58 -08:00
Harshavardhana
d65debb6bc
fix: comply with RFC6750 UserInfo endpoint requirements ( #16592 )
2023-02-10 22:20:25 +05:30
Anis Elleuch
c8ffa59d28
Periodically refresh buckets metadata from the backend disks ( #16561 )
...
fixes #16553
2023-02-09 10:29:20 -08:00
Anis Elleuch
fadc46b906
Add the access key and parent user in the audit log ( #16572 )
2023-02-08 11:05:26 -08:00
Anis Elleuch
095b518802
Show a better error msg when internal data encryption key is incorrect ( #16549 )
2023-02-07 05:22:54 -08:00
Harshavardhana
14cf8f1b22
upgrade deps for minio/pkg v1.6.1 to include groups conditions ( #16538 )
2023-02-06 09:27:29 -08:00
Harshavardhana
5996c8c4d5
feat: allow offline disks on a fresh start ( #16541 )
2023-02-06 09:26:09 -08:00
Harshavardhana
aa8b9572b9
remove double ENABLED help output ( #16528 )
2023-02-03 05:52:52 -08:00
Klaus Post
0b7989aa4b
Fix Kafka initialization crash ( #16523 )
2023-02-02 19:21:19 +05:30
Harshavardhana
d19cbc81b5
fix: do not return IAM/Bucket metadata replication errors to client ( #16486 )
2023-01-26 11:11:54 -08:00
Harshavardhana
65c104a589
add x-amz-id-2 to indicate the node that received the request ( #16474 )
2023-01-25 09:14:10 -08:00
Harshavardhana
3683673fb0
add missing gorilla/mux migration, update credits ( #16461 )
2023-01-23 08:46:37 -08:00
Poorna
1b02e046c2
Fix bandwidth monitoring to be per remote target ( #16360 )
2023-01-19 18:52:16 +05:30
Aditya Manthramurthy
698862ec5d
Fix transports/timeouts related regressions ( #16427 )
2023-01-18 10:06:38 +05:30
Harshavardhana
3db658e51e
use correct xml package for custom MarshalXML() ( #16421 )
2023-01-17 05:08:33 +05:30
Allan Roger Reid
9815dac48f
fix: allow bind on ipv6 loopback failures ( #16388 )
2023-01-11 08:47:39 +05:30
Anis Elleuch
1ece3d1dfe
Add comment field to service accounts ( #16380 )
2023-01-10 21:57:52 +04:00
Anis Elleuch
2146ed4033
xl: Quit early when EC config is incorrect ( #16390 )
...
Co-authored-by: Anis Elleuch <anis@min.io>
2023-01-09 23:07:45 -08:00
Anis Elleuch
1c85652cff
lint: Fix in darwin environment ( #16368 )
2023-01-05 10:12:01 -08:00
Klaus Post
6a04067514
fix: tweak read buffer size to reduce over-reading ( #16338 )
2023-01-01 08:14:20 -08:00
Harshavardhana
75faef888e
disable builds for go1.18 ( #16332 )
2022-12-30 11:37:07 -08:00
Harshavardhana
b67d97b1ba
add missing fields in audit logs for non-compressed handlers ( #16328 )
2022-12-30 10:20:19 -08:00
Anis Elleuch
acc9c033ed
debug: Add X-Amz-Request-ID to lock/unlock calls ( #16309 )
2022-12-23 19:49:07 -08:00
Anis Elleuch
e57e946206
Do not save credentials in config.json ( #16275 )
2022-12-19 12:27:06 -08:00
Harshavardhana
80fc3a8a52
use newDynamicTimeoutWithOpts() when appropriate ( #16266 )
2022-12-15 13:11:37 -08:00
Klaus Post
988a2e8fed
Faster startup of large distributed systems with latency ( #16259 )
2022-12-15 08:31:21 -08:00
Harshavardhana
bad229e16e
fix: support event name s3:Restore:* ( #16257 )
2022-12-14 05:12:07 -08:00
Harshavardhana
c73ea27ed7
do not log checksum mismatch error, client received the error ( #16246 )
2022-12-14 01:57:40 -08:00
Aditya Manthramurthy
9e6cc847f8
Add HTTP2 config option for policy plugin ( #16225 )
2022-12-13 14:28:48 -08:00
Anis Elleuch
939c0100a6
log: Do not interpret verbs in object names in console output ( #16233 )
2022-12-13 08:27:40 -08:00
Aditya Manthramurthy
2d60bf8c50
Refactor HTTP transports ( #16222 )
2022-12-12 20:31:21 -08:00
Aditya Manthramurthy
a469e6768d
Add LDAP DNS SRV record lookup support ( #16201 )
2022-12-12 11:27:27 -08:00
Harshavardhana
2fc182d8e6
fix: iso8601TimeFormat padding issue for certain nanoseconds ( #16207 )
2022-12-12 10:28:30 -08:00
Aditya Manthramurthy
e06127566d
Add IAM API to attach/detach policies for LDAP ( #16182 )
2022-12-09 13:08:33 -08:00
jiuker
8edc2faaa9
reuse sha256 in config GetSettings ( #16188 )
2022-12-08 03:03:24 -08:00
Javier Adriel
04ae9058ed
Populate end_session_endpoint ( #16183 )
2022-12-06 16:56:37 -08:00
Aditya Manthramurthy
a30cfdd88f
Bump up madmin-go to v2 ( #16162 )
2022-12-06 13:46:50 -08:00
Anis Elleuch
932d2c3c62
Add X-Amz-Request-Id to internode calls ( #16146 )
2022-12-06 09:27:26 -08:00
Harshavardhana
419f351df3
avoid logging gzipped body in trace output ( #16172 )
2022-12-05 13:21:27 -08:00
Klaus Post
a713aee3d5
Run staticcheck on CI ( #16170 )
2022-12-05 11:18:50 -08:00
Andreas Auernhammer
d882ba2cb4
kms: add support for KES enclaves ( #16139 )
...
Signed-off-by: Andreas Auernhammer <hi@aead.dev>
2022-12-04 02:34:24 -08:00
Harshavardhana
9b35c72349
fix: a crash in KMS cert reload function ( #16158 )
2022-12-02 11:19:05 -08:00
Aditya Manthramurthy
87cbd41265
feat: Allow at most one claim based OpenID IDP ( #16145 )
2022-11-29 15:40:49 -08:00
Harshavardhana
be92cf5959
change dependency from amqp -> amqp091 (RabbitMQ) official ( #16142 )
2022-11-28 16:05:06 -08:00
Anis Elleuch
1f1dcdce65
move HTTP recorder to an internal library ( #16128 )
2022-11-28 10:20:27 -08:00
Shireesh Anjal
98a67a3776
Improvements in logger and audit webhooks ( #16102 )
2022-11-28 08:03:26 -08:00
Andreas Auernhammer
9b1e70e4f9
kms: fix possible deadlock due to nested RLock calls. ( #16136 )
...
Signed-off-by: Andreas Auernhammer <hi@aead.dev>
2022-11-28 07:31:07 -08:00
Harshavardhana
09d4f8cd0f
avoid serializing decryptKey() every 15mins ( #16135 )
...
if the certs are the same in an environment where the
cert files are symlinks (e.g Kubernetes), then we resort
to reloading certs every 15mins - we can avoid reload
of the kes client instance. Ensure that the price to pay
for contending with the lock must happen when necessary.
2022-11-28 01:14:33 -08:00
Anis Elleuch
97eb7dbf5f
notify: Return detailed err msg when connecting to target fails ( #16118 )
2022-11-24 07:59:19 -08:00
Harshavardhana
08103870a5
update single drive setup error message ( #16098 )
2022-11-18 14:47:38 -08:00
Anis Elleuch
993e586855
config: return XMinioConfigNotFound code for non existing config ( #16065 )
2022-11-18 10:28:14 -08:00
Shireesh Anjal
5246e3be84
Send health diagnostics data as part of callhome ( #16006 )
2022-11-15 13:53:05 -08:00
Harshavardhana
962d1f1a71
choose default values upon incorrect storage_class value ( #16058 )
2022-11-12 10:18:21 -08:00
Harshavardhana
6d76db9d6c
improve server startup error when pools are incorrect ( #16056 )
2022-11-11 19:40:45 -08:00
Ray
66239f30ce
configuring the nats target to reconnect forever ( #16050 )
2022-11-11 12:42:41 -08:00
jiuker
bf89f79694
save deploymentID to avoid mutating request entry in Audit ( #16053 )
2022-11-11 12:42:15 -08:00
Klaus Post
5b242f1d11
Add Audit target metrics ( #16044 )
2022-11-10 10:20:21 -08:00
Krishnan Parthasarathi
6eef9b4a23
lifecycle: simplify Eval and HasActiveRules ( #16036 )
2022-11-10 07:17:45 -08:00
Krishnan Parthasarathi
7ba281728f
ilm: fix x-amz-expiration header evaluation ( #16029 )
2022-11-09 04:20:34 -08:00
Klaus Post
bbc312fce6
Add notification queue metrics ( #16026 )
2022-11-08 16:36:47 -08:00
Harshavardhana
1b0dfb0f58
remove printing map() checksums ( #16028 )
2022-11-08 13:29:24 -08:00
Aditya Manthramurthy
76d822bf1e
Add LDAP policy entities API ( #15908 )
2022-11-07 14:35:09 -08:00
Klaus Post
ddeca9f12a
fix: filter rest errors and logs returned ( #16019 )
2022-11-07 10:38:08 -08:00
Harshavardhana
21251d8c22
initialize streaming events without lazy initialization ( #16016 )
2022-11-07 08:01:24 -08:00
Harshavardhana
944c62daf4
skip flaky tests on windows OS ( #16015 )
2022-11-07 00:11:21 -08:00
Harshavardhana
9547b7d0e9
add deadlineConnections on remoteTransport ( #16010 )
2022-11-05 11:09:21 -07:00
Harshavardhana
76c4ea7682
force all internal MinIO operations to be under UTC ( #16009 )
2022-11-04 16:44:38 -07:00
Harshavardhana
7be65f66b8
support HS256 series of JWT signature for OpenID connect ( #15993 )
2022-11-03 16:41:53 -07:00
Anis Elleuch
7721595aa9
config: Deprecated delay/max_wait/scanner and introduce speed ( #15941 )
2022-11-01 08:04:07 -07:00
Harshavardhana
4892a766a8
do not panic if webhook returns an error ( #15970 )
2022-10-30 16:45:53 -07:00
Poorna
d765b89a63
improve validation for replication resync API ( #15964 )
2022-10-28 23:21:33 -07:00
Klaus Post
71954faa3a
mark pubsub type safe via generics ( #15961 )
2022-10-28 10:55:42 -07:00
Shireesh Anjal
6d22e74d11
mark SUBNET config keys as sensitive info ( #15962 )
2022-10-28 10:54:44 -07:00
Harshavardhana
ec77d28e62
make subnet subsys dynamic and simplify callhome ( #15927 )
2022-10-27 00:20:01 -07:00
Harshavardhana
23b329b9df
remove gateway completely ( #15929 )
2022-10-24 17:44:15 -07:00
Poorna
e4e90b53c1
fix: delete-marker replication check properly ( #15923 )
2022-10-21 14:45:06 -07:00
Anis Elleuch
ac85c2af76
lifecycle: refactor rules filtering and tagging support ( #15914 )
2022-10-21 10:46:53 -07:00
Aditya Manthramurthy
2d16e74f38
Add LDAP IDP Configuration APIs ( #15840 )
2022-10-19 11:00:10 -07:00
Klaus Post
bd3dfad8b9
Add concurrent Snowball extraction + options ( #15836 )
2022-10-18 13:50:21 -07:00
Aditya Manthramurthy
85fc7cea97
Pass role ARN for OIDC providers to console ( #15862 )
2022-10-15 12:57:03 -07:00
Harshavardhana
328d660106
support CRC32 Checksums on single drive setup ( #15873 )
2022-10-15 11:58:47 -07:00
Harshavardhana
59e33b3b21
validate setBucketTarget properly as per BucketExists() call ( #15860 )
2022-10-13 17:46:49 -07:00
Poorna
0e3c92c027
attempt delete marker replication after object is replicated ( #15857 )
...
Ensure delete marker replication success, especially since the
recent optimizations to heal on HEAD, LIST and GET can force
replication attempts on delete marker before underlying object
version could have synced.
2022-10-13 17:45:23 -07:00
hellivan
b0b573052a
fix: unrecognized openid config parameters ( #15847 )
2022-10-12 12:19:44 -07:00
Javier Adriel
2939000342
Add metrics, version and apis handlers ( #15839 )
2022-10-12 12:08:03 -07:00
Anis Elleuch
afd4279cd8
lock tests: Initialize different DRWMutex for each lock ( #15833 )
2022-10-10 15:14:32 -07:00
Harshavardhana
3c4ef4338f
marshal retention XML in expected format ( #15821 )
2022-10-08 13:11:00 -07:00
Aditya Manthramurthy
64cf887b28
use LDAP config from minio/pkg to share with console ( #15810 )
2022-10-07 22:12:36 -07:00
Krishnan Parthasarathi
6d6a731d6d
Handle overlapping and conflicting ILM rules ( #15812 )
2022-10-07 14:36:23 -07:00
Poorna
b6b26dba87
fix: GetObjectRetention to parse in ISO8601 time format ( #15809 )
2022-10-06 13:53:56 -07:00
Harshavardhana
2a13cc28f2
feat: implement support batch replication ( #15554 )
2022-10-05 23:00:43 -07:00
Javier Adriel
3307aa1260
Implement KMS handlers ( #15737 )
2022-10-04 10:05:09 -07:00
Harshavardhana
f696a221af
allow tagging policy condition for GetObject ( #15777 )
2022-10-02 12:29:29 -07:00
Anis Elleuch
6287e8c571
fix: race when accessing REST TCP dial values ( #15770 )
2022-09-29 09:27:58 -07:00
Anis Elleuch
f69a98ce49
fix: loading Audit kafka configuration loading ( #15766 )
2022-09-29 08:35:08 -07:00
Daryl White
d44f3526dc
Update links to documentation site ( #15750 )
2022-09-28 21:28:45 -07:00
Harshavardhana
41b633f5ea
support tagging based policy conditions ( #15763 )
2022-09-28 11:25:46 -07:00
Anis Elleuch
86bb48792c
non-blocking initialization of bucket target notifications ( #15571 )
2022-09-27 17:23:28 -07:00
Harshavardhana
94dbb4a427
fix: generalize SC config and also skip healing sub-sys under SD ( #15757 )
2022-09-26 09:04:54 -07:00
Anis Elleuch
048a46ec2a
Add RPC tcp timeout/errs and AVG duration to prometheus ( #15747 )
2022-09-26 09:04:26 -07:00
Harshavardhana
877bd95fa3
remove unused package internal/smart ( #15758 )
2022-09-25 08:44:53 -07:00
Poorna
8ea6fb368d
Add auto configuration of replication workers ( #15636 )
2022-09-24 16:20:28 -07:00
Poorna
5fd5ddea23
relax retention date validation on replication target ( #15752 )
2022-09-23 21:19:03 -07:00
Anis Elleuch
20c89ebbb3
freeze before exit when _MINIO_DEBUG_NO_EXIT is defined ( #15709 )
...
this is to ensure keep k8s pods running, when they reach a "crashloop" stage
2022-09-22 11:57:27 -07:00
Krishnan Parthasarathi
6f56ba80b3
lifecycle: Assign unique id to rules with empty id ( #15731 )
2022-09-22 10:51:54 -07:00
Klaus Post
ff12080ff5
Remove deprecated io/ioutil ( #15707 )
2022-09-19 11:05:16 -07:00
Javier Adriel
0b6175b742
Implement KMS methods for keys, policies and identities ( #15673 )
2022-09-19 11:04:40 -07:00
Klaus Post
c22f3ca7a8
fix: S3 Select CSV -> JSON with variable field count ( #15677 )
...
When there are fewer fields than expected, output fewer fields.
2022-09-12 17:00:59 -07:00
Anis Elleuch
4a92134235
prometheus: track errors during REST read/write calls ( #15678 )
...
minio_inter_node_traffic_errors_total currently does not track
requests body write/read errors of internode REST communications.
This commit fixes this by wrapping resp.Body.
2022-09-12 12:40:51 -07:00
Poorna
6b9fd256e1
Persist in-memory replication stats to disk ( #15594 )
...
to avoid relying on scanner-calculated replication metrics.
This will improve the accuracy of the replication stats reported.
This PR also adds on to #15556 by handing replication
traffic that could not be queued by available workers to the
MRF queue so that entries in `PENDING` status are healed faster.
2022-09-12 12:40:02 -07:00
Harshavardhana
228c6686f8
allow non-standards fallback for all http.TimeFormats ( #15662 )
...
fixes #15645
2022-09-07 07:24:54 -07:00
Krishnan Parthasarathi
5ce1448049
audit: account for response headers separately ( #15610 )
2022-09-01 12:51:04 -07:00
Klaus Post
8e4a45ec41
fix: encrypt checksums in metadata ( #15620 )
2022-08-31 08:13:23 -07:00
Abirdcfly
d4e0f13bb3
chore: remove duplicate word in comments ( #15607 )
...
Signed-off-by: Abirdcfly <fp544037857@gmail.com>
Signed-off-by: Abirdcfly <fp544037857@gmail.com>
2022-08-30 08:26:43 -07:00
Anis Elleuch
1f28a3bb80
Avoid messages from go test output ( #15601 )
...
A lot of warning messages are printed in CI/CD failures generated by go
test. Avoid that by requiring at least Error level for logging when
doing go test.
2022-08-30 08:23:40 -07:00
Klaus Post
a9f1ad7924
Add extended checksum support ( #15433 )
2022-08-29 16:57:16 -07:00
Harshavardhana
97376f6e8f
improve performance for inlined data ( #15603 )
...
inlined data often is bigger than the allowed
O_DIRECT alignment, so potentially we can write
'xl.meta' without O_DSYNC instead we can rely on
O_DIRECT + fdatasync() instead.
This PR allows O_DIRECT on inlined data that
would gain the benefits of performing O_DIRECT,
eventually performing an fdatasync() at the end.
Performance boost can be observed here for small
objects < 128KiB. The performance boost is mainly
seen on HDD, and marginal on NVMe setups.
2022-08-29 11:19:29 -07:00
Harshavardhana
433b6fa8fe
upgrade golang-lint to the latest ( #15600 )
2022-08-26 12:52:29 -07:00
Krishnan Parthasarathi
99fbfe2421
Add concurrency to healing objects on a fresh disk ( #15575 )
2022-08-25 13:07:15 -07:00
Aditya Manthramurthy
18dffb26e7
Allow querying a single target in config get API ( #15587 )
2022-08-25 00:17:05 -07:00
Aditya Manthramurthy
c54c13831a
fix: error types in internal config package ( #15559 )
2022-08-20 15:11:49 -07:00
Harshavardhana
ae4ee95d25
change default lock retry interval to 50ms ( #15560 )
...
competing calls on the same object on versioned bucket
mutating calls on the same object may unexpected have
higher delays.
This can be reproduced with a replicated bucket
overwriting the same object writes, deletes repeatedly.
For longer locks like scanner keep the 1sec interval
2022-08-19 16:21:05 -07:00
Aditya Manthramurthy
21831b3fe2
fix: cleanup config KV parsing using madmin helpers ( #15552 )
2022-08-18 15:55:17 -07:00
Aditya Manthramurthy
701a82642b
Ensure subnet+callhome configs are single sub-sys ( #15542 )
2022-08-16 17:46:46 -07:00
Harshavardhana
48640b1de2
fix: trim arn:aws:kms from incoming SSE aws-kms-key-id ( #15540 )
2022-08-16 11:28:30 -07:00
Anis Elleuch
5682685c80
Introduce disk io stats metrics ( #15512 )
2022-08-16 07:13:49 -07:00
Aditya Manthramurthy
ee2afcf70b
Move some config constants to madmin ( #15529 )
2022-08-13 14:50:16 -07:00
Aditya Manthramurthy
9986e103cf
Fix env var output in config get/export APIs ( #15528 )
...
Fix a bug where env vars are not output when the config for the
subsystem is specified solely via env vars.
2022-08-13 10:39:01 -07:00
Shireesh Anjal
316c492842
Upgrade madmin-go to latest version (v1.4.15) ( #15510 )
2022-08-10 07:36:13 -07:00
Harshavardhana
a406bb0288
restrict number of disks used for scanning buckets upto GOMAXPROCS ( #15492 )
...
control scanner parallelism to avoid higher CPU
usage on nodes that have more drives but an old CPU.
2022-08-08 16:16:44 -07:00
Harshavardhana
1823ab6808
LDAP/OpenID must be initialized IAM Init() ( #15491 )
...
This allows for LDAP/OpenID to be non-blocking,
allowing for unreachable Identity targets to be
initialized in IAM.
2022-08-08 16:16:27 -07:00
Aditya Manthramurthy
3d94c38ec4
Add env variables to configuration APIs output ( #15465 )
...
Config export and config get APIs now include environment
variables set on the server
2022-08-04 22:21:52 -07:00
ebozduman
b57e7321e7
Replaces 'disk'=>'drive' visible to end user ( #15464 )
2022-08-04 16:10:08 -07:00
Andreas Auernhammer
d774a3309b
kes: automatically reload KES client certificate ( #15450 )
...
This commit adds support for automatically reloading
the MinIO client certificate for authentication to KES.
The client certificate will now be reloaded:
- when the private key / certificate file changes
- when a SIGHUP signal is received
- every 15 minutes
Fixes #14869
Signed-off-by: Andreas Auernhammer <hi@aead.dev>
2022-08-02 16:58:09 -07:00
Harshavardhana
026b87e39b
do not crash for unwrapErrs return nil ( #15456 )
...
fixes #15454
2022-08-02 15:10:11 -07:00
Harshavardhana
fd349103e8
fix: allow P-384/P-512 constant time implementation ( #15445 )
...
since go1.18.x P-384/P-512 are now constant time
implementations, enable them.
2022-08-01 09:27:16 -07:00
Anis Elleuch
10b49eb4fb
Fix resetting a config with a non default target name ( #15448 )
...
mc admin config reset <alias> notify_webhook:something was not working
properly.
The reason is that GetSubSys() was not calculating the target
name properly because it is quitting early when the number of config
inputs ('notify_webhook:something' in this case) is equal to 1.
This commit will make the code calculates always calculate the target
name if found.
2022-08-01 07:52:23 -07:00
Anis Elleuch
3856d078d2
fix: set 20000 as maximum parallel event calls ( #15435 )
...
This is needed to avoid consuming a lot of goroutines when a target is
very slow or there is a bug in a target library.
2022-07-30 12:12:33 -07:00
Harshavardhana
5e763b71dc
use logger.LogOnce to reduce printing disconnection logs ( #15408 )
...
fixes #15334
- re-use net/url parsed value for http.Request{}
- remove gosimple, structcheck and unusued due to https://github.com/golangci/golangci-lint/issues/2649
- unwrapErrs upto leafErr to ensure that we store exactly the correct errors
2022-07-27 09:44:59 -07:00
jiuker
6b4f833a12
convert repeated error checks into single function in logger ( #15387 )
2022-07-25 17:53:03 -07:00
Eng Zer Jun
0a3b1ad4eb
test: use T.TempDir to create temporary test directory ( #15400 )
...
This commit replaces `ioutil.TempDir` with `t.TempDir` in tests. The
directory created by `t.TempDir` is automatically removed when the test
and all its subtests complete.
Prior to this commit, temporary directory created using `ioutil.TempDir`
needs to be removed manually by calling `os.RemoveAll`, which is omitted
in some tests. The error handling boilerplate e.g.
defer func() {
if err := os.RemoveAll(dir); err != nil {
t.Fatal(err)
}
}
is also tedious, but `t.TempDir` handles this for us nicely.
Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-07-25 12:37:26 -07:00
Klaus Post
3795b2c8ba
Add compression scheme to header ( #15395 )
...
For easier debugging. We still do not return compressed size for security reasons.
2022-07-24 07:15:49 -07:00
Taran Pelkey
e83930333b
Allow DelKVS to delete specific sub-system fields. ( #15354 )
2022-07-22 14:48:23 -07:00
Harshavardhana
b0d70a0e5e
support additional claim info in Auditing STS calls ( #15381 )
...
Bonus: Adds a missing AuditLog from AssumeRoleWithCertificate API
Fixes #9529
2022-07-22 11:12:03 -07:00
Aditya Manthramurthy
2393a13f86
Allow site replication config with multiple IDPs ( #15361 )
...
Fixes a bug that did not let site replication be configured when
multiple IDPs are configured.
2022-07-21 19:52:23 -07:00
Harshavardhana
8249cd4406
fix: allow payload verification error to be returned ( #15364 )
...
without reading the reader the error is ignored
by the custom unmarshaller written by ObjectLegalHold
data structure.
2022-07-21 01:24:03 -07:00
Andreas Auernhammer
242d06274a
kms: add context.Context to KMS API calls ( #15327 )
...
This commit adds a `context.Context` to the
the KMS `{Stat, CreateKey, GenerateKey}` API
calls.
The context will be used to terminate external calls
as soon as the client requests gets canceled.
A follow-up PR will add a `context.Context` to
the remaining `DecryptKey` API call.
Signed-off-by: Andreas Auernhammer <hi@aead.dev>
2022-07-18 18:54:27 -07:00
Harshavardhana
b6eb8dff64
Add decommission compression+encryption enabled tests ( #15322 )
...
update compression environment variables to follow
the expected sub-system style, however support fallback
mode.
2022-07-17 08:43:14 -07:00
Harshavardhana
785b429737
add reconnect duration allows for verifying disconnect intervals ( #15306 )
2022-07-15 14:41:24 -07:00
Poorna
53cc561048
Default DeleteReplication rule status if unspecified. ( #15301 )
...
Since this is a MinIO specific extension in the replication config,
default this to Disabled to allow other sdks to be used to configure
replication rules.
Co-authored-by: Poorna Krishnamoorthy <poorna@minio.io>
2022-07-14 16:27:09 -07:00
LHHDZ
df911c9b9e
correct RefreshCall & UnlockCall of DefaultTimeouts ( #15288 )
2022-07-14 07:20:48 -07:00
Klaus Post
0149382cdc
Add padding to compressed+encrypted files ( #15282 )
...
Add up to 256 bytes of padding for compressed+encrypted files.
This will obscure the obvious cases of extremely compressible content
and leave a similar output size for a very wide variety of inputs.
This does *not* mean the compression ratio doesn't leak information
about the content, but the outcome space is much smaller,
so often *less* information is leaked.
2022-07-13 07:52:15 -07:00
