1
0
mirror of https://github.com/minio/minio.git synced 2025-01-21 11:43:16 -05:00

570 Commits

Author SHA1 Message Date
Javier Adriel
3307aa1260
Implement KMS handlers () 2022-10-04 10:05:09 -07:00
Harshavardhana
f696a221af
allow tagging policy condition for GetObject () 2022-10-02 12:29:29 -07:00
Anis Elleuch
6287e8c571
fix: race when accessing REST TCP dial values () 2022-09-29 09:27:58 -07:00
Anis Elleuch
f69a98ce49
fix: loading Audit kafka configuration loading () 2022-09-29 08:35:08 -07:00
Daryl White
d44f3526dc
Update links to documentation site () 2022-09-28 21:28:45 -07:00
Harshavardhana
41b633f5ea
support tagging based policy conditions () 2022-09-28 11:25:46 -07:00
Anis Elleuch
86bb48792c
non-blocking initialization of bucket target notifications () 2022-09-27 17:23:28 -07:00
Harshavardhana
94dbb4a427
fix: generalize SC config and also skip healing sub-sys under SD () 2022-09-26 09:04:54 -07:00
Anis Elleuch
048a46ec2a
Add RPC tcp timeout/errs and AVG duration to prometheus () 2022-09-26 09:04:26 -07:00
Harshavardhana
877bd95fa3
remove unused package internal/smart () 2022-09-25 08:44:53 -07:00
Poorna
8ea6fb368d
Add auto configuration of replication workers () 2022-09-24 16:20:28 -07:00
Poorna
5fd5ddea23
relax retention date validation on replication target () 2022-09-23 21:19:03 -07:00
Anis Elleuch
20c89ebbb3
freeze before exit when _MINIO_DEBUG_NO_EXIT is defined ()
this is to ensure keep k8s pods running, when they reach a "crashloop" stage
2022-09-22 11:57:27 -07:00
Krishnan Parthasarathi
6f56ba80b3
lifecycle: Assign unique id to rules with empty id () 2022-09-22 10:51:54 -07:00
Klaus Post
ff12080ff5
Remove deprecated io/ioutil () 2022-09-19 11:05:16 -07:00
Javier Adriel
0b6175b742
Implement KMS methods for keys, policies and identities () 2022-09-19 11:04:40 -07:00
Klaus Post
c22f3ca7a8
fix: S3 Select CSV -> JSON with variable field count ()
When there are fewer fields than expected, output fewer fields.
2022-09-12 17:00:59 -07:00
Anis Elleuch
4a92134235
prometheus: track errors during REST read/write calls ()
minio_inter_node_traffic_errors_total currently does not track
requests body write/read errors of internode REST communications.

This commit fixes this by wrapping resp.Body.
2022-09-12 12:40:51 -07:00
Poorna
6b9fd256e1
Persist in-memory replication stats to disk ()
to avoid relying on scanner-calculated replication metrics.
This will improve the accuracy of the replication stats reported.

This PR also adds on to  by handing replication
traffic that could not be queued by available workers to the 
MRF queue so that entries in `PENDING` status are healed faster.
2022-09-12 12:40:02 -07:00
Harshavardhana
228c6686f8
allow non-standards fallback for all http.TimeFormats ()
fixes 
2022-09-07 07:24:54 -07:00
Krishnan Parthasarathi
5ce1448049
audit: account for response headers separately () 2022-09-01 12:51:04 -07:00
Klaus Post
8e4a45ec41
fix: encrypt checksums in metadata () 2022-08-31 08:13:23 -07:00
Abirdcfly
d4e0f13bb3
chore: remove duplicate word in comments ()
Signed-off-by: Abirdcfly <fp544037857@gmail.com>

Signed-off-by: Abirdcfly <fp544037857@gmail.com>
2022-08-30 08:26:43 -07:00
Anis Elleuch
1f28a3bb80
Avoid messages from go test output ()
A lot of warning messages are printed in CI/CD failures generated by go
test. Avoid that by requiring at least Error level for logging when
doing go test.
2022-08-30 08:23:40 -07:00
Klaus Post
a9f1ad7924
Add extended checksum support () 2022-08-29 16:57:16 -07:00
Harshavardhana
97376f6e8f
improve performance for inlined data ()
inlined data often is bigger than the allowed
O_DIRECT alignment, so potentially we can write
'xl.meta' without O_DSYNC instead we can rely on
O_DIRECT + fdatasync() instead.

This PR allows O_DIRECT on inlined data that
would gain the benefits of performing O_DIRECT,
eventually performing an fdatasync() at the end.

Performance boost can be observed here for small
objects < 128KiB. The performance boost is mainly
seen on HDD, and marginal on NVMe setups.
2022-08-29 11:19:29 -07:00
Harshavardhana
433b6fa8fe
upgrade golang-lint to the latest () 2022-08-26 12:52:29 -07:00
Krishnan Parthasarathi
99fbfe2421
Add concurrency to healing objects on a fresh disk () 2022-08-25 13:07:15 -07:00
Aditya Manthramurthy
18dffb26e7
Allow querying a single target in config get API () 2022-08-25 00:17:05 -07:00
Aditya Manthramurthy
c54c13831a
fix: error types in internal config package () 2022-08-20 15:11:49 -07:00
Harshavardhana
ae4ee95d25
change default lock retry interval to 50ms ()
competing calls on the same object on versioned bucket
mutating calls on the same object may unexpected have
higher delays.

This can be reproduced with a replicated bucket
overwriting the same object writes, deletes repeatedly.

For longer locks like scanner keep the 1sec interval
2022-08-19 16:21:05 -07:00
Aditya Manthramurthy
21831b3fe2
fix: cleanup config KV parsing using madmin helpers () 2022-08-18 15:55:17 -07:00
Aditya Manthramurthy
701a82642b
Ensure subnet+callhome configs are single sub-sys () 2022-08-16 17:46:46 -07:00
Harshavardhana
48640b1de2
fix: trim arn:aws:kms from incoming SSE aws-kms-key-id () 2022-08-16 11:28:30 -07:00
Anis Elleuch
5682685c80
Introduce disk io stats metrics () 2022-08-16 07:13:49 -07:00
Aditya Manthramurthy
ee2afcf70b
Move some config constants to madmin () 2022-08-13 14:50:16 -07:00
Aditya Manthramurthy
9986e103cf
Fix env var output in config get/export APIs ()
Fix a bug where env vars are not output when the config for the
subsystem is specified solely via env vars.
2022-08-13 10:39:01 -07:00
Shireesh Anjal
316c492842
Upgrade madmin-go to latest version (v1.4.15) () 2022-08-10 07:36:13 -07:00
Harshavardhana
a406bb0288
restrict number of disks used for scanning buckets upto GOMAXPROCS ()
control scanner parallelism to avoid higher CPU
usage on nodes that have more drives but an old CPU.
2022-08-08 16:16:44 -07:00
Harshavardhana
1823ab6808
LDAP/OpenID must be initialized IAM Init() ()
This allows for LDAP/OpenID to be non-blocking,
allowing for unreachable Identity targets to be
initialized in IAM.
2022-08-08 16:16:27 -07:00
Aditya Manthramurthy
3d94c38ec4
Add env variables to configuration APIs output ()
Config export and config get APIs now include environment 
variables set on the server
2022-08-04 22:21:52 -07:00
ebozduman
b57e7321e7
Replaces 'disk'=>'drive' visible to end user () 2022-08-04 16:10:08 -07:00
Andreas Auernhammer
d774a3309b
kes: automatically reload KES client certificate ()
This commit adds support for automatically reloading
the MinIO client certificate for authentication to KES.

The client certificate will now be reloaded:
 - when the private key / certificate file changes
 - when a SIGHUP signal is received
 - every 15 minutes

Fixes 

Signed-off-by: Andreas Auernhammer <hi@aead.dev>
2022-08-02 16:58:09 -07:00
Harshavardhana
026b87e39b
do not crash for unwrapErrs return nil ()
fixes 
2022-08-02 15:10:11 -07:00
Harshavardhana
fd349103e8
fix: allow P-384/P-512 constant time implementation ()
since go1.18.x P-384/P-512 are now constant time
implementations, enable them.
2022-08-01 09:27:16 -07:00
Anis Elleuch
10b49eb4fb
Fix resetting a config with a non default target name ()
mc admin config reset <alias> notify_webhook:something was not working
properly.

The reason is that GetSubSys() was not calculating the target
name properly because it is quitting early when the number of config
inputs ('notify_webhook:something' in this case) is equal to 1.

This commit will make the code calculates always calculate the target
name if found.
2022-08-01 07:52:23 -07:00
Anis Elleuch
3856d078d2
fix: set 20000 as maximum parallel event calls ()
This is needed to avoid consuming a lot of goroutines when a target is
very slow or there is a bug in a target library.
2022-07-30 12:12:33 -07:00
Harshavardhana
5e763b71dc
use logger.LogOnce to reduce printing disconnection logs ()
fixes 

- re-use net/url parsed value for http.Request{}
- remove gosimple, structcheck and unusued due to https://github.com/golangci/golangci-lint/issues/2649
- unwrapErrs upto leafErr to ensure that we store exactly the correct errors
2022-07-27 09:44:59 -07:00
jiuker
6b4f833a12
convert repeated error checks into single function in logger () 2022-07-25 17:53:03 -07:00
Eng Zer Jun
0a3b1ad4eb
test: use T.TempDir to create temporary test directory ()
This commit replaces `ioutil.TempDir` with `t.TempDir` in tests. The
directory created by `t.TempDir` is automatically removed when the test
and all its subtests complete.

Prior to this commit, temporary directory created using `ioutil.TempDir`
needs to be removed manually by calling `os.RemoveAll`, which is omitted
in some tests. The error handling boilerplate e.g.
	defer func() {
		if err := os.RemoveAll(dir); err != nil {
			t.Fatal(err)
		}
	}
is also tedious, but `t.TempDir` handles this for us nicely.

Reference: https://pkg.go.dev/testing#T.TempDir

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-07-25 12:37:26 -07:00