Commit Graph

9807 Commits

Author SHA1 Message Date
Anis Elleuch e93867488b
actively cancel listIAMConfigItems to avoid goroutine leak (#15471)
listConfigItems creates a goroutine but sometimes callers will
exit without properly asking listAllIAMConfigItems() to stop sending
results, hence a goroutine leak.

Create a new context and cancel it for each listAllIAMConfigItems
call.
2022-08-04 13:20:43 -07:00
Minio Trusted c08790edd2 upgrade helm v4.0.10 2022-08-04 09:09:22 -07:00
Kourosh Tafreshi a46baddbc4
Add OIDC to the HelmChart (#15469) 2022-08-04 09:07:51 -07:00
Harshavardhana 3bd9615d0e
fix: log if there is readDir() failure with ListBuckets (#15461)
This is actionable and must be logged.

Bonus: also honor umask by using 0o666 for all Open() syscalls.
2022-08-04 07:23:05 -07:00
Minio Trusted 2871cb5775 upgrade helm v4.0.9 2022-08-02 23:10:44 -07:00
Harshavardhana a6e0ec4e6f
Add support converting non-inlined to inlined (#15444)
This is a feature to allow for inode compaction on
large clusters that use a lot of small files spread
across a large heirarchy.
2022-08-02 23:10:22 -07:00
Minio Trusted e956369c4e Update yaml files to latest version RELEASE.2022-08-02T23-59-16Z 2022-08-03 01:45:42 +00:00
Minio Trusted 76f950c663 upgrade to minio-go/v7 v7.0.34 2022-08-02 16:59:16 -07:00
Andreas Auernhammer d774a3309b
kes: automatically reload KES client certificate (#15450)
This commit adds support for automatically reloading
the MinIO client certificate for authentication to KES.

The client certificate will now be reloaded:
 - when the private key / certificate file changes
 - when a SIGHUP signal is received
 - every 15 minutes

Fixes #14869

Signed-off-by: Andreas Auernhammer <hi@aead.dev>
2022-08-02 16:58:09 -07:00
Anis Elleuch b3edb25377
bloom: healObject to mark a path dirty only for dangling objects (#15458)
The path is marked dirty automatically when healObject() is called, which is
wrong. HealObject() is called during self-healing and this will lead to
an increase in the false positive result of the bloom filter.

Also move NSUpdated() from renameData() and call it directly in
CompleteMultipart and PutObject, this is not a functional change but
it will make it less prone to errors in the future.
2022-08-02 16:57:39 -07:00
Harshavardhana 026b87e39b
do not crash for unwrapErrs return `nil` (#15456)
fixes #15454
2022-08-02 15:10:11 -07:00
Harshavardhana 53a816b17a
fix: readdir fallback on root of the drive (#15457)
fixes #15452
2022-08-02 14:57:36 -07:00
Harshavardhana 043aaa792d
fix: intrument os.OpenFile differently for Reads and Writes (#15449)
allows us to trace latency for READs or WRITEs
2022-08-01 13:22:43 -07:00
dorman aad9cb208a
helm: modify user secret volumes mount path name (#15443) 2022-08-01 12:28:04 -07:00
Minio Trusted edf081c6a2 update minio-go v7.0.33 2022-08-01 10:44:35 -07:00
Harshavardhana fd349103e8
fix: allow P-384/P-512 constant time implementation (#15445)
since go1.18.x P-384/P-512 are now constant time
implementations, enable them.
2022-08-01 09:27:16 -07:00
Anis Elleuch 10b49eb4fb
Fix resetting a config with a non default target name (#15448)
mc admin config reset <alias> notify_webhook:something was not working
properly.

The reason is that GetSubSys() was not calculating the target
name properly because it is quitting early when the number of config
inputs ('notify_webhook:something' in this case) is equal to 1.

This commit will make the code calculates always calculate the target
name if found.
2022-08-01 07:52:23 -07:00
Anis Elleuch 3856d078d2
fix: set 20000 as maximum parallel event calls (#15435)
This is needed to avoid consuming a lot of goroutines when a target is
very slow or there is a bug in a target library.
2022-07-30 12:12:33 -07:00
Minio Trusted 6b4cb35f4f Update yaml files to latest version RELEASE.2022-07-30T05-21-40Z 2022-07-30 05:50:06 +00:00
Shireesh Anjal e6eab2091f
fix: Incorrect ServersCount in cluster.info (#15431)
The `ServersCount` field in cluster.info is expected to contain the
number of nodes, and not number of endpoints.
2022-07-29 22:21:40 -07:00
Harshavardhana 3cdb609cca
allow root users to return appropriate policy in AccountInfo (#15437)
fixes #15436

This fixes a regression caused after the removal of "consoleAdmin"
policy usage for 'root users' in PR #15402
2022-07-29 20:58:03 -07:00
Minio Trusted d6a7f62ff5 update helm v4.0.8 2022-07-29 16:39:54 -07:00
Minio Trusted 72f170f5d2 update console to v0.19.4 2022-07-29 15:07:53 -07:00
Minio Trusted 824d52a82b Update yaml files to latest version RELEASE.2022-07-29T19-40-48Z 2022-07-29 22:06:57 +00:00
Minio Trusted 067ebab9d8 update object-locking docs and word them appropriately 2022-07-29 12:40:48 -07:00
Anis Elleuch 6be6c0d2e3
Update kafka library to v1.35.0 (#15434)
There is a known rare issue in the current version 1.30.0 described here
https://github.com/Shopify/sarama/issues/2241.

Update the library to 1.35.0

Bonus:  update shirou/gopsutil v3.22.5 to v3.22.6 to fix a compilation
error for OpenBSD
2022-07-29 11:34:45 -07:00
Harshavardhana aa874010e2
fix: regression in resolving the right versions (#15430)
fix: regression in resolving right versions

commit d480022711 caused a regression in real
resolver, by picking up incorrect versionID.
2022-07-29 10:03:53 -07:00
Cesar Celis Hernandez 8ec888d13d
feat: update binary once and push it to other servers (#15407) 2022-07-29 08:34:30 -07:00
Harshavardhana 916f274c83
choose starting concurrency based on number of local disks (#15428)
smaller setups may have less drives per server choosing
the concurrency based on number of local drives, and let
the MinIO server change the overall concurrency as
necessary.
2022-07-29 00:00:06 -07:00
Aditya Manthramurthy 7ac53c07af
fix: passing application configuration to console (#15409)
This is an update to MinIO server after swagger codegen related build
fixes added after issues introduced in 39fd7b0b3b
2022-07-28 18:30:24 -07:00
Harshavardhana bc72e4226e
do not allow filesystem fallback in server download (#15429)
It is possible for anyone with admin access to relatively
to get any content of any random OS location by simply
providing the file with 'mc admin update alias/ /etc/passwd`.

Workaround is to disable 'admin:ServiceUpdate' action. Everyone
is advised to upgrade to this patch.

Thanks to @alevsk for finding this bug.
2022-07-28 17:44:21 -07:00
Poorna 5e0776e96a
replication: Include replica object versions for resync (#15427) 2022-07-28 13:43:02 -07:00
Anis Elleuch 2f1ef02d35
Do not update directory access time (#15426)
Most setups will have relatime it only updates the access time 
following a change in the directory.
2022-07-28 12:40:48 -07:00
Minio Trusted db8442584e update helm chart v4.0.7 2022-07-27 20:54:38 -07:00
Naveen d46cf50760
chore(deps): Included dependency review (#14958)
> Dependency Review GitHub Action in your repository to enforce dependency
> reviews on your pull requests.
> The action scans for vulnerable versions of dependencies introduced by package version
> changes in pull requests,
> and warns you about the associated security vulnerabilities.
> This gives you better visibility of what's changing in a pull request,
> and helps prevent vulnerabilities from being added to your repository.

https://docs.github.com/en/code-security/supply-chain-security/understanding-your-software-supply-chain/about-dependency-review#dependency-review-enforcement

Signed-off-by: Naveen <172697+naveensrinivasan@users.noreply.github.com>
2022-07-27 20:53:26 -07:00
Andrew Hibbert 0357121d17
Optionally run a command when completing jobs (#15017) 2022-07-27 20:51:16 -07:00
Harshavardhana aff236e20e
fix: cluster healthcheck for single drive setups (#15415)
single drive setups must return '200 OK' if
drive is accessible, current master returns '503'
2022-07-27 16:46:34 -07:00
Harshavardhana cbd70d26b5
optimize speedtest for smaller setups (#15414)
this has been observed in multiple environments
where the setups are small `speedtest` naturally
fails with default '10s' and the concurrency
of '32' is big for such clusters.

choose a smaller value i.e equal to number of
drives in such clusters and let 'autotune'
increase the concurrency instead.
2022-07-27 14:41:59 -07:00
Harshavardhana 5e763b71dc
use logger.LogOnce to reduce printing disconnection logs (#15408)
fixes #15334

- re-use net/url parsed value for http.Request{}
- remove gosimple, structcheck and unusued due to https://github.com/golangci/golangci-lint/issues/2649
- unwrapErrs upto leafErr to ensure that we store exactly the correct errors
2022-07-27 09:44:59 -07:00
Aditya Manthramurthy 7e4e7a66af
Remove internal usage of consoleAdmin (#15402)
"consoleAdmin" was used as the policy for root derived accounts, but this
lead to unexpected bugs when an administrator modified the consoleAdmin
policy

This change avoids evaluating a policy for root derived accounts as by
default no policy is mapped to the root user. If a session policy is
attached to a root derived account, it will be evaluated as expected.
2022-07-26 19:06:55 -07:00
Shireesh Anjal 906947a285
fix: typo in json key ClusterInfo DeploymentID (#15406)
deployement_id -> deployment_id
2022-07-26 19:05:33 -07:00
Minio Trusted bfc70bc74e Update yaml files to latest version RELEASE.2022-07-26T00-53-03Z 2022-07-26 06:56:37 +00:00
jiuker 6b4f833a12
convert repeated error checks into single function in logger (#15387) 2022-07-25 17:53:03 -07:00
Poorna 426c902b87
site replication: fix healing of bucket deletes. (#15377)
This PR changes the handling of bucket deletes for site 
replicated setups to hold on to deleted bucket state until 
it syncs to all the clusters participating in site replication.
2022-07-25 17:51:32 -07:00
Anis Elleuch e4b51235f8
upgrade: Split in two steps to ensure a stable retry (#15396)
Currently, if one server in a distributed setup fails to upgrade 
due to any reasons, it is not possible to upgrade again unless 
nodes are restarted.

To fix this, split the upgrade process into two steps :

- download the new binary on all servers
- If successful, overwrite the old binary with the new one
2022-07-25 17:49:47 -07:00
Harshavardhana 4c6498d726
move all CI/CD to go1.18 (#15401) 2022-07-25 15:27:20 -07:00
Eng Zer Jun 0a3b1ad4eb
test: use `T.TempDir` to create temporary test directory (#15400)
This commit replaces `ioutil.TempDir` with `t.TempDir` in tests. The
directory created by `t.TempDir` is automatically removed when the test
and all its subtests complete.

Prior to this commit, temporary directory created using `ioutil.TempDir`
needs to be removed manually by calling `os.RemoveAll`, which is omitted
in some tests. The error handling boilerplate e.g.
	defer func() {
		if err := os.RemoveAll(dir); err != nil {
			t.Fatal(err)
		}
	}
is also tedious, but `t.TempDir` handles this for us nicely.

Reference: https://pkg.go.dev/testing#T.TempDir

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-07-25 12:37:26 -07:00
Anis Elleuch f23f442d33
Add cluster info to inspect/profiling archive (#15360)
Add cluster info to inspect and profiling archive.

In addition to the existing data generation for both inspect and profiling,
cluster.info file is added. This latter contains some info of the cluster.
The generation of cluster.info is is done as the last step and it can fail
if it exceed 10 seconds.
2022-07-25 09:11:35 -07:00
Minio Trusted e465c3587b Update yaml files to latest version RELEASE.2022-07-24T17-09-31Z 2022-07-24 17:37:58 +00:00
Minio Trusted 7109b6d414 update console to v0.19.3 2022-07-24 10:09:31 -07:00