Commit Graph

9980 Commits

Author SHA1 Message Date
Cesar Celis Hernandez
8ec888d13d
feat: update binary once and push it to other servers (#15407) 2022-07-29 08:34:30 -07:00
Harshavardhana
916f274c83
choose starting concurrency based on number of local disks (#15428)
smaller setups may have less drives per server choosing
the concurrency based on number of local drives, and let
the MinIO server change the overall concurrency as
necessary.
2022-07-29 00:00:06 -07:00
Aditya Manthramurthy
7ac53c07af
fix: passing application configuration to console (#15409)
This is an update to MinIO server after swagger codegen related build
fixes added after issues introduced in 39fd7b0b3b
2022-07-28 18:30:24 -07:00
Harshavardhana
bc72e4226e
do not allow filesystem fallback in server download (#15429)
It is possible for anyone with admin access to relatively
to get any content of any random OS location by simply
providing the file with 'mc admin update alias/ /etc/passwd`.

Workaround is to disable 'admin:ServiceUpdate' action. Everyone
is advised to upgrade to this patch.

Thanks to @alevsk for finding this bug.
2022-07-28 17:44:21 -07:00
Poorna
5e0776e96a
replication: Include replica object versions for resync (#15427) 2022-07-28 13:43:02 -07:00
Anis Elleuch
2f1ef02d35
Do not update directory access time (#15426)
Most setups will have relatime it only updates the access time 
following a change in the directory.
2022-07-28 12:40:48 -07:00
Minio Trusted
db8442584e update helm chart v4.0.7 2022-07-27 20:54:38 -07:00
Naveen
d46cf50760
chore(deps): Included dependency review (#14958)
> Dependency Review GitHub Action in your repository to enforce dependency
> reviews on your pull requests.
> The action scans for vulnerable versions of dependencies introduced by package version
> changes in pull requests,
> and warns you about the associated security vulnerabilities.
> This gives you better visibility of what's changing in a pull request,
> and helps prevent vulnerabilities from being added to your repository.

https://docs.github.com/en/code-security/supply-chain-security/understanding-your-software-supply-chain/about-dependency-review#dependency-review-enforcement

Signed-off-by: Naveen <172697+naveensrinivasan@users.noreply.github.com>
2022-07-27 20:53:26 -07:00
Andrew Hibbert
0357121d17
Optionally run a command when completing jobs (#15017) 2022-07-27 20:51:16 -07:00
Harshavardhana
aff236e20e
fix: cluster healthcheck for single drive setups (#15415)
single drive setups must return '200 OK' if
drive is accessible, current master returns '503'
2022-07-27 16:46:34 -07:00
Harshavardhana
cbd70d26b5
optimize speedtest for smaller setups (#15414)
this has been observed in multiple environments
where the setups are small `speedtest` naturally
fails with default '10s' and the concurrency
of '32' is big for such clusters.

choose a smaller value i.e equal to number of
drives in such clusters and let 'autotune'
increase the concurrency instead.
2022-07-27 14:41:59 -07:00
Harshavardhana
5e763b71dc
use logger.LogOnce to reduce printing disconnection logs (#15408)
fixes #15334

- re-use net/url parsed value for http.Request{}
- remove gosimple, structcheck and unusued due to https://github.com/golangci/golangci-lint/issues/2649
- unwrapErrs upto leafErr to ensure that we store exactly the correct errors
2022-07-27 09:44:59 -07:00
Aditya Manthramurthy
7e4e7a66af
Remove internal usage of consoleAdmin (#15402)
"consoleAdmin" was used as the policy for root derived accounts, but this
lead to unexpected bugs when an administrator modified the consoleAdmin
policy

This change avoids evaluating a policy for root derived accounts as by
default no policy is mapped to the root user. If a session policy is
attached to a root derived account, it will be evaluated as expected.
2022-07-26 19:06:55 -07:00
Shireesh Anjal
906947a285
fix: typo in json key ClusterInfo DeploymentID (#15406)
deployement_id -> deployment_id
2022-07-26 19:05:33 -07:00
Minio Trusted
bfc70bc74e Update yaml files to latest version RELEASE.2022-07-26T00-53-03Z 2022-07-26 06:56:37 +00:00
jiuker
6b4f833a12
convert repeated error checks into single function in logger (#15387) 2022-07-25 17:53:03 -07:00
Poorna
426c902b87
site replication: fix healing of bucket deletes. (#15377)
This PR changes the handling of bucket deletes for site 
replicated setups to hold on to deleted bucket state until 
it syncs to all the clusters participating in site replication.
2022-07-25 17:51:32 -07:00
Anis Elleuch
e4b51235f8
upgrade: Split in two steps to ensure a stable retry (#15396)
Currently, if one server in a distributed setup fails to upgrade 
due to any reasons, it is not possible to upgrade again unless 
nodes are restarted.

To fix this, split the upgrade process into two steps :

- download the new binary on all servers
- If successful, overwrite the old binary with the new one
2022-07-25 17:49:47 -07:00
Harshavardhana
4c6498d726
move all CI/CD to go1.18 (#15401) 2022-07-25 15:27:20 -07:00
Eng Zer Jun
0a3b1ad4eb
test: use T.TempDir to create temporary test directory (#15400)
This commit replaces `ioutil.TempDir` with `t.TempDir` in tests. The
directory created by `t.TempDir` is automatically removed when the test
and all its subtests complete.

Prior to this commit, temporary directory created using `ioutil.TempDir`
needs to be removed manually by calling `os.RemoveAll`, which is omitted
in some tests. The error handling boilerplate e.g.
	defer func() {
		if err := os.RemoveAll(dir); err != nil {
			t.Fatal(err)
		}
	}
is also tedious, but `t.TempDir` handles this for us nicely.

Reference: https://pkg.go.dev/testing#T.TempDir

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-07-25 12:37:26 -07:00
Anis Elleuch
f23f442d33
Add cluster info to inspect/profiling archive (#15360)
Add cluster info to inspect and profiling archive.

In addition to the existing data generation for both inspect and profiling,
cluster.info file is added. This latter contains some info of the cluster.
The generation of cluster.info is is done as the last step and it can fail
if it exceed 10 seconds.
2022-07-25 09:11:35 -07:00
Minio Trusted
e465c3587b Update yaml files to latest version RELEASE.2022-07-24T17-09-31Z 2022-07-24 17:37:58 +00:00
Minio Trusted
7109b6d414 update console to v0.19.3 2022-07-24 10:09:31 -07:00
Minio Trusted
8c97f3e9bc update minio-go/v7 v7.0.32 2022-07-24 09:28:19 -07:00
Klaus Post
3795b2c8ba
Add compression scheme to header (#15395)
For easier debugging. We still do not return compressed size for security reasons.
2022-07-24 07:15:49 -07:00
Harshavardhana
7725425e05
fix: fork os.MkdirAll to optimize cases where parent exists (#15379)
a/b/c/d/ where `a/b/c/` exists results in additional syscalls
such as an Lstat() call to verify if the `a/b/c/` exists
and its a directory.

We do not need to do this on MinIO since the parent prefixes
if exist, we can simply return success without spending
additional syscalls.

Also this implementation attempts to simply use Access() calls
to avoid os.Stat() calls since the latter does memory allocation
for things we do not need to use.

Access() is simpler since we have a predictable structure on
the backend and we know exactly how our path structures are.
2022-07-24 00:43:11 -07:00
Minio Trusted
b2f4948bbe update helm v4.0.6 2022-07-23 20:34:14 -07:00
Minio Trusted
f802d2ba83 Update yaml files to latest version RELEASE.2022-07-24T01-54-52Z 2022-07-24 02:31:24 +00:00
Daniel Valdivia
ce8548a1a2
Console v0.19.2 (#15390)
Signed-off-by: Daniel Valdivia <18384552+dvaldivia@users.noreply.github.com>
2022-07-23 18:54:52 -07:00
Minio Trusted
490dec981a update go mod tidy -compat=1.17 2022-07-22 15:31:16 -07:00
Aditya Manthramurthy
39fd7b0b3b
Pass multiple IDP config to console (#15270)
This change passes multiple IDP config via a struct 
rather than env variables.
2022-07-22 15:28:02 -07:00
Taran Pelkey
e83930333b
Allow DelKVS to delete specific sub-system fields. (#15354) 2022-07-22 14:48:23 -07:00
Harshavardhana
b0d70a0e5e
support additional claim info in Auditing STS calls (#15381)
Bonus: Adds a missing AuditLog from AssumeRoleWithCertificate API

Fixes #9529
2022-07-22 11:12:03 -07:00
Denis Krivenko
ff5a5c1ee0
helm: Add runtimeClassName support to vanilla helm chart (#15385) 2022-07-22 10:25:41 -07:00
Mathieu Parent
290a53d735
helm: Use existingSecretKey as in the user example (#15386) 2022-07-22 10:25:22 -07:00
Aditya Manthramurthy
2393a13f86
Allow site replication config with multiple IDPs (#15361)
Fixes a bug that did not let site replication be configured when
multiple IDPs are configured.
2022-07-21 19:52:23 -07:00
Poorna
7d8c8de827
single drive: Remove bucket metadata on DeleteBucket (#15378)
from disk and in-memory map
2022-07-21 19:51:53 -07:00
jiuker
3faef829c5
expect full quorum for writing 'format.json' everywhere (#15362) 2022-07-21 18:04:17 -07:00
Poorna
7560fb6f9a
save IAM export assets relative at a folder prefix (#15355) 2022-07-21 17:51:33 -07:00
Harshavardhana
2fddcc6a11
upgrade mqtt library to v1.4.1 (#15366)
mainly to address some connect()/reconnect() packet
exhaustion issues, that were found in some deployments.
2022-07-21 17:49:28 -07:00
Klaus Post
69bf39f42e
fix: make complete multipart uploads faster encrypted/compressed backends (#15375)
- Only fetch the parts we need and abort as soon as one is missing.
- Only fetch the number of parts requested by "ListObjectParts".
2022-07-21 16:47:58 -07:00
MohammadReza
f4d5c861f3
update grafana dashboard (#15357) 2022-07-21 15:17:44 -07:00
Minio Trusted
564a0afae1 Revert "tests: Add context cancelation (#15374)"
This reverts commit 1e332f0eb1.

Reverting this as tests are failing randomly.
2022-07-21 13:58:56 -07:00
Klaus Post
1e332f0eb1
tests: Add context cancelation (#15374)
A huge number of goroutines would build up from various monitors

When creating test filesystems provide a context so they can shut down when no longer needed.
2022-07-21 11:52:18 -07:00
Poorna
cab8d3d568
feat: add API to return list of objects waiting to be replicated (#15091) 2022-07-21 11:05:44 -07:00
Klaus Post
be8c4cb24a
fix: support multiple validateAdminReq actions (#15372)
handle multiple validateAdminReq actions and remove duplicate error responses.
2022-07-21 10:26:59 -07:00
Harshavardhana
65166e4ce4
fix: readQuorum calculation when defaultParityCount is 0 (#15363)
when parity is '0' the readQuorum must be equal
to the number of data disks.
2022-07-21 07:25:54 -07:00
Harshavardhana
8249cd4406
fix: allow payload verification error to be returned (#15364)
without reading the reader the error is ignored
by the custom unmarshaller written by ObjectLegalHold
data structure.
2022-07-21 01:24:03 -07:00
Harshavardhana
c6ecaf68ed update CREDITS with latest dependencies 2022-07-21 00:49:38 -07:00
Harshavardhana
d3f89fa6e3
remove unnecessary logs in IAM store (#15356) 2022-07-20 08:19:12 -07:00