Commit Graph

9807 Commits

Author SHA1 Message Date
Minio Trusted 8c97f3e9bc update minio-go/v7 v7.0.32 2022-07-24 09:28:19 -07:00
Klaus Post 3795b2c8ba
Add compression scheme to header (#15395)
For easier debugging. We still do not return compressed size for security reasons.
2022-07-24 07:15:49 -07:00
Harshavardhana 7725425e05
fix: fork os.MkdirAll to optimize cases where parent exists (#15379)
a/b/c/d/ where `a/b/c/` exists results in additional syscalls
such as an Lstat() call to verify if the `a/b/c/` exists
and its a directory.

We do not need to do this on MinIO since the parent prefixes
if exist, we can simply return success without spending
additional syscalls.

Also this implementation attempts to simply use Access() calls
to avoid os.Stat() calls since the latter does memory allocation
for things we do not need to use.

Access() is simpler since we have a predictable structure on
the backend and we know exactly how our path structures are.
2022-07-24 00:43:11 -07:00
Minio Trusted b2f4948bbe update helm v4.0.6 2022-07-23 20:34:14 -07:00
Minio Trusted f802d2ba83 Update yaml files to latest version RELEASE.2022-07-24T01-54-52Z 2022-07-24 02:31:24 +00:00
Daniel Valdivia ce8548a1a2
Console v0.19.2 (#15390)
Signed-off-by: Daniel Valdivia <18384552+dvaldivia@users.noreply.github.com>
2022-07-23 18:54:52 -07:00
Minio Trusted 490dec981a update go mod tidy -compat=1.17 2022-07-22 15:31:16 -07:00
Aditya Manthramurthy 39fd7b0b3b
Pass multiple IDP config to console (#15270)
This change passes multiple IDP config via a struct 
rather than env variables.
2022-07-22 15:28:02 -07:00
Taran Pelkey e83930333b
Allow DelKVS to delete specific sub-system fields. (#15354) 2022-07-22 14:48:23 -07:00
Harshavardhana b0d70a0e5e
support additional claim info in Auditing STS calls (#15381)
Bonus: Adds a missing AuditLog from AssumeRoleWithCertificate API

Fixes #9529
2022-07-22 11:12:03 -07:00
Denis Krivenko ff5a5c1ee0
helm: Add runtimeClassName support to vanilla helm chart (#15385) 2022-07-22 10:25:41 -07:00
Mathieu Parent 290a53d735
helm: Use existingSecretKey as in the user example (#15386) 2022-07-22 10:25:22 -07:00
Aditya Manthramurthy 2393a13f86
Allow site replication config with multiple IDPs (#15361)
Fixes a bug that did not let site replication be configured when
multiple IDPs are configured.
2022-07-21 19:52:23 -07:00
Poorna 7d8c8de827
single drive: Remove bucket metadata on DeleteBucket (#15378)
from disk and in-memory map
2022-07-21 19:51:53 -07:00
jiuker 3faef829c5
expect full quorum for writing 'format.json' everywhere (#15362) 2022-07-21 18:04:17 -07:00
Poorna 7560fb6f9a
save IAM export assets relative at a folder prefix (#15355) 2022-07-21 17:51:33 -07:00
Harshavardhana 2fddcc6a11
upgrade mqtt library to v1.4.1 (#15366)
mainly to address some connect()/reconnect() packet
exhaustion issues, that were found in some deployments.
2022-07-21 17:49:28 -07:00
Klaus Post 69bf39f42e
fix: make complete multipart uploads faster encrypted/compressed backends (#15375)
- Only fetch the parts we need and abort as soon as one is missing.
- Only fetch the number of parts requested by "ListObjectParts".
2022-07-21 16:47:58 -07:00
MohammadReza f4d5c861f3
update grafana dashboard (#15357) 2022-07-21 15:17:44 -07:00
Minio Trusted 564a0afae1 Revert "tests: Add context cancelation (#15374)"
This reverts commit 1e332f0eb1.

Reverting this as tests are failing randomly.
2022-07-21 13:58:56 -07:00
Klaus Post 1e332f0eb1
tests: Add context cancelation (#15374)
A huge number of goroutines would build up from various monitors

When creating test filesystems provide a context so they can shut down when no longer needed.
2022-07-21 11:52:18 -07:00
Poorna cab8d3d568
feat: add API to return list of objects waiting to be replicated (#15091) 2022-07-21 11:05:44 -07:00
Klaus Post be8c4cb24a
fix: support multiple validateAdminReq actions (#15372)
handle multiple validateAdminReq actions and remove duplicate error responses.
2022-07-21 10:26:59 -07:00
Harshavardhana 65166e4ce4
fix: readQuorum calculation when defaultParityCount is 0 (#15363)
when parity is '0' the readQuorum must be equal
to the number of data disks.
2022-07-21 07:25:54 -07:00
Harshavardhana 8249cd4406
fix: allow payload verification error to be returned (#15364)
without reading the reader the error is ignored
by the custom unmarshaller written by ObjectLegalHold
data structure.
2022-07-21 01:24:03 -07:00
Harshavardhana c6ecaf68ed update CREDITS with latest dependencies 2022-07-21 00:49:38 -07:00
Harshavardhana d3f89fa6e3
remove unnecessary logs in IAM store (#15356) 2022-07-20 08:19:12 -07:00
Harshavardhana ce8397f7d9
use partInfo only for intermediate part.x.meta (#15353) 2022-07-19 18:56:24 -07:00
Klaus Post cae9aeca00
fix: reused field crash in PartIndices (#15351)
`PartIndices` may be set if xlMetaV2Version is reused.

Clear before unmarshaling and add sanity check when reading.
2022-07-19 16:49:46 -07:00
Klaus Post f939d1c183
Independent Multipart Uploads (#15346)
Do completely independent multipart uploads.

In distributed mode, a lock was held to merge each multipart 
upload as it was added. This lock was highly contested and 
retries are expensive (timewise) in distributed mode.

Instead, each part adds its metadata information uniquely. 
This eliminates the per object lock required for each to merge.
The metadata is read back and merged by "CompleteMultipartUpload" 
without locks when constructing final object.

Co-authored-by: Harshavardhana <harsha@minio.io>
2022-07-19 08:35:29 -07:00
Andreas Auernhammer 242d06274a
kms: add `context.Context` to KMS API calls (#15327)
This commit adds a `context.Context` to the
the KMS `{Stat, CreateKey, GenerateKey}` API
calls.

The context will be used to terminate external calls
as soon as the client requests gets canceled.

A follow-up PR will add a `context.Context` to
the remaining `DecryptKey` API call.

Signed-off-by: Andreas Auernhammer <hi@aead.dev>
2022-07-18 18:54:27 -07:00
Poorna 957e3ed729
export IAM: include site replicator svcacct (#15339) 2022-07-18 17:38:53 -07:00
Jeff Haynie ed02ee4ef4
fix: issue when a Helm create user job returns more than once (#15321) 2022-07-18 12:09:44 -07:00
Daniel Valdivia ba9691a0ad
Console v0.19.1 (#15338)
Signed-off-by: Daniel Valdivia <18384552+dvaldivia@users.noreply.github.com>
2022-07-18 11:45:20 -07:00
Minio Trusted e7eb94de6b Update yaml files to latest version RELEASE.2022-07-17T15-43-14Z 2022-07-17 22:06:11 +00:00
Harshavardhana b6eb8dff64
Add decommission compression+encryption enabled tests (#15322)
update compression environment variables to follow
the expected sub-system style, however support fallback
mode.
2022-07-17 08:43:14 -07:00
Harshavardhana 7da9e3a6f8
support encrypted/compressed objects properly during decommission (#15320)
fixes #15314
2022-07-16 19:35:24 -07:00
Anis Elleuch 876970baea
Exclude upload-ids with incomplete part upload in multipart listing (#15318)
Uploading a part object can leave an inconsistent state inside
.minio.sys/multipart where data are uploaded but xl.meta is not
committed yet.

Do not list upload-ids that have this state in the multipart listing.
2022-07-16 13:25:58 -07:00
LHHDZ e68e76e143
fix: data race, which caused tests execution to fail (#15313) 2022-07-16 07:57:55 -07:00
Mathieu Parent 2bc7ca2d34
helm: add annotations for ServiceMonitor (#15020) 2022-07-16 01:04:27 -07:00
Minio Trusted e94eb9af10 update helm v4.0.5
Signed-off-by: Minio Trusted <trusted@minio.io>
2022-07-15 23:42:56 -07:00
Jon Kartago Lamida 3018b21ab8
fix: failure to createUser used by make-user-job helm chart (#15293) 2022-07-15 23:22:21 -07:00
Steven Kriegler 0b605c3383
Allow topologySpreadConstraints configuration (#14684)
The default replica value is 16 (right now) which can lead to massive
resource consumption on one node in smaller clusters. The idea for this
addition is to allow users to specify how the pods (replicas) are being
spread across the cluster. It gives more control over this Helm Release
in smaller clusters where most worker nodes have taints.

As this Kubernetes feature exists since Kubernetes 1.19 and is only
useful for a replica count > 1, this was taken into account.
2022-07-15 21:05:38 -07:00
Harshavardhana e7ac1ea54c
allow decommission to continue when healing (#15312)
Bonus:

- heal buckets in-case during startup the new
  pools have bucket missing.
2022-07-15 21:03:23 -07:00
Harshavardhana 5ac6d91525
support 'admin update' for hotfix versions (#15308)
hotfixed versions are rejected as invalid,
allow `mc admin update` from hotfix repos.
2022-07-15 16:00:34 -07:00
Harshavardhana 1cd6713e24
copy query values before update to preserve the expected keys (#15310)
in success_action_redirect we were missing required
query params as per S3 spec - updated tests.
2022-07-15 15:04:48 -07:00
Harshavardhana 785b429737
add reconnect duration allows for verifying disconnect intervals (#15306) 2022-07-15 14:41:24 -07:00
Minio Trusted 4aecd8d039 Update yaml files to latest version RELEASE.2022-07-15T03-44-22Z 2022-07-15 06:05:11 +00:00
Harshavardhana 1b339ea062
allow force delete on decom pool (#15302)
Bonus:

- skip suspended pool from being
  considered for multipart uploads

- add more context for decomErrors()
2022-07-14 20:44:22 -07:00
Harshavardhana 236ef03dbd
fix: skip objects expired via lifecycle rules during decommission (#15300) 2022-07-14 16:47:09 -07:00