Commit Graph

12040 Commits

Author SHA1 Message Date
Harshavardhana
91f91d8f47
fix: a regression in IAM policy reload routine() (#19421)
all policy reloading is broken since last release since

48deccdc40

fixes #19417
2024-04-05 14:26:41 -07:00
Harshavardhana
a207bd6790
turn-off Nlink readdir() optimization for NFS/CIFS (#19420)
fixes #19418
fixes #19416
2024-04-05 08:17:08 -07:00
Harshavardhana
96d226c0b1
remove frivolous log about abort-multipart failure in replication (#19413) 2024-04-05 04:39:55 -07:00
Krishnan Parthasarathi
a86d98826d
Set object's original modTime when being restored (#19414)
Set object's modTime when being restored

restored here refers to making a temporary local copy in the hot tier
for a tiered object using the RestoreObject API
2024-04-05 04:39:31 -07:00
Harshavardhana
1bb670ecba
use new generics based LRU from hashicorp (#19409)
we have been using an LRU caching for internode
auth tokens, migrate to using a typed implementation
and also do not cache auth tokens when its an error.
2024-04-04 11:58:48 -07:00
Aditya Manthramurthy
c9e9a8e2b9
fix: ldap: use validated base DNs (#19406)
This fixes a regression from #19358 which prevents policy mappings
created in the latest release from being displayed in policy entity
listing APIs.

This is due to the possibility that the base DNs in the LDAP config are
not in a normalized form and #19358 introduced normalized of mapping
keys (user DNs and group DNs). When listing, we check if the policy
mappings are on entities that parse as valid DNs that are descendants of
the base DNs in the config.

Test added that demonstrates a failure without this fix.
2024-04-04 11:36:18 -07:00
jiuker
272367ccd2
feat: add memlimit flags for setMaxResources (#19400) 2024-04-04 05:06:57 -07:00
Anis Eleuch
95bf4a57b6
logging: Add subsystem to log API (#19002)
Create new code paths for multiple subsystems in the code. This will
make maintaing this easier later.

Also introduce bugLogIf() for errors that should not happen in the first
place.
2024-04-04 05:04:40 -07:00
Harshavardhana
2228eb61cb
Add more tests for ARN and its format (#19408)
Original work from #17566 modified to fit the new requirements
2024-04-04 01:31:34 -07:00
Alexander Thaller
5f07eb2d17
Add env variable MINIO_IDENTITY_OPENID_REDIRECT_URI to statefulset (#18949)
Using oidc.redirectUri in the values.yaml only works for the deployment.

When using the statefulset the environment variable
MINIO_IDENTITY_OPENID_REDIRECT_URI is not set. This leads to errors with
oicd providers. For example keycloak throws the error 'invalid
redirect_uri'.

This pull request fixes that.
2024-04-03 23:34:45 -07:00
Shubhendu
d96d696841
Dont use deprecated angular (#19396)
Support for Angular would be stopped with newer versions of grafana

Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>
2024-04-03 19:01:53 -07:00
Harshavardhana
e18c0ab9bf update vulncheck to go1.21.9
Signed-off-by: Harshavardhana <harsha@minio.io>
2024-04-03 19:00:59 -07:00
Andreas Auernhammer
faeb2b7e79
use GenerateKey as more reliable KMS health-check (#19404)
This commit replaces the `KMS.Stat` API call with a
`KMS.GenerateKey` call. This approach is more reliable
since data key generation also works when the KMS backend
is unavailable (temp. offline), but KES has cached the
key. Ref: KES offline caching.

With this change, it is less likely that MinIO readiness
checks fail in cases where the KMS backend is offline.

Signed-off-by: Andreas Auernhammer <github@aead.dev>
2024-04-03 14:13:20 -07:00
Anis Eleuch
97ce11cb6b
Avoid using a nil transport when the config is not initialized (#19405)
Make sure to pass a nil pointer as a Transport to minio-go  when the API config
is not initialized, this will make sure that we do not pass an interface
with a known type but a nil value.

This will also fix the update of the API remote_transport_deadline
configuration without requiring the cluster restart.
2024-04-03 11:27:05 -07:00
Harshavardhana
d7daae4762
update to latest deps (#19399)
Signed-off-by: Harshavardhana <harsha@minio.io>
2024-04-03 09:28:32 -07:00
jiuker
3d86ae12bc
feat: support EdDSA/Ed25519 for oss (#19397) 2024-04-02 16:02:35 -07:00
Sveinn
ba46ee5dfa
Adding console targets back into systemtarget log slice (#19398) 2024-04-02 15:56:14 -07:00
Klaus Post
912bbb2f1d
Always return slice with cap (#19395)
Documentation promised this - so we should do it as well. Try to get a buffer and stash if it isn't big enough.
2024-04-02 08:56:18 -07:00
Harshavardhana
4f660a8eb7
fix: missing metrics for healed objects (#19392)
all healed successful objects via queueHealTask
in a non-blocking heal weren't being reported
correctly, this PR fixes this comprehensively.
2024-04-01 23:48:36 -07:00
Praveen raj Mani
ae4fb1b72e
Prioritize the bucket configs first during the decommissioning (#19393) 2024-04-01 23:48:26 -07:00
Klaus Post
b435806d91
Reduce big message RPC allocations (#19390)
Use `ODirectPoolSmall` buffers for inline data in PutObject.

Add a separate call for inline data that will fetch a buffer for the inline data before unmarshal.
2024-04-01 16:42:09 -07:00
Minio Trusted
06929258bc Update yaml files to latest version RELEASE.2024-03-30T09-41-56Z 2024-03-30 18:32:38 +00:00
Harshavardhana
cb577835d9 add curl to hotfix, release.fips 2024-03-30 02:41:56 -07:00
Harshavardhana
7f35f74f14 add missing curl for other platforms 2024-03-29 12:10:49 -07:00
Klaus Post
3d6194e93c
Remove empty replication stats (#19385)
When sending final stats upstream also trim empty ReplicationStats.
2024-03-29 11:57:52 -07:00
Harshavardhana
72c7845f7e
add static curl to container (#19383) 2024-03-29 08:08:52 -07:00
Harshavardhana
1c99597a06
update() inlineBlock settings properly in storageClass config (#19382) 2024-03-29 08:07:06 -07:00
Harshavardhana
feb9d8480b
add auditing for healing objects (#19379) 2024-03-28 16:46:19 -07:00
Aditya Manthramurthy
4e670458b8
fix: CI warnings (#19380) 2024-03-28 16:44:49 -07:00
Aditya Manthramurthy
48deccdc40
fix: sts accounts map refresh and fewer list calls (#19376)
This fixes a bug where STS Accounts map accumulates accounts in memory
and never removes expired accounts and the STS Policy mappings were not
being refreshed.

The STS purge routine now runs with every IAM credentials load instead
of every 4th time.

The listing of IAM files is now cached on every IAM load operation to
prevent re-listing for STS accounts purging/reload.

Additionally this change makes each server pick a time for IAM loading
that is randomly distributed from a 10 minute interval - this is to
prevent server from thundering while performing the IAM load.

On average, IAM loading will happen between every 5-15min after the
previous IAM load operation completes.
2024-03-28 16:43:50 -07:00
Andi Bräu
2eee744e34
Fix issue [#19314], resolve the absence of the sed command in ub… (#19315)
Fix issue [minio#19314], resolve the absence of the sed command in ubi-micro by replacing it with echo.

Signed-off-by: Andreas Bräu <ab@andi95.de>
Co-authored-by: jiuker <2818723467@qq.com>
2024-03-28 16:20:50 -07:00
Kaan Kabalak
3f72439b8a
Suppress error log for force-deleting object in locked bucket (#19378) 2024-03-28 14:37:42 -07:00
Shubhendu
468a9fae83
Enable replication of SSE-C objects (#19107)
If site replication enabled across sites, replicate the SSE-C
objects as well. These objects could be read from target sites
using the same client encryption keys.

Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>
2024-03-28 10:44:56 -07:00
Shubhendu
d87f91720b
Split the replication dashboard in cluster and node level (#19374)
Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>
2024-03-28 10:15:39 -07:00
Klaus Post
aa0eec16ab
Remove empty replication stats when sending update (#19375)
When sending update and there is no replication stats - remove the struct.

Will remove an unneeded alloc on the receiver.
2024-03-28 10:13:07 -07:00
Shubhendu
d63e603040
Pre populate the server names using a query (#19367)
User doesn't need to remember and enter the server values,
rather they can select from the pre populated list.

Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>
2024-03-28 08:14:26 -07:00
jiuker
8222a640ac
fix: slice append lose the data for NSScanner (#19373) 2024-03-28 08:13:36 -07:00
Aditya Manthramurthy
7e45d84ace
ldap: improve normalization of DN values (#19358)
Instead of relying on user input values, we use the DN value returned by
the LDAP server.

This handles cases like when a mapping is set on a DN value
`uid=svc.algorithm,OU=swengg,DC=min,DC=io` with a user input value (with
unicode variation) of `uid=svc﹒algorithm,OU=swengg,DC=min,DC=io`. The
LDAP server on lookup of this DN returns the normalized value where the
unicode dot character `SMALL FULL STOP` (in the user input), gets
replaced with regular full stop.
2024-03-27 23:45:26 -07:00
Harshavardhana
139a606f0a
use bigger partSize per part for tiering to MinIO (#19361)
Bonus: remove persistent md5sum calculation, turn-off
sha256 as well. Instead we always enable crc32c which
is enough for payload verification also support for
trailing headers checksum.
2024-03-27 23:45:08 -07:00
Harshavardhana
289223b6de
expire ILM all versions verify quorum on action (#19359) 2024-03-27 23:44:52 -07:00
Harshavardhana
c61dd16a1e
fix: avoid fan-out DeletePrefix calls for batch-expire and ILM (#19365) 2024-03-27 20:18:15 -07:00
Harshavardhana
3e38fa54a5
set max versions to be IntMax to avoid premature failures (#19360)
let users/customers set relevant values make default value
to be non-applicable.
2024-03-27 18:08:07 -07:00
jiuker
4a02189ba0
feat: add env to choose which node to start decom (#19310)
add a temporary env  _MINIO_DECOM_ENDPOINT to choose 
the node to start decom from, in situations when first node
first pool is not available.
2024-03-27 16:18:40 -07:00
Shubhendu
3d4fc28ec9
Render node graphs by node (#19356)
As total drives count, online vs offline are per node basis, its
corect to select node for which graphs need to be rendered.

Set prometheus scrape jobs to fetch metrics from all nodes. A sample
scrape job for node metrics could be as below

```
- job_name: minio-job-node
  bearer_token: <token>
  metrics_path: /minio/v2/metrics/node
  scheme: https
  tls_config:
    insecure_skip_verify: true
  static_configs:
  - targets: [tenant1-ss-0-0.tenant1-hl.tenant-ns.svc.cluster.local:9000,tenant1-ss-0-1.tenant1-hl.tenant-ns.svc.cluster.local:9000,tenant1-ss-0-2.tenant1-hl.tenant-ns.svc.cluster.local:9000,tenant1-ss-0-3.tenant1-hl.tenant-ns.svc.cluster.local:9000]
```

Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>
2024-03-27 10:41:08 -07:00
jiuker
ec3a3bb10d
fix: Remove unnecessary loops for searchParent (#19353) 2024-03-27 08:12:14 -07:00
Harshavardhana
364d3a0ac9
fix: new staticheck and linter issues reported (#19340) 2024-03-27 08:10:40 -07:00
Harshavardhana
cb536a73eb use pkger v2.2.9 2024-03-26 16:49:14 -07:00
Minio Trusted
428155add9 Update yaml files to latest version RELEASE.2024-03-26T22-10-45Z 2024-03-26 23:12:03 +00:00
Poorna
8bce123bba
fix: precondition check for multipart with existing object replication (#19349) 2024-03-26 15:10:45 -07:00
Harshavardhana
0a56dbde2f
allow configuring inline shard size value (#19336) 2024-03-26 15:06:19 -07:00