Commit Graph

11865 Commits

Author SHA1 Message Date
Harshavardhana
7e3166475d
simplify common functions in replication (#19480) 2024-04-11 17:27:32 -07:00
Klaus Post
5206c0e883
Inspect: Add error if no results (#19476)
When no results match or another error occurs, add an error to the stream. Keep the "inspect-input.txt" as the only thing in the zip for reference.

Example:

```
λ mc support inspect --airgap myminio/testbucket/fjghfjh/**
mc: Using public key from C:\Users\klaus\mc\support_public.pem
File data successfully downloaded as inspect-data.enc

λ inspect inspect-data.enc
Using private key from support_private.pem
output written to inspect-data.zip
2024/04/11 14:10:51 next stream: GetRawData: No files matched the given pattern

λ unzip -l inspect-data.zip
Archive:  inspect-data.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
      222  2024-04-11 14:10   inspect-input.txt
---------                     -------
      222                     1 file

λ
```

Modifies inspect to read until end of stream to report the error.

Bonus: Add legacy commandline params
2024-04-11 14:22:47 -07:00
Harshavardhana
41ec038523
remove permission denied error for being drive error (#19478) 2024-04-11 14:22:15 -07:00
Shireesh Anjal
08d3d06a06
Add drive metrics in metrics-v3 (#19452)
Add following metrics:

- used_inodes
- total_inodes
- healing
- online
- reads_per_sec
- reads_kb_per_sec
- reads_await
- writes_per_sec
- writes_kb_per_sec
- writes_await
- perc_util

To be able to calculate the `per_sec` values, we capture the IOStats-related 
data in the beginning (along with the time at which they were captured), 
and compare them against the current values subsequently. This is because 
dividing by "time since server uptime." doesn't work in k8s environments.
2024-04-11 10:46:34 -07:00
Harshavardhana
074febd9e1
remove SetDiskLoc() rely on the endpoint values instead (#19475)
the disk location never changes in the lifetime of a
MinIO cluster, even if it did validate this close to the
disk instead at the higher layer.

Return appropriate errors indicating an invalid drive, so
that the drive is not recognized as part of a valid
drive.
2024-04-11 10:45:28 -07:00
Harshavardhana
aa8d25797b
update versioning tests to cover CopyObject() (#19472)
adds tests to cover #19468
2024-04-11 02:50:52 -07:00
Alex
8d7d4adb91
Updated Console UI to v1.2.0 (#19467) 2024-04-11 01:31:16 -07:00
Poorna
ffa91f9794
fix CopyObject with replace overwriting inline status (#19468)
Fixes #19450 - internal inline-data header can get overwritten
during copy with replace before this fix.
2024-04-10 23:42:51 -07:00
Harshavardhana
0c31e61343
allow protection from invalid config values (#19460)
we have had numerous reports on some config
values not having default values, causing
features misbehaving and not having default
values set properly.

This PR tries to address all these concerns
once and for all.

Each new sub-system that gets added

- must check for invalid keys
- must have default values set
- must not "return err" when being saved into
  a global state() instead collate as part of
  other subsystem errors allow other sub-systems
  to independently initialize.
2024-04-10 18:10:30 -07:00
Harshavardhana
9b926f7dbe
avoid busy loops in bad path component (#19466)
use it in places where we are looking
for such bad path components.
2024-04-10 18:08:52 -07:00
Harshavardhana
35d8728990
handle missing LDAP normalization in SetPolicy() API (#19465) 2024-04-10 15:37:42 -07:00
Allan Roger Reid
f7ed9a75ba
Allow specifying the local server with env variable _MINIO_SERVER_LOCAL (#19453)
* Allow specifying the local server, with env variable _MINIO_SERVER_LOCAL, in systems where the hostname cannot be resolved to local IP

* Limit scope of the _MINIO_SERVER_LOCAL solution to only containerized implementations
2024-04-10 09:34:59 -07:00
jiuker
9496c17e13
doc: add Content-Type to s3zip (#19455) 2024-04-10 09:28:27 -07:00
jiuker
ed64e91f06
fix: noHost for collectLocalMetric (#19457) 2024-04-10 09:28:08 -07:00
jiuker
a481825ae1
fix: unknow contentType for ArchiveFileHandler (#19451) 2024-04-09 03:41:25 -07:00
Harshavardhana
7bb0f32332
make if-none-match PUT/POST RFC compliant (#19448)
fixes #19442
2024-04-09 01:17:49 -07:00
Anis Eleuch
c6f8dc431e
Add a warning when the total size of an object versions exceeds 1 TiB (#19435) 2024-04-08 10:45:03 -07:00
Alexander Thaller
78f177b8ee
Allow setting readOnlyRootFilesystem in securityContext (#19437) 2024-04-08 09:31:05 -07:00
Anis Eleuch
787c44c39d
batch-repl: Do not allow both source/target to be remote (#19434)
Return an error when the user specifies endpoints for both source
and target. This can generate many type of errors as the code considers
a deployment remote if its endpoint is specified.
2024-04-08 07:11:38 -07:00
Anis Eleuch
f06fee0364
heal: Add more per disk healing result in the audit (#19427)
HealObject() does not return an error in some cases, for example, when
an object is successfully reconstructed in one disk but fails with other
disks, another case is when a disk does not have the object is temporarily
disconnected

Add the After heal drives result in the audit output for better
analysis.
2024-04-08 02:26:14 -07:00
Harshavardhana
c957e0d426
fix: increase the tiering part size to 128MiB (#19424)
also introduce 8MiB buffer to read from for
bigger parts
2024-04-08 02:22:27 -07:00
Harshavardhana
04101d472f
fix: add fallbackDisks for disk healing (#19425) 2024-04-08 02:22:13 -07:00
Minio Trusted
51fc145161 Update yaml files to latest version RELEASE.2024-04-06T05-26-02Z 2024-04-06 06:44:30 +00:00
Taran Pelkey
9d63bb1b41
Added new API errors for LDAP (#19415)
* change internal errors to named errors

* Change names
2024-04-05 22:26:02 -07:00
Aditya Manthramurthy
8ff2a7a2b9
fix: IAM import/export: remove sts group handling (#19422)
There are no separate STS group mappings to be handled.

Also add tests for basic import/export sanity.
2024-04-05 20:13:35 -07:00
Harshavardhana
91f91d8f47
fix: a regression in IAM policy reload routine() (#19421)
all policy reloading is broken since last release since

48deccdc40

fixes #19417
2024-04-05 14:26:41 -07:00
Harshavardhana
a207bd6790
turn-off Nlink readdir() optimization for NFS/CIFS (#19420)
fixes #19418
fixes #19416
2024-04-05 08:17:08 -07:00
Harshavardhana
96d226c0b1
remove frivolous log about abort-multipart failure in replication (#19413) 2024-04-05 04:39:55 -07:00
Krishnan Parthasarathi
a86d98826d
Set object's original modTime when being restored (#19414)
Set object's modTime when being restored

restored here refers to making a temporary local copy in the hot tier
for a tiered object using the RestoreObject API
2024-04-05 04:39:31 -07:00
Harshavardhana
1bb670ecba
use new generics based LRU from hashicorp (#19409)
we have been using an LRU caching for internode
auth tokens, migrate to using a typed implementation
and also do not cache auth tokens when its an error.
2024-04-04 11:58:48 -07:00
Aditya Manthramurthy
c9e9a8e2b9
fix: ldap: use validated base DNs (#19406)
This fixes a regression from #19358 which prevents policy mappings
created in the latest release from being displayed in policy entity
listing APIs.

This is due to the possibility that the base DNs in the LDAP config are
not in a normalized form and #19358 introduced normalized of mapping
keys (user DNs and group DNs). When listing, we check if the policy
mappings are on entities that parse as valid DNs that are descendants of
the base DNs in the config.

Test added that demonstrates a failure without this fix.
2024-04-04 11:36:18 -07:00
jiuker
272367ccd2
feat: add memlimit flags for setMaxResources (#19400) 2024-04-04 05:06:57 -07:00
Anis Eleuch
95bf4a57b6
logging: Add subsystem to log API (#19002)
Create new code paths for multiple subsystems in the code. This will
make maintaing this easier later.

Also introduce bugLogIf() for errors that should not happen in the first
place.
2024-04-04 05:04:40 -07:00
Harshavardhana
2228eb61cb
Add more tests for ARN and its format (#19408)
Original work from #17566 modified to fit the new requirements
2024-04-04 01:31:34 -07:00
Alexander Thaller
5f07eb2d17
Add env variable MINIO_IDENTITY_OPENID_REDIRECT_URI to statefulset (#18949)
Using oidc.redirectUri in the values.yaml only works for the deployment.

When using the statefulset the environment variable
MINIO_IDENTITY_OPENID_REDIRECT_URI is not set. This leads to errors with
oicd providers. For example keycloak throws the error 'invalid
redirect_uri'.

This pull request fixes that.
2024-04-03 23:34:45 -07:00
Shubhendu
d96d696841
Dont use deprecated angular (#19396)
Support for Angular would be stopped with newer versions of grafana

Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>
2024-04-03 19:01:53 -07:00
Harshavardhana
e18c0ab9bf update vulncheck to go1.21.9
Signed-off-by: Harshavardhana <harsha@minio.io>
2024-04-03 19:00:59 -07:00
Andreas Auernhammer
faeb2b7e79
use GenerateKey as more reliable KMS health-check (#19404)
This commit replaces the `KMS.Stat` API call with a
`KMS.GenerateKey` call. This approach is more reliable
since data key generation also works when the KMS backend
is unavailable (temp. offline), but KES has cached the
key. Ref: KES offline caching.

With this change, it is less likely that MinIO readiness
checks fail in cases where the KMS backend is offline.

Signed-off-by: Andreas Auernhammer <github@aead.dev>
2024-04-03 14:13:20 -07:00
Anis Eleuch
97ce11cb6b
Avoid using a nil transport when the config is not initialized (#19405)
Make sure to pass a nil pointer as a Transport to minio-go  when the API config
is not initialized, this will make sure that we do not pass an interface
with a known type but a nil value.

This will also fix the update of the API remote_transport_deadline
configuration without requiring the cluster restart.
2024-04-03 11:27:05 -07:00
Harshavardhana
d7daae4762
update to latest deps (#19399)
Signed-off-by: Harshavardhana <harsha@minio.io>
2024-04-03 09:28:32 -07:00
jiuker
3d86ae12bc
feat: support EdDSA/Ed25519 for oss (#19397) 2024-04-02 16:02:35 -07:00
Sveinn
ba46ee5dfa
Adding console targets back into systemtarget log slice (#19398) 2024-04-02 15:56:14 -07:00
Klaus Post
912bbb2f1d
Always return slice with cap (#19395)
Documentation promised this - so we should do it as well. Try to get a buffer and stash if it isn't big enough.
2024-04-02 08:56:18 -07:00
Harshavardhana
4f660a8eb7
fix: missing metrics for healed objects (#19392)
all healed successful objects via queueHealTask
in a non-blocking heal weren't being reported
correctly, this PR fixes this comprehensively.
2024-04-01 23:48:36 -07:00
Praveen raj Mani
ae4fb1b72e
Prioritize the bucket configs first during the decommissioning (#19393) 2024-04-01 23:48:26 -07:00
Klaus Post
b435806d91
Reduce big message RPC allocations (#19390)
Use `ODirectPoolSmall` buffers for inline data in PutObject.

Add a separate call for inline data that will fetch a buffer for the inline data before unmarshal.
2024-04-01 16:42:09 -07:00
Minio Trusted
06929258bc Update yaml files to latest version RELEASE.2024-03-30T09-41-56Z 2024-03-30 18:32:38 +00:00
Harshavardhana
cb577835d9 add curl to hotfix, release.fips 2024-03-30 02:41:56 -07:00
Harshavardhana
7f35f74f14 add missing curl for other platforms 2024-03-29 12:10:49 -07:00
Klaus Post
3d6194e93c
Remove empty replication stats (#19385)
When sending final stats upstream also trim empty ReplicationStats.
2024-03-29 11:57:52 -07:00