Commit Graph

12204 Commits

Author SHA1 Message Date
Harshavardhana eff0ea43aa
fix: typo in BucketUsageMetrics group registration in v3 metrics (#20090)
```
curl http://localhost:9000/minio/metrics/v3/cluster/usage/buckets
```

Did not work as documented, due to the fact that there was a typo
in the bucket usage metrics registration group. This endpoint is
a cluster endpoint and does not require any `buckets` argument.
2024-07-14 11:11:42 -07:00
Minio Trusted 3b602bb532 Update yaml files to latest version RELEASE.2024-07-13T01-46-15Z 2024-07-13 02:08:28 +00:00
Alex 459985f0fa
Update Console to v1.6.3 (#20084)
Signed-off-by: Benjamin Perez <benjamin@bexsoft.net>
2024-07-12 18:46:15 -07:00
Harshavardhana d0080046c2 allow sysfs tuning from tuned 2024-07-12 16:31:18 -07:00
Harshavardhana 7fcb428622
do not print unexpected logs (#20083) 2024-07-12 13:51:54 -07:00
dependabot[bot] 4ea6f94ed8
Bump github.com/nats-io/nats-streaming-server from 0.24.3 to 0.24.6 (#20082)
Bumps [github.com/nats-io/nats-streaming-server](https://github.com/nats-io/nats-streaming-server) from 0.24.3 to 0.24.6.
- [Release notes](https://github.com/nats-io/nats-streaming-server/releases)
- [Changelog](https://github.com/nats-io/nats-streaming-server/blob/main/.goreleaser.yml)
- [Commits](https://github.com/nats-io/nats-streaming-server/compare/v0.24.3...v0.24.6)

---
updated-dependencies:
- dependency-name: github.com/nats-io/nats-streaming-server
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-07-12 11:45:09 -07:00
Klaus Post 83adc2eebf
Fix ListObjects aborting after 3 minute on async request (#20074)
When creating the async listing, if the first request does not return within 3 
minutes, it is stopped, since it isn't being kept alive.

Keep updating `lastHandout` while we are waiting for the initial request to be fulfilled.
2024-07-12 09:23:16 -07:00
Poorna 989c318a28
replication: make large workers configurable (#20077)
This PR also improves throttling by reducing tokens requested
from rate limiter based on available tokens to avoid exceeding
throttle wait deadlines
2024-07-12 07:57:31 -07:00
Frank Wessels ef802f2b2c
Updated dependencies for ARM SVE support (#20081) 2024-07-12 07:36:29 -07:00
Taran Pelkey f5d2fbc84c
Add DecodeDN and QuickNormalizeDN functions to LDAP config (#20076) 2024-07-11 18:04:53 -07:00
Allan Roger Reid e139673969
Audit failure in batch job key rotate (#20073) 2024-07-11 16:13:15 -07:00
Harshavardhana a8c6465f22
hide some deprecated fields from 'get' output (#20069)
also update wording on `subnet license="" api_key=""`
2024-07-10 13:16:44 -07:00
Minio Trusted 27538e2d22 Update yaml files to latest version RELEASE.2024-07-10T18-41-49Z 2024-07-10 19:27:17 +00:00
Taran Pelkey 6c6f0987dc
Add groups to policy entities (#20052)
* Add groups to policy entities

* update comment

---------

Co-authored-by: Harshavardhana <harsha@minio.io>
2024-07-10 11:41:49 -07:00
Austin Chang 5f64658faa
clarify error message for root user credential (#20043)
Signed-off-by: Austin Chang <austin880625@gmail.com>
2024-07-10 09:57:01 -07:00
Anis Eleuch ce183cb2b4
heal: List and heal again for any listing error (#19999)
When a fresh drive healing is finished, add more checks for the drive listing
errors. If any, re-list and heal again. Although this is an infrequent use
case to have listPathRaw() returning nil when minDisks is set to 1, we
still need to handle all possible use cases to avoid missing healing
any object.

Also, check for HealObject result to decide of an object is healed in the
fresh disk since HealObject returns nil if an object is healed in any
disk, and not in the new fresh drive.
2024-07-10 09:55:36 -07:00
Klaus Post b3bac73c0f
Clarify post policy error message (#20067)
It is not really clear that the listed keys are missing.

Clarify the error
2024-07-10 07:18:44 -07:00
Anis Eleuch e726d8ff0f
list: Hide objects/versions with pending/failed replicated deletion (#20047)
In regular listing, this commit will avoid showing an object when its
latest version has a pending or failed deletion. In replicated setup.
It will also prevent showing older versions in the same case.
2024-07-09 15:26:42 -07:00
Shubhendu f4230777b3
Log replication errors once (#20063)
Also, sort the error map for multiple sites in ascending order
of deployment IDs, so that the error message generated is always
definitive order and same.

Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>
2024-07-09 10:10:31 -07:00
Krishnan Parthasarathi 380233d646
batch: Update job info object on success (#20053) 2024-07-08 18:45:54 -07:00
Allan Roger Reid d592bc0c1c
Fix documentation for removal of delete markers ILM rule (#20056) 2024-07-08 18:45:38 -07:00
Klaus Post 0d0b0aa599
Abstract grid connections (#20038)
Add `ConnDialer` to abstract connection creation.

- `IncomingConn(ctx context.Context, conn net.Conn)` is provided as an entry point for 
   incoming custom connections.

- `ConnectWS` is provided to create web socket connections.
2024-07-08 14:44:00 -07:00
Anis Eleuch b433bf14ba
Add typos check to Makefile (#20051) 2024-07-08 14:39:49 -07:00
Minio Trusted cf371da346 Update yaml files to latest version RELEASE.2024-07-04T14-25-45Z 2024-07-04 14:58:08 +00:00
Klaus Post 107d951893
Log ILM failed object name (#20040)
Log so we know which object we are dealing with.

Log each object once.
2024-07-04 07:25:45 -07:00
Shireesh Anjal 22c53b1c70
Remove license update job (#20037) 2024-07-03 11:49:48 -07:00
Mark Theunissen 88926ad8e9
return appropriate error upon tier update for incorrect credentials (#20034) 2024-07-03 00:17:20 -07:00
Harshavardhana 32d04091a2
resume any batch jobs in a goroutine (#20035)
Bonus: move batch job initialization to the last item after all other initialization, 
            allowing for faster startup time for different subsystems.
2024-07-03 00:16:05 -07:00
Harshavardhana b6d4a77b94 update vulncheck 2024-07-02 14:34:59 -07:00
Harshavardhana be84a4fd68
do not proxy invalid object names (#20031) 2024-07-02 14:28:55 -07:00
Anis Eleuch 2ec1f404ac
info: Always refresh the root disk status (#20023)
Add root drive status in the disk info cache function, so unmounting a
drive without restarting a local node reflects the correct value.
2024-07-02 13:41:29 -07:00
Klaus Post 2040559f71
Fix SkipReader performance with small initial read (#20030)
If `SkipReader` is called with a small initial buffer it may be doing a huge number if Reads to skip the requested number of bytes. If a small buffer is provided grab a 32K buffer and use that.

Fixes slow execution of `testAPIGetObjectWithMPHandler`.

Bonuses:

* Use `-short` with `-race` test.
* Do all suite test types with `-short`.
* Enable compressed+encrypted in `testAPIGetObjectWithMPHandler`.
* Disable big file tests in `testAPIGetObjectWithMPHandler` when using `-short`.
2024-07-02 08:13:05 -07:00
Anis Eleuch ca0ce4c6ef
tests: Fix setting max openfds as memory limit (#20029)
The code was advertenly passing max openfds to debug.SetMemoryLimit(),
fixing this accelerate go test in my machine.

This is only a testing bug, since the server context has always a valid
MaxMem, so the buggy code was never called in users environments.
2024-07-02 08:09:36 -07:00
Anis Eleuch 757cf413cb
Add batch status API (#19679)
Currently the status of a completed or failed batch is held in the
memory, a simple restart will lose the status and the user will not
have any visibility of the job that was long running.

In addition to the metrics, add a new API that reads the batch status
from the drives. A batch job will be cleaned up three days after
completion.

Also add the batch type in the batch id, the reason is that the batch
job request is removed immediately when the job is finished, then we
do not know the type of batch job anymore, hence a difficulty to locate
the job report
2024-07-02 01:17:52 -07:00
Anis Eleuch b35acb3dbc
heal: Add support of healing particular pool/set (#20024) 2024-07-01 15:02:25 -07:00
Sveinn e404abf103
Letting password enable auth bypass caPublicKey (only if passauth is … (#20022) 2024-07-01 15:02:01 -07:00
jiuker f7ff19cb18
fix: warning for decommissioned pool while start (#20019) 2024-07-01 07:38:46 -07:00
Minio Trusted f736702da8 Update yaml files to latest version RELEASE.2024-06-29T01-20-47Z 2024-06-29 16:45:31 +00:00
Poorna 91faaa1387
fix panic in batch replicate (#20014)
Fixes:

```
panic: send on closed channel
	panic: close of closed channel

goroutine 878 [running]:
github.com/minio/minio/internal/ioutil.SafeClose[...](...)
	/Users/kp/code/src/github.com/minio/minio/internal/ioutil/ioutil.go:407
github.com/minio/minio/cmd.(*erasureServerPools).Walk.func2.2()
	/Users/kp/code/src/github.com/minio/minio/cmd/erasure-server-pool.go:2229 +0xc0
panic({0x108c25e60?, 0x1090b28d0?})
	/usr/local/go/src/runtime/panic.go:770 +0x124
github.com/minio/minio/cmd.(*erasureServerPools).Walk.func2.3({{0x1400e397316, 0x5}, {0x1400d88b8a8, 0x8}, {0x1f99d80, 0xede101c42, 0x0}, 0x3bc, 0x0, 0x0, ...})
	/Users/kp/code/src/github.com/minio/minio/cmd/erasure-server-pool.go:2235 +0xb4
github.com/minio/minio/cmd.(*erasureServerPools).Walk.func2()
	/Users/kp/code/src/github.com/minio/minio/cmd/erasure-server-pool.go:2277 +0xabc
created by github.com/minio/minio/cmd.(*erasureServerPools).Walk in goroutine 575
	/Users/kp/code/src/github.com/minio/minio/cmd/erasure-server-pool.go:2210 +0x33c
```
2024-06-28 18:20:47 -07:00
Poorna 68a9f521d5
fix object lock metadata filter (#20011) 2024-06-28 18:20:27 -07:00
Harshavardhana f365a98029
fix: hot-reloading STS credential policy documents (#20012)
* fix: hot-reloading STS credential policy documents
* Support Role ARNs hot load policies (#28)

---------

Co-authored-by: Anis Eleuch <vadmeste@users.noreply.github.com>
2024-06-28 16:17:22 -07:00
Minio Trusted 47bbc272df Update yaml files to latest version RELEASE.2024-06-28T09-06-49Z 2024-06-28 14:11:36 +00:00
Anis Eleuch aebac90013
tests: Fix minor issue in the config yaml file testing (#20005)
Convert x86_64 to amd64 in the test script to correctly download mc binary.
2024-06-28 02:06:49 -07:00
Taran Pelkey 7ca4ba77c4
Update tests to use AttachPolicy(LDAP) instead of deprecated SetPolicy (#19972) 2024-06-28 02:06:25 -07:00
Poorna 13512170b5
list: Do not decrypt SSE-S3 Etags in a non encrypted format (#20008) 2024-06-27 19:44:56 -07:00
Krishnan Parthasarathi 154fcaeb56
Allow rebalance start when it's stopped/completed (#20009) 2024-06-27 17:22:30 -07:00
Anis Eleuch 722118386d
iam: Hot load of the policy during request authorization (#20007)
Hot load a policy document when during account authorization evaluation
to avoid returning 403 during server startup, when not all policies are
already loaded.

Add this support for group policies as well.
2024-06-27 17:03:07 -07:00
Harshavardhana 709612cb37
fix: rebalance upon pool expansion would crash when in progress (#20004)
you can attempt a rebalance first i.e, start with 2 pools.

```
mc admin rebalance start alias/
```

and after that you can add a new pool, this would
potentially crash.

```
Jun 27 09:22:19 xxx minio[7828]: panic: runtime error: invalid memory address or nil pointer dereference
Jun 27 09:22:19 xxx minio[7828]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x58 pc=0x22cc225]
Jun 27 09:22:19 xxx minio[7828]: goroutine 1 [running]:
Jun 27 09:22:19 xxx minio[7828]: github.com/minio/minio/cmd.(*erasureServerPools).findIndex(...)
```
2024-06-27 11:35:34 -07:00
Harshavardhana b35d083872
fix; change retry-after 60sec for 503s and 10s for 429s (#19996) 2024-06-26 01:32:06 -07:00
Harshavardhana 5e7b243bde
extend cluster health to return errors for IAM, and Bucket metadata (#19995)
Bonus: make API freeze to be opt-in instead of default
2024-06-26 00:44:34 -07:00