Commit Graph

11445 Commits

Author SHA1 Message Date
jiuker
d5e8dac1cf fix: canceling the heal caused goroutine to leak. (#18322) 2023-10-26 07:53:06 -07:00
Poorna
96ec8fcba1 Preserve replica timestamps in multipart (#18318)
Also a backward compatibility fix to use x-amz-replica-status
if present as replication status.
2023-10-25 21:24:10 -07:00
Harshavardhana
0663eb69ed fix: do not preserve mtime during CopyObject() metadata updates (#18316)
mtime must be preserved only if destination mtime is set.

fixes #18314
2023-10-25 14:30:56 -07:00
Harshavardhana
0594d37230 update go mod and CREDITS (#18289) 2023-10-25 08:32:59 -07:00
Andreas Auernhammer
3cc30bcc18 upgrade container base image to ubi-9 (#18313)
This commit updates the container base image from
ubi:8.8 to ubi:9.2.

Signed-off-by: Andreas Auernhammer <github@aead.dev>
2023-10-25 08:32:18 -07:00
Minio Trusted
99c1a642a4 Update yaml files to latest version RELEASE.2023-10-25T06-33-25Z 2023-10-25 07:52:49 +00:00
Harshavardhana
c60f54e5be make ListMultipart/ListParts more reliable skip healing disks (#18312)
this PR also fixes old flaky tests, by properly marking disk offline-based tests.
RELEASE.2023-10-25T06-33-25Z
2023-10-24 23:33:25 -07:00
Harshavardhana
483389f2e2 set diskMaxConcurrent to 32 if nrRequests is lower 2023-10-24 17:21:12 -07:00
Harshavardhana
c0f2f84285 avoid racy replicationCount checks (#18311)
resync status may not be upto-date by
the time the resync is over due to how
the timer is triggered.

diff is sufficient to know if replication
happened or not.
2023-10-24 15:30:42 -07:00
Harshavardhana
069d118329 fix: listObjectParts to prefer local and single disks (#18309) 2023-10-24 13:51:57 -07:00
Harshavardhana
a7b1834772 fix: flaky and stupid tests in root lockdown (#18308) 2023-10-24 13:22:44 -07:00
Klaus Post
6415dec37a Improve multipart listing speed (#18307) 2023-10-24 12:06:06 -07:00
Klaus Post
74253e1ddc Fix BackendInfo() race (#18305)
`GetParityForSC` has a value receiver, so Config is copied before the lock is obtained.

Make it pointer receiver.

Fixes:

```
WARNING: DATA RACE
Read at 0x0000079cdd10 by goroutine 190:
  github.com/minio/minio/cmd.(*erasureServerPools).BackendInfo()
      github.com/minio/minio/cmd/erasure-server-pool.go:579 +0x6f
  github.com/minio/minio/cmd.(*erasureServerPools).LocalStorageInfo()
      github.com/minio/minio/cmd/erasure-server-pool.go:614 +0x3c6
  github.com/minio/minio/cmd.(*peerRESTServer).LocalStorageInfoHandler()
      github.com/minio/minio/cmd/peer-rest-server.go:347 +0x4ea
  github.com/minio/minio/cmd.(*peerRESTServer).LocalStorageInfoHandler-fm()
...

WARNING: DATA RACE
Read at 0x0000079cdd10 by goroutine 190:
  github.com/minio/minio/cmd.(*erasureServerPools).BackendInfo()
      github.com/minio/minio/cmd/erasure-server-pool.go:579 +0x6f
  github.com/minio/minio/cmd.(*erasureServerPools).LocalStorageInfo()
      github.com/minio/minio/cmd/erasure-server-pool.go:614 +0x3c6
  github.com/minio/minio/cmd.(*peerRESTServer).LocalStorageInfoHandler()
      github.com/minio/minio/cmd/peer-rest-server.go:347 +0x4ea
  github.com/minio/minio/cmd.(*peerRESTServer).LocalStorageInfoHandler-fm()
```
2023-10-24 08:15:41 -07:00
Minio Trusted
01b3fb91e5 Update yaml files to latest version RELEASE.2023-10-24T04-42-36Z 2023-10-24 05:05:12 +00:00
Harshavardhana
2dc917e87f maxConcurrent must be set only once per node (#18303) RELEASE.2023-10-24T04-42-36Z 2023-10-23 21:42:36 -07:00
Aditya Manthramurthy
0a284a1a10 fix: SR: Add more info when IAM config differs (#18302)
Provide details on what IAM info mismatched when the validation fails
2023-10-23 21:16:40 -07:00
Harshavardhana
5c8339e1e8 fix: veeam SOS API to higher layers (#18287)
- support populating usage info from scanner info
- support populating quota for the bucket via quota
  settings for the bucket
2023-10-23 13:55:45 -07:00
Harshavardhana
fd37418da2 fix: allow server not initialized error to be retried (#18300)
Since relaxing quorum the error across pools
for ListBuckets(), GetBucketInfo() we hit a
situation where loading IAM could potentially
return an error for second pool that server
is not initialized.

We need to handle this, let the pool come online
and retry transparently - this PR fixes that.
2023-10-23 12:30:20 -07:00
Harshavardhana
bbfea29c2b use object modTime for the event sequencer ID (#18285)
always set modTime after lock is acquired in
completemultipart stage to make sure that the
modTime is not racy.
2023-10-20 19:28:05 -07:00
Harshavardhana
aa703dc903 relax write quorum requirement for ListBuckets()/HeadBucket() (#18288)
Also fix error handling for HeadBucket() to be pool specific
2023-10-20 17:50:21 -07:00
Krishnan Parthasarathi
8cd80fec8c Add unit test for lifecycle.FilterRules (#18284) 2023-10-19 21:33:28 -07:00
Harshavardhana
780882efcf do not check for query params to be signed headers (#18283)
x-amz-signed-headers is meant for HTTP headers only
not for query params, using that to verify things
further can lead to failure.

The generated presigned URL with custom metadata
is already kosher (tamper proof).

fixes #18281
2023-10-19 21:32:49 -07:00
dependabot[bot]
c5636143c6 build(deps): bump github.com/nats-io/nats-server/v2 from 2.9.20 to 2.9.23 (#18282)
build(deps): bump github.com/nats-io/nats-server/v2

Bumps [github.com/nats-io/nats-server/v2](https://github.com/nats-io/nats-server) from 2.9.20 to 2.9.23.
- [Release notes](https://github.com/nats-io/nats-server/releases)
- [Changelog](https://github.com/nats-io/nats-server/blob/main/.goreleaser.yml)
- [Commits](https://github.com/nats-io/nats-server/compare/v2.9.20...v2.9.23)

---
updated-dependencies:
- dependency-name: github.com/nats-io/nats-server/v2
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-10-19 16:40:59 -07:00
Klaus Post
ba6218b354 fix: resource metrics "concurrent map iteration and map write" (#18273)
`resourceMetricsMap` has no protection against concurrent reads and writes.

Add a mutex and don't use maps from the last iteration.

Bug introduced in #18057

Fixes #18271
2023-10-18 13:28:50 -07:00
Harshavardhana
8e32de3ba9 cache DiskInfo() metrics call separately (#18270) 2023-10-18 11:17:32 -07:00
Klaus Post
e37508fb8f fix: linter errors in Windows specific code (#18276) 2023-10-18 11:08:15 -07:00
Klaus Post
b46a717425 Remove unused config migration (#18277)
None of the migration is called. Remove dead code.
2023-10-18 11:05:24 -07:00
Klaus Post
7926df0b80 Fix globalDeploymentID race (#18275)
globalDeploymentID was being read while it was being set.

Fixes race:

```
WARNING: DATA RACE
Write at 0x0000079605a0 by main goroutine:
  github.com/minio/minio/cmd.connectLoadInitFormats()
      github.com/minio/minio/cmd/prepare-storage.go:269 +0x14f0
  github.com/minio/minio/cmd.waitForFormatErasure()
      github.com/minio/minio/cmd/prepare-storage.go:294 +0x21d
...

Previous read at 0x0000079605a0 by goroutine 105:
  github.com/minio/minio/cmd.newContext()
      github.com/minio/minio/cmd/utils.go:817 +0x31e
  github.com/minio/minio/cmd.adminMiddleware.func1()
      github.com/minio/minio/cmd/admin-router.go:110 +0x96
  net/http.HandlerFunc.ServeHTTP()
      net/http/server.go:2136 +0x47
  github.com/minio/minio/cmd.setBucketForwardingMiddleware.func1()
      github.com/minio/minio/cmd/generic-handlers.go:460 +0xb1a
  net/http.HandlerFunc.ServeHTTP()
      net/http/server.go:2136 +0x47
...
```
2023-10-18 08:06:57 -07:00
Krishnan Parthasarathi
557df666fd Don't skip rules with ExpiredObjectDeleteMarker (#18256) 2023-10-16 22:46:46 -07:00
Harshavardhana
f91b257f50 choose different max_concurrent requests per drive based on HDD/NVMe (#18254)
currently the default for all drives is 512, which is a lot
for HDDs the recent testing has revealed moving this to 32
for HDDs seems like a fair value.
2023-10-16 17:18:13 -07:00
Aditya Manthramurthy
28a2d1eb3d Allow OpenID ARN resource ID to start with a - (#18255) 2023-10-16 13:50:51 -07:00
Minio Trusted
a0ae1489e5 Update yaml files to latest version RELEASE.2023-10-16T04-13-43Z 2023-10-16 05:58:49 +00:00
Harshavardhana
edfb310a59 fix: always load ENVs from files first as soon as server starts (#18247)
This is a regression from #18231, however reading from ENV files
must happen well before any parsing logic is invoked.
RELEASE.2023-10-16T04-13-43Z
2023-10-15 21:13:43 -07:00
Minio Trusted
a2312028b9 Update yaml files to latest version RELEASE.2023-10-14T05-17-22Z 2023-10-14 06:27:13 +00:00
Poorna
78f1f69d57 fix site replication resync status (#18245)
To persist status changes on disk upon completion.

Adds new tests to handle this functionality.
RELEASE.2023-10-14T05-17-22Z
2023-10-13 22:17:22 -07:00
Harshavardhana
e1e33077e8 fix: tests and resync replication status (#18244) 2023-10-13 17:03:34 -07:00
Aditya Manthramurthy
b3e7de010d Remove usage of errors.Join for go1.19 compat (#18243) 2023-10-13 15:14:16 -07:00
Satish Michael
f5b04865f4 Helm Chart: Added "MINIO_IDENTITY_OPENID_REDIRECT_URI" Env Var (#18236)
added redirect uri env

Signed-off-by: Satish Kumar Kadarkarai Main <michael.satish@gmail.com>
2023-10-13 07:46:54 -07:00
Shireesh Anjal
bf1c6edb76 Revert "Capture network device info in health report" (#18241)
Introducing a new version of healthinfo struct for adding this info is
not correct. It needs to be implemented differently without adding a new
version.

This reverts commit 8737025d940f80360ed4b3686b332db5156f6659.
2023-10-13 07:46:36 -07:00
jiuker
2ac7fee017 fix: missing fileName will upload failed when PostPolicyBucketHandler (#18240) 2023-10-13 07:31:23 -07:00
Klaus Post
128256e3ab Add event counters (#18232)
Export metric for global events sent and skipped for the lifetime of the server.
2023-10-12 15:39:22 -07:00
Shireesh Anjal
a66a7f3e97 Capture network device info in health report (#18213) 2023-10-12 15:33:31 -07:00
jiuker
20b79f8945 fix: env depend on the flag (#18231) 2023-10-12 15:32:38 -07:00
Klaus Post
9a877734b2 Fix various poolmeta races (#18230)
There is a fundamental race condition in `newErasureServerPools`, where setObjectLayer is 
called before the poolMeta has been loaded/populated.

We add a placeholder value to this field but disable all saving of the value, so we don't risk 
overwriting the value on disk. Once the value has been loaded or created, it is replaced with 
the proper value, which will also be saved.

Also fixes various accesses of `poolMeta` that were done without locks.

We make the `poolMeta.IsSuspended` return false, even if we shouldn't risk out-of-bounds 
reads anymore.
2023-10-12 15:30:42 -07:00
Harshavardhana
409c391850 implement helpers to get relevant info instead of FileInfo() (#18228) 2023-10-12 15:29:59 -07:00
Klaus Post
763ff085a6 Add CI tests for next branch (#18224) 2023-10-12 06:15:10 -07:00
Shubhendu
5b9656374c Error if target went offline (#18221)
If target went offline while MinIO was down, error once
while trying to send message. If target goes offline during
MinIO server running, it already comes through ping() call
and errors out if target offline.

Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>
2023-10-12 06:13:57 -07:00
dependabot[bot]
b32014549c build(deps): bump golang.org/x/net from 0.15.0 to 0.17.0 (#18219)
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.15.0 to 0.17.0.
- [Commits](https://github.com/golang/net/compare/v0.15.0...v0.17.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-10-12 01:54:36 -07:00
dependabot[bot]
9476d212bc build(deps): bump golang.org/x/net from 0.14.0 to 0.17.0 in /docs/debugging/s3-verify (#18218)
build(deps): bump golang.org/x/net in /docs/debugging/s3-verify

Bumps [golang.org/x/net](https://github.com/golang/net) from 0.14.0 to 0.17.0.
- [Commits](https://github.com/golang/net/compare/v0.14.0...v0.17.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-10-12 00:39:12 -07:00
jiuker
000928d34e fix: should call func globalOSMetrics.time(s)() when updateOSMetrics (#18209) 2023-10-12 00:08:13 -07:00