Commit Graph

111 Commits

Author SHA1 Message Date
Poorna
8c9974bc0f
site replication: avoid propagating bucket b/w settings (#18399)
replication mode and bucket bandwidth are one-way and should not be
propagated to peer cluster.

Regression from #18062
2023-11-08 00:40:25 -08:00
jiuker
079b6c2b50
fix: add err when all bucket resync failed (#18401) 2023-11-08 00:40:08 -08:00
jiuker
c3e456e7e6
fix: no resyncid when site-replication cancel (#18392) 2023-11-06 01:53:31 -08:00
jiuker
8a56af439c
fix: siteReplicationSys.startResync return no buckets return if error (#18374) 2023-11-02 16:00:03 -07:00
Aditya Manthramurthy
0a284a1a10
fix: SR: Add more info when IAM config differs (#18302)
Provide details on what IAM info mismatched when the validation fails
2023-10-23 21:16:40 -07:00
Klaus Post
7926df0b80
Fix globalDeploymentID race (#18275)
globalDeploymentID was being read while it was being set.

Fixes race:

```
WARNING: DATA RACE
Write at 0x0000079605a0 by main goroutine:
  github.com/minio/minio/cmd.connectLoadInitFormats()
      github.com/minio/minio/cmd/prepare-storage.go:269 +0x14f0
  github.com/minio/minio/cmd.waitForFormatErasure()
      github.com/minio/minio/cmd/prepare-storage.go:294 +0x21d
...

Previous read at 0x0000079605a0 by goroutine 105:
  github.com/minio/minio/cmd.newContext()
      github.com/minio/minio/cmd/utils.go:817 +0x31e
  github.com/minio/minio/cmd.adminMiddleware.func1()
      github.com/minio/minio/cmd/admin-router.go:110 +0x96
  net/http.HandlerFunc.ServeHTTP()
      net/http/server.go:2136 +0x47
  github.com/minio/minio/cmd.setBucketForwardingMiddleware.func1()
      github.com/minio/minio/cmd/generic-handlers.go:460 +0xb1a
  net/http.HandlerFunc.ServeHTTP()
      net/http/server.go:2136 +0x47
...
```
2023-10-18 08:06:57 -07:00
Poorna
78f1f69d57
fix site replication resync status (#18245)
To persist status changes on disk upon completion.

Adds new tests to handle this functionality.
2023-10-13 22:17:22 -07:00
Poorna
50a8f13e85
site replication: allow setting bandwidth default for bucket (#18062)
This can still be overridden at the bucket level
2023-09-25 15:50:52 -07:00
Poorna
b733e6e83c
site replication turn off retry login for admin API calls (#18039)
additionally also mark site offline if n/w is down
2023-09-15 18:01:47 -07:00
Aditya Manthramurthy
1c99fb106c
Update to minio/pkg/v2 (#17967) 2023-09-04 12:57:37 -07:00
Poorna
b48bbe08b2
Add additional info for replication metrics API (#17293)
to track the replication transfer rate across different nodes,
number of active workers in use and in-queue stats to get
an idea of the current workload.

This PR also adds replication metrics to the site replication
status API. For site replication, prometheus metrics are
no longer at the bucket level - but at the cluster level.

Add prometheus metric to track credential errors since uptime
2023-08-30 01:00:59 -07:00
Anis Eleuch
0cde37be50
Reduce the number of calls to import bucket metadata (#17899)
For each bucket, save the bucket metadata 
once, call the site replication hook once
2023-08-25 07:59:16 -07:00
jiuker
02cc18ff29 refactor the perf client for TTFB and TotalResponseTime (#17901) 2023-08-24 10:21:08 -07:00
Poorna
875f4076ec
site replication: avoid retries when peer is offline (#17853) 2023-08-14 21:31:41 -07:00
jiuker
2dbb1cff4a
feat: support perf site replication (#17477) 2023-07-05 22:28:26 -07:00
Aditya Manthramurthy
5a1612fe32
Bump up madmin-go and pkg deps (#17469) 2023-06-19 17:53:08 -07:00
Anis Eleuch
d8e6e76e89
site-repl: Better error msg when setting sync in a local cluster (#17407) 2023-06-15 12:44:22 -07:00
Poorna
a9082e4f79
site replication: cancel ongoing op properly (#17428) 2023-06-15 08:05:08 -07:00
Harshavardhana
26a0803388
various compliance related fixes (#17401)
- getObjectTagging to be allowed for anonymous policies
- return correct errors for invalid retention period
- return sorted list of tags for an object
- putObjectTagging must return 200 OK not 204 OK
- return 409 ErrObjectLockConfigurationNotAllowed for existing buckets
2023-06-12 13:22:07 -07:00
Poorna
f86b9abf32
site removal: update site config and reload targets after update (#17327) 2023-06-01 10:19:56 -07:00
Harshavardhana
fc03be7891
simplify bucket metadata lookups for versioning/object locking (#17253) 2023-05-22 12:05:14 -07:00
Aditya Manthramurthy
9d96b18df0
Add "name" and "description" params to service acc (#17172) 2023-05-17 17:05:36 -07:00
Harshavardhana
7ae69accc0
allow root user to be disabled via config settings (#17089) 2023-04-28 12:24:14 -07:00
Poorna
74040b457b
Allow setting sync mode for site replication (#16876) 2023-03-24 14:41:23 -07:00
Aditya Manthramurthy
6305b206e1
fix: site-repl should heal STS with virtual parent (#16792) 2023-03-10 16:21:51 -08:00
ferhat elmas
714283fae2
cleanup ignored static analysis (#16767) 2023-03-06 08:56:10 -08:00
Praveen raj Mani
4d708cebe9
Support adding service accounts with expiration (#16430)
Co-authored-by: Harshavardhana <harsha@minio.io>
2023-02-27 10:10:22 -08:00
Aditya Manthramurthy
9ed4fc9687
Remove globalOpenIDConfig (#16708) 2023-02-25 21:01:37 -08:00
Aditya Manthramurthy
e05f3d5d84
Remove globalLDAPConfig (#16706) 2023-02-25 08:07:22 +05:30
Poorna
8a08861dd9
fix: healing of replication config for endpoint changes (#16648) 2023-02-20 16:06:13 +05:30
Harshavardhana
b94733ab31
avoid locks when unnecessary in SiteReplicationMetaInfo() (#16650) 2023-02-18 05:35:22 -08:00
Poorna
eb7d3da994
Fix site replication status reporting of quota (#16626) 2023-02-16 08:08:35 +05:30
Harshavardhana
ee6d96eb46
Periodically remove stale buckets from in-memory (#16597) 2023-02-13 08:09:52 -08:00
Poorna
52aeebebea
Fix site replication meta info call to be non-blocking (#16526)
Co-authored-by: Harshavardhana <harsha@minio.io>
2023-02-08 21:16:53 -08:00
Harshavardhana
d8daabae9b
make site replication healing safer (#16560) 2023-02-07 21:44:42 -08:00
Harshavardhana
d19cbc81b5
fix: do not return IAM/Bucket metadata replication errors to client (#16486) 2023-01-26 11:11:54 -08:00
Harshavardhana
b4ef5ff294
remove unnecessary code checking for supported features (#16423) 2023-01-17 19:37:47 +05:30
Anis Elleuch
1ece3d1dfe
Add comment field to service accounts (#16380) 2023-01-10 21:57:52 +04:00
Harshavardhana
b882310e2b
avoid locks for internal and invalid buckets in MakeBucket() (#16302) 2022-12-23 07:46:00 -08:00
Poorna
6423e4c767
Remove site replication config if it succeeded locally (#16279) 2022-12-22 01:31:20 -08:00
Poorna
d37e514733
Cleanup remote targets automatically on replication config removal. (#16221) 2022-12-14 03:24:06 -08:00
Aditya Manthramurthy
a30cfdd88f
Bump up madmin-go to v2 (#16162) 2022-12-06 13:46:50 -08:00
Klaus Post
a713aee3d5
Run staticcheck on CI (#16170) 2022-12-05 11:18:50 -08:00
Anis Elleuch
04948b4d55
fix: checking for stale STS account under site replication (#16109) 2022-11-22 07:26:33 -08:00
Poorna
d6bc141bd1
feat: Add support for site level resync (#15753) 2022-11-14 07:16:40 -08:00
Anis Elleuch
3b1a9b9fdf
Use the same lock for the scanner and site replication healing (#15985) 2022-11-08 08:55:55 -08:00
Poorna
4f5d38a4b1
site replication edit: validate endpoint belongs to deployment (#16000) 2022-11-03 16:23:45 -07:00
Harshavardhana
0d49b365ff
converge SNSD deployments into single code (#15988) 2022-11-01 16:41:01 -07:00
Klaus Post
0f0e154315
fix: inconsistent replication delete marker timestamps (#15956) 2022-10-27 09:46:52 -07:00
Poorna
ce8456a1a9
proxy multipart to peers via multipart uploadID (#15926) 2022-10-25 10:52:29 -07:00