MyFavMirrors
/
minio
mirror of https://github.com/minio/minio.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
10,974 Commits 5 Branches 503 Tags 186 MiB
Commit Graph

207 Commits

Author SHA1 Message Date
Poorna dfaf735073
replication: fix queuing of large uploads (#17831) 
Fixes regression from #17687
 2023-08-10 15:48:42 -07:00
Harshavardhana b732a673dc
reduce logging in bucket replication in retry scenarios (#17820) 2023-08-08 13:27:40 -07:00
Poorna 26c23b30f4
replication: set context timeout for NewMultipartUpload calls (#17807) 2023-08-05 12:27:07 -07:00
Poorna 311380f8cb
replication resync: fix queueing (#17775) 
Assign resync of all versions of object to the same worker to avoid locking
contention. Fixes parallel resync implementation in #16707
 2023-08-01 11:51:15 -07:00
Poorna 1a42693d68
replication: limit larger uploads to a subset of workers (#17687) 
Limit large uploads (> 128MiB)  to a max of 10 workers, intent is to avoid
larger uploads from using all replication bandwidth, giving room for smaller
uploads to sync faster.
 2023-07-25 20:02:02 -07:00
Harshavardhana 005a4a275a
add more bootstrap messages to provide latency (#17650) 
- simplify refreshing bucket metadata, wait() to
  depend on how fast the bucket metadata can load.

- simplify resync to start resync in single pass.
 2023-07-14 04:00:29 -07:00
Poorna 5e2f8d7a42
replication: Simplify mrf requeueing and add backlog handler (#17171) 
Simplify MRF queueing and add backlog handler

- Limit re-tries to 3 to avoid repeated re-queueing. Fall offs
to be re-tried when the scanner revisits this object or upon access.

- Change MRF to have each node process only its MRF entries.

- Collect MRF backlog by the node to allow for current backlog visibility
 2023-07-12 23:51:33 -07:00
Kaan Kabalak f64d62b01d
Fix style of logOnceIf calls w/unique identifiers (#17631) 2023-07-11 13:17:45 -07:00
Poorna e8c98c3246
Avoid extra GetObjectInfo call in DeleteObject API (#17599) 
Optimize DeleteObject API to avoid extra 
GetObjectInfo call on the replicating side.

For receiving side, it is just a regular
DeleteObject call.

Bonus: Fix a corner case where version purged is 
absent on target (either due to replication not yet
complete or target version already deleted in a
one-way replication or when replication was disabled). 

In such cases, mark version purge complete.
 2023-07-10 07:57:56 -07:00
Klaus Post ff5988f4e0
Reduce allocations (#17584) 
* Reduce allocations

* Add stringsHasPrefixFold which can compare string prefixes, while ignoring case and not allocating.
* Reuse all msgp.Readers
* Reuse metadata buffers when not reading data.

* Make type safe. Make buffer 4K instead of 8.

* Unslice
 2023-07-06 16:02:08 -07:00
Kaan Kabalak 21fbe88e1f
Print certain log messages once per error (#17484) 2023-06-24 20:29:13 -07:00
jiuker b6b68be052
fix: replication check for duplicate endpoints detection with wrong route (#17474) 2023-06-20 09:27:54 -07:00
Aditya Manthramurthy 5a1612fe32
Bump up madmin-go and pkg deps (#17469) 2023-06-19 17:53:08 -07:00
Harshavardhana 1443b5927a
allow quorum fileInfo to pick same parityBlocks (#17454) 
Bonus: allow replication to proceed for 503 errors such as
with error code SlowDownRead
 2023-06-18 18:20:15 -07:00
Poorna c4d0c49a5f
ensure metadata updates go to same pool where version exists (#17451) 
This PR also returns the replication status in 
proxy calls and defers replication attempt if 
HEAD on object version returned a error different
from NoSuchKey
 2023-06-17 07:30:53 -07:00
Poorna Krishnamoorthy f986b0c493 replication: perform bucket resync in parallel (#16707) 
Default number of parallel resync operations for a bucket to 10
to speed up resync.
 2023-06-11 16:09:55 -07:00
Harshavardhana b210ea79bc
do not save MTime in newMultipartUpload() to avoid side-affects (#17340) 2023-06-02 14:38:09 -07:00
Poorna e95825a42e
replication: use latest object info for metrics update (#17333) 2023-06-01 18:52:55 -07:00
Poorna 2131046427
replication: fix audit log reporting (#17222) 2023-05-16 15:35:08 -07:00
Klaus Post aaf1abc993
simplify HardLimitReader by using LimitReader for internal usage (#17218) 2023-05-16 13:14:37 -07:00
jiuker 413549bcf5
fix: loadStatsFromDisk() should return nil for configNotFound (#17217) 2023-05-16 12:23:38 -07:00
Poorna e07c2ab868
Use hash.NewLimitReader for internal multipart calls (#17191) 2023-05-12 11:19:08 -07:00
Poorna c5c1426262
Validate if replication config being added is self referential (#17142) 2023-05-06 13:35:43 -07:00
Harshavardhana 6825bd7e75
fix: inlined objects don't need to honor long locks (#17039) 2023-04-17 12:16:37 -07:00
Harshavardhana c06e0bfef9
set correct `Host:` value for replication event notification (#16984) 2023-04-06 10:20:53 -07:00
Anis Eleuch d90d0c8931
Use one http response recorder per external http call (#16938) 2023-03-31 09:37:29 -07:00
Allan Roger Reid 483b226cc1
fix: avoid logging when object/version not found in replication (#16919) 2023-03-29 15:02:45 -07:00
Harshavardhana 8e02660a0d
update all our deps (#16899) 2023-03-28 03:45:24 -07:00
Poorna fb6ab1cca2
fix: allow replication of 'null' delete markers (#16773) 2023-03-08 07:03:29 -08:00
Poorna ee54643004
Avoid unnecessary replication heal attempts (#16769) 2023-03-07 07:43:38 -08:00
Poorna c33a237067
fix: under site replication disallow remote target modification (#16628) 2023-02-15 20:22:13 -08:00
jiuker a15b6f21b8
remove incorrect use of WaitGroup (#16596) 2023-02-12 20:59:45 -08:00
Poorna 876e1a91b2
replication: Fix typo checking PreconditionFailed status code (#16517) 2023-02-02 19:22:02 +05:30
Poorna 820d94447c
replication: fix target bucket passed on GET proxy (#16495) 2023-01-27 10:24:51 -08:00
Poorna ed20134a7b
replication: detect proxy header presence correctly (#16489) 2023-01-27 01:29:32 -08:00
Harshavardhana e64b9f6751
fix: disallow SSE-C encrypted objects on replicated buckets (#16467) 2023-01-24 15:46:33 -08:00
Poorna ddad231921
replication: Avoid logging PreConditionFailed error (#16450) 2023-01-21 07:33:04 +05:30
Poorna 1b02e046c2
Fix bandwidth monitoring to be per remote target (#16360) 2023-01-19 18:52:16 +05:30
Harshavardhana 2937711390
fix: DeleteObject() API with versionId under replication (#16325) 2022-12-28 22:48:33 -08:00
Anis Elleuch acc9c033ed
debug: Add X-Amz-Request-ID to lock/unlock calls (#16309) 2022-12-23 19:49:07 -08:00
Poorna de0b43de32
persist replication stats with leader lock (#16282) 2022-12-22 14:25:13 -08:00
Poorna 6423e4c767
Remove site replication config if it succeeded locally (#16279) 2022-12-22 01:31:20 -08:00
Harshavardhana 2fc182d8e6
fix: iso8601TimeFormat padding issue for certain nanoseconds (#16207) 2022-12-12 10:28:30 -08:00
Aditya Manthramurthy a30cfdd88f
Bump up madmin-go to v2 (#16162) 2022-12-06 13:46:50 -08:00
Klaus Post a713aee3d5
Run staticcheck on CI (#16170) 2022-12-05 11:18:50 -08:00
Klaus Post 1cd875de1e
Persist updated metadata (#16160) 2022-12-02 08:35:04 -08:00
Anis Elleuch 641ab24aec
repl: resync orchestrator to use global shared lock (#16154) 2022-12-01 12:10:09 -08:00
Klaus Post a22b4adf4c
distribute replication ops based on names (#16083) 2022-11-17 15:20:09 -08:00
Klaus Post b7bb122be8
fix: replication auto-scaling deadlock (#16084) 2022-11-17 07:35:02 -08:00
Klaus Post 8a07000e58
fix: refactor getReplicationDiff for safe use (#16051) 2022-11-15 07:59:21 -08:00