Commit Graph

191 Commits

Author SHA1 Message Date
Harshavardhana b210ea79bc
do not save MTime in newMultipartUpload() to avoid side-affects (#17340) 2023-06-02 14:38:09 -07:00
Poorna e95825a42e
replication: use latest object info for metrics update (#17333) 2023-06-01 18:52:55 -07:00
Poorna 2131046427
replication: fix audit log reporting (#17222) 2023-05-16 15:35:08 -07:00
Klaus Post aaf1abc993
simplify HardLimitReader by using LimitReader for internal usage (#17218) 2023-05-16 13:14:37 -07:00
jiuker 413549bcf5
fix: loadStatsFromDisk() should return nil for configNotFound (#17217) 2023-05-16 12:23:38 -07:00
Poorna e07c2ab868
Use hash.NewLimitReader for internal multipart calls (#17191) 2023-05-12 11:19:08 -07:00
Poorna c5c1426262
Validate if replication config being added is self referential (#17142) 2023-05-06 13:35:43 -07:00
Harshavardhana 6825bd7e75
fix: inlined objects don't need to honor long locks (#17039) 2023-04-17 12:16:37 -07:00
Harshavardhana c06e0bfef9
set correct `Host:` value for replication event notification (#16984) 2023-04-06 10:20:53 -07:00
Anis Eleuch d90d0c8931
Use one http response recorder per external http call (#16938) 2023-03-31 09:37:29 -07:00
Allan Roger Reid 483b226cc1
fix: avoid logging when object/version not found in replication (#16919) 2023-03-29 15:02:45 -07:00
Harshavardhana 8e02660a0d
update all our deps (#16899) 2023-03-28 03:45:24 -07:00
Poorna fb6ab1cca2
fix: allow replication of 'null' delete markers (#16773) 2023-03-08 07:03:29 -08:00
Poorna ee54643004
Avoid unnecessary replication heal attempts (#16769) 2023-03-07 07:43:38 -08:00
Poorna c33a237067
fix: under site replication disallow remote target modification (#16628) 2023-02-15 20:22:13 -08:00
jiuker a15b6f21b8
remove incorrect use of WaitGroup (#16596) 2023-02-12 20:59:45 -08:00
Poorna 876e1a91b2
replication: Fix typo checking PreconditionFailed status code (#16517) 2023-02-02 19:22:02 +05:30
Poorna 820d94447c
replication: fix target bucket passed on GET proxy (#16495) 2023-01-27 10:24:51 -08:00
Poorna ed20134a7b
replication: detect proxy header presence correctly (#16489) 2023-01-27 01:29:32 -08:00
Harshavardhana e64b9f6751
fix: disallow SSE-C encrypted objects on replicated buckets (#16467) 2023-01-24 15:46:33 -08:00
Poorna ddad231921
replication: Avoid logging PreConditionFailed error (#16450) 2023-01-21 07:33:04 +05:30
Poorna 1b02e046c2
Fix bandwidth monitoring to be per remote target (#16360) 2023-01-19 18:52:16 +05:30
Harshavardhana 2937711390
fix: DeleteObject() API with versionId under replication (#16325) 2022-12-28 22:48:33 -08:00
Anis Elleuch acc9c033ed
debug: Add X-Amz-Request-ID to lock/unlock calls (#16309) 2022-12-23 19:49:07 -08:00
Poorna de0b43de32
persist replication stats with leader lock (#16282) 2022-12-22 14:25:13 -08:00
Poorna 6423e4c767
Remove site replication config if it succeeded locally (#16279) 2022-12-22 01:31:20 -08:00
Harshavardhana 2fc182d8e6
fix: iso8601TimeFormat padding issue for certain nanoseconds (#16207) 2022-12-12 10:28:30 -08:00
Aditya Manthramurthy a30cfdd88f
Bump up madmin-go to v2 (#16162) 2022-12-06 13:46:50 -08:00
Klaus Post a713aee3d5
Run staticcheck on CI (#16170) 2022-12-05 11:18:50 -08:00
Klaus Post 1cd875de1e
Persist updated metadata (#16160) 2022-12-02 08:35:04 -08:00
Anis Elleuch 641ab24aec
repl: resync orchestrator to use global shared lock (#16154) 2022-12-01 12:10:09 -08:00
Klaus Post a22b4adf4c
distribute replication ops based on names (#16083) 2022-11-17 15:20:09 -08:00
Klaus Post b7bb122be8
fix: replication auto-scaling deadlock (#16084) 2022-11-17 07:35:02 -08:00
Klaus Post 8a07000e58
fix: refactor getReplicationDiff for safe use (#16051) 2022-11-15 07:59:21 -08:00
Poorna d6bc141bd1
feat: Add support for site level resync (#15753) 2022-11-14 07:16:40 -08:00
Poorna 34d28dd79f
replication: Avoid blocking on mrf save (#16045) 2022-11-10 10:20:02 -08:00
Klaus Post 2894dd4d1a
fix: hold lock while serializing replication stats (#16007) 2022-11-04 09:59:14 -07:00
Klaus Post 0f0e154315
fix: inconsistent replication delete marker timestamps (#15956) 2022-10-27 09:46:52 -07:00
Harshavardhana 23b329b9df
remove gateway completely (#15929) 2022-10-24 17:44:15 -07:00
Anis Elleuch fc6c794972
Audit dangling object removal (#15933) 2022-10-24 11:35:07 -07:00
Poorna e4e90b53c1
fix: delete-marker replication check properly (#15923) 2022-10-21 14:45:06 -07:00
Harshavardhana 59e33b3b21
validate setBucketTarget properly as per BucketExists() call (#15860) 2022-10-13 17:46:49 -07:00
Poorna 0e3c92c027 attempt delete marker replication after object is replicated (#15857)
Ensure delete marker replication success, especially since the
recent optimizations to heal on HEAD, LIST and GET can force
replication attempts on delete marker before underlying object
version could have synced.
2022-10-13 17:45:23 -07:00
Harshavardhana 97112c69be
fix: replication stats() to not crash under any situation (#15851)
Co-authored-by: Daniel Valdivia <18384552+dvaldivia@users.noreply.github.com>
2022-10-12 15:47:41 -07:00
Anis Elleuch e856e10ac2
ignore VersionNotFound in addition to ObjectNotFound while replicating (#15814) 2022-10-07 16:11:41 -07:00
Poorna 8ea6fb368d
Add auto configuration of replication workers (#15636) 2022-09-24 16:20:28 -07:00
Harshavardhana 50a8ba6a6f
fix: parse and save retainUntilDate in correct time format (#15741) 2022-09-23 08:49:27 -07:00
Harshavardhana 124544d834
add pre-conditions support for PUT calls during replication (#15674)
PUT shall only proceed if pre-conditions are met, the new
code uses

- x-minio-source-mtime
- x-minio-source-etag

to verify if the object indeed needs to be replicated
or not, allowing us to avoid StatObject() call.
2022-09-14 18:44:04 -07:00
Poorna a0fb0c1835
panic if replication config could not be read from disk (#15685)
If replication config could not be read from bucket metadata for some
reason, issue a panic so that unexpected replication outcomes can
be avoided for replicated buckets.

For similar reasons, adding a panic while fetching object-lock config
if it failed for reason other than non-existence of config.
2022-09-13 21:23:33 -07:00
Poorna 6b9fd256e1
Persist in-memory replication stats to disk (#15594)
to avoid relying on scanner-calculated replication metrics.
This will improve the accuracy of the replication stats reported.

This PR also adds on to #15556 by handing replication
traffic that could not be queued by available workers to the 
MRF queue so that entries in `PENDING` status are healed faster.
2022-09-12 12:40:02 -07:00