Commit Graph

4734 Commits

Author SHA1 Message Date
Harshavardhana
97112c69be
fix: replication stats() to not crash under any situation (#15851)
Co-authored-by: Daniel Valdivia <18384552+dvaldivia@users.noreply.github.com>
2022-10-12 15:47:41 -07:00
Javier Adriel
2939000342
Add metrics, version and apis handlers (#15839) 2022-10-12 12:08:03 -07:00
Harshavardhana
41e1654f9a
remove spurious logging for object not found (#15842) 2022-10-12 04:28:21 -07:00
Harshavardhana
e3cb0278ce
honor specified target prefix under batch replication (#15834) 2022-10-11 14:36:06 -07:00
Harshavardhana
0c81f1bdb3
indicate how long it took to bring the drive online (#15835) 2022-10-11 11:33:56 -07:00
Klaus Post
6220875803
Add missing server info fields (#15826) 2022-10-11 11:31:26 -07:00
Aditya Manthramurthy
64cf887b28
use LDAP config from minio/pkg to share with console (#15810) 2022-10-07 22:12:36 -07:00
Harshavardhana
927a879052
authenticate the request first for headObject() (#15820) 2022-10-07 21:45:53 -07:00
Anis Elleuch
dfe0c96b87
preserve Version and DeleteMarker sort order in the list XML response (#15819) 2022-10-07 16:12:36 -07:00
Anis Elleuch
e856e10ac2
ignore VersionNotFound in addition to ObjectNotFound while replicating (#15814) 2022-10-07 16:11:41 -07:00
Harshavardhana
928feb0889
remove unused debug param from evalActionFromLifecycle (#15813) 2022-10-07 10:24:12 -07:00
Anis Elleuch
158d0e26a2
decom: Ignore object/version error during deletion (#15806) 2022-10-06 09:41:58 -07:00
Harshavardhana
78385bfbeb
set bucket creation timestamp properly for legacy FS backend (#15800) 2022-10-06 02:46:31 -07:00
Harshavardhana
2a13cc28f2 feat: implement support batch replication (#15554) 2022-10-05 23:00:43 -07:00
Lenin Alevski
4bdf41a6c7
Removing unused getUpdateReaderFromFile function (#15794)
Signed-off-by: Lenin Alevski <alevsk.8772@gmail.com>
2022-10-05 07:58:27 -07:00
Klaus Post
3c605c93fe
warn when 0 parity has been set as default parity (#15790) 2022-10-04 22:41:42 -07:00
Anis Elleuch
121f18a443
Use admin request check for ReplicationDiff handler (#15793) 2022-10-04 17:47:31 -07:00
Harshavardhana
538aeef27a
fix: heal service accounts for LDAP users in site replication (#15785) 2022-10-04 10:41:47 -07:00
Poorna
be0d2537b7
site replication: fix typo in meta collection (#15792) 2022-10-04 10:19:17 -07:00
Javier Adriel
3307aa1260
Implement KMS handlers (#15737) 2022-10-04 10:05:09 -07:00
Harshavardhana
57cfdfd8fb
remove 'perf' tests from health diagnostics (#15780) 2022-10-03 00:18:41 -07:00
Harshavardhana
f696a221af
allow tagging policy condition for GetObject (#15777) 2022-10-02 12:29:29 -07:00
Harshavardhana
2aac50571d
fix: de-duplicate conflicting object names on namespace (#15772) 2022-09-30 15:44:21 -07:00
Shireesh Anjal
45edd27ad7
Re-load config after 'mc admin config reset' (#15771) 2022-09-30 10:55:53 -07:00
Daryl White
d44f3526dc
Update links to documentation site (#15750) 2022-09-28 21:28:45 -07:00
Harshavardhana
41b633f5ea
support tagging based policy conditions (#15763) 2022-09-28 11:25:46 -07:00
Anis Elleuch
86bb48792c
non-blocking initialization of bucket target notifications (#15571) 2022-09-27 17:23:28 -07:00
Harshavardhana
94dbb4a427
fix: generalize SC config and also skip healing sub-sys under SD (#15757) 2022-09-26 09:04:54 -07:00
Anis Elleuch
048a46ec2a
Add RPC tcp timeout/errs and AVG duration to prometheus (#15747) 2022-09-26 09:04:26 -07:00
Poorna
8ea6fb368d
Add auto configuration of replication workers (#15636) 2022-09-24 16:20:28 -07:00
Harshavardhana
b04c0697e1
validate correct ETag for the parts sent during CompleteMultipart (#15751) 2022-09-23 21:17:08 -07:00
Harshavardhana
50a8ba6a6f
fix: parse and save retainUntilDate in correct time format (#15741) 2022-09-23 08:49:27 -07:00
Anis Elleuch
20c89ebbb3
freeze before exit when _MINIO_DEBUG_NO_EXIT is defined (#15709)
this is to ensure keep k8s pods running, when they reach a "crashloop" stage
2022-09-22 11:57:27 -07:00
Krishnan Parthasarathi
6f56ba80b3
lifecycle: Assign unique id to rules with empty id (#15731) 2022-09-22 10:51:54 -07:00
Anis Elleuch
6e84283c66
fix: ignoring O_DIRECT in case of erasure single disk (#15734)
fixes #15733 
fixes #15735
2022-09-22 10:41:06 -07:00
Harshavardhana
9d6fddcfdf
persist the non-default creds in config (#15711) 2022-09-21 16:14:47 -07:00
jiuker
749ce107ee
fix: context leak with replication endpoint hearbeat (#15721) 2022-09-21 03:08:45 -07:00
Poorna
aec2aa3497
site replication: clear config if remove --all specified (#15716) 2022-09-20 14:32:23 -07:00
Klaus Post
ff12080ff5
Remove deprecated io/ioutil (#15707) 2022-09-19 11:05:16 -07:00
Minio Trusted
d89f6af6c4 avoid replication stats crash in Prometheus 2022-09-16 17:09:45 -07:00
Harshavardhana
2c68a19dfd
upgrade all deps and update CREDITS (#15650) 2022-09-16 01:59:45 -07:00
Harshavardhana
9e5853ecc0
optimize double reads by reusing results from checkUploadIDExists() (#15692)
Move to using `xl.meta` data structure to keep temporary partInfo,
this allows for a future change where we move to different parts to
different drives.
2022-09-15 12:43:49 -07:00
Harshavardhana
124544d834
add pre-conditions support for PUT calls during replication (#15674)
PUT shall only proceed if pre-conditions are met, the new
code uses

- x-minio-source-mtime
- x-minio-source-etag

to verify if the object indeed needs to be replicated
or not, allowing us to avoid StatObject() call.
2022-09-14 18:44:04 -07:00
Poorna
b910904fa6
change replication stats save path for windows (#15690) 2022-09-14 13:49:13 -07:00
Klaus Post
eee1ce305c
When listing, do not count delete markers (#15689)
When limiting listing do not count delete, since they may be discarded.

Extend limit, since we may be discarding the forward-to marker.

Fix directories always being sent to resolve, since they didn't return as match.
2022-09-14 12:11:27 -07:00
Klaus Post
5c61c3ccdc
Fix flaky TestGetObjectWithOutdatedDisks (#15687)
On occasion this test fails:

```
2022-09-12T17:22:44.6562737Z === RUN   TestGetObjectWithOutdatedDisks
2022-09-12T17:22:44.6563751Z     erasure-object_test.go:1214: Test 2: Expected data to have md5sum = `c946b71bb69c07daf25470742c967e7c`, found `7d16d23f07072af1a809707ba101ae07`
2
```

Theory: Both objects are written with the same timestamp due to lower timer resolution on Windows. This results in secondary resolution, which is deterministic, but random.

Solution: Instead of hacking in a wait we request the specific version we want. Should still keep the test relevant.

Bonus: Remote action dependency for vulncheck
2022-09-14 08:17:39 -07:00
Poorna
a0fb0c1835
panic if replication config could not be read from disk (#15685)
If replication config could not be read from bucket metadata for some
reason, issue a panic so that unexpected replication outcomes can
be avoided for replicated buckets.

For similar reasons, adding a panic while fetching object-lock config
if it failed for reason other than non-existence of config.
2022-09-13 21:23:33 -07:00
Aditya Manthramurthy
e152b2a975
Pass groups claim into condition values (#15679)
This allows using `jwt:groups` as a multi-valued condition key in policies.
2022-09-13 09:45:36 -07:00
Poorna
6b9fd256e1
Persist in-memory replication stats to disk (#15594)
to avoid relying on scanner-calculated replication metrics.
This will improve the accuracy of the replication stats reported.

This PR also adds on to #15556 by handing replication
traffic that could not be queued by available workers to the 
MRF queue so that entries in `PENDING` status are healed faster.
2022-09-12 12:40:02 -07:00
Klaus Post
ff9a74b91f
Add fast max-keys=1 support for Listing (#15670)
Add a listing option to stop when the limit is reached.  
This can be used by stateless listings for fast results.
2022-09-09 08:13:06 -07:00