Commit Graph

4842 Commits

Author SHA1 Message Date
Harshavardhana 9547b7d0e9
add deadlineConnections on remoteTransport (#16010) 2022-11-05 11:09:21 -07:00
Klaus Post 808ecfe0f2
merge versions across sets when listing (#16003) 2022-11-04 11:33:22 -07:00
Klaus Post 2894dd4d1a
fix: hold lock while serializing replication stats (#16007) 2022-11-04 09:59:14 -07:00
jiuker fd8750e959
fix: http body must be drained in downloadBinary() (#16001) 2022-11-04 08:22:38 -07:00
Poorna 4f5d38a4b1
site replication edit: validate endpoint belongs to deployment (#16000) 2022-11-03 16:23:45 -07:00
Anis Elleuch 7e73fc2870
Implement inspect data API v2 (#15474)
Co-authored-by: Klaus Post <klauspost@gmail.com>
2022-11-02 13:36:38 -07:00
Harshavardhana 0d49b365ff
converge SNSD deployments into single code (#15988) 2022-11-01 16:41:01 -07:00
Anis Elleuch 7721595aa9
config: Deprecated delay/max_wait/scanner and introduce speed (#15941) 2022-11-01 08:04:07 -07:00
Harshavardhana fd6f6fc8df
cleanup stale parent multipart directories (#15980) 2022-11-01 08:00:02 -07:00
Aditya Manthramurthy 4fb47cd568
fix: update admin IDP APIs to be more RESTful (#15896) 2022-10-31 14:52:26 -07:00
Klaus Post ecc932d5dd
Clean entire tmp-old on restart (#15979) 2022-10-31 07:27:50 -07:00
Harshavardhana b57fbff7c1
ignore background healInfo in single drive setup (#15968) 2022-10-31 07:26:10 -07:00
Poorna d765b89a63
improve validation for replication resync API (#15964) 2022-10-28 23:21:33 -07:00
Harshavardhana 6e4acf0504
add a message of removal for gateway and hide the command (#15965) 2022-10-28 14:11:20 -07:00
Klaus Post 71954faa3a
mark pubsub type safe via generics (#15961) 2022-10-28 10:55:42 -07:00
Klaus Post 0f0e154315
fix: inconsistent replication delete marker timestamps (#15956) 2022-10-27 09:46:52 -07:00
Harshavardhana 136d41775f
remove numAvailableDisks check as it doesn't serve any purpose (#15954) 2022-10-27 09:05:24 -07:00
Harshavardhana ec77d28e62
make subnet subsys dynamic and simplify callhome (#15927) 2022-10-27 00:20:01 -07:00
Klaus Post 86420a1f46
Store multipart checksums (#15953) 2022-10-26 18:14:58 -07:00
Poorna 7dd8b6c8ed
ensure ILM expiry creates non null deleteMarker for versioned bucket (#15947) 2022-10-26 16:09:27 -07:00
Anis Elleuch 533c9d4fe3
fix: lockName to disallow parallel same erasure set healing (#15951) 2022-10-26 12:43:54 -07:00
Anis Elleuch a35ef155fc
return appropriate error status code in the lock handler (#15950) 2022-10-26 09:51:26 -07:00
Poorna 8dd3c41b2a
allow MakeBucket errors to be handled lazily (#15945)
remote error is not required to be passed back to the 
client - this is mostly because we have healing that should 
eventually, catch up on this and heal the bucket.
2022-10-25 23:32:37 -07:00
Krishnan Parthasarathi 4523da6543
feat: introduce pool-level rebalance (#15483) 2022-10-25 12:36:57 -07:00
Poorna ce8456a1a9
proxy multipart to peers via multipart uploadID (#15926) 2022-10-25 10:52:29 -07:00
Poorna 9ce1884732
reject editing bucket replication config when site replication is enabled (#15937) 2022-10-24 20:24:32 -07:00
Harshavardhana 23b329b9df
remove gateway completely (#15929) 2022-10-24 17:44:15 -07:00
Krishnan Parthasarathi 0c34e51a75
Filter out tiering metadata during CopyObject (#15936) 2022-10-24 16:32:31 -07:00
Anis Elleuch fc6c794972
Audit dangling object removal (#15933) 2022-10-24 11:35:07 -07:00
Klaus Post 86d543d0f6
Check for s3zip content offset (#15924) 2022-10-21 15:37:48 -07:00
Poorna e4e90b53c1
fix: delete-marker replication check properly (#15923) 2022-10-21 14:45:06 -07:00
Anis Elleuch 58d776daa0
Set CONSOLE_MINIO_SERVER to 127.0.0.1 by default (#15887) 2022-10-21 14:42:28 -07:00
Krishnan Parthasarathi f6b2e89109
Pass encrypted etag as is for immediate tiering (#15925) 2022-10-21 14:40:50 -07:00
Anis Elleuch ac85c2af76
lifecycle: refactor rules filtering and tagging support (#15914) 2022-10-21 10:46:53 -07:00
Shireesh Anjal 5aba2aedb3
Do not freeze s3 traffic in healthinfo api (#15912) 2022-10-21 00:34:32 -07:00
Harshavardhana a8332efa94
fix: Delete() of bucket metadata should not parse the config (#15904) 2022-10-19 17:55:09 -07:00
Aditya Manthramurthy 3dbef72dc7
fix: AccountInfo API for roleARN based accounts (#15907) 2022-10-19 17:54:41 -07:00
Aditya Manthramurthy 2d16e74f38
Add LDAP IDP Configuration APIs (#15840) 2022-10-19 11:00:10 -07:00
Anis Elleuch de5070446d
Deprecate --listeners flag (#15900) 2022-10-19 08:45:50 -07:00
Harshavardhana 374abd1e7d
add filter support for tags and metadata in batch replication (#15885) 2022-10-18 21:22:21 -07:00
Anis Elleuch 0506d9e83d
storage: Return errDiskNotFound when a peer is during shutdown (#15868) 2022-10-18 13:50:46 -07:00
Klaus Post bd3dfad8b9
Add concurrent Snowball extraction + options (#15836) 2022-10-18 13:50:21 -07:00
Harshavardhana 9fff315555
do not need to trace ignored objects (#15894) 2022-10-18 13:47:55 -07:00
Anis Elleuch 18fb86b7be
convert context.DeadlineExceed to offline disk in DiskInfo() (#15886) 2022-10-18 03:01:16 -07:00
Harshavardhana 58a8275e84
do not assume invalid buf to be non-xl.meta (#15843) 2022-10-17 09:39:21 -07:00
Aditya Manthramurthy 85fc7cea97
Pass role ARN for OIDC providers to console (#15862) 2022-10-15 12:57:03 -07:00
Harshavardhana 328d660106
support CRC32 Checksums on single drive setup (#15873) 2022-10-15 11:58:47 -07:00
Harshavardhana c68910005b
validate bucket before attempting batch replication (#15861) 2022-10-15 11:58:31 -07:00
Harshavardhana c79bcc8838 Revert "convert context.DeadlineExceed to offline disk in DiskInfo() (#15869)"
This reverts commit 0fe58dbb34.
2022-10-14 20:37:50 -07:00
Anis Elleuch 0fe58dbb34
convert context.DeadlineExceed to offline disk in DiskInfo() (#15869) 2022-10-14 19:32:13 -07:00
Harshavardhana 6cb2f56395 Revert "Revert "tests: Add context cancelation (#15374)""
This reverts commit 564a0afae1.
2022-10-14 03:08:40 -07:00
Harshavardhana 59e33b3b21
validate setBucketTarget properly as per BucketExists() call (#15860) 2022-10-13 17:46:49 -07:00
Poorna 0e3c92c027 attempt delete marker replication after object is replicated (#15857)
Ensure delete marker replication success, especially since the
recent optimizations to heal on HEAD, LIST and GET can force
replication attempts on delete marker before underlying object
version could have synced.
2022-10-13 17:45:23 -07:00
Anis Elleuch db7a9b2c37
heal-info: Return the endpoint of a disk with unknown state (#15854) 2022-10-13 16:41:44 -07:00
Harshavardhana 44097faec1
support deleteMarkers and all versions in batch replication (#15858) 2022-10-13 14:42:10 -07:00
Klaus Post bf3da5081f
Omit empty checksums in responses (#15850) 2022-10-13 00:49:46 -07:00
Harshavardhana 5532982857
do not disable IsKubernetes(), IsDocker() checks with MINIO_CI_CD (#15852) 2022-10-12 23:40:48 -07:00
Anis Elleuch 783dd875f7
refactor objectQuorumFromMeta() to search for parity quorum (#15844) 2022-10-12 16:42:45 -07:00
Harshavardhana 97112c69be
fix: replication stats() to not crash under any situation (#15851)
Co-authored-by: Daniel Valdivia <18384552+dvaldivia@users.noreply.github.com>
2022-10-12 15:47:41 -07:00
Javier Adriel 2939000342
Add metrics, version and apis handlers (#15839) 2022-10-12 12:08:03 -07:00
Harshavardhana 41e1654f9a
remove spurious logging for object not found (#15842) 2022-10-12 04:28:21 -07:00
Harshavardhana e3cb0278ce
honor specified target prefix under batch replication (#15834) 2022-10-11 14:36:06 -07:00
Harshavardhana 0c81f1bdb3
indicate how long it took to bring the drive online (#15835) 2022-10-11 11:33:56 -07:00
Klaus Post 6220875803
Add missing server info fields (#15826) 2022-10-11 11:31:26 -07:00
Aditya Manthramurthy 64cf887b28
use LDAP config from minio/pkg to share with console (#15810) 2022-10-07 22:12:36 -07:00
Harshavardhana 927a879052
authenticate the request first for headObject() (#15820) 2022-10-07 21:45:53 -07:00
Anis Elleuch dfe0c96b87
preserve Version and DeleteMarker sort order in the list XML response (#15819) 2022-10-07 16:12:36 -07:00
Anis Elleuch e856e10ac2
ignore VersionNotFound in addition to ObjectNotFound while replicating (#15814) 2022-10-07 16:11:41 -07:00
Harshavardhana 928feb0889
remove unused debug param from evalActionFromLifecycle (#15813) 2022-10-07 10:24:12 -07:00
Anis Elleuch 158d0e26a2
decom: Ignore object/version error during deletion (#15806) 2022-10-06 09:41:58 -07:00
Harshavardhana 78385bfbeb
set bucket creation timestamp properly for legacy FS backend (#15800) 2022-10-06 02:46:31 -07:00
Harshavardhana 2a13cc28f2 feat: implement support batch replication (#15554) 2022-10-05 23:00:43 -07:00
Lenin Alevski 4bdf41a6c7
Removing unused getUpdateReaderFromFile function (#15794)
Signed-off-by: Lenin Alevski <alevsk.8772@gmail.com>
2022-10-05 07:58:27 -07:00
Klaus Post 3c605c93fe
warn when 0 parity has been set as default parity (#15790) 2022-10-04 22:41:42 -07:00
Anis Elleuch 121f18a443
Use admin request check for ReplicationDiff handler (#15793) 2022-10-04 17:47:31 -07:00
Harshavardhana 538aeef27a
fix: heal service accounts for LDAP users in site replication (#15785) 2022-10-04 10:41:47 -07:00
Poorna be0d2537b7
site replication: fix typo in meta collection (#15792) 2022-10-04 10:19:17 -07:00
Javier Adriel 3307aa1260
Implement KMS handlers (#15737) 2022-10-04 10:05:09 -07:00
Harshavardhana 57cfdfd8fb
remove 'perf' tests from health diagnostics (#15780) 2022-10-03 00:18:41 -07:00
Harshavardhana f696a221af
allow tagging policy condition for GetObject (#15777) 2022-10-02 12:29:29 -07:00
Harshavardhana 2aac50571d
fix: de-duplicate conflicting object names on namespace (#15772) 2022-09-30 15:44:21 -07:00
Shireesh Anjal 45edd27ad7
Re-load config after 'mc admin config reset' (#15771) 2022-09-30 10:55:53 -07:00
Daryl White d44f3526dc
Update links to documentation site (#15750) 2022-09-28 21:28:45 -07:00
Harshavardhana 41b633f5ea
support tagging based policy conditions (#15763) 2022-09-28 11:25:46 -07:00
Anis Elleuch 86bb48792c
non-blocking initialization of bucket target notifications (#15571) 2022-09-27 17:23:28 -07:00
Harshavardhana 94dbb4a427
fix: generalize SC config and also skip healing sub-sys under SD (#15757) 2022-09-26 09:04:54 -07:00
Anis Elleuch 048a46ec2a
Add RPC tcp timeout/errs and AVG duration to prometheus (#15747) 2022-09-26 09:04:26 -07:00
Poorna 8ea6fb368d
Add auto configuration of replication workers (#15636) 2022-09-24 16:20:28 -07:00
Harshavardhana b04c0697e1
validate correct ETag for the parts sent during CompleteMultipart (#15751) 2022-09-23 21:17:08 -07:00
Harshavardhana 50a8ba6a6f
fix: parse and save retainUntilDate in correct time format (#15741) 2022-09-23 08:49:27 -07:00
Anis Elleuch 20c89ebbb3
freeze before exit when _MINIO_DEBUG_NO_EXIT is defined (#15709)
this is to ensure keep k8s pods running, when they reach a "crashloop" stage
2022-09-22 11:57:27 -07:00
Krishnan Parthasarathi 6f56ba80b3
lifecycle: Assign unique id to rules with empty id (#15731) 2022-09-22 10:51:54 -07:00
Anis Elleuch 6e84283c66
fix: ignoring O_DIRECT in case of erasure single disk (#15734)
fixes #15733 
fixes #15735
2022-09-22 10:41:06 -07:00
Harshavardhana 9d6fddcfdf
persist the non-default creds in config (#15711) 2022-09-21 16:14:47 -07:00
jiuker 749ce107ee
fix: context leak with replication endpoint hearbeat (#15721) 2022-09-21 03:08:45 -07:00
Poorna aec2aa3497
site replication: clear config if remove --all specified (#15716) 2022-09-20 14:32:23 -07:00
Klaus Post ff12080ff5
Remove deprecated io/ioutil (#15707) 2022-09-19 11:05:16 -07:00
Minio Trusted d89f6af6c4 avoid replication stats crash in Prometheus 2022-09-16 17:09:45 -07:00
Harshavardhana 2c68a19dfd
upgrade all deps and update CREDITS (#15650) 2022-09-16 01:59:45 -07:00
Harshavardhana 9e5853ecc0
optimize double reads by reusing results from checkUploadIDExists() (#15692)
Move to using `xl.meta` data structure to keep temporary partInfo,
this allows for a future change where we move to different parts to
different drives.
2022-09-15 12:43:49 -07:00