MyFavMirrors
/
minio
mirror of https://github.com/minio/minio.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
11,613 Commits 5 Branches 503 Tags 186 MiB
Commit Graph

89 Commits

Author SHA1 Message Date
Anis Eleuch 68dde2359f
log: Add logger.Event to send to console and other logger targets (#19060) 
Add a new function logger.Event() to send the log to Console and
http/kafka log webhooks. This will include some internal events such as
disk healing and rebalance/decommissioning
 2024-02-15 15:13:30 -08:00
Anis Eleuch 6ae97aedc9
xl: Disable rename2 in decommissioning/rebalance (#18964) 
Always disable rename2 optimization in decom/rebalance
 2024-02-03 14:03:30 -08:00
Harshavardhana 32e668eb94
update() stale rebalance stats() object during pool expansion (#18882) 
it is entirely possible that a rebalance process which was running
when it was asked to "stop" it failed to write its last statistics
to the disk.

After this a pool expansion can cause disruption and all S3 API
calls would fail at IsPoolRebalancing() function.

This PRs makes sure that we update rebalance.bin under such
conditions to avoid any runtime crashes.
 2024-01-27 10:14:03 -08:00
Harshavardhana da55499db0
fix: reject clients that do not send proper payload (#18701) 2023-12-22 01:26:17 -08:00
Harshavardhana 109a9e3f35
skip ILM expired objects from healing (#18569) 2023-12-01 07:56:24 -08:00
Anis Eleuch 8317557f70
decom: Fix listing quorum to be equal to deletion quorum (#18476) 
With an odd number of drives per erasure set setup, the write/quorum is
the half + 1; however the decommissioning listing will still list those
objects and does not consider those as stale.

Fix it by using (N+1)/2 formula.

Co-authored-by: Anis Elleuch <anis@min.io>
 2023-11-17 21:09:09 -08:00
Klaus Post 9a877734b2
Fix various poolmeta races (#18230) 
There is a fundamental race condition in `newErasureServerPools`, where setObjectLayer is 
called before the poolMeta has been loaded/populated.

We add a placeholder value to this field but disable all saving of the value, so we don't risk 
overwriting the value on disk. Once the value has been loaded or created, it is replaced with 
the proper value, which will also be saved.

Also fixes various accesses of `poolMeta` that were done without locks.

We make the `poolMeta.IsSuspended` return false, even if we shouldn't risk out-of-bounds 
reads anymore.
 2023-10-12 15:30:42 -07:00
Poorna 9dc29d7687
Avoid ILM expiry on deleted versions that are yet to replicate (#18175) 
Fixes #18167
 2023-10-06 06:55:15 -06:00
Anis Eleuch 22d2dbc4e6
decom: Fix infinite retry when the decom is canceled (#18143) 
Also, use rand.Float64() since it is thread-safe; otherwise go race
will complain.
 2023-09-30 00:02:29 -07:00
jiuker 9947c01c8e
feat: SSE-KMS use uuid instead of read all data to md5. (#17958) 2023-09-18 10:00:54 -07:00
Harshavardhana fa6d082bfd
reduce all major allocations in replication path (#18032) 
- remove targetClient for passing around via replicationObjectInfo{}
- remove cloing to object info unnecessarily
- remove objectInfo from replicationObjectInfo{} (only require necessary fields)
 2023-09-16 02:28:06 -07:00
Aditya Manthramurthy 1c99fb106c
Update to minio/pkg/v2 (#17967) 2023-09-04 12:57:37 -07:00
Harshavardhana bddd53d6d2
fix: retry listing in decommissioning if it fails perpetually (#17682) 2023-07-19 13:09:37 -07:00
Harshavardhana dfd7cca0d2
fix: allow cancel of decom only when its in progress (#17607) 2023-07-10 07:55:38 -07:00
Harshavardhana d2f5c3621f
fix: add additional decommission traces for ILM expired content (#17522) 
current decommission traces were missing for

- Skipped ILM expired versions
- Skipped single DELETE marked version
- A success or failure in decommissioning DELETE marker
- allow additional info to be shared in DecomStatus() API
 2023-06-27 11:59:40 -07:00
Harshavardhana eefa047974
fix: keep decommission in a go-routine (#17496) 
This was removed by mistake in #17491
 2023-06-23 12:29:32 -07:00
Harshavardhana d315d012a4
decom: during multiple pool decom preserve current pool status (#17491) 
removal of completed pools must retain pool status of other
pools in draining, to resume any remaining draining operations.
 2023-06-23 07:44:18 -07:00
Harshavardhana 9af6c6ceef
under rebalance look for expired versions v/s remaining versions (#17482) 
A continuation of PR #17479 for rebalance behavior must
also match the decommission behavior.

Fixes bug where rebalance would ignore rebalancing object
versions after one of the version returned "ObjectNotFound"
 2023-06-21 13:23:20 -07:00
Harshavardhana ccc5801112
always look for expired versions v/s remaining versions (#17479) 
while decommissioning it can so happen that the non-current
versions are all expired but there is a DEL marker as the
latest version.

For such objects, we should not decommission them instead
calculate the remaining versions and if the remaining versions
is one and that version is a DEL marker consider such
an object not to be scheduled for decommissioning.
 2023-06-21 08:49:28 -07:00
Harshavardhana 15911c85f6
safely ignore out of band deletions while decommissioning (#17473) 2023-06-20 08:31:42 -07:00
Aditya Manthramurthy 5a1612fe32
Bump up madmin-go and pkg deps (#17469) 2023-06-19 17:53:08 -07:00
Anis Eleuch 38342b1df5
decom: Parallelize decommissining (#17364) 2023-06-07 14:27:51 -07:00
Anis Eleuch 1436858347
log: Add a log when saving pool.bin fails (#17338) 
Co-authored-by: Anis Elleuch <anis@min.io>
 2023-06-04 14:20:21 -07:00
Harshavardhana b210ea79bc
do not save MTime in newMultipartUpload() to avoid side-affects (#17340) 2023-06-02 14:38:09 -07:00
Harshavardhana 9b5829c16e
avoid decommissioning DEL markers with single versions (#17274) 2023-05-25 09:18:49 -07:00
Krishnan Parthasarathi 3e128c116e
Add lifecycle event source to audit log tags (#17248) 2023-05-22 15:28:56 -07:00
Harshavardhana 06557fe8be
allow decommissioned pools to be removed while others are finishing (#17221) 2023-05-16 16:00:57 -07:00
Klaus Post aaf1abc993
simplify HardLimitReader by using LimitReader for internal usage (#17218) 2023-05-16 13:14:37 -07:00
Poorna e07c2ab868
Use hash.NewLimitReader for internal multipart calls (#17191) 2023-05-12 11:19:08 -07:00
Anis Eleuch 883c98e26f
fix: remove objects when there are skipped versions due to ILM in decom (#17198) 2023-05-12 10:37:38 -07:00
Harshavardhana 3637aad36e
do not count ILM expired objects and other skipped objects (#17184) 2023-05-11 13:35:16 -07:00
Harshavardhana b92cdea578
fix: start using pkg/workers to spawn parallel workers (#17170) 2023-05-09 16:37:31 -07:00
Krishnan Parthasarathi e7cac8acef
Add tags to auditLogLifecycle (#17081) 2023-04-26 17:49:00 -07:00
Harshavardhana 6825bd7e75
fix: inlined objects don't need to honor long locks (#17039) 2023-04-17 12:16:37 -07:00
Poorna d1e775313d
support decommissioning of tiered objects (#16751) 2023-03-16 07:48:05 -07:00
Harshavardhana b984bf8d1a
allow expiration of all versions during Listing() (#16757) 2023-03-09 15:15:30 -08:00
Harshavardhana 31188e9327
add parallel workers in batch replication (#16609) 2023-02-13 12:07:58 -08:00
Florian Schwab d67a846ec4
allow restarting of decommissioning if completed, failed or canceld (#16464) 2023-01-24 07:07:59 -08:00
Harshavardhana 095fc0561d
feat: allow decom of multiple pools (#16416) 2023-01-16 21:36:34 +05:30
Anis Elleuch 475a88b555
fix: error out if an object is found after a full decom (#16277) 2023-01-12 05:52:51 +05:30
Harshavardhana b882310e2b
avoid locks for internal and invalid buckets in MakeBucket() (#16302) 2022-12-23 07:46:00 -08:00
Aditya Manthramurthy a30cfdd88f
Bump up madmin-go to v2 (#16162) 2022-12-06 13:46:50 -08:00
Harshavardhana 5a8df7efb3
re-implement StorageInfo to be a peer call (#16155) 2022-12-01 14:31:35 -08:00
jiuker fe8eed963e
fix: wrapped error will not equal in decommissioning (#16113) 2022-11-24 08:00:42 -08:00
Harshavardhana 853c4de75a
allow changing endpoints in distributed setups (#16071) 2022-11-16 07:59:10 -08:00
Harshavardhana 1f3db03bf0
allow changing argument for path for SNSD setup (#16013) 2022-11-07 00:11:58 -08:00
Krishnan Parthasarathi 4523da6543
feat: introduce pool-level rebalance (#15483) 2022-10-25 12:36:57 -07:00
Anis Elleuch fc6c794972
Audit dangling object removal (#15933) 2022-10-24 11:35:07 -07:00
Anis Elleuch ac85c2af76
lifecycle: refactor rules filtering and tagging support (#15914) 2022-10-21 10:46:53 -07:00
Harshavardhana 928feb0889
remove unused debug param from evalActionFromLifecycle (#15813) 2022-10-07 10:24:12 -07:00