MyFavMirrors
/
minio
mirror of https://github.com/minio/minio.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
11,122 Commits 5 Branches 503 Tags 186 MiB
Commit Graph

199 Commits

Author SHA1 Message Date
Harshavardhana a2ab21e91c
add max-keys=2 optimization for spark workloads (#18154) 
comment in the code provides more detailed explanation
on what this PR entails and its assumptions.

this PR reduces the amount of listing() by an order
of magnitude, however there are other such calls that
still needs further optimization that shall be done
in subsequent PRs.
 2023-10-02 07:52:59 -06:00
Harshavardhana c3d70e0795
cache usage, prefix-usage, and buckets for AccountInfo up to 10 secs (#18051) 
AccountInfo is quite frequently called by the Console UI 
login attempts, when many users are logging in it is important
that we provide them with better responsiveness.

- ListBuckets information is cached every second
- Bucket usage info is cached for up to 10 seconds
- Prefix usage (optional) info is cached for up to 10 secs

Failure to update after cache expiration, would still
allow login which would end up providing information
previously cached.

This allows for seamless responsiveness for the Console UI
logins, and overall responsiveness on a heavily loaded
system.
 2023-09-18 22:13:03 -07:00
Aditya Manthramurthy 1c99fb106c
Update to minio/pkg/v2 (#17967) 2023-09-04 12:57:37 -07:00
Harshavardhana dde1a12819
fix: validate incoming uploadID to be base64 encoded (#17865) 
Bonus fixes include

- do not have to write final xl.meta (renameData) does this
  already, saves some IOPs.

- make sure to purge the multipart directory properly using
  a recursive delete, otherwise this can easily pile up and
  rely on the stale uploads cleanup.

fixes #17863
 2023-08-17 09:37:55 -07:00
Harshavardhana 64aa7feabd
allow specifying lower disks for Walk() (#17829) 
useful when you may want Walk() with
reduced quorum requirements.
 2023-08-14 21:32:39 -07:00
Anis Eleuch 7fcfde7f07
s3: Pick a pool with >85% if all other pools are in suspended state (#17826) 2023-08-10 11:06:31 -07:00
Anis Eleuch a436fd513b
track client disconnections properly for all ListObjects calls (#17804) 
Currently ListObjects* calls were returning 200 OK for timed-out clients,
this makes debugging via `mc admin trace` very hard.
 2023-08-04 15:57:27 -07:00
Harshavardhana 114fab4c70
export cluster health as prometheus metrics (#17741) 2023-07-28 01:16:53 -07:00
Harshavardhana bdddf597f6
shuffle buckets randomly before being scanned (#17644) 
this randomness is needed to avoid scanning
the same buckets across different erasure sets,
in the same order.

allow random buckets to be scanned instead
allowing a wider spread of ILM, replication
checks.

Additionally do not loop over twice to fill
the channel, fill the channel regardless of
having bucket new or old.
 2023-07-14 02:25:40 -07:00
Kaan Kabalak f64d62b01d
Fix style of logOnceIf calls w/unique identifiers (#17631) 2023-07-11 13:17:45 -07:00
Harshavardhana aae6846413
feat: allow expiration of all versions via ILM Expiration action (#17521) 
Following extension allows users to specify immediate purge of
all versions as soon as the latest version of this object has
expired.

```
<LifecycleConfiguration>
    <Rule>
        <ID>ClassADocRule</ID>
        <Filter>
           <Prefix>classA/</Prefix>
        </Filter>
        <Status>Enabled</Status>
        <Expiration>
             <Days>3650</Days>
	     <ExpiredObjectAllVersions>true</ExpiredObjectAllVersions>
        </Expiration>
    </Rule>
    ...
```
 2023-06-28 22:12:28 -07:00
Kaan Kabalak 21fbe88e1f
Print certain log messages once per error (#17484) 2023-06-24 20:29:13 -07:00
Harshavardhana 1f8b9b4bd5
fix: do not listAndHeal() inline with PutObject() (#17499) 
there is a possibility that slow drives can actually add latency
to the overall call, leading to a large spike in latency.

this can happen if there are other parallel listObjects()
calls to the same drive, in-turn causing each other to sort
of serialize.

this potentially improves performance and makes PutObject()
also non-blocking.
 2023-06-24 19:31:04 -07:00
Harshavardhana 65c31fab12
fix: do not crash rebalance code instead set the object layer (#17465) 
fixes #17421
 2023-06-20 09:28:23 -07:00
Aditya Manthramurthy 5a1612fe32
Bump up madmin-go and pkg deps (#17469) 2023-06-19 17:53:08 -07:00
Poorna c4d0c49a5f
ensure metadata updates go to same pool where version exists (#17451) 
This PR also returns the replication status in 
proxy calls and defers replication attempt if 
HEAD on object version returned a error different
from NoSuchKey
 2023-06-17 07:30:53 -07:00
Klaus Post c839b64f6a
fix: compressed+encrypted block overhead (#17289) 2023-05-26 10:57:07 -07:00
Krishnan Parthasarathi 3e128c116e
Add lifecycle event source to audit log tags (#17248) 2023-05-22 15:28:56 -07:00
Harshavardhana fc03be7891
simplify bucket metadata lookups for versioning/object locking (#17253) 2023-05-22 12:05:14 -07:00
Harshavardhana 5569acd95c
disallow EC:0 if not set during server startup (#17141) 2023-05-04 14:44:30 -07:00
Harshavardhana 1d0211d395
allow deletes on directory objects to perform permanent deletes (#17132) 2023-05-04 14:43:52 -07:00
Harshavardhana b53376a3a4
change directory objects to never create new versions (#17109) 2023-05-02 16:09:33 -07:00
Krishnan Parthasarathi e7cac8acef
Add tags to auditLogLifecycle (#17081) 2023-04-26 17:49:00 -07:00
Praveen raj Mani 72802a5972
Use 'minio/pkg/sync/errgroup' and 'minio/pkg/workers' (#17069) 2023-04-25 22:57:40 -07:00
Harshavardhana b1f3935c5b
allow ListObjects() when a prefix is an object (#17074) 2023-04-25 22:41:54 -07:00
Harshavardhana 6825bd7e75
fix: inlined objects don't need to honor long locks (#17039) 2023-04-17 12:16:37 -07:00
Harshavardhana f3682b6149
allow writes to pools with inconsistent xl.meta (#17008) 2023-04-11 11:17:46 -07:00
Poorna dc8fdcb9c9
fix: error checking in DeleteBucket (#16929) 2023-03-30 11:54:08 -07:00
Anis Eleuch 1346561b9d
return quorum error instead of insufficient storage error (#16874) 2023-03-22 16:22:37 -07:00
Minio Trusted 4bc52897b2 Update yaml files to latest version RELEASE.2023-03-22T06-36-24Z 2023-03-22 21:16:15 +00:00
Harshavardhana 12047702f5
fix: tweak the maintenance=true to satisfy baremetal first (#16864) 2023-03-21 08:48:38 -07:00
Poorna d1e775313d
support decommissioning of tiered objects (#16751) 2023-03-16 07:48:05 -07:00
Klaus Post 628042e65e
tests: Protect globalLocalDrives against races (#16800) 2023-03-13 06:04:20 -07:00
Harshavardhana b984bf8d1a
allow expiration of all versions during Listing() (#16757) 2023-03-09 15:15:30 -08:00
Poorna fb6ab1cca2
fix: allow replication of 'null' delete markers (#16773) 2023-03-08 07:03:29 -08:00
ferhat elmas 714283fae2
cleanup ignored static analysis (#16767) 2023-03-06 08:56:10 -08:00
Klaus Post d07089ceac
Fix scanner deadlock on lost global lock (#16726) 2023-02-28 21:34:45 -08:00
Klaus Post 9acf1024e4
Remove bloom filter (#16682) 
Removes the bloom filter since it has so limited usability, often gets saturated anyway and adds a bunch of complexity to the scanner.

Also removes a tiny bit of CPU by each write operation.
 2023-02-24 09:03:31 +05:30
Klaus Post d0f4cc89a5
Re-add Veeam Listing workaround (#16593) 2023-02-10 10:48:39 -08:00
Klaus Post 03b94f907f
fix: deleted object names for directory objects (#16448) 2023-01-20 21:16:06 +05:30
Harshavardhana b4ef5ff294
remove unnecessary code checking for supported features (#16423) 2023-01-17 19:37:47 +05:30
jiuker c8e1154f1e
fix: reading from erasureDisks must be protected via read lock() (#16407) 2023-01-13 04:16:23 -08:00
Anis Elleuch 2146ed4033
xl: Quit early when EC config is incorrect (#16390) 
Co-authored-by: Anis Elleuch <anis@min.io>
 2023-01-09 23:07:45 -08:00
Harshavardhana a15a2556c3
converge listBuckets() as a peer call (#16346) 2023-01-03 23:39:40 -08:00
Harshavardhana f1bbb7fef5
vectorize cluster-wide calls such as bucket operations (#16313) 2023-01-03 08:16:39 -08:00
Harshavardhana 5b8fe2e89a
allow locks with object affinity to spread across pools (#16312) 2022-12-23 20:55:45 -08:00
Anis Elleuch acc9c033ed
debug: Add X-Amz-Request-ID to lock/unlock calls (#16309) 2022-12-23 19:49:07 -08:00
Harshavardhana b882310e2b
avoid locks for internal and invalid buckets in MakeBucket() (#16302) 2022-12-23 07:46:00 -08:00
Anis Elleuch 89db3fdb5d
Do not return an error when version disparity is detected (#16269) 2022-12-16 08:52:12 -08:00
Aditya Manthramurthy a30cfdd88f
Bump up madmin-go to v2 (#16162) 2022-12-06 13:46:50 -08:00