MyFavMirrors
/
minio
mirror of https://github.com/minio/minio.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
11,001 Commits 5 Branches 503 Tags 186 MiB
Commit Graph

287 Commits

Author SHA1 Message Date
Harshavardhana 124e28578c
remove strict persistence requirements for List() .metacache objects (#17917) 
.metacache objects are transient in nature, and are better left to
use page-cache effectively to avoid using more IOPs on the disks.

this allows for incoming calls to be not taxed heavily due to
multiple large batch listings.
 2023-08-25 07:58:11 -07:00
Harshavardhana dde1a12819
fix: validate incoming uploadID to be base64 encoded (#17865) 
Bonus fixes include

- do not have to write final xl.meta (renameData) does this
  already, saves some IOPs.

- make sure to purge the multipart directory properly using
  a recursive delete, otherwise this can easily pile up and
  rely on the stale uploads cleanup.

fixes #17863
 2023-08-17 09:37:55 -07:00
Harshavardhana 6e860b6dc5
count all versions as part of DeleteAllVersionsAction (#17821) 2023-08-09 08:55:19 -07:00
Harshavardhana cb089dcb52
error out by default beyond 10000 versions per object (#17803) 
```
You've exceeded the limit on the number of versions you can create on this object
```
 2023-08-04 10:40:21 -07:00
Harshavardhana 0153f96a20
add deadlines for readMetadata() in listing (#17776) 
Bonus: also skip spending time looking for xl.json

- Listing()
- Delete()
 2023-08-01 21:52:31 -07:00
Harshavardhana 81be718674
fix: optimize DiskInfo() call avoid metrics when not needed (#17763) 2023-07-31 15:20:48 -07:00
Harshavardhana 73edd5b8fd
introduce 'mc admin config set alias/ api odirect=on' (#17753) 
change disable_odirect=off -> odirect=on to make it
easier to understand, instead of making it double
negative.
 2023-07-31 00:12:53 -07:00
Harshavardhana f13cfcb83e
allow disabling O_DIRECT for write ops (#17751) 
on really slow systems, O_DIRECT simply kills the drives
allow for a way to disable them.
 2023-07-29 15:17:56 -07:00
Harshavardhana 731e03fe5a
add ReadFileStream deadline for disk call (#17745) 
timeout the reader side if hung via disk max timeout
 2023-07-28 15:37:53 -07:00
Harshavardhana 47dcfcbdd4
introduce deadlines on READ operations (#17724) 2023-07-27 07:33:05 -07:00
Harshavardhana b28bcad11b
avoid Access() calls on known bucket paths (#17719) 2023-07-26 11:31:40 -07:00
Poorna f95129894d
Use decrypted object size while computing object size summary (#17717) 
Corrects an issue with encrypted versioned objects being reported under
`unversioned` bin in the object version histogram
 2023-07-24 17:13:25 -07:00
Harshavardhana 14e1ace552
remove serializing WalkDir() across all buckets/prefixes on SSDs (#17707) 
slower drives get knocked off because they are too slow via 
active monitoring, we do not need to block calls arbitrarily.

Serializing adds latencies for already slow calls, remove
it for SSDs/NVMEs

Also, add a selection with context when writing to `out <-`
channel, to avoid any potential blocks.
 2023-07-24 09:30:19 -07:00
Krishnan Parthasarathi 0120ff93bc
admin-info: add DeleteMarkers count (#17659) 2023-07-18 10:49:40 -07:00
Harshavardhana 24e86d0c59
avoid passing around poolIdx, setIdx instead pass the relevant disks (#17660) 2023-07-17 09:52:05 -07:00
Kaan Kabalak f64d62b01d
Fix style of logOnceIf calls w/unique identifiers (#17631) 2023-07-11 13:17:45 -07:00
Harshavardhana 82075e8e3a
use strconv variants to improve on performance per 'op' (#17626) 
```
BenchmarkItoa
BenchmarkItoa-8         	673628088	         1.946 ns/op	       0 B/op	       0 allocs/op
BenchmarkFormatInt
BenchmarkFormatInt-8    	592919769	         2.012 ns/op	       0 B/op	       0 allocs/op
BenchmarkSprint
BenchmarkSprint-8       	26149144	        49.06 ns/op	       2 B/op	       1 allocs/op
BenchmarkSprintBool
BenchmarkSprintBool-8   	26440180	        45.92 ns/op	       4 B/op	       1 allocs/op
BenchmarkFormatBool
BenchmarkFormatBool-8   	1000000000	         0.2558 ns/op	       0 B/op	       0 allocs/op
```
 2023-07-11 07:46:58 -07:00
Kaan Kabalak bd6842d917
Further print log messages once per error (#17618) 2023-07-10 07:59:57 -07:00
Kaan Kabalak 21fbe88e1f
Print certain log messages once per error (#17484) 2023-06-24 20:29:13 -07:00
Aditya Manthramurthy 5a1612fe32
Bump up madmin-go and pkg deps (#17469) 2023-06-19 17:53:08 -07:00
Harshavardhana 1443b5927a
allow quorum fileInfo to pick same parityBlocks (#17454) 
Bonus: allow replication to proceed for 503 errors such as
with error code SlowDownRead
 2023-06-18 18:20:15 -07:00
jiuker 5a21b1f353
fix: Delete dir failed when .DS_Store in it (#17352) 2023-06-06 10:12:06 -07:00
Klaus Post bb6f4d7633
Remove redundant checkFormatJSON logging (#17134) 2023-05-04 07:28:37 -07:00
Klaus Post e8c0a50862
optimization use small blocks up to 64KB (#17107) 2023-05-01 09:47:49 -07:00
Harshavardhana 8c874884fc
fix: do not copy context in DiskInfo cache (#17085) 2023-04-26 12:13:54 -07:00
Anis Eleuch 224d9a752f
fix: the race in healing tracker code (#17048) 2023-04-18 14:49:56 -07:00
Krishnan Parthasarathi 6877578bbc
Update minio_node_bucket_scans_finished metrics (#17006) 2023-04-11 19:21:34 -07:00
ferhat elmas 056ca0c68e
refactor: reuse open file in storage interface (#16970) 2023-04-09 23:09:28 -07:00
Krishnan Parthasarathi 25f7a8e406
Indicate RenameData is called by healObject (#16997) 2023-04-09 10:25:37 -07:00
Shubhendu 4c204707fd
Correct to remove `null` version while ILM rule application (#16971) 
Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>
Co-authored-by: Harshavardhana <harsha@minio.io>
 2023-04-06 14:10:01 -07:00
jiuker 5fa3665074
add isSysErrNotDir to openFileNoSync (#16958) 2023-04-04 08:00:08 -07:00
Harshavardhana b984bf8d1a
allow expiration of all versions during Listing() (#16757) 2023-03-09 15:15:30 -08:00
ferhat elmas 714283fae2
cleanup ignored static analysis (#16767) 2023-03-06 08:56:10 -08:00
Klaus Post 9acf1024e4
Remove bloom filter (#16682) 
Removes the bloom filter since it has so limited usability, often gets saturated anyway and adds a bunch of complexity to the scanner.

Also removes a tiny bit of CPU by each write operation.
 2023-02-24 09:03:31 +05:30
Klaus Post fd6622458b
Add detailed scanner trace output and notifications (#16668) 2023-02-21 09:33:33 -08:00
Klaus Post 6a04067514
fix: tweak read buffer size to reduce over-reading (#16338) 2023-01-01 08:14:20 -08:00
Krishnan Parthasarathi 2fa35def2c
Fix DeleteObject when only free versions remain (#16289) 2022-12-21 16:24:07 -08:00
Harshavardhana 20ef5e7a6a
avoid double deletes() when no more versions (#16206) 2022-12-12 01:40:04 -08:00
Klaus Post 3eb2d086b2
Replace filepathx with fork (#16192) 2022-12-08 10:42:44 -08:00
Aditya Manthramurthy a30cfdd88f
Bump up madmin-go to v2 (#16162) 2022-12-06 13:46:50 -08:00
Anis Elleuch 1bae32dc96
xl: Delete older data-dir when replacing an existing version-id (#16176) 2022-12-06 13:43:18 -08:00
Klaus Post cc1d8f0057
Check for abandoned data when healing (#16122) 2022-11-28 10:20:55 -08:00
Krishnan Parthasarathi 6eef9b4a23
lifecycle: simplify Eval and HasActiveRules (#16036) 2022-11-10 07:17:45 -08:00
Klaus Post ecc932d5dd
Clean entire tmp-old on restart (#15979) 2022-10-31 07:27:50 -07:00
Harshavardhana 23b329b9df
remove gateway completely (#15929) 2022-10-24 17:44:15 -07:00
Harshavardhana 58a8275e84
do not assume invalid buf to be non-xl.meta (#15843) 2022-10-17 09:39:21 -07:00
Anis Elleuch 6e84283c66
fix: ignoring O_DIRECT in case of erasure single disk (#15734) 
fixes #15733 
fixes #15735
 2022-09-22 10:41:06 -07:00
Klaus Post ff12080ff5
Remove deprecated io/ioutil (#15707) 2022-09-19 11:05:16 -07:00
Harshavardhana 2d9b5a65f1
verify RenameData() versions to be consistent (#15649) 
xl.meta gets written and never rolled back, however
we definitely need to validate the state that is
persisted on the disk, if there are inconsistencies

- more than write quorum we should return an error
  to the client

- if write quorum was achieved however there are
  inconsistent xl.meta's we should simply trigger
  an MRF on them
 2022-09-05 16:51:37 -07:00
Harshavardhana 97376f6e8f
improve performance for inlined data (#15603) 
inlined data often is bigger than the allowed
O_DIRECT alignment, so potentially we can write
'xl.meta' without O_DSYNC instead we can rely on
O_DIRECT + fdatasync() instead.

This PR allows O_DIRECT on inlined data that
would gain the benefits of performing O_DIRECT,
eventually performing an fdatasync() at the end.

Performance boost can be observed here for small
objects < 128KiB. The performance boost is mainly
seen on HDD, and marginal on NVMe setups.
 2022-08-29 11:19:29 -07:00