Shubhendu
468a9fae83
Enable replication of SSE-C objects ( #19107 )
...
If site replication enabled across sites, replicate the SSE-C
objects as well. These objects could be read from target sites
using the same client encryption keys.
Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>
2024-03-28 10:44:56 -07:00
Harshavardhana
6d381f7c0a
relax pre-emptive GetBucketInfo() for multi-object delete ( #19035 )
2024-02-12 08:46:46 -08:00
Harshavardhana
afd19de5a9
fix: allow configuring excess versions alerting ( #19028 )
...
Bonus: enable audit alerts for object versions
beyond the configured value, default is '100'
versions per object beyond which scanner will
alert for each such objects.
2024-02-11 23:41:53 -08:00
Harshavardhana
dd2542e96c
add codespell action ( #18818 )
...
Original work here, #18474 , refixed and updated.
2024-01-17 23:03:17 -08:00
Harshavardhana
b3314e97a6
re-use the same local drive used by remote-peer ( #18645 )
...
historically, we have always kept storage-rest-server
and a local storage API separate without much trouble,
since they both can independently operate due to no
special state() between them.
however, over some time, we have added state()
such as
- drive monitoring threads now there will be "2" of
them per drive instead of just 1.
- concurrent tokens available per drive are now twice
instead of just single shared, allowing unexpectedly
high amount of I/O to go through.
- applying serialization by using walkMutexes can now
be adequately honored for both remote callers and local
callers.
2023-12-13 19:27:55 -08:00
Harshavardhana
fba883839d
feat: bring new HDD related performance enhancements ( #18239 )
...
Optionally allows customers to enable
- Enable an external cache to catch GET/HEAD responses
- Enable skipping disks that are slow to respond in GET/HEAD
when we have already achieved a quorum
2023-11-22 13:46:17 -08:00
Anis Eleuch
70fbcfee4a
Implement batch snowball ( #18485 )
2023-11-22 10:51:46 -08:00
Daniel López Guimaraes
a1792ca0d1
fix: relax enforcing filename on PostPolicy ( #18336 )
...
The filename is not required to be on the form data.
2023-10-30 21:06:32 -07:00
jiuker
2ac7fee017
fix: missing fileName will upload failed when PostPolicyBucketHandler ( #18240 )
2023-10-13 07:31:23 -07:00
Harshavardhana
6829ae5b13
completely remove drive caching layer from gateway days ( #18217 )
...
This has already been deprecated for close to a year now.
2023-10-11 21:18:17 -07:00
Harshavardhana
74e0c9ab9b
reduce unnecessary logging, simplify certain error handling ( #18196 )
...
remove a bunch of unnecessary logs
2023-10-10 00:33:42 -07:00
jiuker
9947c01c8e
feat: SSE-KMS use uuid instead of read all data to md5. ( #17958 )
2023-09-18 10:00:54 -07:00
Aditya Manthramurthy
1c99fb106c
Update to minio/pkg/v2 ( #17967 )
2023-09-04 12:57:37 -07:00
Harshavardhana
b1c1f02132
use buffers for pathJoin, to re-use buffers. ( #17960 )
...
```
benchmark old ns/op new ns/op delta
BenchmarkPathJoin/PathJoin-8 79.6 55.3 -30.53%
benchmark old allocs new allocs delta
BenchmarkPathJoin/PathJoin-8 2 1 -50.00%
benchmark old bytes new bytes delta
BenchmarkPathJoin/PathJoin-8 48 24 -50.00%
```
2023-08-31 17:58:48 -07:00
Anis Eleuch
49638fa533
s3: Delete Bucket should not recreate bucket if it does not exist ( #17676 )
...
Also return Bucket Not Found error in the same use case.
2023-07-18 09:32:19 -07:00
Poorna
e8c98c3246
Avoid extra GetObjectInfo call in DeleteObject API ( #17599 )
...
Optimize DeleteObject API to avoid extra
GetObjectInfo call on the replicating side.
For receiving side, it is just a regular
DeleteObject call.
Bonus: Fix a corner case where version purged is
absent on target (either due to replication not yet
complete or target version already deleted in a
one-way replication or when replication was disabled).
In such cases, mark version purge complete.
2023-07-10 07:57:56 -07:00
Harshavardhana
0bc34952eb
fix: under FanOut API avoid repeated md5sum calculation ( #17572 )
...
md5sum calculation has a high CPU overhead, avoid calculating
it repeatedly for similar fanOut calls.
To fix following CPU profiler result
```
(pprof) top10
Showing nodes accounting for 678.68s, 84.67% of 801.54s total
Dropped 1072 nodes (cum <= 4.01s)
Showing top 10 nodes out of 156
flat flat% sum% cum cum%
332.54s 41.49% 41.49% 332.54s 41.49% runtime/internal/syscall.Syscall6
228.39s 28.49% 69.98% 228.39s 28.49% crypto/md5.block
48.07s 6.00% 75.98% 48.07s 6.00% runtime.memmove
28.91s 3.61% 79.59% 28.91s 3.61% github.com/minio/highwayhash.updateAVX2
8.25s 1.03% 80.61% 8.25s 1.03% runtime.futex
8.25s 1.03% 81.64% 10.81s 1.35% runtime.step
6.99s 0.87% 82.52% 22.35s 2.79% runtime.pcvalue
6.67s 0.83% 83.35% 38.90s 4.85% runtime.mallocgc
5.77s 0.72% 84.07% 32.61s 4.07% runtime.gentraceback
4.84s 0.6% 84.67% 10.49s 1.31% runtime.lock2
```
2023-07-05 03:16:05 -07:00
Klaus Post
216069d0da
Remove 'null' version ID from directory object response ( #17495 )
...
Fixes #17494
Regression from #17132
2023-06-23 13:26:00 -07:00
Aditya Manthramurthy
5a1612fe32
Bump up madmin-go and pkg deps ( #17469 )
2023-06-19 17:53:08 -07:00
Harshavardhana
1443b5927a
allow quorum fileInfo to pick same parityBlocks ( #17454 )
...
Bonus: allow replication to proceed for 503 errors such as
with error code SlowDownRead
2023-06-18 18:20:15 -07:00
Harshavardhana
6806537eb3
event args list for fanOut notification must be sized same ( #17450 )
...
without this fan-out API can crash if client cancels
the on-going request.
2023-06-18 07:09:20 -07:00
Klaus Post
4a562d6732
fix: fanout error response - error must be string for marshaling ( #17433 )
...
Uses https://github.com/minio/minio-go/pull/1839
2023-06-15 09:21:53 -07:00
Harshavardhana
f32efd5429
more compliance related fixes ( #17408 )
...
- lifecycle must return InvalidArgument for rule errors
- do not return `null` versionId in HTTP header
- reject mixed SSE uploads with correct error message
2023-06-13 13:52:33 -07:00
Harshavardhana
26a0803388
various compliance related fixes ( #17401 )
...
- getObjectTagging to be allowed for anonymous policies
- return correct errors for invalid retention period
- return sorted list of tags for an object
- putObjectTagging must return 200 OK not 204 OK
- return 409 ErrObjectLockConfigurationNotAllowed for existing buckets
2023-06-12 13:22:07 -07:00
Harshavardhana
be45ffd8a4
return 204 status code for DeleteBucketTagging ( #17400 )
2023-06-11 20:49:02 -07:00
jiuker
14f6ac9222
fix: fail large content in DeleteMultipleObjects() early ( #17321 )
2023-05-31 10:58:14 -07:00
Anis Eleuch
6425fec366
s3: Add x-minio-error-code header for S3 HEAD requests ( #17283 )
2023-05-26 10:13:18 -07:00
Harshavardhana
d0a0eb9738
support fan-out objects via PostUpload() ( #17233 )
2023-05-24 22:51:07 -07:00
Klaus Post
5677f73794
Add PostObject Checksum ( #17244 )
2023-05-23 07:58:33 -07:00
Harshavardhana
ef2fc0f99e
fix: reduce using memory and temporary files. ( #17206 )
2023-05-15 14:08:54 -07:00
Harshavardhana
1d0211d395
allow deletes on directory objects to perform permanent deletes ( #17132 )
2023-05-04 14:43:52 -07:00
Praveen raj Mani
72802a5972
Use 'minio/pkg/sync/errgroup' and 'minio/pkg/workers' ( #17069 )
2023-04-25 22:57:40 -07:00
Harshavardhana
d1737199ed
fix: delete DNS upon success, update failure message ( #17059 )
2023-04-21 12:12:31 -07:00
Harshavardhana
e700be8cd6
fix: return appropriate Location header for MakeBucket() ( #16820 )
2023-03-15 13:40:40 -07:00
jiuker
cd3a2de5a3
add isValidLocation to common parseLocation ( #16690 )
2023-02-25 08:09:20 +05:30
Klaus Post
fd6622458b
Add detailed scanner trace output and notifications ( #16668 )
2023-02-21 09:33:33 -08:00
Harshavardhana
84fe4fd156
fix: multiObjectDelete by passing versionId for authorization ( #16562 )
2023-02-08 08:01:00 +05:30
Harshavardhana
14cf8f1b22
upgrade deps for minio/pkg v1.6.1 to include groups conditions ( #16538 )
2023-02-06 09:27:29 -08:00
Harshavardhana
d19cbc81b5
fix: do not return IAM/Bucket metadata replication errors to client ( #16486 )
2023-01-26 11:11:54 -08:00
Harshavardhana
e64b9f6751
fix: disallow SSE-C encrypted objects on replicated buckets ( #16467 )
2023-01-24 15:46:33 -08:00
Harshavardhana
31b0decd46
migrate to minio/mux from gorilla/mux ( #16456 )
2023-01-23 16:42:47 +05:30
Harshavardhana
b4ef5ff294
remove unnecessary code checking for supported features ( #16423 )
2023-01-17 19:37:47 +05:30
Harshavardhana
f1bbb7fef5
vectorize cluster-wide calls such as bucket operations ( #16313 )
2023-01-03 08:16:39 -08:00
Harshavardhana
b882310e2b
avoid locks for internal and invalid buckets in MakeBucket() ( #16302 )
2022-12-23 07:46:00 -08:00
Harshavardhana
2433698372
fix: remove unnecessary logs for client conn errors ( #16261 )
2022-12-15 08:25:05 -08:00
Harshavardhana
b03dd1af17
remove hard limit for number of buckets ( #16194 )
2022-12-08 12:24:03 -08:00
Aditya Manthramurthy
a30cfdd88f
Bump up madmin-go to v2 ( #16162 )
2022-12-06 13:46:50 -08:00
Poorna
8dd3c41b2a
allow MakeBucket errors to be handled lazily ( #15945 )
...
remote error is not required to be passed back to the
client - this is mostly because we have healing that should
eventually, catch up on this and heal the bucket.
2022-10-25 23:32:37 -07:00
Harshavardhana
23b329b9df
remove gateway completely ( #15929 )
2022-10-24 17:44:15 -07:00
Harshavardhana
a8332efa94
fix: Delete() of bucket metadata should not parse the config ( #15904 )
2022-10-19 17:55:09 -07:00
