Commit Graph

228 Commits

Author SHA1 Message Date
Krishnan Parthasarathi
e7cac8acef Add tags to auditLogLifecycle (#17081) 2023-04-26 17:49:00 -07:00
Praveen raj Mani
72802a5972 Use 'minio/pkg/sync/errgroup' and 'minio/pkg/workers' (#17069) 2023-04-25 22:57:40 -07:00
Harshavardhana
b1f3935c5b allow ListObjects() when a prefix is an object (#17074) 2023-04-25 22:41:54 -07:00
Krishnan Parthasarathi
fae9000304 heal: Pick maximally occuring modTime in quorum (#17071) 2023-04-25 10:13:57 -07:00
Harshavardhana
84f31ed45d simplify MRF, converge it to regular healing (#17026) 2023-04-19 07:47:42 -07:00
Harshavardhana
6825bd7e75 fix: inlined objects don't need to honor long locks (#17039) 2023-04-17 12:16:37 -07:00
Klaus Post
c133979b8e Add part count to checksum (#17035) 2023-04-14 09:44:45 -07:00
Poorna
cd6dec49c0 Add trace support for ilm activity (#16993) 2023-04-11 19:22:32 -07:00
Harshavardhana
c06e0bfef9 set correct Host: value for replication event notification (#16984) 2023-04-06 10:20:53 -07:00
Poorna
699a24f7e5 batch: validate versioning on src/tgt buckets (#16955) 2023-04-04 10:50:11 -07:00
Harshavardhana
216a471bbb on quorum DeleteObject() errors attempt an MRF (#16932) 2023-03-31 08:15:41 -07:00
Harshavardhana
6c11dbffd5 add crash protection from backend modifications (#16846) 2023-03-20 09:08:42 -07:00
Poorna
d1e775313d support decommissioning of tiered objects (#16751) 2023-03-16 07:48:05 -07:00
Harshavardhana
e0f4dd6027 remove unncessary logs from WalkDir(), PutObject() (#16818) 2023-03-15 11:52:23 -07:00
ferhat elmas
714283fae2 cleanup ignored static analysis (#16767) 2023-03-06 08:56:10 -08:00
Klaus Post
9acf1024e4 Remove bloom filter (#16682)
Removes the bloom filter since it has so limited usability, often gets saturated anyway and adds a bunch of complexity to the scanner.

Also removes a tiny bit of CPU by each write operation.
2023-02-24 09:03:31 +05:30
Harshavardhana
a0f06eac2a add Veeam SOS API first implementation (#16688) 2023-02-22 19:54:57 +05:30
Krishnan Parthasarathi
d136ac0596 Don't close transition task channel on server exit (#16627) 2023-02-15 22:09:25 -08:00
Krishnan Parthasarathi
cea2ca8c8e Add restore-status header for multipart objects (#16508) 2023-01-31 07:53:45 +05:30
Harshavardhana
67fce4a5b3 fix: dangling delete() upon success should return 404 (#16494) 2023-01-27 12:43:45 -08:00
Anis Elleuch
d98116559b Use async healing in PutObject call (#16431) 2023-01-19 00:54:22 -08:00
Harshavardhana
2937711390 fix: DeleteObject() API with versionId under replication (#16325) 2022-12-28 22:48:33 -08:00
Anis Elleuch
acc9c033ed debug: Add X-Amz-Request-ID to lock/unlock calls (#16309) 2022-12-23 19:49:07 -08:00
Krishnan Parthasarathi
2fa35def2c Fix DeleteObject when only free versions remain (#16289) 2022-12-21 16:24:07 -08:00
Anis Elleuch
89db3fdb5d Do not return an error when version disparity is detected (#16269) 2022-12-16 08:52:12 -08:00
Harshavardhana
dfe73629a3 fix: delete marker discrepancies via DeleteObject() API (#16195) 2022-12-08 18:15:16 -08:00
Aditya Manthramurthy
a30cfdd88f Bump up madmin-go to v2 (#16162) 2022-12-06 13:46:50 -08:00
Klaus Post
a713aee3d5 Run staticcheck on CI (#16170) 2022-12-05 11:18:50 -08:00
Klaus Post
1cd875de1e Persist updated metadata (#16160) 2022-12-02 08:35:04 -08:00
Poorna
63fc6ba2cd preserve replicated ETag properly on target (#16129) 2022-11-26 14:43:32 -08:00
Harshavardhana
91f45c4aa6 avoid inconsistent versions healing when versions are large (#16066) 2022-11-14 18:35:26 -08:00
Anis Elleuch
7260241511 Remove some logs caused by external apps (#16027) 2022-11-08 13:29:05 -08:00
Harshavardhana
fd6f6fc8df cleanup stale parent multipart directories (#15980) 2022-11-01 08:00:02 -07:00
Harshavardhana
136d41775f remove numAvailableDisks check as it doesn't serve any purpose (#15954) 2022-10-27 09:05:24 -07:00
Poorna
7dd8b6c8ed ensure ILM expiry creates non null deleteMarker for versioned bucket (#15947) 2022-10-26 16:09:27 -07:00
Anis Elleuch
fc6c794972 Audit dangling object removal (#15933) 2022-10-24 11:35:07 -07:00
Anis Elleuch
ac85c2af76 lifecycle: refactor rules filtering and tagging support (#15914) 2022-10-21 10:46:53 -07:00
Harshavardhana
928feb0889 remove unused debug param from evalActionFromLifecycle (#15813) 2022-10-07 10:24:12 -07:00
Harshavardhana
9e5853ecc0 optimize double reads by reusing results from checkUploadIDExists() (#15692)
Move to using `xl.meta` data structure to keep temporary partInfo,
this allows for a future change where we move to different parts to
different drives.
2022-09-15 12:43:49 -07:00
Harshavardhana
124544d834 add pre-conditions support for PUT calls during replication (#15674)
PUT shall only proceed if pre-conditions are met, the new
code uses

- x-minio-source-mtime
- x-minio-source-etag

to verify if the object indeed needs to be replicated
or not, allowing us to avoid StatObject() call.
2022-09-14 18:44:04 -07:00
Harshavardhana
8e997eba4a fix: trigger Heal when xl.meta needs healing during PUT (#15661)
This PR is a continuation of the previous change instead
of returning an error, instead trigger a spot heal on the
'xl.meta' and return only after the healing is complete.

This allows for future GETs on the same resource to be
consistent for any version of the object.
2022-09-07 07:25:39 -07:00
Harshavardhana
2d9b5a65f1 verify RenameData() versions to be consistent (#15649)
xl.meta gets written and never rolled back, however
we definitely need to validate the state that is
persisted on the disk, if there are inconsistencies

- more than write quorum we should return an error
  to the client

- if write quorum was achieved however there are
  inconsistent xl.meta's we should simply trigger
  an MRF on them
2022-09-05 16:51:37 -07:00
Harshavardhana
5ea629beb2 avoid printing io.ErrUnexpectedEOF for .metacache objects (#15642) 2022-09-02 12:47:17 -07:00
Klaus Post
8e4a45ec41 fix: encrypt checksums in metadata (#15620) 2022-08-31 08:13:23 -07:00
Klaus Post
a9f1ad7924 Add extended checksum support (#15433) 2022-08-29 16:57:16 -07:00
Poorna
471467d310 fix: ensure metadata update happens after deletemarker replication (#15564)
Fixes regression caused by #15521
2022-08-22 15:59:06 -07:00
Harshavardhana
d350b666ff feat: add idempotent delete marker support (#15521)
The bottom line is delete markers are a nuisance,
most applications are not version aware and this
has simply complicated the version management.

AWS S3 gave an unnecessary complication overhead
for customers, they need to now manage these
markers by applying ILM settings and clean
them up on a regular basis.

To make matters worse all these delete markers
get replicated as well in a replicated setup,
requiring two ILM settings on each site.

This PR is an attempt to address this inferior
implementation by deviating MinIO towards an
idempotent delete marker implementation i.e
MinIO will never create any more than single
consecutive delete markers.

This significantly reduces operational overhead
by making versioning more useful for real data.

This is an S3 spec deviation for pragmatic reasons.
2022-08-18 16:41:59 -07:00
Anis Elleuch
b3edb25377 bloom: healObject to mark a path dirty only for dangling objects (#15458)
The path is marked dirty automatically when healObject() is called, which is
wrong. HealObject() is called during self-healing and this will lead to
an increase in the false positive result of the bloom filter.

Also move NSUpdated() from renameData() and call it directly in
CompleteMultipart and PutObject, this is not a functional change but
it will make it less prone to errors in the future.
2022-08-02 16:57:39 -07:00
Harshavardhana
aa874010e2 fix: regression in resolving the right versions (#15430)
fix: regression in resolving right versions

commit d480022711 caused a regression in real
resolver, by picking up incorrect versionID.
2022-07-29 10:03:53 -07:00
Harshavardhana
ce8397f7d9 use partInfo only for intermediate part.x.meta (#15353) 2022-07-19 18:56:24 -07:00