Commit Graph

140 Commits

Author SHA1 Message Date
Harshavardhana
bb639d9f29
remove double reads delete versions (#13544)
deleting collection of versions belonging
to same object, we can avoid re-reading
the xl.meta from the disk instead purge
all the requested versions in-memory,

the tradeoff is to allocate a map to de-dup
the versions, allow disks to be read only
once per object.

additionally reduce the data transfer between
nodes by shortening msgp data values.
2021-11-01 10:50:07 -07:00
Harshavardhana
4ed0eb7012
remove double reads updating object metadata (#13542)
Removes RLock/RUnlock for updating metadata,
since we already take a write lock to update
metadata, this change removes reading of xl.meta
as well as an additional lock, the performance gain
should increase 3x theoretically for

- PutObjectRetention
- PutObjectLegalHold

This optimization is mainly for Veeam like
workloads that require a certain level of iops
from these API calls, we were losing iops.
2021-10-30 08:22:04 -07:00
Harshavardhana
d693431183
fix: ReadFileStream should return an error when size mismatches (#13435)
offset+length should match the Size() of the individual parts
return 'errFileCorrupt' otherwise, to trigger healing of the individual 
parts do not error out prematurely when healing such bitrot's upon
successful parts being written to the client.

another issue this PR fixes is to not return and error to
the client if we have just triggered a heal on a specific
part of the object, instead continue to read all the content
and let the heal happen asynchronously later.
2021-10-13 19:49:14 -07:00
Harshavardhana
200caab82b
fix: multi-pool setup make sure acquire locks properly (#13280)
This was a regression introduced in '14bb969782'
this has the potential to cause corruption when
there are concurrent overwrites attempting to update
the content on the namespace.

This PR adds a situation where PutObject(), CopyObject()
compete properly for the same locks with NewMultipartUpload()
however it ends up turning off competing locks for the actual
object with GetObject() and DeleteObject() - since they do not
compete due to concurrent I/O on a versioned bucket it can lead
to loss of versions.

This PR fixes this bug with multi-pool setup with replication
that causes corruption of inlined data due to lack of competing
locks in a multi-pool setup.

Instead CompleteMultipartUpload holds the necessary
locks when finishing the transaction, knowing the exact
location of an object to schedule the multipart upload
doesn't need to compete in this manner, a pool id location
for existing object.
2021-09-22 21:46:24 -07:00
Krishnan Parthasarathi
31d7cc2cd4
erasure: Set fi.IsLatest when adding a new version (#13277) 2021-09-22 19:17:09 -07:00
Poorna Krishnamoorthy
c4373ef290
Add support for multi site replication (#12880) 2021-09-18 13:31:35 -07:00
Harshavardhana
03b7bebc96
fix: invalid quorum calculation in TransitionObject (#13125)
Quorum calculation should be based on the
existing metadata, custom quorum calculation
can lead to unreadable content.
2021-09-01 08:57:42 -07:00
Harshavardhana
35f2552fc5
reduce extra getObjectInfo() calls during ILM transition (#13091)
* reduce extra getObjectInfo() calls during ILM transition

This PR also changes expiration logic to be non-blocking,
scanner is now free from additional costs incurred due
to slower object layer calls and hitting the drives.

* move verifying expiration inside locks
2021-08-27 17:06:47 -07:00
Harshavardhana
bbf3576f70
remove unecessary metadata structs in applyTransitionAction() (#13059) 2021-08-24 12:24:00 -07:00
Poorna Krishnamoorthy
674c6f7a7b
fix: resync of replication of delete markers (#12932)
Fixes #12919
2021-08-23 14:48:22 -07:00
Krishnan Parthasarathi
b7e3651d3c
Set free-version id in case of version/version-suspended buckets (#12982)
This free-version id may be used to track tiered object contents of the
object (version) being deleted.
2021-08-17 08:59:48 -07:00
Klaus Post
24722ddd02
Remove inline data hack (#12946)
move the code down to the storage layer,
this logic decouples the inline data from the 
size parameter making it flexible and future
proof.
2021-08-13 08:25:54 -07:00
Klaus Post
3eac02f676
Use metadata reader in ReadVersion (#12942)
Use `readMetadata` when reading version 
information without data requested. 

Reduces IO on inlined data.

Bonus: Inline compressed data as well when 
compression is enabled.
2021-08-12 10:05:24 -07:00
Harshavardhana
40a2fa8e81
fix: add more optimizations to putMetacacheObject() (#12916)
- avoid extra lookup for 'xl.meta' since we are
  definitely sure that it doesn't exist.

- use this in newMultipartUpload() as well

- also additionally do not write with O_DSYNC
  to avoid loading the drives, instead create
  'xl.meta' for listing operations without
  O_DSYNC since these are ephemeral objects.

- do the same with newMultipartUpload() since
  it gets synced when the PutObjectPart() is
  attempted, we do not need to tax newMultipartUpload()
  instead.
2021-08-10 11:12:22 -07:00
Harshavardhana
54ab3a1d5b
implement putMetacacheObject() optimizing List operations (#12903)
removes unexpected features from regular putObject() such as

- increasing parity when disks are down, avoids
  a lot of DiskInfo() calls.

- triggering MRF for metacache objects
  if disks are offline

- avoiding renames from temporary location
  to actual namespace, not needed since
  metacache files are unique.
2021-08-09 06:58:54 -07:00
Harshavardhana
035882d292
fix: remove parentIsObject() check (#12851)
we will allow situations such as

```
a/b/1.txt
a/b
```

and

```
a/b
a/b/1.txt
```

we are going to document that this usecase is
not supported and we will never support it, if
any application does this users have to delete
the top level parent to make sure namespace is
accessible at lower level.

rest of the situations where the prefixes get
created across sets are supported as is.
2021-08-03 13:26:57 -07:00
Krishnan Parthasarathi
0a62ae4e61
Revert ignoring inlined objects for transition (#12843) 2021-07-30 16:45:17 -07:00
Harshavardhana
a51799d9f0
feat: Add support for audit notifications for transition (#12842)
This PR adds audit notifications for transitioning objects,
similar to audit logging for expiration and replication
traffic.
2021-07-30 12:45:25 -07:00
Harshavardhana
f175ff8f66
add healing fixes for delete-marker (#12788)
- delete-markers are incorrectly reported
  as corrupt with wrong data sent to client
  'mc admin heal -r' on objects with delete
  marker will report as 'grey' incorrectly.

- do not heal delete-markers during HeadObject()
  this can lead to inconsistent order of heals
  on the object, although this is not an issue
  in terms of order of versions it is rather
  simpler to keep the same order on all drives.

- defaultHealResult() should handle 'err == nil'
  case such that valid cases should be handled
  as 'drive' status OK.
2021-07-26 08:01:41 -07:00
Krishnan Parthasarathi
6ea083d197
Add deployment-id and source bucket to transitioned object name (#12693)
This allows remote bucket admin to identify the origin of transitioned
objects by simply inspecting the object prefixes.

e.g let's take a remote tier TIER-1 pointing to a remote bucket (prefix)
testbucket/testprefix-1. The remote bucket admin can list all transitioned objects
from a MinIO deployment identified by '2e78e906-1c5d-4f94-8689-9df44cafde39' and
source bucket 'mybucket' like so,

```
$ ./mc ls -r minio-tier-target/testbucket/testprefix-1/2e78e906-1c5d-4f94-8689-9df44cafde39/mybucket/
[2021-07-12 17:15:50 PDT]   160B 48/fb/48fbc0e6-3a73-458b-9337-8e722c619ca4
[2021-07-12 16:58:46 PDT]   160B 7d/1c/7d1c96bd-031a-48d4-99ea-b1304e870830
```
2021-07-20 10:49:52 -07:00
Krishnan Parthasarathi
29eea52e14
Skip transitioning of object versions if inlined (#12705) 2021-07-16 09:38:27 -07:00
Anis Elleuch
b0b4696a64
heal: Add MRF metrics to background heal API response (#12398)
This commit gathers MRF metrics from 
all nodes in a cluster and return it to the caller. This will show information about the 
number of objects in the MRF queues 
waiting to be healed.
2021-07-15 22:32:06 -07:00
Harshavardhana
affee27b05
fix: speed up erasure code upgrade checks (#12626)
DiskInfo() calls can stagger and wait if run
serially timing out 10secs per drive, to avoid
this lets check DiskInfo in parallel to avoid
delays when nodes get disconnected.
2021-07-08 01:04:37 -07:00
Krishnan Parthasarathi
a1df230518
Add a 'free' version to track deletion of tiered object content (#12470) 2021-06-30 19:32:07 -07:00
Harshavardhana
41caf89cf4
fix: apply pre-conditions first on object metadata (#12545)
This change in error flow complies with AWS S3 behavior
for applications depending on specific error conditions.

fixes #12543
2021-06-24 09:44:00 -07:00
Anis Elleuch
7722b91e1d
s3: Force a prefix removal using a special header (#12504)
An S3 client can send `x-minio-force-delete: true` to remove a prefix.
2021-06-15 18:43:14 -07:00
Klaus Post
d524544494
Fix nil disk check in parity upgrade feature (#12444)
Fixes #12443
2021-06-04 09:38:19 -07:00
Aditya Manthramurthy
30a3921d3e
[Tiering] Support remote tiers with object versioning (#12342)
- Adds versioning support for S3 based remote tiers that have versioning
enabled. This ensures that when reading or deleting we specify the specific
version ID of the object. In case of deletion, this is important to ensure that
the object version is actually deleted instead of simply being marked for
deletion.

- Stores the remote object's version id in the tier-journal. Tier-journal file
version is not bumped up as serializing the new struct version is
compatible with old journals without the remote object version id.

- `storageRESTVersion` is bumped up as FileInfo struct now includes a
`TransitionRemoteVersionID` member.

- Azure and GCS support for this feature will be added subsequently.

Co-authored-by: Krishnan Parthasarathi <krisis@users.noreply.github.com>
2021-06-03 14:26:51 -07:00
Harshavardhana
1f262daf6f
rename all remaining packages to internal/ (#12418)
This is to ensure that there are no projects
that try to import `minio/minio/pkg` into
their own repo. Any such common packages should
go to `https://github.com/minio/pkg`
2021-06-01 14:59:40 -07:00
Harshavardhana
81d5688d56
move the dependency to minio/pkg for common libraries (#12397) 2021-05-28 15:17:01 -07:00
Harshavardhana
89bb9f17d7
fix: when parityDrives hits > len(storageDisks)/2, keep maxParity (#12387)
Additionally move out `x-minio-internal-erasure-upgraded` from HTTP headers
list, as its an internal header, rename elsewhere accordingly.
2021-05-27 13:38:04 -07:00
Klaus Post
acc452b7ce
Add more erasure codes on degraded systems. (#11852)
In cases where a cluster is degraded, we do not uphold our consistency 
guarantee and we will write fewer erasure codes and rely on healing 
to recreate the missing shards.

In some cases replacing known bad disks in practice take days.
We want to change the behavior of a known degraded system to keep
the erasure code promise of the storage class for each object.

This will create the objects with the same confidence as a fully 
functional cluster. The tradeoff will be that objects created 
during a partial outage will take up slightly more space.

This means that when the storage class is EC:4, there should 
always be written 4 parity shards, even if some disks are unavailable.

When an object is created on a set, the disks are immediately 
checked. If any disks are unavailable additional parity shards 
will be made for each offline disk, up to 50% of the number of disks.

We add an internal metadata field with the actual and intended 
erasure code level, this can optionally be picked up later by 
the scanner if we decide that data like this should be re-sharded.
2021-05-27 11:38:09 -07:00
Harshavardhana
4840974d7a
fix: inline data upon overwrites should be readable (#12369)
This PR fixes two bugs

- Remove fi.Data upon overwrite of objects from inlined-data to non-inlined-data
- Workaround for an existing bug on disk with latest releases to ignore fi.Data
  and instead read from the disk for non-inlined-data
- Addtionally add a reserved metadata header to indicate data is inlined for
  a given version.
2021-05-25 16:33:06 -07:00
Klaus Post
cde6469b88
Fix hanging erasure writes (#12253)
However, this slice is also used for closing the writers, so close is never called on these.

Furthermore when an error is returned from a write it is now reported to the reader.

bonus: remove unused heal param from `newBitrotWriter`.

* Remove copy, now that we don't mutate.
2021-05-17 08:32:28 -07:00
Harshavardhana
f1e479d274
remove more duplicate bloom filter trackers (#12302)
At some places bloom filter tracker was getting
updated for `.minio.sys/tmp` bucket, there is no
reason to update bloom filters for those.

And add a missing bloom filter update for MakeBucket()

Bonus: purge unused function deleteEmptyDir()
2021-05-17 08:25:48 -07:00
Harshavardhana
2ab9dc7609 do not update bloomFilters for temporary objects 2021-05-15 19:54:07 -07:00
Harshavardhana
d84261aa6d
fix: ensure proper usage of DataDir (#12300)
- GetObject() should always use a common dataDir to
  read from when it starts reading, this allows the
  code in erasure decoding to have sane expectations.

- Healing should always heal on the common dataDir, this
  allows the code in dangling object detection to purge
  dangling content.

These both situations can happen under certain types of
retries during PUT when server is restarting etc, some
namespace entries might be left over.
2021-05-14 16:50:47 -07:00
Harshavardhana
e84f533c6c
add missing wait groups for certain io.Pipe() usage (#12264)
wait groups are necessary with io.Pipes() to avoid
races when a blocking function may not be expected
and a Write() -> Close() before Read() races on each
other. We should avoid such situations..

Co-authored-by: Klaus Post <klauspost@gmail.com>
2021-05-11 09:18:37 -07:00
Harshavardhana
1aa5858543
move madmin to github.com/minio/madmin-go (#12239) 2021-05-06 08:52:02 -07:00
Krishnan Parthasarathi
860bf1bab2
Add IsRemote method on FileInfo, ObjectInfo (#12209)
Provides a convenient method to know if an object's contents are in its remote
tier.
2021-05-04 08:40:42 -07:00
Harshavardhana
0d3ddf7286
fix: improve NewObjectReader implementation for careful cleanup usage (#12199)
cleanup functions should never be cleaned before the reader is
instantiated, this type of design leads to situations where order
of lockers and places for them to use becomes confusing.

Allow WithCleanupFuncs() if the caller wishes to add cleanupFns
to be run upon close() or an error during initialization of the
reader.

Also make sure streams are closed before we unlock the resources,
this allows for ordered cleanup of resources.
2021-04-30 18:37:58 -07:00
Harshavardhana
64f6020854
fix: cleanup locking, cancel context upon lock timeout (#12183)
upon errors to acquire lock context would still leak,
since the cancel would never be called. since the lock
is never acquired - proactively clear it before returning.
2021-04-29 20:55:21 -07:00
Harshavardhana
336c8ac99f
fix: do not heal when disks are down (#12186)
HeadObject() was erroneously attempting
a heal when disks are down, avoid it.
2021-04-29 09:54:16 -07:00
Anis Elleuch
9e797532dc
lock: Always cancel the returned Get(R)Lock context (#12162)
* lock: Always cancel the returned Get(R)Lock context

There is a leak with cancel created inside the locking mechanism. The
cancel purpose was to cancel operations such erasure get/put that are
holding non-refreshable locks.

This PR will ensure the created context.Cancel is passed to the unlock
API so it will cleanup and avoid leaks.

* locks: Avoid returning nil cancel in local lockers

Since there is no Refresh mechanism in the local locking mechanism, we
do not generate a new context or cancel. Currently, a nil cancel
function is returned but this can cause a crash. Return a dummy function
instead.
2021-04-27 16:12:50 -07:00
Poorna Krishnamoorthy
4be0f92067
Fix multipart restore to remove part match (#12161)
Part ETags are not available after multipart finalizes, removing this
check as not useful.

Signed-off-by: Poorna Krishnamoorthy <poorna@minio.io>
Co-authored-by: Harshavardhana <harsha@minio.io>
2021-04-26 18:24:06 -07:00
Harshavardhana
2966823818
use jsoniter for json marshal/unmarshal in KMS (#12146)
Signed-off-by: Harshavardhana <harsha@minio.io>
2021-04-26 16:01:52 -07:00
Harshavardhana
4eb9b6eaf8
preserve metadata multipart restore (#12139)
avoid re-read of xl.meta instead just use
the success criteria from PutObjectPart()
and check the ETag matches per Part, if
they match then the parts have been
successfully restored as is.

Signed-off-by: Harshavardhana <harsha@minio.io>
2021-04-24 19:07:27 -07:00
Poorna Krishnamoorthy
5d954ea228
fix: versionID and MTime for restored object (#12145)
Signed-off-by: Poorna Krishnamoorthy <poorna@minio.io>
2021-04-24 19:04:35 -07:00
Harshavardhana
cbfdf97abf Use CompleteMultipartUpload in RestoreTransitionedObject
Signed-off-by: Krishnan Parthasarathi <kp@minio.io>
2021-04-23 11:58:53 -07:00
Krishnan Parthasarathi
3831027c54 fix: compiler errors in restoreTransitionedObject (#12120) 2021-04-23 11:58:53 -07:00