Add decommission compression+encryption enabled tests (#15322)

update compression environment variables to follow
the expected sub-system style, however support fallback
mode.
This commit is contained in:
Harshavardhana
2022-07-17 08:43:14 -07:00
committed by GitHub
parent 7da9e3a6f8
commit b6eb8dff64
18 changed files with 233 additions and 48 deletions

View File

@@ -244,6 +244,7 @@ Replication from a source bucket to multiple destination buckets is supported. F
Note that on the source side, the `X-Amz-Replication-Status` changes from `PENDING` to `COMPLETED` after replication succeeds to each of the targets. On the destination side, a `X-Amz-Replication-Status` status of `REPLICA` indicates that the object was replicated successfully. Any replication failures are automatically re-attempted during a periodic disk scanner cycle.
### Interaction with extended Bucket Versioning configuration
When Bucket Versioning with excluded prefixes are configured objects matching these prefixes are excluded from being versioned and replicated.
```
@@ -261,6 +262,7 @@ When Bucket Versioning with excluded prefixes are configured objects matching th
<!-- .. up to 10 prefixes in all -->
</VersioningConfiguration>
```
In the above sample config, objects under prefixes matching any of the `ExcludedPrefixes` glob patterns will neither be versioned nor replicated.
## Explore Further

View File

@@ -131,7 +131,7 @@ Inline data is optional. If no inline data is present, it is encoded as 0 bytes.
| Entry | Encoding | Content
| --------------------|-----------------------------|----------------------------------------
| xlMetaInlineDataVer | byte | version identifier
| id -> data | msgp map[string][]byte | Map of string id -> byte content
| id -> data | msgp `map[string][]byte` | Map of string id -> byte content
Currently only xlMetaInlineDataVer == 1 exists.

View File

@@ -64,12 +64,15 @@ Similarly to suspend versioning set the configuration with Status set to `Suspen
```
## MinIO extension to Bucket Versioning
### Motivation
**PLEASE READ: This feature is meant for advanced usecases only where the setup is using bucket versioning or with replicated buckets, use this feature to optimize versioning behavior for some specific applications. MinIO experts will evaluate and guide on the benefits for your application, please reach out to us on https://subnet.min.io.**
**PLEASE READ: This feature is meant for advanced usecases only where the setup is using bucket versioning or with replicated buckets, use this feature to optimize versioning behavior for some specific applications. MinIO experts will evaluate and guide on the benefits for your application, please reach out to us on <https://subnet.min.io>.**
Spark/Hadoop workloads which use Hadoop MR Committer v1/v2 algorithm upload objects to a temporary prefix in a bucket. These objects are 'renamed' to a different prefix on Job commit. Object storage admins are forced to configure separate ILM policies to expire these objects and their versions to reclaim space.
### Solution
To exclude objects under a list of prefix (glob) patterns from being versioned, you can send the following versioning configuration with Status set to `Enabled`.
```
@@ -92,6 +95,7 @@ To exclude objects under a list of prefix (glob) patterns from being versioned,
```
### Features
- Objects matching these prefixes will behave as though versioning were suspended. These objects **will not** be replicated if bucket has replication configured.
- Objects matching these prefixes will also not leave `null` delete markers, dramatically reduces namespace pollution while keeping the benefits of replication.
- Users with explicit permissions or the root credential can configure the versioning state of any bucket.