Commit Graph

26 Commits

Author SHA1 Message Date
Krishnan Parthasarathi
6d5bc045bc
Disallow ExpiredObjectAllVersions with object lock (#19792)
Relaxes restrictions on Expiration and NoncurrentVersionExpiration
placed by https://github.com/minio/minio/pull/19785.
ref: https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lock-managing.html#object-lock-managing-lifecycle

> Object lifecycle management configurations continue functioning
normally on protected objects, including placing delete markers.
However, a locked version of an object cannot be deleted by a S3
Lifecycle expiration policy. Object Lock is maintained regardless of
the object's storage class and throughout S3 Lifecycle
transitions between storage classes.
2024-05-22 18:12:48 -07:00
jiuker
391baa1c9a
test: add reject ilm rule test case (#19788) 2024-05-22 04:26:59 -07:00
jiuker
9906b3ade9
fix: reject ilm rule when bucket LockEnabled (#19785) 2024-05-21 23:50:03 -07:00
Krishnan Parthasarathi
7926401cbd
ilm: Handle DeleteAllVersions action differently for DEL markers (#19481)
i.e., this rule element doesn't apply to DEL markers.

This is a breaking change to how ExpiredObejctDeleteAllVersions
functions today. This is necessary to avoid the following highly probable
footgun scenario in the future.

Scenario:
The user uses tags-based filtering to select an object's time to live(TTL). 
The application sometimes deletes objects, too, making its latest
version a DEL marker. The previous implementation skipped tag-based filters
if the newest version was DEL marker, voiding the tag-based TTL. The user is
surprised to find objects that have expired sooner than expected.

* Add DelMarkerExpiration action

This ILM action removes all versions of an object if its
the latest version is a DEL marker.

```xml
<DelMarkerObjectExpiration>
    <Days> 10 </Days>
</DelMarkerObjectExpiration>
```

1. Applies only to objects whose,
  • The latest version is a DEL marker.
  • satisfies the number of days criteria
2. Deletes all versions of this object
3. Associated rule can't have tag-based filtering

Includes,
- New bucket event type for deletion due to DelMarkerExpiration
2024-04-30 18:11:10 -07:00
Krishnan Parthasarathi
2007dd26ae
ilm: Expire if object past expected expiry date (#19230)
When an object qualifies for both tiering and expiration rules and is
past its expiration date, it should be expired without requiring to tier
it, even when tiering event occurs before expiration.
2024-03-08 22:41:22 -08:00
Krishnan Parthasarathi
a93214ea63
ilm: ObjectSizeLessThan and ObjectSizeGreaterThan (#18500) 2023-11-22 13:42:39 -08:00
Krishnan Parthasarathi
8cd80fec8c
Add unit test for lifecycle.FilterRules (#18284) 2023-10-19 21:33:28 -07:00
Krishnan Parthasarathi
557df666fd
Don't skip rules with ExpiredObjectDeleteMarker (#18256) 2023-10-16 22:46:46 -07:00
Harshavardhana
aae6846413
feat: allow expiration of all versions via ILM Expiration action (#17521)
Following extension allows users to specify immediate purge of
all versions as soon as the latest version of this object has
expired.

```
<LifecycleConfiguration>
    <Rule>
        <ID>ClassADocRule</ID>
        <Filter>
           <Prefix>classA/</Prefix>
        </Filter>
        <Status>Enabled</Status>
        <Expiration>
             <Days>3650</Days>
	     <ExpiredObjectAllVersions>true</ExpiredObjectAllVersions>
        </Expiration>
    </Rule>
    ...
```
2023-06-28 22:12:28 -07:00
Krishnan Parthasarathi
0ec722bc54
Add tags to NewerNoncurrentVersions audit event (#17110) 2023-05-02 12:56:33 -07:00
Krishnan Parthasarathi
6eef9b4a23
lifecycle: simplify Eval and HasActiveRules (#16036) 2022-11-10 07:17:45 -08:00
Krishnan Parthasarathi
7ba281728f
ilm: fix x-amz-expiration header evaluation (#16029) 2022-11-09 04:20:34 -08:00
Anis Elleuch
ac85c2af76
lifecycle: refactor rules filtering and tagging support (#15914) 2022-10-21 10:46:53 -07:00
Krishnan Parthasarathi
6d6a731d6d
Handle overlapping and conflicting ILM rules (#15812) 2022-10-07 14:36:23 -07:00
Krishnan Parthasarathi
6f56ba80b3
lifecycle: Assign unique id to rules with empty id (#15731) 2022-09-22 10:51:54 -07:00
Anis Elleuch
b0e2c2da78
lifecycle: Support tags with special characters (#14906)
Object tags can have special characters such as whitespace. However
the current code doesn't properly consider those characters while
evaluating the lifecycle document.

ObjectInfo.UserTags contains an url encoded form of object tags
(e.g. key+1=val)

This commit fixes the issue by using the tags package to parse object tags.
2022-05-14 10:25:55 -07:00
Harshavardhana
f527c708f2
run gofumpt cleanup across code-base (#14015) 2022-01-02 09:15:06 -08:00
Krishnan Parthasarathi
44a9339c0a
Newer noncurrent versions (#13815)
- Rename MaxNoncurrentVersions tag to NewerNoncurrentVersions

Note: We apply overlapping NewerNoncurrentVersions rules such that 
we honor the highest among applicable limits. e.g if 2 overlapping rules 
are configured with 2 and 3 noncurrent versions to be retained, we 
will retain 3.

- Expire newer noncurrent versions after noncurrent days
- MinIO extension: allow noncurrent days to be zero, allowing expiry 
  of noncurrent version as soon as more than configured 
  NewerNoncurrentVersions are present.
- Allow NewerNoncurrentVersions rules on object-locked buckets
- No x-amz-expiration when NewerNoncurrentVersions configured
- ComputeAction should skip rules with NewerNoncurrentVersions > 0
- Add unit tests for lifecycle.ComputeAction
- Support lifecycle rules with MaxNoncurrentVersions
- Extend ExpectedExpiryTime to work with zero days
- Fix all-time comparisons to be relative to UTC
2021-12-14 09:41:44 -08:00
Krishnan Parthasarathi
3da9ee15d3
Add MaxNoncurrentVersions to NoncurrentExpiration action (#13580)
This unit allows users to limit the maximum number of noncurrent 
versions of an object.

To enable this rule you need the following *ilm.json*
```
cat >> ilm.json <<EOF
{
    "Rules": [
        {
            "ID": "test-max-noncurrent",
            "Status": "Enabled",
            "Filter": {
                "Prefix": "user-uploads/"
            },
            "NoncurrentVersionExpiration": {
                "MaxNoncurrentVersions": 5
            }
        }
    ]
}
EOF
mc ilm import myminio/mybucket < ilm.json
```
2021-11-19 17:54:10 -08:00
Poorna K
3dfcca68e6
fix race in TestComputeActions test (#13564) 2021-11-02 14:20:15 -07:00
Krishnan Parthasarathi
45d145a823
fix: immediate tiering for NoncurrentVersionTransition (#13464) 2021-10-18 17:24:30 -07:00
Krishnan Parthasarathi
f3aeed77e5
Add immediate inline tiering support (#13298) 2021-10-01 11:58:17 -07:00
Anis Elleuch
37bef900fd
ilm: More compliance to spec related to expired delete markers (#12887)
<Days> tag can remove expired delete markers according to AWS S3 spec.

<NoncurrentVersionExpiration> cannot remove expired delete markers in
anyway.
2021-08-05 11:21:21 -07:00
Krishnan Parthasarathi
209e6d00c6
Use ObjectInfo.ToLifecycleOpts instead of literal values (#12772)
Promote getLifecycleTransitionTier to a method on lifecycle.Lifecycle.
2021-07-21 19:12:44 -07:00
Krishnan Parthasarathi
d0963974a5
pkg/lifecycle: Add SetPredictionHeaders method (#12755)
This method is used to add expected expiration and transition time 
for an object in GET/HEAD Object response headers.

Also fixed bugs in lifecycle.PredictTransitionTime and
getLifecycleTransitionTier in handling current and 
non-current versions.
2021-07-20 17:36:55 -07:00
Harshavardhana
1f262daf6f
rename all remaining packages to internal/ (#12418)
This is to ensure that there are no projects
that try to import `minio/minio/pkg` into
their own repo. Any such common packages should
go to `https://github.com/minio/pkg`
2021-06-01 14:59:40 -07:00