ilm: Handle DeleteAllVersions action differently for DEL markers (#19481)

i.e., this rule element doesn't apply to DEL markers.

This is a breaking change to how ExpiredObejctDeleteAllVersions
functions today. This is necessary to avoid the following highly probable
footgun scenario in the future.

Scenario:
The user uses tags-based filtering to select an object's time to live(TTL). 
The application sometimes deletes objects, too, making its latest
version a DEL marker. The previous implementation skipped tag-based filters
if the newest version was DEL marker, voiding the tag-based TTL. The user is
surprised to find objects that have expired sooner than expected.

* Add DelMarkerExpiration action

This ILM action removes all versions of an object if its
the latest version is a DEL marker.

```xml
<DelMarkerObjectExpiration>
    <Days> 10 </Days>
</DelMarkerObjectExpiration>
```

1. Applies only to objects whose,
  • The latest version is a DEL marker.
  • satisfies the number of days criteria
2. Deletes all versions of this object
3. Associated rule can't have tag-based filtering

Includes,
- New bucket event type for deletion due to DelMarkerExpiration
This commit is contained in:
Krishnan Parthasarathi
2024-04-30 18:11:10 -07:00
committed by GitHub
parent 8161411c5d
commit 7926401cbd
11 changed files with 471 additions and 89 deletions

View File

@@ -63,6 +63,7 @@ const (
ObjectManyVersions
ObjectLargeVersions
PrefixManyFolders
ILMDelMarkerExpirationDelete
objectSingleTypesEnd
// Start Compound types that require expansion:
@@ -199,6 +200,8 @@ func (name Name) String() string {
return "s3:ObjectRemoved:NoOP"
case ObjectRemovedDeleteAllVersions:
return "s3:ObjectRemoved:DeleteAllVersions"
case ILMDelMarkerExpirationDelete:
return "s3:LifecycleDelMarkerExpiration:Delete"
case ObjectReplicationAll:
return "s3:Replication:*"
case ObjectReplicationFailed:
@@ -324,6 +327,8 @@ func ParseName(s string) (Name, error) {
return ObjectRemovedNoOP, nil
case "s3:ObjectRemoved:DeleteAllVersions":
return ObjectRemovedDeleteAllVersions, nil
case "s3:LifecycleDelMarkerExpiration:Delete":
return ILMDelMarkerExpirationDelete, nil
case "s3:Replication:*":
return ObjectReplicationAll, nil
case "s3:Replication:OperationFailedReplication":

View File

@@ -68,6 +68,8 @@ func TestNameString(t *testing.T) {
{ObjectCreatedPut, "s3:ObjectCreated:Put"},
{ObjectRemovedAll, "s3:ObjectRemoved:*"},
{ObjectRemovedDelete, "s3:ObjectRemoved:Delete"},
{ObjectRemovedDeleteAllVersions, "s3:ObjectRemoved:DeleteAllVersions"},
{ILMDelMarkerExpirationDelete, "s3:LifecycleDelMarkerExpiration:Delete"},
{ObjectRemovedNoOP, "s3:ObjectRemoved:NoOP"},
{ObjectCreatedPutRetention, "s3:ObjectCreated:PutRetention"},
{ObjectCreatedPutLegalHold, "s3:ObjectCreated:PutLegalHold"},
@@ -219,6 +221,7 @@ func TestParseName(t *testing.T) {
{"s3:ObjectAccessed:*", ObjectAccessedAll, false},
{"s3:ObjectRemoved:Delete", ObjectRemovedDelete, false},
{"s3:ObjectRemoved:NoOP", ObjectRemovedNoOP, false},
{"s3:LifecycleDelMarkerExpiration:Delete", ILMDelMarkerExpirationDelete, false},
{"", blankName, true},
}