Add Full Object Checksums and CRC64-NVME (#20855)

Backport of AIStor PR 247.

Add support for full object checksums as described here:

https://docs.aws.amazon.com/AmazonS3/latest/userguide/checking-object-integrity.html

New checksum types are fully supported. Mint tests from https://github.com/minio/minio-go/pull/2026 are now passing.

Includes fixes from https://github.com/minio/minio/pull/20743 for mint tests.

Add using checksums as validation for object content. Fixes #20845 #20849

Fixes checksum replication (downstream PR 250)
This commit is contained in:
Klaus Post
2025-01-20 06:49:07 -08:00
committed by GitHub
parent 779ec8f0d4
commit 827004cd6d
18 changed files with 665 additions and 168 deletions

View File

@@ -170,12 +170,16 @@ const (
MinIOServerStatus = "x-minio-server-status"
// Content Checksums
AmzChecksumAlgo = "x-amz-checksum-algorithm"
AmzChecksumCRC32 = "x-amz-checksum-crc32"
AmzChecksumCRC32C = "x-amz-checksum-crc32c"
AmzChecksumSHA1 = "x-amz-checksum-sha1"
AmzChecksumSHA256 = "x-amz-checksum-sha256"
AmzChecksumMode = "x-amz-checksum-mode"
AmzChecksumAlgo = "x-amz-checksum-algorithm"
AmzChecksumCRC32 = "x-amz-checksum-crc32"
AmzChecksumCRC32C = "x-amz-checksum-crc32c"
AmzChecksumSHA1 = "x-amz-checksum-sha1"
AmzChecksumSHA256 = "x-amz-checksum-sha256"
AmzChecksumCRC64NVME = "x-amz-checksum-crc64nvme"
AmzChecksumMode = "x-amz-checksum-mode"
AmzChecksumType = "x-amz-checksum-type"
AmzChecksumTypeFullObject = "FULL_OBJECT"
AmzChecksumTypeComposite = "COMPOSITE"
// Post Policy related
AmzMetaUUID = "X-Amz-Meta-Uuid"