Commit Graph

45 Commits

Author SHA1 Message Date
Harshavardhana a35cbb3ff3
update healthcheck tests for new prometheus endpoint (#11333) 2021-01-22 11:04:52 -08:00
Yingrong Zhao 6df6ac0f34
fix testMultipartUploadFailure to properly cleanup (#11137) 2020-12-21 08:23:27 -08:00
Harshavardhana 6128304f6e fix: regression introduced in aws-sdk-go tests
regression was introduced by cce5d7152a
which incorrectly implemented the tests and fixed a wrong requirement,
CompleteMultipart should never succeed in the tests.
2020-12-18 21:14:27 -08:00
Yingrong Zhao cce5d7152a
fix `testListMultipartUpload` in aws-sdk-go (#11122)
Currently, the test doesn't complete nor abort the multipart upload
after tesing the list operation. It caused the `cleanup()` to fail since
the object can't not be deleted which causes the bucket deletion to
fail.

This PR adds a `CompleteMultipartUpload` call to commit the object so
the cleanup can successfully delete the object and bucket.
2020-12-16 13:24:10 -08:00
Harshavardhana e083471ec4
use argon2 with sync.Pool for better memory management (#11019) 2020-12-03 19:23:19 -08:00
ebozduman 303be1866d
Adds "x-amz-usr-agent" and "x-id" params to be used in authentication of presignedURL (#10792) 2020-12-02 02:02:49 -08:00
Harshavardhana ae4ded7fd1
fix: s3select tests with new minio-py SDK (#10995) 2020-11-29 13:05:37 -08:00
Venkata K Sadineni d8e07f2c41
go mod update golang.org/x/text to address a CVE (#10805) 2020-11-01 19:42:48 -08:00
Mark Clarkson 9dda9fb903
fix: https healthcheck mint test (#10622) 2020-10-05 08:21:41 -07:00
Harshavardhana 2acb530ccd
update rulesguard with new rules (#10392)
Co-authored-by: Nitish Tiwari <nitish@minio.io>
Co-authored-by: Praveen raj Mani <praveen@minio.io>
2020-09-01 16:58:13 -07:00
飞雪无情 ea5094e842
Add missing go mod libraries for mint. (#10347)
The aws-sdk-go,healthcheck,security module in mint
lacks required dependency libraries,now add them in go.mod
2020-08-25 17:51:12 -07:00
Harshavardhana 98ca770f81
mint fixes for go1.14 (#10294)
add go.mod to ensure that mint can be
built with go1.14.x
2020-08-19 20:07:37 -07:00
Harshavardhana 9179cdfc9d
update mint tests with new minio-py APIs (#10238) 2020-08-10 14:32:17 -07:00
Harshavardhana ce303f5c7e
update node for mint 14.x (#10236) 2020-08-10 11:25:40 -07:00
Harshavardhana 1e2ebc9945
feat: time to bring back http2.0 support (#10230)
Bonus move our CI/CD to go1.14
2020-08-10 09:02:29 -07:00
Harshavardhana 4915433bd2
Support bucket versioning (#9377)
- Implement a new xl.json 2.0.0 format to support,
  this moves the entire marshaling logic to POSIX
  layer, top layer always consumes a common FileInfo
  construct which simplifies the metadata reads.
- Implement list object versions
- Migrate to siphash from crchash for new deployments
  for object placements.

Fixes #2111
2020-06-12 20:04:01 -07:00
kannappanr 7214a0160a
allow bucket policy to set/removed in NAS gateway (#9706) 2020-05-28 08:31:16 -07:00
Harshavardhana 7ea026ff1d
fix: reply back user-metadata in lower case form (#9697)
some clients such as veeam expect the x-amz-meta to
be sent in lower cased form, while this does indeed
defeats the HTTP protocol contract it is harder to
change these applications, while these applications
get fixed appropriately in future.

x-amz-meta is usually sent in lowercased form
by AWS S3 and some applications like veeam
incorrectly end up relying on the case sensitivity
of the HTTP headers.

Bonus fixes

 - Fix the iso8601 time format to keep it same as
   AWS S3 response
 - Increase maxObjectList to 50,000 and use
   maxDeleteList as 10,000 whenever multi-object
   deletes are needed.
2020-05-25 16:51:32 -07:00
P R 3f6d624c7b
add gateway object tagging support (#9124) 2020-05-23 11:09:35 -07:00
Harshavardhana 1bc32215b9
enable full linter across the codebase (#9620)
enable linter using golangci-lint across
codebase to run a bunch of linters together,
we shall enable new linters as we fix more
things the codebase.

This PR fixes the first stage of this
cleanup.
2020-05-18 09:59:45 -07:00
Anis Elleuch dfadf70a7f
mint: Add more SQL tests (#9540) 2020-05-15 11:20:57 -07:00
kannappanr af0309371e
mint tests s3cmd remove copy object test (#9595)
test is failing in Azure gateway as a mapping from 
x-amz-storage-class to Azure storage tier is failing 
and there is no way to not send storage class in s3cmd.
2020-05-13 19:55:34 -07:00
Harshavardhana 36e88cbd50
fix mint tests in awscli to ignore NotImplemented properly (#9561) 2020-05-08 22:32:10 -07:00
Harshavardhana a1de9cec58
cleanup object-lock/bucket tagging for gateways (#9548)
This PR is to ensure that we call the relevant object
layer APIs for necessary S3 API level functionalities
allowing gateway implementations to return proper
errors as NotImplemented{}

This allows for all our tests in mint to behave
appropriately and can be handled appropriately as
well.
2020-05-08 13:44:44 -07:00
Harshavardhana 60d415bb8a
deprecate/remove global WORM mode (#9436)
global WORM mode is a complex piece for which
the time has passed, with the advent of S3 compatible
object locking and retention implementation global
WORM is sort of deprecated, this has been mentioned
in our documentation for some time, now the time
has come for this to go.
2020-04-24 16:37:05 -07:00
Harshavardhana 6817c5ea58
migrate mint tests to latest versions (#9424) 2020-04-22 16:06:58 -07:00
Harshavardhana 3ff5bf2369
fix: convert storage class into azure tiers (#9381) 2020-04-19 13:42:56 -07:00
Harshavardhana 8bae956df6
allow copyObject to rotate storageClass of objects (#9362)
Added additional mint tests as well to verify, this
functionality.

Fixes #9357
2020-04-16 17:42:44 -07:00
Anis Elleuch 9902c9baaa
sql: Add support of escape quote in CSV (#9231)
This commit modifies csv parser, a fork of golang csv
parser to support a custom quote escape character.

The quote escape character is used to escape the quote
character when a csv field contains a quote character
as part of data.
2020-04-01 15:39:34 -07:00
Klaus Post 8d98662633
re-implement data usage crawler to be more efficient (#9075)
Implementation overview: 

https://gist.github.com/klauspost/1801c858d5e0df391114436fdad6987b
2020-03-18 16:19:29 -07:00
Anis Elleuch 35ecc04223
Support configurable quote character parameter in Select (#8955) 2020-03-13 22:09:34 -07:00
Nitish Tiwari 7e819d00ea
Fix Error Code for ObjectTagging Parsing (#8971)
Also add Mint tests
2020-02-11 17:42:28 -08:00
Nesty92 b1bfd75fcf
Aws SDK go functional test: add SSE-C CopyObject test (#8938) 2020-02-06 00:21:03 -08:00
Nesty92 f6a7d4d29b
Add CreateBucketError, ListMultipartUploads functional test (#8886) 2020-01-29 07:10:03 -08:00
Nitish Tiwari 61c17c8933 Add ObjectTagging Support (#8754)
This PR adds support for AWS S3 ObjectTagging API as explained here
https://docs.aws.amazon.com/AmazonS3/latest/dev/object-tagging.html
2020-01-20 08:45:59 -08:00
poornas 60e60f68dd Add support for object locking with legal hold. (#8634) 2020-01-16 15:41:56 -08:00
Praveen raj Mani 842d0241ed Fix aws-sdk-php functional test cases for GCS gateway (#8613)
Fixes #8570
2019-12-16 12:41:49 +05:30
Harshavardhana 5d3d57c12a
Start using error wrapping with fmt.Errorf (#8588)
Use fatih/errwrap to fix all the code to use
error wrapping with fmt.Errorf()
2019-12-02 09:28:01 -08:00
Bala FA fb48ca5020 Add Get/Put Bucket Lock Configuration API support (#8120)
This feature implements [PUT Bucket object lock configuration][1] and
[GET Bucket object lock configuration][2]. After object lock
configuration is set, existing and new objects are set to WORM for
specified duration. Currently Governance mode works exactly like
Compliance mode.

Fixes #8101

[1] https://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketPUTObjectLockConfiguration.html
[2] https://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketGETObjectLockConfiguration.html
2019-11-12 14:50:18 -08:00
Harshavardhana b512241300
Add metrics healthcheck test with JWT (#8287) 2019-09-22 23:30:38 -07:00
Praveen raj Mani ad75683bde Authorize prometheus endpoint with bearer token (#7640) 2019-09-22 20:27:12 +05:30
poornas 4553db3872 mint: turn off progress for awscli sync test (#8158)
Bumping up to awscli 1.11.177 which supports --no-progress flag
2019-08-30 10:26:48 -07:00
Christian Muehlhaeuser c5faba55c1 Comment: Typo Fix (#7958) 2019-07-21 05:55:09 +01:00
Kanagaraj M 12353caf35 Fix: Support Unicode delimiters in s3 select (#7931) 2019-07-17 19:10:17 +01:00
Harshavardhana 1dc25bcf5f
Add mint tests into MinIO repo (#7886) 2019-07-09 18:32:39 -07:00