Commit Graph

51 Commits

Author SHA1 Message Date
Harshavardhana 2fd7545b6c
fix: move to go1.15 (#10660) 2020-10-13 18:33:18 -07:00
Mark Clarkson 9dda9fb903
fix: https healthcheck mint test (#10622) 2020-10-05 08:21:41 -07:00
Harshavardhana 2acb530ccd
update rulesguard with new rules (#10392)
Co-authored-by: Nitish Tiwari <nitish@minio.io>
Co-authored-by: Praveen raj Mani <praveen@minio.io>
2020-09-01 16:58:13 -07:00
飞雪无情 ea5094e842
Add missing go mod libraries for mint. (#10347)
The aws-sdk-go,healthcheck,security module in mint
lacks required dependency libraries,now add them in go.mod
2020-08-25 17:51:12 -07:00
Harshavardhana 98ca770f81
mint fixes for go1.14 (#10294)
add go.mod to ensure that mint can be
built with go1.14.x
2020-08-19 20:07:37 -07:00
Harshavardhana ab801ad3d4
build edge containers with go1.14 (#10279) 2020-08-19 12:08:11 +05:30
Harshavardhana 9179cdfc9d
update mint tests with new minio-py APIs (#10238) 2020-08-10 14:32:17 -07:00
Harshavardhana ce303f5c7e
update node for mint 14.x (#10236) 2020-08-10 11:25:40 -07:00
Harshavardhana 1e2ebc9945
feat: time to bring back http2.0 support (#10230)
Bonus move our CI/CD to go1.14
2020-08-10 09:02:29 -07:00
poornas 16d7b90adf
Remove nuget from docker mint build (#9864) 2020-06-17 14:47:57 -07:00
Harshavardhana e11e4bcbc7 stick to go1.13.x for mint 2020-06-12 22:47:47 -07:00
Harshavardhana 4915433bd2
Support bucket versioning (#9377)
- Implement a new xl.json 2.0.0 format to support,
  this moves the entire marshaling logic to POSIX
  layer, top layer always consumes a common FileInfo
  construct which simplifies the metadata reads.
- Implement list object versions
- Migrate to siphash from crchash for new deployments
  for object placements.

Fixes #2111
2020-06-12 20:04:01 -07:00
kannappanr 7214a0160a
allow bucket policy to set/removed in NAS gateway (#9706) 2020-05-28 08:31:16 -07:00
Harshavardhana 7ea026ff1d
fix: reply back user-metadata in lower case form (#9697)
some clients such as veeam expect the x-amz-meta to
be sent in lower cased form, while this does indeed
defeats the HTTP protocol contract it is harder to
change these applications, while these applications
get fixed appropriately in future.

x-amz-meta is usually sent in lowercased form
by AWS S3 and some applications like veeam
incorrectly end up relying on the case sensitivity
of the HTTP headers.

Bonus fixes

 - Fix the iso8601 time format to keep it same as
   AWS S3 response
 - Increase maxObjectList to 50,000 and use
   maxDeleteList as 10,000 whenever multi-object
   deletes are needed.
2020-05-25 16:51:32 -07:00
P R 3f6d624c7b
add gateway object tagging support (#9124) 2020-05-23 11:09:35 -07:00
Harshavardhana 1bc32215b9
enable full linter across the codebase (#9620)
enable linter using golangci-lint across
codebase to run a bunch of linters together,
we shall enable new linters as we fix more
things the codebase.

This PR fixes the first stage of this
cleanup.
2020-05-18 09:59:45 -07:00
Anis Elleuch dfadf70a7f
mint: Add more SQL tests (#9540) 2020-05-15 11:20:57 -07:00
kannappanr af0309371e
mint tests s3cmd remove copy object test (#9595)
test is failing in Azure gateway as a mapping from 
x-amz-storage-class to Azure storage tier is failing 
and there is no way to not send storage class in s3cmd.
2020-05-13 19:55:34 -07:00
Harshavardhana 36e88cbd50
fix mint tests in awscli to ignore NotImplemented properly (#9561) 2020-05-08 22:32:10 -07:00
Harshavardhana a1de9cec58
cleanup object-lock/bucket tagging for gateways (#9548)
This PR is to ensure that we call the relevant object
layer APIs for necessary S3 API level functionalities
allowing gateway implementations to return proper
errors as NotImplemented{}

This allows for all our tests in mint to behave
appropriately and can be handled appropriately as
well.
2020-05-08 13:44:44 -07:00
Harshavardhana 60d415bb8a
deprecate/remove global WORM mode (#9436)
global WORM mode is a complex piece for which
the time has passed, with the advent of S3 compatible
object locking and retention implementation global
WORM is sort of deprecated, this has been mentioned
in our documentation for some time, now the time
has come for this to go.
2020-04-24 16:37:05 -07:00
Harshavardhana 6817c5ea58
migrate mint tests to latest versions (#9424) 2020-04-22 16:06:58 -07:00
Harshavardhana 3ff5bf2369
fix: convert storage class into azure tiers (#9381) 2020-04-19 13:42:56 -07:00
Harshavardhana 8bae956df6
allow copyObject to rotate storageClass of objects (#9362)
Added additional mint tests as well to verify, this
functionality.

Fixes #9357
2020-04-16 17:42:44 -07:00
Anis Elleuch 9902c9baaa
sql: Add support of escape quote in CSV (#9231)
This commit modifies csv parser, a fork of golang csv
parser to support a custom quote escape character.

The quote escape character is used to escape the quote
character when a csv field contains a quote character
as part of data.
2020-04-01 15:39:34 -07:00
Klaus Post 8d98662633
re-implement data usage crawler to be more efficient (#9075)
Implementation overview: 

https://gist.github.com/klauspost/1801c858d5e0df391114436fdad6987b
2020-03-18 16:19:29 -07:00
Anis Elleuch 35ecc04223
Support configurable quote character parameter in Select (#8955) 2020-03-13 22:09:34 -07:00
Nitish Tiwari 7e819d00ea
Fix Error Code for ObjectTagging Parsing (#8971)
Also add Mint tests
2020-02-11 17:42:28 -08:00
Nesty92 b1bfd75fcf
Aws SDK go functional test: add SSE-C CopyObject test (#8938) 2020-02-06 00:21:03 -08:00
Nesty92 f6a7d4d29b
Add CreateBucketError, ListMultipartUploads functional test (#8886) 2020-01-29 07:10:03 -08:00
Nitish Tiwari 61c17c8933 Add ObjectTagging Support (#8754)
This PR adds support for AWS S3 ObjectTagging API as explained here
https://docs.aws.amazon.com/AmazonS3/latest/dev/object-tagging.html
2020-01-20 08:45:59 -08:00
poornas 60e60f68dd Add support for object locking with legal hold. (#8634) 2020-01-16 15:41:56 -08:00
Harshavardhana 64fde1ab95
xl/zones: return errNoHealRequired when no heal is required (#8821)
Zone abstraction of object layer was returning `nil`
incorrectly under situations where disk healing is
not required. Returning `nil` is considered as healing
successful, which leads to unexpected ReloadFormat()
peer notification calls during startup.

This PR fixes this behavior properly for zones.
2020-01-15 17:19:13 -08:00
Anis Elleuch 169e8742fc mint: Use https for access to maven reposity (#8824)
Maven repository requires HTTPS now. This lead to issues
building mint image in aws-sdk-java & minio-java.

The PR fixes the issue and also bump aws sdk version in
aws-sdk-java to the latest.
2020-01-15 15:48:49 -08:00
Harshavardhana d4a390028a node 6.x is EOL'ed upgrade to latest stable (#8702) 2019-12-26 08:27:35 +05:30
kannappanr eaa838a63f mint: Change aws cli version (#8699) 2019-12-24 19:58:19 -08:00
Praveen raj Mani 842d0241ed Fix aws-sdk-php functional test cases for GCS gateway (#8613)
Fixes #8570
2019-12-16 12:41:49 +05:30
Harshavardhana 5d3d57c12a
Start using error wrapping with fmt.Errorf (#8588)
Use fatih/errwrap to fix all the code to use
error wrapping with fmt.Errorf()
2019-12-02 09:28:01 -08:00
Bala FA fb48ca5020 Add Get/Put Bucket Lock Configuration API support (#8120)
This feature implements [PUT Bucket object lock configuration][1] and
[GET Bucket object lock configuration][2]. After object lock
configuration is set, existing and new objects are set to WORM for
specified duration. Currently Governance mode works exactly like
Compliance mode.

Fixes #8101

[1] https://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketPUTObjectLockConfiguration.html
[2] https://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketGETObjectLockConfiguration.html
2019-11-12 14:50:18 -08:00
Harshavardhana 26863009c0 Load certs even if they are symlinks (#8494) 2019-11-08 11:59:20 +05:30
Harshavardhana b512241300
Add metrics healthcheck test with JWT (#8287) 2019-09-22 23:30:38 -07:00
Praveen raj Mani ad75683bde Authorize prometheus endpoint with bearer token (#7640) 2019-09-22 20:27:12 +05:30
Yao Zongyou 0e1408844b fix get container id bug (#8196) 2019-09-19 10:19:51 +05:30
Harshavardhana 94e5cb7576
Migrate to go1.13 to avail all new features (#8203)
Read more https://blog.golang.org/go1.13
2019-09-08 16:44:15 -07:00
poornas 4553db3872 mint: turn off progress for awscli sync test (#8158)
Bumping up to awscli 1.11.177 which supports --no-progress flag
2019-08-30 10:26:48 -07:00
Harshavardhana b83413b167 Use GOPROXY to speed up builds (#7984)
Read more here https://proxy.golang.org proposal 
for go1.13
2019-07-30 22:27:11 +05:30
Christian Muehlhaeuser c5faba55c1 Comment: Typo Fix (#7958) 2019-07-21 05:55:09 +01:00
Kanagaraj M 12353caf35 Fix: Support Unicode delimiters in s3 select (#7931) 2019-07-17 19:10:17 +01:00
Harshavardhana 5a52bc7ff6
Fix mint hub.docker.com builds (#7908) 2019-07-11 11:45:57 -07:00
poornas 045e1fed2b Fix dotnet tests build to be project structure agnostic (#7906) 2019-07-11 12:51:30 +05:30