b5dc4b5873
Fix CopyObject with metadata for GCS Gateway ( #4971 )
2017-09-26 11:04:42 -07:00
70fec0a53f
azure: add stateless gateway support ( #4874 )
...
Previously init multipart upload stores metadata of an object which is
used for complete multipart. This patch makes azure gateway to store
metadata information of init multipart object in azure in the name of
'minio.sys.tmp/multipart/v1/<UPLOAD-ID>/meta.json' and uses this
information on complete multipart.
2017-09-19 16:08:08 -07:00
79ba4d3f33
refactor ObjectLayer PutObject and PutObjectPart ( #4925 )
...
This change refactor the ObjectLayer PutObject and PutObjectPart
functions. Instead of passing an io.Reader and a size to PUT operations
ObejectLayer expects an HashReader.
A HashReader verifies the MD5 sum (and SHA256 sum if required) of the object.
This change updates all all PutObject(Part) calls and removes unnecessary code
in all ObjectLayer implementations.
Fixes #4923
2017-09-19 12:40:27 -07:00
b10fa507b2
set http transport config for gateway ( #4765 )
...
This change sets the http config for the minio client used by the
minio server in gateway mode.
Fixes #4765
2017-08-08 16:23:52 -07:00
eb787d8613
gateway-gcs: remove files older than 2 weeks in minio.sys.temp ( #4599 ).
...
Rename ##minio## to {minio}.
2017-07-20 15:36:48 -07:00
bc73a1a1cb
gcs: Save partNumber as part of backend format. ( #4666 )
...
Fixes #4637
2017-07-13 23:20:16 -07:00
ce7af3aae1
gcs: Fix writer/reader go-routine leaks and code re-use ( #4651 )
...
This PR serves to fix following things in GCS gateway.
- fixes leaks in object reader and writer, not getting closed
under certain situations. This led to go-routine leaks.
- apparent confusing issue in case of complete multipart upload,
where it is currently possible for an entirely different
object name to concatenate parts of a different object name
if you happen to know the upload-id and parts of the object.
This is a very rare scenario but it is possible.
- succint usage of certain parts of code base and re-use.
2017-07-11 09:25:19 -07:00
e91e9e8a38
GCS ListObjectV2 honours continuationToken ( #4608 )
2017-06-29 11:19:55 -07:00
53e7fdc847
create subcomposed objects if total parts are > 32 ( #4593 )
2017-06-27 22:27:05 -07:00
eaa41e4086
gcs: Check if the given project id argument exists ( #4583 )
...
Using GCS resource manager API, check if the provided
project id is already created and associated to the current
user account.
2017-06-23 22:10:29 -07:00
0a6e9a1834
gateway-gcs: cleanup minio.sys.temp before deleting the bucket ( #4582 )
...
fixes #4560
fixes #4569
2017-06-23 17:57:25 -07:00
15b65a8342
Added AnonListObjectsV2 support to GCS ( #4584 )
2017-06-23 17:35:45 -07:00
13ab8e17e2
gateway-gcs: use minio.sys.temp/multipart/v1 as url base ( #4562 )
2017-06-21 10:27:44 -07:00
3928c1e14c
gateway/gcs: Change in multipart backend format ( #4455 )
2017-06-17 16:00:41 -07:00
f99f218999
Add support for reading and saving config on Gateway. ( #4463 )
...
This is also a first step towards supporting bucket
notification for gateway.
2017-06-16 16:01:41 -07:00
4fb5fc72d7
GCS gateway allows apps to supply their own marker ( #4495 )
...
Most s3 compatible apps use object keys returned in listing as
marker. This change allows this behaviour with gateway-gcs too.
2017-06-16 15:02:07 -07:00
d86973dcca
Allow bucket creation in different regions, closes #4287 and #4241
...
* I needed to remove the region check from PutBucketHandler
2017-06-16 15:02:07 -07:00
8085ba4494
Filter out internal object prefix during listing ( #4435 )
...
We use ZZZZ-Minio/ prefix internally in our GCS gateway which should be
filtered out in the response to ListObjects.
2017-06-16 15:02:07 -07:00
9bd0eb1a9e
Set default ETag value if vendor returns empty md5 string ( #4409 )
...
The ETag is constructed from md5 atttribute of object attributes
returned by the vendor's Composer. The md5 attribute comes back
as nil for large uploads. Instead the CRC32C should be used.
Refer to https://cloud.google.com/storage/docs/hashes-etags
Fixes #4397
2017-06-16 15:02:07 -07:00
e4e0abfc05
fix: Check project id before starting gateway ( #4412 )
2017-06-16 15:02:07 -07:00
2aa76e7407
Change md5Sum to etag ( #4399 )
2017-06-16 14:58:49 -07:00
0dab038858
Cleanup and update the PR with the master branch.
2017-06-16 14:55:32 -07:00
a76556ec1b
Map only default region us-east-1 to gcs us region
2017-06-16 14:54:37 -07:00
91c7bb65c5
gateway/gcs: send proper error responses for Get/SetBucket policies. ( #4338 )
...
Fixes #4323
2017-06-16 14:54:37 -07:00
5d602034ea
gateway: Use default params when no args provided ( #4315 )
...
For S3 & Azure, use default parameters when no arguments (endpoint) are
provided. This also avoids a crash.
2017-06-16 14:54:37 -07:00
9c50a9f567
Fix ListObjectParts to list properly all parts - closes #4322
2017-06-16 14:54:37 -07:00
52122c0309
Fix uploadIDMarker handling.
2017-06-16 14:54:37 -07:00
3b9d313c87
Fix issue with AbortMultipartUpload, closes #4322
2017-06-16 14:54:37 -07:00
bfff251e2a
Fix issue with UNSIGNED payloads.
...
Additionally also fixes escaping slashes in
temporary multipart names
2017-06-16 14:54:37 -07:00
52b500cce9
Verify multipart etag during complete, closes #4288
2017-06-16 14:54:37 -07:00
c63cdca11f
Support iterating through ListObjectParts using NextPartNumberMarker, closes #4284
2017-06-16 14:54:37 -07:00
4430085981
Add region to gcs gateway example
2017-06-16 14:54:37 -07:00
5c78415b31
Verify md5 content hash, closes #4285
2017-06-16 14:54:37 -07:00
bd67117756
Use maxKeys for iterator
2017-06-16 14:54:37 -07:00
f3e5e9fb29
Support marker, closes #4286
2017-06-16 14:54:37 -07:00
Aditya Manthramurthy
Bala FA
Andreas Auernhammer
Krishna Srinivas
Harshavardhana
Nitish Tiwari
A. Elleuch
Krishnan Parthasarathi
Remco Verhoef
poornas
Anis Elleuch