Commit Graph

34 Commits

Author SHA1 Message Date
Harshavardhana 4487f70f08 Revert all GetObjectNInfo related PRs (#6398)
* Revert "Encrypted reader wrapped in NewGetObjectReader should be closed (#6383)"

This reverts commit 53a0bbeb5b.

* Revert "Change SelectAPI to use new GetObjectNInfo API (#6373)"

This reverts commit 5b05df215a.

* Revert "Implement GetObjectNInfo object layer call (#6290)"

This reverts commit e6d740ce09.
2018-08-31 13:10:12 -07:00
Aditya Manthramurthy e6d740ce09 Implement GetObjectNInfo object layer call (#6290)
This combines calling GetObjectInfo and GetObject while returning a
io.ReadCloser for the object's body. This allows the two operations to
be under a single lock, fixing a race between getting object info and
reading the object body.
2018-08-27 15:28:23 +05:30
kannappanr c7946ab9ab
Remove unnecessary error log messages (#6186) 2018-08-16 12:57:49 -07:00
kannappanr 726e75611e
Do not log BucketNotFound errors on minio console (#6114) 2018-07-03 11:04:55 -07:00
wd256 25f9b0bc3b Handle ListObjectsV2 start-after parameter in ObjectLayer (#6078) 2018-07-01 09:52:45 +05:30
Harshavardhana 92a6676a2f Avoid unnecessary logging for policy not found errors (#6104) 2018-06-29 06:30:10 -07:00
Annanay Agarwal 78abe5234e Add functionality to make cache limit configurable (#5991) 2018-06-25 10:24:12 -07:00
Nitish Tiwari eb0e56ccf6 Update content-language field for Azure gateway (#6061)
Fixes https://github.com/minio/minio-go/issues/996
2018-06-21 09:46:45 -07:00
Krishna Srinivas 69b9d6fbee Add user-agent for azure blob requests (#6002) 2018-06-01 15:44:48 -07:00
Harshavardhana 71c4ff9d10 Fix tabs/indents in help message (#5962) 2018-05-21 11:11:57 -07:00
Krishna Srinivas 9de8fefa90 Fix azure list-objects not to list minio.sys.tmp/ (#5895) 2018-05-09 01:38:27 -07:00
wd256 7f207156ec Copy an object without user-defined metadata for Azure gateway (#5872) 2018-05-02 10:20:00 +05:30
Krishna Srinivas 9816264eed Support for ListObjectsV1 style marker for Azure gateway (#5856)
fixes #4948
2018-04-27 16:08:46 -07:00
Bala FA 0d52126023 Enhance policy handling to support SSE and WORM (#5790)
- remove old bucket policy handling
- add new policy handling
- add new policy handling unit tests

This patch brings support to bucket policy to have more control not
limiting to anonymous.  Bucket owner controls to allow/deny any rest
API.

For example server side encryption can be controlled by allowing
PUT/GET objects with encryptions including bucket owner.
2018-04-24 15:53:30 -07:00
Harshavardhana ccdb7bc286 Fix s3 compatibility fixes for getBucketLocation,headBucket,deleteBucket (#5842)
- getBucketLocation
- headBucket
- deleteBucket

Should return 404 or NoSuchBucket even for invalid bucket names, invalid
bucket names are only validated during MakeBucket operation
2018-04-24 08:57:33 +05:30
ebozduman f16bfda2f2 Remove panic() and handle it appropriately (#5807)
This is an effort to remove panic from the source. 
Add a new call called CriticialIf, that calls LogIf and exits. 
Replace panics with one of CriticalIf, FatalIf and a return of error.
2018-04-19 17:24:43 -07:00
kannappanr cef992a395
Remove error package and cause functions (#5784) 2018-04-10 09:36:37 -07:00
kannappanr f8a3fd0c2a
Create logger package and rename errorIf to LogIf (#5678)
Removing message from error logging
Replace errors.Trace with LogIf
2018-04-05 15:04:40 -07:00
poornas 98315b8e60 Update disk cache documentation (#5743) 2018-04-02 10:25:08 +05:30
Nitish Tiwari 328076f773
Fix help message to be uniform across env vars (#5735) 2018-03-30 01:59:21 +05:30
Harshavardhana 35b3913d22 Fix block id length upto 8bytes only for azure gateway. (#5731)
This PR also reverts commit 2f9354b17e
to bring back 8 byte block id requirement for azure gateway.
2018-03-29 09:54:47 -07:00
poornas a3e806ed61 Add disk based edge caching support. (#5182)
This PR adds disk based edge caching support for minio server.

Cache settings can be configured in config.json to take list of disk drives,
cache expiry in days and file patterns to exclude from cache or via environment
variables MINIO_CACHE_DRIVES, MINIO_CACHE_EXCLUDE and MINIO_CACHE_EXPIRY

Design assumes that Atime support is enabled and the list of cache drives is
fixed.
 - Objects are cached on both GET and PUT/POST operations.
 - Expiry is used as hint to evict older entries from cache, or if 80% of cache
   capacity is filled.
 - When object storage backend is down, GET, LIST and HEAD operations fetch
   object seamlessly from cache.

Current Limitations
 - Bucket policies are not cached, so anonymous operations are not supported in
   offline mode.
 - Objects are distributed using deterministic hashing among list of cache
   drives specified.If one or more drives go offline, or cache drive
   configuration is altered - performance could degrade to linear lookup.

Fixes #4026
2018-03-28 14:14:06 -07:00
Harshavardhana 2f9354b17e Use uuid.New() for uploadID in azure gateway (#5721) 2018-03-28 11:17:25 -07:00
Harshavardhana 35e64573fa Add missing env doc for MINIO_DOMAIN (#5705) 2018-03-26 09:11:39 -07:00
Krishna Srinivas e452377b24 Add context to the object-interface methods.
Make necessary changes to xl fs azure sia
2018-03-15 16:28:25 -07:00
Harshavardhana 4af89543cf Update minio-go dependencies to latest 5.0.0 release (#5640)
With following changes

- Add SSE and refactor encryption API (#942) <Andreas Auernhammer>
- add copyObject test changing metadata and preserving etag (#944) <Harshavardhana>
- Add SSE-C tests for multipart, copy, get range operations (#941) <Harshavardhana>
- Removing conditional check for notificationInfoCh in api-notication (#940) <Matthew Magaldi>
- Honor prefix parameter in ListBucketPolicies API (#929) <kannappanr>
- test for empty objects uploaded with SSE-C headers (#927) <kannappanr>
- Encryption headers should also be set during initMultipart (#930) <Harshavardhana>
- Add support for Content-Language metadata header (#928) <kannappanr>
- Fix check for duplicate notification configuration entries (#917) <kannappanr>
- allow OS to cleanup sockets in TIME_WAIT (#925) <Harshavardhana>
- Sign V2: Fix signature calculation in virtual host style (#921) <A. Elleuch>
- bucket policy: Support json string in Principal field (#919) <A. Elleuch>
- Fix copyobject failure for empty files (#918) <kannappanr>
- Add new constructor NewWithOptions to SDK (#915) <poornas>
- Support redirect headers to sign again with new Host header. (#829) <Harshavardhana>
- Fail in PutObject if invalid user metadata is passed <Harshavadhana>
- PutObjectOptions Header: Don't include invalid header <Isaac Hess>
- increase max retry count to 10 (#913) <poornas>
- Add new regions for Paris and China west. (#905) <Harshavardhana>
- fix s3signer to use req.Host header (#899) <Bartłomiej Nogaś>
2018-03-14 19:38:29 +05:30
Harshavardhana 9af254a82f Remove stable sort usage when not needed (#5586)
Stable sort is needed when we are sorting based on two or more
distinct elements. When equal elements are indistinguishable,
such as with integers, or more generally, any data where the
entire element is the key like `PartNumber`, stability is not
an issue.
2018-02-28 08:33:00 +05:30
Harshavardhana 0ea54c9858 Change CopyObject{Part} to single srcInfo argument (#5553)
Refactor such that metadata and etag are
combined to a single argument `srcInfo`.

This is a precursor change for #5544 making
it easier for us to provide encryption/decryption
functions.
2018-02-21 14:18:47 +05:30
poornas 4f73fd9487 Unify gateway and object layer. (#5487)
* Unify gateway and object layer. Bring bucket policies into
object layer.
2018-02-09 15:19:30 -08:00
Krishna Srinivas 45c35b3544 Autocorrect user provided Azure endpoint (#5417)
fixes #5373
2018-01-29 10:30:08 -08:00
Andreas Auernhammer d0a43af616 replace all "crypto/sha256" with "github.com/minio/sha256-simd" (#5391)
This change replaces all imports of "crypto/sha256" with
"github.com/minio/sha256-simd". The sha256-simd package
is faster on ARM64 (NEON instructions) and can take advantage
of AVX-512 in certain scenarios.

Fixes #5374
2018-01-17 10:54:31 -08:00
poornas 0bb6247056 Move nslocking from s3 layer to object layer (#5382)
Fixes #5350
2018-01-13 10:04:52 +05:30
Harshavardhana eb7c690ea9 Support in-place upgrades of new minio binary and releases. (#4961)
This PR allows 'minio update' to not only shows update banner
but also allows for in-place upgrades.

Updates are done safely by validating the downloaded
sha256 of the binary.

Fixes #4781
2017-12-15 12:33:42 -08:00
Harshavardhana eb2894233c Convert gateways into respective packages (#5200)
- Make azure gateway a package
- Make b2 gateway a package
- Make gcs gateway a package
- Make s3 gateway a package
- Make sia gateway a package
2017-12-05 17:58:09 -08:00