Commit Graph

3206 Commits

Author SHA1 Message Date
Brendan Ashworth 583e4ecff6 pkg/fs: optimize GetObject syscalls for common case
In the common case, GetObject is called on a bucket that exists and an
object that exists and is not a directory. It should be optimized for
this case, thus error-related syscalls are pushed back until they are
necessary.

This should not impact performance negatively in the uncommon case, and
instead drops two otherwise unnecessary os.Stat's in the common case.

The race conditions around a proper error being returned were present
beforehand.

It also renames 'err' to 'e'.
2016-03-13 13:56:33 -07:00
Brendan Ashworth b2257682e4 pkg/fs: add benchmark for GetObject
This commit adds a benchmark for GetObject. It doesn't leverage the I/O
as much because it uses short text for data, just 58 chars.
2016-03-13 11:13:06 -07:00
Harshavardhana 09789de586 Merge branch 'awwalker-new-copy-headers' 2016-03-12 10:56:11 -08:00
awwalker 9a5e3299fc api/object: Add CopyObject to support match/modified copy headers
Adds support for the following request headers:

- x-amz-copy-source-if-match
- x-amz-copy-source-if-none-match
- x-amz-copy-source-if-unmodified-since
- x-amz-copy-source-if-modified-since

Fixes #1176
2016-03-12 10:54:23 -08:00
Harshavardhana 2c09df24ee Merge pull request #1225 from hackintoshrao/fs-bucket-tests
tests: Add tests for ListBuckets
2016-03-12 01:26:56 -08:00
Karthic Rao 53a76439a2 test for GetBucketInfo 2016-03-12 14:31:30 +05:30
Anand Babu (AB) Periasamy 6b0af08885 Merge pull request #1224 from harshavardhana/simplify
cleanup: Remove unecessary packages and tests. Simplify.
2016-03-11 22:58:43 -08:00
Harshavardhana 5282a79eda cleanup: Remove unecessary packages and tests. Simplify. 2016-03-11 19:53:55 -08:00
Anand Babu (AB) Periasamy 3d8b9afa8f Merge pull request #1222 from harshavardhana/cleanup-fix
cleanup: Rename ObjectMetadata as ObjectInfo.
2016-03-11 17:14:22 -08:00
Harshavardhana 52751d81cb cleanup: Rename ObjectMetadata as ObjectInfo.
Fixes #1215
2016-03-11 16:58:08 -08:00
Harshavardhana c81f4b0228 Merge pull request #1218 from hackintoshrao/better-fs-util-test
Test: Better structuring of fs-utils test
2016-03-11 06:54:43 -08:00
Karthic Rao ec8c1d4ef6 Better structuring of fs-utils test 2016-03-11 19:19:47 +05:30
Sreeram Boyapati 62bd44f873 buildscripts: Enable user to choose an arch to build
- Building minio for all architectures takes a lot of time.
    Choose the one user needs
2016-03-11 10:17:56 +05:30
Anand Babu (AB) Periasamy b5c77b641d Merge pull request #1209 from harshavardhana/err-naming
error: Add proper prefixes for s3Error codes.
2016-03-10 18:59:16 -08:00
Harshavardhana fdf3d64793 error: Add proper prefixes for s3Error codes.
This patch adds 'Err' prefix for all standard API
error codes and also adds a proper type for them.
2016-03-10 18:38:46 -08:00
Anand Babu (AB) Periasamy 373d335d94 Merge pull request #1214 from brendanashworth/improve-listbuckets
ListBuckets test & improvement, IsValid{Bucket,Object}Name fix, test, docs
2016-03-10 18:20:18 -08:00
Anand Babu (AB) Periasamy b16025abf4 Merge pull request #1213 from harshavardhana/presigned
auth: Detect anonymous as the last resort.
2016-03-10 18:11:43 -08:00
Harshavardhana 166ef09c3d auth: Detect anonymous as the last resort. 2016-03-10 17:55:36 -08:00
Harshavardhana e54aa10201 Merge pull request #1212 from harshavardhana/venodr
vendor: Update ui-assets with new changes and release.
2016-03-10 17:55:25 -08:00
Harshavardhana 5606232567 vendor: Update ui-assets with new changes and release. 2016-03-10 17:36:32 -08:00
Harshavardhana 9352cb87c6 Merge pull request #1211 from harshavardhana/vendorize
vendor: Add minio-go vendor updates.
2016-03-10 15:50:14 -08:00
Harshavardhana e781959d5b vendor: Add minio-go vendor updates. 2016-03-10 14:33:15 -08:00
Harshavardhana af295f3600 Merge pull request #1186 from balamurugana/devel
api: refactor list object handling in fs backend
2016-03-10 13:43:49 -08:00
Bala.FA c70bc2209e api: refactor list object handling in fs backend
When list object is invoked, it creates a goroutine if not available
for given parameters else uses existing goroutine.  These goroutines
are alive for 15 seconds for further continuation list object request
else they exit.

Fixes #1076
2016-03-11 02:20:51 +05:30
Harshavardhana 5cb546d288 Merge pull request #1210 from krishnasrinivas/port-check-fix
startup: specify the network - tcp4/tcp6 for ListenTCP()
2016-03-10 08:54:02 -08:00
Krishna Srinivas 010e775b17 startup: specify the network - tcp4/tcp6 for ListenTCP() 2016-03-10 17:17:47 +05:30
Harshavardhana 2cba605514 Merge pull request #1206 from fwessels/notice-typo
Fix typo
2016-03-09 09:25:24 -08:00
Harshavardhana d740d3aaae Merge pull request #1208 from krishnasrinivas/port-check
startup: do not start minio server if port is not free. Fixes #1207
2016-03-09 09:25:05 -08:00
Anand Babu (AB) Periasamy 5ac4afa4d1 Merge pull request #1080 from harshavardhana/bucket-policy
accessPolicy: Implement Put, Get, Delete access policy.
2016-03-09 07:11:39 -08:00
Krishna Srinivas ea7ea427ca startup: do not start minio server if port is not free. Fixes #1207 2016-03-09 19:39:40 +05:30
frankw 027f7efbdb Fix typo 2016-03-09 13:55:58 +01:00
Harshavardhana d5057b3c51 accessPolicy: Implement Put, Get, Delete access policy.
This patch implements Get,Put,Delete bucket policies

Supporting - http://docs.aws.amazon.com/AmazonS3/latest/dev/access-policy-language-overview.html

Currently supports following actions.

   "*":                             true,
   "s3:*":                          true,
   "s3:GetObject":                  true,
   "s3:ListBucket":                 true,
   "s3:PutObject":                  true,
   "s3:CreateBucket":               true,
   "s3:GetBucketLocation":          true,
   "s3:DeleteBucket":               true,
   "s3:DeleteObject":               true,
   "s3:AbortMultipartUpload":       true,
   "s3:ListBucketMultipartUploads": true,
   "s3:ListMultipartUploadParts":   true,

following conditions for "StringEquals" and "StringNotEquals"

   "s3:prefix", "s3:max-keys"
2016-03-08 17:44:50 -08:00
Harshavardhana 846410c563 Merge pull request #1205 from harshavardhana/patch-3
Fix docker documentation.
2016-03-08 11:45:45 -08:00
Harshavardhana f268762ec7 Fix docker documentation. 2016-03-08 11:23:24 -08:00
Atul Jha 41739e0913 Fix doc as per community member suggestion.
Added suggestion mentioned by our community member - fixes #1116
2016-03-08 10:42:56 -08:00
Brendan Ashworth cd3eb63c4a pkg/fs: test, document, and fix IsValid{Bucket,Object}Name
This commit improves the docs for both functions (more Go-like) and
drops an unnecessary condition in IsValidBucketName. This also drops a
condition in IsValidObjectName where "" (empty string) was a valid
object name. This has been fixed and will no longer return true.

This commit also adds tests for both functions, including a regression
test for the bug fix.
2016-03-07 19:59:24 -08:00
Brendan Ashworth a5d0bef4e2 pkg/fs: test, bench, and drop unnecessary check in ListBuckets
There is now a simple test and a benchmark for ListBuckets. I also
dropped an unnecessary check that was simply repeated from above,
guaranteed to be true.
2016-03-07 19:58:33 -08:00
Harshavardhana 114f9de5eb Merge pull request #1201 from harshavardhana/time
handlers: Cleanup time handler helpers.
2016-03-07 12:05:26 -08:00
Harshavardhana 761cb2c740 handlers: Cleanup time handlers helpers. 2016-03-07 10:47:45 -08:00
Harshavardhana 59fbb6d081 Merge pull request #1199 from brendanashworth/improvements-fs
pkg/fs improvements
2016-03-07 00:31:57 -08:00
Brendan Ashworth fab45aae40 pkg/fs: add bucket test and benchmarks
Lots of useful benchmarks and a simple test addition!
2016-03-07 00:07:11 -08:00
Brendan Ashworth 7399d8ceaa pkg/fs: skip unnecessary os.Stat system call 2016-03-07 00:07:11 -08:00
Brendan Ashworth 0a0451a0fb pkg/fs: DRY SetBucketMetadata
It had a lot of code that was the same as GetBucketMetadata, so instead
call GBM from SBM so as to reduce doing the same thing in two different
spots. Theoretically this will induce a small overhead as now at least
two calls of denormalizeBucket are made, although this shouldn't be
noticeable.
2016-03-07 00:07:11 -08:00
Brendan Ashworth 294ea814bf pkg/fs: for locks, prefer defer and read-only ops
This commit prefers the use of 'defer' for fs.Unlock (and fs.RUnlock)
because it is more idiomatic Go and reduces repetition in the code,
lending to a cleaner code base.

It also switches a few uses of the lock to read-only locks, which should
improve performance of those functions dramatically in certain contexts.
2016-03-07 00:07:11 -08:00
Anand Babu (AB) Periasamy d54a8a9c07 Merge pull request #1195 from harshavardhana/delete-objects
api: Implement multiple objects Delete api - fixes #956
2016-03-06 18:52:39 -08:00
Harshavardhana aed62788d9 api: Implement multiple objects Delete api - fixes #956
This API takes input XML input in following form.

```
<?xml version="1.0" encoding="UTF-8"?>
<Delete>
    <Quiet>true</Quiet>
    <Object>
         <Key>Key</Key>
    </Object>
    <Object>
         <Key>Key</Key>
    </Object>
    ...
</Delete>
```

and responds the list of successful deletes, list of errors
for all the deleted objects.

```
<?xml version="1.0" encoding="UTF-8"?>
<DeleteResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Deleted>
    <Key>sample1.txt</Key>
  </Deleted>
  <Error>
    <Key>sample2.txt</Key>
    <Code>AccessDenied</Code>
    <Message>Access Denied</Message>
  </Error>
</DeleteResult>
```
2016-03-06 18:31:50 -08:00
Harshavardhana 6f842124ad Merge pull request #1197 from brendanashworth/improve-DRY-1
api: DRY code and add new test
2016-03-06 16:30:48 -08:00
Brendan Ashworth adf74ffdb0 api: DRY code and add new test
This commit makes code cleaner and reduces the repetitions in the code
base. Specifically, it reduces the clutter in setObjectHeaders. It also
merges encodeSuccessResponse and encodeErrorResponse together because
they served no purpose differently. Finally, it adds a simple test for
generateRequestID.
2016-03-06 13:26:27 -08:00
Harshavardhana e883583804 Merge pull request #1192 from harshavardhana/color
console: Fix console color printing on windows.
2016-03-04 13:50:49 -08:00
Harshavardhana 164dfe2ec9 console: Fix console color printing on windows.
Print colored accessKeyID and secretAccessKey are unreadable on windows
on command prompts and powershell.

Use the console package from minio client.
2016-03-04 10:07:19 -08:00