7270ca4157
pkg/wildcard: Simplify the wildcard logic further. ( #2555 )
2016-08-27 00:27:17 -07:00
ec4260d260
api: BucketNotification should disallow duplicate notification. ( #2539 )
...
Added checks to look for duplicated notification configs.
Fixes #2472
2016-08-23 18:42:30 -07:00
4dec50ba51
build: Add platform specific fixes.
2016-08-16 14:40:41 -07:00
76d56c6ff2
typo: Fix typos across the codebase. ( #2442 )
2016-08-15 02:44:48 -07:00
7e46055a15
api/handlers: Implement streaming signature v4 support. ( #2370 )
...
* api/handlers: Implement streaming signature v4 support.
Fixes #2326
* tests: Add tests for quick/safe
2016-08-08 20:56:29 -07:00
b23605a2b5
pkg/objcache: Add more tests. ( #2371 )
2016-08-06 10:22:14 -07:00
064c51162d
api: Add new ListenBucketNotificationHandler. ( #2336 )
...
This API is precursor before implementing `minio lambda` and `mc` continous replication.
This new api is an extention to BucketNofication APIs.
// Request
```
GET /bucket?notificationARN=arn:minio:lambda:us-east-1:10:minio HTTP/1.1
...
...
```
// Response
```
{"Records": ...}
...
...
...
{"Records": ...}
```
2016-08-04 22:01:58 -07:00
de5d5ff241
pkg/crypto: Deprecate cgo sha256 version. ( #2354 )
2016-08-04 03:19:36 -07:00
2e0742e309
bucket policy: Support for '?' wildcard. ( #2353 )
...
- Support for '?' wildcard for resource matching.
- Wildcard package is added with Match functions.
- Wildcard.Match supports '*' and wild.MatchExtended supports both '*'
and '?' wildcards in the pattern string.
- Tests for the same for the wide range of cases.
2016-08-04 00:41:32 -07:00
69fd196471
Object-cache: enforce cache size to be less than RAM. ( #2338 )
2016-08-02 10:04:35 -07:00
043ddbd834
optimize memory allocation during erasure-read by using temporary buffer pool. ( #2259 )
...
* XL/erasure-read: optimize memory allocation during erasure-read by using temporary buffer pool.
With the change the buffer needed during GetObject by erasureReadFile is allocated only once.
2016-07-25 14:17:01 -07:00
c0c8a8430e
XL/PutObject: Add single putObject and multipart caching. ( #2115 )
...
- Additionally adds test cases as well for object cache.
- Adds auto-expiry with expiration and cleanup time interval.
Fixes #2080
Fixes #2091
2016-07-08 20:34:27 -07:00
01cbacd803
object-cache: use golang bytes.Buffer and bytes.NewReader instead of custom implementation. ( #2108 )
2016-07-06 01:29:49 -07:00
8a028a9efb
handler/PUT: Handle signature verification through a custom reader. ( #2066 )
...
Change brings in a new signVerifyReader which provides a io.Reader
compatible reader, additionally implements Verify() function.
Verify() function validates the signature present in the incoming
request. This approach is choosen to avoid complexities involved
in using io.Pipe().
Thanks to Krishna for his inputs on this.
Fixes #2058
Fixes #2054
Fixes #2087
2016-07-05 01:04:50 -07:00
1ad5fb8f76
posix: checkDiskFree() also checks free inodes. ( #2086 )
...
Previously checkDiskFree() checks for free available space. This
patch enables checkDiskFree() also checks for free inodes in linux and
free clusters in windows.
Fixes #2075
2016-07-03 22:34:45 -07:00
d2557bb538
XL: GetObject caching implemented for XL. ( #2017 )
...
The object cache implementation is XL cache, which defaults
to 8GB worth of read cache. Currently GetObject() transparently
writes to this cache upon first client read and then subsequently
serves reads from the same cache.
Currently expiration is not implemented.
2016-07-03 17:15:38 -07:00
e10934a88e
bitrot: Start using blake2b algorithm and remove sha512 usage. ( #1957 )
...
Fixes #1952
2016-06-22 17:13:26 -07:00
4ab57f7d60
server: terminal width should fallback to 80x25. ( #1895 )
...
Some environments might disable access to `/dev/tty`, fall
back to '80' in such scenarios.
Move to 'cheggaaa/pb' package for better cross platform
support on fetching terminal width.
Fixes #1891
2016-06-12 19:35:28 -07:00
51f3d4e0ca
XL/multipart: statPart should ignore errDiskNotFound. ( #1862 )
...
startPart should also take uploadId and partName as arguments.
2016-06-07 18:15:04 -07:00
51bb613fdf
pkg/safe: remove temporary file on failure ( #1774 )
2016-05-28 15:15:53 -07:00
64b0976e1b
Remove probe and tasker. ( #1733 )
...
Fixes #1717
2016-05-24 18:43:33 -07:00
49141eb3e4
http: Remove minhttp package and use standard Golang http. ( #1587 )
...
The functionality provided by minhttp will be implemented
cleanly through our own APIs. Since we are not going to
send SIGUSR2 and manage configuration in that manner, it
doesn't make sense to use minhttp.
Fixes #1586
2016-05-10 18:03:00 -07:00
d1fa1d9352
Remove binary files from previous commit.
2016-05-10 15:49:17 -07:00
26e2c4bf4d
Replacing fastsha256 with crypto/sha256 package from golang standard package ( #1584 )
2016-05-10 14:20:11 -07:00
5133ea50bd
xl/fs: Make i/o operations atomic. ( #1496 )
2016-05-05 20:28:22 -07:00
7ae40eb1bb
minhttp: Remove probe usage, move to golang error. ( #1459 )
...
Fixes #1454
2016-05-03 01:07:34 -07:00
4e34e03dd4
xl/fs: Split object layer into interface. ( #1415 )
2016-04-29 14:24:10 -07:00
a98a7fb1ad
Implement XL layer - preliminary work.
2016-04-25 12:47:31 -07:00
ff4e04d942
atomic/fs: use safe package for atomic writes, even in multipart.
2016-04-06 16:05:30 -07:00
379e0abf03
cleanup: Remove old donut/xl code and erasure implementation.
...
This is a change to bring in 'klauspost/reedsolomon' library
in #1270 patch.
2016-04-02 17:30:35 -07:00
6037fe66e9
minio: Simplify for `gosimple` tool complaints.
2016-04-02 17:28:54 -07:00
efc80343e3
fs: Break fs package to top-level and introduce ObjectAPI interface.
...
ObjectAPI interface brings in changes needed for XL ObjectAPI layer.
The new interface for any ObjectAPI layer is as below
```
// ObjectAPI interface.
type ObjectAPI interface {
// Bucket resource API.
DeleteBucket(bucket string) *probe.Error
ListBuckets() ([]BucketInfo, *probe.Error)
MakeBucket(bucket string) *probe.Error
GetBucketInfo(bucket string) (BucketInfo, *probe.Error)
// Bucket query API.
ListObjects(bucket, prefix, marker, delimiter string, maxKeys int) (ListObjectsResult, *probe.Error)
ListMultipartUploads(bucket string, resources BucketMultipartResourcesMetadata) (BucketMultipartResourcesMetadata, *probe.Error)
// Object resource API.
GetObject(bucket, object string, startOffset int64) (io.ReadCloser, *probe.Error)
GetObjectInfo(bucket, object string) (ObjectInfo, *probe.Error)
PutObject(bucket string, object string, size int64, data io.Reader, metadata map[string]string) (ObjectInfo, *probe.Error)
DeleteObject(bucket, object string) *probe.Error
// Object query API.
NewMultipartUpload(bucket, object string) (string, *probe.Error)
PutObjectPart(bucket, object, uploadID string, partID int, size int64, data io.Reader, md5Hex string) (string, *probe.Error)
ListObjectParts(bucket, object string, resources ObjectResourcesMetadata) (ObjectResourcesMetadata, *probe.Error)
CompleteMultipartUpload(bucket string, object string, uploadID string, parts []CompletePart) (ObjectInfo, *probe.Error)
AbortMultipartUpload(bucket, object, uploadID string) *probe.Error
}
```
2016-04-01 15:58:39 -07:00
1502e2f29f
Merge pull request #1265 from vadmeste/add_fbsd_support
...
Add simple FreeBSD support, make the minio project compilable
2016-03-26 22:28:07 -07:00
9dca46e156
signature: Use a layered approach for signature verification.
...
Signature calculation has now moved out from being a package to
top-level as a layered mechanism.
In case of payload calculation with body, go-routines are initiated
to simultaneously write and calculate shasum. Errors are sent
over the writer so that the lower layer removes the temporary files
properly.
2016-03-26 15:21:05 -07:00
663f24064b
Add simple FreeBSD support, make the minio project compilable
2016-03-26 22:39:34 +01:00
aaf97ea02c
config/main: Re-write config files - add to new config v3
...
- New config format.
```
{
"version": "3",
"address": ":9000",
"backend": {
"type": "fs",
"disk": "/path"
},
"credential": {
"accessKey": "WLGDGYAQYIGI833EV05A",
"secretKey": "BYvgJM101sHngl2uzjXS/OBF/aMxAN06JrJ3qJlF"
},
"region": "us-east-1",
"logger": {
"file": {
"enable": false,
"fileName": "",
"level": "error"
},
"syslog": {
"enable": false,
"address": "",
"level": "debug"
},
"console": {
"enable": true,
"level": "fatal"
}
}
}
```
New command lines in lieu of supporting XL.
Minio initialize filesystem backend.
~~~
$ minio init fs <path>
~~~
Minio initialize XL backend.
~~~
$ minio init xl <url1>...<url16>
~~~
For 'fs' backend it starts the server.
~~~
$ minio server
~~~
For 'xl' backend it waits for servers to join.
~~~
$ minio server
... [PROGRESS BAR] of servers connecting
~~~
Now on other servers execute 'join' and they connect.
~~~
....
minio join <url1> -- from <url2> && minio server
minio join <url1> -- from <url3> && minio server
...
...
minio join <url1> -- from <url16> && minio server
~~~
2016-03-23 19:16:09 -07:00
c8570edaab
Issue of 'mc ls' when prefix is a directory fixed, tests added for GetObjectInfo
2016-03-23 19:42:04 +05:30
7be79b507b
Changing to empty response when prefixDir doesn't exist
2016-03-23 04:46:10 +05:30
ff41c050d5
Formatting issues fixed.
2016-03-22 15:55:29 +05:30
b55922effe
Fix for Istruncated set to true under certain conditions.
...
Optimizing List Objects by using binary sort to discard entries in cases
where prefix or marker is set.
Adding test coverage to ListObjects.
Adding benchmark to ListObjects.
2016-03-22 10:09:16 +05:30
99af0444b7
Handling maxKeys=0 case with a empty response
2016-03-18 15:16:30 +05:30
88714e7c8e
bucketpolicy: Improve bucket policy validation, avoid nested rules.
...
Bucket policy validation is more stricter now, to avoid nested
rules. The reason to do this is keep the rules simpler and more
meaningful avoiding conflicts.
This patch implements stricter checks.
Example policy to be generally avoided.
```
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"s3:GetObject",
"s3:DeleteObject"
],
"Effect": "Allow",
"Principal": {
"AWS": [
"*"
]
},
"Resource": [
"arn:aws:s3:::jarjarbing/*"
]
},
{
"Action": [
"s3:GetObject",
"s3:DeleteObject"
],
"Effect": "Deny",
"Principal": {
"AWS": [
"*"
]
},
"Resource": [
"arn:aws:s3:::jarjarbing/restic/key/*"
]
}
]
}
```
2016-03-15 17:50:23 -07:00
583e4ecff6
pkg/fs: optimize GetObject syscalls for common case
...
In the common case, GetObject is called on a bucket that exists and an
object that exists and is not a directory. It should be optimized for
this case, thus error-related syscalls are pushed back until they are
necessary.
This should not impact performance negatively in the uncommon case, and
instead drops two otherwise unnecessary os.Stat's in the common case.
The race conditions around a proper error being returned were present
beforehand.
It also renames 'err' to 'e'.
2016-03-13 13:56:33 -07:00
b2257682e4
pkg/fs: add benchmark for GetObject
...
This commit adds a benchmark for GetObject. It doesn't leverage the I/O
as much because it uses short text for data, just 58 chars.
2016-03-13 11:13:06 -07:00
53a76439a2
test for GetBucketInfo
2016-03-12 14:31:30 +05:30
5282a79eda
cleanup: Remove unecessary packages and tests. Simplify.
2016-03-11 19:53:55 -08:00
52751d81cb
cleanup: Rename ObjectMetadata as ObjectInfo.
...
Fixes #1215
2016-03-11 16:58:08 -08:00
ec8c1d4ef6
Better structuring of fs-utils test
2016-03-11 19:19:47 +05:30
fdf3d64793
error: Add proper prefixes for s3Error codes.
...
This patch adds 'Err' prefix for all standard API
error codes and also adds a proper type for them.
2016-03-10 18:38:46 -08:00
373d335d94
Merge pull request #1214 from brendanashworth/improve-listbuckets
...
ListBuckets test & improvement, IsValid{Bucket,Object}Name fix, test, docs
2016-03-10 18:20:18 -08:00
Harshavardhana
karthic rao
Krishna Srinivas
Bala FA
Harshavardhana
karthic rao
Anis Elleuch
Karthic Rao
Brendan Ashworth
Anand Babu (AB) Periasamy