Commit Graph

3281 Commits

Author SHA1 Message Date
Krishna Srinivas
a3b4199e9b FS/Multipart: Allow for parallel uploads of parts with same part number. (#2587)
CBL client does not close connection when the backup process is stopped, this causes
read() on the stream on the server side to block and hence the lock held on the part
is not released. When the backup process is restarted, we again try to lock on the
part and this will block. Using a unique tmp name and not locking it fixes the problem.
2016-08-31 16:01:32 -07:00
Krishna Srinivas
3aa0574c6b FS/multipart: Append the parts to a file as and when the parts arrive. (#2513) 2016-08-31 13:42:57 -07:00
Andrew Stone
3118195e92 docker: Minimize image size (#2585)
This takes the image size from 823.1MB (current edge) to 281.8MB.

The following steps are taken:
  0) Use go-alpine for a much smaller base image
  1) Perform all actions in 1 step to eliminate diff images
  2) Remove unused files created after setup
2016-08-31 13:32:41 -07:00
Harshavardhana
5716f1c199 Fix incorrect error strings. (#2580) 2016-08-30 13:47:09 -07:00
Harshavardhana
2020afd200 server: http.Server do not add deadlines causes issues. (#2581)
Adding deadlines is a no go since Golang doesn't back off
the timers if there is an active i/o in progress.

It is meant to be for applications to handle this themselves
and manually progress the deadlines.

Fixes #2561
2016-08-30 12:52:19 -07:00
Bala FA
7431acb2c4 web: add handlers for set/get bucket policy. (#2486)
Refer #1858
2016-08-30 10:04:50 -07:00
Harshavardhana
2f9975c76c notifications: GetObjectInfo() later if bucket notifications are set. (#2575) 2016-08-30 03:42:37 -07:00
Dee Koder
d9bd73f4e8 docs: Simplified install instructions for home-brew. (#2571) 2016-08-29 14:08:04 -07:00
koolhead17
8484d1c0ca Doc: Modified README.md to add brew package manager install steps. (#2557) 2016-08-28 15:44:56 -07:00
Harshavardhana
7270ca4157 pkg/wildcard: Simplify the wildcard logic further. (#2555) 2016-08-27 00:27:17 -07:00
Brendan Ashworth
b67c8970c9 bucket-handlers: fix missing return statement (#2553)
If the location was invalid, it would write an error response but then
continue to attempt to make the bucket. Whether or not it would succeed,
it would attempt to call response.WriteHeaders twice in a row, which
would cause a message to be logged to the server console (bad).

Here is the relevant Go code:
c80e0d374b/src/net/http/server.go (L878-L881)
2016-08-25 20:00:47 -07:00
Mohit Agarwal
418921de89 minor cleanup
- Reused contains() from utils.go at a couple of places
- Cleanup in return statements and boolean checks
2016-08-24 22:54:34 -07:00
Harshavardhana
ec4260d260 api: BucketNotification should disallow duplicate notification. (#2539)
Added checks to look for duplicated notification configs.

Fixes #2472
2016-08-23 18:42:30 -07:00
Harshavardhana
c39d3db7a0 server/tls: allocate tls.Config{} properly. (#2537)
There is a golang bug which crashes the server, allocate tls.Config properly to avoid
this problem.

```
2016/08/22 20:21:03 http: panic serving 127.0.0.1:40744: runtime error: invalid memory address or nil pointer dereference
goroutine 38 [running]:
net/http.(*conn).serve.func1(0xc820526680)
    /home/harsha/.gimme/versions/go1.6.2.linux.amd64/src/net/http/server.go:1389 +0xc1
panic(0xbefa80, 0xc820010140)
    /home/harsha/.gimme/versions/go1.6.2.linux.amd64/src/runtime/panic.go:443 +0x4e9
crypto/tls.(*Conn).serverHandshake(0xc820368300, 0x0, 0x0)
    /home/harsha/.gimme/versions/go1.6.2.linux.amd64/src/crypto/tls/handshake_server.go:43 +0x4d6
crypto/tls.(*Conn).Handshake(0xc820368300, 0x0, 0x0)
    /home/harsha/.gimme/versions/go1.6.2.linux.amd64/src/crypto/tls/conn.go:1035 +0x169
net/http.(*conn).serve(0xc820526680)
    /home/harsha/.gimme/versions/go1.6.2.linux.amd64/src/net/http/server.go:1405 +0x382
created by net/http.(*Server).Serve
    /home/harsha/.gimme/versions/go1.6.2.linux.amd64/src/net/http/server.go:2137 +0x44e
```

Fixes #2536
2016-08-23 00:20:15 -07:00
Harshavardhana
2da0cfc904 signature: Use the client sent region if region is properly validated. (#2534)
A properly validated region string from the client should be once
validated properly.

Fixes #2521
2016-08-22 19:43:36 -07:00
Harshavardhana
9dd0e3dc44 config: Initialize notify for fresh configs. (#2531) 2016-08-22 16:04:04 -07:00
Dee Koder
4722c94653 typo: Fix capitalization in heading. (#2529) 2016-08-22 12:01:21 -07:00
Krishna Srinivas
45c928e2f5 boot: checkPortAvailability() should fail only for EADDRINUSE error and ignore other errors. (#2527)
fixes #2510
2016-08-22 10:20:01 -07:00
Anis Elleuch
07506358ff Refactor Heal RPC and add Shutdown RPC (#2488) 2016-08-21 12:06:53 -07:00
Harshavardhana
975eb31973 api/bucketPolicy: Use minio-go/pkg/set and fix bucket policy regression. (#2506)
Current master has a regression 'mc policy <policy-type> alias/bucket/prefix'
does not work anymore, due to the way new minio-go changes do json marshalling.
This led to a regression on server side when a ``prefix`` is provided
policy is rejected as malformed from th server which is not the case with
AWS S3.

This patch uses the new ``minio-go/pkg/set`` package to address the
unmarshalling problems.

Fixes #2503
2016-08-20 03:16:38 -07:00
koolhead17
a3c509fd23 Doc: Modified examples in how-to-run-multiple-minio-server-instances-on-single-machine.md (#2497) 2016-08-19 16:09:21 -07:00
GarimaKapoor
63bb78cfc6 Update README.md 2016-08-19 11:46:03 -07:00
Jesse Lucas
f2fd8b0265 Adding test coverage for server-mux.go by creating TestListenAndServeTLS(). (#2493)
Editing config.go to add lock on customConfigPath to avoid race condition
when setGlobalConfigPath() and getConfigPath() are called concurrently.
2016-08-19 01:29:50 -07:00
Jesse Lucas
a8052889fe server-mux_test.go updating test to fix race condition with TestListenAndServe (#2485) 2016-08-18 19:45:03 -07:00
Harshavardhana
bccf549463 server: Move all the top level files into cmd folder. (#2490)
This change brings a change which was done for the 'mc'
package to allow for clean repo and have a cleaner
github drop in experience.
2016-08-18 16:23:42 -07:00
Anis Elleuch
73d1a46f3e Allow asterisk character in key names (#2487) 2016-08-18 12:00:01 -07:00
Harshavardhana
95c16f51cb notification: Handle ARN with empty account and name types. (#2482)
Fixes #2481
2016-08-17 16:34:03 -07:00
Harshavardhana
810dcbf34b fs: ListObjects should populate ETag properly if fs.json is available. (#2480)
Fixes #2470
2016-08-17 13:26:08 -07:00
Harshavardhana
cb77586508 controller: Fix controller cli arguments. 2016-08-17 11:41:04 -07:00
Krishna Srinivas
e2498edb45 contoller: Implement controlled healing and trigger (#2381)
This patch introduces new command line 'control' 

- minio control 

TO manage minio server connecting through GoRPC API frontend.

- minio control heal 

Is implemented for healing objects.
2016-08-17 11:36:33 -07:00
Jesse Lucas
0b7dfab17a Refactoring MuxServer Close() method to always wait for conns to close before returning. Adding lock around ServerMux listener setting to protect against data race. Adding additional tests to server-mux_test.go to make sure open connections are closed and for ListenAndServe. (#2467) 2016-08-17 00:18:23 -07:00
Krishna Srinivas
674fdc4304 redirect-middleware: Redirect to /minio only if client is a browser and the request is not V4 signed. (#2469)
fixes #2356
2016-08-17 00:16:59 -07:00
karthic rao
10feb1af3f tests: object handlers: Unit tests for Get and Copy Object handlers (#2451) 2016-08-16 19:24:23 -07:00
GarimaKapoor
d2b924cca8 Added Goreportcard badge. 2016-08-16 17:59:59 -07:00
Brendan Ashworth
13390d0c95 tests: add tests for signature-v4.go (#2458)
This adds tests for the functions `doesPresignedSignatureMatch` and
`doesPolicySignatureMatch`.
2016-08-16 17:02:35 -07:00
Harshavardhana
0a3d43273f vendor: sha256 32bit updated. (#2459) 2016-08-16 16:19:29 -07:00
Harshavardhana
4dec50ba51 build: Add platform specific fixes. 2016-08-16 14:40:41 -07:00
Krishna Srinivas
f2bffe6086 fs/delete-object: fs.json filepath was incorrect. (#2448) 2016-08-16 10:05:08 -07:00
Krishna Srinivas
8e2f64aea4 fs/multipart: save metadata(fs.json) for multipart uploads. (#2450) 2016-08-16 10:04:40 -07:00
Harshavardhana
c054e633fd utils: Shutdown channel should be bufferred. 2016-08-15 21:01:24 -07:00
Harshavardhana
e86dfcf41e api: Change listen bucket notification to be TopicConfiguration. (#2447) 2016-08-15 20:56:43 -07:00
Anis Elleuch
3b9dbd748b tests: Web handlers (#2429) 2016-08-15 16:13:03 -07:00
Harshavardhana
3d1bb8f439 tests: Fix hasExtendedHeader tests with env variable. 2016-08-15 16:09:08 -07:00
Krishna Srinivas
bb8a425d49 When updating the meta file, write to temp file first and then rename to the actual location.
This prevents appending the metadata to the metadata-file when a file is reuploaded.
2016-08-15 15:55:59 -07:00
Harshavardhana
0e745fdb05 fs: Enable fs.json with env MINIO_ENABLE_FSMETA 2016-08-15 15:53:48 -07:00
Anis Elleuch
51d7749c3e Check if eventN is initialized before notifying in Upload web handler (#2435) 2016-08-15 12:15:46 -07:00
Harshavardhana
76d56c6ff2 typo: Fix typos across the codebase. (#2442) 2016-08-15 02:44:48 -07:00
Harshavardhana
b41bfcbf2f utils: Fix unit tests issue. (#2441) 2016-08-15 01:59:28 -07:00
Yurii Rashkovskii
341171f326 Problem: AWS documentation defines event timestamp as 1970-01-01T00:00:00.000Z (#2440)
While Minio is using 20160814T124605Z

(See http://docs.aws.amazon.com/AmazonS3/latest/dev/notification-content-structure.html)

Solution: adhere to AWS documentation
2016-08-15 01:50:07 -07:00
karthic rao
a3592228f5 bug-fix: fix for tests failure when cache is disabled (#2439) 2016-08-15 01:25:41 -07:00