Commit Graph

4870 Commits

Author SHA1 Message Date
Anis Elleuch d8e4d3c9c8 POSTForm: Return http 303 if redirect is specified (#3479)
success_action_redirect in the sent Form means that the server needs to return 303 in addition to a well specific redirection url, this commit adds this feature
2016-12-20 09:32:17 -08:00
Harshavardhana faa6b1e925 vendorize deps for snappy, blake2b and sha256 (#3476)
Bring in new optimization and portability changes.

Fixes https://github.com/minio/minio-go/issues/578
2016-12-19 19:32:55 -08:00
Krishnan Parthasarathi 85c6bb9809 server: Sort disk arguments for consistent ordering (#3469)
This is important in a distributed setup, where the server hosting the
first disk formats a fresh setup. Sorting ensures that all servers
arrive at the same 'first' server.

Note: This change doesn't protect against different disk arguments
with some disks being same across servers.
2016-12-19 17:04:31 -08:00
Anis Elleuch 5404dddcea PostForm: Save supported headers in obj metadata (#3474)
Supported Headers like Content-Type, Cache-Control, Content-Encoding, X-Amz-* , etc.. are now saved in object metadata
2016-12-19 16:14:04 -08:00
Anis Elleuch 4692fdbb8f PostForm: Follow success_action_status requirement (#3467)
S3 spec requires that Post Object response depends on the passed success_action_status. This commit implements that requirement.
2016-12-18 13:39:56 -08:00
Bala FA 1875a47495 rpcclient: fix leaky connection (#3471)
Previously, more than one goroutine calls RPCClient.dial(), each
goroutine gets a new rpc.Client but only one such client is stored
into RPCClient object.  This leads to leaky connection at the server
side.  This is fixed by taking lock at top of dial() and release on
return.
2016-12-17 18:17:40 -08:00
Harshavardhana 9c9f390350 fs: validate filesystem path argument properly. (#3470)
FS should fail for invalid paths like

 - file:///
 - ftp://
 - http://
2016-12-17 13:43:26 -08:00
Bala FA 1b2b16998f Remove regexp usage (#3456)
This patch removes regexp usage in cacheControlHandler.ServeHTTP() and
server-main.checkEndpointsSyntax()
2016-12-17 11:00:16 -08:00
koolhead17 7a17b2a585 Browser: Added character limit for Bucket Name. (#3454) 2016-12-16 09:59:37 -08:00
Aditya Manthramurthy f7766b49aa Amend ZFS line in README - to make it about Minio on FreeNAS (#3461) 2016-12-16 09:59:02 -08:00
Krishnan Parthasarathi b2f920a868 Add service API handler stubs for status, stop and restart (#3417) 2016-12-15 22:26:15 -08:00
Anis Elleuch 8ceb969445 tests: Use testTmpDir var to specify tmp directory (#3459)
To be able to specify the directory where tests will be done.  This way,
it will be easier to run Minio tests on a mounted directory like NFS, ..
2016-12-15 22:25:05 -08:00
Harshavardhana 0db484c8f6 signv2: Do not use path encoding for query values. (#3458)
Use query unescape before comparing signature.
2016-12-15 14:56:18 -08:00
dcharbonnier 5c481fbf6e Fix Minio Quickstart Guide link (#3457) 2016-12-15 11:59:16 -08:00
Aditya Manthramurthy 8e6e9301ce Add support for Kafka as a notifications target (#2869) (#3439) 2016-12-15 08:23:48 -08:00
Harshavardhana 664ff063a1 server: checkEndpoints syntax properly. (#3451) 2016-12-14 20:42:19 -08:00
koolhead17 d9fd6f9a96 docs: Removed $/# from code block to make code copy easier. (#3448) 2016-12-14 17:45:47 -08:00
Harshavardhana b28ff50126 lock/server: Check if the lock server itself is skewed back. (#3447) 2016-12-14 08:02:32 -08:00
Karthic Rao 3fe2d77b70 Adding functions for resetting globals. (#3421) 2016-12-13 11:51:48 -08:00
Krishnan Parthasarathi ab49498fc3 server: Exit gracefully if no endpoint is local to it. (#3442) 2016-12-13 11:18:31 -08:00
Minio Trusted 46fdd70114 Fix docker file and avoid creating README.md 2016-12-13 09:31:19 -08:00
Krishnan Parthasarathi 29d72b84c0 loginServer should be a member of lockServer (#3441)
Add a unit test to catch a missing LoginHandler method in lockServer.
2016-12-13 08:01:47 -08:00
Anand Babu (AB) Periasamy f53fcdf10e keep make messages consistent (#3437) 2016-12-13 00:26:45 -08:00
Krishna Srinivas 8e665105b1 Presign-v4: Allow requests that were signed slightly ahead of the current time. (#3435) 2016-12-12 16:32:00 -08:00
Harshavardhana 2062add05f fs/posix: On windows use helpers and init format.json properly. (#3434)
Fixes #3433
2016-12-12 15:43:41 -08:00
Krishnan Parthasarathi 6b4e6bcebf Move LoginHandler into LoginServer which others embed (#3431)
* Move LoginHandler into LoginServer which others embed

* Add unit tests for loginServer
2016-12-12 08:11:23 -08:00
Harshavardhana 2d6f8153fa format: Check properly for disks in valid formats. (#3427)
There was an error in how we validated disk formats,
if one of the disk was formatted and was formatted with
FS would cause confusion and object layer would never
initialize essentially go into an infinite loop.

Validate pre-emptively and also check for FS format
properly.
2016-12-11 15:18:55 -08:00
Anis Elleuch 5c10f4adf0 presign v2: include resp headers in signature calc (#3428)
Include response headers when presigning an url using signature v2 algorithm
2016-12-11 14:32:25 -08:00
Harshavardhana 4daa0d2cee lock: Moving locking to handler layer. (#3381)
This is implemented so that the issues like in the
following flow don't affect the behavior of operation.

```
GetObjectInfo()
.... --> Time window for mutation (no lock held)
.... --> Time window for mutation (no lock held)
GetObject()
```

This happens when two simultaneous uploads are made
to the same object the object has returned wrong
info to the client.

Another classic example is "CopyObject" API itself
which reads from a source object and copies to
destination object.

Fixes #3370
Fixes #2912
2016-12-10 16:15:12 -08:00
Harshavardhana cd0f350c02 env: Bring back MINIO_BROWSER env. (#3423)
Set MINIO_BROWSER=off to disable web browser completely.

Fixes #3422
2016-12-10 00:42:22 -08:00
Krishna Srinivas ac554bf663 FS/Multipart: Fix race between PutObjectPart and Complete/Abort multi… (#3419)
FS/Multipart: Fix race between PutObjectPart and Complete/Abort multipart. close(timeoutCh) on complete/abort so that a racing PutObjectPart does not leave a dangling go-routine.

Fixes #3351
2016-12-09 16:10:18 -08:00
Harshavardhana b363709c11 caching: Optimize memory allocations. (#3405)
This change brings in changes at multiple places

 - Reuse buffers at almost all locations ranging
   from rpc, fs, xl, checksum etc.
 - Change caching behavior to disable itself
   under low memory conditions i.e < 8GB of RAM.
 - Only objects cached are of size 1/10th the size
   of the cache for example if 4GB is the cache size
   the maximum object size which will be cached
   is going to be 400MB. This change is an
   optimization to cache more objects rather
   than few larger objects.
 - If object cache is enabled default GC
   percent has been reduced to 20% in lieu
   with newly found behavior of GC. If the cache
   utilization reaches 75% of the maximum value
   GC percent is reduced to 10% to make GC
   more aggressive.
 - Do not use *bytes.Buffer* due to its growth
   requirements. For every allocation *bytes.Buffer*
   allocates an additional buffer for its internal
   purposes. This is undesirable for us, so
   implemented a new cappedWriter which is capped to a
   desired size, beyond this all writes rejected.

Possible fix for #3403.
2016-12-08 20:35:07 -08:00
Anis Elleuch 410b579e87 startup: Show elapsed time in disks format process (#3413) 2016-12-07 10:22:00 -08:00
Karthic Rao 7b7c0bba58 Use a non member mutex lock for serverConfig access. (#3411)
- This is to ensure that the any new config references made to the
  serverConfig is also backed by a mutex lock.
- Otherwise any new config assigment will also replace the member mutex
  which is currently used for safe access.
2016-12-07 03:41:54 -08:00
Anis Elleuch 0cef971832 Fix max cache size calculation when system RAM is inferior to the default cache size (#3410) 2016-12-06 16:09:26 -08:00
Anis Elleuch 5c9a95df32 srv-mux: do not print peek protocol EOF err msg (#3402)
EOF err message in Peek Protocol is shown when a client closes the
connection in the middle of peek protocol, this commit hides it since it
doesn't make sense to show it
2016-12-05 14:49:32 -08:00
Anis Elleuch 3b455d6137 tests: Add tests for xl-v1-list-objects-heal (#3399) 2016-12-05 09:40:33 -08:00
Anis Elleuch b2a0e5754b bucket-handlers: More tests for post form handler (#3392) 2016-12-04 12:23:19 -08:00
Anis Elleuch 63d9bb626a postform: fix check when ${filename} is provided (#3391)
Checking key condition when ${filename} is provided wasn't working well,
this patch fixes the wrong behavior
2016-12-04 10:30:52 -08:00
Anis Elleuch 372da5eaf5 tests: Enhance checkPostPolicy() coverage (#3389) 2016-12-03 12:41:07 -08:00
Harshavardhana cf17fc7774 fs: PutObject create 0byte objects properly. (#3387)
Current code always appends to a file only if 1byte or
more was sent on the wire was affecting both PutObject
and PutObjectPart uploads.

This patch fixes such a situation and resolves #3385
2016-12-03 11:53:12 -08:00
Krishnan Parthasarathi 67509453d3 FS: sync abortMultipart cleanup and bg append (#3388)
backgroundAppend type's abort method should wait for appendParts to finish
writing ongoing appending of parts in the background before cleaning up
the part files.
2016-12-02 23:33:06 -08:00
Harshavardhana d31f256020 Fail on lint errors during CI build. 2016-12-02 18:08:12 -08:00
Harshavardhana d67f47927c api: Fix the formatting issues in last patch. 2016-12-02 17:39:21 -08:00
Anis Elleuch 85bb5870a9 Post Policy Form: exhaustive post policy check (#3386)
Add support of all conditions check described in
http://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-HTTPPOSTConstructPolicy.html
and simplify a little bit the existing code
2016-12-02 17:00:33 -08:00
Harshavardhana 4a9b205a15 docs: Add missing comments for exported functions. 2016-12-02 14:39:23 -08:00
koolhead17 f3b346cbb3 docs: removed the Edge tag reference. (#3366) 2016-12-02 10:47:18 -08:00
Harshavardhana ff4ce0ee14 fs/xl: Combine input checks into re-usable functions. (#3383)
Repeated code around both object layers are moved
and combined into simple re-usable functions.
2016-12-01 23:15:17 -08:00
Anis Elleuch 918924796f fs: Enable shutdown test with faulty disks (#3380) 2016-12-01 13:59:06 -08:00
Krishnan Parthasarathi feb6685359 posix: Use preparePath only for paths used with syscall or os functions (#3377) 2016-11-30 20:56:15 -08:00