Commit Graph

3868 Commits

Author SHA1 Message Date
Krishnan Parthasarathi
b408d0e87d Add aws:Referer condition key support. (#3641)
This change implements bucket policy enhancements required to restrict access based on HTTP referer.
See https://docs.aws.amazon.com/AmazonS3/latest/dev/example-bucket-policies.html#example-bucket-policies-use-case-4 for more information.

Fixes #3540
2017-01-29 19:45:11 -08:00
Nitish Tiwari
69b81af93e Fix Minio quick start guide URL (#3655) 2017-01-29 07:21:30 -08:00
Harshavardhana
cf558ff6d1 docs: Fix TLS doc to be docs.minio.io compatible 2017-01-28 22:46:29 -08:00
Harshavardhana
5af7bd6a01 docs: Fix a typo in TLS doc description 2017-01-28 22:41:52 -08:00
Harshavardhana
73f4f29110 docs: Restructure docs, move the files to their relevant location. (#3648)
Also combines windows TLS docs with single doc with Linux.
2017-01-28 19:45:30 -08:00
Harshavardhana
ccd949d8ca Add -v for race tests to avoid travis and appveyor timeouts. (#3647)
This is added so that CI's don't wrong fully timeout on
little longer running tests.
2017-01-27 12:31:02 -08:00
Anis Elleuch
e1bc99e4fe xl: Fix GET of an empty multiparted object (#3646)
GetObject returns unsatisfied range error when we try to download an object
uploaded using multipart mechanism.
2017-01-27 10:51:02 -08:00
Alex Ellis
02194ee3c6 Fix issue with webhook events not closing Response body, leaving connections open with remote server (#3645) 2017-01-27 09:50:08 -08:00
Harshavardhana
85f2b74cfd jwt: Cache the bcrypt password hash. (#3526)
Creds don't require secretKeyHash to be calculated
everytime, cache it instead and re-use.

This is an optimization for bcrypt.

Relevant results from the benchmark done locally, negative
value means improvement in this scenario.

```
benchmark                       old ns/op     new ns/op     delta
BenchmarkAuthenticateNode-4     160590992     80125647      -50.11%
BenchmarkAuthenticateWeb-4      160556692     80432144      -49.90%

benchmark                       old allocs     new allocs     delta
BenchmarkAuthenticateNode-4     87             75             -13.79%
BenchmarkAuthenticateWeb-4      87             75             -13.79%

benchmark                       old bytes     new bytes     delta
BenchmarkAuthenticateNode-4     15222         9785          -35.72%
BenchmarkAuthenticateWeb-4      15222         9785          -35.72%
```
2017-01-26 16:51:51 -08:00
Krishna Srinivas
152cdf1c05 fs: Move traceError() to lower functions where possible. (#3633) 2017-01-26 15:40:10 -08:00
Krishna Srinivas
17dd1c19df cleanup: refactor common code between FS and XL listDirFactory. (#3639) 2017-01-26 15:39:22 -08:00
Anis Elleuch
8e49a3d047 Simplify running cmd.Main() for external tests (#3636)
An external test that runs cmd.Main() has a difficulty to set cmd arguments
and MINIO_{ACCESS,SECRET}_KEY values, this commit changes a little the current
behavior in a way that helps external tests.
2017-01-26 15:22:41 -08:00
Krishna Srinivas
cccf77229d cleanup: Move code in *-multipart-common.go to *-multipart.go (#3638)
The code in *-multipart-common.go is not common anymore.
2017-01-26 12:51:12 -08:00
Harshavardhana
dafdc74605 fs: if fs.json is empty ignore it while reading metadata. (#3634)
This is needed so that we don't send wrong errors
on previously failed PutObject() which would have
left a stale `fs.json` entry.
2017-01-26 10:19:07 -08:00
Anis Elleuch
c753b5dfeb madmin: All APIs return ErrorResponse error type (#3632)
Avoid returning errors.New("Got HTTP Status") in all APIs
2017-01-25 16:01:29 -08:00
Krishna Srinivas
82373e3d50 fs: cleanup - do not cache size of metafiles (#3630)
* Remove Size() method and size field from lock.LockedFile
* WriteTo method of fsMeta and uploadsV1 now takes concrete type *lock.LockedFile
2017-01-25 12:29:06 -08:00
Anis Elleuch
76b8abcd8e madmin: Fix args order in listObjectsHeal() (#3629)
The order of marker and delimiter and in listObjectsHeal() internal function
are switched. That will give wrong result in case of a non recursive objects
heal list.
2017-01-25 09:58:49 -08:00
Harshavardhana
dbb5408906 Add ARM64 build binary for README.md
Fixes #3626
2017-01-24 20:28:54 -08:00
Harshavardhana
18c9d49a32 Build browser with new changes and update yarn.lock 2017-01-24 19:11:37 -08:00
Harshavardhana
d41dcb784b Move to blake2b-simd due to perf problems in golang.org/x/crypto
Ref https://github.com/golang/go/issues/18563
2017-01-24 18:07:34 -08:00
Krishna Srinivas
659d5aabd1 browser: Access to private paths redirects to login. (#3622) 2017-01-24 12:08:00 -08:00
Krishna Srinivas
b4343a28b7 browser: add yarn.lock and .gitignore files. (#3621) 2017-01-24 11:56:30 -08:00
Krishnan Parthasarathi
0e693e0284 Add dry-run query param for HealFormat API (#3618) 2017-01-24 08:11:05 -08:00
Anis Elleuch
fc880f9b23 admin: Enhance set credentials test (#3619)
Add more test cases and ignore access and secret keys set from the env
2017-01-24 08:08:36 -08:00
Krishna Srinivas
cead24b0f7 miniobrowser: Bring Minio browser source into minio repo. (#3617) 2017-01-23 18:07:22 -08:00
Krishna Srinivas
8489f22fe2 signature-v2: Use request.RequestURI for signature calculation. (#3616)
* signature-v2: Use request.RequestURI for signature calculation.

* Use splitStr instead of strings.Split
2017-01-23 17:01:44 -08:00
Anis Elleuch
fc6f804865 server-mux: Keep listening after Accept() err (#3613)
Accept() can return errors like: `too many open files`, no need to totally quit listening in this case.
2017-01-23 09:55:34 -08:00
Anis Elleuch
d1d89116f1 admin: Add version to service Status API response (#3605)
Add server's version field to service status API:

"version":{
	"version":"DEVELOPMENT.GOGET",
	"commitID":"DEVELOPMENT.GOGET"
}
2017-01-23 08:56:06 -08:00
Anis Elleuch
e1f64141a2 presign-v2: Compute signature on encoded URL path (#3612)
Encode the path of the passed presigned url before calculating the signature. This fixes
presigning objects whose names contain characters that are found encoded in urls.
2017-01-23 08:54:32 -08:00
Krishnan Parthasarathi
586058f079 Implement mgmt REST APIs to heal storage format. (#3604)
* Implement heal format REST API handler
* Implement admin peer rpc handler to re-initialize storage
* Implement HealFormat API in pkg/madmin
* Update pkg/madmin API.md to incl. HealFormat
* Added unit tests for ReInitDisks rpc handler and HealFormatHandler
2017-01-23 00:32:55 -08:00
Krishna Srinivas
4e926b292f vendor-update: Minio Browser (#3609) 2017-01-22 21:02:09 -08:00
Anis Elleuch
47358e3104 server-mux: Add tcp idle read timeout (#3607)
Avoid many idle client connections i.e client didn't send any data until a given stipulated amount of time. 

Default chosen here is 30 seconds.
2017-01-22 14:48:27 -08:00
Harshavardhana
3640c63289 server/mux: PeekProtocol() should return error and connection be closed. (#3608)
For TLS peekProtocol do not assume the incoming request to be a TLS
connection perform a handshake() instead and validate.

Also add some security related defaults to `tls.Config`.
2017-01-22 12:14:00 -08:00
Harshavardhana
51fa4f7fe3 Make PutObject a nop for an object which ends with "/" and size is '0' (#3603)
This helps majority of S3 compatible applications while not returning
an error upon directory create request.

Fixes #2965
2017-01-20 16:33:01 -08:00
Andrei Kopats
c3f7d1026f fs: start even if there are not enough free space (#3606) 2017-01-20 09:30:20 -08:00
Harshavardhana
80f1387877 Initialize peers properly for localhost. (#3600)
This introduced a regression.

Fixes #3594
2017-01-19 11:32:13 -08:00
Andreas Kohn
0674fa43ff Handle the region for GetBucketLocation and PutBucket properly (#3596)
This adjusts the code for these two handlers to match the logic in ListBucketHandler.

Fixes #3595
2017-01-19 11:31:51 -08:00
Harshavardhana
a17f1e875c server/mux: Close the connection even if buffer.Flush() returns error. (#3599)
It is possible that buf.Flush() might return an error, leading to a
potential leak in active sockets.
2017-01-19 11:19:57 -08:00
Jeffery Utter
9e1f1b50e0 Don't Check Available Inodes on NFS (#3598)
In some cases (such as with VirutualBox, this value gets hardcoded
to 1000, which is less than the required minimum of 10000.

Fixes #3592
2017-01-19 10:39:44 -08:00
Anis Elleuch
0715032598 heal: Add ListBucketsHeal object API (#3563)
ListBucketsHeal will list which buckets that need to be healed:
  * ListBucketsHeal() (buckets []BucketInfo, err error)
2017-01-19 09:34:18 -08:00
Harshavardhana
dfc2ef3004 storage/rpc: Remove network error restriction. (#3591)
This restriction has lots of side affects, since
we do not have a mechanism to clear states like
this it is better not to keep them.

Network errors are common and can occur with
simple cable removal etc. Since we already have
a retry mechanism this error count and stateful
nature can bring problems on a long running
cluster.
2017-01-18 12:55:57 -08:00
Harshavardhana
62f8343879 Add constants for commonly used values. (#3588)
This is a consolidation effort, avoiding usage
of naked strings in codebase. Whenever possible
use constants which can be repurposed elsewhere.

This also fixes `goconst ./...` reported issues.
2017-01-18 12:24:34 -08:00
Anis Elleuch
f803bb4b3d admin: Add service Set Credentials API (#3580) 2017-01-17 14:25:59 -08:00
Andy Brown
20a65981bd add delete and options methods to CORS whitelist (#3589) 2017-01-17 14:20:05 -08:00
Harshavardhana
09b450d610 Fix fs tests to avoid deleting /usr to certain systems. 2017-01-17 14:05:07 -08:00
Krishnan Parthasarathi
c194b9f5f1 Implement mgmt REST APIs for heal subcommands (#3533)
The heal APIs supported in this change are,
- listing of objects to be healed.
- healing a bucket.
- healing an object.
2017-01-17 10:02:58 -08:00
Harshavardhana
98a6a2bcab obj: Return objectInfo for CompleteMultipartUpload(). (#3587)
This patch avoids doing GetObjectInfo() in similar way
how we did for PutOject().
2017-01-16 19:23:43 -08:00
Harshavardhana
1c699d8d3f fs: Re-implement object layer to remember the fd (#3509)
This patch re-writes FS backend to support shared backend sharing locks for safe concurrent access across multiple servers.
2017-01-16 17:05:00 -08:00
Harshavardhana
a054c73e22 Add slack replace gitter (#3584) 2017-01-16 15:26:26 -08:00
Harshavardhana
b580ad24e2 server/http: Add missing keep alive for incoming tcp connections. (#3585)
This was seen reproducing a bug with @gowithplanb. Windows Cloud
Berry clients do not close their respective connections.
2017-01-16 03:38:06 -08:00