Commit Graph

88 Commits

Author SHA1 Message Date
Krishna Srinivas
cea4cfa3a8 Implement S3 Gateway to third party cloud storage providers. (#3756)
Currently supported backend is Azure Blob Storage.

```
export MINIO_ACCESS_KEY=azureaccountname
export MINIO_SECRET_KEY=azureaccountkey
minio gateway azure
```
2017-03-16 12:21:58 -07:00
Harshavardhana
47ac410ab0 Code cleanup - simplify server side code. (#3870)
Fix all the issues reported by `gosimple` tool.
2017-03-08 10:00:47 -08:00
Krishna Srinivas
0bae3330e8 browser: Send correct arguments for RemoveObjects web handler. (#3848)
Fixes #3839
2017-03-03 17:07:17 -08:00
Anis Elleuch
0c8c463a63 tests: Fix web handlers testing with faulty disks (#3845) 2017-03-03 15:46:28 -08:00
Harshavardhana
e5d4e7aa9d web: Validate if bucket names are reserved (#3841)
Both '.minio.sys' and 'minio' should be never allowed
to be created from web-ui and then fail to list it
by filtering them out.

Fixes #3840
2017-03-03 03:01:42 -08:00
Krishna Srinivas
91cf54f895 web-handlers: Support removal of multiple objects at once. (#3810) 2017-02-28 19:07:28 -08:00
Rushan
52d6678bf0 Browser: Implement multi select user interface for object listings (#3730) 2017-02-22 14:51:38 -08:00
Krishna Srinivas
6800902b43 web-handlers: Implement API to download files as a zip file. (#3715) 2017-02-08 23:39:08 -08:00
Harshavardhana
31dff87903 Honor envs properly for access and secret key. (#3703)
Also changes the behavior of `secretKeyHash` which is
not necessary to be sent over the network, each node
has its own secretKeyHash to validate.

Fixes #3696
Partial(fix) #3700 (More changes needed with some code cleanup)
2017-02-07 12:51:43 -08:00
Anis Elleuch
ed4fcb63f7 Require content-length in POST & Upload requests (#3671)
Avoid passing size = -1 to PutObject API by requiring content-length
header in POST request (as AWS S3 does) and in Upload web handler.
Post handler is modified to completely store multipart file to know
its size before sending it to PutObject().
2017-02-02 10:45:00 -08:00
Harshavardhana
85f2b74cfd jwt: Cache the bcrypt password hash. (#3526)
Creds don't require secretKeyHash to be calculated
everytime, cache it instead and re-use.

This is an optimization for bcrypt.

Relevant results from the benchmark done locally, negative
value means improvement in this scenario.

```
benchmark                       old ns/op     new ns/op     delta
BenchmarkAuthenticateNode-4     160590992     80125647      -50.11%
BenchmarkAuthenticateWeb-4      160556692     80432144      -49.90%

benchmark                       old allocs     new allocs     delta
BenchmarkAuthenticateNode-4     87             75             -13.79%
BenchmarkAuthenticateWeb-4      87             75             -13.79%

benchmark                       old bytes     new bytes     delta
BenchmarkAuthenticateNode-4     15222         9785          -35.72%
BenchmarkAuthenticateWeb-4      15222         9785          -35.72%
```
2017-01-26 16:51:51 -08:00
Harshavardhana
62f8343879 Add constants for commonly used values. (#3588)
This is a consolidation effort, avoiding usage
of naked strings in codebase. Whenever possible
use constants which can be repurposed elsewhere.

This also fixes `goconst ./...` reported issues.
2017-01-18 12:24:34 -08:00
Harshavardhana
1c699d8d3f fs: Re-implement object layer to remember the fd (#3509)
This patch re-writes FS backend to support shared backend sharing locks for safe concurrent access across multiple servers.
2017-01-16 17:05:00 -08:00
Krishna Srinivas
12a7a15daa browser: Allow anonymous browsing of readable buckets. (#3515) 2017-01-11 13:26:42 -08:00
Harshavardhana
b0cfceb211 event: Enhance event message struct to provide origin server. (#3557)
`principalId` i.e user identity is kept as AccessKey in
accordance with S3 spec.

Additionally responseElements{} are added starting with

`x-amz-request-id` is a hexadecimal of the event time itself in nanosecs.
`x-minio-origin-server` - points to the server generating the event.

Fixes #3556
2017-01-10 16:43:48 -08:00
Harshavardhana
cae62ce543 browser: Handle proper login errors. (#3518)
Also additionally log the remote address.

Fixes #3514
2017-01-03 01:33:00 -08:00
Bala.FA
6d10f4c19a Adopt dsync interface changes and major cleanup on RPC server/client.
* Rename GenericArgs to AuthRPCArgs
* Rename GenericReply to AuthRPCReply
* Remove authConfig.loginMethod and add authConfig.ServiceName
* Rename loginServer to AuthRPCServer
* Rename RPCLoginArgs to LoginRPCArgs
* Rename RPCLoginReply to LoginRPCReply
* Version and RequestTime are added to LoginRPCArgs and verified by
  server side, not client side.
* Fix data race in lockMaintainence loop.
2017-01-02 20:57:42 +05:30
Bala FA
e8ce3b64ed Generate and use access/secret keys properly (#3498) 2016-12-26 10:21:23 -08:00
Harshavardhana
6efee2072d objectLayer: Check for format.json in a wrapped disk. (#3311)
This is needed to validate if the `format.json` indeed exists
when a fresh node is brought online.

This wrapped implementation also connects to the remote node
by attempting a re-login. Subsequently after a successful
connect `format.json` is validated as well.

Fixes #3207
2016-11-23 15:48:10 -08:00
Bala FA
825000bc34 Use humanize constants for KiB, MiB and GiB units. (#3322) 2016-11-22 18:18:22 -08:00
Harshavardhana
dd93f808c8 web: Add more data for jsonrpc responses. (#3296)
This change adds more richer error response
for JSON-RPC by interpreting object layer
errors to corresponding meaningful errors
for the web browser.

```go
&json2.Error{
   Message: "Bucket Name Invalid, Only lowercase letters, full stops, and numbers are allowed.",
}
```

Additionally this patch also allows PresignedGetObject()
to take expiry parameter to have variable expiry.
2016-11-22 11:12:38 -08:00
Harshavardhana
c91d3791f9 heal: Add healing support for bucket, bucket metadata files. (#3252)
This patch implements healing in general but it is only used
as part of quickHeal().

Fixes #3237
2016-11-16 16:42:23 -08:00
Harshavardhana
398421b9f5 xl/bootup: Server bootup shouldn't return for missing buckets. (#3255)
Ref #3196
2016-11-14 15:45:00 -08:00
Harshavardhana
2f373684f5 Fix the server startup messages and help text. (#3211) 2016-11-09 23:37:12 -08:00
Harshavardhana
33c771bb3e tests: Add tests for browser peer rpc. Fixes #3062 (#3189) 2016-11-07 11:43:35 -08:00
Harshavardhana
d9674f7524 Improve coverage of web-handlers.go (#3157)
This patch additionally relaxes the requirement for
accesskeys to be in a regexy set of values.

Fixes #3063
2016-11-02 14:45:11 -07:00
Harshavardhana
5782ec3ada Fix peers and web UIVersion validation. (#3048) 2016-10-23 12:32:35 -07:00
Aditya Manthramurthy
0aabc1d8d9 Use Peer RPC to propagate bucket policy changes (#2891) 2016-10-13 09:19:04 -07:00
Harshavardhana
6494b77d41 server: Add more elaborate startup messages. (#2731)
These messages based on our prep stage during XL
and prints more informative message regarding
drive information.

This change also does a much needed refactoring.
2016-10-05 12:48:07 -07:00
Krishna Srinivas
61a18ed48f sha256: Verify sha256 along with md5sum, signature is verified on the request early. (#2813) 2016-10-02 15:51:49 -07:00
Harshavardhana
cfbab22237 web: Remove bucket policy when we have no more statements. (#2779) 2016-09-26 03:11:22 -07:00
Harshavardhana
be0e06c0aa web: Simplify and rename GetAllBucketPolicy --> ListAllBucketPolicies. (#2778) 2016-09-25 21:53:19 -07:00
Krishna Srinivas
1e53316241 Add tests for presigned-get (#2767)
* web-handlers: support for presigned-get json-rpc call for MinioBrowser's "share" feature.

* Add tests for presigned-get
2016-09-23 01:25:49 -07:00
Bala FA
aa579bbc20 web: add method to get all policies for given bucket name. (#2756)
Refer #1858
2016-09-22 23:06:45 -07:00
Anis Elleuch
fc783f8407 More tests for web handlers (#2755)
* Return negative values of Total and Free in StorageInfo() when we fail to get disk info

* Return consistent messages in web handlers when the server is not initialized
2016-09-22 16:35:12 -07:00
Anis Elleuch
90417d2dd6 Check for bucket existence in Set/Get/Remove bucket policy workflow + tests (#2745) 2016-09-21 16:38:50 -07:00
Bala FA
7431acb2c4 web: add handlers for set/get bucket policy. (#2486)
Refer #1858
2016-08-30 10:04:50 -07:00
Harshavardhana
bccf549463 server: Move all the top level files into cmd folder. (#2490)
This change brings a change which was done for the 'mc'
package to allow for clean repo and have a cleaner
github drop in experience.
2016-08-18 16:23:42 -07:00