Commit Graph

32 Commits

Author SHA1 Message Date
Harshavardhana f248089523 api: Implement bucket notification. (#2271)
* Implement basic S3 notifications through queues

Supports multiple queues and three basic queue types:

1. NilQueue -- messages don't get sent anywhere
2. LogQueue -- messages get logged
3. AmqpQueue -- messages are sent to an AMQP queue

* api: Implement bucket notification.

Supports two different queue types

- AMQP
- ElasticSearch.

* Add support for redis
2016-07-23 22:51:12 -07:00
Harshavardhana e5dd917c37 handlers/generic: Remove support for 'x-minio-date' (#2064) 2016-07-02 01:51:17 -07:00
Frank a4fef436c8 Fix formatting for adding ETag for exposed headers (#1687) 2016-05-18 19:17:17 -07:00
Harshavardhana 30b0b4deba storage/server/client: Enable storage server, enable client storage. 2016-04-16 16:25:53 -07:00
Harshavardhana 02ad48466d error: Signature errors should be returned with APIErrorCode.
The reasoning is that we can reply back with wide range of
S3 error responses, which would provide more richer context
to S3 client.

Fixes #1267
2016-03-31 23:28:40 -07:00
Harshavardhana aa8c9bad54 routers: Move API and Web routers into their own files.
This is done to ensure we have a clean way to add new routers such as

  - diskRouter
  - configRouter
  - lockRouter
2016-03-27 13:28:36 -07:00
Harshavardhana 9dca46e156 signature: Use a layered approach for signature verification.
Signature calculation has now moved out from being a package to
top-level as a layered mechanism.

In case of payload calculation with body, go-routines are initiated
to simultaneously write and calculate shasum. Errors are sent
over the writer so that the lower layer removes the temporary files
properly.
2016-03-26 15:21:05 -07:00
Harshavardhana aaf97ea02c config/main: Re-write config files - add to new config v3
- New config format.

```
{
	"version": "3",
	"address": ":9000",
    "backend": {
          "type": "fs",
          "disk": "/path"
    },
	"credential": {
		"accessKey": "WLGDGYAQYIGI833EV05A",
		"secretKey": "BYvgJM101sHngl2uzjXS/OBF/aMxAN06JrJ3qJlF"
	},
	"region": "us-east-1",
	"logger": {
		"file": {
			"enable": false,
			"fileName": "",
			"level": "error"
		},
		"syslog": {
			"enable": false,
			"address": "",
			"level": "debug"
		},
		"console": {
			"enable": true,
			"level": "fatal"
		}
	}
}
```

New command lines in lieu of supporting XL.

Minio initialize filesystem backend.
~~~
$ minio init fs <path>
~~~

Minio initialize XL backend.
~~~
$ minio init xl <url1>...<url16>
~~~

For 'fs' backend it starts the server.
~~~
$ minio server
~~~

For 'xl' backend it waits for servers to join.
~~~
$ minio server
... [PROGRESS BAR] of servers connecting
~~~

Now on other servers execute 'join' and they connect.
~~~
....
minio join <url1> -- from <url2> && minio server
minio join <url1> -- from <url3> && minio server
...
...
minio join <url1> -- from <url16> && minio server
~~~
2016-03-23 19:16:09 -07:00
Harshavardhana fdf3d64793 error: Add proper prefixes for s3Error codes.
This patch adds 'Err' prefix for all standard API
error codes and also adds a proper type for them.
2016-03-10 18:38:46 -08:00
Harshavardhana d5057b3c51 accessPolicy: Implement Put, Get, Delete access policy.
This patch implements Get,Put,Delete bucket policies

Supporting - http://docs.aws.amazon.com/AmazonS3/latest/dev/access-policy-language-overview.html

Currently supports following actions.

   "*":                             true,
   "s3:*":                          true,
   "s3:GetObject":                  true,
   "s3:ListBucket":                 true,
   "s3:PutObject":                  true,
   "s3:CreateBucket":               true,
   "s3:GetBucketLocation":          true,
   "s3:DeleteBucket":               true,
   "s3:DeleteObject":               true,
   "s3:AbortMultipartUpload":       true,
   "s3:ListBucketMultipartUploads": true,
   "s3:ListMultipartUploadParts":   true,

following conditions for "StringEquals" and "StringNotEquals"

   "s3:prefix", "s3:max-keys"
2016-03-08 17:44:50 -08:00
Harshavardhana 761cb2c740 handlers: Cleanup time handlers helpers. 2016-03-07 10:47:45 -08:00
Harshavardhana aed62788d9 api: Implement multiple objects Delete api - fixes #956
This API takes input XML input in following form.

```
<?xml version="1.0" encoding="UTF-8"?>
<Delete>
    <Quiet>true</Quiet>
    <Object>
         <Key>Key</Key>
    </Object>
    <Object>
         <Key>Key</Key>
    </Object>
    ...
</Delete>
```

and responds the list of successful deletes, list of errors
for all the deleted objects.

```
<?xml version="1.0" encoding="UTF-8"?>
<DeleteResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Deleted>
    <Key>sample1.txt</Key>
  </Deleted>
  <Error>
    <Key>sample2.txt</Key>
    <Code>AccessDenied</Code>
    <Message>Access Denied</Message>
  </Error>
</DeleteResult>
```
2016-03-06 18:31:50 -08:00
Harshavardhana 3ff8a1b719 api: Implement CopyObject s3 API, doing server side copy.
Fixes #1172
2016-02-27 19:51:59 -08:00
Harshavardhana 2520298734 Merge pull request #1170 from krishnasrinivas/caching
caching: disable caching of index.html and enable caching for other UI asset files.
2016-02-27 00:02:51 -08:00
Krishna Srinivas af7170675d caching: disable caching of index.html and enable caching for other UI asset files. 2016-02-27 12:22:26 +05:30
Harshavardhana ae6e774377 api: Return NotImplemented for MultiDelete and CopyObject APIs 2016-02-26 15:57:30 -08:00
Krishna Srinivas e509bcb2b9 UI: serve index.html if the requested file is not found in the assets bundle. 2016-02-24 00:36:30 +05:30
Anand Babu (AB) Periasamy b31dac9162 Merge pull request #1144 from harshavardhana/definitions
cleanup: Remove definitions and move them to its relative places acco…
2016-02-23 01:50:17 +05:30
Harshavardhana 800b19d8e5 cleanup: Remove definitions and move them to its relative places accordingly
- Move fs-definitions.go and break them into fs-datatypes.go, fs-bucket-acl.go
  and fs-utils.go
- Move api-definitions.go to api-response.go, where they should be.
- Move web-definitions to its related handlers.
2016-02-22 10:41:27 -08:00
Harshavardhana 5da1972d1f router: Fix "/minio" router for web. 2016-02-21 22:59:01 -08:00
Harshavardhana 91a7b13529 web: Handle private bucket match from prefix to exact match.
Filter out 'privateBucket' if any from listBuckets output.
2016-02-20 02:22:56 -08:00
Harshavardhana dd9aaa855c web/rpc: Merge ports with API server.
Fixes #1081 and #1130
2016-02-17 20:28:15 -08:00
Harshavardhana 5a9333a67b signature: Rewrite signature handling and move it into a library. 2016-02-16 17:28:16 -08:00
Harshavardhana 9b29af8bbe listObjects: list objects minor optimization.
Minor optimization.

- Add 1000 entries buffered channel for walkerCh.
- Reset marker after the lexical order has reached and
  compare only if the marker is set.
2016-02-09 21:45:19 -08:00
Harshavardhana 012fbe756b handlers: Fix the naming of all handlers. 2016-02-04 15:02:53 -08:00
Krishna Srinivas a344e7713a browser-caching: enable browser caching for WebUI 2016-02-05 03:54:05 +05:30
Krishna Srinivas 81b255511f CORS: cors handling should be before auth handling. cors should allow PUT. 2016-01-28 22:54:03 +05:30
Harshavardhana 3c71c5c80c s3cmd: Handle support for s3cmd. 2015-12-09 17:16:53 -08:00
Harshavardhana 57430fe183 ignore-handlers: Enhance ignore handlers to cater for bucket resources with or without separators
Fixes an issue which we saw with minio-py
2015-12-07 12:06:27 -08:00
Harshavardhana bd0436bf98 Change default options for cors to handle HEAD and allow all headers 2015-10-30 11:56:50 -07:00
Harshavardhana 1256ca86d0 Add fs separator 2015-10-17 12:05:12 -07:00
Harshavardhana 7fea9cb550 Add rpc signature handler 2015-10-08 22:28:11 -07:00