MyFavMirrors/minio
1
0
Fork 0
mirror of https://github.com/minio/minio.git synced 2025-02-03 18:06:00 -05:00
Code Issues Packages Projects Releases Wiki Activity
4,185 Commits 4 Branches 508 Tags
Commit Graph

70 Commits

Author SHA1 Message Date
Aditya Manthramurthy
a02575ebf9 Bump up minio-go to (fixes #4243) (#4256) 2017-05-04 13:43:23 -07:00
Krishna Srinivas
e5b2e25caf gateway-s3: vendor-update minio-go (#4220) 2017-05-02 18:46:39 -07:00
Remco Verhoef
3a539ce660 Implement gateway S3 support (#3940) 2017-04-27 11:26:00 -07:00
Harshavardhana
f1d7780167 lock: Vendorize all the new changes made in minio/dsync (#4154) 
Fixes #4139
 2017-04-19 14:22:35 -07:00
Bala FA
de204a0a52 Add extensive endpoints validation (#4019) 2017-04-11 15:44:27 -07:00
Krishna Srinivas
cea4cfa3a8 Implement S3 Gateway to third party cloud storage providers. (#3756) 
Currently supported backend is Azure Blob Storage.

```
export MINIO_ACCESS_KEY=azureaccountname
export MINIO_SECRET_KEY=azureaccountkey
minio gateway azure
```
 2017-03-16 12:21:58 -07:00
Harshavardhana
3edff1501e Vendor upstream redis library instead of our fork. (#3913) 
We forked the upstream to address a build issue on
go 1.6 - that is long done and we don't need
to manage our forks anymore.
 2017-03-16 08:22:47 -07:00
Harshavardhana
9eb8e375c5 cli: Make sure to add --help flag for subcommands. (#3773) 
--help is now back and prints properly with command
help template.
 2017-02-19 20:46:06 -08:00
Harshavardhana
25eeb88a8f cli: Bring upstream changes to minio server. 2017-02-15 17:30:31 -08:00
Andreas Auernhammer
f38222c0cc update the blake2b implementation (#3724) 
Fixes a performance bug caused by SSE-AVX register savings on amd64.
 2017-02-09 15:01:00 -08:00
Harshavardhana
d41dcb784b Move to blake2b-simd due to perf problems in golang.org/x/crypto 
Ref https://github.com/golang/go/issues/18563
 2017-01-24 18:07:34 -08:00
Krishna Srinivas
cead24b0f7 miniobrowser: Bring Minio browser source into minio repo. (#3617) 2017-01-23 18:07:22 -08:00
Krishna Srinivas
4e926b292f vendor-update: Minio Browser (#3609) 2017-01-22 21:02:09 -08:00
Bala.FA
6d10f4c19a Adopt dsync interface changes and major cleanup on RPC server/client. 
* Rename GenericArgs to AuthRPCArgs
* Rename GenericReply to AuthRPCReply
* Remove authConfig.loginMethod and add authConfig.ServiceName
* Rename loginServer to AuthRPCServer
* Rename RPCLoginArgs to LoginRPCArgs
* Rename RPCLoginReply to LoginRPCReply
* Version and RequestTime are added to LoginRPCArgs and verified by
  server side, not client side.
* Fix data race in lockMaintainence loop.
 2017-01-02 20:57:42 +05:30
Andreas Auernhammer
1ac36a95aa replace blake2b implementation (#3481) 
* replace blake2b implementation
replace the blake2b-simd with the golang/x/crypto implementation

```
name        old time/op    new time/op     delta
Size64-8       715ns ±13%      614ns ± 3%    ~     (p=0.084 n=6+6)
Size128-8      612ns ± 5%      634ns ± 8%    ~     (p=0.084 n=6+6)
Size1K-8      2.18µs ± 5%     2.09µs ± 7%    ~     (p=0.084 n=6+6)
Size8K-8      13.1µs ± 2%     13.4µs ± 3%    ~     (p=0.084 n=6+6)
Size32K-8     48.5µs ± 1%     49.5µs ± 3%    ~     (p=0.775 n=6+6)
Size128K-8     199µs ± 0%      198µs ± 3%    ~     (p=0.468 n=6+6)

name        old speed      new speed       delta
Size64-8    92.6MB/s ±11%  104.2MB/s ± 3%    ~     (p=0.139 n=6+6)
Size128-8    208MB/s ± 6%    202MB/s ± 8%    ~     (p=0.102 n=6+6)
Size1K-8     466MB/s ± 7%    492MB/s ± 7%    ~     (p=0.139 n=6+6)
Size8K-8     621MB/s ± 2%    610MB/s ± 3%    ~     (p=0.102 n=6+6)
Size32K-8    672MB/s ± 2%    669MB/s ± 1%    ~     (p=0.818 n=6+6)
Size128K-8   657MB/s ± 1%    672MB/s ± 0%  +2.28%  (p=0.002 n=6+6)

name        old time/op   new time/op   delta
Size64-4      334ns ± 1%    243ns ± 0%  -27.14%  (p=0.029 n=4+4)
Size128-4     296ns ± 1%    242ns ± 0%  -18.21%  (p=0.029 n=4+4)
Size1K-4     1.44µs ± 0%   1.28µs ± 0%  -10.83%  (p=0.029 n=4+4)
Size8K-4     10.0µs ± 0%    9.4µs ± 0%   -6.23%  (p=0.029 n=4+4)
Size32K-4    39.8µs ± 1%   37.3µs ± 0%   -6.31%  (p=0.029 n=4+4)
Size128K-4    162µs ± 3%    149µs ± 0%   -7.72%  (p=0.029 n=4+4)

name        old speed     new speed     delta
Size64-4    192MB/s ± 1%  263MB/s ± 0%  +37.24%  (p=0.029 n=4+4)
Size128-4   431MB/s ± 0%  526MB/s ± 0%  +22.04%  (p=0.029 n=4+4)
Size1K-4    713MB/s ± 0%  800MB/s ± 0%  +12.17%  (p=0.029 n=4+4)
Size8K-4    815MB/s ± 0%  869MB/s ± 0%   +6.64%  (p=0.029 n=4+4)
Size32K-4   823MB/s ± 1%  878MB/s ± 0%   +6.72%  (p=0.029 n=4+4)
Size128K-4  810MB/s ± 3%  877MB/s ± 0%   +8.23%  (p=0.029 n=4+4)
```
See: https://go-review.googlesource.com/#/c/34319/
 2016-12-21 14:20:01 -08:00
Harshavardhana
1d134c1a94 admin: Fix wrong vendorized location for s3signer and s3utils. 2016-12-20 15:18:37 -08:00
Harshavardhana
faa6b1e925 vendorize deps for snappy, blake2b and sha256 (#3476) 
Bring in new optimization and portability changes.

Fixes https://github.com/minio/minio-go/issues/578
 2016-12-19 19:32:55 -08:00
Krishnan Parthasarathi
a609a4126c Vendorize recent fixes in dsync (#3365) 
* Update 'i' only if lock grant read from buffered channel
 2016-11-28 23:03:46 -08:00
Harshavardhana
12c1abed98 Vendorize with bug fixes from minio browser. (#3341) 
This patch brings in changes from miniobrowser repo.

- Bucket policy UI and functionality fixes by @krishnasrinivas
- Bucket policy implementation by @balamurugana
- UI changes and new functionality changing password etc. @rushenn
- UI and new functionality for sharing URLs, deleting files
  @rushenn and @krishnasrinivas.
- Other misc fixes by @vadmeste @brendanashworth
 2016-11-23 17:31:11 -08:00
Harshavardhana
7a5bbf7a2e Revert "Vendor update for dsync, fixing major go routine leak issue. (#3308)" 
This reverts commit 273228fafa3e43dce6b298a2f34905c811a1446a.
 2016-11-23 15:17:41 -08:00
Karthic Rao
273228fafa Vendor update for dsync, fixing major go routine leak issue. (#3308) 2016-11-21 10:47:48 -08:00
Frank
ea406754a6 New dsync and added ForceUnlock to lock rpc server (#2956) 
* Update dsync and added ForceUnlock function
* Added test cases for ForceUnlock
 2016-10-17 01:53:29 -07:00
Harshavardhana
a681af6953 Update minio browser with new changes. (#2940) 
- Bucket policy set/unset support.
- Shareable URL support.
- Delete object support.
 2016-10-15 08:51:53 -07:00
Frank
e53a9f6cab Update vendorized version of dsync that relaxes read quorum to N/2 (#2874) 2016-10-07 08:02:59 -07:00
Bala FA
aa579bbc20 web: add method to get all policies for given bucket name. (#2756) 
Refer #1858
 2016-09-22 23:06:45 -07:00
Frank
df2ef64d20 Upgrade to new dsync version incl. stale lock detection (#2708) 2016-09-16 00:30:55 -07:00
Harshavardhana
f655592ff5 vendorize new dsync with new changes of logging. 2016-09-13 21:18:30 -07:00
Harshavardhana
9605fde04d controller/auth: Implement JWT based authorization for controller. (#2544) 
Fixes #2474
 2016-09-13 21:18:30 -07:00
Krishnan Parthasarathi
bda6bcd5be Layered rpc-client implementation (#2512) 2016-09-13 21:18:30 -07:00
Krishnan Parthasarathi
c33d1b8ee6 Vendorize minio/dsync for server-side read lock (#2484) 
- Prevention of stale lock accumulation.
- Removal of dead code.
 2016-09-13 21:18:30 -07:00
Harshavardhana
83074ed57e Add missing rpc-client.go - missing in previous commit. 2016-09-13 21:18:30 -07:00
Krishnan Parthasarathi
64a7e6992e Vendorize rpc reconnect changes from minio/dsync (#2405) 2016-09-13 21:18:30 -07:00
Krishnan Parthasarathi
b0f3f94163 unify single-node and distributed namespace locking (#2401) 2016-09-13 21:18:30 -07:00
Krishnan Parthasarathi
b7c169d71d object layer initialization using distributed locking (#2397) 
* vendorized latest minio/dsync

* wip - object layer initialization using distributed locking
 2016-09-13 21:18:30 -07:00
Krishnan Parthasarathi
e55926e8cf distribute: Make server work with multiple remote disks 
This change initializes rpc servers associated with disks that are
local. It makes object layer initialization on demand, namely on the
first request to the object layer.

Also adds lock RPC service vendorized minio/dsync
 2016-09-13 21:18:30 -07:00
Bala FA
7431acb2c4 web: add handlers for set/get bucket policy. (#2486) 
Refer #1858
 2016-08-30 10:04:50 -07:00
Harshavardhana
975eb31973 api/bucketPolicy: Use minio-go/pkg/set and fix bucket policy regression. (#2506) 
Current master has a regression 'mc policy <policy-type> alias/bucket/prefix'
does not work anymore, due to the way new minio-go changes do json marshalling.
This led to a regression on server side when a ``prefix`` is provided
policy is rejected as malformed from th server which is not the case with
AWS S3.

This patch uses the new ``minio-go/pkg/set`` package to address the
unmarshalling problems.

Fixes #2503
 2016-08-20 03:16:38 -07:00
Harshavardhana
0a3d43273f vendor: sha256 32bit updated. (#2459) 2016-08-16 16:19:29 -07:00
Frank
f239fcac67 Switched to faster minio/sha256-simd implementation (#2320) 2016-07-28 14:44:37 -07:00
Harshavardhana
f248089523 api: Implement bucket notification. (#2271) 
* Implement basic S3 notifications through queues

Supports multiple queues and three basic queue types:

1. NilQueue -- messages don't get sent anywhere
2. LogQueue -- messages get logged
3. AmqpQueue -- messages are sent to an AMQP queue

* api: Implement bucket notification.

Supports two different queue types

- AMQP
- ElasticSearch.

* Add support for redis
 2016-07-23 22:51:12 -07:00
Harshavardhana
169c72cdab vendor: Bring new updates from blake2b-simd repo. (#2094) 
This vendorization is needed to bring in new improvements
and support for AVX2 and SSE.

Fixes #2081
 2016-07-06 18:24:31 -07:00
Harshavardhana
48ac34919f browser: Add new release for ui-assets.go (#2070) 
update `ui-assets.go` using `x-amz-date` for JSON rpc.
 2016-07-02 10:54:17 -07:00
Harshavardhana
d484157d67 XL/bitrot: Migrate to new blake2b-simd SIMD optimized implementation. (#2031) 
Thanks for Frank Wessels <fwessels@xs4all.nl> for all the heavy lifting work.

Comparative benchmarks are as below.
```
benchmark                old ns/op     new ns/op     delta
BenchmarkHash64-4        742           411           -44.61%
BenchmarkHash128-4       681           346           -49.19%
BenchmarkWrite1K-4       4239          1497          -64.69%
BenchmarkWrite8K-4       33633         11514         -65.77%
BenchmarkWrite32K-4      134091        45947         -65.73%
BenchmarkWrite128K-4     537976        183643        -65.86%

benchmark                old MB/s     new MB/s     speedup
BenchmarkHash64-4        86.18        155.51       1.80x
BenchmarkHash128-4       187.96       369.10       1.96x
BenchmarkWrite1K-4       241.55       683.87       2.83x
BenchmarkWrite8K-4       3897.06      11383.41     2.92x
BenchmarkWrite32K-4      977.48       2852.63      2.92x
BenchmarkWrite128K-4     243.64       713.73       2.93x
```

Fixes #2030
 2016-06-29 02:06:35 -07:00
Harshavardhana
2e4ab71303 Web: Update with ui changes. (#1808) 2016-05-31 02:01:02 -07:00
Harshavardhana
444d1f8a65 miniobrowser: Vendorize to new changes in miniobrowser. 2016-04-21 20:35:48 -07:00
Harshavardhana
8603185f2f browser: Add new ui-assets.go 2016-04-08 01:47:30 -07:00
Harshavardhana
ba3a5805c1 vendorize: Add updated ui-assets.go. 2016-04-02 17:27:36 -07:00
Krishna Srinivas
331890c4c8 UI-handler: remove minio-go dependancy. 2016-04-01 13:56:32 +05:30
Harshavardhana
aaf97ea02c config/main: Re-write config files - add to new config v3 
- New config format.

```
{
	"version": "3",
	"address": ":9000",
    "backend": {
          "type": "fs",
          "disk": "/path"
    },
	"credential": {
		"accessKey": "WLGDGYAQYIGI833EV05A",
		"secretKey": "BYvgJM101sHngl2uzjXS/OBF/aMxAN06JrJ3qJlF"
	},
	"region": "us-east-1",
	"logger": {
		"file": {
			"enable": false,
			"fileName": "",
			"level": "error"
		},
		"syslog": {
			"enable": false,
			"address": "",
			"level": "debug"
		},
		"console": {
			"enable": true,
			"level": "fatal"
		}
	}
}
```

New command lines in lieu of supporting XL.

Minio initialize filesystem backend.
~~~
$ minio init fs <path>
~~~

Minio initialize XL backend.
~~~
$ minio init xl <url1>...<url16>
~~~

For 'fs' backend it starts the server.
~~~
$ minio server
~~~

For 'xl' backend it waits for servers to join.
~~~
$ minio server
... [PROGRESS BAR] of servers connecting
~~~

Now on other servers execute 'join' and they connect.
~~~
....
minio join <url1> -- from <url2> && minio server
minio join <url1> -- from <url3> && minio server
...
...
minio join <url1> -- from <url16> && minio server
~~~
 2016-03-23 19:16:09 -07:00
Harshavardhana
5606232567 vendor: Update ui-assets with new changes and release. 2016-03-10 17:36:32 -08:00