Commit Graph

142 Commits

Author SHA1 Message Date
Harshavardhana efbf7dbc0f api: Bucket notification add filter rules check and validate. (#2272)
These filtering techniques are used to validate
object names for their prefix and suffix.
2016-07-25 17:53:55 -07:00
Harshavardhana f248089523 api: Implement bucket notification. (#2271)
* Implement basic S3 notifications through queues

Supports multiple queues and three basic queue types:

1. NilQueue -- messages don't get sent anywhere
2. LogQueue -- messages get logged
3. AmqpQueue -- messages are sent to an AMQP queue

* api: Implement bucket notification.

Supports two different queue types

- AMQP
- ElasticSearch.

* Add support for redis
2016-07-23 22:51:12 -07:00
Harshavardhana 4cfbdb1bf0 server: Remove deadcode/deprecated code. (#2088) 2016-07-04 14:46:38 -07:00
Harshavardhana 57146fbbb8 main: minio --help should print regardless of root. (#1985)
Remove root check entirely.

Fixes #1964
2016-06-26 03:03:52 -07:00
Harshavardhana de21126f7e XL: Re-align the code again. 2016-06-02 01:54:06 -07:00
Harshavardhana 64b0976e1b Remove probe and tasker. (#1733)
Fixes #1717
2016-05-24 18:43:33 -07:00
Krishnan Parthasarathi 584813e214 Used MINIO_PROFILE_DIR for saving profile information of a minio server (#1722)
To specify the directory where profiling information should be saved
  ```
    export MINIO_PROFILE_DIR=/path/to/profile/dir
  ```
By default, profiling information would be saved in a directory created
using ioutil.TempDir, which would be displayed in stdout on starting the
minio server.
2016-05-22 22:11:39 -07:00
Anand Babu (AB) Periasamy b8405ca172 simplify profiler cleanup 2016-05-19 19:19:32 -07:00
Krishnan Parthasarathi 2f05aacbf2 Stop profiling on exit of main goroutine (#1670)
* Stop profiling on exit of main goroutine

Previously, profiling was stopped since Stop() method was called on exit of cli.BeforeFunc.
This lead to profiling to be stopped prematurely.

* Moved profiling switch statement to a separate func
2016-05-19 14:50:54 -07:00
Krishnan Parthasarathi 596fe65e84 Write pprof output files under config dir supplied (#1660)
Since config dir, supplied as command line argument, is parsed after pprof
output directory is determined, pprof output files are  written in ~/.minio/profile
directory instead of <configDir>/profile/. This change fixes this behaviour.
2016-05-17 11:44:40 -07:00
Harshavardhana 9472299308 logging: Log only for unhandled errors, remove all the debug logging. (#1652)
This patch brings in the removal of debug logging altogether, instead
we bring in the functionality of being able to trace the errors properly
pointing back to the origination of the problem.

To enable tracing you need to enable "MINIO_TRACE" set to "1" or "true"
environment variable which would print back traces whenever there is an
error which is unhandled or at the handler layer.

By default this tracing is turned off and only user level logging is
provided.
2016-05-16 14:31:28 -07:00
Harshavardhana b66c3bf35e server: Enable server profiling as needed. (#1565) 2016-05-09 16:18:56 -07:00
Harshavardhana 4e34e03dd4 xl/fs: Split object layer into interface. (#1415) 2016-04-29 14:24:10 -07:00
Harshavardhana 984903cce1 server: Add global namespace lock. (#1398)
Fixes #1393
2016-04-29 01:29:09 -07:00
Harshavardhana e9fba04b36 logging: Enable logging across storage fs layer. (#1367)
Adds log.Debugf at all the layer - fixes #1074
2016-04-24 00:36:00 -07:00
Harshavardhana 484ba91b08 config: Migrate to the new version. Remove backend details.
Migrate to new config format v4.
```
{
	"version": "4",
	"credential": {
		"accessKey": "WLGDGYAQYIGI833EV05A",
		"secretKey": "BYvgJM101sHngl2uzjXS/OBF/aMxAN06JrJ3qJlF"
	},
	"region": "us-east-1",
	"logger": {
		"console": {
			"enable": true,
			"level": "fatal"
		},
		"file": {
			"enable": false,
			"fileName": "",
			"level": "error"
		},
		"syslog": {
			"enable": false,
			"address": "",
			"level": "debug"
		}
	}
}
```

This patch also updates [minio cli spec](./minio.md)
2016-04-02 17:29:31 -07:00
Harshavardhana 3538c9f598 minio: Server upon start displays a message if update is available.
This code also handles to turn itself off when network is not
available and if request fails. Also prints only when the update
is available.
2016-03-24 20:03:51 -07:00
Harshavardhana aaf97ea02c config/main: Re-write config files - add to new config v3
- New config format.

```
{
	"version": "3",
	"address": ":9000",
    "backend": {
          "type": "fs",
          "disk": "/path"
    },
	"credential": {
		"accessKey": "WLGDGYAQYIGI833EV05A",
		"secretKey": "BYvgJM101sHngl2uzjXS/OBF/aMxAN06JrJ3qJlF"
	},
	"region": "us-east-1",
	"logger": {
		"file": {
			"enable": false,
			"fileName": "",
			"level": "error"
		},
		"syslog": {
			"enable": false,
			"address": "",
			"level": "debug"
		},
		"console": {
			"enable": true,
			"level": "fatal"
		}
	}
}
```

New command lines in lieu of supporting XL.

Minio initialize filesystem backend.
~~~
$ minio init fs <path>
~~~

Minio initialize XL backend.
~~~
$ minio init xl <url1>...<url16>
~~~

For 'fs' backend it starts the server.
~~~
$ minio server
~~~

For 'xl' backend it waits for servers to join.
~~~
$ minio server
... [PROGRESS BAR] of servers connecting
~~~

Now on other servers execute 'join' and they connect.
~~~
....
minio join <url1> -- from <url2> && minio server
minio join <url1> -- from <url3> && minio server
...
...
minio join <url1> -- from <url16> && minio server
~~~
2016-03-23 19:16:09 -07:00
Harshavardhana 164dfe2ec9 console: Fix console color printing on windows.
Print colored accessKeyID and secretAccessKey are unreadable on windows
on command prompts and powershell.

Use the console package from minio client.
2016-03-04 10:07:19 -08:00
Anand Babu (AB) Periasamy bbca70e13b rewrite minio runtime checks 2016-02-15 17:56:56 +05:30
Harshavardhana 62f6ffb6db xl: Moved to minio/minio - fixes #1112 2016-02-11 15:43:36 -08:00
Harshavardhana 7a3409c309 fs: Cleanup Golang errors to be called 'e' and probe to be called as 'err'
- Replace the ACL checks back, remove them when bucket
  policy is implemented.
- Move FTW (File Tree Walk) into ioutils package.
2016-02-04 13:43:52 -08:00
Harshavardhana df91661ec6 flags: Remove anonymous, ratelimit, json and web-address flags.
- Web address now uses the port + 1 from the API address port directly.
- Remove ratelimiting, ratelimiting will be achieved if necessary through
  iptables.
- Remove json flag, not needed anymore.
- Remove anonymous flag, server will be no more anonymous for play.minio.io
  we will use demo credentials.
2016-02-02 18:37:09 -08:00
Harshavardhana 0a9496462a jwt: Add JWT support for minio server.
Please read JWT.md before using this feature.
2016-01-22 17:38:05 -08:00
Harshavardhana 836f5204af minio: Add config-folder option.
Fixes #997
2015-12-07 12:34:09 -08:00
Harshavardhana ab3fd8ea7f cli: vendorize to new CLI package updates.
- Fix a new line issue for minioHelpTemplate.
- Fixes #974
2015-11-21 09:01:18 -08:00
Krishna Srinivas f77851bee0 docker: second --ldflags was overriding the first --ldflags option 2015-11-07 15:21:01 -08:00
Krishna Srinivas 440bec28d9 docker: the docker image will now contain just the static binary 2015-11-06 20:44:58 -08:00
Harshavardhana 15909e5463 logger: Improve logger input argument handling and colorize outputs 2015-11-03 22:51:42 -08:00
Harshavardhana 74b62f1798 minio/probe: Add missing Commit-ID for probe appinfo 2015-11-02 15:05:57 -08:00
Harshavardhana 49260abad4 Fix description typo 2015-10-27 11:38:28 -07:00
Harshavardhana 3566d08c52 Update new changes in probe and add setAppInfo 2015-10-25 11:11:29 -07:00
Anand Babu (AB) Periasamy 8e68591933 updated probe 2015-10-23 20:09:14 -07:00
Anand Babu (AB) Periasamy b077bb9165 move description above usage 2015-10-21 22:27:20 -07:00
Harshavardhana 56003fded7 Add logger command - also migrate from old config to newer config 2015-10-21 00:02:16 -07:00
Harshavardhana b9ea18b8b8 Implement accessLog handler 2015-10-19 13:07:09 -07:00
Harshavardhana 179d2d7dac Add initial cut of auto expiry of objects 2015-10-19 01:34:31 -07:00
Harshavardhana c065be656c Implement min-free-disk as a subcommand, deprecate flag 2015-10-19 00:59:20 -07:00
Harshavardhana 5b2fa33bdb Implementing min-free-disk 2015-10-18 00:23:14 -07:00
Harshavardhana 47f1ffa1f3 Implement update command 2015-10-17 15:04:54 -07:00
Harshavardhana 94b0243341 Update minio micro services description 2015-10-16 11:40:47 -07:00
Harshavardhana 762b798767 Migrate this project to minio micro services code 2015-10-16 11:26:08 -07:00
Harshavardhana ee377c9bff Enforce signature v4 tests all the time, server defaults to only authenticated requests.
All requests must be authenticated to minio server from now on by using keys generated at
``${HOME}/.minio/users.json`` - from ``minio controller`` during its first time run.

Add a new hidden option ``--anonymous`` for running server in unauthenticated mode.
2015-10-07 10:43:27 -07:00
Harshavardhana f0a8dbecae Add --json output formatter for server 2015-10-05 00:20:49 -07:00
Anand Babu (AB) Periasamy 45146cc138 setting GOMAXPROCS is no longer 2015-09-20 16:06:16 -07:00
Harshavardhana 674631f9d8 Improve code further - this time further simplification of names 2015-09-19 21:21:39 -07:00
Krishna Srinivas e600bd6b4f Controller Service proxies rpc calls to the corresponding servers 2015-09-19 19:37:20 -07:00
Anand Babu (AB) Periasamy d1f1b7ac31 new version format and some cleanup 2015-09-18 23:27:04 -07:00
Harshavardhana 7093a05ab1 Version is a package now, will be re-used across codebase. 2015-09-17 20:17:33 -07:00
Harshavardhana 77c71bd596 Add trie to verify wrong inputs, and provide meaningful messages 2015-09-17 16:49:08 -07:00