Commit Graph

96 Commits

Author SHA1 Message Date
Harshavardhana b90cefdb88 Merge remote-tracking branch 'origin/feature-gcs' 2017-06-22 11:52:12 -07:00
Krishnan Parthasarathi 146bc3e638 Add MINIO_REGION to server help message (#4558)
* Add e.g for setting MINIO_REGION env variable
* Add MINIO_REGION to region table
2017-06-20 15:02:18 -07:00
Harshavardhana f99f218999 Add support for reading and saving config on Gateway. (#4463)
This is also a first step towards supporting bucket
notification for gateway.
2017-06-16 16:01:41 -07:00
splinter98 8293f546af Add support for MQTT server as a notification target (#4474)
This implementation is similar to AMQP notifications:

* Notifications are published on a single topic as a JSON feed
* Topic is configurable, as is the QoS. Uses the paho.mqtt.golang
  library for the mqtt connection, and supports connections over tcp
  and websockets, with optional secure tls support.
* Additionally the minio server configuration has been bumped up
  so mqtt configuration can be added.
* Configuration migration code is added with tests.

MQTT is an ISO standard M2M/IoT messaging protocol and was
originally designed for applications for limited bandwidth
networks. Today it's use is growing in the IoT space.
2017-06-14 17:27:49 -07:00
Aditya Manthramurthy 986aa8fabf Bypass network in lock requests to local server (#4465)
This makes lock RPCs similar to other RPCs where requests to the local
server bypass the network. Requests to the local lock-subsystem may
bypass the network layer and directly access the locking
data-structures.

This incidentally fixes #4451.
2017-06-05 12:25:04 -07:00
Aditya Manthramurthy a0e02f43e1 Fix and cleanup update message and improve related tests (#4361)
Fixes #4232
2017-05-31 09:22:00 -07:00
Harshavardhana 610dbe3479 config: Do not migrate config file if not needed. (#4264)
Also improve the error message returned by `pkg/quick`.

Fixes #4233
2017-05-06 10:16:59 -07:00
Bala FA de204a0a52 Add extensive endpoints validation (#4019) 2017-04-11 15:44:27 -07:00
Harshavardhana b927523223 server: Introduce a new env MINIO_REGION. (#4078)
This is implemented to be able to override region
through command line just like how access and
secret keys are provided.
2017-04-09 10:44:10 -07:00
Bala FA 6e9c91f43a fix: use its own lock in serverConfigV17 (#4014)
Previously serverConfigV17 used a global lock that made any instance of
serverConfigV17 depended on single global serverConfigMu.

This patch fixes by having individual lock per instances.
2017-03-30 22:26:24 -07:00
Bala FA 2df8160f6a server: handle command line and env variables at one place. (#3975) 2017-03-30 11:21:19 -07:00
Harshavardhana 1caad902cb config/path: Figure out absolute paths properly on windows. (#3996)
The following form of arguments such as

```
minio.exe -C some_dir server dir
```

has stopped working because of lack of handling of
absolute paths for config directory. Always calculate
absolute path for any relative paths on any operating
system.

The following fix converts all config directory relative
paths into absolute paths.

Fixes #3991
2017-03-29 08:55:33 -07:00
Krishna Srinivas 9ee83b89bb config: Appropriate error message when newer config file is found (#3972) 2017-03-28 18:41:16 -07:00
Bala FA 6e63904048 browser-flag: wrapped bool type denotes browser on/off flag. (#3963)
Statically typed BrowserFlag prevents any arbitrary string value
usage. The wrapped bool marshals/unmarshals JSON according to the
typed value ie string value "on" represents boolean true and "off" as
boolean false.
2017-03-26 12:00:27 -07:00
Bala FA d3cb79a57c Refactor logger (#3924)
This patch fixes below

* Previously fatalIf() never writes log other than first logging target.
* quiet flag is not honored to show progress messages other than startup messages.
* Removes console package usage for progress messages.
2017-03-23 16:36:00 -07:00
Bala FA 1c97dcb10a Add UTCNow() function. (#3931)
This patch adds UTCNow() function which returns current UTC time.

This is equivalent of UTCNow() == time.Now().UTC()
2017-03-18 11:28:41 -07:00
Krishna Srinivas cea4cfa3a8 Implement S3 Gateway to third party cloud storage providers. (#3756)
Currently supported backend is Azure Blob Storage.

```
export MINIO_ACCESS_KEY=azureaccountname
export MINIO_SECRET_KEY=azureaccountkey
minio gateway azure
```
2017-03-16 12:21:58 -07:00
Harshavardhana f3334159a4 config: Relax browser and region to be empty. (#3912)
- If browser field is missing or empty
  then default to "on".

- If region field is empty or missing then
  default to "us-east-1" (S3 spec behavior)
2017-03-16 11:06:17 -07:00
Bala FA 21d73a3eef Simplify credential usage. (#3893) 2017-03-16 00:16:06 -07:00
Anis Elleuch ae4361cc45 config: Check for duplicated entries in all scopes (#3872)
Validate Minio config by checking if there is double json key
in any scope level. The returned error contains the json path
to the duplicated key.
2017-03-15 16:30:34 -07:00
Bala FA 8a9852220d Make unit testable cert parsing functions. (#3863) 2017-03-08 19:20:01 -08:00
Bala FA bff4d29415 Remove commands and commandsTree global variables. (#3855) 2017-03-06 19:35:26 -08:00
Bala FA 98d17d2a97 Remove globalQuiet and globalConfigDir global variables (#3830) 2017-03-02 14:21:30 -08:00
Bala FA 208dd15245 Remove globalMaxCacheSize and globalCacheExpiry variables (#3826)
This patch fixes below

* Remove global variables globalMaxCacheSize and globalCacheExpiry.
* Make global variables into constant in objcache package.
2017-03-02 10:34:37 -08:00
Harshavardhana cff45db1b9 cli: Use ADDRESS:PORT to clarify --address behavior (#3803)
Currently we document as IP:PORT which doesn't provide
if someone can use HOSTNAME:PORT. This is a change
to clarify this by calling it as ADDRESS:PORT which
encompasses both a HOSTNAME and an IP.

Fixes #3799
2017-02-24 14:19:20 -08:00
Harshavardhana 9eb8e375c5 cli: Make sure to add --help flag for subcommands. (#3773)
--help is now back and prints properly with command
help template.
2017-02-19 20:46:06 -08:00
Harshavardhana 611bd68739 cli: Add new features from CLI.
use `.HelpName` for template to pass down
proper command names. Also treat flags
to be optional in all of them.
2017-02-15 17:45:11 -08:00
Harshavardhana 13c3b9cbcb main/cli: Don't print hidden flags/commands in help template. (#3748)
Always use .VisibleFlags and .VisibleCommands to not print
Hidden flags as expected from help template.
2017-02-15 02:25:38 -08:00
Harshavardhana 9df01035da Remove XL references in public docs to Erasure. (#3725)
Ref #3722
2017-02-09 23:26:44 -08:00
Krishnan Parthasarathi ce9aa2f2b2 Add uptime to ServiceStatus (#3690) 2017-02-08 00:13:02 -08:00
Harshavardhana 31dff87903 Honor envs properly for access and secret key. (#3703)
Also changes the behavior of `secretKeyHash` which is
not necessary to be sent over the network, each node
has its own secretKeyHash to validate.

Fixes #3696
Partial(fix) #3700 (More changes needed with some code cleanup)
2017-02-07 12:51:43 -08:00
Anis Elleuch b6ebf2aba8 server-mux: Simplify graceful shutdown behavior (#3681)
`*http.Server` is no more used, doing some cleanup.
2017-02-03 22:53:30 -08:00
Bala FA cc1575f944 fix: do port availability check only on macOS. (#3654)
On macOS, if a process already listens on 127.0.0.1:PORT, net.Listen() falls back
to IPv6 address ie minio will start listening on IPv6 address whereas another
(non-)minio process is listening on IPv4 of given port.
To avoid this error sutiation we check for port availability only for macOS.

Note: checkPortAvailability() tries to listen on given port and closes it.
It is possible to have a disconnected client in this tiny window of time.
2017-01-30 00:44:36 -08:00
Krishnan Parthasarathi 586058f079 Implement mgmt REST APIs to heal storage format. (#3604)
* Implement heal format REST API handler
* Implement admin peer rpc handler to re-initialize storage
* Implement HealFormat API in pkg/madmin
* Update pkg/madmin API.md to incl. HealFormat
* Added unit tests for ReInitDisks rpc handler and HealFormatHandler
2017-01-23 00:32:55 -08:00
Harshavardhana 80f1387877 Initialize peers properly for localhost. (#3600)
This introduced a regression.

Fixes #3594
2017-01-19 11:32:13 -08:00
Harshavardhana 62f8343879 Add constants for commonly used values. (#3588)
This is a consolidation effort, avoiding usage
of naked strings in codebase. Whenever possible
use constants which can be repurposed elsewhere.

This also fixes `goconst ./...` reported issues.
2017-01-18 12:24:34 -08:00
Harshavardhana 1c699d8d3f fs: Re-implement object layer to remember the fd (#3509)
This patch re-writes FS backend to support shared backend sharing locks for safe concurrent access across multiple servers.
2017-01-16 17:05:00 -08:00
Harshavardhana 08b6cfb082 ssl: Set a global boolean to enable SSL across Minio (#3558)
We have been using `isSSL()` everywhere we can set
a global value once and re-use it again.
2017-01-11 13:59:51 -08:00
Harshavardhana b0cfceb211 event: Enhance event message struct to provide origin server. (#3557)
`principalId` i.e user identity is kept as AccessKey in
accordance with S3 spec.

Additionally responseElements{} are added starting with

`x-amz-request-id` is a hexadecimal of the event time itself in nanosecs.
`x-minio-origin-server` - points to the server generating the event.

Fixes #3556
2017-01-10 16:43:48 -08:00
Krishnan Parthasarathi 85c6bb9809 server: Sort disk arguments for consistent ordering (#3469)
This is important in a distributed setup, where the server hosting the
first disk formats a fresh setup. Sorting ensures that all servers
arrive at the same 'first' server.

Note: This change doesn't protect against different disk arguments
with some disks being same across servers.
2016-12-19 17:04:31 -08:00
Harshavardhana 9c9f390350 fs: validate filesystem path argument properly. (#3470)
FS should fail for invalid paths like

 - file:///
 - ftp://
 - http://
2016-12-17 13:43:26 -08:00
Bala FA 1b2b16998f Remove regexp usage (#3456)
This patch removes regexp usage in cacheControlHandler.ServeHTTP() and
server-main.checkEndpointsSyntax()
2016-12-17 11:00:16 -08:00
Krishnan Parthasarathi b2f920a868 Add service API handler stubs for status, stop and restart (#3417) 2016-12-15 22:26:15 -08:00
Harshavardhana 664ff063a1 server: checkEndpoints syntax properly. (#3451) 2016-12-14 20:42:19 -08:00
Krishnan Parthasarathi ab49498fc3 server: Exit gracefully if no endpoint is local to it. (#3442) 2016-12-13 11:18:31 -08:00
Harshavardhana cd0f350c02 env: Bring back MINIO_BROWSER env. (#3423)
Set MINIO_BROWSER=off to disable web browser completely.

Fixes #3422
2016-12-10 00:42:22 -08:00
Anis Elleuch 01f625824a args: Honor config-dir & quiet wherever they are (#3356)
setGlobalsFromContext() is added to set global variables after parsing
command line arguments. Thus, global flags will be honored wherever
they are placed in minio command.
2016-11-28 12:15:36 -08:00
Harshavardhana dd74e5a809 Revert "init: Honor config-dir flag when it is passed as global or local flag (#3337)"
This reverts commit e2ef95af7d.

This is reverted since the previous patch caused crashes.
2016-11-23 17:31:36 -08:00
Anis Elleuch e2ef95af7d init: Honor config-dir flag when it is passed as global or local flag (#3337)
setGlobalsFromContext() is added to sets global variable after parsing
command line arguments.
2016-11-23 17:13:40 -08:00
Harshavardhana 6efee2072d objectLayer: Check for `format.json` in a wrapped disk. (#3311)
This is needed to validate if the `format.json` indeed exists
when a fresh node is brought online.

This wrapped implementation also connects to the remote node
by attempting a re-login. Subsequently after a successful
connect `format.json` is validated as well.

Fixes #3207
2016-11-23 15:48:10 -08:00