Commit Graph

3361 Commits

Author SHA1 Message Date
Harshavardhana
ba2ba328da server: Fixes for various conditions
- Fix distributed branch to be able to run FS version.
- Fix distributed branch to be able to run XL local disks.
- Ignore initialization failures of notification and bucket
  policies, the codepath should load whatever is possible.
2016-09-13 21:18:30 -07:00
Anis Elleuch
67b8080144 Fix control lock rpc name in control lock cmd (#2627) 2016-09-13 21:18:30 -07:00
Anis Elleuch
239a34ca97 Add tests for regular and streaming v4 PutObject Handler (#2618) 2016-09-13 21:18:30 -07:00
Krishna Srinivas
81d8263ae2 binary-update: Do not fetch update info for minio binary compiled from source.
fixes #2494
2016-09-13 21:18:30 -07:00
Krishna Srinivas
b4e4846e9f PutObject: object layer now returns ObjectInfo instead of md5sum to avoid extra GetObjectInfo call. (#2599)
From the S3 layer after PutObject we were calling GetObjectInfo for bucket notification. This can
be avoided if PutObjectInfo returns ObjectInfo.

fixes #2567
2016-09-13 21:18:30 -07:00
Krishna Srinivas
92e49eab5a FS/Multipart: Do not rename append files to another tmp file as the append files are already in tmp location. (#2612) 2016-09-13 21:18:30 -07:00
Harshavardhana
c4a7b950a0 fs: Fix asynchronous multipart bug.
Construct part path properly.
2016-09-13 21:18:30 -07:00
Karthic Rao
1ce339abeb Fixing ineffssaign errors (#2608) 2016-09-13 21:18:30 -07:00
Aditya Manthramurthy
a1f922315b Add docker-compose file to run Minio in distributed mode (#2606)
Serves as a starting point to run a Minio cluster using Docker. The
file can be used as configuration for the docker-compose tool to start
4 Minio servers in distributed mode.

* Add a docker-compose.yml file to run 4 minio server instances in
  distributed mode

* Update Docker.md with command to use the file
2016-09-13 21:18:30 -07:00
Anis Elleuch
3e284162d7 Add global flags to all commands and subcommands (#2605) 2016-09-13 21:18:30 -07:00
Anis Elleuch
ff99392102 Enhance minio server help template (#2603) 2016-09-13 21:18:30 -07:00
Krishna Srinivas
9358ee011b logging: Print stack trace in case of errors.
fixes #1827
2016-09-13 21:18:30 -07:00
Harshavardhana
37cbcae6ba xl: Remove an unecessary lock with isBucketExist() (#2593)
Fixes #2566
2016-09-13 21:18:30 -07:00
Harshavardhana
ae64b7fac8 XL: Handle object layer initialization properly.
Initialization when disk was down the network disk
reported an incorrect error rather than errDiskNotFound.

This resulted in incorrect error handling during
prepInitStorage() stage.

Fixes #2577
2016-09-13 21:18:30 -07:00
Anis Elleuch
d936ed90ae Avoid testing on system errors strings in posix (#2583) 2016-09-13 21:18:30 -07:00
Krishna Srinivas
7cc77eba45 XL/Healing: errDiskNotFound is the only pardonable error in xlShouldHeal. (#2586)
This is so that we try to heal a file for all the "bad" cases except when the disk is down.
2016-09-13 21:18:30 -07:00
Karthic Rao
07d232c7b4 instrumentation: instrumentation for locks. (#2584)
- Instrumentation for locks.
- Detailed test coverage.
- Adding RPC control handler to fetch lock instrumentation.
- RPC control handlers suite tests with a test RPC server.
2016-09-13 21:18:30 -07:00
Krishnan Parthasarathi
de67bca211 Move formatting of disks out of object layer initialization (#2572) 2016-09-13 21:18:30 -07:00
Anis Elleuch
5c4dbc966f Add Howto setup minio distributed with Docker 2016-09-13 21:18:30 -07:00
Harshavardhana
bca1385683 lock: Fix support single node XL locking as well. 2016-09-13 21:18:30 -07:00
Harshavardhana
f655592ff5 vendorize new dsync with new changes of logging. 2016-09-13 21:18:30 -07:00
Harshavardhana
cbe87cb2ed Fix fd-leak in rpcClient close it pro-actively. 2016-09-13 21:18:30 -07:00
Anis Elleuch
0513b3ed07 Add Heal Disk Metadata RPC API + tests (#2556) 2016-09-13 21:18:30 -07:00
Frank
7f92165c79 Single use DRWMutex, RPCClient refactor and added missing cases to lock-rpc-server (#2560)
This PR contains various fixes for the distributed release:
- Use DRWMutex in namespace-lock only for a single Lock()/RLock() call in conformance to server-side rw-locking as implemented in minio/dsync
- Implement missing cases in lock-rpc-server to catch Unlock() for active read locks and RUnlock() for an active write lock
- Refactor RPCClient to release local mutex while making actual RPC.Call()
2016-09-13 21:18:30 -07:00
Harshavardhana
780ccc26f7 server: Validate server arguments for duplicates. (#2554)
- Validates invalid format inputs.
- Validates duplicate entries.
- Validates sufficient amount of disks.

Partially fixes #2502
2016-09-13 21:18:30 -07:00
Harshavardhana
339425fd52 server: Fetch StorageInfo() from underlying disks transparently. (#2549)
Fixes #2511
2016-09-13 21:18:30 -07:00
Harshavardhana
fa6e9540a8 server: We shouldn't exit the server in lazy init. (#2548)
Avoid fatalIf instead these are non-critical errors,
continue running the server.

 - initializing bucket notifications
 - initializing bucket policies.
 - migrating bucket policies failure.

Fixes #2547
2016-09-13 21:18:30 -07:00
Harshavardhana
9605fde04d controller/auth: Implement JWT based authorization for controller. (#2544)
Fixes #2474
2016-09-13 21:18:30 -07:00
Anis Elleuch
200d327737 List only objects that need healing (#2546) 2016-09-13 21:18:30 -07:00
Harshavardhana
e1b0985b5b rpc: Refactor authentication and login changes. (#2543)
- Cache login requests.
- Converge validating distributed setup.
2016-09-13 21:18:30 -07:00
Krishnan Parthasarathi
c8dfc4cda4 auth-rpc: Reset token on disconnect (#2542) 2016-09-13 21:18:30 -07:00
Bala FA
7922a54c9a rpc-client: remove unwanted nil check of rpcClient. (#2538) 2016-09-13 21:18:30 -07:00
Krishnan Parthasarathi
bda6bcd5be Layered rpc-client implementation (#2512) 2016-09-13 21:18:30 -07:00
Harshavardhana
7e3e24b394 rpc: client login should ignore server versions. 2016-09-13 21:18:30 -07:00
Harshavardhana
bb0466f4ce control: Fix controller CLI handling with distributed server object layer.
Object layer initialization is done lazily fix it.
2016-09-13 21:18:30 -07:00
Harshavardhana
8797952409 server: Add server command line for running in distributed mode 2016-09-13 21:18:30 -07:00
Krishnan Parthasarathi
c33d1b8ee6 Vendorize minio/dsync for server-side read lock (#2484)
- Prevention of stale lock accumulation.
- Removal of dead code.
2016-09-13 21:18:30 -07:00
Krishnan Parthasarathi
1f67c18222 Lock/Rlock rpc reply was incorrect (#2479) 2016-09-13 21:18:30 -07:00
Krishnan Parthasarathi
a4691611a7 Move initGracefulShutdown before objectLayer initialization (#2468) 2016-09-13 21:18:30 -07:00
awwalker
7c7eb1475d splitNetPath: Add support for windows paths including volumeNames e.g ip:C:\network\path 2016-09-13 21:18:30 -07:00
Harshavardhana
0bce3d6d63 Fix web handler initialize with distributed lazy init. 2016-09-13 21:18:30 -07:00
Krishnan Parthasarathi
804d91ef61 storage/rpc-client: Reconnect on network disconnect (#2436) 2016-09-13 21:18:30 -07:00
Krishnan Parthasarathi
229600ce9b Implement RLock, RUnlock rpc handlers (#2437)
This would make it simplify dsync.RWMutex's algorithm to acquire a
distributed read lock and be tolerant to N/2-1 failures.
2016-09-13 21:18:30 -07:00
Harshavardhana
43098df9d8 rpc: Re-factor ReadFile behavior client <--> server.
Current code did not marshal/unmarshal buffers properly from
the server, The reason being buffers have to be allocated and
sent properly back to client to be consumable.
2016-09-13 21:18:30 -07:00
Harshavardhana
6908a0dcd4 Extract rpc server wrapped errors and translate to storage error. 2016-09-13 21:18:30 -07:00
Harshavardhana
cae5761f16 rpc/client: Add missing rpcTokens for each rpc calls. 2016-09-13 21:18:30 -07:00
Harshavardhana
83074ed57e Add missing rpc-client.go - missing in previous commit. 2016-09-13 21:18:30 -07:00
Krishnan Parthasarathi
64a7e6992e Vendorize rpc reconnect changes from minio/dsync (#2405) 2016-09-13 21:18:30 -07:00
Harshavardhana
61af764f8a Add rpc layer authentication. 2016-09-13 21:18:30 -07:00
Harshavardhana
b4172ad3c8 Rename rpc-{client,server} storage-rpc-{client,server} 2016-09-13 21:18:30 -07:00