Aditya Manthramurthy
6533927237
Lock-free rate-limit algorithm + bug-fix ( #2694 )
2016-09-14 11:27:37 -07:00
Harshavardhana
da9ae574df
server: We should fail properly during server startup. ( #2689 )
...
Fixes #2688
2016-09-14 01:11:03 -07:00
Harshavardhana
16e4a7c200
Merge pull request #2657 from minio/distributed
...
Distributed XL support
2016-09-13 22:34:49 -07:00
Harshavardhana
ee7e70c992
tests: Add tests for ListMultipartUploads, DeleteMultipleObjects. ( #2649 )
...
Additionally adds PostPolicyHandler tests.
2016-09-13 21:22:31 -07:00
Krishna Srinivas
54a9f59a13
Init: Print SQS ARNs after globalEventNotifier is inited. ( #2682 )
...
fixes #2681
2016-09-13 21:18:30 -07:00
Harshavardhana
e6fd664331
tests: Fix format-config tests.
2016-09-13 21:18:30 -07:00
Karthic Rao
b247ec9352
tests: refactor object-handler tests. ( #2656 )
...
- Move the initialization to a common executor for object Layer API
tests.d
2016-09-13 21:18:30 -07:00
Harshavardhana
43befab8ef
Change distributed server wording.
2016-09-13 21:18:30 -07:00
Harshavardhana
eae0281c64
tests: Add GetBucketLocation, HeadBucket tests. ( #2644 )
2016-09-13 21:18:30 -07:00
Karthic Rao
8bd78fbdfb
performance: gjson parsing for readXLMeta, listParts, getObjectInfo. ( #2631 )
...
- Using gjson for constructing xlMetaV1{} in realXLMeta.
- Test for parsing constructing xlMetaV1{} using gjson.
- Changes made since benchmarks showed 30-40% improvement in speed.
- Follow up comments in issue https://github.com/minio/minio/issues/2208
for more details.
- gjson parsing of parts from xl.json for listParts.
- gjson parsing of statInfo from xl.json for getObjectInfo.
- Vendorizing gjson dependency.
2016-09-13 21:18:30 -07:00
Krishnan Parthasarathi
66459a4ce0
Add unit-tests for formatting disks during initialization ( #2635 )
...
* Add unit-tests for formatting disks during initialization
- Fixed corresponding code at places where it was deviating from the
tabular spec.
* Added more test cases and simplified algo
... based on feedback from ``go test -coverprofile``.
2016-09-13 21:18:30 -07:00
Harshavardhana
182109f0de
xl: Heal format.json
properly on fresh disks.
2016-09-13 21:18:30 -07:00
Harshavardhana
9998e9ea19
api: Response timeFormat do not need to have nano-second precision.
...
Fixes an error reported by s3verify.
2016-09-13 21:18:30 -07:00
Harshavardhana
ba2ba328da
server: Fixes for various conditions
...
- Fix distributed branch to be able to run FS version.
- Fix distributed branch to be able to run XL local disks.
- Ignore initialization failures of notification and bucket
policies, the codepath should load whatever is possible.
2016-09-13 21:18:30 -07:00
Anis Elleuch
67b8080144
Fix control lock rpc name in control lock cmd ( #2627 )
2016-09-13 21:18:30 -07:00
Anis Elleuch
239a34ca97
Add tests for regular and streaming v4 PutObject Handler ( #2618 )
2016-09-13 21:18:30 -07:00
Krishna Srinivas
81d8263ae2
binary-update: Do not fetch update info for minio binary compiled from source.
...
fixes #2494
2016-09-13 21:18:30 -07:00
Krishna Srinivas
b4e4846e9f
PutObject: object layer now returns ObjectInfo instead of md5sum to avoid extra GetObjectInfo call. ( #2599 )
...
From the S3 layer after PutObject we were calling GetObjectInfo for bucket notification. This can
be avoided if PutObjectInfo returns ObjectInfo.
fixes #2567
2016-09-13 21:18:30 -07:00
Krishna Srinivas
92e49eab5a
FS/Multipart: Do not rename append files to another tmp file as the append files are already in tmp location. ( #2612 )
2016-09-13 21:18:30 -07:00
Harshavardhana
c4a7b950a0
fs: Fix asynchronous multipart bug.
...
Construct part path properly.
2016-09-13 21:18:30 -07:00
Karthic Rao
1ce339abeb
Fixing ineffssaign errors ( #2608 )
2016-09-13 21:18:30 -07:00
Aditya Manthramurthy
a1f922315b
Add docker-compose file to run Minio in distributed mode ( #2606 )
...
Serves as a starting point to run a Minio cluster using Docker. The
file can be used as configuration for the docker-compose tool to start
4 Minio servers in distributed mode.
* Add a docker-compose.yml file to run 4 minio server instances in
distributed mode
* Update Docker.md with command to use the file
2016-09-13 21:18:30 -07:00
Anis Elleuch
3e284162d7
Add global flags to all commands and subcommands ( #2605 )
2016-09-13 21:18:30 -07:00
Anis Elleuch
ff99392102
Enhance minio server help template ( #2603 )
2016-09-13 21:18:30 -07:00
Krishna Srinivas
9358ee011b
logging: Print stack trace in case of errors.
...
fixes #1827
2016-09-13 21:18:30 -07:00
Harshavardhana
37cbcae6ba
xl: Remove an unecessary lock with isBucketExist() ( #2593 )
...
Fixes #2566
2016-09-13 21:18:30 -07:00
Harshavardhana
ae64b7fac8
XL: Handle object layer initialization properly.
...
Initialization when disk was down the network disk
reported an incorrect error rather than errDiskNotFound.
This resulted in incorrect error handling during
prepInitStorage() stage.
Fixes #2577
2016-09-13 21:18:30 -07:00
Anis Elleuch
d936ed90ae
Avoid testing on system errors strings in posix ( #2583 )
2016-09-13 21:18:30 -07:00
Krishna Srinivas
7cc77eba45
XL/Healing: errDiskNotFound is the only pardonable error in xlShouldHeal. ( #2586 )
...
This is so that we try to heal a file for all the "bad" cases except when the disk is down.
2016-09-13 21:18:30 -07:00
Karthic Rao
07d232c7b4
instrumentation: instrumentation for locks. ( #2584 )
...
- Instrumentation for locks.
- Detailed test coverage.
- Adding RPC control handler to fetch lock instrumentation.
- RPC control handlers suite tests with a test RPC server.
2016-09-13 21:18:30 -07:00
Krishnan Parthasarathi
de67bca211
Move formatting of disks out of object layer initialization ( #2572 )
2016-09-13 21:18:30 -07:00
Anis Elleuch
5c4dbc966f
Add Howto setup minio distributed with Docker
2016-09-13 21:18:30 -07:00
Harshavardhana
bca1385683
lock: Fix support single node XL locking as well.
2016-09-13 21:18:30 -07:00
Harshavardhana
f655592ff5
vendorize new dsync with new changes of logging.
2016-09-13 21:18:30 -07:00
Harshavardhana
cbe87cb2ed
Fix fd-leak in rpcClient close it pro-actively.
2016-09-13 21:18:30 -07:00
Anis Elleuch
0513b3ed07
Add Heal Disk Metadata RPC API + tests ( #2556 )
2016-09-13 21:18:30 -07:00
Frank
7f92165c79
Single use DRWMutex, RPCClient refactor and added missing cases to lock-rpc-server ( #2560 )
...
This PR contains various fixes for the distributed release:
- Use DRWMutex in namespace-lock only for a single Lock()/RLock() call in conformance to server-side rw-locking as implemented in minio/dsync
- Implement missing cases in lock-rpc-server to catch Unlock() for active read locks and RUnlock() for an active write lock
- Refactor RPCClient to release local mutex while making actual RPC.Call()
2016-09-13 21:18:30 -07:00
Harshavardhana
780ccc26f7
server: Validate server arguments for duplicates. ( #2554 )
...
- Validates invalid format inputs.
- Validates duplicate entries.
- Validates sufficient amount of disks.
Partially fixes #2502
2016-09-13 21:18:30 -07:00
Harshavardhana
339425fd52
server: Fetch StorageInfo() from underlying disks transparently. ( #2549 )
...
Fixes #2511
2016-09-13 21:18:30 -07:00
Harshavardhana
fa6e9540a8
server: We shouldn't exit the server in lazy init. ( #2548 )
...
Avoid fatalIf instead these are non-critical errors,
continue running the server.
- initializing bucket notifications
- initializing bucket policies.
- migrating bucket policies failure.
Fixes #2547
2016-09-13 21:18:30 -07:00
Harshavardhana
9605fde04d
controller/auth: Implement JWT based authorization for controller. ( #2544 )
...
Fixes #2474
2016-09-13 21:18:30 -07:00
Anis Elleuch
200d327737
List only objects that need healing ( #2546 )
2016-09-13 21:18:30 -07:00
Harshavardhana
e1b0985b5b
rpc: Refactor authentication and login changes. ( #2543 )
...
- Cache login requests.
- Converge validating distributed setup.
2016-09-13 21:18:30 -07:00
Krishnan Parthasarathi
c8dfc4cda4
auth-rpc: Reset token on disconnect ( #2542 )
2016-09-13 21:18:30 -07:00
Bala FA
7922a54c9a
rpc-client: remove unwanted nil check of rpcClient. ( #2538 )
2016-09-13 21:18:30 -07:00
Krishnan Parthasarathi
bda6bcd5be
Layered rpc-client implementation ( #2512 )
2016-09-13 21:18:30 -07:00
Harshavardhana
7e3e24b394
rpc: client login should ignore server versions.
2016-09-13 21:18:30 -07:00
Harshavardhana
bb0466f4ce
control: Fix controller CLI handling with distributed server object layer.
...
Object layer initialization is done lazily fix it.
2016-09-13 21:18:30 -07:00
Harshavardhana
8797952409
server: Add server command line for running in distributed mode
2016-09-13 21:18:30 -07:00
Krishnan Parthasarathi
c33d1b8ee6
Vendorize minio/dsync for server-side read lock ( #2484 )
...
- Prevention of stale lock accumulation.
- Removal of dead code.
2016-09-13 21:18:30 -07:00