Commit Graph

60 Commits

Author SHA1 Message Date
Harshavardhana
a8ab02a73a v4/presign: Fix presign requests when there are more signed headers. (#3222)
This fix removes a wrong logic which fails for requests which
have more signed headers in a presign request.

Fixes #3217
2016-11-10 21:57:15 -08:00
Aditya Manthramurthy
cf022de4d5 Add tests for s3PeerAPIHandlers (Fixes #3067) (#3242) 2016-11-10 16:43:04 -08:00
Bala FA
9c2cfb5cb6 tests: Add missing unit test cases for AbortMultipartUploadHandler(). (#3200)
Fixes #3070
2016-11-08 16:25:00 -08:00
Bala FA
91a0ade908 tests: add unit test for HeadObjectHandler (#3197)
Fixes #3068
2016-11-07 16:02:27 -08:00
Aditya Manthramurthy
85a5c358d8 Add bucket metadata state client/handler (Fixes #3022) (#3152)
- Adds an interface to update in-memory bucket metadata state called
  BucketMetaState - this interface has functions to:
     - update bucket notification configuration,
     - bucket listener configuration,
     - bucket policy configuration, and
     - send bucket event

- This interface is implemented by `localBMS` a type for manipulating
  local node in-memory bucket metadata, and by `remoteBMS` a type for
  manipulating remote node in-memory bucket metadata.

- The remote node interface, makes an RPC call, but the local node
  interface does not - it updates in-memory bucket state directly.

- Rename mkPeersFromEndpoints to makeS3Peers and refactored it.

- Use arrayslice instead of map in s3Peers struct

- `s3Peers.SendUpdate` now receives an arrayslice of peer indexes to
  send the request to, with a special nil value slice indicating that
  all peers should be sent the update.

- `s3Peers.SendUpdate` now returns an arrayslice of errors, representing
  errors from peers when sending an update. The array positions
  correspond to peer array s3Peers.peers

Improve globalS3Peers:

- Make isDistXL a global `globalIsDistXL` and remove from s3Peers

- Make globalS3Peers an array of (address, bucket-meta-state) pairs.

- Fix code and tests.
2016-11-07 12:09:24 -08:00
Harshavardhana
33c771bb3e tests: Add tests for browser peer rpc. Fixes #3062 (#3189) 2016-11-07 11:43:35 -08:00
Anis Elleuch
79601d27f2 Use endpoint url when printing disks status in distributed mode (#3151) 2016-11-02 08:51:06 -07:00
Karthic Rao
8bffa78f7f Fix Instance type during benchmarks. (#3147)
- The benchmark initialization function was not taking into account the
  instance type (FS/XL), was using XL ObjectLayer even for FS
  benchmarks.
- This was leading to incorrect benchmark results for FS related
  benchmarks.
- The fix takes into account the instance type (FS/XL) and correctly
  returns FS backend for FS benchmarks.
2016-11-01 10:21:16 -07:00
Karthic Rao
3e8cb8c937 build: ineffassign fixes. (#3134) 2016-10-30 23:32:46 -07:00
Harshavardhana
9e2d0ac50b Move to URL based syntax formatting. (#3092)
For command line arguments we are currently following

- <node-1>:/path ... <node-n>:/path

This patch changes this to

- http://<node-1>/path ... http://<node-n>/path
2016-10-27 03:30:52 -07:00
Anis Elleuch
a15dc5fed5 Print message when creating the config file (#3089) 2016-10-26 18:44:22 -07:00
Krishnan Parthasarathi
2c9b406f6c Add TLS based tests to functional test suite (#3083) 2016-10-26 02:30:31 -07:00
Harshavardhana
485c0ea8bf tests: Combine v2 tests with the Suite itself. (#3088) 2016-10-25 13:34:14 -07:00
Harshavardhana
ece559afe2 api: Do not use sqs for ListenBucketNotification. (#3023)
Add more tests. Fixes #3024
2016-10-21 01:25:17 -07:00
Krishna Srinivas
32c3a558e9 distributed-XL: Support to run one minio process per export even on the same machine. (#2999)
fixes #2983
2016-10-20 18:31:02 -07:00
Anis Elleuch
41f9ab1c69 Translate storage access denied error to S3 Access Denied response (#3015) 2016-10-20 16:09:55 -07:00
Aditya Manthramurthy
0aabc1d8d9 Use Peer RPC to propagate bucket policy changes (#2891) 2016-10-13 09:19:04 -07:00
Krishnan Parthasarathi
b59bac670a Handle err returned by rpc.Server.RegisterName (#2910) 2016-10-12 23:13:24 -07:00
Mateusz Gajewski
73982c8cb6 Listen bucket notification for multiple prefixes/suffixes (#2911)
* Listen bucket notification for multiple prefixes/suffixes

* After review fixes
2016-10-12 11:02:15 -07:00
Aditya Manthramurthy
6199aa0707 Peer RPCs for bucket notifications (#2877)
* Implements a Peer RPC router that sends info to all Minio servers in the cluster.
* Bucket notifications are propagated to all nodes via this RPC router.
* Bucket listener configuration is persisted to separate object layer
  file (`listener.json`) and peer RPCs are used to communicate changes
  throughout the cluster.
* When events are generated, RPC calls to send them to other servers
  where bucket listeners may be connected is implemented.
* Some bucket notification tests are now disabled as they cannot work in
  the new design.
* Minor fix in `funcFromPC` to use `path.Join`
2016-10-12 01:03:50 -07:00
Krishnan Parthasarathi
a5921b5743 Use same timestamp for all chunks in chunked signature (#2908) 2016-10-11 23:46:51 -07:00
Karthic Rao
ff91ecb177 tests: Adding unknown signature type test for API handlers. (#2905) 2016-10-11 20:38:10 -07:00
Harshavardhana
fa8ea41cd9 lock/instrumentation: Cleanup and print in user friendly form. (#2807) 2016-10-11 00:50:27 -07:00
Karthic Rao
3ac6790ca2 tests: Add Object Layer nil test for bucket-handler API's (#2899) 2016-10-11 00:00:02 -07:00
Krishnan Parthasarathi
2d5e988a6d Refactor streaming signatureV4 w/ state machine (#2862)
* Refactor streaming signatureV4 w/ state machine

- Used state machine to make transitions between reading chunk header,
  chunk data and trailer explicit.

* debug: add print/panic statements to gather more info on CI failure

* Persist lastChunk status between Read() on ChunkReader

... remove panic() which was added as interim aid for debugging.

* Add unit-tests to cover v4 streaming signature
2016-10-10 01:42:32 -07:00
Harshavardhana
3cfb23750a control: Implement service command 'stop,restart,status'. (#2883)
- stop - stops all the servers.
- restart - restart all the servers.
- status - prints status of storage info about the cluster.
2016-10-09 23:03:10 -07:00
Karthic Rao
e213172431 tests: Missing anonymous tests for bucket-handlers. (#2885) 2016-10-09 09:21:37 -07:00
Karthic Rao
09463265ce tests: Adding anonymous requests tests for bucket policy handlers. (#2882) 2016-10-08 01:04:26 -07:00
Karthic Rao
8f4cf2a7d0 tests: anonymous/unsigned tests for object handler API's . (#2881) 2016-10-07 23:28:50 -07:00
Karthic Rao
d1df5e0ae1 tests: Add helper function for API handler anonymous request tests. (#2876)
- Add helper function for API handler anonymous request tests.
- Add PutObject Part Anonymous request case using the new helper
  function to validate its functionality.
2016-10-07 11:16:11 -07:00
Karthic Rao
97f4989945 tests: cleaning up. (#2875)
- Clean up PutObjectPart and ListObjectPart API handler tests.
- Add more comments, make the tests more readable.
- Add verification for HTTP response status code.
- Initialize the test using object Layer.
- Move to Go 1.7 sub tests.
2016-10-07 08:02:37 -07:00
Karthic Rao
a8105ec068 - Test utility function for easy asserting of cases wherein objectLayer (#2865)
is `nil` in API handlers.
- Remove the existing tests for the `nil` check and use the new method
  to test for object layer being `nil`.
2016-10-06 13:34:33 -07:00
Harshavardhana
64f37bbf5b rpc: Add RPC client tests. (#2858) 2016-10-06 02:30:54 -07:00
Harshavardhana
6494b77d41 server: Add more elaborate startup messages. (#2731)
These messages based on our prep stage during XL
and prints more informative message regarding
drive information.

This change also does a much needed refactoring.
2016-10-05 12:48:07 -07:00
Krishna Srinivas
95f544657a Signature-V2: use raw resource/query from the request for signature calculation. (#2833) 2016-10-05 09:18:53 -07:00
Krishnan Parthasarathi
4f902d42b2 Add unit-tests for ListObjectParts API handler (#2826)
* Add missing uploadID test
... make variables in test code unexported.
* Add ServerNotInitialized test for ListObjectPartsHandler
* Add tests for ListObjectParts with signatureV2 and Anonymous requests
* Add failure test cases for ListObjectParts
2016-10-03 08:54:57 -07:00
Krishnan Parthasarathi
ddeb8242d8 PutObjectPartHandler unit-tests (#2810) 2016-10-01 08:23:26 -07:00
Harshavardhana
5885ffc8ae signature: Add legacy signature v2 support transparently. (#2811)
Add new tests as well.
2016-09-30 14:32:13 -07:00
Krishnan Parthasarathi
740ecf530c Add PutBucketNotification, ListenBucketNotification handler unit-tests. (#2787) 2016-09-28 01:08:03 -07:00
Harshavardhana
6aa2fc95c0 Revert "bucket: refactor policies and fix bugs related to enforcing policies. (#2766)"
This reverts commit ca5ca8332b.
2016-09-26 19:32:33 -07:00
Harshavardhana
ca5ca8332b bucket: refactor policies and fix bugs related to enforcing policies. (#2766)
This patch also addresses the problem of double caching at
object layer once at XL and another at handler layer.
2016-09-22 23:47:48 -07:00
Karthic Rao
b8903d842c api/complete-multipart: fixes and tests. (#2719)
* api/complete-multipart: tests and simplification.

- Removing the logic of sending white space characters.
- Fix for incorrect HTTP response status for certain cases.
- Tests for New Multipart Upload and Complete Multipart Upload.

* tests: test for Delelete Object API handler
2016-09-21 20:08:08 -07:00
Krishnan Parthasarathi
559ad38b8c Add bucket-notification-handler tests (#2750) 2016-09-21 17:41:34 -07:00
Anis Elleuch
b89a1cd482 tests: Implemented more tests for fs-v1*.go (#2686) 2016-09-16 13:06:49 -07:00
Harshavardhana
7d37dea449 tests: Add more streaming signature tests. (#2713)
Part fix for #2621
2016-09-16 02:45:42 -07:00
Harshavardhana
03430d0db8 tests: Add ListBucketHandler tests. (#2701)
part-3 final fix for #2412
2016-09-14 23:53:42 -07:00
Harshavardhana
ee7e70c992 tests: Add tests for ListMultipartUploads, DeleteMultipleObjects. (#2649)
Additionally adds PostPolicyHandler tests.
2016-09-13 21:22:31 -07:00
Karthic Rao
b247ec9352 tests: refactor object-handler tests. (#2656)
- Move the initialization to a common executor for object Layer API
  tests.d
2016-09-13 21:18:30 -07:00
Harshavardhana
eae0281c64 tests: Add GetBucketLocation, HeadBucket tests. (#2644) 2016-09-13 21:18:30 -07:00
Anis Elleuch
239a34ca97 Add tests for regular and streaming v4 PutObject Handler (#2618) 2016-09-13 21:18:30 -07:00