0ff359ca0e
Fix early init. problem for notifications ( Fixes #2972 ) ( #2977 )
2016-10-17 16:38:29 -07:00
f8e13fb00e
server: Startup sequence should be more idempotent. ( #2974 )
...
Fixes #2971 - honors ignore-disks option properly.
Fixes #2969 - change the net.Dial to have a timeout of 3secs.
2016-10-17 14:31:33 -07:00
686a610fc3
api: Nanosecond precision for API responses is valid with S3. ( #2957 )
...
Wqe need to be compatible as well fixes #2955
2016-10-17 08:44:55 -07:00
b89609dc2e
XL: Filter out md5Sum from user defined headers ( #2962 )
2016-10-17 08:41:33 -07:00
fa50312220
Avoid returning disk corrupted by servers in the middle of init all disks formats ( #2964 )
2016-10-17 08:39:55 -07:00
fee3f99a6e
xl: heal bucket should validate if bucket exists first. ( #2953 )
...
Fixes #2944
2016-10-17 02:10:23 -07:00
ea406754a6
New dsync and added ForceUnlock to lock rpc server ( #2956 )
...
* Update dsync and added ForceUnlock function
* Added test cases for ForceUnlock
2016-10-17 01:53:29 -07:00
d02cb963d5
Fix listen-bucket ( Fixes #2942 ) ( #2949 )
...
Don't close socket while re-initializing notify-listeners, as the rpc
client object is shared between notify-listeners and peer clients.
Also, improves SendRPC() readability by using GetPeerClient().
2016-10-16 20:52:10 -07:00
334cdb5d64
XL total/free space calculation is done inside xl module ( #2945 )
2016-10-16 14:24:15 -07:00
5c3639c1b7
Redirect /minio to /minio/ when requests come from browsers ( #2937 )
2016-10-15 06:21:51 -07:00
903574db90
copy-object: Do not use ETag of source as MD5 as it will not be MD5 if source was uploaded as multipart. ( #2938 )
...
fixes #2934
2016-10-15 06:20:55 -07:00
f463d3ce42
Fix a crash when service shutdown is signaled and object API is not ready yet ( #2939 )
2016-10-15 06:20:16 -07:00
17eeec6895
Bucket policy propagation ( Fixes #2930 ) ( #2932 )
...
Fixes a serialisation bug - encoding/gob does not directly support
serializing `map[string]interface{}`, so we serialise to JSON and send a
byte array in the RPC call, and deserialize and update on the receiver.
2016-10-14 22:49:51 -07:00
070d3610ff
tests: V2 Signature tests for object-handlers. ( #2931 )
2016-10-14 20:52:46 -07:00
f22862aa28
heal: Refactor heal command. ( #2901 )
...
- return errors for heal operation through rpc replies.
- implement rotating wheel for healing status.
Fixes #2491
2016-10-14 19:57:40 -07:00
2f520ed92f
Remove errors package, add comments and simplify. ( #2925 )
2016-10-14 12:31:00 -07:00
c03ce0f74a
Display SSL expiry warnings ( #2925 )
2016-10-14 12:30:36 -07:00
0320a77dc0
HealBucket: create the bucket if it is missing in one of the disks. ( #2924 )
2016-10-14 11:12:17 -07:00
31be826f51
Fix missing error check for jsonrpc.Server.RegisterService() ( #2921 )
2016-10-13 17:34:10 -07:00
17e49a9ed2
signature-v2 fix. ( #2918 )
...
- Return errors similar to V4 Sign processsing.
- Return ErrMissing fields when Auth Header fields are missing.
- Return InvalidAccessID when accessID doesn't match.
* tests: Adding V2 signature tests for bucket handler API's.
2016-10-13 09:25:56 -07:00
0aabc1d8d9
Use Peer RPC to propagate bucket policy changes ( #2891 )
2016-10-13 09:19:04 -07:00
55f6828750
Do not print update message unless there is an update. ( #2919 )
2016-10-13 09:17:08 -07:00
6303f26330
Protect map from concurrent access ( Fixes #2915 ) ( #2916 )
...
Protects the Peers RPC clients map from concurrent access to fix a data race condition.
2016-10-13 01:33:50 -07:00
b59bac670a
Handle err returned by rpc.Server.RegisterName ( #2910 )
2016-10-12 23:13:24 -07:00
84acc820c7
Fix free drive space calculation in XL mode ( #2917 )
2016-10-12 20:22:15 -07:00
fdaa129a5b
Fix dockerfile container image. ( #2892 )
2016-10-12 18:09:08 -07:00
df59967f59
Avoid checking date header of web requests by properly applying generic handlers ( #2914 )
2016-10-12 12:58:36 -07:00
73982c8cb6
Listen bucket notification for multiple prefixes/suffixes ( #2911 )
...
* Listen bucket notification for multiple prefixes/suffixes
* After review fixes
2016-10-12 11:02:15 -07:00
6199aa0707
Peer RPCs for bucket notifications ( #2877 )
...
* Implements a Peer RPC router that sends info to all Minio servers in the cluster.
* Bucket notifications are propagated to all nodes via this RPC router.
* Bucket listener configuration is persisted to separate object layer
file (`listener.json`) and peer RPCs are used to communicate changes
throughout the cluster.
* When events are generated, RPC calls to send them to other servers
where bucket listeners may be connected is implemented.
* Some bucket notification tests are now disabled as they cannot work in
the new design.
* Minor fix in `funcFromPC` to use `path.Join`
2016-10-12 01:03:50 -07:00
a5921b5743
Use same timestamp for all chunks in chunked signature ( #2908 )
2016-10-11 23:46:51 -07:00
ff91ecb177
tests: Adding unknown signature type test for API handlers. ( #2905 )
2016-10-11 20:38:10 -07:00
a6357502c1
Correct typo in error string ( #2902 )
2016-10-11 08:56:02 -07:00
fa8ea41cd9
lock/instrumentation: Cleanup and print in user friendly form. ( #2807 )
2016-10-11 00:50:27 -07:00
3ac6790ca2
tests: Add Object Layer nil test for bucket-handler API's ( #2899 )
2016-10-11 00:00:02 -07:00
268b96058f
ns-lock: lock namespace during FS object operations. ( #2896 )
2016-10-10 10:20:04 -07:00
0d031c432b
Fix typo in serverVersion ( #2894 )
2016-10-10 10:11:56 -07:00
6e8f3224c5
Test coverage for lock rpc server ( #2893 )
...
* Add test coverage for removeEntry and removeEntryIfExists
* Initial test framework for Lock/Unlock functionality
* Add clarification comments
* Add test coverage code for RLock() and RUnlock()
* Add test coverage for Expired() function
* Have all lock-rpc-server test functions start with the same prefix
* Properly initialize JWT security token
2016-10-10 10:11:29 -07:00
9c53e9f4c3
tests: Enhance coverage for bucket policy handlers. ( #2895 )
2016-10-10 09:29:56 -07:00
2d5e988a6d
Refactor streaming signatureV4 w/ state machine ( #2862 )
...
* Refactor streaming signatureV4 w/ state machine
- Used state machine to make transitions between reading chunk header,
chunk data and trailer explicit.
* debug: add print/panic statements to gather more info on CI failure
* Persist lastChunk status between Read() on ChunkReader
... remove panic() which was added as interim aid for debugging.
* Add unit-tests to cover v4 streaming signature
2016-10-10 01:42:32 -07:00
3cfb23750a
control: Implement service command 'stop,restart,status'. ( #2883 )
...
- stop - stops all the servers.
- restart - restart all the servers.
- status - prints status of storage info about the cluster.
2016-10-09 23:03:10 -07:00
57f75b1d9b
Ignore copy conditions when ETag is not available ( #2888 )
2016-10-09 16:21:42 -07:00
e213172431
tests: Missing anonymous tests for bucket-handlers. ( #2885 )
2016-10-09 09:21:37 -07:00
f5f007e183
Test: Add test case for xl.HealObject() ( #2884 )
...
fixes #2842
2016-10-08 17:08:17 -07:00
09463265ce
tests: Adding anonymous requests tests for bucket policy handlers. ( #2882 )
2016-10-08 01:04:26 -07:00
8f4cf2a7d0
tests: anonymous/unsigned tests for object handler API's . ( #2881 )
2016-10-07 23:28:50 -07:00
30183c4a9a
tests: cleanup and unsigned request test. ( #2880 )
...
- Cleaning up of ListMultipartUpload API test for improving readability,
code maintainance and extensibility.
- Moving ListMultipartUploads to Go 1.7 sub tests.
- Using the new Anonymous request helper function for
ListMultipartUploads.
2016-10-07 20:16:57 -07:00
d1df5e0ae1
tests: Add helper function for API handler anonymous request tests. ( #2876 )
...
- Add helper function for API handler anonymous request tests.
- Add PutObject Part Anonymous request case using the new helper
function to validate its functionality.
2016-10-07 11:16:11 -07:00
f1bc9343a1
prep: Initialization should wait instead of exit the servers. ( #2872 )
...
- Servers do not exit for invalid credentials instead they print and wait.
- Servers do not exit for version mismatch instead they print and wait.
- Servers do not exit for time differences between nodes they print and wait.
2016-10-07 11:15:55 -07:00
97f4989945
tests: cleaning up. ( #2875 )
...
- Clean up PutObjectPart and ListObjectPart API handler tests.
- Add more comments, make the tests more readable.
- Add verification for HTTP response status code.
- Initialize the test using object Layer.
- Move to Go 1.7 sub tests.
2016-10-07 08:02:37 -07:00
1e5e213d24
auth: Make sure we initialize or change config before RPC requests. ( #2867 )
2016-10-06 13:35:56 -07:00
a8105ec068
- Test utility function for easy asserting of cases wherein objectLayer ( #2865 )
...
is `nil` in API handlers.
- Remove the existing tests for the `nil` check and use the new method
to test for object layer being `nil`.
2016-10-06 13:34:33 -07:00
bb9be02228
minio-browser: do not redirect to /minio if MINIO_BROWSER=off ( #2863 )
...
fixes #2837
2016-10-06 08:30:32 -07:00
64f37bbf5b
rpc: Add RPC client tests. ( #2858 )
2016-10-06 02:30:54 -07:00
0fc96fa25c
Refactor bucket policy handler test to use API test initializer. ( #2859 )
2016-10-06 02:02:42 -07:00
2d8c6f8288
unit test for bucketPolicyConditionMatch function. ( #2857 )
2016-10-06 00:23:46 -07:00
b94211bd66
api: ListObjectsV1 compliance with AWS S3. ( #2856 )
...
XSD - xml schema definition for SOAP operations
on S3 provides positional restrictions on XML
output.
Fix the response by re-arranging the positions in
accordance with S3 behavior.
Fixes #2849
2016-10-05 20:12:47 -07:00
6494b77d41
server: Add more elaborate startup messages. ( #2731 )
...
These messages based on our prep stage during XL
and prints more informative message regarding
drive information.
This change also does a much needed refactoring.
2016-10-05 12:48:07 -07:00
63a7ca1af0
web: fix jwt token expiry set to one day by default. ( #2819 )
...
Fixes #2818
2016-10-05 10:18:55 -07:00
95f544657a
Signature-V2: use raw resource/query from the request for signature calculation. ( #2833 )
2016-10-05 09:18:53 -07:00
740a919e25
config: Use migrateV8 to v9 function properly. ( #2852 )
2016-10-05 02:28:04 -07:00
402c92beda
Add listObjectParts test w/ unknown request signature type ( #2847 )
2016-10-04 07:57:35 -07:00
6a9013b97c
misspell fixes. ( #2835 )
2016-10-04 00:09:21 -07:00
73b50aea2d
Add preSign auth type tests for ListObjectPartsHandler and PutObjectPartHandler ( #2834 )
2016-10-03 22:05:33 -07:00
315e66858c
Add PostgreSQL notifier ( #2739 ) ( #2824 )
...
* The user is required to specify a table name and database connection
information in the configuration file.
* INSERTs and DELETEs are done via prepared statements for speed.
* Assumes a table structure, and requires PostgreSQL 9.5 or above due to
the use of UPSERT.
* Creates the table if it does not exist with the given table name using
a query like:
CREATE TABLE myminio (
key varchar PRIMARY KEY,
value JSONB
);
* Vendors some required libraries.
2016-10-03 17:29:55 -07:00
4f902d42b2
Add unit-tests for ListObjectParts API handler ( #2826 )
...
* Add missing uploadID test
... make variables in test code unexported.
* Add ServerNotInitialized test for ListObjectPartsHandler
* Add tests for ListObjectParts with signatureV2 and Anonymous requests
* Add failure test cases for ListObjectParts
2016-10-03 08:54:57 -07:00
61a18ed48f
sha256: Verify sha256 along with md5sum, signature is verified on the request early. ( #2813 )
2016-10-02 15:51:49 -07:00
b5a6dd1395
Avoid path-cleaning policy resources for a better compliance with S3 ( #2823 )
2016-10-01 21:30:25 -07:00
83e6e1060e
Layer LimitReader responsibly allowing sign verification to work ( #2821 )
2016-10-01 09:37:40 -07:00
ddeb8242d8
PutObjectPartHandler unit-tests ( #2810 )
2016-10-01 08:23:26 -07:00
5ecba587f7
api: Relax object name validation. ( #2814 )
...
Fixes #2812
2016-09-30 16:56:36 -07:00
db3da97a50
signature/v2: Fix presigned requests.
2016-09-30 15:22:00 -07:00
5885ffc8ae
signature: Add legacy signature v2 support transparently. ( #2811 )
...
Add new tests as well.
2016-09-30 14:32:13 -07:00
9fb1c89f81
Add TLS encryption capability to RPC clients ( #2789 )
2016-09-29 23:42:37 -07:00
1e6afac3bd
Add NATS notifier ( #2795 )
2016-09-29 23:42:10 -07:00
64083b9227
signature: Region changes should be handled just like AWS. ( #2805 )
...
- PutBucket happens with 'us-east-1'.
- ListBuckets happens with any region.
- GetBucketLocation happens with 'us-east-1' and location is returned.
2016-09-29 15:51:00 -07:00
5fdd768903
Make addition of TopicConfig to globalEventNotifier go-routine safe ( #2806 )
2016-09-28 22:46:19 -07:00
428629f577
Add unit tests for server-main.go ( #2802 )
2016-09-28 11:19:07 -07:00
1edd74dda2
update: Deprecate the usage of update=yes query param. ( #2801 )
...
Fixes #2799
2016-09-28 02:41:21 -07:00
740ecf530c
Add PutBucketNotification, ListenBucketNotification handler unit-tests. ( #2787 )
2016-09-28 01:08:03 -07:00
10d2ef5449
Remove comments relating to deprecated MINIO_DEBUG envvar ( #2797 )
2016-09-27 18:28:46 -07:00
8ea571c7f7
Remove MINIO_DEBUG environment variable ( #2794 )
...
Removes the unimplemented settings of MINIO_DEBUG=mem and makes
MINIO_DEBUG=lock the default behaviour.
2016-09-27 14:35:43 -07:00
ca3022d545
api: Change ListenBucketNotification with new API format. ( #2791 )
...
Take prefix, suffix and events as query params.
2016-09-27 13:17:43 -07:00
9417614a8e
Recalculate free minimum disk space ( #2788 )
...
* Fix calculating free space disk by using blocks available for unprivileged user
* Use fixed minimal free disk space instead of percentage
2016-09-27 12:46:38 -07:00
70d52bbc4c
Add unit test for rate-limit-handler ( #2661 ) ( #2784 )
2016-09-26 21:31:12 -07:00
6aa2fc95c0
Revert "bucket: refactor policies and fix bugs related to enforcing policies. ( #2766 )"
...
This reverts commit ca5ca8332b
2016-09-26 19:32:33 -07:00
cfbab22237
web: Remove bucket policy when we have no more statements. ( #2779 )
2016-09-26 03:11:22 -07:00
be0e06c0aa
web: Simplify and rename GetAllBucketPolicy --> ListAllBucketPolicies. ( #2778 )
2016-09-25 21:53:19 -07:00
1c941fd787
rpc: Should validate server versions. ( #2775 )
...
Fixes #2764
2016-09-24 03:34:45 -07:00
669783f875
Purge stale object cache entry ( #2770 )
2016-09-23 19:55:28 -07:00
27e474b3d2
Improve code coverage in bucket-notification-handlers.go ( #2759 )
...
* Fix incorrect test cases for bucket-notification handler
* Add tests covering failure cases for bucket notification
2016-09-23 13:32:51 -07:00
1e53316241
Add tests for presigned-get ( #2767 )
...
* web-handlers: support for presigned-get json-rpc call for MinioBrowser's "share" feature.
* Add tests for presigned-get
2016-09-23 01:25:49 -07:00
ca5ca8332b
bucket: refactor policies and fix bugs related to enforcing policies. ( #2766 )
...
This patch also addresses the problem of double caching at
object layer once at XL and another at handler layer.
2016-09-22 23:47:48 -07:00
aa579bbc20
web: add method to get all policies for given bucket name. ( #2756 )
...
Refer #1858
2016-09-22 23:06:45 -07:00
e375d822da
bucket: SetBucketPolicy should save a valid Version and validate. ( #2762 )
2016-09-22 22:27:21 -07:00
fc783f8407
More tests for web handlers ( #2755 )
...
* Return negative values of Total and Free in StorageInfo() when we fail to get disk info
* Return consistent messages in web handlers when the server is not initialized
2016-09-22 16:35:12 -07:00
ef22330563
Require go 1.7.1 to build Minio server ( #2727 )
2016-09-22 10:33:52 -07:00
1148f95292
ineffassign fixes ( #2758 )
2016-09-21 23:03:54 -07:00
f7430ec09c
use runtime/debug.Stack() in leak detect test ( #2757 )
2016-09-21 22:04:35 -07:00
b8903d842c
api/complete-multipart: fixes and tests. ( #2719 )
...
* api/complete-multipart: tests and simplification.
- Removing the logic of sending white space characters.
- Fix for incorrect HTTP response status for certain cases.
- Tests for New Multipart Upload and Complete Multipart Upload.
* tests: test for Delelete Object API handler
2016-09-21 20:08:08 -07:00
32f097b4d6
Controller rpc tests ( #2709 )
...
* Test code for controller-handler operations:
* Heal operations
* List operation
* Switch to "testing" lib, moving away from gocheck
* Minor refactors
* Remove extra call to initGracefulShutdown
* Remove dead code in mainControl:
Dead code found by the TestControlMain() test function that always
passes.
* Add tests for control-*-main.go
2016-09-21 19:58:50 -07:00
Aditya Manthramurthy
Harshavardhana
Krishnan Parthasarathi
Anis Elleuch
Frank
Krishna Srinivas
Karthic Rao
Mateusz Gajewski
Mateusz Gajewski
Bala FA
Krishna Srinivas