Harshavardhana
ade803a923
Verify if request date is 5minutes late, reject such a request as
...
it could be a possible replay attack.
This commit also fixes #505 , by returning MethodNotAllowed instead of NotImplemented
2015-04-27 03:55:10 -07:00
Harshavardhana
9232ce3b4e
Now client requests for ACL changes are honored through PutBucketACL API
2015-04-27 03:04:29 -07:00
Harshavardhana
c99d96dbc2
Now donut supports bucket ACL's, bringing in this support for dl.minio.io
2015-04-27 02:09:32 -07:00
Harshavardhana
8beca83090
Limit memory mode at 100 buckets and return error appropriately
...
~~~
mc: <ERROR> Failed to create bucket for URL [http://localhost:9000/newbucket-101 ]. Reason: [You have attempted to create more buckets than allowed.].
~~~
Response from server
2015-04-26 22:15:39 -07:00
Harshavardhana
2d96d5ad57
Enhance error responses for request limit and bring some code from api errors
...
~~~
mc: <ERROR> Failed to create bucket for URL [http://localhost:9000/newbucket-101 ]. Reason: [Reduce your request rate.].
~~~
Client sees proper errors now.
2015-04-26 22:10:59 -07:00
Frederick F. Kautz IV
8a57006b36
Merge pull request #504 from fkautz/pr_out_making_lock_much_less_aggressive_fixes_multiple_uploads_blocking
2015-04-26 20:11:50 -07:00
Frederick F. Kautz IV
a974e4d7c7
Making lock much less aggressive, fixes multiple uploads blocking
2015-04-26 20:08:37 -07:00
Frederick F. Kautz IV
258bf55e31
Bandwidth quota now supports 100-Continue
2015-04-26 20:07:05 -07:00
Frederick F. Kautz IV
9431a54a27
finish me
2015-04-26 19:25:48 -07:00
Frederick F. Kautz IV
202fc26719
Revert "Exporting WriteErrorResponse"
...
This reverts commit e05d46a0b5
.
2015-04-26 17:02:49 -07:00
Frederick F. Kautz IV
e05d46a0b5
Exporting WriteErrorResponse
2015-04-26 16:28:54 -07:00
Frederick F. Kautz IV
64f7d04cd1
Adding connection limits
2015-04-26 16:14:51 -07:00
Frederick F. Kautz IV
7867ee89fa
Bandwidth quotas now work on data returned from server
2015-04-26 15:49:55 -07:00
Frederick F. Kautz IV
e2403cbc56
Quotas are more accurate, occur on Read()
2015-04-26 15:07:57 -07:00
Frederick F. Kautz IV
bbb493080f
Switching back to BigEndian
2015-04-25 18:16:21 -07:00
Frederick F. Kautz IV
3a48f9fe4d
Adding quotas based upon type
2015-04-25 18:07:02 -07:00
Harshavardhana
b75f46aaa0
Use standard functions to achieve byteorder converstion
2015-04-25 13:18:15 -07:00
Frederick F. Kautz IV
e2475925ea
quota handlers now log against ip properly against a duration
2015-04-24 20:49:33 -07:00
Harshavardhana
b3a9d8b058
Merge pull request #483 from harshavardhana/pr_out_memory_now_handles_submitting_large_files_fixes_482
2015-04-24 15:25:38 -07:00
Harshavardhana
c5d8ca245d
Memory now handles submitting large files - fixes #482
2015-04-24 15:22:22 -07:00
Frederick F. Kautz IV
fda50d52c8
Add initial framework for quota
2015-04-24 00:16:29 -07:00
Frederick F. Kautz IV
5a607a6762
Moving factories and start minio method to server package
2015-04-23 16:13:50 -07:00
Frederick F. Kautz IV
5c515d3138
Simplifying server config handling
2015-04-23 12:44:22 -07:00
Harshavardhana
5ae8d34f76
Donut mode will now take multiple paths as argument, updated docs as well
2015-04-23 11:36:14 -07:00
Harshavardhana
a5729e8c02
Disable verifying ACL until we get webcli up and running
2015-04-23 03:21:08 -07:00
Harshavardhana
bcc4449de7
Add proper help and several other cleanup
2015-04-23 02:42:50 -07:00
Harshavardhana
e95604ff86
Finishing all the test case support for ACL and other fixes
2015-04-23 01:24:46 -07:00
Frederick F. Kautz IV
0f8626ab1a
Refactoring cli to take commands
2015-04-23 01:05:44 -07:00
Harshavardhana
848c4ee31c
Further fixes for ACL support, currently code is disabled in all the handlers
...
Disabled because due to lack of testing support. Once we get that in we can
uncomment them back.
2015-04-22 19:31:45 -07:00
Harshavardhana
1c0ff2c758
ACL driver/storage layer support
2015-04-22 19:31:45 -07:00
Harshavardhana
c8713fd650
Bring in full fledged acl support
2015-04-22 18:26:35 -07:00
Harshavardhana
2c1455af1b
Implement x-amz-acl tests
2015-04-22 18:26:35 -07:00
Harshavardhana
107e077ec0
Implement x-amz-acl handling
2015-04-22 18:26:35 -07:00
Frederick F. Kautz IV
c56f7380d6
Memory driver now limits by size
2015-04-22 17:50:26 -07:00
Harshavardhana
b121c8588f
Remove bucketpolicy handlers and all its references
2015-04-22 14:46:51 -07:00
Harshavardhana
3f33643d39
File driver to take the high road
2015-04-16 02:16:39 -07:00
Frederick F. Kautz IV
59a81a73c6
Adding lru to memory driver, not wired to command line opts
2015-04-15 12:49:41 -07:00
Frederick F. Kautz IV
0cab3fd151
Better locking in memory driver
2015-04-15 11:48:56 -07:00
Harshavardhana
148935392d
Fix golint complaint
2015-04-08 23:42:59 -07:00
Frederick F. Kautz IV
26d8deee40
Removing unneeded files
2015-04-08 20:58:42 -07:00
Frederick F. Kautz IV
7212b3406a
Adding comments to iodine
2015-04-08 19:44:44 -07:00
Frederick F. Kautz IV
c12d5ea8a9
Adding iodine error conversion
2015-04-08 19:32:37 -07:00
Harshavardhana
38155e86ac
iodine is part of minio now
2015-04-08 18:05:48 -07:00
Harshavardhana
7fd1cc073c
Add everything back into one project hood, still missing iodine
2015-04-08 17:47:56 -07:00
Frederick F. Kautz IV
6241c9880b
Moving object storage drivers back to minio
2015-04-08 16:28:14 -07:00
Harshavardhana
2a21b7d639
Implement headBucketHandler() and its related tests
...
Remove internal testify/mock and use upstream - update all godeps
2015-04-07 01:16:32 -07:00
Frederick F. Kautz IV
ea961ef16d
Updating to use objectdriver
2015-04-06 18:00:18 -07:00
Frederick F. Kautz IV
e71cc8d833
Adding Debug logger to log package
2015-04-06 12:15:19 -07:00
Harshavardhana
319feb52f1
erasure is now external pull it and update from minio-io/erasure
2015-04-05 23:51:24 -07:00
Frederick F. Kautz IV
052817ff91
Fix fd leak
2015-04-05 21:33:24 -07:00
Harshavardhana
d324c4b061
Return proper return for all drivers when BucketNotFound
2015-04-05 15:55:10 -07:00
Harshavardhana
0475d7d056
Migrate to external Donut package
2015-04-05 14:59:24 -07:00
Frederick F. Kautz IV
2f44438843
Reformatting imports
2015-04-04 13:59:59 -07:00
Frederick F. Kautz IV
be4bc44b4d
Replacing gopkg.in/check.v1 with minio-io/check
2015-04-04 13:37:30 -07:00
Frederick F. Kautz IV
e6abca1a63
Adding iodine to object HEAD verb
2015-04-04 12:46:34 -07:00
Frederick F. Kautz IV
46badca07e
Removing Godeps from pkg/api
2015-04-03 20:40:21 -07:00
Frederick F. Kautz IV
4171ad8b69
Adding test for invalid range in api
2015-04-03 20:35:39 -07:00
Harshavardhana
4079dd1d12
Commit testify,objx, gorilla/mux, minio-io/erasure gopkg.in/check.v1
2015-04-03 18:57:47 -07:00
Harshavardhana
4b25d4e8db
Use verifyError instead of parsResponse, also get rid of mxj dependency
2015-04-03 18:56:47 -07:00
Frederick F. Kautz IV
9954e5c280
Better error handling + tests for get object
2015-04-03 18:53:27 -07:00
Frederick F. Kautz IV
db1e6e5988
Adding tests for put bucket errors, fixed mismatching error
2015-04-03 18:37:39 -07:00
Frederick F. Kautz IV
ebf1e784b3
Adding iodine and tests for for list buckets errors
2015-04-03 18:29:31 -07:00
Frederick F. Kautz IV
5865295f5d
Adding better error support to api
2015-04-03 18:06:30 -07:00
Frederick F. Kautz IV
8757ccf9a3
Fixing race condition in api_test.go
2015-04-01 20:30:57 -07:00
Harshavardhana
85e547d854
Parse response headers in tests
2015-04-01 20:07:51 -07:00
Harshavardhana
92f408af5d
Add erasure to godep
2015-04-01 16:56:43 -07:00
Frederick F. Kautz IV
818b6d2e33
Minor cleanup of memory get partial object
2015-04-01 15:50:59 -07:00
Frederick F. Kautz IV
0c5ed6f3ba
Add partial object support to donut driver
2015-04-01 15:48:12 -07:00
Frederick F. Kautz IV
563a914046
Fix whitespace
2015-04-01 12:30:58 -07:00
Frederick F. Kautz IV
83f0f0971f
Adding comments on why we subtract a second from date1 in an api test
2015-04-01 12:29:08 -07:00
Frederick F. Kautz IV
1f699a094b
Minor modification for creating temp dir in api test
2015-04-01 12:07:33 -07:00
Frederick F. Kautz IV
5b1350b4bf
api test now states which driver is currently running
2015-04-01 12:06:22 -07:00
Harshavardhana
a56098b8c9
Make sure in API we handle all error type exclusively, and return certain values exclusively
2015-03-31 20:26:16 -07:00
Frederick F. Kautz IV
6136960cfd
Moving api_test to fixtures for drivers
2015-03-31 19:53:25 -07:00
Frederick F. Kautz IV
959d90fcf7
Adding test for partial objects
2015-03-31 19:07:45 -07:00
Frederick F. Kautz IV
62c4c997c2
Simplifying error handling in api bucket handlers
2015-03-30 14:27:32 -07:00
Frederick F. Kautz IV
318d9d5aa8
API Bucket Handlers errors simplified
2015-03-30 11:40:28 -07:00
Frederick F. Kautz IV
7d657eb55c
Simplifying error response writing for object handlers
2015-03-29 22:03:13 -07:00
Frederick F. Kautz IV
c3cda5d9a8
Adding comments to errors
2015-03-29 22:03:13 -07:00
Frederick F. Kautz IV
ded090a8b0
Normalizing imports in api
2015-03-29 21:41:38 -07:00
Frederick F. Kautz IV
8cbc1c2346
More api cleanup
2015-03-29 21:38:23 -07:00
Frederick F. Kautz IV
452f0f8335
Cleaning up more api code
2015-03-29 21:06:51 -07:00
Frederick F. Kautz IV
d644233054
Rename writeObjectHeaders to setObjectHeaders, fix writing multiple headers bug
2015-03-29 18:49:32 -07:00
Harshavardhana
b748cf359a
More simplification to erasure also add two more test functions
2015-03-29 14:52:13 -07:00
Frederick F. Kautz IV
86b32c2147
Minor change from trace to error log
2015-03-28 18:47:36 -07:00
Frederick F. Kautz IV
d5db891094
Simplfying code in api, round 1. Decoupling functions which both write headers and data.
2015-03-28 18:01:00 -07:00
Frederick F. Kautz IV
b165efdfcb
Refactoring pkg/utils/log, copying from golang's log and modifying
2015-03-28 15:58:48 -07:00
Harshavardhana
f3f0a3a5f7
Merge pull request #403 from harshavardhana/pr_out_let_donut_driver_use_log_errorln_wrapper
2015-03-28 12:43:42 -07:00
Harshavardhana
767f4d1d8f
Let donut driver use log.Errorln wrapper
2015-03-28 12:43:26 -07:00
Frederick F. Kautz IV
c901d0fb32
Fix for readers closing in erasure early
2015-03-27 23:49:14 -07:00
Frederick F. Kautz IV
96e26335e2
Adding tests and a fix for get object where bucket is not found
2015-03-27 23:23:22 -07:00
Frederick F. Kautz IV
ad370f341b
Fixing donut location
2015-03-27 12:44:13 -07:00
Frederick F. Kautz IV
53439d7768
Updating iodine to match new api
2015-03-26 15:55:06 -07:00
Frederick F. Kautz IV
c5f10df957
Removing TODO
2015-03-26 11:33:50 -07:00
Frederick F. Kautz IV
eaeeb4bef0
Refactoring erasure, each block is encoded by a function. Iodine included.
2015-03-26 11:32:39 -07:00
Frederick F. Kautz IV
31746fdeb2
Adding iodine to donutObjectWriter.Write
2015-03-26 10:49:03 -07:00
Frederick F. Kautz IV
bfbb97c98f
Merge pull request #393 from fkautz/pr_out_adding_iodine_to_object_writer
2015-03-26 10:42:57 -07:00
Frederick F. Kautz IV
f7a300400e
Merge pull request #392 from fkautz/pr_out_adding_iodine_to_node_local
2015-03-26 10:42:50 -07:00
Frederick F. Kautz IV
caf7e41499
Adding iodine to object_writer
2015-03-26 10:41:40 -07:00
Frederick F. Kautz IV
5cdf2564a5
Adding iodine to node_local
2015-03-26 10:21:24 -07:00
Frederick F. Kautz IV
70a2217eb2
AddNode fails with proper error code when bucketid is malformed
2015-03-26 10:02:40 -07:00
Frederick F. Kautz IV
cc64417481
Adding iodine to donut
2015-03-26 09:57:21 -07:00
Frederick F. Kautz IV
40e311a919
Updating iodine to support new iodine.Error(error, map[string]string) error method
2015-03-25 19:12:55 -07:00
Harshavardhana
68f36f5e08
Merge pull request #385 from abperiasamy/erasure_cache_tbls_matrix
2015-03-25 19:11:07 -07:00
Anand Babu (AB) Periasamy
b44d2100db
cache decode matrix tables and matrix
2015-03-25 19:09:34 -07:00
Harshavardhana
2d0be3e3b2
fix golint complaints
2015-03-25 19:01:36 -07:00
Anand Babu (AB) Periasamy
bd39768de2
janitor duty on erasure-decode
2015-03-25 18:02:03 -07:00
Frederick F. Kautz IV
eff48971ad
Adding iodine to donut bucket
2015-03-25 17:05:26 -07:00
Frederick F. Kautz IV
d52dd5ed5b
Adding iodine to pkg/server/server.go
2015-03-25 17:04:26 -07:00
Harshavardhana
58082cd8dc
Add gocyclo into source build, choosing cyclomatic complexity tolerance of 15 and below
2015-03-25 15:57:17 -07:00
Harshavardhana
c366fdd2d8
loop only on readers once
2015-03-25 14:48:46 -07:00
Harshavardhana
e1f4ed4650
Comment exported functions GetEncodedBlocksLen, GetEncodedBlockLen
2015-03-25 13:32:35 -07:00
Harshavardhana
2cbd15b690
Golint fixes
2015-03-25 13:25:25 -07:00
Harshavardhana
d967babbaa
Remove unnecessary if <block>
2015-03-25 12:58:15 -07:00
Frederick F. Kautz IV
be229473b5
Updating iodine
2015-03-25 00:43:48 -07:00
Harshavardhana
0f8021abd1
Fix nil pointer dereference when err is nil
2015-03-25 00:37:25 -07:00
Frederick F. Kautz IV
1be3766ae6
Adding iodine annotation to donut.Start(path)
2015-03-25 00:27:45 -07:00
Frederick F. Kautz IV
12a17f1ed7
Merge pull request #364 from fkautz/pr_out_using_iodine_in_donut_start_
2015-03-25 00:25:41 -07:00
Frederick F. Kautz IV
d06f5afdfc
Using iodine in donut.Start()
2015-03-25 00:25:27 -07:00
Harshavardhana
b3c3a8d6f2
Avoid empty if else body..
2015-03-25 00:24:41 -07:00
Harshavardhana
ada98c56c0
Fix minio hang in donut driver.
...
remove sending err to errorChannel until we figure out the right fix.
2015-03-25 00:15:36 -07:00
Frederick F. Kautz IV
1ae74591f6
Adding iodine
2015-03-24 23:54:49 -07:00
Frederick F. Kautz IV
dd5ba865ff
Merge remote-tracking branch 'origin/master' into HEAD
...
Conflicts:
pkg/drivers/donut/donut.go
pkg/storage/donut/bucket.go
pkg/storage/donut/donut.go
pkg/storage/donut/donut_test.go
pkg/storage/donut/donutdriver.go
2015-03-24 21:04:08 -07:00
Frederick F. Kautz IV
cf2550b267
Populate buckets at start
2015-03-24 19:58:03 -07:00
Harshavardhana
653ea50a59
Erasure technique is now uint8, handle it appropriately
2015-03-24 19:12:52 -07:00
Anand Babu (AB) Periasamy
fa9b1f341b
erasure encode janitor duty
2015-03-24 18:47:53 -07:00
Harshavardhana
6a87ed001c
Handle errors properly during erasure Decoding, also populate technique and verify
2015-03-24 18:14:13 -07:00
Harshavardhana
0474439b43
Code restructuring, assigning proper subsystems to each
...
- Drivers contain
* donut/*
* file/*
* memory/*
- Storage format contains
* donut/*
- GetObject() --> renamed to GetObjectReader()
- Deleted stale objectwriter.go, renamed donutwriter.go to object_writer.go
Simplifying, and documenting codebase further
2015-03-24 06:47:10 -07:00
Frederick F. Kautz IV
3c4012f1e7
Adding donut backend and setting as default
2015-03-23 20:07:22 -07:00
Harshavardhana
7c1c4b8a02
Merge pull request #351 from harshavardhana/pr_out_implement_log_package_as_drop_in_replacement_for_handling_debug_log_level
2015-03-23 18:55:14 -07:00
Harshavardhana
914962bd93
Implement log package as drop-in replacement for handling Debug log-level
2015-03-23 18:18:11 -07:00
Frederick F. Kautz IV
f91f5e59ee
Running gofmt manually, auto gofmt didn't simplify code
2015-03-23 12:26:41 -07:00
Frederick F. Kautz IV
56a298fd6a
Refactoring method variable names for erasureWriter struct
2015-03-23 12:25:20 -07:00
Frederick F. Kautz IV
d57e9d6c18
Adding sys. as reserved in erasureWriter
2015-03-23 11:56:24 -07:00
Frederick F. Kautz IV
b2f47ef446
Switching content hashing from sha512 to md5
2015-03-23 11:46:19 -07:00
Harshavardhana
b8e9834faa
Rename parms as --> params for brevity and misc cleanup
2015-03-22 20:26:01 -07:00
Frederick F. Kautz IV
b2c3172095
Adding sha512 support for pre-encoded data and verification on decode
2015-03-22 20:11:25 -07:00
Frederick F. Kautz IV
befac7d047
Exorcising donutbox
2015-03-22 19:55:15 -07:00
Frederick F. Kautz IV
eb8005cc65
Reading correct block size in erasure coded data
2015-03-22 18:49:30 -07:00
Anand Babu (AB) Periasamy
b9b0b35edf
removed private k and m and some more cleanups
2015-03-22 18:22:09 -07:00
Frederick F. Kautz IV
293e438830
Merge pull request #341 from abperiasamy/janitor
2015-03-22 17:34:25 -07:00
Anand Babu (AB) Periasamy
3a76229551
encoded size function is now public + some janitor duty
2015-03-22 17:17:53 -07:00
Harshavardhana
3929135c34
Make sure we use O_EXCL with O_CREATE to make sure we don't trip over existing file
2015-03-22 15:55:40 -07:00
Harshavardhana
2fa9320df2
De-couple donut into smaller files, useful for ease in external integration
2015-03-22 15:43:37 -07:00
Frederick F. Kautz IV
55f3659bb3
Removing log tracers
2015-03-22 15:39:50 -07:00
Frederick F. Kautz IV
03f51eac29
Adding list objects to donut
2015-03-22 15:33:52 -07:00
Harshavardhana
4b9fbd3b3a
DonutfileWriter should return value of file.Close()
2015-03-22 14:51:18 -07:00
Frederick F. Kautz IV
ced6edc9fe
Minor change, removing unnecessary commented method
2015-03-22 12:44:45 -07:00
Frederick F. Kautz IV
99cf484488
Adding GetObjectMetadata to donut
2015-03-22 12:36:51 -07:00
Frederick F. Kautz IV
703af6c3ce
Removing gob from donut driver
2015-03-22 12:17:12 -07:00