MyFavMirrors
/
minio
mirror of https://github.com/minio/minio.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
2,138 Commits 5 Branches 503 Tags 186 MiB
Commit Graph

818 Commits

Author SHA1 Message Date
Anis Elleuch 7e5a26f42c Protect the exported method Delete() 2015-05-31 20:29:42 +01:00
Harshavardhana 82a0eac659 Add filesystem factory functions and add related cli options 2015-05-30 03:23:06 -07:00
Harshavardhana 0cc63706bb Resurrect back fileystem driver 2015-05-30 03:12:01 -07:00
Anis Elleuch d360d0fec3 Forbid the upload of files bigger than the memory backend capacity 2015-05-28 22:07:02 +01:00
Harshavardhana 554a244b47 Add donut OSX support 2015-05-26 02:14:34 -07:00
Harshavardhana e46660a966 Use http.TimeFormat to return, S3 expects us to return in GMT not UTC 2015-05-25 20:17:04 -07:00
Harshavardhana 19250296c6 Handle authorization header better 2015-05-24 20:59:53 -07:00
Harshavardhana 9666f2e5bf Relax minium object size limit, one can really upload a 0byte file 
For example
~~~
$ mc ls https://s3.amazonaws.com/ferenginar/test.go
[2015-05-23 12:02:06 PDT]     0B test.go

$ curl -i -X HEAD https://s3.amazonaws.com/ferenginar/test.go
HTTP/1.1 200 OK
x-amz-id-2: ZMWLriPH+uQJ8IsaMHmuNOU/FXvdSq+s6O7ugUI9hZ695XJTNAZ1utKxh03w5Jcf
x-amz-request-id: F5A11F533B74DD8F
Date: Sat, 23 May 2015 19:19:29 GMT
Last-Modified: Sat, 23 May 2015 19:02:06 GMT
ETag: "d41d8cd98f00b204e9800998ecf8427e"
Accept-Ranges: bytes
Content-Type: binary/octet-stream
Content-Length: 0
Server: AmazonS3
~~~
 2015-05-23 12:19:16 -07:00
Harshavardhana e11f9110b6 add missing validate access keys after being extracted from AuthHeader 2015-05-21 19:23:47 -07:00
Harshavardhana ad60b97d52 Fix some donut TODO's regarding bucket created date 2015-05-21 11:13:49 -07:00
Harshavardhana cd817d34df Finish last bits of api testsuite on server 2015-05-21 02:57:03 -07:00
Harshavardhana f00bfd38da Bump gocyclo back to 16 2015-05-19 18:32:50 -07:00
Harshavardhana 3d61ba5930 Fix duplicate map allocation 2015-05-18 13:14:01 -07:00
Harshavardhana e4c0d574dc Update totalnumber of multiparts inside CreateObjectPart(), also introduce support for UploadIDMarker and Prefix for listMultipartUploads 2015-05-18 12:24:36 -07:00
Harshavardhana 7ce47130fd HEAD request should have Content-Length for only successful response, there is no response body for errors, just header is sufficient - fixes #603 
http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.13

"""  in the case of the HEAD method, the size of the entity-body that would have been sent had the request been a GET. """
 2015-05-15 21:55:59 -07:00
Harshavardhana 317637fcb8 Remove redundant WriteHeader calls 2015-05-15 20:58:28 -07:00
Harshavardhana 5c5761ae16 Remove unnecessary GetObjectMetadata prefix argument 2015-05-15 19:25:17 -07:00
Harshavardhana 6275238a47 For multiparts use its own cache avoids deadlocks during object expiration 
Using same object cache as regular objects for multiparts causes deadlocks during object expiration routine.

As a side affect this also fixes the bug of listing multiparts during regular 'listing' of objects.
 2015-05-14 20:26:59 -07:00
Harshavardhana c41ebf7535 Fixing a race in NewMultipartUpload() session for simultaneous requests, observed at play.minio.io 2015-05-14 19:11:19 -07:00
Harshavardhana 8fb427a892 Add more tests for multipart now in driver api_testsuite 2015-05-14 17:28:39 -07:00
Harshavardhana dba7c94571 Add tests for abortMultipartUpload, listobjectparts, listmultipartuploads 2015-05-14 17:07:30 -07:00
Harshavardhana 1bd94ec6ab An attempt to implement ListMultipartUploads() 2015-05-14 16:22:47 -07:00
Harshavardhana 82b043005a Avoid one last memory copy 2015-05-14 11:26:55 -07:00
Harshavardhana 5498c90788 Add a new validateContentTypeHandler{}, verify Accept header earlier 2015-05-13 12:22:15 -07:00
Harshavardhana de56909d55 More scripts and code from Minio-io to minio 2015-05-11 16:43:38 -07:00
Harshavardhana 8d06504068 Change all minio-io path to minio 2015-05-11 16:23:10 -07:00
Harshavardhana e39de4f603 Fix typo introduced in previous commit 2015-05-11 14:24:31 -07:00
Harshavardhana f45635524f Remove SignatureV2 support, bring in SignatureV4 header only validation for now 2015-05-11 13:38:40 -07:00
Harshavardhana b48b7f51ef Use convenience functions for url.Values over looping through keys, fix a bug in listObjectParts() 2015-05-10 11:32:41 -07:00
Harshavardhana 14b5828505 Handle partNumberMarker with listObjectParts now and other fixes 2015-05-09 19:39:00 -07:00
Frederick F. Kautz IV cf559fa48e Limiting upload id size 2015-05-09 18:28:50 -07:00
Frederick F. Kautz IV 61a9ea0f4b Adding support for memory driver test 2015-05-09 18:15:27 -07:00
Frederick F. Kautz IV 058604aa21 Adding tests and fixes for multipart uploads uncovered from tests 2015-05-09 17:42:14 -07:00
Harshavardhana 8860aa0c8c Implement AbortMultipart 2015-05-09 16:09:50 -07:00
Harshavardhana 18c7f620cc Add listparts support 2015-05-09 13:55:55 -07:00
Harshavardhana 04e388d620 Handle two different styles of time format, s3cmd now compatible 2015-05-08 16:01:59 -07:00
Harshavardhana 01b474fca3 Move expiration code into its own and add ticker select case 2015-05-08 12:37:49 -07:00
Harshavardhana 23a545673d Verify incoming md5sum for invidual parts in complete multipart request and some simplification of the code 
Add two new functions for session cleanup
 2015-05-08 03:59:05 -07:00
Harshavardhana 616241a2c1 Re-enable bucket deletion, this time with uploadID sitting there 2015-05-08 02:28:38 -07:00
Harshavardhana 2ea10c798b Implement proper errors for Multipart 2015-05-08 02:02:51 -07:00
Harshavardhana 82c3656f79 Reply back CompleteMultipartUploadResult properly with final ETag computed 
- Now s3 libraries and also objectstorage-go work properly
 2015-05-07 23:01:20 -07:00
Frederick F. Kautz IV ae5c99d7bb Router should work with ?uploads unescaped now 2015-05-07 21:37:34 -07:00
Frederick F. Kautz IV 9dc7b82b39 Merge pull request #576 from fkautz/pr_out_adding_multipart_support 2015-05-07 20:00:20 -07:00
Frederick F. Kautz IV f050d5e974 Adding multipart support 2015-05-07 19:55:30 -07:00
Frederick F. Kautz IV ed71c28c37 Fixing panic in intelligent, attempted to delete nonexistent key 2015-05-07 19:12:49 -07:00
Frederick F. Kautz IV d0c4334834 Adding feature flags 2015-05-07 13:56:54 -07:00
Harshavardhana 1df28f3060 Get rid of sub-domain code and some other formatting changes 2015-05-07 00:21:40 -07:00
Harshavardhana 6d3d92694f Add Marker support for donut, also add a testing code 2015-05-05 22:18:24 -07:00
Harshavardhana 75028c2ad1 Add lexicographic Marker/NextMarker support for recursive listing of objects. 
Also update times when an object is accessed logic
 2015-05-05 18:05:34 -07:00
Frederick F. Kautz IV 24af2222c9 Adding xmlns 2015-05-05 12:54:34 -07:00
Harshavardhana 193a6606db Fix a potential race between ExpireObjects() and Set() over items map 2015-05-04 22:36:47 -07:00
Harshavardhana ed1259d6f0 Deprecate LRU use just map[string]interface{} and call it intelligent 'cache' 2015-05-04 22:01:56 -07:00
Harshavardhana 63edb1e9a0 Nullify list memory to nil as we remove the element 2015-05-04 17:22:08 -07:00
Harshavardhana d63064b8af Optimize memory usage in GetPartialObject() for memory driver 2015-05-04 15:26:56 -07:00
Harshavardhana c8f31d97a8 Modify LRU further to add object expiration 2015-05-04 04:32:44 -07:00
Harshavardhana 670f997b07 Bring blockingWriter from client and use it here 2015-05-04 03:55:43 -07:00
Harshavardhana d0df548eb5 Use new LRU inside memory driver 2015-05-04 03:55:43 -07:00
Harshavardhana f7caef2d26 Change CreateObject() to take size argument from content-length 2015-05-04 03:55:43 -07:00
Harshavardhana c342ce1588 New LRU based on GroupCache which keeps track of memory rather than entries 2015-05-04 03:55:19 -07:00
Frederick F. Kautz IV 3cf045fd1b Deleting key from lastAccessedObjects on eviction 2015-05-01 22:01:07 -07:00
Harshavardhana 9ec8925f1c Rename IsValidObject() to IsValidObjectName() 2015-05-01 20:31:26 -07:00
Frederick F. Kautz IV 33ecba8e49 Reimagining memory settings cli input 2015-05-01 18:12:58 -07:00
Frederick F. Kautz IV 25c268ac88 Attempt to fix memory lock on Get preventing write 2015-05-01 17:45:40 -07:00
Frederick F. Kautz IV 85f1b46492 Removing log message from test 2015-05-01 17:32:18 -07:00
Frederick F. Kautz IV 7cf86e6114 Fix docker support 2015-05-01 15:23:39 -07:00
Harshavardhana 2c7b30e5e1 Even mux vars don't help, handle it by looking at req.URL.Path 2015-05-01 12:41:17 -07:00
Frederick F. Kautz IV b86b26e7e5 Exposing expiration for memory driver 2015-04-30 21:20:54 -07:00
Harshavardhana c01fb92eca Merge pull request #557 from harshavardhana/pr_out_make_builddate_universal 2015-04-30 21:19:04 -07:00
Harshavardhana bbc32d6ac0 Disable domain based routing for now 2015-04-30 21:15:53 -07:00
Harshavardhana f3078d047c Get bucketName from mux.Vars instead of heuristic code, resolves subdomain issues 2015-04-30 21:11:32 -07:00
Frederick F. Kautz IV e911cc332e Merge pull request #554 from fkautz/pr_out_disable_eviction_on_0_expiration_time 2015-04-30 19:40:46 -07:00
Harshavardhana 8e3d48bf35 Use UTC() everywhere 
ref - https://github.com/golang/go/issues/1988, no standard function to set timeZone
 2015-04-30 19:38:32 -07:00
Frederick F. Kautz IV 94e24f1a05 Disable eviction on 0 expiration time 2015-04-30 19:34:35 -07:00
Harshavardhana 72395798c8 Merge pull request #552 from harshavardhana/pr_out_minio_config_directory_should_be_0700_not_os_modedir 2015-04-30 17:40:15 -07:00
Harshavardhana 2de81c8a3c Minio config directory should be 0700 not os.ModeDir 2015-04-30 15:42:49 -07:00
Frederick F. Kautz IV 8e2f7f9493 Wrapping functions that require gc so that references are released before running 2015-04-30 12:38:12 -07:00
Harshavardhana d815e6adfd Changes to CreateObject() now returns back md5 along with any error 
- This change is necessary to avoid the racy calls to GetObjectMetadata()
- This change is also necessary since one has to reply back md5sum with
  PUT object response header
 2015-04-30 03:38:14 -07:00
Harshavardhana fd2203b1b7 Some more improvements to connection limit 2015-04-30 01:23:44 -07:00
Harshavardhana f8a16dd22b Memory now evicts bucket if no more objects in memory struct 
- To avoid race in expiration while accessing memory driver structs with
  in two competing write locks. Use lru Len() instead to know exact length
  to schedule for eviction.

- squash both bucket and object structs, instead use a separate map to keep
  mutable info of lastAccessTime which can be independently used inside the
  expiration routine.
 2015-04-30 00:31:52 -07:00
Frederick F. Kautz IV ed0a9a9e06 Removing entity too large errors from memory driver 2015-04-30 00:09:54 -07:00
Frederick F. Kautz IV aff2b3b06a Adding debug statements for testing 2015-04-29 21:13:32 -07:00
Frederick F. Kautz IV 82a16b8848 Make minio work with curl and browsers again 2015-04-29 20:17:26 -07:00
Frederick F. Kautz IV 3a1386165f Fixing race condition 2015-04-29 18:54:59 -07:00
Harshavardhana 03bca9a001 Fix out of memory issue 2015-04-29 18:33:19 -07:00
Harshavardhana 25d4c0f6fa A small logical change in messaging and logging 2015-04-29 17:30:54 -07:00
Frederick F. Kautz IV f596c744ed Adding error message on connection limit met 2015-04-29 17:21:23 -07:00
Harshavardhana a89c4a57d3 Return resources as its available, not a new value 2015-04-29 16:47:14 -07:00
Harshavardhana 251c99176a Merge pull request #537 from harshavardhana/pr_out_add_proper_content_length_for_error_and_success_responses 2015-04-29 15:32:12 -07:00
Frederick F. Kautz IV 0da04c6f17 Expirations are now based on last accessed instead of time created 2015-04-29 15:28:45 -07:00
Harshavardhana 92e4301414 Add proper content-length for error and success responses 
- All compliance issues with S3 API for Put,Get,List (Bucket,Object) respectively
 - Encodes and returns back proper HTTP headers
 2015-04-29 15:28:28 -07:00
Frederick F. Kautz IV 5fc5f79ab7 Adding lru 2015-04-29 15:08:26 -07:00
Frederick F. Kautz IV 8fcb697844 Simplifying memory sleep logic 2015-04-29 13:53:09 -07:00
Frederick F. Kautz IV 5cc9418ca7 Adding expiration 2015-04-29 13:02:21 -07:00
Frederick F. Kautz IV a608524ebf api logging is now in its own package 2015-04-29 12:24:53 -07:00
Harshavardhana d1d5f7a78d Collate success response into writeSuccessResponse(), add docs 2015-04-29 10:52:04 -07:00
Harshavardhana b18bb230da maxObjectSize and minObjectSize limitation added at putObjectHandler() 
Put() replies back with

   - EntityTooLarge with > 5GB in single PUT operation
   - EntityTooSmall with < 1B in single PUT operation
   - IncompleteBody with ho Content-Length found in HTTP request header
 2015-04-29 02:19:53 -07:00
Frederick F. Kautz IV 172cd18455 Adding initial logging framework to api_router 2015-04-28 20:37:08 -07:00
Frederick F. Kautz IV 88cca647ea Adding json logging to stderr 2015-04-28 18:11:53 -07:00
Frederick F. Kautz IV de091068e8 Removing quotas, will handle with iptables 2015-04-28 12:14:34 -07:00
Frederick F. Kautz IV f2257a643c Critical sections are now protected by lock 2015-04-27 19:59:58 -07:00
Harshavardhana 605edbc307 Add some logging with debug 2015-04-27 19:26:24 -07:00
Harshavardhana 7bdcf4bbf3 Merge pull request #515 from fkautz/pr_out_fix_multiple_response_writeheader_calls 
Fix multiple response.WriteHeader calls
 2015-04-27 19:21:47 -07:00
Frederick F. Kautz IV db5870be28 Fix multiple response.WriteHeader calls 2015-04-27 18:30:21 -07:00
Frederick F. Kautz IV fc2537eef9 Merge pull request #513 from fkautz/pr_out_adding_software_fallback 2015-04-27 14:41:41 -07:00
Frederick F. Kautz IV 289c859675 Adding software fallback 2015-04-27 13:55:40 -07:00
Harshavardhana ade803a923 Verify if request date is 5minutes late, reject such a request as 
it could be a possible replay attack.

This commit also fixes #505, by returning MethodNotAllowed instead of NotImplemented
 2015-04-27 03:55:10 -07:00
Harshavardhana 9232ce3b4e Now client requests for ACL changes are honored through PutBucketACL API 2015-04-27 03:04:29 -07:00
Harshavardhana c99d96dbc2 Now donut supports bucket ACL's, bringing in this support for dl.minio.io 2015-04-27 02:09:32 -07:00
Harshavardhana 8beca83090 Limit memory mode at 100 buckets and return error appropriately 
~~~
mc: <ERROR> Failed to create bucket for URL [http://localhost:9000/newbucket-101]. Reason: [You have attempted to create more buckets than allowed.].
~~~

Response from server
 2015-04-26 22:15:39 -07:00
Harshavardhana 2d96d5ad57 Enhance error responses for request limit and bring some code from api errors 
~~~
mc: <ERROR> Failed to create bucket for URL [http://localhost:9000/newbucket-101]. Reason: [Reduce your request rate.].
~~~

Client sees proper errors now.
 2015-04-26 22:10:59 -07:00
Frederick F. Kautz IV 8a57006b36 Merge pull request #504 from fkautz/pr_out_making_lock_much_less_aggressive_fixes_multiple_uploads_blocking 2015-04-26 20:11:50 -07:00
Frederick F. Kautz IV a974e4d7c7 Making lock much less aggressive, fixes multiple uploads blocking 2015-04-26 20:08:37 -07:00
Frederick F. Kautz IV 258bf55e31 Bandwidth quota now supports 100-Continue 2015-04-26 20:07:05 -07:00
Frederick F. Kautz IV 9431a54a27 finish me 2015-04-26 19:25:48 -07:00
Frederick F. Kautz IV 202fc26719 Revert "Exporting WriteErrorResponse" 
This reverts commit e05d46a0b5.
 2015-04-26 17:02:49 -07:00
Frederick F. Kautz IV e05d46a0b5 Exporting WriteErrorResponse 2015-04-26 16:28:54 -07:00
Frederick F. Kautz IV 64f7d04cd1 Adding connection limits 2015-04-26 16:14:51 -07:00
Frederick F. Kautz IV 7867ee89fa Bandwidth quotas now work on data returned from server 2015-04-26 15:49:55 -07:00
Frederick F. Kautz IV e2403cbc56 Quotas are more accurate, occur on Read() 2015-04-26 15:07:57 -07:00
Frederick F. Kautz IV bbb493080f Switching back to BigEndian 2015-04-25 18:16:21 -07:00
Frederick F. Kautz IV 3a48f9fe4d Adding quotas based upon type 2015-04-25 18:07:02 -07:00
Harshavardhana b75f46aaa0 Use standard functions to achieve byteorder converstion 2015-04-25 13:18:15 -07:00
Frederick F. Kautz IV e2475925ea quota handlers now log against ip properly against a duration 2015-04-24 20:49:33 -07:00
Harshavardhana b3a9d8b058 Merge pull request #483 from harshavardhana/pr_out_memory_now_handles_submitting_large_files_fixes_482 2015-04-24 15:25:38 -07:00
Harshavardhana c5d8ca245d Memory now handles submitting large files - fixes #482 2015-04-24 15:22:22 -07:00
Frederick F. Kautz IV fda50d52c8 Add initial framework for quota 2015-04-24 00:16:29 -07:00
Frederick F. Kautz IV 5a607a6762 Moving factories and start minio method to server package 2015-04-23 16:13:50 -07:00
Frederick F. Kautz IV 5c515d3138 Simplifying server config handling 2015-04-23 12:44:22 -07:00
Harshavardhana 5ae8d34f76 Donut mode will now take multiple paths as argument, updated docs as well 2015-04-23 11:36:14 -07:00
Harshavardhana a5729e8c02 Disable verifying ACL until we get webcli up and running 2015-04-23 03:21:08 -07:00
Harshavardhana bcc4449de7 Add proper help and several other cleanup 2015-04-23 02:42:50 -07:00
Harshavardhana e95604ff86 Finishing all the test case support for ACL and other fixes 2015-04-23 01:24:46 -07:00
Frederick F. Kautz IV 0f8626ab1a Refactoring cli to take commands 2015-04-23 01:05:44 -07:00
Harshavardhana 848c4ee31c Further fixes for ACL support, currently code is disabled in all the handlers 
Disabled because due to lack of testing support. Once we get that in we can
uncomment them back.
 2015-04-22 19:31:45 -07:00
Harshavardhana 1c0ff2c758 ACL driver/storage layer support 2015-04-22 19:31:45 -07:00
Harshavardhana c8713fd650 Bring in full fledged acl support 2015-04-22 18:26:35 -07:00
Harshavardhana 2c1455af1b Implement x-amz-acl tests 2015-04-22 18:26:35 -07:00
Harshavardhana 107e077ec0 Implement x-amz-acl handling 2015-04-22 18:26:35 -07:00
Frederick F. Kautz IV c56f7380d6 Memory driver now limits by size 2015-04-22 17:50:26 -07:00
Harshavardhana b121c8588f Remove bucketpolicy handlers and all its references 2015-04-22 14:46:51 -07:00
Harshavardhana 3f33643d39 File driver to take the high road 2015-04-16 02:16:39 -07:00
Frederick F. Kautz IV 59a81a73c6 Adding lru to memory driver, not wired to command line opts 2015-04-15 12:49:41 -07:00
Frederick F. Kautz IV 0cab3fd151 Better locking in memory driver 2015-04-15 11:48:56 -07:00
Harshavardhana 148935392d Fix golint complaint 2015-04-08 23:42:59 -07:00
Frederick F. Kautz IV 26d8deee40 Removing unneeded files 2015-04-08 20:58:42 -07:00
Frederick F. Kautz IV 7212b3406a Adding comments to iodine 2015-04-08 19:44:44 -07:00
Frederick F. Kautz IV c12d5ea8a9 Adding iodine error conversion 2015-04-08 19:32:37 -07:00
Harshavardhana 38155e86ac iodine is part of minio now 2015-04-08 18:05:48 -07:00
Harshavardhana 7fd1cc073c Add everything back into one project hood, still missing iodine 2015-04-08 17:47:56 -07:00
Frederick F. Kautz IV 6241c9880b Moving object storage drivers back to minio 2015-04-08 16:28:14 -07:00
Harshavardhana 2a21b7d639 Implement headBucketHandler() and its related tests 
Remove internal testify/mock and use upstream - update all godeps
 2015-04-07 01:16:32 -07:00
Frederick F. Kautz IV ea961ef16d Updating to use objectdriver 2015-04-06 18:00:18 -07:00
Frederick F. Kautz IV e71cc8d833 Adding Debug logger to log package 2015-04-06 12:15:19 -07:00
Harshavardhana 319feb52f1 erasure is now external pull it and update from minio-io/erasure 2015-04-05 23:51:24 -07:00
Frederick F. Kautz IV 052817ff91 Fix fd leak 2015-04-05 21:33:24 -07:00
Harshavardhana d324c4b061 Return proper return for all drivers when BucketNotFound 2015-04-05 15:55:10 -07:00
Harshavardhana 0475d7d056 Migrate to external Donut package 2015-04-05 14:59:24 -07:00
Frederick F. Kautz IV 2f44438843 Reformatting imports 2015-04-04 13:59:59 -07:00
Frederick F. Kautz IV be4bc44b4d Replacing gopkg.in/check.v1 with minio-io/check 2015-04-04 13:37:30 -07:00
Frederick F. Kautz IV e6abca1a63 Adding iodine to object HEAD verb 2015-04-04 12:46:34 -07:00
Frederick F. Kautz IV 46badca07e Removing Godeps from pkg/api 2015-04-03 20:40:21 -07:00
Frederick F. Kautz IV 4171ad8b69 Adding test for invalid range in api 2015-04-03 20:35:39 -07:00
Harshavardhana 4079dd1d12 Commit testify,objx, gorilla/mux, minio-io/erasure gopkg.in/check.v1 2015-04-03 18:57:47 -07:00
Harshavardhana 4b25d4e8db Use verifyError instead of parsResponse, also get rid of mxj dependency 2015-04-03 18:56:47 -07:00
Frederick F. Kautz IV 9954e5c280 Better error handling + tests for get object 2015-04-03 18:53:27 -07:00
Frederick F. Kautz IV db1e6e5988 Adding tests for put bucket errors, fixed mismatching error 2015-04-03 18:37:39 -07:00
Frederick F. Kautz IV ebf1e784b3 Adding iodine and tests for for list buckets errors 2015-04-03 18:29:31 -07:00
Frederick F. Kautz IV 5865295f5d Adding better error support to api 2015-04-03 18:06:30 -07:00
Frederick F. Kautz IV 8757ccf9a3 Fixing race condition in api_test.go 2015-04-01 20:30:57 -07:00
Harshavardhana 85e547d854 Parse response headers in tests 2015-04-01 20:07:51 -07:00
Harshavardhana 92f408af5d Add erasure to godep 2015-04-01 16:56:43 -07:00
Frederick F. Kautz IV 818b6d2e33 Minor cleanup of memory get partial object 2015-04-01 15:50:59 -07:00
Frederick F. Kautz IV 0c5ed6f3ba Add partial object support to donut driver 2015-04-01 15:48:12 -07:00
Frederick F. Kautz IV 563a914046 Fix whitespace 2015-04-01 12:30:58 -07:00
Frederick F. Kautz IV 83f0f0971f Adding comments on why we subtract a second from date1 in an api test 2015-04-01 12:29:08 -07:00
Frederick F. Kautz IV 1f699a094b Minor modification for creating temp dir in api test 2015-04-01 12:07:33 -07:00
Frederick F. Kautz IV 5b1350b4bf api test now states which driver is currently running 2015-04-01 12:06:22 -07:00
Harshavardhana a56098b8c9 Make sure in API we handle all error type exclusively, and return certain values exclusively 2015-03-31 20:26:16 -07:00
Frederick F. Kautz IV 6136960cfd Moving api_test to fixtures for drivers 2015-03-31 19:53:25 -07:00
Frederick F. Kautz IV 959d90fcf7 Adding test for partial objects 2015-03-31 19:07:45 -07:00
Frederick F. Kautz IV 62c4c997c2 Simplifying error handling in api bucket handlers 2015-03-30 14:27:32 -07:00
Frederick F. Kautz IV 318d9d5aa8 API Bucket Handlers errors simplified 2015-03-30 11:40:28 -07:00
Frederick F. Kautz IV 7d657eb55c Simplifying error response writing for object handlers 2015-03-29 22:03:13 -07:00
Frederick F. Kautz IV c3cda5d9a8 Adding comments to errors 2015-03-29 22:03:13 -07:00
Frederick F. Kautz IV ded090a8b0 Normalizing imports in api 2015-03-29 21:41:38 -07:00
Frederick F. Kautz IV 8cbc1c2346 More api cleanup 2015-03-29 21:38:23 -07:00
Frederick F. Kautz IV 452f0f8335 Cleaning up more api code 2015-03-29 21:06:51 -07:00
Frederick F. Kautz IV d644233054 Rename writeObjectHeaders to setObjectHeaders, fix writing multiple headers bug 2015-03-29 18:49:32 -07:00
Harshavardhana b748cf359a More simplification to erasure also add two more test functions 2015-03-29 14:52:13 -07:00
Frederick F. Kautz IV 86b32c2147 Minor change from trace to error log 2015-03-28 18:47:36 -07:00
Frederick F. Kautz IV d5db891094 Simplfying code in api, round 1. Decoupling functions which both write headers and data. 2015-03-28 18:01:00 -07:00
Frederick F. Kautz IV b165efdfcb Refactoring pkg/utils/log, copying from golang's log and modifying 2015-03-28 15:58:48 -07:00
Harshavardhana f3f0a3a5f7 Merge pull request #403 from harshavardhana/pr_out_let_donut_driver_use_log_errorln_wrapper 2015-03-28 12:43:42 -07:00
Harshavardhana 767f4d1d8f Let donut driver use log.Errorln wrapper 2015-03-28 12:43:26 -07:00
Frederick F. Kautz IV c901d0fb32 Fix for readers closing in erasure early 2015-03-27 23:49:14 -07:00
Frederick F. Kautz IV 96e26335e2 Adding tests and a fix for get object where bucket is not found 2015-03-27 23:23:22 -07:00
Frederick F. Kautz IV ad370f341b Fixing donut location 2015-03-27 12:44:13 -07:00
Frederick F. Kautz IV 53439d7768 Updating iodine to match new api 2015-03-26 15:55:06 -07:00
Frederick F. Kautz IV c5f10df957 Removing TODO 2015-03-26 11:33:50 -07:00
Frederick F. Kautz IV eaeeb4bef0 Refactoring erasure, each block is encoded by a function. Iodine included. 2015-03-26 11:32:39 -07:00
Frederick F. Kautz IV 31746fdeb2 Adding iodine to donutObjectWriter.Write 2015-03-26 10:49:03 -07:00
Frederick F. Kautz IV bfbb97c98f Merge pull request #393 from fkautz/pr_out_adding_iodine_to_object_writer 2015-03-26 10:42:57 -07:00
Frederick F. Kautz IV f7a300400e Merge pull request #392 from fkautz/pr_out_adding_iodine_to_node_local 2015-03-26 10:42:50 -07:00
Frederick F. Kautz IV caf7e41499 Adding iodine to object_writer 2015-03-26 10:41:40 -07:00
Frederick F. Kautz IV 5cdf2564a5 Adding iodine to node_local 2015-03-26 10:21:24 -07:00
Frederick F. Kautz IV 70a2217eb2 AddNode fails with proper error code when bucketid is malformed 2015-03-26 10:02:40 -07:00
Frederick F. Kautz IV cc64417481 Adding iodine to donut 2015-03-26 09:57:21 -07:00
Frederick F. Kautz IV 40e311a919 Updating iodine to support new iodine.Error(error, map[string]string) error method 2015-03-25 19:12:55 -07:00
Harshavardhana 68f36f5e08 Merge pull request #385 from abperiasamy/erasure_cache_tbls_matrix 2015-03-25 19:11:07 -07:00
Anand Babu (AB) Periasamy b44d2100db cache decode matrix tables and matrix 2015-03-25 19:09:34 -07:00
Harshavardhana 2d0be3e3b2 fix golint complaints 2015-03-25 19:01:36 -07:00
Anand Babu (AB) Periasamy bd39768de2 janitor duty on erasure-decode 2015-03-25 18:02:03 -07:00
Frederick F. Kautz IV eff48971ad Adding iodine to donut bucket 2015-03-25 17:05:26 -07:00
Frederick F. Kautz IV d52dd5ed5b Adding iodine to pkg/server/server.go 2015-03-25 17:04:26 -07:00
Harshavardhana 58082cd8dc Add gocyclo into source build, choosing cyclomatic complexity tolerance of 15 and below 2015-03-25 15:57:17 -07:00
Harshavardhana c366fdd2d8 loop only on readers once 2015-03-25 14:48:46 -07:00
Harshavardhana e1f4ed4650 Comment exported functions GetEncodedBlocksLen, GetEncodedBlockLen 2015-03-25 13:32:35 -07:00
Harshavardhana 2cbd15b690 Golint fixes 2015-03-25 13:25:25 -07:00
Harshavardhana d967babbaa Remove unnecessary if <block> 2015-03-25 12:58:15 -07:00
Frederick F. Kautz IV be229473b5 Updating iodine 2015-03-25 00:43:48 -07:00
Harshavardhana 0f8021abd1 Fix nil pointer dereference when err is nil 2015-03-25 00:37:25 -07:00
Frederick F. Kautz IV 1be3766ae6 Adding iodine annotation to donut.Start(path) 2015-03-25 00:27:45 -07:00
Frederick F. Kautz IV 12a17f1ed7 Merge pull request #364 from fkautz/pr_out_using_iodine_in_donut_start_ 2015-03-25 00:25:41 -07:00
Frederick F. Kautz IV d06f5afdfc Using iodine in donut.Start() 2015-03-25 00:25:27 -07:00
Harshavardhana b3c3a8d6f2 Avoid empty if else body.. 2015-03-25 00:24:41 -07:00
Harshavardhana ada98c56c0 Fix minio hang in donut driver. 
remove sending err to errorChannel until we figure out the right fix.
 2015-03-25 00:15:36 -07:00
Frederick F. Kautz IV 1ae74591f6 Adding iodine 2015-03-24 23:54:49 -07:00
Frederick F. Kautz IV dd5ba865ff Merge remote-tracking branch 'origin/master' into HEAD 
Conflicts:
	pkg/drivers/donut/donut.go
	pkg/storage/donut/bucket.go
	pkg/storage/donut/donut.go
	pkg/storage/donut/donut_test.go
	pkg/storage/donut/donutdriver.go
 2015-03-24 21:04:08 -07:00
Frederick F. Kautz IV cf2550b267 Populate buckets at start 2015-03-24 19:58:03 -07:00
Harshavardhana 653ea50a59 Erasure technique is now uint8, handle it appropriately 2015-03-24 19:12:52 -07:00
Anand Babu (AB) Periasamy fa9b1f341b erasure encode janitor duty 2015-03-24 18:47:53 -07:00
Harshavardhana 6a87ed001c Handle errors properly during erasure Decoding, also populate technique and verify 2015-03-24 18:14:13 -07:00
Harshavardhana 0474439b43 Code restructuring, assigning proper subsystems to each 
- Drivers contain
    * donut/*
    * file/*
    * memory/*

 - Storage format contains
    * donut/*
      - GetObject() --> renamed to GetObjectReader()
      - Deleted stale objectwriter.go, renamed donutwriter.go to object_writer.go

Simplifying, and documenting codebase further
 2015-03-24 06:47:10 -07:00
Frederick F. Kautz IV 3c4012f1e7 Adding donut backend and setting as default 2015-03-23 20:07:22 -07:00
Harshavardhana 7c1c4b8a02 Merge pull request #351 from harshavardhana/pr_out_implement_log_package_as_drop_in_replacement_for_handling_debug_log_level 2015-03-23 18:55:14 -07:00
Harshavardhana 914962bd93 Implement log package as drop-in replacement for handling Debug log-level 2015-03-23 18:18:11 -07:00
Frederick F. Kautz IV f91f5e59ee Running gofmt manually, auto gofmt didn't simplify code 2015-03-23 12:26:41 -07:00
Frederick F. Kautz IV 56a298fd6a Refactoring method variable names for erasureWriter struct 2015-03-23 12:25:20 -07:00
Frederick F. Kautz IV d57e9d6c18 Adding sys. as reserved in erasureWriter 2015-03-23 11:56:24 -07:00
Frederick F. Kautz IV b2f47ef446 Switching content hashing from sha512 to md5 2015-03-23 11:46:19 -07:00
Harshavardhana b8e9834faa Rename parms as --> params for brevity and misc cleanup 2015-03-22 20:26:01 -07:00
Frederick F. Kautz IV b2c3172095 Adding sha512 support for pre-encoded data and verification on decode 2015-03-22 20:11:25 -07:00
Frederick F. Kautz IV befac7d047 Exorcising donutbox 2015-03-22 19:55:15 -07:00
Frederick F. Kautz IV eb8005cc65 Reading correct block size in erasure coded data 2015-03-22 18:49:30 -07:00
Anand Babu (AB) Periasamy b9b0b35edf removed private k and m and some more cleanups 2015-03-22 18:22:09 -07:00
Frederick F. Kautz IV 293e438830 Merge pull request #341 from abperiasamy/janitor 2015-03-22 17:34:25 -07:00
Anand Babu (AB) Periasamy 3a76229551 encoded size function is now public + some janitor duty 2015-03-22 17:17:53 -07:00
Harshavardhana 3929135c34 Make sure we use O_EXCL with O_CREATE to make sure we don't trip over existing file 2015-03-22 15:55:40 -07:00
Harshavardhana 2fa9320df2 De-couple donut into smaller files, useful for ease in external integration 2015-03-22 15:43:37 -07:00
Frederick F. Kautz IV 55f3659bb3 Removing log tracers 2015-03-22 15:39:50 -07:00
Frederick F. Kautz IV 03f51eac29 Adding list objects to donut 2015-03-22 15:33:52 -07:00
Harshavardhana 4b9fbd3b3a DonutfileWriter should return value of file.Close() 2015-03-22 14:51:18 -07:00
Frederick F. Kautz IV ced6edc9fe Minor change, removing unnecessary commented method 2015-03-22 12:44:45 -07:00
Frederick F. Kautz IV 99cf484488 Adding GetObjectMetadata to donut 2015-03-22 12:36:51 -07:00
Frederick F. Kautz IV 703af6c3ce Removing gob from donut driver 2015-03-22 12:17:12 -07:00
Harshavardhana 95affcb119 Fix build issues for new DonutDriver() 2015-03-22 02:53:21 -07:00
Frederick F. Kautz IV 933d44c656 New donut driver with local file encoding 2015-03-22 00:40:21 -07:00
Harshavardhana a8056c4c12 Return error during JoinFiles back to the caller for inspection 2015-03-20 03:17:08 -07:00
Harshavardhana 2b0b5f014a Update license header Mini should be Minimalist really 2015-03-19 14:35:50 -07:00
Harshavardhana c3602967ab Fix bucketpolicy to handle s3 and minio policy names separately. 
Also fix an apparent bug in GetBucketPolicy{}
 2015-03-19 12:22:14 -07:00
Harshavardhana 6eb5315249 Resource list updates 2015-03-19 00:47:34 -07:00
Harshavardhana a1baed3bbc Refactor to move config handling into api/ 2015-03-18 22:49:02 -07:00
Harshavardhana 7ce3ab3193 Restructure minio api, move signature checks from utils to Api. 2015-03-18 18:16:19 -07:00
Frederick F. Kautz IV 2c69433588 Adding interfaces for donut driver 2015-03-18 14:27:10 -07:00
Frederick F. Kautz IV bf86e78714 Renaming files to match 2015-03-18 12:26:44 -07:00
Frederick F. Kautz IV 1c9e550ee6 Renaming encoded driver to donutstorage 2015-03-18 12:25:37 -07:00
Frederick F. Kautz IV d7f00194b5 Adding interfaces for implementing against 2015-03-17 22:33:59 -07:00
Frederick F. Kautz IV 982a52e0d4 Merge pull request #320 from fkautz/pr_out_removing_block_size_from_donut_writer_no_longer_necessary 2015-03-17 14:25:08 -07:00
Frederick F. Kautz IV d6be8d3517 Removing block size from donut writer, no longer necessary 2015-03-17 14:24:59 -07:00
Harshavardhana 74b3d092f2 Now MinioAPI handles Content-MD5 set during putObject() 
- This change also facilitates proper error response
    in case of md5 mismatch or corruption
  - TODO a test function needs to be implemented
 2015-03-17 13:32:13 -07:00
Harshavardhana 42006c2ab0 Merge pull request #318 from harshavardhana/pr_out_remove_contenttype_pseudo_func_ 2015-03-17 00:47:46 -07:00
Harshavardhana dccf32e5c4 Remove contenttype pseudo func() 2015-03-17 00:47:22 -07:00
Frederick F. Kautz IV 90b337a3e8 Fixing order of input and layout in time.Parse 2015-03-17 00:47:17 -07:00
Frederick F. Kautz IV 25725c442a Moving to more accurate RFC3339 2015-03-17 00:45:50 -07:00
Frederick F. Kautz IV e36fa95f1f Moving from iso8601 to RFC3339 2015-03-17 00:42:08 -07:00
Frederick F. Kautz IV 598eb5192e Encoded now passes tests, enabling. 2015-03-17 00:33:49 -07:00
Harshavardhana 6d71935ea5 Encoded storage type now feature complete 2015-03-16 23:42:45 -07:00
Frederick F. Kautz IV 34cf6682fe list objects in buckets 2015-03-16 20:43:05 -07:00
Frederick F. Kautz IV d99435963b Merge pull request #311 from fkautz/pr_out_switching_from_base64_to_hex_for_etag 2015-03-16 18:39:07 -07:00
Frederick F. Kautz IV 1a88b36c55 Switching from base64 to hex for etag 2015-03-16 18:38:42 -07:00
Harshavardhana 81f1d3c195 Merge pull request #310 from harshavardhana/pr_out_make_blocksize_as_constant 2015-03-16 18:20:22 -07:00
Harshavardhana b495f15178 Make blockSize as constant 2015-03-16 18:20:01 -07:00
Frederick F. Kautz IV 211b084e50 Performing etag off of bytes rather than md5 string 2015-03-16 18:15:37 -07:00
Frederick F. Kautz IV 583c32c774 Metadata now stores hex md5, not base64. Minio API now converts md5 hex to base64. 2015-03-16 18:03:22 -07:00
Frederick F. Kautz IV 10ae089c46 Better handling of metadata in encoded 2015-03-16 17:40:59 -07:00
Harshavardhana 15f68972a5 Some necessary cleanup 
- Rename 'fs' as 'file' for brevity
  - Rename 'inmemory' as 'memory' for brevity
  - Change everywhere else
 2015-03-16 11:46:16 -07:00
Harshavardhana 765fde13f1 Remove x509 package, janitor continuation monad 2015-03-15 19:44:55 -07:00
Harshavardhana bf04d125fd Merge pull request #304 from harshavardhana/pr_out_move_from_fmt_errorf_to_errors_new 2015-03-15 19:42:23 -07:00
Harshavardhana 596eeb097e Move from fmt.Errorf to errors.New 
In many cases fmt.Errorf is good enough, but since
error is an interface, you can use arbitrary data
structures as error values, to allow callers to
inspect the details of the error.
 2015-03-15 19:41:16 -07:00
Frederick F. Kautz IV a589ddaddf Decoding and returning objects now works 2015-03-15 19:02:00 -07:00
Frederick F. Kautz IV 682bf085c4 Metadata and writing the object are now both committed together. 2015-03-15 17:42:33 -07:00
Frederick F. Kautz IV 96d0a089b8 Adding create bucket to encoded 2015-03-15 12:28:24 -07:00
Harshavardhana 3f95c55067 Skip uinimplemented tests and some cleanup 2015-03-15 01:23:57 -07:00
Frederick F. Kautz IV 3e453a394a Working in memory donut driver 2015-03-15 00:41:06 -07:00
Frederick F. Kautz IV 15d5e74837 More testing 2015-03-14 19:29:54 -07:00
Frederick F. Kautz IV 7e61696901 Inmemory donutbox is now thread safe 2015-03-14 19:21:34 -07:00
Frederick F. Kautz IV 85da9fc627 Adding start of inmem donut 2015-03-14 18:02:27 -07:00
Frederick F. Kautz IV 54472c18fe Renaming singledisk to encoded 2015-03-14 11:37:43 -07:00
Frederick F. Kautz IV 348c4d9a88 Removing old donut format 2015-03-14 11:36:19 -07:00
Frederick F. Kautz IV 2f4b92eb8d Minor refactor to prepare for file version of donut 2015-03-14 11:34:45 -07:00