MyFavMirrors
/
minio
mirror of https://github.com/minio/minio.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
1,669 Commits 5 Branches 503 Tags 186 MiB
Commit Graph

333 Commits

Author SHA1 Message Date
Harshavardhana 42c0287943 Use errorChannels only for services not for drivers, reduce them to use simple functions 2015-06-28 23:59:47 -07:00
Harshavardhana 91e5f648cb Move to Set() not Append() due to large memory reference copy 2015-06-28 10:13:12 -07:00
Harshavardhana ac4d8fe478 Add free() method for proxyReader to aggressively de-allocate Read data, to handle certain out of memory conditions 
There are still some more out there
 2015-06-27 20:43:25 -07:00
Harshavardhana 05f8654e3d Add Append() method to trove cache for appending data to an existing key 
This largely avoids a large buffer copy which would accumulate inside proxyReader{}

This patch also implements "initialize()" function to init and populate data
on all the existing buckets, avoiding the redundant ListBuckets() invoked by
every API call.
 2015-06-27 20:25:24 -07:00
Harshavardhana 367772b988 Make sure to populate on disk data into memory upon first API requests 2015-06-27 18:25:21 -07:00
Harshavardhana 45a7eab804 An attempt to bring in memory layer into donut driver 2015-06-27 17:23:34 -07:00
Harshavardhana f3c25bcfc4 Keeping the lexical order same add optimizations, provide a comprehensive response from ListObjects() 2015-06-27 13:12:44 -07:00
Harshavardhana ae66ae42c4 Rename functions for their purpose 2015-06-27 12:39:11 -07:00
Harshavardhana 9a4680475f Object metadata was wrongly misconstrued to be mutable, handle it 2015-06-26 23:22:53 -07:00
Harshavardhana 3aa6d90c5e Avoid racy maps, read from disk on success return quickly. Many more optimizations 2015-06-26 19:49:37 -07:00
Harshavardhana 68974918ac Donut moves to typed metadata, removing the necessity for strconv, parsing and string converstions 2015-06-26 16:23:12 -07:00
Harshavardhana 767d3743ee Add new metadata definitions and use them 2015-06-26 13:34:09 -07:00
Harshavardhana fb9adb5524 Donut cleanup another set 
- Make sure to close all readers
- Fix errors in api_testsuite c.Assert(err, IsNil) should be done right after each function call
 2015-06-25 18:54:34 -07:00
Harshavardhana eec66f195a Take all the ListObjects into bucket handlers 
Earlier the listing would wait for all the objects to be processed
this is essentially very time consuming considering even for 100,000
files.
 2015-06-25 18:04:29 -07:00
Harshavardhana 45e9d25931 Remove more bloated code - simplify 2015-06-25 13:02:08 -07:00
Harshavardhana 82dcbf262d Add simple locking for donut API for now - fixes #671 2015-06-25 11:29:11 -07:00
Harshavardhana 03b4d3b308 Go vet fixes for donut 2015-06-24 21:07:03 -07:00
Harshavardhana 57a2b53178 Removing further bloated code simplifying 2015-06-24 21:03:39 -07:00
Harshavardhana a2c205ff2e Use external package disk for donut. 2015-06-24 21:03:02 -07:00
Harshavardhana 841ff01412 Move disk into its own package, remove bloated code 2015-06-24 20:13:47 -07:00
Harshavardhana 1682c748ac Remove unnecessary interfaces from donut, cleanup 2015-06-24 19:43:38 -07:00
Harshavardhana ba2d3dea74 Fix wrong TMPFS listing in donut filesystem map 2015-06-23 23:16:06 -07:00
Harshavardhana 2fd52ca284 Add missing strongly typed errors for Donut 2015-06-23 11:54:44 -07:00
Harshavardhana 3bf64f5669 Verify d.donut to be non-nil, usually happens when multiple go versions compilations are linked - possible cause for #659 2015-06-20 11:03:17 -07:00
Harshavardhana 641f07cecf Use filepath everywhere instead of path.{} functions for portability - fixes #656 2015-06-18 16:02:45 -07:00
Harshavardhana b793f53d48 Minior change filter() to filterObjects() 2015-06-17 20:35:44 -07:00
Harshavardhana d4e71beb54 Add time.Since on buckets created time to avoid deleting buckets spuriously during eviction of objects 2015-06-13 11:28:04 -07:00
Harshavardhana 636788997b Minor optimization avoiding one unnecessary copy 2015-06-11 14:23:05 -07:00
Harshavardhana 64b014369c Return back entity too large for createObject requests bigger than cache size 2015-06-08 17:10:07 -07:00
Harshavardhana ab1976bc14 Load from persisted multipart session, now multipart uploads can resume even after restarting minio fs server 2015-06-08 15:15:37 -07:00
Harshavardhana 2976f109df More multipart fixes, filter out parts and multipart metadata from listObjects 2015-06-08 14:08:36 -07:00
Harshavardhana e08d59be00 Make sure to delete uploadid's from active session file properly 2015-06-08 12:25:51 -07:00
Harshavardhana 0d3710cb9b Store in json file instead of gob for fs metadata 2015-06-08 12:08:01 -07:00
Harshavardhana 21916dc0a1 Fix a deadlock in trove when evicting objects when we reach the maxSize used 2015-06-06 16:02:52 -07:00
Harshavardhana 097f70a3c5 Enable filesystem for API tests, this patch also implements ListMultipartUploads() 2015-06-01 20:57:13 -07:00
Harshavardhana bfb9209689 Skip files and weird directories from listBuckets(), allowing this so that minio fs mode can be used as ftp service 2015-06-01 16:43:43 -07:00
Harshavardhana d86ba5f336 Implement multipart support 
- implemented NewMultipartUpload()
 - implemented CreateObjectPart()
 - implemented ListObjectParts()
 - implemented AbortMultipartUpload()
 - implemented CompleteMultipartUpload()
 2015-06-01 16:25:11 -07:00
Harshavardhana a91e519692 Make caching a package trove, and use it inside memory driver 2015-05-31 18:41:56 -07:00
Anis Elleuch 7e5a26f42c Protect the exported method Delete() 2015-05-31 20:29:42 +01:00
Harshavardhana 0cc63706bb Resurrect back fileystem driver 2015-05-30 03:12:01 -07:00
Anis Elleuch d360d0fec3 Forbid the upload of files bigger than the memory backend capacity 2015-05-28 22:07:02 +01:00
Harshavardhana 554a244b47 Add donut OSX support 2015-05-26 02:14:34 -07:00
Harshavardhana ad60b97d52 Fix some donut TODO's regarding bucket created date 2015-05-21 11:13:49 -07:00
Harshavardhana f00bfd38da Bump gocyclo back to 16 2015-05-19 18:32:50 -07:00
Harshavardhana 3d61ba5930 Fix duplicate map allocation 2015-05-18 13:14:01 -07:00
Harshavardhana e4c0d574dc Update totalnumber of multiparts inside CreateObjectPart(), also introduce support for UploadIDMarker and Prefix for listMultipartUploads 2015-05-18 12:24:36 -07:00
Harshavardhana 5c5761ae16 Remove unnecessary GetObjectMetadata prefix argument 2015-05-15 19:25:17 -07:00
Harshavardhana 6275238a47 For multiparts use its own cache avoids deadlocks during object expiration 
Using same object cache as regular objects for multiparts causes deadlocks during object expiration routine.

As a side affect this also fixes the bug of listing multiparts during regular 'listing' of objects.
 2015-05-14 20:26:59 -07:00
Harshavardhana c41ebf7535 Fixing a race in NewMultipartUpload() session for simultaneous requests, observed at play.minio.io 2015-05-14 19:11:19 -07:00
Harshavardhana 8fb427a892 Add more tests for multipart now in driver api_testsuite 2015-05-14 17:28:39 -07:00