Harshavardhana
1d31c76dd6
Merge pull request #694 from harshavardhana/pr_out_move_memory_code_out_add_it_as_layer_on_top_of_existing_cache_code_wip
2015-06-30 17:12:29 +00:00
Harshavardhana
8f61d6b6be
Move memory code out, add it as layer on top of existing donut code
...
Just like how http.Handlers can be overlayed on top of each other
with each implementing ServeHTTP().
drivers.Driver can be overlayed on top of each other in similar manner
which would implement the drivers.Driver interface.
API <----> cache <----> donut <----> donut(format)
2015-06-30 10:09:12 -07:00
Harshavardhana
fe3c618cc7
Merge pull request #693 from harshavardhana/pr_out_add_dummy_driver_for_community_to_submit_new_drivers
2015-06-29 23:48:37 +00:00
Harshavardhana
ab6e16bb41
Add dummy driver for community to submit new drivers
2015-06-29 16:43:50 -07:00
Harshavardhana
12de98fb62
Rename memory driver as cache
2015-06-29 16:43:50 -07:00
Harshavardhana
2571342451
Filesystem goes the high road *again*
2015-06-29 16:43:42 -07:00
Harshavardhana
c4c67581dc
Merge pull request #692 from harshavardhana/pr_out_isvalidbucket_is_sufficient_we_don_t_need_to_verify_for_
2015-06-29 22:27:11 +00:00
Harshavardhana
f74d6138da
IsValidBucket() is sufficient we don't need to verify for "."
2015-06-29 15:15:54 -07:00
Harshavardhana
b5a5861c8f
Merge pull request #691 from harshavardhana/pr_out_handle_couple_of_cases_of_oom_conditions_move_caching_to_getobject_rather_than_putobject_
...
Handle couple of cases of OOM conditions, move caching to GetObject() rather than PutObject()
2015-06-29 19:33:55 +00:00
Harshavardhana
3109909355
Handle couple of cases of OOM conditions, move caching to GetObject() rather than PutObject()
2015-06-29 12:28:50 -07:00
Harshavardhana
d07d0c670a
Return back proper errors in writeObjectData(), rename few functions
2015-06-29 11:46:35 -07:00
Harshavardhana
be816145a9
Merge pull request #690 from harshavardhana/pr_out_put_object_on_successful_write_returns_full_metadata_to_avoid_subsequent_getobjectmetadata_calls_in_driver
2015-06-29 18:21:33 +00:00
Harshavardhana
10c807f233
Put object on successful write returns full metadata, to avoid subsequent GetObjectMetadata() calls in driver
2015-06-29 11:15:46 -07:00
Harshavardhana
6921328b93
Avoid frivolous GetObjectMetadata() calls at driver level, return back all the information in donut ListObjects()
2015-06-29 11:14:58 -07:00
Harshavardhana
f05ad062ee
Merge pull request #689 from harshavardhana/pr_out_expand_http_server_struct_to_store_more_values
2015-06-29 07:17:31 +00:00
Harshavardhana
d8f7896a43
Expand http server struct to store more values
2015-06-29 00:12:28 -07:00
Harshavardhana
63f9647c80
Merge pull request #688 from harshavardhana/pr_out_use_errorchannels_only_for_services_not_for_drivers_reduce_them_to_use_simple_functions
2015-06-29 07:02:24 +00:00
Harshavardhana
42c0287943
Use errorChannels only for services not for drivers, reduce them to use simple functions
2015-06-28 23:59:47 -07:00
Harshavardhana
b2bf90afbd
Merge pull request #687 from harshavardhana/pr_out_move_to_set_not_append_due_to_large_memory_reference_copy
2015-06-28 17:15:06 +00:00
Harshavardhana
91e5f648cb
Move to Set() not Append() due to large memory reference copy
2015-06-28 10:13:12 -07:00
Harshavardhana
22abe1b397
Merge pull request #686 from harshavardhana/pr_out_add_free_method_for_proxyreader_to_aggressively_de_allocate_read_data_to_handle_certain_out_of_memory_conditions
...
Add free() method for proxyReader to aggressively de-allocate Read data, to handle certain out of memory conditions
2015-06-28 03:45:48 +00:00
Harshavardhana
ac4d8fe478
Add free() method for proxyReader to aggressively de-allocate Read data, to handle certain out of memory conditions
...
There are still some more out there
2015-06-27 20:43:25 -07:00
Harshavardhana
a4d20d1e75
Merge pull request #685 from harshavardhana/pr_out_add_append_method_to_trove_cache_for_appending_data_to_an_existing_key
...
Add Append() method to trove cache for appending data to an existing key
2015-06-28 03:28:06 +00:00
Harshavardhana
05f8654e3d
Add Append() method to trove cache for appending data to an existing key
...
This largely avoids a large buffer copy which would accumulate inside proxyReader{}
This patch also implements "initialize()" function to init and populate data
on all the existing buckets, avoiding the redundant ListBuckets() invoked by
every API call.
2015-06-27 20:25:24 -07:00
Harshavardhana
762aae7c32
Merge pull request #684 from harshavardhana/pr_out_make_sure_to_populate_on_disk_data_into_memory_upon_first_api_requests
2015-06-28 01:28:29 +00:00
Harshavardhana
367772b988
Make sure to populate on disk data into memory upon first API requests
2015-06-27 18:25:21 -07:00
Harshavardhana
350e6eb5bb
Merge pull request #683 from harshavardhana/pr_out_add_proper_command_paramters_for_donut
2015-06-28 00:41:13 +00:00
Harshavardhana
07a6aafc94
Add proper command paramters for donut
2015-06-27 17:39:22 -07:00
Harshavardhana
c65969077d
Merge pull request #682 from harshavardhana/pr_out_an_attempt_to_bring_in_memory_layer_into_donut_driver
2015-06-28 00:25:27 +00:00
Harshavardhana
45a7eab804
An attempt to bring in memory layer into donut driver
2015-06-27 17:23:34 -07:00
Harshavardhana
7ab16b5b83
Merge pull request #681 from harshavardhana/pr_out_keeping_the_lexical_order_same_add_optimizations_provide_a_comprehensive_response_from_listobjects_
...
Keeping the lexical order same add optimizations, provide a comprehensive response from ListObjects()
2015-06-27 20:15:24 +00:00
Harshavardhana
f3c25bcfc4
Keeping the lexical order same add optimizations, provide a comprehensive response from ListObjects()
2015-06-27 13:12:44 -07:00
Harshavardhana
795e48d492
Merge pull request #680 from harshavardhana/pr_out_rename_functions_for_their_purpose
2015-06-27 19:42:04 +00:00
Harshavardhana
ae66ae42c4
Rename functions for their purpose
2015-06-27 12:39:11 -07:00
Harshavardhana
9c2e861470
Merge pull request #679 from harshavardhana/pr_out_object_metadata_was_wrongly_misconstrued_to_be_mutable_handle_it
2015-06-27 06:26:21 +00:00
Harshavardhana
9a4680475f
Object metadata was wrongly misconstrued to be mutable, handle it
2015-06-26 23:22:53 -07:00
Harshavardhana
39f26acbc9
Merge pull request #678 from harshavardhana/pr_out_handle_racy_map_updates_in_listobjects_on_a_bucket
...
Avoid racy maps, read from disk on success return quickly. Many more optimizations
2015-06-27 02:59:50 +00:00
Harshavardhana
3aa6d90c5e
Avoid racy maps, read from disk on success return quickly. Many more optimizations
2015-06-26 19:49:37 -07:00
Harshavardhana
aab4937084
Merge pull request #677 from harshavardhana/pr_out_moving_to_more_typed_responses_this_removes_all_the_necessity_for_strconv
...
Donut moves to typed metadata, removing the necessity for strconv, parsing and string converstions
2015-06-26 23:26:17 +00:00
Harshavardhana
68974918ac
Donut moves to typed metadata, removing the necessity for strconv, parsing and string converstions
2015-06-26 16:23:12 -07:00
Harshavardhana
e3d8a9e0f1
Merge pull request #676 from harshavardhana/pr_out_add_new_metadata_definitions_and_use_them_wip
2015-06-26 20:38:02 +00:00
Harshavardhana
767d3743ee
Add new metadata definitions and use them
2015-06-26 13:34:09 -07:00
Harshavardhana
0cb3f76a91
Merge pull request #675 from harshavardhana/pr_out_import_quick_key_value_store_from_minio_client_for_persistent_state_files_primarily_for_donut
...
Import quick key value store from Minio Client for persistent state files, primarily for donut
2015-06-26 02:59:28 +00:00
Harshavardhana
9958e34772
Import quick key value store from Minio Client for persistent state files, primarily for donut
2015-06-25 19:57:31 -07:00
Harshavardhana
bd0dccd8f1
Merge pull request #674 from harshavardhana/pr_out_donut_cleanup_another_set
...
Donut cleanup another set
2015-06-26 01:59:45 +00:00
Harshavardhana
fb9adb5524
Donut cleanup another set
...
- Make sure to close all readers
- Fix errors in api_testsuite c.Assert(err, IsNil) should be done right after each function call
2015-06-25 18:54:34 -07:00
Harshavardhana
eec66f195a
Take all the ListObjects into bucket handlers
...
Earlier the listing would wait for all the objects to be processed
this is essentially very time consuming considering even for 100,000
files.
2015-06-25 18:04:29 -07:00
Harshavardhana
8405c4d42f
Merge pull request #673 from harshavardhana/pr_out_remove_more_bloated_code_simplify
...
Remove more bloated code - simplify
2015-06-25 21:14:43 +00:00
Harshavardhana
45e9d25931
Remove more bloated code - simplify
2015-06-25 13:02:08 -07:00
Harshavardhana
7ade42165f
Merge pull request #672 from harshavardhana/pr_out_add_simple_locking_for_donut_api_for_now_fixes_671
2015-06-25 18:33:08 +00:00
