Commit Graph

9770 Commits

Author SHA1 Message Date
Harshavardhana d8f7896a43 Expand http server struct to store more values 2015-06-29 00:12:28 -07:00
Harshavardhana 63f9647c80 Merge pull request #688 from harshavardhana/pr_out_use_errorchannels_only_for_services_not_for_drivers_reduce_them_to_use_simple_functions 2015-06-29 07:02:24 +00:00
Harshavardhana 42c0287943 Use errorChannels only for services not for drivers, reduce them to use simple functions 2015-06-28 23:59:47 -07:00
Harshavardhana b2bf90afbd Merge pull request #687 from harshavardhana/pr_out_move_to_set_not_append_due_to_large_memory_reference_copy 2015-06-28 17:15:06 +00:00
Harshavardhana 91e5f648cb Move to Set() not Append() due to large memory reference copy 2015-06-28 10:13:12 -07:00
Harshavardhana 22abe1b397 Merge pull request #686 from harshavardhana/pr_out_add_free_method_for_proxyreader_to_aggressively_de_allocate_read_data_to_handle_certain_out_of_memory_conditions
Add free() method for proxyReader to aggressively de-allocate Read data, to handle certain out of memory conditions
2015-06-28 03:45:48 +00:00
Harshavardhana ac4d8fe478 Add free() method for proxyReader to aggressively de-allocate Read data, to handle certain out of memory conditions
There are still some more out there
2015-06-27 20:43:25 -07:00
Harshavardhana a4d20d1e75 Merge pull request #685 from harshavardhana/pr_out_add_append_method_to_trove_cache_for_appending_data_to_an_existing_key
Add Append() method to trove cache for appending data to an existing key
2015-06-28 03:28:06 +00:00
Harshavardhana 05f8654e3d Add Append() method to trove cache for appending data to an existing key
This largely avoids a large buffer copy which would accumulate inside proxyReader{}

This patch also implements "initialize()" function to init and populate data
on all the existing buckets, avoiding the redundant ListBuckets() invoked by
every API call.
2015-06-27 20:25:24 -07:00
Harshavardhana 762aae7c32 Merge pull request #684 from harshavardhana/pr_out_make_sure_to_populate_on_disk_data_into_memory_upon_first_api_requests 2015-06-28 01:28:29 +00:00
Harshavardhana 367772b988 Make sure to populate on disk data into memory upon first API requests 2015-06-27 18:25:21 -07:00
Harshavardhana 350e6eb5bb Merge pull request #683 from harshavardhana/pr_out_add_proper_command_paramters_for_donut 2015-06-28 00:41:13 +00:00
Harshavardhana 07a6aafc94 Add proper command paramters for donut 2015-06-27 17:39:22 -07:00
Harshavardhana c65969077d Merge pull request #682 from harshavardhana/pr_out_an_attempt_to_bring_in_memory_layer_into_donut_driver 2015-06-28 00:25:27 +00:00
Harshavardhana 45a7eab804 An attempt to bring in memory layer into donut driver 2015-06-27 17:23:34 -07:00
Harshavardhana 7ab16b5b83 Merge pull request #681 from harshavardhana/pr_out_keeping_the_lexical_order_same_add_optimizations_provide_a_comprehensive_response_from_listobjects_
Keeping the lexical order same add optimizations, provide a comprehensive response from ListObjects()
2015-06-27 20:15:24 +00:00
Harshavardhana f3c25bcfc4 Keeping the lexical order same add optimizations, provide a comprehensive response from ListObjects() 2015-06-27 13:12:44 -07:00
Harshavardhana 795e48d492 Merge pull request #680 from harshavardhana/pr_out_rename_functions_for_their_purpose 2015-06-27 19:42:04 +00:00
Harshavardhana ae66ae42c4 Rename functions for their purpose 2015-06-27 12:39:11 -07:00
Harshavardhana 9c2e861470 Merge pull request #679 from harshavardhana/pr_out_object_metadata_was_wrongly_misconstrued_to_be_mutable_handle_it 2015-06-27 06:26:21 +00:00
Harshavardhana 9a4680475f Object metadata was wrongly misconstrued to be mutable, handle it 2015-06-26 23:22:53 -07:00
Harshavardhana 39f26acbc9 Merge pull request #678 from harshavardhana/pr_out_handle_racy_map_updates_in_listobjects_on_a_bucket
Avoid racy maps, read from disk on success return quickly. Many more optimizations
2015-06-27 02:59:50 +00:00
Harshavardhana 3aa6d90c5e Avoid racy maps, read from disk on success return quickly. Many more optimizations 2015-06-26 19:49:37 -07:00
Harshavardhana aab4937084 Merge pull request #677 from harshavardhana/pr_out_moving_to_more_typed_responses_this_removes_all_the_necessity_for_strconv
Donut moves to typed metadata, removing the necessity for strconv, parsing and string converstions
2015-06-26 23:26:17 +00:00
Harshavardhana 68974918ac Donut moves to typed metadata, removing the necessity for strconv, parsing and string converstions 2015-06-26 16:23:12 -07:00
Harshavardhana e3d8a9e0f1 Merge pull request #676 from harshavardhana/pr_out_add_new_metadata_definitions_and_use_them_wip 2015-06-26 20:38:02 +00:00
Harshavardhana 767d3743ee Add new metadata definitions and use them 2015-06-26 13:34:09 -07:00
Harshavardhana 0cb3f76a91 Merge pull request #675 from harshavardhana/pr_out_import_quick_key_value_store_from_minio_client_for_persistent_state_files_primarily_for_donut
Import quick key value store from Minio Client for persistent state files, primarily for donut
2015-06-26 02:59:28 +00:00
Harshavardhana 9958e34772 Import quick key value store from Minio Client for persistent state files, primarily for donut 2015-06-25 19:57:31 -07:00
Harshavardhana bd0dccd8f1 Merge pull request #674 from harshavardhana/pr_out_donut_cleanup_another_set
Donut cleanup another set
2015-06-26 01:59:45 +00:00
Harshavardhana fb9adb5524 Donut cleanup another set
- Make sure to close all readers
- Fix errors in api_testsuite c.Assert(err, IsNil) should be done right after each function call
2015-06-25 18:54:34 -07:00
Harshavardhana eec66f195a Take all the ListObjects into bucket handlers
Earlier the listing would wait for all the objects to be processed
this is essentially very time consuming considering even for 100,000
files.
2015-06-25 18:04:29 -07:00
Harshavardhana 8405c4d42f Merge pull request #673 from harshavardhana/pr_out_remove_more_bloated_code_simplify
Remove more bloated code - simplify
2015-06-25 21:14:43 +00:00
Harshavardhana 45e9d25931 Remove more bloated code - simplify 2015-06-25 13:02:08 -07:00
Harshavardhana 7ade42165f Merge pull request #672 from harshavardhana/pr_out_add_simple_locking_for_donut_api_for_now_fixes_671 2015-06-25 18:33:08 +00:00
Harshavardhana 82dcbf262d Add simple locking for donut API for now - fixes #671 2015-06-25 11:29:11 -07:00
Harshavardhana 5abcb7f348 Merge pull request #670 from harshavardhana/pr_out_go_vet_fixes_for_donut 2015-06-25 04:08:49 +00:00
Harshavardhana 03b4d3b308 Go vet fixes for donut 2015-06-24 21:07:03 -07:00
Harshavardhana 57a2b53178 Removing further bloated code simplifying 2015-06-24 21:03:39 -07:00
Harshavardhana a2c205ff2e Use external package disk for donut. 2015-06-24 21:03:02 -07:00
Harshavardhana 841ff01412 Move disk into its own package, remove bloated code 2015-06-24 20:13:47 -07:00
Harshavardhana 1682c748ac Remove unnecessary interfaces from donut, cleanup 2015-06-24 19:43:38 -07:00
Harshavardhana b915cc3611 Merge pull request #669 from harshavardhana/pr_out_add_sha256_and_sha512_windows_compatibility_layer 2015-06-24 21:43:26 +00:00
Harshavardhana 3498872467 Add sha256 and sha512 windows compatibility layer 2015-06-24 14:39:41 -07:00
Harshavardhana 2af863cefc Merge pull request #668 from harshavardhana/pr_out_fix_a_bug_on_windows_regarding_blocksse3_calculation 2015-06-24 21:29:34 +00:00
Harshavardhana 77d35b87d4 Fix a bug on windows regarding blockSSE3 calculation 2015-06-24 14:24:33 -07:00
Harshavardhana 1056e7e180 Merge pull request #667 from harshavardhana/pr_out_add_windows_code_for_sha1_and_crc32c 2015-06-24 21:19:54 +00:00
Harshavardhana f1410731db Add windows code for sha1 and crc32c 2015-06-24 14:16:32 -07:00
Harshavardhana fbee8f8122 Merge pull request #665 from harshavardhana/pr_out_fix_wrong_tmpfs_listing_in_document_filesystem_map 2015-06-23 23:17:21 -07:00
Harshavardhana ba2d3dea74 Fix wrong TMPFS listing in donut filesystem map 2015-06-23 23:16:06 -07:00