Commit Graph

7587 Commits

Author SHA1 Message Date
Harshavardhana 01c0e45b5c Add crc32c checksum support for erasure chunks
Now upon conditions

 - if checksum '.cksum' file is missing
 - or corrupted chunk (mismatching chunk)
 - or missing chunk entirely

Decoding works consistently
2014-12-10 00:58:15 -08:00
Harshavardhana 72ddb345a1 Merge pull request #99 from harshavardhana/pr_out_make_get_put_more_robust 2014-12-09 03:37:08 -08:00
Harshavardhana f6a5b61b00 Make get/put more robust
- init folder if not present already in 'get()' call
  - put.go was referencing wrong args - fix it
  - if no OBJECTNAME is specified during 'get' - grab a
    list of files under "~/.minio"
  - Provide new API GetList() for ObjectStorage struct
2014-12-09 03:32:31 -08:00
Harshavardhana 3fe2c2af01 Merge pull request #98 from harshavardhana/pr_out_add_sha1_avx2_supported_fast_implementation_with_some_crc32c_cleanup 2014-12-09 01:01:05 -08:00
Harshavardhana 43680cd7b6 Add sha1 - AVX2 supported fast implementation, with some crc32c cleanup 2014-12-09 01:00:02 -08:00
Frederick F. Kautz IV 93e57118f8 Merge pull request #97 from fkautz/pr_out_adding_append_storage_driver 2014-12-08 23:37:54 -08:00
Frederick F. Kautz IV 1d7b3866cb Adding append storage driver 2014-12-08 23:37:25 -08:00
Frederick F. Kautz IV 76f21c1cfc Merge pull request #96 from fkautz/pr_out_removing_printlns_from_test_code 2014-12-08 20:40:08 -08:00
Frederick F. Kautz IV af8eb7a55d Removing printlns from test code 2014-12-08 20:39:30 -08:00
Frederick F. Kautz IV a40d834a42 Merge pull request #95 from fkautz/pr_out_adding_extremely_simple_fs_based_get_and_put_in_erasure_demo 2014-12-08 20:23:02 -08:00
Frederick F. Kautz IV d1ebf935da Adding extremely simple fs based get and put in erasure-demo 2014-12-08 20:20:25 -08:00
Frederick F. Kautz IV c2917f0d64 Merge pull request #94 from fkautz/pr_out_refactoring_file_storage_driver_to_fsstorage 2014-12-08 15:27:37 -08:00
Frederick F. Kautz IV 0cf80e075e Refactoring file storage driver to fsstorage 2014-12-08 14:29:25 -08:00
Frederick F. Kautz IV 8a5b8ac90f Merge pull request #93 from fkautz/pr_out_cpu_tests_now_actually_test_properly_on_linux_and_return_bool_instead_of_int 2014-12-08 14:25:11 -08:00
Frederick F. Kautz IV e71998c584 CPU tests now actually test properly on linux and return bool instead of int 2014-12-08 13:10:18 -08:00
Harshavardhana 5d4c02ad4a Merge pull request #92 from harshavardhana/pr_out_fixes_71_some_crazy_races_inside_erasure_and_simplify_the_code 2014-12-07 01:34:23 -08:00
Harshavardhana 7314b5e37d Fixes #71 - some crazy races inside erasure and simplify the code
- This change also brings in changing 'unsigned char' to 'uint8_t'
for brevity
2014-12-07 01:33:49 -08:00
Harshavardhana d57700922d Merge pull request #91 from harshavardhana/pr_out_remove_cpufeatures_c_code_since_pkgs_crc32c_cpu_should_be_used_instead
Remove cpufeatures C code, since pkgs/crc32c/cpu should be used instead
2014-12-05 17:26:03 -08:00
Harshavardhana e414d5f67b Remove cpufeatures C code, since pkgs/crc32c/cpu should be used instead 2014-12-05 17:25:42 -08:00
Harshavardhana f68baeaab2 Merge pull request #90 from harshavardhana/pr_out_provide_joinfileswithprefix_case_with_gochannel
Provide JoinFilesWithPrefix case with Gochannel
2014-12-05 17:22:19 -08:00
Harshavardhana 9a5003dd25 Provide JoinFilesWithPrefix case with Gochannel 2014-12-05 16:02:46 -08:00
Harshavardhana d34d86c050 Merge pull request #87 from fkautz/pr_out_adding_more_unit_tests_to_strbyteconv_fixes_69
Adding more unit tests to strbyteconv. Fixes #69.
2014-12-04 23:11:08 -08:00
Frederick F. Kautz IV 5abcaf4d58 Adding more unit tests to strbyteconv. Fixes #69. 2014-12-04 23:03:21 -08:00
Harshavardhana 1b283441a4 Merge pull request #80 from harshavardhana/pr_out_add_git_and_mercurial_as_dependencies 2014-12-04 01:57:15 -08:00
Harshavardhana c3fca84f25 Add git and mercurial as dependencies 2014-12-04 01:56:08 -08:00
Harshavardhana 69ae3b3310 Merge pull request #79 from abperiasamy/pr_out_updated_developers_guide 2014-12-04 01:22:34 -08:00
Anand Babu (AB) Periasamy 0464e2f8cc updated developers guide 2014-12-04 01:18:26 -08:00
Harshavardhana 0a42a82df5 Merge pull request #78 from harshavardhana/pr_out_avoid_linux_to_be_hard_dependency_on_build 2014-12-04 01:13:28 -08:00
Harshavardhana 0c4dd8f5fa Avoid linux to be hard dependency on build 2014-12-04 01:12:10 -08:00
Harshavardhana a2e0a41f7e Merge pull request #76 from harshavardhana/pr_out_fast_crc_implementations_ported_from_intel_s_efforts 2014-12-03 01:35:54 -08:00
Harshavardhana fb34c5290c Fast CRC implementations ported from Intel's efforts
Provides fast CRC32C with PCLMULQDQ instructions in Golang

The white papers on CRC32C calculations with PCLMULQDQ instruction can be
  downloaded from:

http://www.intel.com/content/dam/www/public/us/en/documents/white-papers/crc-iscsi-polynomial-crc32-instruction-paper.pdf
http://www.intel.com/content/dam/www/public/us/en/documents/white-papers/fast-crc-computation-paper.pdf
2014-12-03 01:20:31 -08:00
Harshavardhana d31050803b Merge pull request #77 from harshavardhana/pr_out_change_instructions_for_go_installation_to_directory_inside_users_home_dir
Change instructions for go installation to directory inside users ${HOME} dir
2014-12-02 13:15:34 -08:00
Harshavardhana 18234c0666 Change instructions for go installation to directory inside users ${HOME} dir 2014-12-02 13:14:55 -08:00
Harshavardhana 39beabbbb6 Merge pull request #74 from fkautz/pr_out_adding_chunking_by_block_to_erasure_demo_via_block_size_parameter
Adding chunking by block to erasure-demo via --block-size parameter
2014-12-01 21:45:17 -08:00
Frederick F. Kautz IV dac14cd7a1 Merge pull request #75 from harshavardhana/pr_out_remove_unnecessary_c_code_and_use_everything_from_golang 2014-12-01 21:10:21 -08:00
Harshavardhana 3a6cac8ada Remove unnecessary C code and use everything from Golang 2014-12-01 21:07:55 -08:00
Frederick F. Kautz IV 46b08681a4 Adding chunking by block to erasure-demo via --block-size parameter 2014-12-01 21:06:36 -08:00
Harshavardhana 59c1197f47 Merge pull request #73 from fkautz/pr_out_adding_file_splitter_and_streaming_byte_chunker
Adding file splitter and streaming byte chunker
2014-12-01 20:47:25 -08:00
Frederick F. Kautz IV 89587e1391 Adding file splitter and streaming byte chunker 2014-12-01 18:51:47 -08:00
Harshavardhana 3b010d4813 Merge pull request #72 from harshavardhana/pr_out_build_restructure_top_level
Build restructure top level
2014-12-01 14:46:34 -08:00
Harshavardhana 6e4583ef1d Build restructure top level 2014-12-01 14:45:50 -08:00
Harshavardhana c2b52590ee Merge pull request #70 from harshavardhana/pr_out_enable_cover_profile_and_test_for_split_and_strbyteconv
Enable cover profile and test for split and strbyteconv
2014-12-01 10:58:34 -08:00
Harshavardhana a4676211cf Enable cover profile and test for split and strbyteconv 2014-12-01 10:40:50 -08:00
Harshavardhana c002f90e0b Merge pull request #28 from harshavardhana/pr_out_enable_way_to_split_files_at_input_bytes
Enable way to split files at input bytes
2014-12-01 01:03:17 -08:00
Harshavardhana 3a18454f5e Merge pull request #67 from fkautz/pr_out_refactoring_input_parsing_for_erasure_demo
Refactoring input parsing for erasure-demo
2014-12-01 01:03:05 -08:00
Harshavardhana 67e9a27b34 Enable way to split files at input bytes 2014-12-01 01:00:56 -08:00
Harshavardhana 2bea464c32 Merge pull request #68 from harshavardhana/pr_out_strbyteconv_stringtobytes_bytestostring_
strbyteconv -    - StringToBytes()    - BytesToString()
2014-12-01 00:37:04 -08:00
Harshavardhana ad5b4b52e8 strbyteconv -
- StringToBytes()
   - BytesToString()
2014-11-30 23:06:47 -08:00
Frederick F. Kautz IV 323a7b8492 Refactoring input parsing for erasure-demo 2014-11-30 22:18:23 -08:00
Harshavardhana 2a2e67cb50 Merge pull request #66 from fkautz/pr_out_fixing_build_where_cover_and_godep_cause_a_break_when_system_is_offline
Fixing build where cover and godep cause a break when system is offline
2014-11-30 21:59:21 -08:00