Commit Graph

10951 Commits

Author SHA1 Message Date
Harshavardhana
803c35ee90 Storage erasure encoding info at StorageEntry and also store CRC at blockheader 2014-12-11 02:11:48 -08:00
Harshavardhana
23a3609a64 Merge pull request #106 from harshavardhana/pr_out_ignore_io_eof_for_gob_decoding 2014-12-11 01:47:49 -08:00
Harshavardhana
c0724b38b9 Ignore io.EOF for gob decoding 2014-12-11 01:45:58 -08:00
Harshavardhana
063d668133 Merge pull request #105 from fkautz/pr_out_erasure_encoded_demo 2014-12-11 00:43:02 -08:00
Frederick F. Kautz IV
568ca4e1bc erasure encoded demo 2014-12-11 00:32:32 -08:00
Frederick F. Kautz IV
e5995f0193 Merge pull request #104 from fkautz/pr_out_erasure_demo_put_now_takes_1_or_2_arguments 2014-12-11 00:20:45 -08:00
Harshavardhana
d442988912 Merge pull request #103 from fkautz/pr_out_objectstorage_refactor_to_take_io_reader_instead_of_byte
ObjectStorage refactor to take io.Reader instead of []byte
2014-12-10 21:40:25 -08:00
Frederick F. Kautz IV
a1040ae657 erasure-demo put now takes 1 or 2 arguments 2014-12-10 20:59:49 -08:00
Frederick F. Kautz IV
19da9760b3 ObjectStorage refactor to take io.Reader instead of []byte 2014-12-10 20:57:59 -08:00
Frederick F. Kautz IV
c7b4e14f64 Merge pull request #102 from fkautz/pr_out_adding_list_path_to_object_storage_definition_and_accompanying_definition_to_fs 2014-12-10 19:19:26 -08:00
Frederick F. Kautz IV
cae9b288b0 Adding List(path) to object storage definition and accompanying definition to fs 2014-12-10 19:15:03 -08:00
Harshavardhana
9142de1dd5 Merge pull request #101 from fkautz/pr_out_fixing_build
Fixing build
2014-12-10 19:00:34 -08:00
Frederick F. Kautz IV
e670608226 Fixing build 2014-12-10 17:43:16 -08:00
Harshavardhana
9dd3dd8b53 Merge pull request #100 from harshavardhana/pr_out_add_crc32c_checksum_support_for_erasure_chunks
Add crc32c checksum support for erasure chunks
2014-12-10 11:53:23 -08:00
Harshavardhana
01c0e45b5c Add crc32c checksum support for erasure chunks
Now upon conditions

 - if checksum '.cksum' file is missing
 - or corrupted chunk (mismatching chunk)
 - or missing chunk entirely

Decoding works consistently
2014-12-10 00:58:15 -08:00
Harshavardhana
72ddb345a1 Merge pull request #99 from harshavardhana/pr_out_make_get_put_more_robust 2014-12-09 03:37:08 -08:00
Harshavardhana
f6a5b61b00 Make get/put more robust
- init folder if not present already in 'get()' call
  - put.go was referencing wrong args - fix it
  - if no OBJECTNAME is specified during 'get' - grab a
    list of files under "~/.minio"
  - Provide new API GetList() for ObjectStorage struct
2014-12-09 03:32:31 -08:00
Harshavardhana
3fe2c2af01 Merge pull request #98 from harshavardhana/pr_out_add_sha1_avx2_supported_fast_implementation_with_some_crc32c_cleanup 2014-12-09 01:01:05 -08:00
Harshavardhana
43680cd7b6 Add sha1 - AVX2 supported fast implementation, with some crc32c cleanup 2014-12-09 01:00:02 -08:00
Frederick F. Kautz IV
93e57118f8 Merge pull request #97 from fkautz/pr_out_adding_append_storage_driver 2014-12-08 23:37:54 -08:00
Frederick F. Kautz IV
1d7b3866cb Adding append storage driver 2014-12-08 23:37:25 -08:00
Frederick F. Kautz IV
76f21c1cfc Merge pull request #96 from fkautz/pr_out_removing_printlns_from_test_code 2014-12-08 20:40:08 -08:00
Frederick F. Kautz IV
af8eb7a55d Removing printlns from test code 2014-12-08 20:39:30 -08:00
Frederick F. Kautz IV
a40d834a42 Merge pull request #95 from fkautz/pr_out_adding_extremely_simple_fs_based_get_and_put_in_erasure_demo 2014-12-08 20:23:02 -08:00
Frederick F. Kautz IV
d1ebf935da Adding extremely simple fs based get and put in erasure-demo 2014-12-08 20:20:25 -08:00
Frederick F. Kautz IV
c2917f0d64 Merge pull request #94 from fkautz/pr_out_refactoring_file_storage_driver_to_fsstorage 2014-12-08 15:27:37 -08:00
Frederick F. Kautz IV
0cf80e075e Refactoring file storage driver to fsstorage 2014-12-08 14:29:25 -08:00
Frederick F. Kautz IV
8a5b8ac90f Merge pull request #93 from fkautz/pr_out_cpu_tests_now_actually_test_properly_on_linux_and_return_bool_instead_of_int 2014-12-08 14:25:11 -08:00
Frederick F. Kautz IV
e71998c584 CPU tests now actually test properly on linux and return bool instead of int 2014-12-08 13:10:18 -08:00
Harshavardhana
5d4c02ad4a Merge pull request #92 from harshavardhana/pr_out_fixes_71_some_crazy_races_inside_erasure_and_simplify_the_code 2014-12-07 01:34:23 -08:00
Harshavardhana
7314b5e37d Fixes #71 - some crazy races inside erasure and simplify the code
- This change also brings in changing 'unsigned char' to 'uint8_t'
for brevity
2014-12-07 01:33:49 -08:00
Harshavardhana
d57700922d Merge pull request #91 from harshavardhana/pr_out_remove_cpufeatures_c_code_since_pkgs_crc32c_cpu_should_be_used_instead
Remove cpufeatures C code, since pkgs/crc32c/cpu should be used instead
2014-12-05 17:26:03 -08:00
Harshavardhana
e414d5f67b Remove cpufeatures C code, since pkgs/crc32c/cpu should be used instead 2014-12-05 17:25:42 -08:00
Harshavardhana
f68baeaab2 Merge pull request #90 from harshavardhana/pr_out_provide_joinfileswithprefix_case_with_gochannel
Provide JoinFilesWithPrefix case with Gochannel
2014-12-05 17:22:19 -08:00
Harshavardhana
9a5003dd25 Provide JoinFilesWithPrefix case with Gochannel 2014-12-05 16:02:46 -08:00
Harshavardhana
d34d86c050 Merge pull request #87 from fkautz/pr_out_adding_more_unit_tests_to_strbyteconv_fixes_69
Adding more unit tests to strbyteconv. Fixes #69.
2014-12-04 23:11:08 -08:00
Frederick F. Kautz IV
5abcaf4d58 Adding more unit tests to strbyteconv. Fixes #69. 2014-12-04 23:03:21 -08:00
Harshavardhana
1b283441a4 Merge pull request #80 from harshavardhana/pr_out_add_git_and_mercurial_as_dependencies 2014-12-04 01:57:15 -08:00
Harshavardhana
c3fca84f25 Add git and mercurial as dependencies 2014-12-04 01:56:08 -08:00
Harshavardhana
69ae3b3310 Merge pull request #79 from abperiasamy/pr_out_updated_developers_guide 2014-12-04 01:22:34 -08:00
Anand Babu (AB) Periasamy
0464e2f8cc updated developers guide 2014-12-04 01:18:26 -08:00
Harshavardhana
0a42a82df5 Merge pull request #78 from harshavardhana/pr_out_avoid_linux_to_be_hard_dependency_on_build 2014-12-04 01:13:28 -08:00
Harshavardhana
0c4dd8f5fa Avoid linux to be hard dependency on build 2014-12-04 01:12:10 -08:00
Harshavardhana
a2e0a41f7e Merge pull request #76 from harshavardhana/pr_out_fast_crc_implementations_ported_from_intel_s_efforts 2014-12-03 01:35:54 -08:00
Harshavardhana
fb34c5290c Fast CRC implementations ported from Intel's efforts
Provides fast CRC32C with PCLMULQDQ instructions in Golang

The white papers on CRC32C calculations with PCLMULQDQ instruction can be
  downloaded from:

http://www.intel.com/content/dam/www/public/us/en/documents/white-papers/crc-iscsi-polynomial-crc32-instruction-paper.pdf
http://www.intel.com/content/dam/www/public/us/en/documents/white-papers/fast-crc-computation-paper.pdf
2014-12-03 01:20:31 -08:00
Harshavardhana
d31050803b Merge pull request #77 from harshavardhana/pr_out_change_instructions_for_go_installation_to_directory_inside_users_home_dir
Change instructions for go installation to directory inside users ${HOME} dir
2014-12-02 13:15:34 -08:00
Harshavardhana
18234c0666 Change instructions for go installation to directory inside users ${HOME} dir 2014-12-02 13:14:55 -08:00
Harshavardhana
39beabbbb6 Merge pull request #74 from fkautz/pr_out_adding_chunking_by_block_to_erasure_demo_via_block_size_parameter
Adding chunking by block to erasure-demo via --block-size parameter
2014-12-01 21:45:17 -08:00
Frederick F. Kautz IV
dac14cd7a1 Merge pull request #75 from harshavardhana/pr_out_remove_unnecessary_c_code_and_use_everything_from_golang 2014-12-01 21:10:21 -08:00
Harshavardhana
3a6cac8ada Remove unnecessary C code and use everything from Golang 2014-12-01 21:07:55 -08:00