Commit Graph

9253 Commits

Author SHA1 Message Date
Frederick F. Kautz IV
c2290dd1d4 Updates for cli and context
cli adds new global options. We'll be able to specify flags that
apply to multiple options now.

context update fixes parallelWriter function, actually writes a value out now.
2014-12-11 09:19:20 -08:00
Harshavardhana
e107c3204d Merge pull request #107 from harshavardhana/pr_out_storage_erasure_encoding_info_at_storageentry_and_also_store_crc_at_blockheader 2014-12-11 02:22:30 -08:00
Harshavardhana
803c35ee90 Storage erasure encoding info at StorageEntry and also store CRC at blockheader 2014-12-11 02:11:48 -08:00
Harshavardhana
23a3609a64 Merge pull request #106 from harshavardhana/pr_out_ignore_io_eof_for_gob_decoding 2014-12-11 01:47:49 -08:00
Harshavardhana
c0724b38b9 Ignore io.EOF for gob decoding 2014-12-11 01:45:58 -08:00
Harshavardhana
063d668133 Merge pull request #105 from fkautz/pr_out_erasure_encoded_demo 2014-12-11 00:43:02 -08:00
Frederick F. Kautz IV
568ca4e1bc erasure encoded demo 2014-12-11 00:32:32 -08:00
Frederick F. Kautz IV
e5995f0193 Merge pull request #104 from fkautz/pr_out_erasure_demo_put_now_takes_1_or_2_arguments 2014-12-11 00:20:45 -08:00
Harshavardhana
d442988912 Merge pull request #103 from fkautz/pr_out_objectstorage_refactor_to_take_io_reader_instead_of_byte
ObjectStorage refactor to take io.Reader instead of []byte
2014-12-10 21:40:25 -08:00
Frederick F. Kautz IV
a1040ae657 erasure-demo put now takes 1 or 2 arguments 2014-12-10 20:59:49 -08:00
Frederick F. Kautz IV
19da9760b3 ObjectStorage refactor to take io.Reader instead of []byte 2014-12-10 20:57:59 -08:00
Frederick F. Kautz IV
c7b4e14f64 Merge pull request #102 from fkautz/pr_out_adding_list_path_to_object_storage_definition_and_accompanying_definition_to_fs 2014-12-10 19:19:26 -08:00
Frederick F. Kautz IV
cae9b288b0 Adding List(path) to object storage definition and accompanying definition to fs 2014-12-10 19:15:03 -08:00
Harshavardhana
9142de1dd5 Merge pull request #101 from fkautz/pr_out_fixing_build
Fixing build
2014-12-10 19:00:34 -08:00
Frederick F. Kautz IV
e670608226 Fixing build 2014-12-10 17:43:16 -08:00
Harshavardhana
9dd3dd8b53 Merge pull request #100 from harshavardhana/pr_out_add_crc32c_checksum_support_for_erasure_chunks
Add crc32c checksum support for erasure chunks
2014-12-10 11:53:23 -08:00
Harshavardhana
01c0e45b5c Add crc32c checksum support for erasure chunks
Now upon conditions

 - if checksum '.cksum' file is missing
 - or corrupted chunk (mismatching chunk)
 - or missing chunk entirely

Decoding works consistently
2014-12-10 00:58:15 -08:00
Harshavardhana
72ddb345a1 Merge pull request #99 from harshavardhana/pr_out_make_get_put_more_robust 2014-12-09 03:37:08 -08:00
Harshavardhana
f6a5b61b00 Make get/put more robust
- init folder if not present already in 'get()' call
  - put.go was referencing wrong args - fix it
  - if no OBJECTNAME is specified during 'get' - grab a
    list of files under "~/.minio"
  - Provide new API GetList() for ObjectStorage struct
2014-12-09 03:32:31 -08:00
Harshavardhana
3fe2c2af01 Merge pull request #98 from harshavardhana/pr_out_add_sha1_avx2_supported_fast_implementation_with_some_crc32c_cleanup 2014-12-09 01:01:05 -08:00
Harshavardhana
43680cd7b6 Add sha1 - AVX2 supported fast implementation, with some crc32c cleanup 2014-12-09 01:00:02 -08:00
Frederick F. Kautz IV
93e57118f8 Merge pull request #97 from fkautz/pr_out_adding_append_storage_driver 2014-12-08 23:37:54 -08:00
Frederick F. Kautz IV
1d7b3866cb Adding append storage driver 2014-12-08 23:37:25 -08:00
Frederick F. Kautz IV
76f21c1cfc Merge pull request #96 from fkautz/pr_out_removing_printlns_from_test_code 2014-12-08 20:40:08 -08:00
Frederick F. Kautz IV
af8eb7a55d Removing printlns from test code 2014-12-08 20:39:30 -08:00
Frederick F. Kautz IV
a40d834a42 Merge pull request #95 from fkautz/pr_out_adding_extremely_simple_fs_based_get_and_put_in_erasure_demo 2014-12-08 20:23:02 -08:00
Frederick F. Kautz IV
d1ebf935da Adding extremely simple fs based get and put in erasure-demo 2014-12-08 20:20:25 -08:00
Frederick F. Kautz IV
c2917f0d64 Merge pull request #94 from fkautz/pr_out_refactoring_file_storage_driver_to_fsstorage 2014-12-08 15:27:37 -08:00
Frederick F. Kautz IV
0cf80e075e Refactoring file storage driver to fsstorage 2014-12-08 14:29:25 -08:00
Frederick F. Kautz IV
8a5b8ac90f Merge pull request #93 from fkautz/pr_out_cpu_tests_now_actually_test_properly_on_linux_and_return_bool_instead_of_int 2014-12-08 14:25:11 -08:00
Frederick F. Kautz IV
e71998c584 CPU tests now actually test properly on linux and return bool instead of int 2014-12-08 13:10:18 -08:00
Harshavardhana
5d4c02ad4a Merge pull request #92 from harshavardhana/pr_out_fixes_71_some_crazy_races_inside_erasure_and_simplify_the_code 2014-12-07 01:34:23 -08:00
Harshavardhana
7314b5e37d Fixes #71 - some crazy races inside erasure and simplify the code
- This change also brings in changing 'unsigned char' to 'uint8_t'
for brevity
2014-12-07 01:33:49 -08:00
Harshavardhana
d57700922d Merge pull request #91 from harshavardhana/pr_out_remove_cpufeatures_c_code_since_pkgs_crc32c_cpu_should_be_used_instead
Remove cpufeatures C code, since pkgs/crc32c/cpu should be used instead
2014-12-05 17:26:03 -08:00
Harshavardhana
e414d5f67b Remove cpufeatures C code, since pkgs/crc32c/cpu should be used instead 2014-12-05 17:25:42 -08:00
Harshavardhana
f68baeaab2 Merge pull request #90 from harshavardhana/pr_out_provide_joinfileswithprefix_case_with_gochannel
Provide JoinFilesWithPrefix case with Gochannel
2014-12-05 17:22:19 -08:00
Harshavardhana
9a5003dd25 Provide JoinFilesWithPrefix case with Gochannel 2014-12-05 16:02:46 -08:00
Harshavardhana
d34d86c050 Merge pull request #87 from fkautz/pr_out_adding_more_unit_tests_to_strbyteconv_fixes_69
Adding more unit tests to strbyteconv. Fixes #69.
2014-12-04 23:11:08 -08:00
Frederick F. Kautz IV
5abcaf4d58 Adding more unit tests to strbyteconv. Fixes #69. 2014-12-04 23:03:21 -08:00
Harshavardhana
1b283441a4 Merge pull request #80 from harshavardhana/pr_out_add_git_and_mercurial_as_dependencies 2014-12-04 01:57:15 -08:00
Harshavardhana
c3fca84f25 Add git and mercurial as dependencies 2014-12-04 01:56:08 -08:00
Harshavardhana
69ae3b3310 Merge pull request #79 from abperiasamy/pr_out_updated_developers_guide 2014-12-04 01:22:34 -08:00
Anand Babu (AB) Periasamy
0464e2f8cc updated developers guide 2014-12-04 01:18:26 -08:00
Harshavardhana
0a42a82df5 Merge pull request #78 from harshavardhana/pr_out_avoid_linux_to_be_hard_dependency_on_build 2014-12-04 01:13:28 -08:00
Harshavardhana
0c4dd8f5fa Avoid linux to be hard dependency on build 2014-12-04 01:12:10 -08:00
Harshavardhana
a2e0a41f7e Merge pull request #76 from harshavardhana/pr_out_fast_crc_implementations_ported_from_intel_s_efforts 2014-12-03 01:35:54 -08:00
Harshavardhana
fb34c5290c Fast CRC implementations ported from Intel's efforts
Provides fast CRC32C with PCLMULQDQ instructions in Golang

The white papers on CRC32C calculations with PCLMULQDQ instruction can be
  downloaded from:

http://www.intel.com/content/dam/www/public/us/en/documents/white-papers/crc-iscsi-polynomial-crc32-instruction-paper.pdf
http://www.intel.com/content/dam/www/public/us/en/documents/white-papers/fast-crc-computation-paper.pdf
2014-12-03 01:20:31 -08:00
Harshavardhana
d31050803b Merge pull request #77 from harshavardhana/pr_out_change_instructions_for_go_installation_to_directory_inside_users_home_dir
Change instructions for go installation to directory inside users ${HOME} dir
2014-12-02 13:15:34 -08:00
Harshavardhana
18234c0666 Change instructions for go installation to directory inside users ${HOME} dir 2014-12-02 13:14:55 -08:00
Harshavardhana
39beabbbb6 Merge pull request #74 from fkautz/pr_out_adding_chunking_by_block_to_erasure_demo_via_block_size_parameter
Adding chunking by block to erasure-demo via --block-size parameter
2014-12-01 21:45:17 -08:00