Commit Graph

11530 Commits

Author SHA1 Message Date
Harshavardhana 0148237e22 Refactor checksum code, and add Objectname hashing per uploaded objects
Add placeholder sha256, sha512 implementations from Intel with AVX, AVX2
and SSE4.1 extensions - with some boilerplate Go code.
2014-12-13 21:42:23 -08:00
Harshavardhana b87839a1c8 Merge pull request #120 from harshavardhana/pr_out_add_license_header 2014-12-13 14:27:59 -08:00
Harshavardhana 99478c524a Add license header 2014-12-13 14:27:06 -08:00
Harshavardhana 6c616f9bd0 Merge pull request #119 from harshavardhana/pr_out_update_documentation_for_crc32c_cpu 2014-12-13 01:39:31 -08:00
Harshavardhana 71c774f99b Update documentation for crc32c, cpu 2014-12-13 01:39:14 -08:00
Harshavardhana 643a293d94 Merge pull request #118 from harshavardhana/pr_out_provide_new_pkgs_utils_for_generic_functions 2014-12-13 01:15:02 -08:00
Harshavardhana 20d46a1977 Provide new pkgs/utils for generic functions 2014-12-13 01:14:36 -08:00
Harshavardhana fb1537248b Merge pull request #117 from harshavardhana/pr_out_how_to_join_community 2014-12-12 23:57:09 -08:00
Harshavardhana 41e9084770 How to Join Community 2014-12-12 23:52:25 -08:00
Harshavardhana 73063deaa6 Merge pull request #116 from harshavardhana/pr_out_fsstorage_should_use_appendstorage_with_offsets 2014-12-12 17:18:53 -08:00
Harshavardhana fc50291cac fsStorage should use appendStorage with offsets
- also takes in 'blockSize' cli option.
 - robustness fixes
2014-12-12 17:18:16 -08:00
Harshavardhana b570b486de Merge pull request #115 from harshavardhana/pr_out_add_list_object_support_for_erasure_and_append_storage_drivers 2014-12-12 02:56:13 -08:00
Harshavardhana 0a0e1111cd Add list() object support for erasure and append storage drivers
- Reply back objects with their protectionlevel and md5sum
  - // TODO hash value
  - Calculate md5sum after "storeBlocks()", to make sure data is committed
2014-12-12 02:52:37 -08:00
Harshavardhana 267aa87ad7 Merge pull request #114 from harshavardhana/pr_out_provide_new_cli_options_for_put_command
Provide new cli options for 'put' command
2014-12-11 22:57:56 -08:00
Harshavardhana f71b323e32 Provide new cli options for 'put' command
- --protection-level
  - --block-size
2014-12-11 22:56:12 -08:00
Harshavardhana 429836593b Merge pull request #113 from harshavardhana/pr_out_build_fixes_after_moving_to_go1_4 2014-12-11 20:59:13 -08:00
Harshavardhana 235949eafe Build fixes after moving to go1.4 2014-12-11 20:57:00 -08:00
Harshavardhana 0bd2b8516e Build fix, fixing without a PR 2014-12-11 20:44:06 -08:00
Harshavardhana 4a7d9a58f4 Merge pull request #112 from harshavardhana/pr_out_calculate_md5sum_for_each_object 2014-12-11 20:36:42 -08:00
Harshavardhana 91d1c938a8 Merge pull request #111 from harshavardhana/pr_out_implement_simple_encoded_storage_in_gateway 2014-12-11 20:36:31 -08:00
Harshavardhana e15edbf393 Implement simple encoded storage in gateway 2014-12-11 20:29:40 -08:00
Harshavardhana a0ec21975d Calculate md5sum for each object 2014-12-11 19:43:58 -08:00
Harshavardhana a56710e318 Merge pull request #110 from harshavardhana/pr_out_use_objectname_with_basename_for_the_filepath 2014-12-11 15:35:53 -08:00
Harshavardhana dddc555491 Use objectName with basename for the filepath 2014-12-11 15:35:25 -08:00
Harshavardhana 94e11db3a8 Merge pull request #109 from harshavardhana/pr_out_add_crc_checks_for_object_get_ 2014-12-11 15:10:51 -08:00
Harshavardhana 1166557da1 Add crc checks for Object Get() 2014-12-11 15:06:29 -08:00
Frederick F. Kautz IV b3af234ee2 Merge pull request #108 from fkautz/pr_out_updates_for_cli_and_context 2014-12-11 09:28:33 -08:00
Frederick F. Kautz IV c2290dd1d4 Updates for cli and context
cli adds new global options. We'll be able to specify flags that
apply to multiple options now.

context update fixes parallelWriter function, actually writes a value out now.
2014-12-11 09:19:20 -08:00
Harshavardhana e107c3204d Merge pull request #107 from harshavardhana/pr_out_storage_erasure_encoding_info_at_storageentry_and_also_store_crc_at_blockheader 2014-12-11 02:22:30 -08:00
Harshavardhana 803c35ee90 Storage erasure encoding info at StorageEntry and also store CRC at blockheader 2014-12-11 02:11:48 -08:00
Harshavardhana 23a3609a64 Merge pull request #106 from harshavardhana/pr_out_ignore_io_eof_for_gob_decoding 2014-12-11 01:47:49 -08:00
Harshavardhana c0724b38b9 Ignore io.EOF for gob decoding 2014-12-11 01:45:58 -08:00
Harshavardhana 063d668133 Merge pull request #105 from fkautz/pr_out_erasure_encoded_demo 2014-12-11 00:43:02 -08:00
Frederick F. Kautz IV 568ca4e1bc erasure encoded demo 2014-12-11 00:32:32 -08:00
Frederick F. Kautz IV e5995f0193 Merge pull request #104 from fkautz/pr_out_erasure_demo_put_now_takes_1_or_2_arguments 2014-12-11 00:20:45 -08:00
Harshavardhana d442988912 Merge pull request #103 from fkautz/pr_out_objectstorage_refactor_to_take_io_reader_instead_of_byte
ObjectStorage refactor to take io.Reader instead of []byte
2014-12-10 21:40:25 -08:00
Frederick F. Kautz IV a1040ae657 erasure-demo put now takes 1 or 2 arguments 2014-12-10 20:59:49 -08:00
Frederick F. Kautz IV 19da9760b3 ObjectStorage refactor to take io.Reader instead of []byte 2014-12-10 20:57:59 -08:00
Frederick F. Kautz IV c7b4e14f64 Merge pull request #102 from fkautz/pr_out_adding_list_path_to_object_storage_definition_and_accompanying_definition_to_fs 2014-12-10 19:19:26 -08:00
Frederick F. Kautz IV cae9b288b0 Adding List(path) to object storage definition and accompanying definition to fs 2014-12-10 19:15:03 -08:00
Harshavardhana 9142de1dd5 Merge pull request #101 from fkautz/pr_out_fixing_build
Fixing build
2014-12-10 19:00:34 -08:00
Frederick F. Kautz IV e670608226 Fixing build 2014-12-10 17:43:16 -08:00
Harshavardhana 9dd3dd8b53 Merge pull request #100 from harshavardhana/pr_out_add_crc32c_checksum_support_for_erasure_chunks
Add crc32c checksum support for erasure chunks
2014-12-10 11:53:23 -08:00
Harshavardhana 01c0e45b5c Add crc32c checksum support for erasure chunks
Now upon conditions

 - if checksum '.cksum' file is missing
 - or corrupted chunk (mismatching chunk)
 - or missing chunk entirely

Decoding works consistently
2014-12-10 00:58:15 -08:00
Harshavardhana 72ddb345a1 Merge pull request #99 from harshavardhana/pr_out_make_get_put_more_robust 2014-12-09 03:37:08 -08:00
Harshavardhana f6a5b61b00 Make get/put more robust
- init folder if not present already in 'get()' call
  - put.go was referencing wrong args - fix it
  - if no OBJECTNAME is specified during 'get' - grab a
    list of files under "~/.minio"
  - Provide new API GetList() for ObjectStorage struct
2014-12-09 03:32:31 -08:00
Harshavardhana 3fe2c2af01 Merge pull request #98 from harshavardhana/pr_out_add_sha1_avx2_supported_fast_implementation_with_some_crc32c_cleanup 2014-12-09 01:01:05 -08:00
Harshavardhana 43680cd7b6 Add sha1 - AVX2 supported fast implementation, with some crc32c cleanup 2014-12-09 01:00:02 -08:00
Frederick F. Kautz IV 93e57118f8 Merge pull request #97 from fkautz/pr_out_adding_append_storage_driver 2014-12-08 23:37:54 -08:00
Frederick F. Kautz IV 1d7b3866cb Adding append storage driver 2014-12-08 23:37:25 -08:00