MyFavMirrors/minio
1
0
Fork 0
mirror of https://github.com/minio/minio.git synced 2025-01-09 14:03:22 -05:00
Code Issues Packages Projects Releases Wiki Activity
226 Commits 4 Branches 506 Tags 190 MiB
0148237e22
Commit Graph

50 Commits

Author SHA1 Message Date
Harshavardhana
0148237e22 Refactor checksum code, and add Objectname hashing per uploaded objects 
Add placeholder sha256, sha512 implementations from Intel with AVX, AVX2
and SSE4.1 extensions - with some boilerplate Go code.
 2014-12-13 21:42:23 -08:00
Harshavardhana
20d46a1977 Provide new pkgs/utils for generic functions 2014-12-13 01:14:36 -08:00
Harshavardhana
fc50291cac fsStorage should use appendStorage with offsets 
- also takes in 'blockSize' cli option.
 - robustness fixes
 2014-12-12 17:18:16 -08:00
Harshavardhana
0a0e1111cd Add list() object support for erasure and append storage drivers 
- Reply back objects with their protectionlevel and md5sum
  - // TODO hash value
  - Calculate md5sum after "storeBlocks()", to make sure data is committed
 2014-12-12 02:52:37 -08:00
Harshavardhana
235949eafe Build fixes after moving to go1.4 2014-12-11 20:57:00 -08:00
Harshavardhana
0bd2b8516e Build fix, fixing without a PR 2014-12-11 20:44:06 -08:00
Harshavardhana
4a7d9a58f4 Merge pull request #112 from harshavardhana/pr_out_calculate_md5sum_for_each_object 2014-12-11 20:36:42 -08:00
Harshavardhana
e15edbf393 Implement simple encoded storage in gateway 2014-12-11 20:29:40 -08:00
Harshavardhana
a0ec21975d Calculate md5sum for each object 2014-12-11 19:43:58 -08:00
Harshavardhana
1166557da1 Add crc checks for Object Get() 2014-12-11 15:06:29 -08:00
Harshavardhana
803c35ee90 Storage erasure encoding info at StorageEntry and also store CRC at blockheader 2014-12-11 02:11:48 -08:00
Harshavardhana
c0724b38b9 Ignore io.EOF for gob decoding 2014-12-11 01:45:58 -08:00
Frederick F. Kautz IV
568ca4e1bc erasure encoded demo 2014-12-11 00:32:32 -08:00
Frederick F. Kautz IV
19da9760b3 ObjectStorage refactor to take io.Reader instead of []byte 2014-12-10 20:57:59 -08:00
Frederick F. Kautz IV
cae9b288b0 Adding List(path) to object storage definition and accompanying definition to fs 2014-12-10 19:15:03 -08:00
Frederick F. Kautz IV
e670608226 Fixing build 2014-12-10 17:43:16 -08:00
Harshavardhana
f6a5b61b00 Make get/put more robust 
- init folder if not present already in 'get()' call
  - put.go was referencing wrong args - fix it
  - if no OBJECTNAME is specified during 'get' - grab a
    list of files under "~/.minio"
  - Provide new API GetList() for ObjectStorage struct
 2014-12-09 03:32:31 -08:00
Harshavardhana
43680cd7b6 Add sha1 - AVX2 supported fast implementation, with some crc32c cleanup 2014-12-09 01:00:02 -08:00
Frederick F. Kautz IV
1d7b3866cb Adding append storage driver 2014-12-08 23:37:25 -08:00
Frederick F. Kautz IV
af8eb7a55d Removing printlns from test code 2014-12-08 20:39:30 -08:00
Frederick F. Kautz IV
0cf80e075e Refactoring file storage driver to fsstorage 2014-12-08 14:29:25 -08:00
Frederick F. Kautz IV
e71998c584 CPU tests now actually test properly on linux and return bool instead of int 2014-12-08 13:10:18 -08:00
Harshavardhana
7314b5e37d Fixes #71 - some crazy races inside erasure and simplify the code 
- This change also brings in changing 'unsigned char' to 'uint8_t'
for brevity
 2014-12-07 01:33:49 -08:00
Harshavardhana
e414d5f67b Remove cpufeatures C code, since pkgs/crc32c/cpu should be used instead 2014-12-05 17:25:42 -08:00
Harshavardhana
f68baeaab2 Merge pull request #90 from harshavardhana/pr_out_provide_joinfileswithprefix_case_with_gochannel 
Provide JoinFilesWithPrefix case with Gochannel
 2014-12-05 17:22:19 -08:00
Harshavardhana
9a5003dd25 Provide JoinFilesWithPrefix case with Gochannel 2014-12-05 16:02:46 -08:00
Frederick F. Kautz IV
5abcaf4d58 Adding more unit tests to strbyteconv. Fixes #69. 2014-12-04 23:03:21 -08:00
Harshavardhana
0c4dd8f5fa Avoid linux to be hard dependency on build 2014-12-04 01:12:10 -08:00
Harshavardhana
fb34c5290c Fast CRC implementations ported from Intel's efforts 
Provides fast CRC32C with PCLMULQDQ instructions in Golang

The white papers on CRC32C calculations with PCLMULQDQ instruction can be
  downloaded from:

http://www.intel.com/content/dam/www/public/us/en/documents/white-papers/crc-iscsi-polynomial-crc32-instruction-paper.pdf
http://www.intel.com/content/dam/www/public/us/en/documents/white-papers/fast-crc-computation-paper.pdf
 2014-12-03 01:20:31 -08:00
Harshavardhana
3a6cac8ada Remove unnecessary C code and use everything from Golang 2014-12-01 21:07:55 -08:00
Harshavardhana
59c1197f47 Merge pull request #73 from fkautz/pr_out_adding_file_splitter_and_streaming_byte_chunker 
Adding file splitter and streaming byte chunker
 2014-12-01 20:47:25 -08:00
Frederick F. Kautz IV
89587e1391 Adding file splitter and streaming byte chunker 2014-12-01 18:51:47 -08:00
Harshavardhana
6e4583ef1d Build restructure top level 2014-12-01 14:45:50 -08:00
Harshavardhana
a4676211cf Enable cover profile and test for split and strbyteconv 2014-12-01 10:40:50 -08:00
Harshavardhana
67e9a27b34 Enable way to split files at input bytes 2014-12-01 01:00:56 -08:00
Harshavardhana
ad5b4b52e8 strbyteconv - 
- StringToBytes()
   - BytesToString()
 2014-11-30 23:06:47 -08:00
Harshavardhana
fa1a13a8bf Cleanup makefile 2014-11-30 15:20:20 -08:00
Harshavardhana
7e79721452 No caps for stdint 2014-11-30 11:57:58 -08:00
Harshavardhana
980ad3b373 Merge pull request #55 from harshavardhana/pr_out_decoding_bug_fixes_42 
Decoding bug fixes #42
 2014-11-29 17:24:40 -08:00
Harshavardhana
4d7c1e3b68 Decoding bug fixes #42 2014-11-29 17:22:31 -08:00
Harshavardhana
a6e75f42dd Merge pull request #53 from fkautz/pr_out_moving_gateway_and_storage_driver_to_packages 
Moving gateway and storage driver to packages
 2014-11-29 16:33:07 -08:00
Frederick F. Kautz IV
96e42a8443 Adding initial documentation to erasure package 2014-11-29 14:59:09 -08:00
Frederick F. Kautz IV
3da8e53ab8 Exposing NewEncoder since GetEncoder is no longer around 2014-11-29 14:54:21 -08:00
Frederick F. Kautz IV
03beef3afc Moving gateway and storage driver to packages 2014-11-29 14:42:22 -08:00
Harshavardhana
17a23876a2 Rename NewEncoder-->newEncoder 2014-11-29 14:41:20 -08:00
Harshavardhana
a86de74983 Merge pull request #50 from fkautz/pr_out_removing_lru_from_erasure_package 
Removing LRU from erasure package
 2014-11-29 14:39:07 -08:00
Harshavardhana
922e6bac0b Erasure restructuring 2014-11-29 14:28:55 -08:00
Frederick F. Kautz IV
9ef1ef5c8c Removing LRU from erasure package 2014-11-29 13:45:44 -08:00
Harshavardhana
f1fcd3be06 Make sure to use SIMD Alignment with 32 2014-11-27 02:15:41 -08:00
Harshavardhana
3cefaf5ad8 Restructure directory, add 'pkgs' folder now 2014-11-24 15:10:25 -08:00