Commit Graph

96 Commits

Author SHA1 Message Date
Frederick F. Kautz IV 7971b66e60 Adding object donut type 2015-03-11 19:56:18 -07:00
Harshavardhana 41653774fc Simplify fs codebase, split them into separate files and more commenting 2015-03-08 18:01:48 -07:00
Harshavardhana 9e89af0e3c Remove redundant uint32 2015-03-07 19:15:51 -08:00
Frederick F. Kautz IV 679c6ace3a Fragment reader implemented 2015-03-07 18:04:20 -08:00
Frederick F. Kautz IV ca1a4b616c Adding read header for erasure 2015-03-07 15:53:41 -08:00
Frederick F. Kautz IV 00381c887a Adding erasure package to wrap versioned erasure. Ensures we always write latest version. 2015-03-07 13:04:02 -08:00
Harshavardhana d992bccd9a Move pkg/storage/erasure to pkg/encoding/erasure - and other cleanups 2015-03-07 00:16:48 -08:00
Frederick F. Kautz IV 856e0100c0 Erasure layer now writes using new technique 2015-03-06 16:37:44 -08:00
Harshavardhana e5af8a3f5d Further fixes -
- All test files have been renamed to their respective <package>_test name,
    this is done in accordance with
      - https://github.com/golang/go/wiki/CodeReviewComments#import-dot
        imports are largely used in testing, but to avoid namespace collision
        and circular dependencies

  - Never use _* in package names other than "_test" change fragment_v1 to expose
    fragment just like 'gopkg.in/check.v1'
2015-03-06 02:04:21 -08:00
Harshavardhana c00d1461b9 Golint cleanup from top level 2015-03-05 23:37:37 -08:00
Frederick F. Kautz IV 1186c47603 Fragment now implements new version 2015-03-05 20:56:02 -08:00
Frederick F. Kautz IV 665af44c70 Renaming donut packages to match layout 2015-03-05 19:44:01 -08:00
Harshavardhana 3a3c8645fc Update comments across the codebase 2015-03-03 02:39:38 -08:00
Harshavardhana c3ad0906e0 Add deadcode code which recursivley goes into all directories and verifies dangling variables. 2015-03-02 18:44:20 -08:00
Frederick F. Kautz IV 6f300c9ef0 Adding test at byte level 2015-03-01 15:13:09 -08:00
Frederick F. Kautz IV 0b08312ddd Specifying more exact types for data 2015-03-01 14:43:11 -08:00
Harshavardhana 9f43659135 Add tests, add NewHeader() for populating Header and ValidateHeader() 2015-03-01 13:19:06 -08:00
Frederick F. Kautz IV 7414dc9b75 Adding data writer 2015-03-01 11:32:30 -08:00
Frederick F. Kautz IV be12a6b3d9 Changing K and M in data.go to match new datatype in erasure package 2015-03-01 01:28:33 -08:00
Frederick F. Kautz IV bb0441917c Making changes to data based on discussion 2015-03-01 01:16:22 -08:00
Harshavardhana 2f04a1299e Avoid allocating 1gig memory, avoid such mistakes 2015-03-01 00:43:12 -08:00
Frederick F. Kautz IV 2916b1278e Adding data.go, does nothing for now. 2015-03-01 00:27:52 -08:00
Frederick F. Kautz IV b4ef522a9b Donut frame is now its own package 2015-03-01 00:02:14 -08:00
Frederick F. Kautz IV 6c2428833a Renaming Write to WriteFrame 2015-02-28 23:52:56 -08:00
Harshavardhana 6b397844b9 Add donut benchmark for 64MB, 128MB, 256MB, 512MB
Current average values

~~~
> minio/pkg/storage/donut/v1 *> go test -bench .
OK: 2 passed
PASS
BenchmarkDonut64M              5         266249634 ns/op         252.05 MB/s
BenchmarkDonut128M             3         436507394 ns/op         307.48 MB/s
BenchmarkDonut256M             2         836116359 ns/op         321.05 MB/s
BenchmarkDonut512M             1        1676738951 ns/op         320.19 MB/s
ok      github.com/minio-io/minio/pkg/storage/donut/v1  8.430s
> minio/pkg/storage/donut/v1 *>
~~~
2015-02-28 17:05:46 -08:00
Harshavardhana 65f22083ca donut_gen works with new Donut Frame 2015-02-28 16:30:18 -08:00
Frederick F. Kautz IV ddc7cf835e Donut now tests amount written should match expected amount to write 2015-02-28 16:09:52 -08:00
Frederick F. Kautz IV d3d1205eb1 Updating comment describing data type 2015-02-28 15:59:52 -08:00
Frederick F. Kautz IV 58a04ee831 New donut frame implemented 2015-02-28 15:54:13 -08:00
Harshavardhana 3f8b3462e6 Remove donut_gen binary and update crc32c function 2015-02-27 14:41:53 -08:00
Frederick F. Kautz IV 2e37791681 Merge pull request #212 from fkautz/pr_out_adding_comparison_for_footer_s_inim 2015-02-27 12:40:10 -08:00
Frederick F. Kautz IV 1f7a3c13a5 Adding comparison for footer's INIM 2015-02-27 12:39:52 -08:00
Frederick F. Kautz IV 2ca62f13a5 Moving donut_gen to its own dir and renaming it as donut_gen_v1 2015-02-27 12:39:14 -08:00
Frederick F. Kautz IV be768d01a2 Seek to beginning for tempbuffer 2015-02-27 12:18:27 -08:00
Frederick F. Kautz IV 5c52da6ebe Adding crc32c to write 2015-02-26 23:04:07 -08:00
Frederick F. Kautz IV ef442289da Making donut use io.ReadWriteSeek instead of io.Writer 2015-02-23 20:27:25 -08:00
Harshavardhana 3151f9b6c3 Add proper entries for GobHeader, Data copy.
This commit also adds 'donut_gen.go' for generating sample donut file
2015-02-21 01:44:27 -08:00
Frederick F. Kautz IV 8129d8fd0b Writing data out with error handling 2015-02-20 23:24:01 -08:00
Frederick F. Kautz IV 98ddbd840e Moving donut code to v1 in donut package 2015-02-20 23:07:43 -08:00
Frederick F. Kautz IV 9833647fb7 Fixing magic number for INIM 2015-02-20 22:42:58 -08:00
Anand Babu (AB) Periasamy ec36a598c9 Magic as numbers are faster to encode and compare 2015-02-20 18:20:21 -08:00
Harshavardhana c4f0aad9b5 Add donut tests, just for verification 2015-02-18 19:46:14 -08:00
Frederick F. Kautz IV f7f34d08e8 Donut structure refined 2015-02-18 17:18:33 -08:00
Anand Babu (AB) Periasamy 9e304b53de donut spec v1 prototype 2015-02-18 15:52:12 -08:00
Frederick F. Kautz IV 05ac09bf69 Adding header version 2015-02-18 10:21:02 -08:00
Frederick F. Kautz IV fe9a5c14a0 Adding scaffolding for writing donut 2015-02-17 17:47:10 -08:00