Frederick F. Kautz IV
7971b66e60
Adding object donut type
2015-03-11 19:56:18 -07:00
Harshavardhana
41653774fc
Simplify fs codebase, split them into separate files and more commenting
2015-03-08 18:01:48 -07:00
Harshavardhana
9e89af0e3c
Remove redundant uint32
2015-03-07 19:15:51 -08:00
Frederick F. Kautz IV
679c6ace3a
Fragment reader implemented
2015-03-07 18:04:20 -08:00
Frederick F. Kautz IV
ca1a4b616c
Adding read header for erasure
2015-03-07 15:53:41 -08:00
Frederick F. Kautz IV
00381c887a
Adding erasure package to wrap versioned erasure. Ensures we always write latest version.
2015-03-07 13:04:02 -08:00
Harshavardhana
d992bccd9a
Move pkg/storage/erasure to pkg/encoding/erasure - and other cleanups
2015-03-07 00:16:48 -08:00
Frederick F. Kautz IV
856e0100c0
Erasure layer now writes using new technique
2015-03-06 16:37:44 -08:00
Harshavardhana
e5af8a3f5d
Further fixes -
...
- All test files have been renamed to their respective <package>_test name,
this is done in accordance with
- https://github.com/golang/go/wiki/CodeReviewComments#import-dot
imports are largely used in testing, but to avoid namespace collision
and circular dependencies
- Never use _* in package names other than "_test" change fragment_v1 to expose
fragment just like 'gopkg.in/check.v1'
2015-03-06 02:04:21 -08:00
Harshavardhana
c00d1461b9
Golint cleanup from top level
2015-03-05 23:37:37 -08:00
Frederick F. Kautz IV
1186c47603
Fragment now implements new version
2015-03-05 20:56:02 -08:00
Frederick F. Kautz IV
665af44c70
Renaming donut packages to match layout
2015-03-05 19:44:01 -08:00
Harshavardhana
3a3c8645fc
Update comments across the codebase
2015-03-03 02:39:38 -08:00
Harshavardhana
c3ad0906e0
Add deadcode code which recursivley goes into all directories and verifies dangling variables.
2015-03-02 18:44:20 -08:00
Frederick F. Kautz IV
6f300c9ef0
Adding test at byte level
2015-03-01 15:13:09 -08:00
Frederick F. Kautz IV
0b08312ddd
Specifying more exact types for data
2015-03-01 14:43:11 -08:00
Harshavardhana
9f43659135
Add tests, add NewHeader() for populating Header and ValidateHeader()
2015-03-01 13:19:06 -08:00
Frederick F. Kautz IV
7414dc9b75
Adding data writer
2015-03-01 11:32:30 -08:00
Frederick F. Kautz IV
be12a6b3d9
Changing K and M in data.go to match new datatype in erasure package
2015-03-01 01:28:33 -08:00
Frederick F. Kautz IV
bb0441917c
Making changes to data based on discussion
2015-03-01 01:16:22 -08:00
Harshavardhana
2f04a1299e
Avoid allocating 1gig memory, avoid such mistakes
2015-03-01 00:43:12 -08:00
Frederick F. Kautz IV
2916b1278e
Adding data.go, does nothing for now.
2015-03-01 00:27:52 -08:00
Frederick F. Kautz IV
b4ef522a9b
Donut frame is now its own package
2015-03-01 00:02:14 -08:00
Frederick F. Kautz IV
6c2428833a
Renaming Write to WriteFrame
2015-02-28 23:52:56 -08:00
Harshavardhana
6b397844b9
Add donut benchmark for 64MB, 128MB, 256MB, 512MB
...
Current average values
~~~
> minio/pkg/storage/donut/v1 *> go test -bench .
OK: 2 passed
PASS
BenchmarkDonut64M 5 266249634 ns/op 252.05 MB/s
BenchmarkDonut128M 3 436507394 ns/op 307.48 MB/s
BenchmarkDonut256M 2 836116359 ns/op 321.05 MB/s
BenchmarkDonut512M 1 1676738951 ns/op 320.19 MB/s
ok github.com/minio-io/minio/pkg/storage/donut/v1 8.430s
> minio/pkg/storage/donut/v1 *>
~~~
2015-02-28 17:05:46 -08:00
Harshavardhana
65f22083ca
donut_gen works with new Donut Frame
2015-02-28 16:30:18 -08:00
Frederick F. Kautz IV
ddc7cf835e
Donut now tests amount written should match expected amount to write
2015-02-28 16:09:52 -08:00
Frederick F. Kautz IV
d3d1205eb1
Updating comment describing data type
2015-02-28 15:59:52 -08:00
Frederick F. Kautz IV
58a04ee831
New donut frame implemented
2015-02-28 15:54:13 -08:00
Harshavardhana
3f8b3462e6
Remove donut_gen binary and update crc32c function
2015-02-27 14:41:53 -08:00
Frederick F. Kautz IV
2e37791681
Merge pull request #212 from fkautz/pr_out_adding_comparison_for_footer_s_inim
2015-02-27 12:40:10 -08:00
Frederick F. Kautz IV
1f7a3c13a5
Adding comparison for footer's INIM
2015-02-27 12:39:52 -08:00
Frederick F. Kautz IV
2ca62f13a5
Moving donut_gen to its own dir and renaming it as donut_gen_v1
2015-02-27 12:39:14 -08:00
Frederick F. Kautz IV
be768d01a2
Seek to beginning for tempbuffer
2015-02-27 12:18:27 -08:00
Frederick F. Kautz IV
5c52da6ebe
Adding crc32c to write
2015-02-26 23:04:07 -08:00
Frederick F. Kautz IV
ef442289da
Making donut use io.ReadWriteSeek instead of io.Writer
2015-02-23 20:27:25 -08:00
Harshavardhana
3151f9b6c3
Add proper entries for GobHeader, Data copy.
...
This commit also adds 'donut_gen.go' for generating sample donut file
2015-02-21 01:44:27 -08:00
Frederick F. Kautz IV
8129d8fd0b
Writing data out with error handling
2015-02-20 23:24:01 -08:00
Frederick F. Kautz IV
98ddbd840e
Moving donut code to v1 in donut package
2015-02-20 23:07:43 -08:00
Frederick F. Kautz IV
9833647fb7
Fixing magic number for INIM
2015-02-20 22:42:58 -08:00
Anand Babu (AB) Periasamy
ec36a598c9
Magic as numbers are faster to encode and compare
2015-02-20 18:20:21 -08:00
Harshavardhana
c4f0aad9b5
Add donut tests, just for verification
2015-02-18 19:46:14 -08:00
Frederick F. Kautz IV
f7f34d08e8
Donut structure refined
2015-02-18 17:18:33 -08:00
Anand Babu (AB) Periasamy
9e304b53de
donut spec v1 prototype
2015-02-18 15:52:12 -08:00
Frederick F. Kautz IV
05ac09bf69
Adding header version
2015-02-18 10:21:02 -08:00
Frederick F. Kautz IV
fe9a5c14a0
Adding scaffolding for writing donut
2015-02-17 17:47:10 -08:00