MyFavMirrors/minio
1
0
Fork 0
mirror of https://github.com/minio/minio.git synced 2024-12-30 09:03:22 -05:00
Code Issues Packages Projects Releases Wiki Activity
1,305 Commits 4 Branches 506 Tags 189 MiB
c99d96dbc2
Commit Graph

231 Commits

Author SHA1 Message Date
Harshavardhana
a42a48a455 Get object range request added 2015-03-12 00:07:27 -07:00
Frederick F. Kautz IV
24d5018037 Rename StoreBucket to CreateBucket 2015-03-11 20:14:40 -07:00
Frederick F. Kautz IV
8c6aa45458 Adding copyright 2015-03-11 19:59:20 -07:00
Frederick F. Kautz IV
7971b66e60 Adding object donut type 2015-03-11 19:56:18 -07:00
Frederick F. Kautz IV
ec9b20a94f Adding stubes for new single disk driver 2015-03-11 13:36:21 -07:00
Harshavardhana
41653774fc Simplify fs codebase, split them into separate files and more commenting 2015-03-08 18:01:48 -07:00
Harshavardhana
01e98eb326 Make BucketPolicy strictly typed 2015-03-08 15:57:20 -07:00
Harshavardhana
8866b2cc5f Add delimiter and prefix tests, also add inmemory to support delimiters 2015-03-08 14:51:31 -07:00
Harshavardhana
7a01db8005 Strict checks for delimiter and prefix if absent or present 2015-03-08 13:01:39 -07:00
Harshavardhana
9e89af0e3c Remove redundant uint32 2015-03-07 19:15:51 -08:00
Harshavardhana
9553eb6b4a Merge pull request #269 from harshavardhana/pr_out_get_object_metadata_needs_to_filter_based_of_prefix_when_delimiter_is_specified 2015-03-07 19:14:41 -08:00
Harshavardhana
f8e2b75e23 Get object metadata needs to filter based of Prefix when delimiter is specified 2015-03-07 19:13:15 -08:00
Frederick F. Kautz IV
b39eac36ea Merge pull request #268 from fkautz/pr_out_fragment_reader_implemented 2015-03-07 18:04:29 -08:00
Frederick F. Kautz IV
679c6ace3a Fragment reader implemented 2015-03-07 18:04:20 -08:00
Harshavardhana
df69038b04 Merge pull request #267 from harshavardhana/pr_out_add_md5_support_for_fsstorage_also_re_purpose_getobjectmetadata_for_listobjects 2015-03-07 17:16:00 -08:00
Harshavardhana
4751ed0e30 Add MD5 support for fsStorage, also re-purpose GetObjectMetadata for ListObjects 2015-03-07 17:15:00 -08:00
Frederick F. Kautz IV
ca1a4b616c Adding read header for erasure 2015-03-07 15:53:41 -08:00
Frederick F. Kautz IV
00381c887a Adding erasure package to wrap versioned erasure. Ensures we always write latest version. 2015-03-07 13:04:02 -08:00
Harshavardhana
d992bccd9a Move pkg/storage/erasure to pkg/encoding/erasure - and other cleanups 2015-03-07 00:16:48 -08:00
Frederick F. Kautz IV
856e0100c0 Erasure layer now writes using new technique 2015-03-06 16:37:44 -08:00
Harshavardhana
e5af8a3f5d Further fixes - 
- All test files have been renamed to their respective <package>_test name,
    this is done in accordance with
      - https://github.com/golang/go/wiki/CodeReviewComments#import-dot
        imports are largely used in testing, but to avoid namespace collision
        and circular dependencies

  - Never use _* in package names other than "_test" change fragment_v1 to expose
    fragment just like 'gopkg.in/check.v1'
 2015-03-06 02:04:21 -08:00
Harshavardhana
c00d1461b9 Golint cleanup from top level 2015-03-05 23:37:37 -08:00
Harshavardhana
540723d7ae Golint cleanup pkg/api 2015-03-05 21:09:19 -08:00
Harshavardhana
76e601b26b Golint cleanup pkg/storage 2015-03-05 21:09:19 -08:00
Frederick F. Kautz IV
1186c47603 Fragment now implements new version 2015-03-05 20:56:02 -08:00
Frederick F. Kautz IV
665af44c70 Renaming donut packages to match layout 2015-03-05 19:44:01 -08:00
Harshavardhana
f2e238199b Handle individual objects too 2015-03-04 01:54:35 -08:00
Harshavardhana
e400e85639 Add proper path prefix to be delimited 2015-03-04 00:54:39 -08:00
Harshavardhana
cc46475c86 Pathprefix is filtered out properly and delimited as well 2015-03-03 19:45:54 -08:00
Harshavardhana
3a3c8645fc Update comments across the codebase 2015-03-03 02:39:38 -08:00
Harshavardhana
137584d658 Add comments 2015-03-03 01:25:45 -08:00
Harshavardhana
c3ad0906e0 Add deadcode code which recursivley goes into all directories and verifies dangling variables. 2015-03-02 18:44:20 -08:00
Frederick F. Kautz IV
32f8c17174 Renaming AppendUstr to AppendUniqStr 2015-03-01 17:06:00 -08:00
Frederick F. Kautz IV
6f300c9ef0 Adding test at byte level 2015-03-01 15:13:09 -08:00
Frederick F. Kautz IV
0b08312ddd Specifying more exact types for data 2015-03-01 14:43:11 -08:00
Harshavardhana
9f43659135 Add tests, add NewHeader() for populating Header and ValidateHeader() 2015-03-01 13:19:06 -08:00
Anand Babu (AB) Periasamy
6515678e87 Merge pull request #236 from fkautz/pr_out_adding_data_writer 
Adding data writer
 2015-03-01 11:36:35 -08:00
Frederick F. Kautz IV
7414dc9b75 Adding data writer 2015-03-01 11:32:30 -08:00
Harshavardhana
eb9a4a83ac Move AppendUstr(), AppendUint() into helpers 2015-03-01 11:19:03 -08:00
Frederick F. Kautz IV
be12a6b3d9 Changing K and M in data.go to match new datatype in erasure package 2015-03-01 01:28:33 -08:00
Harshavardhana
33601ff567 Merge pull request #233 from harshavardhana/pr_out_make_k_m_to_be_uint8_and_technique_becomes_its_own_type 2015-03-01 01:26:42 -08:00
Harshavardhana
b77c62e25d Make K,M to be uint8 and Technique becomes its own type 2015-03-01 01:25:28 -08:00
Frederick F. Kautz IV
bb0441917c Making changes to data based on discussion 2015-03-01 01:16:22 -08:00
Harshavardhana
2f04a1299e Avoid allocating 1gig memory, avoid such mistakes 2015-03-01 00:43:12 -08:00
Frederick F. Kautz IV
2916b1278e Adding data.go, does nothing for now. 2015-03-01 00:27:52 -08:00
Frederick F. Kautz IV
b4ef522a9b Donut frame is now its own package 2015-03-01 00:02:14 -08:00
Frederick F. Kautz IV
6c2428833a Renaming Write to WriteFrame 2015-02-28 23:52:56 -08:00
Harshavardhana
6b397844b9 Add donut benchmark for 64MB, 128MB, 256MB, 512MB 
Current average values

~~~
> minio/pkg/storage/donut/v1 *> go test -bench .
OK: 2 passed
PASS
BenchmarkDonut64M              5         266249634 ns/op         252.05 MB/s
BenchmarkDonut128M             3         436507394 ns/op         307.48 MB/s
BenchmarkDonut256M             2         836116359 ns/op         321.05 MB/s
BenchmarkDonut512M             1        1676738951 ns/op         320.19 MB/s
ok      github.com/minio-io/minio/pkg/storage/donut/v1  8.430s
> minio/pkg/storage/donut/v1 *>
~~~
 2015-02-28 17:05:46 -08:00
Harshavardhana
65f22083ca donut_gen works with new Donut Frame 2015-02-28 16:30:18 -08:00
Frederick F. Kautz IV
ddc7cf835e Donut now tests amount written should match expected amount to write 2015-02-28 16:09:52 -08:00
Frederick F. Kautz IV
d3d1205eb1 Updating comment describing data type 2015-02-28 15:59:52 -08:00
Frederick F. Kautz IV
58a04ee831 New donut frame implemented 2015-02-28 15:54:13 -08:00
Harshavardhana
0c2d58bc6d Implement proper delimiter and prefix handling 
With this change Minio server now responds with, delimited
'object names' in conjunction with prefix filtering

~~~
<ListBucketResult>
  <Name>example-bucket</Name>
  <Prefix></Prefix>
  <Marker></Marker>
  <MaxKeys>1000</MaxKeys>
  <Delimiter>/</Delimiter>
  <IsTruncated>false</IsTruncated>
  <Contents>
    <Key>sample.html</Key>
    <LastModified>2011-02-26T01:56:20.000Z</LastModified>
    <ETag>example-bucket#sample.html</ETag>
    <Size>142863</Size>
    <Owner>
      <ID>minio</ID>
      <DisplayName>minio</DisplayName>
    </Owner>
    <StorageClass>STANDARD</StorageClass>
  </Contents>
  <CommonPrefixes>
    <Prefix>photos/</Prefix>
  </CommonPrefixes>
</ListBucketResult>
~~~

~~~
<ListBucketResult>
  <Name>example-bucket</Name>
  <Prefix>photos/2006/</Prefix>
  <Marker></Marker>
  <MaxKeys>1000</MaxKeys>
  <Delimiter>/</Delimiter>
  <IsTruncated>false</IsTruncated>

  <CommonPrefixes>
    <Prefix>photos/2006/feb/</Prefix>
  </CommonPrefixes>
  <CommonPrefixes>
    <Prefix>photos/2006/jan/</Prefix>
  </CommonPrefixes>
</ListBucketResult>
~~~
 2015-02-28 14:47:19 -08:00
Harshavardhana
d5e5a3b3e6 Use validString 2015-02-27 19:49:18 -08:00
Harshavardhana
6ebb48b4ea Add object name validation 2015-02-27 19:42:04 -08:00
Harshavardhana
53669a0854 Implement delimiter, path prefix 2015-02-27 16:23:05 -08:00
Harshavardhana
3f8b3462e6 Remove donut_gen binary and update crc32c function 2015-02-27 14:41:53 -08:00
Frederick F. Kautz IV
2e37791681 Merge pull request #212 from fkautz/pr_out_adding_comparison_for_footer_s_inim 2015-02-27 12:40:10 -08:00
Frederick F. Kautz IV
1f7a3c13a5 Adding comparison for footer's INIM 2015-02-27 12:39:52 -08:00
Frederick F. Kautz IV
2ca62f13a5 Moving donut_gen to its own dir and renaming it as donut_gen_v1 2015-02-27 12:39:14 -08:00
Frederick F. Kautz IV
be768d01a2 Seek to beginning for tempbuffer 2015-02-27 12:18:27 -08:00
Frederick F. Kautz IV
5c52da6ebe Adding crc32c to write 2015-02-26 23:04:07 -08:00
Frederick F. Kautz IV
ef442289da Making donut use io.ReadWriteSeek instead of io.Writer 2015-02-23 20:27:25 -08:00
Harshavardhana
f4b5519451 Update erasure documentation 2015-02-23 13:39:16 -08:00
Harshavardhana
51e80eaa6d Add domain and subdomain support for MinioAPI 
This change brings in domain and subdomain support

   - ./minio --domain "yourminiodomain.com"

This change brings in a much needed feature by keeping
bucketnames as part of your 'DNS' name.

All your existing applications can be migrated off from s3 to
Minio without little to no modifications.

NOTE: Setting up DNS for your `buckets` is out of scope of this feature
 2015-02-23 02:25:01 -08:00
Harshavardhana
3151f9b6c3 Add proper entries for GobHeader, Data copy. 
This commit also adds 'donut_gen.go' for generating sample donut file
 2015-02-21 01:44:27 -08:00
Frederick F. Kautz IV
8129d8fd0b Writing data out with error handling 2015-02-20 23:24:01 -08:00
Frederick F. Kautz IV
98ddbd840e Moving donut code to v1 in donut package 2015-02-20 23:07:43 -08:00
Frederick F. Kautz IV
9833647fb7 Fixing magic number for INIM 2015-02-20 22:42:58 -08:00
Anand Babu (AB) Periasamy
ec36a598c9 Magic as numbers are faster to encode and compare 2015-02-20 18:20:21 -08:00
Harshavardhana
c4f0aad9b5 Add donut tests, just for verification 2015-02-18 19:46:14 -08:00
Frederick F. Kautz IV
f7f34d08e8 Donut structure refined 2015-02-18 17:18:33 -08:00
Frederick F. Kautz IV
ab80c0ce12 Merge pull request #180 from abperiasamy/donut 2015-02-18 15:57:16 -08:00
Anand Babu (AB) Periasamy
9e304b53de donut spec v1 prototype 2015-02-18 15:52:12 -08:00
Harshavardhana
fab954f13f Run govet and fix 2015-02-18 15:33:55 -08:00
Frederick F. Kautz IV
05ac09bf69 Adding header version 2015-02-18 10:21:02 -08:00
Harshavardhana
a3e5121f7b Make channels more aware of errors and shutdown on error - some cleanup 2015-02-18 00:24:19 -08:00
Frederick F. Kautz IV
fe9a5c14a0 Adding scaffolding for writing donut 2015-02-17 17:47:10 -08:00
Harshavardhana
f04beab7c2 Skip policy files 2015-02-16 18:37:09 -08:00
Harshavardhana
eeae64935e Implement bucket policy handler and with galore of cleanup 2015-02-16 18:20:40 -08:00
Harshavardhana
966cdd4f05 Reply in xml from validate and ignore handlers 2015-02-11 22:00:45 -08:00
Harshavardhana
ac4f07906c Implement S3 Style ErrorCodes and Response 
This patchset also brings in lot of cleanup in terms of minioapi codebase
 2015-02-11 21:27:28 -08:00
Harshavardhana
680848bdcb Add license header 2015-02-08 03:00:32 -08:00
Frederick F. Kautz IV
288bc3fcc7 Adding content type to backend storage drivers 2015-02-04 17:32:40 -08:00
Frederick F. Kautz IV
7152712f34 Accessing dir as object should return ObjectNotFound 2015-02-03 16:45:00 -08:00
Frederick F. Kautz IV
a092b44cf1 Adding test to ensure list objects returns an error for non existant buckets to match previous bug fix and fix for inmemory 2015-02-01 10:48:31 -08:00
Harshavardhana
5d197dd063 Reply back errors properly 2015-02-01 03:10:28 -08:00
Frederick F. Kautz IV
54026dbc78 Setting order when retrieving buckets 2015-02-01 00:16:25 -08:00
Frederick F. Kautz IV
7ca87e06aa Adding list bucket tests and fixing inmemory behavior. 2015-01-29 11:23:14 -08:00
Harshavardhana
82baccb8bf Prefix if empty skip it before IsValidBucket 2015-01-29 01:07:37 -08:00
Frederick F. Kautz IV
ca7b1a7fc6 Fix build 2015-01-28 14:00:08 -08:00
Harshavardhana
2ccecd3f32 Add list buckets for 'fs' 2015-01-27 18:43:55 -08:00
Frederick F. Kautz IV
5daa408e2c Creating subdirectories in fs now works 2015-01-27 17:47:11 -08:00
Frederick F. Kautz IV
044dc9af80 Adding more tests, including ordering of objects in prefix 2015-01-27 13:11:22 -08:00
Harshavardhana
aa4bc2549e Merge pull request #72 from harshavardhana/pr_out_add_erasure_package_in_its_full_form_v1_0 2015-01-27 12:57:51 -08:00
Harshavardhana
2547163cb2 Add erasure package in its full form v1.0 2015-01-27 12:55:17 -08:00
Frederick F. Kautz IV
74ef65daac Removing log from fs_test.go 2015-01-27 12:02:08 -08:00
Frederick F. Kautz IV
163a6c35db Adding fs storage 2015-01-27 11:09:55 -08:00
Frederick F. Kautz IV
49641e3410 Adding storage modifications for upcoming fs system 2015-01-26 15:45:10 -08:00
Harshavardhana
f3a27169fc Avoid 'clang' optimization, keep it -O0 2015-01-26 12:22:08 -08:00
Frederick F. Kautz IV
2368e7c936 Adding error support to all commands in storage interface 2015-01-25 15:35:08 -08:00
Frederick F. Kautz IV
d37bbfec41 Adding IsTruncated to minioapi 2015-01-25 13:32:39 -08:00
Frederick F. Kautz IV
39208601dc Adding more api suite tests 2015-01-25 13:06:36 -08:00
Frederick F. Kautz IV
0da193e6d6 Adding paging count tests 2015-01-25 12:57:58 -08:00
Frederick F. Kautz IV
82a69d3ab9 Adding initial test suites 2015-01-25 11:22:38 -08:00
Harshavardhana
357a81e879 Add bucket to be part of object struct, reply back with BucketNotFound 
for erroneous Object Put requests.

Other minor cleanups, to follow

  - https://github.com/golang/go/wiki/CodeReviewComments
 2015-01-24 15:35:05 -08:00
Frederick F. Kautz IV
8d1c447aac Adding HEAD verb to minioapi 2015-01-22 14:25:53 -08:00
Frederick F. Kautz IV
3a3d9989d1 Merge pull request #39 from fkautz/pr_out_making_in_memory_storage_data_type_private 2015-01-21 17:15:29 -08:00
Frederick F. Kautz IV
079cdcd909 Making in memory storage data type private 2015-01-21 17:15:05 -08:00
Frederick F. Kautz IV
8efd23b755 Migrating from unix ts to time.Time 2015-01-21 17:12:47 -08:00
Frederick F. Kautz IV
53a8536984 Fixing regex 2015-01-21 15:29:31 -08:00
Frederick F. Kautz IV
856781b2a4 Simplifying bucket name convention and making convention public 2015-01-21 15:22:15 -08:00
Frederick F. Kautz IV
ae0b88f319 Simplfying bucket naming convention 2015-01-21 15:20:35 -08:00
Frederick F. Kautz IV
9ca1e37235 Adding etag to headers 2015-01-21 15:02:08 -08:00
Frederick F. Kautz IV
c23fa26830 Extracting storage api to interface 2015-01-21 12:58:33 -08:00
Harshavardhana
d44404dd81 Provide ETag with sha256Sum of input object data 2015-01-21 10:55:33 -08:00
Harshavardhana
5b67da7d96 Add PutBucket and ListBuckets service 2015-01-21 00:52:59 -08:00
Harshavardhana
e22ae2475a Add license headers and other cleanup 2015-01-20 23:16:24 -08:00
Frederick F. Kautz IV
9260e6404a Adding error when object exists 2015-01-20 19:13:57 -08:00
Frederick F. Kautz IV
cc4329fb12 List objects in a bucket. 2015-01-20 18:40:22 -08:00
Frederick F. Kautz IV
53190e1210 Initial work for xml list objects 2015-01-20 16:08:14 -08:00
Frederick F. Kautz IV
36268f07ea Adding error handling to storage get call 2015-01-19 17:39:20 -08:00
Frederick F. Kautz IV
c791c7bca4 Store objects in memory map 2015-01-19 10:30:40 -08:00
Frederick F. Kautz IV
718794205f wip 2015-01-18 17:05:50 -08:00
Frederick F. Kautz IV
c766f3617b Populating http handler with test scaffolding 2015-01-18 16:16:02 -08:00
Frederick F. Kautz IV
d3d1c1afb3 Minor refactoring, moving http request logic from storage to server 2015-01-18 15:50:53 -08:00
Frederick F. Kautz IV
f356599e22 Adding initial web server and storage server module infrastructure 2015-01-18 15:06:24 -08:00
Frederick F. Kautz IV
f550e84cf4 Removing old server 
Removing storage drivers
 2015-01-18 14:54:46 -08:00
Harshavardhana
68de9ac19e Further restructure 2015-01-14 12:40:43 -08:00
Harshavardhana
432275e966 Full restructure in accordance with 
- pkg/{subsystem}/{package} style
  - modify Makefile to reflect the new style,
    consolidate various entries
  - add a dummy ``main.go`` at top level
 2015-01-14 11:29:04 -08:00
Harshavardhana
6b36b5c551 A full restructure 2014-12-29 21:22:26 -08:00