Commit Graph

266 Commits

Author SHA1 Message Date
Harshavardhana
988d39a5b6 Migrate to golang1.5 release with GO15VENDOREXPERIMENT=1 enabled 2015-08-22 18:35:37 -07:00
Harshavardhana
74587886d2 Refactoring minio server command and flags 2015-08-20 13:07:33 -07:00
Harshavardhana
61175ef091 Migrate to govendor to avoid limitations of godep
- over the course of a project history every maintainer needs to update
  its dependency packages, the problem essentially with godep is manipulating
  GOPATH - this manipulation leads to static objects created at different locations
  which end up conflicting with the overall functionality of golang.

  This also leads to broken builds. There is no easier way out of this other than
  asking developers to do 'godep restore' all the time. Which perhaps as a practice
  doesn't sound like a clean solution. On the other hand 'godep restore' has its own
  set of problems.

- govendor is a right tool but a stop gap tool until we wait for golangs official
  1.5 version which fixes this vendoring issue once and for all.

- govendor provides consistency in terms of how import paths should be handled unlike
  manipulation GOPATH.

  This has advantages
    - no more compiled objects being referenced in GOPATH and build time GOPATH
      manging which leads to conflicts.
    - proper import paths referencing the exact package a project is dependent on.

 govendor is simple and provides the minimal necessary tooling to achieve this.

 For now this is the right solution.
2015-08-12 19:24:57 -07:00
Harshavardhana
5d3379ed7e deprecate 'make.go', go back to Makefile - make.go is not genversion.go 2015-07-31 17:16:54 -07:00
Harshavardhana
aabfd541e1 Merge cmd/donut into minio cmd, deprecate controller RPC request 2015-07-31 12:57:15 -07:00
Harshavardhana
e4543489fe Use updatedeps script to update godeps 2015-07-16 12:57:32 -07:00
Harshavardhana
4498662c16 Remove scsi non portable code, instead "donut make" implements functionality to instantiate a donut 2015-07-15 11:55:57 -07:00
Harshavardhana
1e80925ca7 Remove dependency on minio/cli for make.go fixes #738 2015-07-13 11:39:28 -07:00
Harshavardhana
55e4d0c6a5 mkdonut now creates a donut processing cli args 2015-07-12 21:21:31 -07:00
Harshavardhana
7615a6bfe5 HEAD shouldn't have any body, handle it in writeErrorResponse() 2015-07-11 10:34:55 -07:00
Harshavardhana
5cfb05465e Add cache, donut tests separately - fix behavior differences
Remove priority queue, implement it using a simpler channels
2015-07-02 21:04:04 -07:00
Harshavardhana
8bde4d4e34 Migrating minio server build to do 'go run make.go' style like mc 2015-06-16 20:23:02 -07:00
Harshavardhana
5101273ec7 Do not attempt to restore while saving new deps 2015-06-04 16:32:35 -07:00
Harshavardhana
5d484e2770 Implement custom date command for cross platform portability 2015-06-04 12:32:23 -07:00
Harshavardhana
82a0eac659 Add filesystem factory functions and add related cli options 2015-05-30 03:23:06 -07:00
Harshavardhana
554a244b47 Add donut OSX support 2015-05-26 02:14:34 -07:00
Harshavardhana
f00bfd38da Bump gocyclo back to 16 2015-05-19 18:32:50 -07:00
Harshavardhana
de56909d55 More scripts and code from Minio-io to minio 2015-05-11 16:43:38 -07:00
Harshavardhana
8d06504068 Change all minio-io path to minio 2015-05-11 16:23:10 -07:00
Harshavardhana
bbace9ac2e Make buildDate universal 2015-04-30 21:16:49 -07:00
Harshavardhana
a60d279817 bump up gocyclo to 18 2015-04-29 15:34:00 -07:00
Anand Babu (AB) Periasamy
b8eb34aded cleaning make:clean 2015-04-28 21:24:59 -07:00
Harshavardhana
096a994c7b Add build date using ldflags 2015-04-27 13:08:51 -07:00
Anand Babu (AB) Periasamy
b010fd0ff3 Version is now based on MD5SUM of its binary 2015-04-24 21:51:52 -07:00
Harshavardhana
c5d8ca245d Memory now handles submitting large files - fixes #482 2015-04-24 15:22:22 -07:00
Harshavardhana
97d1f03a87 Fix build for travis 2015-04-11 17:31:01 -07:00
Harshavardhana
c63bdf79fa Remove docs, minimal docs going to be at minio.io instead and rest amazon docs 2015-04-10 21:21:59 -07:00
Harshavardhana
7fd1cc073c Add everything back into one project hood, still missing iodine 2015-04-08 17:47:56 -07:00
Harshavardhana
637d2d687a Add full rebuild for minio command to avoid godep stale issues 2015-04-02 16:11:52 -07:00
Harshavardhana
9c0b17c239 Build everything by default 2015-04-02 14:34:05 -07:00
Frederick F. Kautz IV
a689769245 Updating godep to minio-io/godep 2015-04-01 20:57:27 -07:00
Harshavardhana
92f408af5d Add erasure to godep 2015-04-01 16:56:43 -07:00
Harshavardhana
4d57ca7c44 Update makefile 2015-03-25 23:10:03 -07:00
Anand Babu (AB) Periasamy
e14ba846b5 make clean build-constants.go 2015-03-25 23:04:27 -07:00
Anand Babu (AB) Periasamy
3c4aa85c69 git commit hash generator 2015-03-25 22:57:30 -07:00
Harshavardhana
58082cd8dc Add gocyclo into source build, choosing cyclomatic complexity tolerance of 15 and below 2015-03-25 15:57:17 -07:00
Harshavardhana
b952855779 Add strict checks with vet, golint and gofmt 2015-03-09 16:17:38 -07:00
Harshavardhana
d992bccd9a Move pkg/storage/erasure to pkg/encoding/erasure - and other cleanups 2015-03-07 00:16:48 -08:00
Harshavardhana
e5af8a3f5d Further fixes -
- All test files have been renamed to their respective <package>_test name,
    this is done in accordance with
      - https://github.com/golang/go/wiki/CodeReviewComments#import-dot
        imports are largely used in testing, but to avoid namespace collision
        and circular dependencies

  - Never use _* in package names other than "_test" change fragment_v1 to expose
    fragment just like 'gopkg.in/check.v1'
2015-03-06 02:04:21 -08:00
Harshavardhana
3a3c8645fc Update comments across the codebase 2015-03-03 02:39:38 -08:00
Harshavardhana
c3ad0906e0 Add deadcode code which recursivley goes into all directories and verifies dangling variables. 2015-03-02 18:44:20 -08:00
Anand Babu (AB) Periasamy
6fc11932ec make test -> test-all 2015-02-23 11:03:40 -08:00
Harshavardhana
a3fe948c8b Remove setup development environment script 2015-02-21 21:38:04 -08:00
Harshavardhana
741e14f749 Remove go cover and remove unncessary Makefiles 2015-02-20 17:44:26 -08:00
Harshavardhana
0a1a47da1b remove govet dependency for now 2015-02-18 15:51:31 -08:00
Harshavardhana
fab954f13f Run govet and fix 2015-02-18 15:33:55 -08:00
Harshavardhana
79455c3f9d No symlink, fail when not in GOPATH 2015-02-18 14:35:49 -08:00
Harshavardhana
0d91e7d540 Move to devscripts to avoid top level dir clutter 2015-02-14 17:28:55 -08:00
Harshavardhana
d036bf3a2a Restructure docs and move mkdocs.yml to top-level 2015-02-08 17:40:39 -08:00
Harshavardhana
19267fa29a Create the non-existent directory 2015-02-05 10:57:26 -08:00
Frederick F. Kautz IV
0646e7ae40 Fixing symlink test to not link if file exists 2015-02-03 17:15:57 -08:00
Harshavardhana
88af9ba666 Consolidate developers docs and Makefile, create symlink automatically 2015-02-01 21:18:46 -08:00
Frederick F. Kautz IV
84c5df8441 Using generic make test ./... and make build ./..., disabled signers code for now to fix build. 2015-01-30 10:58:47 -08:00
Harshavardhana
ac99968796 Add x509 tests and should be part of the build 2015-01-28 17:12:59 -08:00
Harshavardhana
d78cd581c5 Authorization validation in accordance with S3 signer AWSv2 2015-01-28 16:14:08 -08:00
Harshavardhana
07a1ad8b2d Remote syso files in 'make clean' 2015-01-27 13:13:25 -08:00
Harshavardhana
2547163cb2 Add erasure package in its full form v1.0 2015-01-27 12:55:17 -08:00
Harshavardhana
4c0804c5bc Cosmetic change 'minio' to be installed with Makefile 2015-01-25 17:45:00 -08:00
Harshavardhana
01d15ca3b2 Implement crc32c for Darwin, update documentation 2015-01-23 19:43:51 -08:00
Harshavardhana
3d43305b1f Make gcc mandatory on OSX, since clang lacks certain GAS features 2015-01-22 18:33:35 -08:00
Harshavardhana
8582391ef6 Add arch=osx for isal to compile properly on MacOSX 2015-01-21 15:04:54 -08:00
Harshavardhana
e7c74fdde2 Add ARCH flags 2015-01-21 13:52:42 -08:00
Harshavardhana
9401262f75 Remove linux specific code for now, bring them in later 2015-01-21 13:47:50 -08:00
Harshavardhana
5b67da7d96 Add PutBucket and ListBuckets service 2015-01-21 00:52:59 -08:00
Frederick F. Kautz IV
f550e84cf4 Removing old server
Removing storage drivers
2015-01-18 14:54:46 -08:00
Harshavardhana
68de9ac19e Further restructure 2015-01-14 12:40:43 -08:00
Anand Babu (AB) Periasamy
d4d43fa018 clean isal files as well 2015-01-14 11:53:46 -08:00
Harshavardhana
432275e966 Full restructure in accordance with
- pkg/{subsystem}/{package} style
  - modify Makefile to reflect the new style,
    consolidate various entries
  - add a dummy ``main.go`` at top level
2015-01-14 11:29:04 -08:00
Harshavardhana
f347a1e590 Merge with Intel ISAL changes from github.com/minio-io/isal
- These changes bring in a much needed Mac OSX port for
    Intel ISAL library
  - At the current stage this MacOSX part of code is
    considered beta
  - pkg/cpu now supports OSX
  - pkg/checksum/crc32c - is still WIP, rest of the packages
    have been validated
2015-01-11 00:39:39 -08:00
Frederick F. Kautz IV
992dea1bf3 Adding index command 2015-01-06 22:26:09 +13:00
Harshavardhana
fff6b589cf Rename split-file --> split 2015-01-03 18:03:21 -08:00
Harshavardhana
6b36b5c551 A full restructure 2014-12-29 21:22:26 -08:00
Harshavardhana
13650e088c Implement md5c function, slower than Golang's implementation
keeping it in repo to make further improvements and also rename

        minio-hash ---> crypto
2014-12-21 03:50:33 -08:00
Frederick F. Kautz IV
25c8bfee24 Adding minio-hash to install via Makefile 2014-12-21 23:16:29 +13:00
Frederick F. Kautz IV
ff55d40432 Adding md5, sha256, sha512 to Makefile 2014-12-21 13:40:09 +13:00
Harshavardhana
d4c05f11c7 Update documentation 2014-12-20 09:09:35 -08:00
Harshavardhana
5268a62166 Add minio-cli stub generator 2014-12-19 00:00:14 -08:00
Harshavardhana
e1b6f5391f Add pkgs/scsi to build 2014-12-17 03:36:10 -08:00
Harshavardhana
3ec5c2fb8f Implement commands
Commands implemented

   $ minio put <objectname> <file>
   $ minio encode <objectname>
   $ minio get <objectname>
   $ minio list

TODO :

   $ minio init
   $ minio verify
    ...
    ...
2014-12-15 14:59:15 -08:00
Frederick F. Kautz IV
05baa6d9a2 Removing fsstorage 2014-12-15 19:15:22 +13:00
Harshavardhana
0148237e22 Refactor checksum code, and add Objectname hashing per uploaded objects
Add placeholder sha256, sha512 implementations from Intel with AVX, AVX2
and SSE4.1 extensions - with some boilerplate Go code.
2014-12-13 21:42:23 -08:00
Harshavardhana
fc50291cac fsStorage should use appendStorage with offsets
- also takes in 'blockSize' cli option.
 - robustness fixes
2014-12-12 17:18:16 -08:00
Harshavardhana
235949eafe Build fixes after moving to go1.4 2014-12-11 20:57:00 -08:00
Harshavardhana
c0724b38b9 Ignore io.EOF for gob decoding 2014-12-11 01:45:58 -08:00
Frederick F. Kautz IV
568ca4e1bc erasure encoded demo 2014-12-11 00:32:32 -08:00
Frederick F. Kautz IV
e670608226 Fixing build 2014-12-10 17:43:16 -08:00
Harshavardhana
f6a5b61b00 Make get/put more robust
- init folder if not present already in 'get()' call
  - put.go was referencing wrong args - fix it
  - if no OBJECTNAME is specified during 'get' - grab a
    list of files under "~/.minio"
  - Provide new API GetList() for ObjectStorage struct
2014-12-09 03:32:31 -08:00
Harshavardhana
43680cd7b6 Add sha1 - AVX2 supported fast implementation, with some crc32c cleanup 2014-12-09 01:00:02 -08:00
Harshavardhana
7314b5e37d Fixes #71 - some crazy races inside erasure and simplify the code
- This change also brings in changing 'unsigned char' to 'uint8_t'
for brevity
2014-12-07 01:33:49 -08:00
Harshavardhana
c3fca84f25 Add git and mercurial as dependencies 2014-12-04 01:56:08 -08:00
Harshavardhana
fb34c5290c Fast CRC implementations ported from Intel's efforts
Provides fast CRC32C with PCLMULQDQ instructions in Golang

The white papers on CRC32C calculations with PCLMULQDQ instruction can be
  downloaded from:

http://www.intel.com/content/dam/www/public/us/en/documents/white-papers/crc-iscsi-polynomial-crc32-instruction-paper.pdf
http://www.intel.com/content/dam/www/public/us/en/documents/white-papers/fast-crc-computation-paper.pdf
2014-12-03 01:20:31 -08:00
Harshavardhana
6e4583ef1d Build restructure top level 2014-12-01 14:45:50 -08:00
Harshavardhana
a4676211cf Enable cover profile and test for split and strbyteconv 2014-12-01 10:40:50 -08:00
Harshavardhana
67e9a27b34 Enable way to split files at input bytes 2014-12-01 01:00:56 -08:00
Frederick F. Kautz IV
d1f9704f25 Fixing build where cover and godep cause a break when system is offline 2014-11-30 21:58:39 -08:00
Harshavardhana
fa1a13a8bf Cleanup makefile 2014-11-30 15:20:20 -08:00
Harshavardhana
eaf5379246 Update readme with clear instructions 2014-11-30 15:09:58 -08:00
Frederick F. Kautz IV
6382ee161e Renaming minio-demo to erasure-demo 2014-11-30 13:54:31 -08:00
Frederick F. Kautz IV
3c09029049 Merging encode and decode to single minio-demo binary 2014-11-29 22:48:37 -08:00
Frederick F. Kautz IV
03beef3afc Moving gateway and storage driver to packages 2014-11-29 14:42:22 -08:00
Harshavardhana
3cefaf5ad8 Restructure directory, add 'pkgs' folder now 2014-11-24 15:10:25 -08:00
Frederick F. Kautz IV
5a3eb47583 Adding minio-decode 2014-11-23 19:32:01 -08:00
Frederick F. Kautz IV
a15e935a33 Adding minio-encode binary 2014-11-22 18:39:59 -08:00
Matthew Farrellee
5047c055f6 stop requiring a cover directory
it is not automatically created. instead, store coverage output in the
current directory.
2014-11-22 21:13:56 -05:00
Frederick F. Kautz IV
9c18624e58 Removing binaries 2014-11-22 15:16:38 -08:00
Frederick F. Kautz IV
1a6b9ccfc8 Fixing isal compile from root, workaround for https://code.google.com/p/go/issues/detail?id=7891 2014-11-22 13:09:05 -08:00
Frederick F. Kautz IV
f8f9f65652 Moving minio command to cmd/minio 2014-11-14 18:20:40 -07:00
Frederick F. Kautz IV
9275afa173 Changing minios to minio 2014-11-07 22:23:52 -08:00
Frederick F. Kautz IV
5878dd5e50 Cleaning up more code, system generated by config 2014-11-06 22:45:27 -08:00
Frederick F. Kautz IV
0c68f52caf Adding bucket management system with in memory storage driver 2014-11-06 21:34:46 -05:00
Frederick F. Kautz IV
e98a77ff1a Moving to godep for dependency management 2014-11-03 20:59:07 -08:00
Frederick F. Kautz IV
fc6a2a45cb Setting up initial cli options and http handlers 2014-11-02 17:33:40 -08:00
Frederick F. Kautz IV
dca2f84bc9 Refactoring miniosd and minios to a single binary minio 2014-11-02 14:33:53 -05:00
Frederick F. Kautz IV
2c73655b7c Working build system, builds gopath dynamically 2014-11-01 19:44:30 -07:00
Frederick F. Kautz IV
26aad4eab2 Adding third_party dir support 2014-11-01 15:20:59 -07:00
Frederick F. Kautz IV
720ccbddee Adding initial makefile and main.go 2014-11-01 04:22:16 -04:00