Commit Graph

196 Commits

Author SHA1 Message Date
Bala FA b0deea27df Refactor s3select to support parquet. (#7023)
Also handle pretty formatted JSON documents.
2019-01-08 16:53:04 -08:00
Harshavardhana bebaff269c Support IPv6 in minio command line (#6947)
Fixes #6946
2018-12-14 13:07:46 +05:30
Harshavardhana 30040fba45 Add windows CI on travis (#6661) 2018-10-18 08:48:53 +05:30
Harshavardhana 0e02328c98 Migrate config.json from config-dir to backend (#6195)
This PR is the first set of changes to move the config
to the backend, the changes use the existing `config.json`
allows it to be migrated such that we can save it in on
backend disks.

In future releases, we will slowly migrate out of the
current architecture.

Fixes #6182
2018-08-15 10:11:47 +05:30
Durgadas Kamath 71979376b5 Add s390x support (#6263) 2018-08-08 15:49:12 -07:00
Rafael Peria de Sene 317e648c0d Add support for ppc64le (#6116)
Signed-off-by: Rafael Peria de Sene <rpsene@br.ibm.com>
2018-07-03 09:42:19 -07:00
Harshavardhana e5e522fc61
docs: fix all Chinese doc links for the new docs site (#6097)
Additionally fix typos, default to US locale words
2018-06-28 16:02:02 -07:00
kannappanr 577d10674d
Cleanup stack trace in error logs (#6045)
Add compile time GOROOT path to the list of prefix
of file paths to be removed.

Add webhandler function names to the slice that
stores function names to terminate logging.
2018-06-18 12:04:46 -07:00
Harshavardhana 73f7a98590 Enable go1.10.1 version (#5666) 2018-04-06 11:07:39 -07:00
Harshavardhana 565dc8b283 Do not check for GOPATH env for builds (#5628)
Since go1.8 GOPATH is not required to set prior, as
it defaults to "${HOME}/go" we only need to check if
go tool detected GOPATH correctly. If yes then we
proceed if not we fail.
2018-03-20 12:05:49 -07:00
Harshavardhana fb96779a8a Add large bucket support for erasure coded backend (#5160)
This PR implements an object layer which
combines input erasure sets of XL layers
into a unified namespace.

This object layer extends the existing
erasure coded implementation, it is assumed
in this design that providing > 16 disks is
a static configuration as well i.e if you started
the setup with 32 disks with 4 sets 8 disks per
pack then you would need to provide 4 sets always.

Some design details and restrictions:

- Objects are distributed using consistent ordering
  to a unique erasure coded layer.
- Each pack has its own dsync so locks are synchronized
  properly at pack (erasure layer).
- Each pack still has a maximum of 16 disks
  requirement, you can start with multiple
  such sets statically.
- Static sets set of disks and cannot be
  changed, there is no elastic expansion allowed.
- Static sets set of disks and cannot be
  changed, there is no elastic removal allowed.
- ListObjects() across sets can be noticeably
  slower since List happens on all servers,
  and is merged at this sets layer.

Fixes #5465
Fixes #5464
Fixes #5461
Fixes #5460
Fixes #5459
Fixes #5458
Fixes #5460
Fixes #5488
Fixes #5489
Fixes #5497
Fixes #5496
2018-02-15 17:45:57 -08:00
Harshavardhana c59f1e3a80 revamp minio build messages (#5519) 2018-02-14 10:29:19 +05:30
Harshavardhana fd3897d0c3 Move to go1.9.4 with recent security release (#5502) 2018-02-08 14:33:22 +05:30
Harshavardhana eb2894233c Convert gateways into respective packages (#5200)
- Make azure gateway a package
- Make b2 gateway a package
- Make gcs gateway a package
- Make s3 gateway a package
- Make sia gateway a package
2017-12-05 17:58:09 -08:00
Harshavardhana 9eb52ec7c7 Remove release scripts for minio. (#5181)
Use `GOOS=<osname> go build`  to build minio for any platform of choice.
2017-11-14 17:05:40 -08:00
Harshavardhana 3d0dced23c Remove go1.9 specific code for windows (#5033)
Following fix https://go-review.googlesource.com/#/c/41834/ has
been merged upstream and released with go1.9.
2017-10-13 15:31:15 +05:30
Bala FA d28b3d8801 Move to go1.9.1 as default environment. (#5041) 2017-10-09 22:23:59 -07:00
Harshavardhana c46c2a6dd6 Fix healthcheck issue with Docker containers (#5011)
Fixes #5010
2017-10-05 13:52:57 -07:00
Bala FA 670e3e0c8b verify: add minio gateway s3 setup for verification tests (#4976) 2017-10-04 11:16:39 -07:00
Bala FA 6d7df1d1cb Use mc functional tests for verifying build (#4934) 2017-09-20 13:22:05 -07:00
Bala FA 1bb3a03099 build: add verify check on `make test` (#4901)
This patch adds basic tests for FS/XL/Distribute setups.
2017-09-12 16:56:33 -07:00
Harshavardhana 58633a383a Deprecate intel-32, arm64, arm support for minio builds. (#4811)
It was decided that we will be deprecating ARM support
for minio builds. ARM users should simply compile from source.

Additionally 32bit version of Linux, Windows and FreeBSD (64bit)
are deprecated.
2017-08-14 11:08:58 -07:00
Nitish Tiwari d4b107adf4 Retry name lookup for kubernetes and docker swarm environment (#4800)
Wait for remote hosts to resolve instead of failing on first host
resolution error, when running in Kubernetes or Docker environment.

Note that

- Waiting is based on exponential back-off mechanism
- If run as a binary, server fails if remote host is not resolvable

This is needed because in orchestration platforms like Kubernetes, remote
hosts are started sequentially and all the hosts are not up initially,
though they are expected to come up in a short time frame
It is difficult to identify a cap on the waiting time due to
non-deterministic nature of infrastructure platforms, so the server waits
infinitely for the hosts to come up, while logging the error messages to
the console.

Fixes: https://github.com/minio/minio/issues/4669
2017-08-13 13:34:10 -07:00
Nitish Tiwari 53f84d6084 Report healthy status for initial 120s, then switch to healthcheck (#4799)
Swarm routes traffic only to containers that report healthy status,
while Minio in distributed mode needs to talk to other peers before
it can respond to healthcheck probe. As the Minio containers are not
able to talk to each other, distributed Minio is not getting started
on Docker Swarm.

With this PR, Minio Healthcheck report healthy status for initial
120s enough for distributed Minio to start. After that normal
Healthcheck resumes. Also changed the healthcheck method name
in accordance with Google shell styleguide.

Fixes: https://github.com/minio/minio/issues/4761
2017-08-12 19:26:35 -07:00
A. Elleuch b4dc6df35c go1.8: Changes to support golang 1.8 (#4759)
QuirkConn is added to replace net.Conn as a workaround to a golang bug:
https://github.com/golang/go/issues/21133
2017-08-06 11:27:33 -07:00
Nitish Tiwari f7889e1af7 Add TLS request for healthcheck (#4740) 2017-08-03 20:05:45 -07:00
Harshavardhana ba6d997d18 build: Error out properly when unsupported arch is given. (#4585) 2017-06-24 01:05:35 -07:00
Nitish Tiwari 9d3d64df2d Added HEALTHCHECK in release Dockerfiles (#4550)
A new script `healthcheck.sh` checks for http status 200/404
2017-06-20 19:24:45 -07:00
Harshavardhana f99987e47c Generate sha1sum as well for release for backward compatibility. (#4475)
Additionally remove support for arm6vl in release, since
go 1.8 the support for armv6 has been dropped and we do
not see high usage events from this platform.
2017-06-06 11:25:06 -07:00
Harshavardhana c63afabc9b build/release: Generate sha256sums also without the release tag. (#4318)
Ref #4306
2017-05-12 21:40:22 -07:00
Harshavardhana fa3d5d0f46 build/release: Generate sha256sums for built binaries. (#4311)
We used to build sha1sum deprecate it and
use sha256sum instead.

Fixes #4306
2017-05-10 11:22:05 -07:00
Harshavardhana e372b5ed67 build: Fix release build names. (#4263)
Currently due to the occurrence of 6 arguments from
`gen-ldflags.go` leads to a bug where the binaries
genenerated have wrong names.

As shown below.

```
If you want to build for all, Just press Enter: linux/amd64
-->    linux/amd64:github.com/minio/minio
$ ls release/linux-amd64/
[2017-05-04 23:08:51 PDT]  17MiB minio
[2017-05-04 23:08:51 PDT]  17MiB minio.2017-05-05T06:08:22Z
[2017-05-04 23:08:51 PDT]    76B minio.shasum
```

This PR fixes this issue by retaining the previous release
binary names.

```
If you want to build for all, Just press Enter: linux/amd64
-->    linux/amd64:github.com/minio/minio
$ ls release/linux-amd64/
[2017-05-04 23:08:51 PDT]  17MiB minio
[2017-05-04 23:08:51 PDT]  17MiB minio.RELEASE.2017-05-05T06-08-22Z
[2017-05-04 23:08:51 PDT]    76B minio.shasum
```
2017-05-05 13:16:58 -07:00
Harshavardhana 0ea8bfaf78 Add waiting on hosts in docker entrypoint for distributed setups. (#4244)
Thanks to Remco Verhoef <remco@dutchcoders.io> for the script.

Fixes #4225
2017-05-04 00:48:13 -07:00
Harshavardhana dc365bca44 build: -s -w should be added by gen-ldflags.go (#4172) 2017-04-24 23:01:38 -07:00
Peter Tribble 2b96d9f706 Enable build on solaris (#4115) 2017-04-23 11:10:18 -07:00
Harshavardhana 6e9ac8db59 docker: Support docker swarm secrets. (#3977)
Fixes #3896
2017-04-08 01:43:40 -07:00
Harshavardhana 28eff0f6c1 build: Improve build messaging, say where we built Minio. (#3973) 2017-03-25 00:33:57 -07:00
Krishnan Parthasarathi e5773e11c6 Make minio server compile on OpenBSD, NetBSD, Solaris (#3719) 2017-02-08 22:27:35 -08:00
Harshavardhana 50796e481d build: Add ARM binary builds for ARMv6 and Aarch64. (#3577) 2017-01-13 18:51:17 -08:00
Matthew Hall cde6496172 checkdeps.sh: support unusual git version strings (for OS X et. al.) (#3512) 2016-12-31 19:57:33 -08:00
Harshavardhana f9025652df Support building specific architectures. 2016-12-27 17:52:24 -08:00
tibbes 33c022fcec Fix checkdeps.sh on Mac (#3306)
Update the check_minimum_version function to use numeric comparison (not
string comparison) on components of version numbers. Fixes the following
output:

```
$ make
Checking deps:
ERROR
OSX version '10.11.6' not supported.
Minimum supported version: 10.8
make: *** [checks] Error 1
```
2016-11-21 12:25:46 -08:00
Anis Elleuch daf6f3a5c0 Fix path comparing in checkgopath (#3215) 2016-11-09 10:09:42 -08:00
Bala FA 5eb4002bf7 cleanup build scripts. (#3192) 2016-11-07 13:28:41 -08:00
Harshavardhana 62dcee3b14 dist: Moved systemd scripts to minio-systemd. (#3136)
Removing this from the repo.
2016-10-31 02:37:32 -07:00
Harshavardhana f72163f856 build: Deprecate requirement of GOROOT (#2803) 2016-09-28 18:49:16 -07:00
Anis Elleuch ef22330563 Require go 1.7.1 to build Minio server (#2727) 2016-09-22 10:33:52 -07:00
Harshavardhana bccf549463 server: Move all the top level files into cmd folder. (#2490)
This change brings a change which was done for the 'mc'
package to allow for clean repo and have a cleaner
github drop in experience.
2016-08-18 16:23:42 -07:00
Matthieu Fronton 402af93da2 Update how-to-install-golang URL (#2406) 2016-08-11 12:01:12 -07:00
Harshavardhana 90c20a8c11 Add codecov for minio. (#2359) 2016-08-04 16:48:50 -07:00
Anand Babu (AB) Periasamy 63c65b4635 filter GOPATH from stack trace (#1755) 2016-05-25 02:32:35 -07:00
Harshavardhana 93666827f4 release: Add freebsd/amd64 build and remove zip, tgz. (#1316) 2016-04-13 23:34:55 -07:00
Harshavardhana 379e0abf03 cleanup: Remove old donut/xl code and erasure implementation.
This is a change to bring in 'klauspost/reedsolomon' library
in #1270 patch.
2016-04-02 17:30:35 -07:00
Harshavardhana 59ee5a547c release: gz doesn't preserve permissions use tar.gz
And fix various other issues with release script.
2016-03-26 23:44:32 -07:00
Anis Elleuch 663f24064b Add simple FreeBSD support, make the minio project compilable 2016-03-26 22:39:34 +01:00
Harshavardhana 41cba3a457 buildscripts: compress release binaries.
Fix update command as well to show compressed files in updates.
2016-03-18 23:30:54 -07:00
Sreeram Boyapati 62bd44f873 buildscripts: Enable user to choose an arch to build
- Building minio for all architectures takes a lot of time.
    Choose the one user needs
2016-03-11 10:17:56 +05:30
Harshavardhana 04d6408c31 Merge pull request #1167 from harshavardhana/fix-release-tag
build: Fix release tag.
2016-02-23 17:16:26 -08:00
Harshavardhana 223245cc45 build: Add release builds, now generated with 'make release'
Currently supported platforms are

    - linux{amd64,arm,386}
    - winows{amd64,386}
    - darwin{amd64}
2016-02-23 15:14:02 -08:00
Harshavardhana 408aa72146 build/vet: Fix all the shadowing reports with go1.6
Golang 1.6 is default version for the build now.

Additionally set 'GODEBUG=cgocheck=0' for now, until
we fix the erasure coding package.

Readmore here https://tip.golang.org/doc/go1.6#cgo
2016-02-23 14:34:39 -08:00
Harshavardhana 2181003609 web: Removing dependency for gpg and downloading assets.
Assets are vendorized from now on and updated for each release.
2016-02-23 13:32:12 -08:00
Harshavardhana 18fd0a0f81 build: Verify yasm version and complain - fixes #1154 2016-02-22 03:17:17 -08:00
Harshavardhana 98ee5fcf55 build: Add spelling checks and check if curl is installed. 2016-02-10 00:18:05 -08:00
Harshavardhana ddc99e3112 build: Add build dependency check for 'gpg' 2016-02-05 03:10:23 -08:00
Krishna Srinivas f77851bee0 docker: second --ldflags was overriding the first --ldflags option 2015-11-07 15:21:01 -08:00
Harshavardhana 7845515f36 build: Versioning now populated through ldflags 2015-11-02 02:37:26 -08:00
Anand Babu (AB) Periasamy 588833d06f build time constants 2015-10-26 02:41:04 -07:00
Harshavardhana 32898c72fa Add arm support for build scripts 2015-10-20 12:03:03 -07:00
Harshavardhana 762b798767 Migrate this project to minio micro services code 2015-10-16 11:26:08 -07:00
Harshavardhana 39c8991e5f Do not use fmt.Println with formatting strings 2015-09-23 09:21:11 -07:00
Harshavardhana d54488f144 Move all server and controller packages into top-level 2015-09-19 01:07:42 -07:00
Anand Babu (AB) Periasamy d1f1b7ac31 new version format and some cleanup 2015-09-18 23:27:04 -07:00
Harshavardhana b5246dbd7d Add version checks for golang 1.5.1 2015-09-17 16:34:06 -07:00
Harshavardhana 988d39a5b6 Migrate to golang1.5 release with GO15VENDOREXPERIMENT=1 enabled 2015-08-22 18:35:37 -07:00
Harshavardhana 61175ef091 Migrate to govendor to avoid limitations of godep
- over the course of a project history every maintainer needs to update
  its dependency packages, the problem essentially with godep is manipulating
  GOPATH - this manipulation leads to static objects created at different locations
  which end up conflicting with the overall functionality of golang.

  This also leads to broken builds. There is no easier way out of this other than
  asking developers to do 'godep restore' all the time. Which perhaps as a practice
  doesn't sound like a clean solution. On the other hand 'godep restore' has its own
  set of problems.

- govendor is a right tool but a stop gap tool until we wait for golangs official
  1.5 version which fixes this vendoring issue once and for all.

- govendor provides consistency in terms of how import paths should be handled unlike
  manipulation GOPATH.

  This has advantages
    - no more compiled objects being referenced in GOPATH and build time GOPATH
      manging which leads to conflicts.
    - proper import paths referencing the exact package a project is dependent on.

 govendor is simple and provides the minimal necessary tooling to achieve this.

 For now this is the right solution.
2015-08-12 19:24:57 -07:00
Harshavardhana c503bf412f Fix dependency checking on osx 2015-07-27 18:19:22 -07:00
Harshavardhana 63c9cf0c4b Move from Minimalist Object Storage to Minio Cloud Storage 2015-07-24 17:51:40 -07:00
Harshavardhana e4543489fe Use updatedeps script to update godeps 2015-07-16 12:57:32 -07:00
Harshavardhana 75788c7a1d Fix go installation check on amazon instance 2015-07-05 18:12:58 -07:00
Anis Elleuch 4bed0aa526 Check if go binary belongs to the go installation pointed by GOROOT env 2015-06-23 17:44:05 +01:00
Harshavardhana 7a060110ff Fix BUILDDEPS paths for golang installation 2015-06-22 12:07:23 -07:00
Harshavardhana 8bde4d4e34 Migrating minio server build to do 'go run make.go' style like mc 2015-06-16 20:23:02 -07:00
Harshavardhana 5d484e2770 Implement custom date command for cross platform portability 2015-06-04 12:32:23 -07:00
Harshavardhana de56909d55 More scripts and code from Minio-io to minio 2015-05-11 16:43:38 -07:00
Anand Babu (AB) Periasamy b010fd0ff3 Version is now based on MD5SUM of its binary 2015-04-24 21:51:52 -07:00
Harshavardhana 97d1f03a87 Fix build for travis 2015-04-11 17:31:01 -07:00
Harshavardhana 92136d49fd Update cli to minio-io/cli 2015-04-02 12:49:40 -07:00
Anand Babu (AB) Periasamy 3c4aa85c69 git commit hash generator 2015-03-25 22:57:30 -07:00
Harshavardhana 58082cd8dc Add gocyclo into source build, choosing cyclomatic complexity tolerance of 15 and below 2015-03-25 15:57:17 -07:00
Harshavardhana 2b0b5f014a Update license header Mini should be Minimalist really 2015-03-19 14:35:50 -07:00
Frederick F. Kautz IV 34c1e58332 Renaming Package to goPackage to fix golint error 2015-03-03 12:56:19 -08:00
Harshavardhana 3a3c8645fc Update comments across the codebase 2015-03-03 02:39:38 -08:00
Harshavardhana 137584d658 Add comments 2015-03-03 01:25:45 -08:00
Harshavardhana 721ec9ef5a Force exported functions to have comments 2015-03-03 01:01:38 -08:00
Harshavardhana c3ad0906e0 Add deadcode code which recursivley goes into all directories and verifies dangling variables. 2015-03-02 18:44:20 -08:00
Harshavardhana a3fe948c8b Remove setup development environment script 2015-02-21 21:38:04 -08:00