Commit Graph

226 Commits

Author SHA1 Message Date
Harshavardhana 4bba2cd034
fix: disallow versioning to be suspended with object lock (#9930) 2020-06-28 08:15:15 -07:00
Harshavardhana 5e529a1c96
simplify context timeout for readiness (#9772)
additionally also add CORS support to restrict
for specific origin, adds a new config and
updated the documentation as well
2020-06-04 14:58:34 -07:00
Harshavardhana 1bc32215b9
enable full linter across the codebase (#9620)
enable linter using golangci-lint across
codebase to run a bunch of linters together,
we shall enable new linters as we fix more
things the codebase.

This PR fixes the first stage of this
cleanup.
2020-05-18 09:59:45 -07:00
Harshavardhana 886ae15464
trimpaths when building minio binaries (#9246) 2020-04-01 10:45:11 -07:00
Harshavardhana d8af244708
Add numeric/date policy conditions (#9233)
add new policy conditions

- NumericEquals
- NumericNotEquals
- NumericLessThan
- NumericLessThanEquals
- NumericGreaterThan
- NumericGreaterThanEquals
- DateEquals
- DateNotEquals
- DateLessThan
- DateLessThanEquals
- DateGreaterThan
- DateGreaterThanEquals
2020-04-01 00:04:25 -07:00
Klaus Post 8d98662633
re-implement data usage crawler to be more efficient (#9075)
Implementation overview: 

https://gist.github.com/klauspost/1801c858d5e0df391114436fdad6987b
2020-03-18 16:19:29 -07:00
Harshavardhana 5aa5dcdc6d
lock: improve locker initialization at init (#8776)
Use reference format to initialize lockers
during startup, also handle `nil` for NetLocker
in dsync and remove *errorLocker* implementation

Add further tuning parameters such as

 - DialTimeout is now 15 seconds from 30 seconds
 - KeepAliveTimeout is not 20 seconds, 5 seconds
   more than default 15 seconds
 - ResponseHeaderTimeout to 10 seconds
 - ExpectContinueTimeout is reduced to 3 seconds
 - DualStack is enabled by default remove setting
   it to `true`
 - Reduce IdleConnTimeout to 30 seconds from
   1 minute to avoid idleConn build up

Fixes #8773
2020-01-10 02:35:06 -08:00
Harshavardhana 2ab8d5e47f Enable build verification with race (#8583) 2019-12-02 15:54:26 -08:00
Harshavardhana 6a4ef2e48e Initialize configs correctly, move notification config (#8367)
This PR also removes deprecated tests, adds checks
to avoid races reproduced on CI/CD.
2019-10-09 11:41:15 +05:30
Harshavardhana 290ad0996f Move etcd, logger, crypto into their own packages (#8366)
- Deprecates _MINIO_PROFILER, `mc admin profile` does the job
- Move ENVs to common location in cmd/config/
2019-10-08 11:17:56 +05:30
Krishnan Parthasarathi 1127293863 Add a basic .dockerignore file to reduce docker context in `make docker` (#8282)
- Add useful default tag to dev docker image build using `make docker`
2019-09-21 04:49:56 +05:30
Vivian Kong fa32c71a56 Build staticcheck as pre-built binaries are not available for s390x (#8220) 2019-09-17 05:59:15 +05:30
Harshavardhana 5512baab21 Fix go mod tidy errors and fix platform independent downloads (#8222)
Refer https://github.com/gin-gonic/gin/issues/1673
2019-09-12 04:57:59 +05:30
Harshavardhana 94e5cb7576
Migrate to go1.13 to avail all new features (#8203)
Read more https://blog.golang.org/go1.13
2019-09-08 16:44:15 -07:00
Nitish Tiwari e5fb6294a7 Remove healthcheck script for Docker image (#8095)
There are multiple possibilities for running MinIO within
a container e.g. configurable address, non-root user etc.
This makes it difficult to identify actual IP / Port to
use to check healthcheck status from within a container.

It is simpler to use external healthcheck mechanisms
like healthcheck command in docker-compose to check
for MinIO health status. This is similar to how checks
work in Kubernetes as well.

This PR removes the healthcheck script used inside
Docker container and ad documentation on how to
use docker-compose based healthcheck mechanism.
2019-08-17 12:44:04 -07:00
Harshavardhana b83413b167 Use GOPROXY to speed up builds (#7984)
Read more here https://proxy.golang.org proposal 
for go1.13
2019-07-30 22:27:11 +05:30
Harshavardhana 55dd017e62 Deprecate auto detection of container user (#7930)
There is no reliable way to handle fallbacks for
MinIO deployments, due to various command line
options and multiple locations which require
access inside container.

Parsing command line options is tricky to figure
out which is the backend disk etc, we did try
to fix this in implementations of check-user.go
but it wasn't complete and introduced more bugs.

This PR simplifies the entire approach to rather
than running Docker container as non-root by default
always, it allows users to opt-in. Such that they
are aware that that is what they are planning to do.

In-fact there are other ways docker containers can
be run as regular users, without modifying our
internal behavior and adding more complexities.
2019-07-17 19:20:55 +01:00
Harshavardhana 91ceae23d0 Add support for customizable user (#7569) 2019-06-10 20:27:42 +05:30
Harshavardhana a69f74533c Add region as part of error XML (#7752) 2019-06-05 16:28:21 -07:00
Joe Stevens a19cf063b5 Fixes for multiplatform dev and testing from forks (#7734)
Add support for correct dependency URLs on all platforms

only build mountinfo.go on linux

make testfile path relative to support fork work
2019-06-04 00:59:40 -07:00
Harshavardhana a71e08efb4
On macOS avoid using GNU specific options (#7621)
Fix Makefile from `cp -uf` to `cp -f`. We are only
interested in `-f` anyways.
2019-05-07 10:39:26 -07:00
Harshavardhana c90999df98 Valid if bucket names are internal (#7476)
This commit fixes a privilege escalation issue against
the S3 and web handlers. An authenticated IAM user
can:

- Read from or write to the internal '.minio.sys'
bucket by simply sending a properly signed
S3 GET or PUT request. Further, the user can
- Read from or write to the internal '.minio.sys'
bucket using the 'Upload'/'Download'/'DownloadZIP'
API by sending a "browser" request authenticated
with its JWT token.
2019-04-03 23:10:37 -07:00
Harshavardhana 313a3a286a Migrate to go1.12 to simplify our cmd/http package (#7302)
Simplify the cmd/http package overall by removing
custom plain text v/s tls connection detection, by
migrating to go1.12 and choose minimum version
to be go1.12

Also remove all the vendored deps, since they
are not useful anymore.
2019-04-02 18:28:39 -07:00
Anis Elleuch e13c99ed82 Makefile: Fix getting misspell code when already installed (#7434)
Grouping misspell installation code so it won't executed when
misspell is found in the current system.
2019-03-27 16:16:17 -07:00
Sidhartha Mani 6bc0de2a75 add go modules file and start running go 1.11 style builds (#7354) 2019-03-19 13:50:58 -07:00
Harshavardhana 2232b0b55f Fix mac build failure for healthcheck binary (#7263) 2019-02-21 08:11:16 +05:30
Nitish Tiwari 1e82c4a7c4
Implement Docker healthcheck script in Go (#7105)
Go script makes it easy to read/maintain. Also updated the timeout
in Dockerfiles from 5s to default 30s and test interval to 1m

Higher timeout makes sense as server may sometimes respond slowly
if under high load as reported in #6974

Fixes #6974
2019-02-20 21:42:03 +05:30
Harshavardhana df35d7db9d Introduce staticcheck for stricter builds (#7035) 2019-02-13 18:29:36 +05:30
Harshavardhana 8e0910ab3e Fix build issues on BSDs in pkg/cpu (#7116)
Also add a cross compile script to test always cross
compilation for some well known platforms and architectures
, we support out of box compilation of these platforms even
if we don't make an official release build.

This script is to avoid regressions in this area when we
add platform dependent code.
2019-01-22 09:27:23 +05:30
Bala FA b0deea27df Refactor s3select to support parquet. (#7023)
Also handle pretty formatted JSON documents.
2019-01-08 16:53:04 -08:00
Harshavardhana b5280ba243
Migrate to Go version 1.11.4 (#7026) 2018-12-28 14:04:39 -08:00
Pontus Leitzler f9779b24ad Enable default vet flags (#6810)
Enable default vet flags except experimental
2018-11-14 10:23:44 -08:00
Andreas Auernhammer f09e7ca764 fix travis CI build (#6620)
This commit fixes the Travis CI build by
correcting the golint import path
2018-10-11 14:58:44 -07:00
Harshavardhana ec2295c3dc Quickly build dev docker images using 'make docker' (#6505)
This PR simplifies the process of developer build of local
docker containers using `make docker`.

You need to provide a TAG i.e

```
TAG=y4m4/minio:exp make docker
```
2018-09-25 10:33:25 -07:00
Harshavardhana a13cd7b7c4 Separate build functional tests from Makefile (#6351)
Recently travis seems to have issues with builds after
merge, so this PR is an experiment to fix this.

Failed: https://travis-ci.org/minio/minio/builds/419769285#L2310

Succeeded: https://travis-ci.org/minio/minio/builds/419565606#L2322

This PR was tested to be working properly on my fork to build
on travis, would need to do the same in minio/minio and see
how it goes through.
2018-08-28 13:57:01 +05:30
Arjun Mishra 7c14cdb60e S3 Select API Support for CSV (#6127)
Add support for trivial where clause cases
2018-08-15 03:30:19 -07:00
Harshavardhana e5e522fc61
docs: fix all Chinese doc links for the new docs site (#6097)
Additionally fix typos, default to US locale words
2018-06-28 16:02:02 -07:00
kannappanr d94500ae26
Makefile: Create bin directory if it doesn't exist (#6050) 2018-06-18 11:45:28 -07:00
Harshavardhana 74328c3061 Implement auto cert reloading (#5963) 2018-05-31 12:30:15 -07:00
Harshavardhana c59f1e3a80 revamp minio build messages (#5519) 2018-02-14 10:29:19 +05:30
Harshavardhana eb2894233c Convert gateways into respective packages (#5200)
- Make azure gateway a package
- Make b2 gateway a package
- Make gcs gateway a package
- Make s3 gateway a package
- Make sia gateway a package
2017-12-05 17:58:09 -08:00
Aditya Manthramurthy d1a6c32d80 Improve make and make install messages (#5207) 2017-11-21 16:22:01 -08:00
Harshavardhana 9eb52ec7c7 Remove release scripts for minio. (#5181)
Use `GOOS=<osname> go build`  to build minio for any platform of choice.
2017-11-14 17:05:40 -08:00
Bala FA d28b3d8801 Move to go1.9.1 as default environment. (#5041) 2017-10-09 22:23:59 -07:00
Bala FA 88938340b3 remove all dead codes (#5019)
Fixes #5012
2017-10-05 12:25:45 -07:00
Bala FA 1bb3a03099 build: add verify check on `make test` (#4901)
This patch adds basic tests for FS/XL/Distribute setups.
2017-09-12 16:56:33 -07:00
Brendan Ashworth c59b995f7b build: ditch verifiers on make (#4679)
This commit ditches running verifiers automatically when just building
the server. It retains the verifiers when running tests.

There is very little point to running the verifiers each time a
developer builds the library but has no intent of running the tests.
They're expensive in time; this commit halves the build time on my
system, from 57 seconds to 29 seconds. This is because verifiers updates
the libraries from GitHub each time, which is slightly wasteful.
Additionally, computing cyclomatic complexity is expensive
computationally and isn't necessary to build the library.

Additionally, this allows the library to be built offline. It no longer
requires internet to run make.
2017-07-15 12:12:03 -07:00
Harshavardhana a86dc8a4c5 cleanup makefile and enable CGO_ENABLED=0 (#4598) 2017-06-26 18:07:06 -07:00
Harshavardhana 432bf7d99e Fail if formatting is wrong in our CI tests. (#4459)
We didn't fail before, we should helps in avoiding
formatting issues to creep into the codebase.
2017-06-02 14:05:51 -07:00
Harshavardhana b78f6fbcc5 Do not send envVars in ServerInfo() (#4422)
Sending envVars along with access and secret
exposes the entire minio server's sensitive
information. This will be an unexpected
situation for all users.

If at all we need to look for things like if
credentials are set through env, we should
only have access to only this information
not the entire set of system envs.
2017-05-24 21:09:23 -07:00