This PR also tries to simplify the approach taken in
object-locking implementation by preferential treatment
given towards full validation.
This in-turn has fixed couple of bugs related to
how policy should have been honored when ByPassGovernance
is provided.
Simplifies code a bit, but also duplicates code intentionally
for clarity due to complex nature of object locking
implementation.
Change distributed locking to allow taking bulk locks
across objects, reduces usually 1000 calls to 1.
Also allows for situations where multiple clients sends
delete requests to objects with following names
```
{1,2,3,4,5}
```
```
{5,4,3,2,1}
```
will block and ensure that we do not fail the request
on each other.
we don't need to validateFormats again once we have obtained
reference format, because it is possible that at this stage
another server is doing a disk heal during startup, once
in a while due to delays we get false positives and our
server doesn't start.
Format in quorum as reference format can be assumed as valid
and we proceed further, until and unless HealFormat re-inits
the disks after a successful heal.
Also use separate port for healing tests to avoid any
conflicts with regular build testing.
Fixes#8884
This is to fix a situation where an object name incorrectly
is sent with '//' in its path heirarchy, we should reject
such object names because they may be hashed to a set where
the object might not originally belong because, this can
cause situations where once object is uploaded we cannot
delete it anymore.
Fixes#8873
Zone abstraction of object layer was returning `nil`
incorrectly under situations where disk healing is
not required. Returning `nil` is considered as healing
successful, which leads to unexpected ReloadFormat()
peer notification calls during startup.
This PR fixes this behavior properly for zones.
Use reference format to initialize lockers
during startup, also handle `nil` for NetLocker
in dsync and remove *errorLocker* implementation
Add further tuning parameters such as
- DialTimeout is now 15 seconds from 30 seconds
- KeepAliveTimeout is not 20 seconds, 5 seconds
more than default 15 seconds
- ResponseHeaderTimeout to 10 seconds
- ExpectContinueTimeout is reduced to 3 seconds
- DualStack is enabled by default remove setting
it to `true`
- Reduce IdleConnTimeout to 30 seconds from
1 minute to avoid idleConn build up
Fixes#8773
Fixes an issue reported by @klauspost and @vadmeste
This PR also allows users to expand their clusters
from single node XL deployment to distributed mode.
Currently, the backend minio server uses the same data directory
as the mint test itself, causing `s3 sync` to fail often.
Now `minio` backend will use a different data directory `/data`
instead of `/mint/data`
Simplify the cmd/http package overall by removing
custom plain text v/s tls connection detection, by
migrating to go1.12 and choose minimum version
to be go1.12
Also remove all the vendored deps, since they
are not useful anymore.
Also add a cross compile script to test always cross
compilation for some well known platforms and architectures
, we support out of box compilation of these platforms even
if we don't make an official release build.
This script is to avoid regressions in this area when we
add platform dependent code.
This PR is the first set of changes to move the config
to the backend, the changes use the existing `config.json`
allows it to be migrated such that we can save it in on
backend disks.
In future releases, we will slowly migrate out of the
current architecture.
Fixes#6182
Add compile time GOROOT path to the list of prefix
of file paths to be removed.
Add webhandler function names to the slice that
stores function names to terminate logging.
Since go1.8 GOPATH is not required to set prior, as
it defaults to "${HOME}/go" we only need to check if
go tool detected GOPATH correctly. If yes then we
proceed if not we fail.
This PR implements an object layer which
combines input erasure sets of XL layers
into a unified namespace.
This object layer extends the existing
erasure coded implementation, it is assumed
in this design that providing > 16 disks is
a static configuration as well i.e if you started
the setup with 32 disks with 4 sets 8 disks per
pack then you would need to provide 4 sets always.
Some design details and restrictions:
- Objects are distributed using consistent ordering
to a unique erasure coded layer.
- Each pack has its own dsync so locks are synchronized
properly at pack (erasure layer).
- Each pack still has a maximum of 16 disks
requirement, you can start with multiple
such sets statically.
- Static sets set of disks and cannot be
changed, there is no elastic expansion allowed.
- Static sets set of disks and cannot be
changed, there is no elastic removal allowed.
- ListObjects() across sets can be noticeably
slower since List happens on all servers,
and is merged at this sets layer.
Fixes#5465Fixes#5464Fixes#5461Fixes#5460Fixes#5459Fixes#5458Fixes#5460Fixes#5488Fixes#5489Fixes#5497Fixes#5496
It was decided that we will be deprecating ARM support
for minio builds. ARM users should simply compile from source.
Additionally 32bit version of Linux, Windows and FreeBSD (64bit)
are deprecated.
Wait for remote hosts to resolve instead of failing on first host
resolution error, when running in Kubernetes or Docker environment.
Note that
- Waiting is based on exponential back-off mechanism
- If run as a binary, server fails if remote host is not resolvable
This is needed because in orchestration platforms like Kubernetes, remote
hosts are started sequentially and all the hosts are not up initially,
though they are expected to come up in a short time frame
It is difficult to identify a cap on the waiting time due to
non-deterministic nature of infrastructure platforms, so the server waits
infinitely for the hosts to come up, while logging the error messages to
the console.
Fixes: https://github.com/minio/minio/issues/4669
Swarm routes traffic only to containers that report healthy status,
while Minio in distributed mode needs to talk to other peers before
it can respond to healthcheck probe. As the Minio containers are not
able to talk to each other, distributed Minio is not getting started
on Docker Swarm.
With this PR, Minio Healthcheck report healthy status for initial
120s enough for distributed Minio to start. After that normal
Healthcheck resumes. Also changed the healthcheck method name
in accordance with Google shell styleguide.
Fixes: https://github.com/minio/minio/issues/4761
Additionally remove support for arm6vl in release, since
go 1.8 the support for armv6 has been dropped and we do
not see high usage events from this platform.
Currently due to the occurrence of 6 arguments from
`gen-ldflags.go` leads to a bug where the binaries
genenerated have wrong names.
As shown below.
```
If you want to build for all, Just press Enter: linux/amd64
--> linux/amd64:github.com/minio/minio
$ ls release/linux-amd64/
[2017-05-04 23:08:51 PDT] 17MiB minio
[2017-05-04 23:08:51 PDT] 17MiB minio.2017-05-05T06:08:22Z
[2017-05-04 23:08:51 PDT] 76B minio.shasum
```
This PR fixes this issue by retaining the previous release
binary names.
```
If you want to build for all, Just press Enter: linux/amd64
--> linux/amd64:github.com/minio/minio
$ ls release/linux-amd64/
[2017-05-04 23:08:51 PDT] 17MiB minio
[2017-05-04 23:08:51 PDT] 17MiB minio.RELEASE.2017-05-05T06-08-22Z
[2017-05-04 23:08:51 PDT] 76B minio.shasum
```
Update the check_minimum_version function to use numeric comparison (not
string comparison) on components of version numbers. Fixes the following
output:
```
$ make
Checking deps:
ERROR
OSX version '10.11.6' not supported.
Minimum supported version: 10.8
make: *** [checks] Error 1
```
Golang 1.6 is default version for the build now.
Additionally set 'GODEBUG=cgocheck=0' for now, until
we fix the erasure coding package.
Readmore here https://tip.golang.org/doc/go1.6#cgo
