Commit Graph

6015 Commits

Author SHA1 Message Date
Praveen raj Mani
2b9b907f9c Enable event persistence in Redis (#7601) 2019-07-23 10:22:08 -07:00
Daryl Finlay
9389a55e5d Cancel PutObjectPart on upload abort (#7940)
Calling ListMultipartUploads fails if an upload is aborted while a
part is being uploaded because the directory for the upload exists
(since fsRenameFile ends up calling os.MkdirAll) but the meta JSON file
doesn't. To fix this we make sure an upload hasn't been aborted during
PutObjectPart by checking the existence of the directory for the upload
while moving the temporary part file into it.
2019-07-22 22:36:15 -07:00
Harshavardhana
87e6533cf3 Add some design docs for distributed setup (#7950) 2019-07-23 07:48:10 +05:30
Christian Muehlhaeuser
38bc3a45db Fixed tautological conditions (#7959)
We already check for err being equal to nil above, no need
to check again.
2019-07-22 17:06:08 -07:00
Christian Muehlhaeuser
c5faba55c1 Comment: Typo Fix (#7958) 2019-07-21 05:55:09 +01:00
Harshavardhana
8b5e6e338c Fix: Only add SRV records that match the bucket name exactly (#7957)
Problem: MinIO incorrectly appends DNS SRV records of buckets that have a prefix match with a given bucket. E.g bucket1 would incorrectly get bucket's DNS records too.
Solution: This fix ensures that we only add SRV records that match the key exactly
2019-07-20 11:29:05 +01:00
poornas
0373a1699b Add error filter to admin trace API (#7923)
This allows MinIO to have the ability to send back only error trace
2019-07-20 01:38:26 +01:00
Krishnan Parthasarathi
559a59220e Add initial support for bucket lifecycle (#7563)
This PR is based off @sinhaashish's PR for object lifecycle
management, which includes support only for,
- Expiration of object
- Filter using object prefix (_not_ object tags)

N B the code for actual expiration of objects will be included in a
subsequent PR.
2019-07-19 21:20:33 +01:00
Yao Zongyou
59e1763816 doc: use make instead of go command to test changes (#7951) 2019-07-19 14:40:39 +01:00
poornas
041a812ba0 trace api: add call stats to trace (#7915)
Stats such as call latency, bytes received and sent have been added
2019-07-18 23:29:17 +01:00
Krishnan Parthasarathi
fbfc9a61ec Add node address information to logs (#7941) 2019-07-18 09:58:37 -07:00
Philipp Dallig
be9baa1464 Fix startup without MINIO_USERNAME and MINIO_GROUPNAME (#7944) 2019-07-18 21:49:49 +05:30
Minio Trusted
b058e32348 Update yaml files to latest version RELEASE.2019-07-17T22-54-12Z 2019-07-17 22:59:33 +00:00
Lucas
ea66a52ed1 Add KMS master key from Docker secret (#7825) 2019-07-17 20:55:26 +01:00
Harshavardhana
55dd017e62 Deprecate auto detection of container user (#7930)
There is no reliable way to handle fallbacks for
MinIO deployments, due to various command line
options and multiple locations which require
access inside container.

Parsing command line options is tricky to figure
out which is the backend disk etc, we did try
to fix this in implementations of check-user.go
but it wasn't complete and introduced more bugs.

This PR simplifies the entire approach to rather
than running Docker container as non-root by default
always, it allows users to opt-in. Such that they
are aware that that is what they are planning to do.

In-fact there are other ways docker containers can
be run as regular users, without modifying our
internal behavior and adding more complexities.
2019-07-17 19:20:55 +01:00
Kanagaraj M
12353caf35 Fix: Support Unicode delimiters in s3 select (#7931) 2019-07-17 19:10:17 +01:00
Harshavardhana
a57c747667 Document vault in prod mode instead of dev mode (#7928) 2019-07-16 01:32:15 +01:00
Anis Elleuch
28661c0413 heal: Trigger auto-heal once each month instead of 24 hours (#7934) 2019-07-16 00:03:42 +01:00
Harshavardhana
04a152be12 Redirect to browser only if browser is enabled (#7914) 2019-07-15 20:01:17 +01:00
Harshavardhana
bce3f8237d Allow users to give anonymous access (#7926)
Current code already allows users to GetPolicy/SetPolicy
there was a missing code in ListAllBucketPolicies to allow
access, this fixes this behavior.

Fixes #7913
2019-07-15 20:00:41 +01:00
Harshavardhana
16a45e5aff
Fix dynamic help vars for sub-commands (#7925)
The fix in #7646 introduced a regression which
was left unnoticed, the fix didn't work for
sub-commands unfortunately. This fixes it
by moving v1.21.0 version of the minio/cli
package.

Fixes #7924
2019-07-12 23:32:27 -07:00
Anis Elleuch
000a60f238 xl: Heal empty parts (#7860)
posix.VerifyFile() doesn't know how to check if a file
is corrupted if that file is empty. We do have the part
size in xl.json so we pass it to VerifyFile to return
an error so healing empty parts can work properly.
2019-07-13 00:29:44 +01:00
Praveen raj Mani
bf278ca36f Enable event persistence in NSQ (#7579) 2019-07-12 10:41:57 +01:00
Ashish Kumar Sinha
97f2bc26b9 Add validations for object name length and prefix (#7746)
fixes #7717
2019-07-12 10:08:12 +05:30
Praveen raj Mani
bba562235b Enable persistent event store in elasticsearch (#7564) 2019-07-12 08:23:20 +05:30
dependabot[bot]
2337e5f803 Bump lodash from 4.17.4 to 4.17.14 in /browser (#7912)
Bumps [lodash](https://github.com/lodash/lodash) from 4.17.4 to 4.17.14.
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](https://github.com/lodash/lodash/compare/4.17.4...4.17.14)

Signed-off-by: dependabot[bot] <support@github.com>
2019-07-11 16:44:47 -07:00
Krishnan Parthasarathi
ffd7b7059c Pass on web-handler arguments properly to log entries (#7894) 2019-07-11 22:37:13 +01:00
Harshavardhana
5c0acbc6fc
Add text/event-stream for long running http connections (#7909)
When MinIO is behind a proxy, proxies end up killing
clients when no data is seen on the connection, adding
the right content-type ensures that proxies do not come
in the way.
2019-07-11 13:19:25 -07:00
Harshavardhana
5a52bc7ff6
Fix mint hub.docker.com builds (#7908) 2019-07-11 11:45:57 -07:00
poornas
045e1fed2b Fix dotnet tests build to be project structure agnostic (#7906) 2019-07-11 12:51:30 +05:30
dependabot[bot]
a861d38532 Bump lodash-es from 4.17.4 to 4.17.14 in /browser (#7910)
Bumps [lodash-es](https://github.com/lodash/lodash) from 4.17.4 to 4.17.14.
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](https://github.com/lodash/lodash/compare/4.17.4...4.17.14)

Signed-off-by: dependabot[bot] <support@github.com>
2019-07-11 10:55:13 +05:30
poornas
20a15567b8 Fix atime support check for disk cache (#7891)
- add a sleep between Stat operations to
accurately detect atime
2019-07-10 23:41:11 +01:00
Krishnan Parthasarathi
94f67ad224 Log error response even if a handler doesn't logBody (#7867) 2019-07-10 11:49:02 -07:00
ebozduman
36ee110563 Regression fix to bring back checkPolicyCond function call (#7897)
Fixes #7895
2019-07-10 10:48:43 +05:30
Harshavardhana
1dc25bcf5f
Add mint tests into MinIO repo (#7886) 2019-07-09 18:32:39 -07:00
Minio Trusted
2d96745156 Update yaml files to latest version RELEASE.2019-07-10T00-34-56Z 2019-07-10 00:39:45 +00:00
mzukowski-reef
9d49688c87 Switch to kurin/blazer from minio/blazer fork for b2 gateway (#7879) 2019-07-09 08:14:02 -07:00
Anis Elleuch
8e09374cb8 Avoid go-prompt to show colored prompt properly in Windows (#7890)
Update prompt shows some weird characters under Windows, the reason
is that go-prompt is used to show a yes/no prompt, since go-prompt
does not seem to have a way to support color/fatih, this PR will
implements its own yes/no prompt with the correct text coloration.
2019-07-09 01:46:04 +01:00
Krishna Srinivas
58d90ed73c Avoid network transfer for bitrot verification during healing (#7375) 2019-07-08 13:51:18 -07:00
Anis Elleuch
e857b6741d Add one log in health checker liveness code (#7861) 2019-07-06 16:38:39 -07:00
poornas
0505ef83b5 Fix host address returned in admin API calls (#7846) 2019-07-05 20:41:35 -07:00
Minio Trusted
22bc15d89b Update yaml files to latest version RELEASE.2019-07-05T21-20-21Z 2019-07-05 21:24:43 +00:00
Krishna Srinivas
a2e904b966 Support any string as delimiter for listing (#7882) 2019-07-05 14:06:12 -07:00
Kaan Kabalak
cc7dc61eb4 Allow folders inside buckets to be opened in a new tab (#7840)
Fixes #7836
2019-07-05 13:21:06 -07:00
Yao Zongyou
c4f480a839 fix csv read bug (#7885) 2019-07-05 12:08:56 -07:00
Yao Zongyou
60831e3299 aggregation functions' argument may already has been cast to numeric (#7876) 2019-07-05 10:38:38 -07:00
Yao Zongyou
037319066f fix unicode support related bugs in s3select (#7877) 2019-07-05 09:43:10 -07:00
Praveen raj Mani
bb871a7c31 Enable event persistence in webhook (#7614) 2019-07-05 15:21:41 +05:30
Harshavardhana
0ebbd3caef
Avoid chown instead fallback to rootpath for user perms (#7874)
Fixes #7864
2019-07-03 18:57:34 -07:00
Ryan Tam
bd56f80250 Fix ignored alias for aggregate result in S3 Select (#7849)
The SQL parser as it stands right now ignores alias for aggregate
result, e.g. `SELECT COUNT(*) AS thing FROM s3object` doesn't actually
return record like `{"thing": 42}`, it returns a record like `{"_1": 42}`.
Column alias for aggregate result is supported in AWS's S3 Select, so
this commit fixes that by respecting the `expr.As` in the expression.

Also improve test for S3 select

On top of testing a simple `SELECT` query, we want to test a few more
"advanced" queries (e.g. aggregation).

Convert existing tests into table driven tests[1], and add the new test
cases with "advanced" queries into them.

[1] - https://github.com/golang/go/wiki/TableDrivenTests
2019-07-03 16:34:54 -07:00