Commit Graph

7922 Commits

Author SHA1 Message Date
Kaan Kabalak cc7dc61eb4 Allow folders inside buckets to be opened in a new tab (#7840)
Fixes #7836
2019-07-05 13:21:06 -07:00
Yao Zongyou c4f480a839 fix csv read bug (#7885) 2019-07-05 12:08:56 -07:00
Yao Zongyou 60831e3299 aggregation functions' argument may already has been cast to numeric (#7876) 2019-07-05 10:38:38 -07:00
Yao Zongyou 037319066f fix unicode support related bugs in s3select (#7877) 2019-07-05 09:43:10 -07:00
Praveen raj Mani bb871a7c31 Enable event persistence in webhook (#7614) 2019-07-05 15:21:41 +05:30
Harshavardhana 0ebbd3caef
Avoid chown instead fallback to rootpath for user perms (#7874)
Fixes #7864
2019-07-03 18:57:34 -07:00
Ryan Tam bd56f80250 Fix ignored alias for aggregate result in S3 Select (#7849)
The SQL parser as it stands right now ignores alias for aggregate
result, e.g. `SELECT COUNT(*) AS thing FROM s3object` doesn't actually
return record like `{"thing": 42}`, it returns a record like `{"_1": 42}`.
Column alias for aggregate result is supported in AWS's S3 Select, so
this commit fixes that by respecting the `expr.As` in the expression.

Also improve test for S3 select

On top of testing a simple `SELECT` query, we want to test a few more
"advanced" queries (e.g. aggregation).

Convert existing tests into table driven tests[1], and add the new test
cases with "advanced" queries into them.

[1] - https://github.com/golang/go/wiki/TableDrivenTests
2019-07-03 16:34:54 -07:00
iliul a39e810965 docs: Fix dead link of HighwayHash (#7847)
Signed-off-by: Lei Liu <liul.stone@gmail.com>
2019-07-03 14:32:58 -07:00
mizuno-keyence 09103991ea [Bugfix] duplicating flag registration (#7853) 2019-07-03 14:31:19 -07:00
Harshavardhana c43f745449
Ensure that we use constants everywhere (#7845)
This allows for canonicalization of the strings
throughout our code and provides a common space
for all these constants to reside.

This list is rather non-exhaustive but captures
all the headers used in AWS S3 API operations
2019-07-02 22:34:32 -07:00
Anis Elleuch 9610a74c19 auto-heal: Use fast scan instead of the deep one (#7868) 2019-07-02 18:53:08 -07:00
Matthew Wegner 0bcd8abc5c doc: "admin user policy" command typo (#7865)
Under "change user policy", the `mc admin set-policy` command is wrong.  It should be `mc admin user set-policy`.
2019-07-02 11:48:26 -07:00
kannappanr 70b350c383
Remove DeploymentID from response headers (#7815)
Response headers need not contain deployment ID.
2019-07-01 12:22:01 -07:00
Krishna Srinivas 338e9a9be9 Put object client disconnect (#7824)
Fail putObject  and postpolicy in case client prematurely disconnects
Use request's context to cancel lock requests on client disconnects
2019-06-28 22:09:17 -07:00
Krishnan Parthasarathi edbd8709ec Simplify PR template to ease new contributors' workflow (#7844) 2019-06-27 15:47:46 -07:00
Minio Trusted 5db60a6c59 Update yaml files to latest version RELEASE.2019-06-27T21-13-50Z 2019-06-27 21:18:52 +00:00
Kaan Kabalak 36c19f1d65 Check bucket & prefix after obj's for new bucket are fetched (#7838)
Fixes #7837
2019-06-27 12:13:18 -07:00
iliul d3f9f8be88 golint: fix redundant code logic (#7842)
Signed-off-by: Lei Liu <liul.stone@gmail.com>
2019-06-27 15:18:33 +05:30
Krishna Srinivas 183ec094c4 Simplify HTTP trace related code (#7833) 2019-06-26 22:41:12 -07:00
Harshavardhana c1d2b3d5c3 Handle HEAD/GET requests for virtual DNS requests (#7839)
r.URL.Path is empty when HEAD bucket with virtual
DNS requests come in since bucket is now part of
r.Host, we should use our domain names and fetch
the right bucket/object names.

This fixes an really old issue in our federation
setups.
2019-06-26 18:21:54 -07:00
Praveen raj Mani be72609d1f Expose version info in prometheus (#7812)
Fixes #7795
2019-06-26 10:36:54 -07:00
Anis Elleuch 48f2c98052 admin: Add Background heal status info API (#7774)
This API returns the information related to the self healing routine.

For the moment, it returns:
- The total number of objects that are scanned
- The last time when an item was scanned
2019-06-25 16:42:24 -07:00
Kanagaraj M 286c663495 list objects in browser ordered by last modified (#7805)
- return all objects in web-handlers listObjects response
- added local pagination to object list ui
- also fixed infinite loader and removed unused fields
2019-06-25 16:31:50 -07:00
Yao Zongyou 941fed8e4a s3Select: call Close on error to release the read lock (#7830) 2019-06-25 13:30:48 -07:00
Yao Zongyou 55092bede1 add timestamp compare support (#7832) 2019-06-25 11:05:37 -07:00
Yao Zongyou 90a3b830f4 fix typo and the string representation of the time.Time value (#7831) 2019-06-25 09:54:14 -07:00
Yao Zongyou 23b9df0694 Fix s3select TRIM function's nil pointer dereference bug (#7817) 2019-06-24 16:59:33 -07:00
Kanagaraj M 48cb271a46 include ip address while doing checkPortAvailability (#7818)
While checking for port availability, ip address should be included.
When a machine has multiple ip addresses, multiple minio instances
or some other applications can be run on same port but different
ip address.

Fixes #7685
2019-06-24 15:02:39 -07:00
Klaus Post 61229b38f7 Update compression README.md (#7823)
- Snappy is not and RLE compressor, it is LZ77 based.
- Add `xz` as a common file type.
- Add most common media container types.
- Never heard of `application/x-spoon`. Google turns up a blank as well.
- Change link to minio blog post on compression & encryption.
2019-06-23 13:52:58 +05:30
Harshavardhana 90ca73af13 Allow trace even if server is not initialized (#7822) 2019-06-21 16:47:51 -07:00
Daniel Valdivia a04b6561a0 Fix a typo on the comment for ListenBucketNotification (#7821) 2019-06-21 11:58:52 -07:00
Andreas Auernhammer 219d841496 remove `encryption key` section for certtool docs (#7820)
This commit removes the encryption key section from
the certool.exe docs because:
 - MinIO does not support any TLS cipher that encrypts
   something with the private key. We only support PFS
   ciphers.
 - The doc comment is not really accurate anyway.
2019-06-21 10:39:02 -07:00
dependabot[bot] baef49b4a2 Bump stringstream from 0.0.5 to 0.0.6 in /browser (#7816)
Bumps [stringstream](https://github.com/mhart/StringStream) from 0.0.5 to 0.0.6.
- [Release notes](https://github.com/mhart/StringStream/releases)
- [Commits](https://github.com/mhart/StringStream/compare/v0.0.5...v0.0.6)

Signed-off-by: dependabot[bot] <support@github.com>
2019-06-21 07:49:51 -07:00
Ashish Kumar Sinha 3022f60561 Stop duplicate entry in Notification.xml (#7690)
fixes #7671
2019-06-21 13:28:02 +05:30
poornas 680fdf6f90 Update doc link (#7814) 2019-06-20 16:49:21 -07:00
Harshavardhana 1af6e8cb72
Add support for session policies in STS APIs (#7747)
This PR adds support for adding session policies
for further restrictions on STS credentials, useful
in situations when applications want to generate
creds for multiple interested parties with different
set of policy restrictions.

This session policy is not mandatory, but optional.

Fixes #7732
2019-06-20 15:28:33 -07:00
Andreas Auernhammer 98d3913a1e enable SSE-KMS pass-through on S3 gateway (#7788)
This commit relaxes the restriction that the MinIO gateway
does not accept SSE-KMS headers. Now, the S3 gateway allows
SSE-KMS headers for PUT and MULTIPART PUT requests and forwards them
to the S3 gateway backend (AWS). This is considered SSE pass-through
mode.

Fixes #7753
2019-06-19 17:37:08 -07:00
Harshavardhana 35c38e4bd8 Add docs regarding trace (#7808) 2019-06-19 14:28:03 -07:00
Minio Trusted e43d3a075c Update yaml files to latest version RELEASE.2019-06-19T18-24-42Z 2019-06-19 18:31:37 +00:00
Harshavardhana 43e0ef4248 Update docs with new settings (#7803) 2019-06-18 17:52:27 -07:00
Harshavardhana cd7d5b59e5
Add DeleteUser() to generate events in etcd (#7804)
Fixes a regression introduced in 6d89435356

Fixes #7797
2019-06-18 15:44:23 -07:00
poornas 299ef9b188 Trace: Replace function name with API prefix (#7794)
This change is required for `Admin Trace`
2019-06-18 13:55:13 -07:00
Harshavardhana b30c436715 [notify] Make sure to return when quorum is missing (#7799)
Fixes a regression introduced in 510ec153b9
2019-06-18 09:23:33 -07:00
Nitish Tiwari 7001fe407f Check local address in healthcheck script to ensure correct MinIO port (#7787)
Fixes #7780
2019-06-17 00:12:36 -07:00
Minio Trusted 59f7266081 Update yaml files to latest version RELEASE.2019-06-15T23-07-18Z 2019-06-15 23:17:28 +00:00
Harshavardhana 99bf4d0c42 [sec] Match ${aws:username} exactly instead of prefix match (#7791)
This PR fixes a security issue where an IAM user based
on his policy is granted more privileges than restricted
by the users IAM policy.

This is due to an issue of prefix based Matcher() function
which was incorrectly matching prefix based on resource
prefixes instead of exact match.
2019-06-15 11:27:17 -07:00
Cody Maloney 7b8beecc81 Move lock to not surround pieces which don't use any internal members. (#7779)
Previously the read/write lock applied both for gateway use cases as
well the object store use case. Nothing from sys is touched or looked
at in the gateway usecase though, so we don't need to lock. Don't lock
to make the gateway policy getting a little more efficient, particularly
as where this is called from (checkRequestAuthType) is quite common.
2019-06-15 10:11:10 -07:00
Praveen raj Mani 510ec153b9 Refreshing notification system should not erase the rules-map of other buckets (#7758)
Fixes #7707
2019-06-15 03:14:27 -07:00
dependabot[bot] e29a37e95c Bump macaddress from 0.2.8 to 0.2.9 in /browser (#7781)
Bumps [macaddress](https://github.com/scravy/node-macaddress) from 0.2.8 to 0.2.9.
- [Release notes](https://github.com/scravy/node-macaddress/releases)
- [Commits](https://github.com/scravy/node-macaddress/compare/0.2.8...0.2.9)

Signed-off-by: dependabot[bot] <support@github.com>
2019-06-15 03:07:56 -07:00
Harshavardhana 4a4048fe27 Migrate minio etcd config to backend config (#7751)
etcd when used in federated setups, currently
mandates that all clusters should have same
config.json, which is too restrictive and makes
federation a restrictive environment.

This change makes it apparent that each cluster
needs to be independently managed if necessary
from `mc admin info` command line.

Each cluster with in federation can have their
own root credentials and as well as separate
regions. This way buckets get further restrictions
and allows for root creds to be not common
across clusters/data centers.

Existing data in etcd gets migrated to backend
on each clusters, upon start. Once done
users can change their config entries
independently.
2019-06-15 03:07:54 -07:00