MyFavMirrors
/
minio
mirror of https://github.com/minio/minio.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
6,452 Commits 5 Branches 503 Tags 186 MiB
Commit Graph

6452 Commits

Author SHA1 Message Date
Harshavardhana 127641731a
Parallelize initialization of storageDisks (#8288) 2019-09-27 16:47:12 -07:00
Klaus Post c1a17c2561 S3 Select: Aggregate AVG/SUM as float (#8326) 
Force sum/average to be calculated as a float.

As noted in #8221

> run SELECT AVG(CAST (Score as int)) FROM S3Object on

```
Name,Score
alice,80
bob,81
```

> AWS S3 gives 80.5 and MinIO gives 80.

This also makes overflows much more unlikely.
 2019-09-27 16:12:03 -07:00
Klaus Post 1c5b05c130 S3 select: Fix output conversion on select * (#8303) 
Fixes #8268
 2019-09-27 12:33:14 -07:00
Harshavardhana 4155f4e49b
trace: Print either Transfer-Encoding or Content-Length (#8314) 
If Transfer-Encoding is set client would have
never set Content-Length as its considered
malformed HTTP request
 2019-09-27 10:19:27 -07:00
Aditya Manthramurthy f3022e891d Add ToC for all STS API docs (#8318) 2019-09-27 09:57:51 -07:00
Minio Trusted c28405a5c2 Update yaml files to latest version RELEASE.2019-09-26T19-42-35Z 2019-09-26 19:47:59 +00:00
Bala FA 2a2ff96ee1 change `ReadPerf` into `ReadThroughput` in NetPerfInfo. (#8316) 
Previously `ReadPerf` was in time.Duration is changed to `ReadThroughput` in uint64.
 2019-09-27 00:01:18 +05:30
Harshavardhana fd53057654 Add InfoCannedPolicy API to fetch only necessary policy (#8307) 
This PR adds
- InfoCannedPolicy() API for efficiency in fetching policies
- Send group memberships for LDAPUser if available
 2019-09-26 23:53:13 +05:30
Harshavardhana 3094615e38 Update LDAP document to reflect work user/group sub-commands (#8313) 
This PR also fixes markdown formatting
 2019-09-26 23:48:26 +05:30
Klaus Post ff726969aa Switch to Snappy -> S2 compression (#8189) 2019-09-25 23:08:24 -07:00
Klaus Post be313f1758 S3 Select: Workaround java buffer size (#8312) 
Updates #7475

The Java implementation has a 128KB buffer and a message must be emitted before that is used. #7475 therefore limits the message size to 128KB. But up to 256 bytes are written to the buffer in each call. This means we must emit a message before shorter than 128KB.

Therefore we change the limit to 128KB minus 256 bytes.
 2019-09-26 04:56:20 +05:30
Aditya Manthramurthy 704be85987 Fix numbering in doc (#8311) 2019-09-26 04:48:59 +05:30
Aditya Manthramurthy c8da04ba5b Add section on user/group policy for AD/LDAP integration (#8310) 2019-09-26 04:45:09 +05:30
Minio Trusted 9ed423b13f Update yaml files to latest version RELEASE.2019-09-25T18-25-51Z 2019-09-25 18:31:05 +00:00
Harshavardhana c8fbc94329
Fix writing 'format.json' and make it atomic (#8296) 
- Choose a unique uuid such that under situations of duplicate
  mounts we do not append to an existing json entry.
- Avoid AppendFile instead use WriteAll() to write the entire
  byte array atomically.
 2019-09-24 18:47:26 -07:00
Harshavardhana be70ef59e7
Update minio-go to latest release (#8297) 
- Change LDAP XML names to follow server changes
- Fix listObjects hang when listing special characters
- set encoding type to url in listobjects API
 2019-09-24 01:01:54 -07:00
Anis Elleuch a790877c01 s3: Encode continuation & next continuation tokens when asked (#8292) 
When url encoding is passed in v2 listing handler, continuationToken
and nextContinuationToken needs to be encoded. The reason is that
both represents an object name/prefix in Minio server and it could
contain a character unsupported by XML specification.
 2019-09-24 05:30:53 +05:30
Harshavardhana 77dc2031a2 Fix LDAP responseXML to be named appropriately (#8285) 
This PR additionally also adds support for missing

- Session policy support for AD/LDAP
- Add API request/response parameters detail
- Update example to take ldap username,
  password input from the command line
- Fixes session policy handling for
  ClientGrants and WebIdentity
 2019-09-24 03:51:16 +05:30
Harshavardhana 975134e42b
Add checks in DiskInfo() to protect against changing mounts (#8286) 2019-09-23 15:16:55 -07:00
Andreas Auernhammer cb7d23cb17 remove SSE-S3 key rotation in CopyObject (#8278) 
This commit removes the SSE-S3 key rotation functionality
from CopyObject since there will be a dedicated Admin-API
for this purpose.

Also update the security documentation to link to mc and
the admin documentation.
 2019-09-24 02:05:04 +05:30
poornas 2e02e1889b Cleanup ResponseWriter function for audit and trace (#8283) 2019-09-24 02:04:28 +05:30
Julio Pintos 3b05e175d7 fix link to let's encrypt doc (#8294) 2019-09-23 23:53:11 +05:30
Harshavardhana b512241300
Add metrics healthcheck test with JWT (#8287) 2019-09-22 23:30:38 -07:00
ebozduman dbf7b1e573 starts-with policy condition support issue (#7937) 2019-09-22 14:20:49 -07:00
Harshavardhana 26985ac632 Fix all failing tests with -race 2019-09-22 11:01:46 -07:00
Praveen raj Mani ad75683bde Authorize prometheus endpoint with bearer token (#7640) 2019-09-22 20:27:12 +05:30
poornas 4925bc3e80 log server startup messages to admin console api (#8264) 2019-09-22 13:54:32 +05:30
Andreas Auernhammer ffded5a930 make the crypto error type a native go type (#8267) 
This commit makes the `crypto.Error` type a native go (string)
type. That allows us to define error values as constants instead
of variables.

For reference see:
 - https://twitter.com/_aead_/status/1118170258215514115?s=20
 - https://dave.cheney.net/2016/04/07/constant-errors
 2019-09-22 01:12:51 -07:00
Krishnan Parthasarathi 1127293863 Add a basic .dockerignore file to reduce docker context in `make docker` (#8282) 
- Add useful default tag to dev docker image build using `make docker`
 2019-09-21 04:49:56 +05:30
Andreas Auernhammer 2b51fe9f26 make SSE request header check comprehensive (#8276) 
This commit refactors the SSE header check
by moving it into the `crypto` package, adds
a unit test for it and makes the check comprehensive.
 2019-09-21 03:26:12 +05:30
Harshavardhana 4780fa5a58 Remove setting net.Conn Deadlines as its not needed anymore (#8269) 
This commit fixes a bug introduced in af6c6a2b35.

Setting deadlines in Go results in arbitrary hangs as reported here
https://github.com/golang/go/issues/34385

Fixes https://github.com/minio/minio/issues/7852
 2019-09-20 23:37:24 +05:30
Klaus Post 520552ffa9 S3 select: flush when reaching limit (#8279) 
Add missing flush when reaching select limit.
 2019-09-20 11:00:17 -07:00
Andreas Auernhammer b823d6d7bd remove the unused code for decrypting `io.Writer` (#8277) 
This commit removes unused code for decrypting
`io.Writer` since the actual implementation only
decrypts `io.Reader`
 2019-09-20 14:51:07 +05:30
Andreas Auernhammer a9d724120f remove TLS 1.3 opt-in code (#8275) 
This commit removes the TLS 1.3 opt-in code.
Since TLS 1.3 is opt-out for >= Go 1.13 this
code is not needed anymore.
 2019-09-20 01:51:44 +05:30
Praveen raj Mani dc0dce9beb Check for connection errors during event transit in elasticsearch and webhook (#8273) 2019-09-19 08:23:43 -07:00
Yao Zongyou 0e1408844b fix get container id bug (#8196) 2019-09-19 10:19:51 +05:30
Andreas Auernhammer e34369c860 prepare SSE-S3 metadata parsing for K/V data key store (#8259) 
This commit allows the MinIO server to parse the metadata if:
 - either the `X-Minio-Internal-Server-Side-Encryption-S3-Key-Id`
   and the `X-Minio-Internal-Server-Side-Encryption-S3-Kms-Sealed-Key`
   entries are present.
 - or *both* headers are not present.

This is in service to support a K/V data key store.
 2019-09-19 04:08:09 +05:30
Minio Trusted 8dc897b5f5 Update yaml files to latest version RELEASE.2019-09-18T21-55-05Z 2019-09-18 22:02:41 +00:00
Praveen raj Mani 456ce4cc92 Add rootCAs support to Kafka & MQTT (#8236) 
Fixes #8211
 2019-09-18 23:43:04 +05:30
Aditya Manthramurthy 696f4ceee2 Add notes on configuring LDAP STS with Microsoft Active Directory (#8260) 2019-09-17 17:21:37 -07:00
Klaus Post dac1cf5a9a S3 Select: Parsing tweaks (#8261) 
* Don't output empty lines.
* Trim whitespace from byte to int/float/bool conversions.
 2019-09-17 17:21:23 -07:00
Harshavardhana cb01516a26 In HDFS gateway fix non-empty folder behavior (#8254) 
To be compatible with our FS and Erasure coded
mode deployments, make sure that we do not send
200 OK for folders which have files inside.

Fixes #8143
 2019-09-18 01:59:59 +05:30
Aditya Manthramurthy dfa1b417a8 Update comments in ldap example client code (#8257) 2019-09-18 01:59:31 +05:30
Krishnan Parthasarathi 31bee6b6ed Remove size query parameter from PerfInfo handler (#8258) 2019-09-18 01:59:12 +05:30
poornas 04b92124c5 fs/xl: Log warning if cache config specified (#8251) 
in non-gateway mode.
 2019-09-16 19:55:52 -07:00
Harshavardhana 5392eee250 Avoid recursion and use a simple loop to merge entries (#8239) 
This avoids stack overflows when there are
lot of entries to be skipped, this PR also
optimizes the code to reuse the buffers.
 2019-09-17 06:08:37 +05:30
Vivian Kong fa32c71a56 Build staticcheck as pre-built binaries are not available for s390x (#8220) 2019-09-17 05:59:15 +05:30
Klaus Post c9b8bd8de2 S3 Select: optimize output (#8238) 
Queue output items and reuse them.
Remove the unneeded type system in sql and just use the Go type system.

In best case this is more than an order of magnitude speedup:

```
BenchmarkSelectAll_1M-12    	       1	1841049400 ns/op	274299728 B/op	 4198522 allocs/op
BenchmarkSelectAll_1M-12    	      14	  84833400 ns/op	169228346 B/op	 3146541 allocs/op
```
 2019-09-17 05:56:27 +05:30
Klaus Post 017456df63 Wait clearing the close channel (#8250) 
Close channel should not be nilled before goroutines have exited.

Fixes potential hang on closing.
 2019-09-16 16:18:01 -07:00
Harshavardhana 14b137aa66 posix/readDir should populate name for DT_UKNOWN (#8240) 
In commit a8296445ad we changed the code to handle
some corner cases on ARM and other platforms, this
PR just avoids the return for unknown filetypes
prematurely and let the name be populated appropriately.

This fixes bug for older XFS implementations such as
in Ubuntu 14.04
 2019-09-17 03:04:01 +05:30