Commit Graph

8862 Commits

Author SHA1 Message Date
Poorna K 03725dc015
Default multipart caching to writethrough (#13613)
when `MINIO_CACHE_COMMIT` is set.

- `writeback` caching applies only to single 
uploads. When cache commit mode is 
`writeback`, default multipart caching to be
synchronous.

- Add writethrough caching for single uploads
2021-11-10 08:12:03 -08:00
Harshavardhana 0a6f9bc1eb
allocate new highwayhash for each string hash (#13623)
fixes #13622
2021-11-09 15:28:08 -08:00
Aditya Manthramurthy 1946922de3
Add CI for etcd IAM backend (#13614)
Runs when ETCD_SERVER env var is set
2021-11-09 09:25:13 -08:00
Minio Trusted edf1f4233b Update yaml files to latest version RELEASE.2021-11-09T03-21-45Z 2021-11-09 04:51:05 +00:00
Harshavardhana f4b55ea7a7 update console to v0.12.2 2021-11-08 19:21:45 -08:00
Aditya Manthramurthy 8dfd1f03e9
fix: IAM initialization crash with etcd store (#13612) 2021-11-08 12:55:27 -08:00
Harshavardhana acf26c5ab7 re-arrange metacache struct to be optimal (#13609) 2021-11-08 10:26:08 -08:00
Klaus Post d9800c8135
fix: make sure to log panic in handlers (#13611) 2021-11-08 09:28:13 -08:00
Harshavardhana 02bef7560f add missing Copyright header 2021-11-08 09:13:15 -08:00
Daniel A. Ochoa 07dd0692b6
Fix hdfs gateway concurrent map writes (#13596)
Co-authored-by: Harshavardhana <harsha@minio.io>
2021-11-08 09:07:58 -08:00
Klaus Post 4f3317effe
Close stream on panic (#13605)
Always close streamHTTPResponse on panic on main thread to avoid 
write/flush after response handler has returned.
2021-11-08 08:41:27 -08:00
Klaus Post 9afdbe3648
fix: RLock UID memory leak (#13607)
UID were misnamed in RLock, leading to memory buildup.

Regression in #13430
2021-11-08 07:35:50 -08:00
Aditya Manthramurthy fe0df01448
fix: locking in some situations for IAM store (#13595)
- Fix a bug where read locks were taken instead of write locks in some situations
- Remove an unnecessary lock when updating based on notifications.
2021-11-07 17:42:32 -08:00
Harshavardhana 12e6907512
apply spelling checks for US locale (#13599) 2021-11-07 01:22:59 -08:00
Harshavardhana 5aef492b4c update disk-caching design guide 2021-11-07 01:21:34 -08:00
Harshavardhana 5d7ed8ff7d update S3 gateway limitation docs 2021-11-06 23:24:48 -07:00
Harshavardhana b1754fc5ff update go.mod and CREDITS 2021-11-06 11:39:17 -07:00
Harshavardhana 19bbf3e142 update CREDITS 2021-11-05 13:53:21 -07:00
jiangfucheng e1755275a0
resume heal from previous object instead of bucket after server restart (#13581) 2021-11-05 13:10:41 -07:00
Harshavardhana 520037e721
move to jwt-go v4 with correct releases (#13586) 2021-11-05 12:20:08 -07:00
Minio Trusted cbb0828ab8 Update yaml files to latest version RELEASE.2021-11-05T09-16-26Z 2021-11-05 10:03:56 +00:00
Andreas Auernhammer 8774d10bdf
sts: always verify the key usage of client certificates (#13583)
This commit makes the MinIO server behavior more consistent
w.r.t. key usage verification.

When MinIO verifies the client certificates it also checks
that the client certificate is valid of client authentication
(or any (i.e. wildcard) usage).

However, the MinIO server used to not verify the client key usage
when client certificate verification was disabled.
Now, the MinIO server verifies the client key usage even when
client certificate verification has been disabled. This makes
the MinIO behavior more consistent from a client's perspective.

Now, a client certificate has to be valid for client authentication
in all cases.

Signed-off-by: Andreas Auernhammer <hi@aead.dev>
2021-11-05 02:16:26 -07:00
Harshavardhana df9f479d58 update console to v0.12.1 2021-11-04 16:43:59 -07:00
Harshavardhana 8bb52c9c2a
fix: ignore disks that are available but not writable (#13585)
This is to allow replacing drives while some drives
while available are not writable.
2021-11-04 16:42:49 -07:00
Aditya Manthramurthy 947c423824
fix: user DN filtering that causes some unnecessary logs (#13584)
Additionally, remove the unnecessary `isUsingLookupBind` field in the LDAP struct
2021-11-04 13:11:20 -07:00
Harshavardhana c3d24fb26d
use single encoder for sending speedtest results (#13579)
Bonus: if runs have PUT higher then capture it anyways
to display an unexpected result, which provides a way
to understand what might be slowing things down on the
system.

For example on a Data24 WDC setup it is clearly visible
there is a bug in the hardware.

```
./mc admin speedtest wdc/
⠧ Running speedtest (With 64 MiB object size, 32 concurrency) PUT: 31 GiB/s GET: 24 GiB/s
⠹ Running speedtest (With 64 MiB object size, 48 concurrency) PUT: 38 GiB/s GET: 24 GiB/s

MinIO 2021-11-04T06:08:33Z, 6 servers, 48 drives
PUT: 38 GiB/s, 605 objs/s
GET: 24 GiB/s, 383 objs/s
```

Reads are almost 14GiB/sec slower than Writes which
is practically not possible.
2021-11-04 12:11:52 -07:00
Pavel M 112f9ae087
claim exp should be integer (#13582)
claim exp can be 

- float64
- json.Number

As per OIDC spec https://openid.net/specs/openid-connect-core-1_0.html#IDToken

Avoid using strings since the upstream library only supports these two types now.
2021-11-04 12:03:43 -07:00
Aditya Manthramurthy 01b9ff54d9
Add LDAP STS tests and workflow for CI (#13576)
Runs LDAP tests with openldap container on GH Actions
2021-11-04 08:16:30 -07:00
Aditya Manthramurthy 64a1904136
Remove unused GlobalServiceDoneCh (#13578) 2021-11-04 08:15:10 -07:00
Aditya Manthramurthy bce6864785
Add tests to verify default server policies (#13575)
Check that they are present and that they can be modified by user
2021-11-03 19:49:05 -07:00
Aditya Manthramurthy ecd54b4cba
Move all IAM storage functionality into iam store type (#13567)
This reverts commit 091a7ae359.

- Ensure all actions accessing storage lock properly.

- Behavior change: policies can be deleted only when they
  are not associated with any active credentials.

Also adds fix for accidental canned policy removal that was present in the
reverted version of the change.
2021-11-03 19:47:49 -07:00
Harshavardhana ca2b288a4b update console to v0.12.0 2021-11-03 16:23:45 -07:00
Harshavardhana 1016fbb8f9
feat: detect starting from windows explorer (#13570)
Windows users often click on the binary without
knowing MinIO is a command-line tool and should be
run from a terminal. Throw a message to guide them
on what to do.

Co-authored-by: Klaus Post <klauspost@gmail.com>
2021-11-03 14:22:13 -07:00
Harshavardhana be3f81c7ec
remove unused activeIOCount in single drive mode (#13574) 2021-11-03 12:29:45 -07:00
Minio Trusted 9f3c151c3c Update yaml files to latest version RELEASE.2021-11-03T03-36-36Z 2021-11-03 06:48:34 +00:00
Harshavardhana 9735f3d8f2 fix missing go.sum changes 2021-11-02 20:36:36 -07:00
Harshavardhana 34680c5ccf
fix: SQL select to honor limits properly for array queries (#13568)
added tests to cover the scenarios as well.
2021-11-02 19:14:46 -07:00
Krishna Srinivas 58934e5881
Support live updates for clients during speedtest (#13566) 2021-11-02 15:27:03 -07:00
Harshavardhana ad3f98b8e7 add util-linux RPM for setpriv command 2021-11-02 14:25:01 -07:00
Poorna K 7c33a33ef3
cache: fix commit value lookup in config (#13551) 2021-11-02 14:20:52 -07:00
Poorna K 3dfcca68e6
fix race in TestComputeActions test (#13564) 2021-11-02 14:20:15 -07:00
Harshavardhana 73b74c94a1
remove unnecessary RPMs to reduce security reports (#13565) 2021-11-02 14:15:46 -07:00
Harshavardhana 18338d60d5 treat all 2xx, 3xx as good status-codes
fixes #13560
2021-11-02 14:12:43 -07:00
Harshavardhana e106070640 update docs to mention the expected behavior for requests_max
fixes #13561
2021-11-02 14:10:21 -07:00
Harshavardhana 091a7ae359 Revert "Move all IAM storage functionality into iam store type (#13541)"
This reverts commit caadcc3ed8.
2021-11-02 13:51:42 -07:00
Krishna Srinivas 70160aeab3
Remove IOPS autotuning and simplify autotune code (#13554) 2021-11-02 13:03:00 -07:00
jandres - moscardo 1aa08f594d
Update README.md prometheus (#13514)
Modify the doc to warn users about Prometheus sending `domain:port`
2021-11-02 12:27:30 -07:00
Harshavardhana 14d8a931fe
re-use io.Copy buffers with 32k pools (#13553)
Borrowed idea from Go's usage of this
optimization for ReadFrom() on client
side, we should re-use the 32k buffers
io.Copy() allocates for generic copy
from a reader to writer.

the performance increase for reads for
really tiny objects is at this range
after this change.

> * Fastest: +7.89% (+1.3 MiB/s) throughput, +7.89% (+1308.1) obj/s
2021-11-02 08:11:50 -07:00
Harshavardhana 30ba85bc67
no need to write storageClass globally (#13555)
fixes #13548
2021-11-02 08:11:20 -07:00
Aditya Manthramurthy caadcc3ed8
Move all IAM storage functionality into iam store type (#13541)
- Ensure all actions accessing storage lock properly.

- Behavior change: policies can be deleted only when they 
  are not associated with any active credentials.
2021-11-01 21:58:07 -07:00