Commit Graph

8638 Commits

Author SHA1 Message Date
Harshavardhana 39d51ce845 fix: add Dockerfile.release* /opt/bin writable 2021-09-09 22:27:33 -07:00
Minio Trusted a216583d95 Update yaml files to latest version RELEASE.2021-09-09T21-37-07Z 2021-09-09 23:50:26 +00:00
Harshavardhana 5c448b1b97
fix: update allowed max chunk payloadsize to 16MiB (#13169)
fixes #13163
2021-09-09 14:37:07 -07:00
Klaus Post 7f49c38e2d
Recover corrupted usage files if any (#13179) 2021-09-09 11:24:22 -07:00
Klaus Post 0e7fdcee30
Healing: Decide healing inlining based on metadata (#13178)
Don't perform an independent evaluation of inlining, but mirror the decision made when uploading the object.

Leads to some objects being inlined or not based on new metrics. Instead respect previous decision.
2021-09-09 08:55:43 -07:00
Poorna Krishnamoorthy 418f8bed6a
Detect multipart uploads correctly in unencrypted case (#13176)
This is a fix building on #13171 to ensure objects uploaded using multipart are 
replicated as multipart
2021-09-09 07:52:49 -07:00
soudhaf 950fe73c4f
fix: minio console nginx config (#13156) 2021-09-08 23:41:06 -07:00
Harshavardhana 0892f1e406
fix: multipart replication and encrypted etag for sse-s3 (#13171)
Replication was not working properly for encrypted
objects in single PUT object for preserving etag,

We need to make sure to preserve etag such that replication
works properly and not gets into infinite loops of copying
due to ETag mismatches.
2021-09-08 22:25:23 -07:00
Poorna Krishnamoorthy 9af4e7b1da
Add healthcheck back for replication targets (#13168)
This will allow objects to relinquish read lock held during
replication earlier if the target is known to be down
without waiting for connection timeout when replication 
is attempted.
2021-09-08 15:34:50 -07:00
Harshavardhana 198a838d00 update console tag to v0.9.8 2021-09-08 14:05:17 -07:00
Harshavardhana aaa3fc3805
fix: remove deprecated LDAP username format support (#13165) 2021-09-08 13:31:51 -07:00
Klaus Post 3c2efd9cf3
Stop async listing earlier (#13160)
Stop async listing if we have not heard back from the client for 3 minutes.

This will stop spending resources on async listings when they are unlikely to get used. 
If the client returns a new listing will be started on the second request.

Stop saving cache metadata to disk. It is cleared on restarts anyway. Removes all 
load/save functionality
2021-09-08 11:06:45 -07:00
Harshavardhana 951b1e6a7a
fix: Optimize listing calls for NFS mounts (#13159)
--no-compat should allow for some optimized
behavior for NFS mounts by removing Stat()
operations.
2021-09-08 08:15:42 -07:00
Harshavardhana 9c5fd6a776 update CREDITS file with latest 2021-09-07 19:17:18 -07:00
Andreas Auernhammer e438dccf19
sts: add support for certificate-based authentication (#12748)
This commit adds a new STS API for X.509 certificate
authentication.

A client can make an HTTP POST request over a TLS connection
and MinIO will verify the provided client certificate, map it to an 
S3 policy and return temp. S3 credentials to the client.

So, this STS API allows clients to authenticate with X.509
certificates over TLS and obtain temp. S3 credentials.

For more details and examples refer to the docs/sts/tls.md
documentation.

Signed-off-by: Andreas Auernhammer <hi@aead.dev>
2021-09-07 19:03:48 -07:00
Andreas Auernhammer 43d2655ee4
tls: add TLS 1.3 ciphers to the list of supported ciphers (#13158)
This commit adds the TLS 1.3 ciphers to the list of
supported ciphers. Now, clients can connect to MinIO
using TLS 1.3

Signed-off-by: Andreas Auernhammer <hi@aead.dev>
2021-09-07 09:57:32 -07:00
Klaus Post b2c92cdaaa
select: Add more compression formats (#13142)
Support Zstandard, LZ4, S2, and snappy as additional 
compression formats for S3 Select.
2021-09-06 09:09:53 -07:00
Anis Elleuch 42b1d92b2a
azure: Fix crash after using ParseForm() for query param lookups (#13147)
Azure storage SDK uses http.Request feature which panics when the
request contains r.Form popuplated.

Azure gateway code creates a new request, however it modifies the
transport to add our metrics code which sets Request.Form during
shouldMeterRequest() call.

This commit simplifies shouldMeterRequest() to avoid setting
request.Form and avoid the crash.
2021-09-04 09:07:24 -07:00
Harshavardhana 1250312287
fail ready/liveness if etcd is unhealthy in gateway mode (#13146) 2021-09-03 17:05:41 -07:00
Klaus Post 308371b434
Clean up ToFileInfo and avoid copy (#13144)
Simplify code and remove an iteration of all versions. Remove unneded copy.
2021-09-03 12:31:32 -07:00
Anis Elleuch 88e6c11746
iam: Retry connection to etcd during initialization (#13143)
Retry connecting to etcd during IAM initialization when etcd is enabled.
2021-09-03 09:10:48 -07:00
Harshavardhana 2ee4ae88f9 update helm release to 3.0.2 2021-09-03 01:11:32 -07:00
Minio Trusted e2b6fb0a6a Update yaml files to latest version RELEASE.2021-09-03T03-56-13Z 2021-09-03 06:45:54 +00:00
Harshavardhana a19e3bc9d9
add more dangling heal related tests (#13140)
also make sure that HealObject() never returns
'ObjectNotFound' or 'VersionNotFound' errors,
as those are meaningless and not useful for the
caller.
2021-09-02 20:56:13 -07:00
Harshavardhana 495c55e6a5
fix: make sure to delete dangling objects during heal (#13138)
heal with --remove was not removing dangling versions
on versioned buckets, this PR fixes this properly.

this is a regression introduced in PR #12617
2021-09-02 17:45:30 -07:00
Poorna Krishnamoorthy a366143c5b
Remove replication permission check (#13135)
Fixes #13105
2021-09-02 09:31:13 -07:00
Harshavardhana b8f5a7db33 update console release to v0.9.6 2021-09-01 23:57:19 -07:00
Harshavardhana f486cfae86 point all docs for containers to quay.io 2021-09-01 18:48:26 -07:00
Harshavardhana 0838732df3 point helm releases to quay.io 2021-09-01 18:47:43 -07:00
Harshavardhana f422f09dff update console to latest master 2021-09-01 18:10:44 -07:00
Harshavardhana aff196ae57 add helm chart support for gateway 2021-09-01 16:27:12 -07:00
Harshavardhana 27c9f8be7a ignore MINIO_CERT_PASSWD to be same on all instances 2021-09-01 16:02:35 -07:00
Harshavardhana 67b6c945e2 whitelist certain ENVs from bootstrap check 2021-09-01 15:10:17 -07:00
Harshavardhana c89aee37b9
fix: log errors for incorrect environment inputs (#13121)
Invalid MINIO_ARGS, MINIO_ENDPOINTS would be silently
ignored when using remoteEnv style, make sure to log
errors to indicate invalid configuration.
2021-09-01 11:34:07 -07:00
Harshavardhana 03b7bebc96
fix: invalid quorum calculation in TransitionObject (#13125)
Quorum calculation should be based on the
existing metadata, custom quorum calculation
can lead to unreadable content.
2021-09-01 08:57:42 -07:00
Harshavardhana f89d0f68d0
fix: missing cleanup of tmp folders in NAS gateway setup (#13124)
console service should be shutdown last once all shutdown
sequences are complete, this is to ensure that we do not
prematurely kill the server before it cleans up the

`.minio.sys/tmp/uuid` folder.

NOTE: this only applies to NAS gateway setup.
2021-08-31 18:52:48 -07:00
Harshavardhana 72a288f73f
fix: add x-amz-server-side-encryption header in ListObjectsM (#13122)
fixes #13046
2021-08-31 17:18:13 -07:00
Harshavardhana 0073aee1ed update to console release v0.9.5 2021-08-31 15:18:43 -07:00
Harshavardhana 0f7a51f461
fix: speedTest between peers keep the connection alive (#13120)
for longer durations keep the speedTest alive instead
of timing them out based on ResponseHeaderTimeout.
2021-08-31 14:08:23 -07:00
Klaus Post 556552340a
listing: Don't log errFileNotFound and friends (#13119) 2021-08-31 09:46:42 -07:00
Harshavardhana 50bce0130a release helm/minio v2.0.1 upgrade MinIO 2021-08-31 02:09:09 -07:00
Minio Trusted 5c6dc63577 Update yaml files to latest version RELEASE.2021-08-31T05-46-54Z 2021-08-31 07:27:43 +00:00
Harshavardhana 2077d27053 init() globalAPIConfig only once 2021-08-30 22:46:54 -07:00
Klaus Post 76b3d3c559
fix: missing close on error for keepAlive connections (#13109)
Add missing close when error is reported
before body is done.
2021-08-30 08:46:46 -07:00
Dominik Hassler 514b2d6f12
fix: build on illumos (Solaris) (#13097) 2021-08-30 08:40:16 -07:00
Klaus Post 470553ff5d
Tweak readall allocation and renameData buffer reuse (#13108)
Use a single allocation for reading the file, not the growing buffer of `io.ReadAll`.

Reuse the write buffer if we can when writing metadata in RenameData.
2021-08-30 08:38:11 -07:00
Harshavardhana 88e0aa1cb2
verify all nodes have same ENVs in bootstrap (#13096) 2021-08-30 08:27:39 -07:00
Harshavardhana 35f2552fc5
reduce extra getObjectInfo() calls during ILM transition (#13091)
* reduce extra getObjectInfo() calls during ILM transition

This PR also changes expiration logic to be non-blocking,
scanner is now free from additional costs incurred due
to slower object layer calls and hitting the drives.

* move verifying expiration inside locks
2021-08-27 17:06:47 -07:00
Anis Elleuch e05886561d
lock: Fix Refresh logic with multi resources lock (#13092)
A multi resources lock is a single lock UID with multiple associated
resources. This is created for example by multi objects delete
operation. This commit changes the behavior of Refresh() to iterate over
all locks having the same UID and refresh them.

Bonus: Fix showing top locks for multi delete objects
2021-08-27 13:07:55 -07:00
Klaus Post 2451b9a75a
fix: hanging operations on PUT with slow IO (#13087)
#11878 added "keepHTTPResponseAlive" to CreateFile requests. 
The problem is that it will begin writing to the response before the 
body is read after 10 seconds. This will abort the writes on the 
client-side, since it assumes the server has received what it wants.

The proposed solution here is to monitor the completion of the body 
before beginning to send keepalive pings.

Fixes observed high number of goroutines stuck in `io.Copy` in 
`github.com/minio/minio/cmd.(*xlStorage).CreateFile` and 
`(*storageRESTClient).CreateFile` stuck in `http.DrainBody`.
2021-08-27 09:16:36 -07:00