Commit Graph

3768 Commits

Author SHA1 Message Date
ebozduman 83c37a44b6
'starts-with' support for 'success_action_status' (#12698) 2021-07-13 13:33:03 -07:00
Harshavardhana 9dae5a7c85
fix: only redirect for GET/HEAD requests (#12702)
fixes #12701
2021-07-13 11:25:08 -07:00
Anis Elleuch aa78505181
Add prefixes usage in Accounting Usage Info (#12687) 2021-07-13 10:42:11 -07:00
Harshavardhana e316873f84
feat: Add support for kakfa audit logger target (#12678) 2021-07-13 09:39:13 -07:00
Harshavardhana 559d075627
fix: properly send group unset policy (#12694)
In distributed setups `mc admin group unset`
was not correctly working due to incorrect
handling of query parameters between peers.
2021-07-13 09:32:35 -07:00
Harshavardhana a655e7f820
fix: use browser redirect URL for IDP callback (#12689)
if browser_redirect_url is set use that for IDP callback
automatically, if we do not have to set REDIRECT_URI
for OpenID callback URL.
2021-07-12 15:21:07 -07:00
Klaus Post 3201bae773
feat: gzip S3/Admin API responses whenever applicable (#12631)
Gzip responses if appropriate, except GetObject requests.
List reponses has an almost 10:1 compression ratio with no
measurable slowdown (in fact it seems a bit faster).
2021-07-12 10:49:50 -07:00
Anis Elleuch 8625cfdfae
update madmin-go dependency, update related code (#12669) 2021-07-12 10:16:10 -07:00
Harshavardhana d86ef6256d fix: ldap:username variable substitution in policies 2021-07-11 18:39:06 -07:00
Harshavardhana cd36019450
fix: allow STS credentials with dynamic policies (#12681)
- ParentUser for OIDC auth changed to `openid:`
  instead of `jwt:` to avoid clashes with variable
  substitution

- Do not pass in random parents into IsAllowed()
  policy evaluation as it can change the behavior
  of looking for correct policies underneath.

fixes #12676
fixes #12680
2021-07-11 17:39:52 -07:00
Harshavardhana 931f73f59a
fix: add IAM dummy store for gateway operations (#12670)
with console addition users cannot login with
root credentials without etcd persistent layer,
allow a dummy store such that such functionalities
can be supported when running as non-persistent
manner, this enables all calls and operations.
2021-07-10 08:32:52 -07:00
Anis Elleuch 9be040dd14
Add option in readDir to enable symlink following of dirs (#12668) 2021-07-09 16:20:51 -07:00
Harshavardhana da0fd5f056
fix: allow customizing console redirection (#12665)
MinIO might be running inside proxies, and
console while being on another port might not be
reachable on a specific port behind such proxies.

For such scenarios customize the redirect URL
such that console can be redirected to correct
proxy endpoint instead.

fixes #12661
2021-07-09 14:27:09 -07:00
Anis Elleuch c8cf4c5eb8
Fix force bucket deletion in distributed mode (#12659)
storageRESTForceDelete is not always passed by client, use r.URL().Query
instead.

Co-authored-by: Anis Elleuch <anis@min.io>
2021-07-09 13:29:38 -07:00
Klaus Post d6a2fe02d3
Add admin file inspector (#12635)
Download files from *any* bucket/path as an encrypted zip file.

The key is included in the response but can be separated so zip 
and the key doesn't have to be sent on the same channel.

Requires https://github.com/minio/pkg/pull/6
2021-07-09 11:29:16 -07:00
Harshavardhana 28adb29db3
feat: Add support to poll users on external SSO (#12592)
Additional support for vendor-specific admin API
integrations for OpenID, to ensure validity of
credentials on MinIO.

Every 5minutes check for validity of credentials
on MinIO with vendor specific IDP.
2021-07-09 11:17:21 -07:00
Harshavardhana b79cdc1611
fix: allow gateway to work with root credentials (#12655) 2021-07-09 10:35:09 -07:00
Harshavardhana 84a64a7e47
fix: accountInfo should use parentUser (#12652)
parentUser is needed to make sure that
dynamic variables in policy work properly.

fixes #12651
2021-07-08 12:43:25 -07:00
Harshavardhana affee27b05
fix: speed up erasure code upgrade checks (#12626)
DiskInfo() calls can stagger and wait if run
serially timing out 10secs per drive, to avoid
this lets check DiskInfo in parallel to avoid
delays when nodes get disconnected.
2021-07-08 01:04:37 -07:00
Harshavardhana 2ce60d772b
fix: update README.md for new release (#12637)
rename all 'docker run' commands to 'podman run'

fixes #12633
2021-07-07 18:07:29 -07:00
Harshavardhana c99d399d09
fix: handle redirects for specific resources (#12629) 2021-07-07 12:04:16 -07:00
Harshavardhana 25f55d6051 fix: order of args ListObjectsV2 S3 gateway 2021-07-06 13:44:14 -07:00
Harshavardhana 039978640f
fix: honor system umask for file creates (#12601)
use 0666 os.FileMode to honor system umask
2021-07-06 12:54:16 -07:00
Harshavardhana 6503c6ac21 fix: support startAfter with S3 gateway
fixes #12604
2021-07-05 18:44:04 -07:00
Klaus Post 05aebc52c2
feat: Implement listing version 3.0 (#12605)
Co-authored-by: Harshavardhana <harsha@minio.io>
2021-07-05 15:34:41 -07:00
Klaus Post a7e2a1a38b
fix: two different scanner update races (#12615) 2021-07-02 11:19:56 -07:00
Klaus Post f706671568
fix: multi-way list entry resolution (#12617)
Fixes brought forward from https://github.com/minio/minio/pull/12605

Fixes resolution when an object is in prefix of another and one zone returns the directory and another the object.

Fixes resolution on single entries that arrive first, so resolution doesn't depend on order.
2021-07-02 09:54:00 -07:00
Harshavardhana 4f6c74a257
simplify audit logging for replication and ILM (#12610)
auditLog should be attempted right before the
return of the function and not multiple times
per function, this ensures that we only trigger
it once per function call.
2021-07-01 14:02:44 -07:00
Krishnan Parthasarathi a1df230518
Add a 'free' version to track deletion of tiered object content (#12470) 2021-06-30 19:32:07 -07:00
Aditya Manthramurthy dc6958b6a1
fix: enhance openid claim missing error (#12608)
The error implies an expected claim is missing even when 
the claim is present. Added an additional error message 
to clarify the problem.
2021-06-30 17:11:23 -07:00
Harshavardhana 5a33d3f05a fix: set the correct IDP salt/passphrase 2021-06-30 16:45:52 -07:00
Harshavardhana 4781e7580b
add support for customizing redirect_uri for IDP (#12607) 2021-06-30 16:08:20 -07:00
Poorna Krishnamoorthy a3f0288262
Use multipart call for replication (#12535)
if object was uploaded with multipart. This is to ensure that
GetObject calls with partNumber in URI request parameters
have same behavior on source and replication target.
2021-06-30 07:44:24 -07:00
Harshavardhana a6ad965799
fix: use NumVersions for list resolver (#12599)
also do not incorrectly double count
objExists unless its selected and it
matches with previous entry.

Bonus: change listQuorum to match with
AskDisks to ensure that we atleast by
default choose all the "drives" that
we asked is consistent.
2021-06-30 07:43:19 -07:00
Shireesh Anjal 4575291f8a
fix: drive state not added in health report (#12603)
In case of FS mode, the drive state was not being added in the health
report. Fixed by hard coding it to "ok".
2021-06-30 07:42:38 -07:00
Harshavardhana ec8d93f756
fix: add missing readTriggerCh close (#12593) 2021-06-29 08:47:15 -07:00
Poorna Krishnamoorthy a69c2a2fb3
Change replication to use read lock instead of writelock (#12581)
Fixes #12573

This PR also adding audit logging for replication activity
2021-06-28 23:58:08 -07:00
Poorna Krishnamoorthy c20fbcd842
fix:(replication/bandwidth) add missing validation in monitored reader (#12582) 2021-06-27 10:11:13 -07:00
Poorna Krishnamoorthy d00783c923
Use rate.Limiter for bandwidth monitoring (#12506)
Bonus: fixes a hang when bandwidth caps are enabled for
synchronous replication
2021-06-24 18:29:30 -07:00
Harshavardhana 8d1bc65757
allow resetting and reapply config on broken clusters (#12554)
Bonus: remove kms_kes as sub-system, since its ENV only.
- also fixes a crash with etcd cluster without KMS
configured and also if KMS decryption is missing.
2021-06-24 16:24:12 -07:00
Harshavardhana cc5656f6d5
allow OPA fallback for STS requests (#12568)
fixes #12547
2021-06-24 12:00:06 -07:00
Harshavardhana 41caf89cf4
fix: apply pre-conditions first on object metadata (#12545)
This change in error flow complies with AWS S3 behavior
for applications depending on specific error conditions.

fixes #12543
2021-06-24 09:44:00 -07:00
Emir İşman 9bf1ac0bb6
[deps]: update jwt-go dependency (#12544)
jwt-go has been renamed to jwt and has a new home. See
https://github.com/dgrijalva/jwt-go/issues/462
2021-06-24 08:41:04 -07:00
Poorna Krishnamoorthy ba6e9682e5
Clean up targets properly on bucket deletion (#12565) 2021-06-24 08:39:58 -07:00
Harshavardhana b4bf82c751
do not heal "backend-encrypted" out-of-band with migration (#12556)
backend-encrypted doesn't need to be explicitly healed anymore
since this file is deleted upon upgrade and migration to the
KMS based encrypted config/IAM credentials.
2021-06-23 12:09:10 -07:00
zxxxhonest 3a0ca7af8c
panic: unaligned 64-bit atomic operation (#12559)
goroutine 1 [running]:
runtime/internal/atomic.panicUnaligned()
        /usr/local/go/src/runtime/internal/atomic/unaligned.go:8 +0x24

golang doc:
// BUG(rsc): On x86-32, the 64-bit functions use instructions unavailable before the Pentium MMX.
//
// On non-Linux ARM, the 64-bit functions use instructions unavailable before the ARMv6k core.
//
// On ARM, x86-32, and 32-bit MIPS,
// it is the caller's responsibility to arrange for 64-bit
// alignment of 64-bit words accessed atomically. The first word in a
// variable or in an allocated struct, array, or slice can be relied upon to be
// 64-bit aligned.
2021-06-23 07:15:43 -07:00
Harshavardhana bcea72ee8a
fix: ForceUnlock should unlock everywhere (#12542) 2021-06-21 17:47:35 -07:00
Klaus Post 14bb969782
lock on multi pool object creation (#12541)
Create write lock on PutObject and CopyObject when on multi-pool setup.

Use the same lock as NewMultipartUpload so all creation calls share the same lock.
2021-06-21 09:25:10 -07:00
Harshavardhana 8f1fe3b761
fix: --console-address when specified endpoints missing (#12534)
Additionally upgrade console dependency for reading
environment variables properly.
2021-06-20 23:04:47 -07:00
Klaus Post 42ad12ebf1
fs: Handle no buckets (#12531)
Since total are sent after each bucket, deleting all would 
not make usage go to zero.
2021-06-18 10:41:36 -07:00