Commit Graph

8560 Commits

Author SHA1 Message Date
Harshavardhana
e124d88788
optimize listing operation concurrency (#12728)
- remove use of getOnlineDisks() instead rely on fallbackDisks()
  when disk return errors like diskNotFound, unformattedDisk
  use other fallback disks to list from, instead of paying the
  price for checking getOnlineDisks()

- optimize getDiskID() further to avoid large write locks when
  looking formatLastCheck time window

This new change allows for a more relaxed fallback for listing
allowing for more tolerance and also eventually gain more
consistency in results even if using '3' disks by default.
2021-07-24 22:03:38 -07:00
Aditya Manthramurthy
de00b641da
[LDAP] Support syncing user-group memberships with LDAP service (#12785)
When configured in Lookup Bind mode, the server now periodically queries the
LDAP IDP service to find changes to a user's group memberships, and saves this
info to update the access policies for all temporary and service account
credentials belonging to LDAP users.
2021-07-24 11:57:36 -07:00
Harshavardhana
e936871b83 update to console v0.8.0 2021-07-23 19:53:56 -07:00
Harshavardhana
922f5da8dc update nats.io to fix critical bug 2021-07-23 18:33:31 -07:00
Harshavardhana
294bafd58b remove unused WebTrace and minio/rpc dependency 2021-07-23 15:50:21 -07:00
Anis Elleuch
23ef25b57a
profiling: Return goroutines with sleep duration (#12775)
Add a new goroutine file which has another printing format. We need it
to see how much time each goroutine was blocked. Easier to detect stops.

Co-authored-by: Anis Elleuch <anis@min.io>
2021-07-23 13:16:53 -07:00
Aditya Manthramurthy
9a31030e74
DEPRECATION Warning: Avoid configuring default LDAP STS Expiry (#12781)
- Show notice when `MINIO_IDENTITY_LDAP_STS_EXPIRY` or the 
  corresponding to the configuration option is used at server startup.
- Once support is removed, the default will be fixed at 1 hour.
- Users may specify expiry directly in the STS API.
- Update docs and help message
- Adds example in ldap.go to configure expiry in STS API.
2021-07-22 16:43:57 -07:00
AlexHuang2021
df2871de53
fix: return error when listing fails to retry healing (#12765) 2021-07-22 12:14:44 -07:00
Aditya Manthramurthy
a3079a7de2
fix: Add support for DurationSeconds in LDAP STS API (#12778) 2021-07-22 12:13:21 -07:00
Aditya Manthramurthy
17a37241f0
fix: regression in AssumeRole session policy handling (fixes #12756) (#12773) 2021-07-22 08:42:07 -07:00
Minio Trusted
026a005b64 Update yaml files to latest version RELEASE.2021-07-22T05-23-32Z 2021-07-22 06:12:29 +00:00
Harshavardhana
b865b9f43d update console to fix assets 2021-07-21 22:23:32 -07:00
Krishnan Parthasarathi
209e6d00c6
Use ObjectInfo.ToLifecycleOpts instead of literal values (#12772)
Promote getLifecycleTransitionTier to a method on lifecycle.Lifecycle.
2021-07-21 19:12:44 -07:00
Minio Trusted
dbd7f74bb9 Update yaml files to latest version RELEASE.2021-07-21T22-15-23Z 2021-07-21 23:46:28 +00:00
Harshavardhana
4a2cb23690 update console to master 2021-07-21 15:15:23 -07:00
Harshavardhana
320e1533c4
use expected MinIO URLs for console (#12770)
when TLS is configured using IPs directly
might interfere and not work properly when
the server is configured with TLS certs but
the certs only have domain certs.

Also additionally allow users to specify
a public accessible URL for console to talk
to MinIO i.e `MINIO_SERVER_URL` this would
allow them to use an external ingress domain
to talk to MinIO. This internally fixes few
problems such as presigned URL generation on
the console UI etc.

This needs to be done additionally for any
MinIO deployments that might have a much more
stricter requirement when running in standalone
mode such as FS or standalone erasure code.
2021-07-21 14:51:16 -07:00
Harshavardhana
7f45e80755 update console to master with more UI fixes 2021-07-21 11:19:36 -07:00
Aditya Manthramurthy
0db1c94e7d
[IDP:LDAP] Cleanup creds for removed LDAP user entries (#12759) 2021-07-20 23:33:12 -07:00
Aditya Manthramurthy
e7a4967726
fix: openID cleanup all creds associated with OIDC user (#12758) 2021-07-20 17:42:10 -07:00
Krishnan Parthasarathi
d0963974a5
pkg/lifecycle: Add SetPredictionHeaders method (#12755)
This method is used to add expected expiration and transition time 
for an object in GET/HEAD Object response headers.

Also fixed bugs in lifecycle.PredictTransitionTime and
getLifecycleTransitionTier in handling current and 
non-current versions.
2021-07-20 17:36:55 -07:00
Krishnan Parthasarathi
6ea083d197
Add deployment-id and source bucket to transitioned object name (#12693)
This allows remote bucket admin to identify the origin of transitioned
objects by simply inspecting the object prefixes.

e.g let's take a remote tier TIER-1 pointing to a remote bucket (prefix)
testbucket/testprefix-1. The remote bucket admin can list all transitioned objects
from a MinIO deployment identified by '2e78e906-1c5d-4f94-8689-9df44cafde39' and
source bucket 'mybucket' like so,

```
$ ./mc ls -r minio-tier-target/testbucket/testprefix-1/2e78e906-1c5d-4f94-8689-9df44cafde39/mybucket/
[2021-07-12 17:15:50 PDT]   160B 48/fb/48fbc0e6-3a73-458b-9337-8e722c619ca4
[2021-07-12 16:58:46 PDT]   160B 7d/1c/7d1c96bd-031a-48d4-99ea-b1304e870830
```
2021-07-20 10:49:52 -07:00
Harshavardhana
ab7584eeac update to latest console fs/gateway 2021-07-19 17:49:54 -07:00
sakkiii
69e0faa278
add more XSS HTTP headers (#12256) 2021-07-19 16:05:02 -07:00
Shireesh Anjal
fa98014bbe
fix: hostname inside --console-address not anonymized (#12743)
In case of non-distributed setup, if the server start command contains a
`--console-address` flag and its value contains a hostname, it is not
getting anonymized.

Fixed by replacing the console host also with `server1`
2021-07-19 15:16:26 -07:00
Ali Malek
c25ad94b7e
Refactor docker-compose, Use YAML variable for common minio configs (#12735) 2021-07-18 18:06:40 -07:00
Harshavardhana
27e07e80bc
fix: accountInfo() error FS mode (#12734) 2021-07-17 01:17:35 -07:00
Harshavardhana
b13bca85d0
send backendInfo as part AccountInfo (#12733)
This is needed for console UI to change
the UI behavior for different modes of
operation.
2021-07-16 14:37:06 -07:00
Krishnan Parthasarathi
29eea52e14
Skip transitioning of object versions if inlined (#12705) 2021-07-16 09:38:27 -07:00
Anis Elleuch
b0b4696a64
heal: Add MRF metrics to background heal API response (#12398)
This commit gathers MRF metrics from 
all nodes in a cluster and return it to the caller. This will show information about the 
number of objects in the MRF queues 
waiting to be healed.
2021-07-15 22:32:06 -07:00
Minio Trusted
ead8778305 Update yaml files to latest version RELEASE.2021-07-15T22-27-34Z 2021-07-16 04:27:54 +00:00
Aditya Manthramurthy
eae9c2f65b
Add changes to ensure session policy is enforced in LDAP STS (#12716)
- Bonus: Fix bug in webidentity sts that doesnt parse session policy correctly.

- update ldap.go to support session policy argument
2021-07-15 15:27:34 -07:00
Poorna Krishnamoorthy
a6ec405443
fix: UpdateObjectVersion should compare versionID through versions (#12726)
fixes #12703
2021-07-15 15:01:59 -07:00
Harshavardhana
9516587a6c update console to master branch
with new fixes

- improve download behavior
- avoid response timeouts
2021-07-15 14:44:18 -07:00
Ravind Kumar
ce800ed347
DOCS: Remove http links in favor of HTTPS (#12725) 2021-07-15 13:31:59 -07:00
Harshavardhana
017722cf85
fix: propagate service account deletes properly (#12717)
service account deletes were not propagating
to remote peers, fix this.
2021-07-14 21:28:53 -07:00
Harshavardhana
10fc30a989
fix crash when config is not properly initialized (#12714)
fixes #12709
2021-07-14 18:27:25 -07:00
Lenin Alevski
92ffe5e5ef
Allow MinIO to load configurations from env file (#12706)
docker-entrypoint.sh will load configuration values from
'config.env' file, this is useful when MinIO is deployed in Kubernetes
environments and want to avoid reading secrets from environment
variables

Signed-off-by: Lenin Alevski <alevsk.8772@gmail.com>
2021-07-14 16:55:59 -07:00
Lazzaretti
e8cbfa7af2
fix: rename error message for MINIO_BROWSER_REDIRECT_URL (#12700)
rename the error message for MINIO_BROWSER_REDIRECT_URL to match the field name
2021-07-14 12:38:25 -07:00
Anis Elleuch
b8f95fb3d4
fix: Use correct replication status in replication healing (#12711)
In case of replication healing, we always store completed status in the
object metadata, which is wrong because replication could fail in the
further retries.
2021-07-14 09:58:46 -07:00
Anis Elleuch
a4b8928660
Move prefix usage in admin AccountInfo API (#12710) 2021-07-14 08:51:10 -07:00
Shireesh Anjal
4d6d4244f1
Anonymize hosts in subnet health report (#12627)
Ensure that hostnames / ip addresses are not printed in the subnet
health report. Anonymize them by replacing them with `servern` where `n`
represents the position of the server in the pool.

This is done by building a `host anonymizer` map that maps every
possible value containing the host e.g. host, host:port,
http://host:port, etc to the corresponding anonymized name and using
this map to replace the values at the time of health report generation.

A different logic is used to anonymize host names in the `procinfo`
data, as the host names are part of an ellipses pattern in the process
start command. Here we just replace the prefix/suffix of the ellipses
pattern with their hashes.
2021-07-14 00:23:22 -07:00
Harshavardhana
8d19efe7e0
feat: increase allowed maximum STS expiration timeout to 365 days (#12704) 2021-07-14 00:08:22 -07:00
ebozduman
83c37a44b6
'starts-with' support for 'success_action_status' (#12698) 2021-07-13 13:33:03 -07:00
Harshavardhana
9dae5a7c85
fix: only redirect for GET/HEAD requests (#12702)
fixes #12701
2021-07-13 11:25:08 -07:00
Anis Elleuch
aa78505181
Add prefixes usage in Accounting Usage Info (#12687) 2021-07-13 10:42:11 -07:00
Harshavardhana
e316873f84
feat: Add support for kakfa audit logger target (#12678) 2021-07-13 09:39:13 -07:00
Harshavardhana
559d075627
fix: properly send group unset policy (#12694)
In distributed setups `mc admin group unset`
was not correctly working due to incorrect
handling of query parameters between peers.
2021-07-13 09:32:35 -07:00
Harshavardhana
e25ae1c01f add console-address flag for spoonfed users 2021-07-12 17:05:38 -07:00
Harshavardhana
a655e7f820
fix: use browser redirect URL for IDP callback (#12689)
if browser_redirect_url is set use that for IDP callback
automatically, if we do not have to set REDIRECT_URI
for OpenID callback URL.
2021-07-12 15:21:07 -07:00
Harshavardhana
c438eda624 update console dependency for derivedKey fix 2021-07-12 14:37:25 -07:00