Commit Graph

8498 Commits

Author SHA1 Message Date
Anis Elleuch
37bef900fd
ilm: More compliance to spec related to expired delete markers (#12887)
<Days> tag can remove expired delete markers according to AWS S3 spec.

<NoncurrentVersionExpiration> cannot remove expired delete markers in
anyway.
2021-08-05 11:21:21 -07:00
Harshavardhana
3863a96bdf remove unexpected spaces and newlines 2021-08-04 17:21:32 -07:00
Ravind Kumar
c89135fd13
Update License section for documentation (#12878) 2021-08-04 17:07:38 -07:00
Harshavardhana
b10f823907
allow diskFillFraction to be 99% (#12879)
larger 4-8TiB sized disks would return
error prematurely even with sufficient
amount of disk space left, increase
diskFillFraction to 1%
2021-08-04 15:23:39 -07:00
Harshavardhana
4d8f81a992
remove replicate set fallback behavior (#12865)
replica set deployment style is an incorrect
style of distributed MinIO deployment, since
statefulsets have been around for many releases
in k8s now.
2021-08-04 15:23:18 -07:00
Harshavardhana
c13cbc64d1 fix multiple typos in documentation 2021-08-04 14:15:45 -07:00
Matt Sarrel
109c8acf4f
fixed typo in metrics README.md (#12874) 2021-08-04 12:48:57 -07:00
Klaus Post
cc60d66909
Fix incremental usage accounting (#12871)
Remote caches were not returned correctly, so they would not get updated on save.

Furthermore make some tweaks for more reliable updates.

Invalidate bloom filter to ensure rescan.
2021-08-04 09:14:14 -07:00
Krishnan Parthasarathi
4197870287
Storage-class headers should depend only on ObjectInfo state (#12864) 2021-08-03 17:35:52 -07:00
Harshavardhana
035882d292
fix: remove parentIsObject() check (#12851)
we will allow situations such as

```
a/b/1.txt
a/b
```

and

```
a/b
a/b/1.txt
```

we are going to document that this usecase is
not supported and we will never support it, if
any application does this users have to delete
the top level parent to make sure namespace is
accessible at lower level.

rest of the situations where the prefixes get
created across sets are supported as is.
2021-08-03 13:26:57 -07:00
Harshavardhana
9371852c7d
fix: getAPIEndpoints() should return public_url (#12852)
fixes #12850
2021-08-02 21:50:20 -07:00
Harshavardhana
ea64a9263c
add deprecation notice for LDAP username format (#12849) 2021-08-02 18:20:06 -07:00
Harshavardhana
510c67de4a update console to v0.8.2 2021-08-02 13:07:49 -07:00
Dmitry Eliseev
27dbe30ecb
Allow to specify secrets separately (#12813) 2021-07-31 21:44:51 -07:00
Krishnan Parthasarathi
0a62ae4e61
Revert ignoring inlined objects for transition (#12843) 2021-07-30 16:45:17 -07:00
Harshavardhana
a51799d9f0
feat: Add support for audit notifications for transition (#12842)
This PR adds audit notifications for transitioning objects,
similar to audit logging for expiration and replication
traffic.
2021-07-30 12:45:25 -07:00
Harshavardhana
e7baf2d7d2 stop all console logging 2021-07-29 23:06:05 -07:00
Shireesh Anjal
56097dfca8
Include system errors in health report (#12776)
Some examples of system errors:

- audit is enabled
- updatedb is installed
2021-07-29 23:05:34 -07:00
Harshavardhana
7281e86d9e update docker-compose to RELEASE.2021-07-30T00-02-00Z 2021-07-29 18:07:06 -07:00
Harshavardhana
d004828b20 fix: speedtest Endpoint formatting 2021-07-29 17:51:30 -07:00
Harshavardhana
4567160e4d update madmin-go to v1.0.20 2021-07-29 17:02:00 -07:00
Harshavardhana
bc6a2b579b update to console v0.8.1 2021-07-29 16:14:10 -07:00
Harshavardhana
3c34e18a4e
allow multipart uploads for single part multipart (#12821)
its possible that some multipart uploads would have
uploaded only single parts so relying on `len(o.Parts)`
alone is not sufficient, we need to look for ETag
pattern to be absolutely sure.
2021-07-28 22:11:55 -07:00
Poorna Krishnamoorthy
b6cd54779c
Increase context timeout for bandwidth throttled reader (#12820)
increase default timeout up to one hour for toy setups.

fixes #12812
2021-07-28 15:20:01 -07:00
Harshavardhana
bfbdb8f0a8
fix: incorrect O_DIRECT behavior for reads (#12811)
O_DIRECT behavior was broken and it was still
caching all the reads, this change properly fixes
this behavior.
2021-07-28 11:20:16 -07:00
Harshavardhana
397637a042 remove x-frame-options HTTP header
fixes #12817
2021-07-28 08:57:22 -07:00
Anis Elleuch
20cacc3b04
fs: Update object modtime when metadata is updated (#12818)
Follow S3 behavior of updating object modtime when object metadata is
updated.
2021-07-28 07:36:38 -07:00
Harshavardhana
a78bc7bfdb rename all access token to id_tokens 2021-07-27 22:38:12 -07:00
Harshavardhana
3735450e7e
fix: allow audience claim to be an array (#12810)
Some incorrect setups might have multiple audiences
where they are trying to use a single authentication
endpoint for multiple services.

Nevertheless OpenID spec allows it to make it
even more confusin for no good reason.

> It MUST contain the OAuth 2.0 client_id of the
> Relying Party as an audience value. It MAY also
> contain identifiers for other audiences. In the
> general case, the aud value is an array of case
> sensitive strings. In the common special case
> when there is one audience, the aud value MAY
> be a single case sensitive string.

fixes #12809
2021-07-27 18:37:51 -07:00
Krishna Srinivas
aa0c28809b
Server side speedtest implementation (#12750) 2021-07-27 12:55:56 -07:00
Minio Trusted
471b4fd0c9 Update yaml files to latest version RELEASE.2021-07-27T02-40-15Z 2021-07-27 05:02:00 +00:00
Harshavardhana
ddcd419b4c
always validate JWT token audience (#12797)
audience for the JWT token should match
the configured client_id, this allows
rejecting valid JWTs not meant for MinIO.
2021-07-26 19:40:15 -07:00
Harshavardhana
a9d9b520ec
remove short circuited healing optimization (#12796)
this healing optimization caused multiple
regressions in healing

- delete-markers incorrectly missing
  heal and returning incorrect healing
  results to client.

- missing individual 'parts' such
  as for restored object or simply
  for all objects just missing few parts.

This optimization is not necessary, we
should proceed to verify all cases possible
not just when metadata is inconsistent.
2021-07-26 16:51:09 -07:00
Harshavardhana
0c666379fe
fix: avoid removing healed parts on dstDataPath (#12795)
destination path and old path will be similar
when healing occurs, this can lead to healed
parts being again purged leading to always an
inconsistent state on an object which might
further cause reduction in quorum eventually.
2021-07-26 15:15:34 -07:00
Harshavardhana
a3f7d575e0
improve delete-marker healing (#12794)
delete-markers missing on drives were
not healed due to few things

disksWithAllParts() does not know-how
to deal with delete markers, add support
for that.

fixes #12787
2021-07-26 11:48:09 -07:00
Harshavardhana
f175ff8f66
add healing fixes for delete-marker (#12788)
- delete-markers are incorrectly reported
  as corrupt with wrong data sent to client
  'mc admin heal -r' on objects with delete
  marker will report as 'grey' incorrectly.

- do not heal delete-markers during HeadObject()
  this can lead to inconsistent order of heals
  on the object, although this is not an issue
  in terms of order of versions it is rather
  simpler to keep the same order on all drives.

- defaultHealResult() should handle 'err == nil'
  case such that valid cases should be handled
  as 'drive' status OK.
2021-07-26 08:01:41 -07:00
Anis Elleuch
39874b77ed
mrf: Avoid rare data race and more simplification (#12791)
This change avoids a rare data race and simplify the function that
returns MRF last activity information.
2021-07-26 08:00:59 -07:00
Aditya Manthramurthy
7106e9394c
[LDAP] Document new LDAP synchronization behavior (#12786) 2021-07-25 10:45:53 -07:00
Harshavardhana
e124d88788
optimize listing operation concurrency (#12728)
- remove use of getOnlineDisks() instead rely on fallbackDisks()
  when disk return errors like diskNotFound, unformattedDisk
  use other fallback disks to list from, instead of paying the
  price for checking getOnlineDisks()

- optimize getDiskID() further to avoid large write locks when
  looking formatLastCheck time window

This new change allows for a more relaxed fallback for listing
allowing for more tolerance and also eventually gain more
consistency in results even if using '3' disks by default.
2021-07-24 22:03:38 -07:00
Aditya Manthramurthy
de00b641da
[LDAP] Support syncing user-group memberships with LDAP service (#12785)
When configured in Lookup Bind mode, the server now periodically queries the
LDAP IDP service to find changes to a user's group memberships, and saves this
info to update the access policies for all temporary and service account
credentials belonging to LDAP users.
2021-07-24 11:57:36 -07:00
Harshavardhana
e936871b83 update to console v0.8.0 2021-07-23 19:53:56 -07:00
Harshavardhana
922f5da8dc update nats.io to fix critical bug 2021-07-23 18:33:31 -07:00
Harshavardhana
294bafd58b remove unused WebTrace and minio/rpc dependency 2021-07-23 15:50:21 -07:00
Anis Elleuch
23ef25b57a
profiling: Return goroutines with sleep duration (#12775)
Add a new goroutine file which has another printing format. We need it
to see how much time each goroutine was blocked. Easier to detect stops.

Co-authored-by: Anis Elleuch <anis@min.io>
2021-07-23 13:16:53 -07:00
Aditya Manthramurthy
9a31030e74
DEPRECATION Warning: Avoid configuring default LDAP STS Expiry (#12781)
- Show notice when `MINIO_IDENTITY_LDAP_STS_EXPIRY` or the 
  corresponding to the configuration option is used at server startup.
- Once support is removed, the default will be fixed at 1 hour.
- Users may specify expiry directly in the STS API.
- Update docs and help message
- Adds example in ldap.go to configure expiry in STS API.
2021-07-22 16:43:57 -07:00
AlexHuang2021
df2871de53
fix: return error when listing fails to retry healing (#12765) 2021-07-22 12:14:44 -07:00
Aditya Manthramurthy
a3079a7de2
fix: Add support for DurationSeconds in LDAP STS API (#12778) 2021-07-22 12:13:21 -07:00
Aditya Manthramurthy
17a37241f0
fix: regression in AssumeRole session policy handling (fixes #12756) (#12773) 2021-07-22 08:42:07 -07:00
Minio Trusted
026a005b64 Update yaml files to latest version RELEASE.2021-07-22T05-23-32Z 2021-07-22 06:12:29 +00:00
Harshavardhana
b865b9f43d update console to fix assets 2021-07-21 22:23:32 -07:00