Commit Graph

3705 Commits

Author SHA1 Message Date
Harshavardhana
ed4941a5f3
fix: calculate dataBlocks properly in healing (#12364) 2021-05-25 09:34:27 -07:00
Harshavardhana
cacdeca8cc
fix: return error for unexpected quorum in pickValidFileInfo (#12363) 2021-05-24 18:31:56 -07:00
Anis Elleuch
abd32065aa
heal: Hold lock when reading xl.meta from disks (#12362)
Lock is hold in healObject() after reading xl.meta from disks the first
time. This commit will held the lock since the beginning of HealObject()

Co-authored-by: Anis Elleuch <anis@min.io>
2021-05-24 13:39:38 -07:00
Harshavardhana
2baabd455b docs: fix per tenant limits docs formatting 2021-05-24 09:37:17 -07:00
Harshavardhana
ebf75ef10d
fix: remove all unused code (#12360) 2021-05-24 09:28:19 -07:00
Klaus Post
f01820a4ee
fix: invalid multipart offset when compressed+encrypted. (#12340)
Fixes `testSSES3EncryptedGetObjectReadSeekFunctional` mint test.

```
{
  "args": {
    "bucketName": "minio-go-test-w53hbpat649nhvws",
    "objectName": "6mdswladz4vfpp2oit1pkn3qd11te5"
  },
  "duration": 7537,
  "error": "We encountered an internal error, please try again.: cause(The requested range \"bytes 251717932 -> -116384170 of 135333762\" is not satisfiable.)",
  "function": "GetObject(bucketName, objectName)",
  "message": "CopyN failed",
  "name": "minio-go: testSSES3EncryptedGetObjectReadSeekFunctional",
  "status": "FAIL"
}
```

Compressed files always start at the beginning of a part so no additional offset should be added.
2021-05-21 14:07:16 -07:00
Harshavardhana
0287711dc9
fix: implement readMetadata common function for re-use (#12353)
Previous PR #12351 added functions to read from the reader
stream to reduce memory usage, use the same technique in
few other places where we are not interested in reading the
data part.
2021-05-21 11:41:25 -07:00
Klaus Post
9d1b6fb37d
Add XL reader without data (#12351)
Add XL metadata reader that reads metadata only on larger files.

Use for scanning and listing for now.
2021-05-21 09:10:54 -07:00
Harshavardhana
32d8a48d4e
reduce memory usage in metacache reader (#12334) 2021-05-20 09:00:11 -07:00
Harshavardhana
6060b755c6
fix: migrate users properly from older releases to newer (#12333) 2021-05-19 19:25:44 -07:00
Krishnan Parthasarathi
cfa94cc35c
Simplify remote tier validation in lifecycle rule validation (#12329) 2021-05-19 18:51:23 -07:00
Klaus Post
2ca9c533ef
feat: implement in-progress partial bucket updates (#12279) 2021-05-19 14:38:30 -07:00
Anis Elleuch
866593fd94
heal: Ignore disks with non quorum modtime and dataDir (#12328) 2021-05-19 12:04:08 -07:00
Harshavardhana
ecb5525c91
fix: muxing order for rejected APIs (#12321) 2021-05-19 09:21:34 -07:00
Klaus Post
c2c803dd30
Fix list entry deduplication (#12325)
File infos would always be the same.

Add numversions as a final tiebreaker.
2021-05-19 09:21:18 -07:00
Harshavardhana
4f5d75f22b
fix: speed up drive mux registration (#12319)
in setups with lots of drives the server
startup is slow, initialize all local drives
in parallel before registering with muxer.

this speeds up when there are multiple pools
and large collection of drives.
2021-05-18 17:25:00 -07:00
Harshavardhana
bb7fbcdc09
fix: generating service accounts for group only LDAP accounts (#12318)
fixes #12315
2021-05-18 15:19:20 -07:00
Andreas Auernhammer
82c53ac260
sse-kms: set KMS key ID response header (#12316)
This commit adds the `X-Amz-Server-Side-Encryption-Aws-Kms-Key-Id`
response header to the GET, HEAD, PUT and Download API.

Based on AWS documentation [1] AWS S3 returns the KMS key ID as part
of the response headers.

[1] https://docs.aws.amazon.com/AmazonS3/latest/userguide/specifying-kms-encryption.html

Signed-off-by: Andreas Auernhammer <aead@mail.de>
2021-05-18 14:21:20 -07:00
Harshavardhana
a70e0da19e
use direntPool, direntNamePool for reusable buffers (#12314)
- in readDirFn re-use buffers from direntPool()
- in readDirN use separate dirent name buffer direntNamePool()
2021-05-18 10:29:50 -07:00
Harshavardhana
c6b7dc012a
fix: use key.Ciphertext for DecryptKey in KeyStatus (#12313)
enhance GlobalKMS.Stat() for kes to actually perform
a network call to check Version() of kes and also
implicitly that its reachable.
2021-05-18 07:22:31 -07:00
Harshavardhana
2daba018d6
reduce allocations on multi-disk clusters (#12311)
multi-disk clusters initialize buffer pools
per disk, this is perhaps expensive and perhaps
not useful, for a running server instance. As this
may disallow re-use of buffers across sets,

this change ensures that buffers across sets
can be re-used at drive level, this can reduce
quite a lot of memory on large drive setups.
2021-05-17 17:49:48 -07:00
Harshavardhana
d610578d84
fix: de-couple IAM migration and loading context from lock context (#12312)
fixes #12307
2021-05-17 16:50:47 -07:00
Harshavardhana
a096a92c63
add io.ErrUnexpectedEOF for config retriable errors (#12309)
fixes #12307
2021-05-17 15:13:14 -07:00
Harshavardhana
3d9873106d
feat: distributed setup can start now with default credentials (#12303)
In lieu of new changes coming for server command line, this
change is to deprecate strict requirement for distributed setups
to provide root credentials.

Bonus: remove MINIO_WORM warning from April 2020, it is time to
remove this warning.
2021-05-17 08:45:22 -07:00
Klaus Post
cde6469b88
Fix hanging erasure writes (#12253)
However, this slice is also used for closing the writers, so close is never called on these.

Furthermore when an error is returned from a write it is now reported to the reader.

bonus: remove unused heal param from `newBitrotWriter`.

* Remove copy, now that we don't mutate.
2021-05-17 08:32:28 -07:00
Klaus Post
55375fa7f6
Update probabilities for bloom filter. (#12305)
See https://github.com/minio/minio/discussions/12285

Results in M=958506 K=7 and 119840 bytes per filter when serialized compared to 26176 bytes before.
2021-05-17 08:31:04 -07:00
Harshavardhana
f1e479d274
remove more duplicate bloom filter trackers (#12302)
At some places bloom filter tracker was getting
updated for `.minio.sys/tmp` bucket, there is no
reason to update bloom filters for those.

And add a missing bloom filter update for MakeBucket()

Bonus: purge unused function deleteEmptyDir()
2021-05-17 08:25:48 -07:00
Harshavardhana
2ab9dc7609 do not update bloomFilters for temporary objects 2021-05-15 19:54:07 -07:00
Harshavardhana
4d876d03e8 fix: do not fail upon faulty/non-writable drives
gracefully start the server, if there are other drives
available - print enough information for administrator
to notice the errors in console.

Bonus: for really large streams use larger buffer for
writes.
2021-05-15 12:57:18 -07:00
Harshavardhana
d84261aa6d
fix: ensure proper usage of DataDir (#12300)
- GetObject() should always use a common dataDir to
  read from when it starts reading, this allows the
  code in erasure decoding to have sane expectations.

- Healing should always heal on the common dataDir, this
  allows the code in dangling object detection to purge
  dangling content.

These both situations can happen under certain types of
retries during PUT when server is restarting etc, some
namespace entries might be left over.
2021-05-14 16:50:47 -07:00
Harshavardhana
5b18c57a54
fix: for deleteBucket delete on dnsStore first (#12298)
attempt a delete on remote DNS store first before
attempting locally, because removing at DNS store
is cheaper than deleting locally, in case of
errors locally we can cheaply recreate the
bucket on dnsStore instead of.
2021-05-14 12:40:54 -07:00
Andreas Auernhammer
a1f70b106f
sse: add support for SSE-KMS bucket configurations (#12295)
This commit adds support for SSE-KMS bucket configurations.
Before, the MinIO server did not support SSE-KMS, and therefore,
it was not possible to specify an SSE-KMS bucket config.

Now, this is possible. For example:
```
mc encrypt set sse-kms some-key <alias>/my-bucket
```

Further, this commit fixes an issue caused by not supporting
SSE-KMS bucket configuration and switching to SSE-KMS as default
SSE method.

Before, the server just checked whether an SSE bucket config was
present (not which type of SSE config) and applied the default
SSE method (which was switched from SSE-S3 to SSE-KMS).

This caused objects to get encrypted with SSE-KMS even though a
SSE-S3 bucket config was present.

This issue is fixed as a side-effect of this commit.

Signed-off-by: Andreas Auernhammer <aead@mail.de>
2021-05-14 00:59:05 -07:00
Poorna Krishnamoorthy
951acf561c
Add support for syncing replica modifications (#11104)
when bidirectional replication is set up.

If ReplicaModifications is enabled in the replication
configuration, sync metadata updates to source if
replication rules are met. By default, if this
configuration is unset, MinIO automatically sync's
metadata updates on replica back to the source.
2021-05-13 19:20:45 -07:00
Harshavardhana
397391c89f
fix: parentUser mapped policy for OIDC creds (#12293)
missing parentUser for OIDC STS creds can lead
to fail to authenticate, this PR attempts to
fix the parentUser policy map for distributed
setups.
2021-05-13 16:21:06 -07:00
Andreas Auernhammer
9cd9f5a0b3
check that we can reach KES server and that the default key exists (#12291)
This commit adds a check to the MinIO server setup that verifies
that MinIO can reach KES, if configured, and that the default key
exists. If the default key does not exist it will create it
automatically.

Signed-off-by: Andreas Auernhammer <aead@mail.de>
2021-05-13 11:13:31 -07:00
Harshavardhana
5c0a7189c7
fix: LDAP authentication with groups only (#12283)
fixes #12282
2021-05-12 21:25:07 -07:00
Harshavardhana
57aed841dd
do not return error for usage-cache version v4 (#12276) 2021-05-12 08:07:02 -07:00
Klaus Post
229d83bb75
feat: add dynamic usage cache (#12229)
A cache structure will be kept with a tree of usages.
The cache is a tree structure where each keeps track 
of its children.

An uncompacted branch contains a count of the files 
only directly at the branch level, and contains link to 
children branches or leaves.

The leaves are "compacted" based on a number of properties.
A compacted leaf contains the totals of all files beneath it.

A leaf is only scanned once every dataUsageUpdateDirCycles,
rarer if the bloom filter for the path is clean and no lifecycles 
are applied. Skipped leaves have their totals transferred from 
the previous cycle.

A clean leaf will be included once every healFolderIncludeProb 
for partial heal scans. When selected there is a one in 
healObjectSelectProb that any object will be chosen for heal scan.

Compaction happens when either:

- The folder (and subfolders) contains less than dataScannerCompactLeastObject objects.
- The folder itself contains more than dataScannerCompactAtFolders folders.
- The folder only contains objects and no subfolders.
- A bucket root will never be compacted.

Furthermore, if a has more than dataScannerCompactAtChildren recursive 
children (uncompacted folders) the tree will be recursively scanned and the 
branches with the least number of objects will be compacted until the limit 
is reached.

This ensures that any branch will never contain an unreasonable amount 
of other branches, and also that small branches with few objects don't 
take up unreasonable amounts of space.

Whenever a branch is scanned, it is assumed that it will be un-compacted
before it hits any of the above limits. This will make the branch rebalance 
itself when scanned if the distribution of objects has changed.

TLDR; With current values: No bucket will ever have more than 10000 
child nodes recursively. No single folder will have more than 2500 child 
nodes by itself. All subfolders are compacted if they have less than 500 
objects in them recursively.

We accumulate the (non-deletemarker) version count for paths as well, 
since we are changing the structure anyway.
2021-05-11 18:36:15 -07:00
Harshavardhana
fe21aa356c
fix: if targetUser empty use parentUser for serviceAccounts (#12275) 2021-05-11 13:02:00 -07:00
Anis Elleuch
56d4d7b8b1
MRF: Better detection of non stable disks (#12252)
MRF does not detect when a node is disconnected and reconnected quickly
this change will ensure that MRF is alerted by comparing the last disk
reconnection timestamp with the last MRF check time.

Signed-off-by: Anis Elleuch <anis@min.io>

Co-authored-by: Klaus Post <klauspost@gmail.com>
2021-05-11 09:19:15 -07:00
Harshavardhana
e84f533c6c
add missing wait groups for certain io.Pipe() usage (#12264)
wait groups are necessary with io.Pipes() to avoid
races when a blocking function may not be expected
and a Write() -> Close() before Read() races on each
other. We should avoid such situations..

Co-authored-by: Klaus Post <klauspost@gmail.com>
2021-05-11 09:18:37 -07:00
Anis Elleuch
0b34dfb479
lock: Timeout Unlock RPC call (#12213)
RPC unlock call needs to be timed out otherwise this can block
indefinitely.

Signed-off-by: Anis Elleuch <anis@min.io>
2021-05-11 02:11:29 -07:00
Harshavardhana
b81fada834
use json unmarshal/marshal from jsoniter in hotpaths (#12269) 2021-05-11 02:02:32 -07:00
Andreas Auernhammer
d8eb7d3e15
kms: replace KES client implementation with minio/kes (#12207)
This commit replaces the custom KES client implementation
with the KES SDK from https://github.com/minio/kes

The SDK supports multi-server client load-balancing and
requests retry out of the box. Therefore, this change reduces
the overall complexity within the MinIO server and there
is no need to maintain two separate client implementations.

Signed-off-by: Andreas Auernhammer <aead@mail.de>
2021-05-10 18:15:11 -07:00
Andreas Auernhammer
c03a06cca8
config: enforce AES-GCM in FIPS mode (#12265)
This commit enforces the usage of AES-256
for config and IAM data en/decryption in FIPS
mode.

Further, it improves the implementation of
`fips.Enabled` by making it a compile time
constant. Now, the compiler is able to evaluate
the any `if fips.Enabled { ... }` at compile time
and eliminate unused code.

Signed-off-by: Andreas Auernhammer <aead@mail.de>
2021-05-10 08:24:11 -07:00
Harshavardhana
2d79d6d847
fix: do not niladic p.writers upon failure (#12255)
p.writers is a verbatim value of bitrotWriter
backed by a pipe() that should never be nil'ed,
instead use the captured errors to skip the writes.

additionally detect also short writes, and reject
them as errors.
2021-05-10 08:20:23 -07:00
Harshavardhana
8b52d70012
fix: IAM not initialized then checkKeyValid() should return 503s (#12260)
currently GetUser() returns 403 when IAM is not initialized
this can lead to applications crashing, instead return 503
so that the applications can retry and backoff.

fixes #12078
2021-05-09 08:14:19 -07:00
Harshavardhana
39d681a04a update fsSimpleRenameFile contrib 2021-05-08 22:31:41 -07:00
Harshavardhana
764721e2c6
add root_disk threshold detection (#12259)
as there is no automatic way to detect if there
is a root disk mounted on / or /var for the container
environments due to how the root disk information
is masked inside overlay root inside container.

this PR brings an environment variable to set
root disk size threshold manually to detect the
root disks in such situations.
2021-05-08 15:40:29 -07:00
Andreas Auernhammer
adaae26bbc
sse-kms: fix single-part object decryption (#12257)
This commit fixes a bug in the single-part object decryption
that is triggered in case of SSE-KMS. Before, it was assumed
that the encryption is either SSE-C or SSE-S3. In case of SSE-KMS
the SSE-C branch was executed. This lead to an invalid SSE-C
algorithm error.

This commit fixes this by inverting the `if-else` logic.
Now, the SSE-C branch only gets executed when SSE-C headers
are present.

Signed-off-by: Andreas Auernhammer <aead@mail.de>
2021-05-07 14:40:57 -07:00
Andreas Auernhammer
0ba8c0a19b
sse-kms: fix assignment to potential nil map (#12250)
This commit fixes a bug introduced by af0c65b.
When there is no / an empty client-provided SSE-KMS
context the `ParseMetadata` may return a nil map
(`kms.Context`).

When unsealing the object key we must check that
the context is nil before assigning a key-value pair.

Signed-off-by: Andreas Auernhammer <aead@mail.de>
2021-05-07 09:16:49 -07:00
Anis Elleuch
cb0b36f8c2
svcacct: Fix updating service account and add missing check (#12251)
UpdateServiceAccount ignores updating fields when not passed from upper
layer, such as empty policy, empty account status, and empty secret key.

This PR will check for a secret key only if it is empty and add more
check on the value of the account status.

Signed-off-by: Anis Elleuch <anis@min.io>
2021-05-07 09:13:30 -07:00
Klaus Post
254698f126
fix: minor allocation improvements in xlMetaV2 (#12133) 2021-05-07 09:11:05 -07:00
Krishnan Parthasarathi
0bab1c1895
Heal restored object contents on disk (#12238) 2021-05-06 16:06:57 -07:00
Harshavardhana
d495cb68d3
fix: crash in prometherus metrics collector (#12244)
node_health metrics crashes in gateway mode,
in gateway mode ignore node health metrics.

fixes #12243
2021-05-06 15:43:34 -07:00
Andreas Auernhammer
af0c65be93
add SSE-KMS support and use SSE-KMS for auto encryption (#12237)
This commit adds basic SSE-KMS support.
Now, a client can specify the SSE-KMS headers
(algorithm, optional key-id, optional context)
such that the object gets encrypted using the
SSE-KMS method. Further, auto-encryption now
defaults to SSE-KMS.

This commit does not try to do any refactoring
and instead tries to implement SSE-KMS as a minimal
change to the code base. However, refactoring the entire
crypto-related code is planned - but needs a separate
effort.

Signed-off-by: Andreas Auernhammer <aead@mail.de>
2021-05-06 15:24:01 -07:00
Nitish Tiwari
776589f0da Add free inode metric for Prometheus (#12225) 2021-05-06 12:50:48 -07:00
Harshavardhana
361940706d
fix: avoid races in NewMultipartUpload under multiple pools (#12233)
It is possible in some scenarios that in multiple pools,
two concurrent calls for the same object as a multipart operation
can lead to duplicate entries on two different pools.

This PR fixes this

- hold locks to serialize multiple callers so that we don't race.
- make sure to look for existing objects on the namespace as well
  not just for existing uploadIDs
2021-05-06 10:45:33 -07:00
Harshavardhana
1aa5858543
move madmin to github.com/minio/madmin-go (#12239) 2021-05-06 08:52:02 -07:00
Harshavardhana
f4623ea8dc fix: validate secret key before updating service accounts 2021-05-05 16:41:47 -07:00
Harshavardhana
b8833c2947 do not change targetUser after permission validation
for service accounts make sure that targetUser is
always the one that is presented/validated from
the incoming request, not the parentUser.
2021-05-05 16:13:52 -07:00
Anis Elleuch
af1b6e3458
iam: Do not create service accounts for non existant IAM users (#12236)
When running MinIO server without LDAP/OpenID, we should error out when
the code tries to create a service account for a non existant regular
user.

Bonus: refactor the check code to be show all cases more clearly

Signed-off-by: Anis Elleuch <anis@min.io>

Co-authored-by: Anis Elleuch <anis@min.io>
2021-05-05 16:04:50 -07:00
Harshavardhana
0eeb0a4e04 Revert "add SSE-KMS support and use SSE-KMS for auto encryption (#11767)"
This reverts commit 26f1fcab7d.
2021-05-05 15:20:46 -07:00
Andreas Auernhammer
26f1fcab7d
add SSE-KMS support and use SSE-KMS for auto encryption (#11767)
This commit adds basic SSE-KMS support.
Now, a client can specify the SSE-KMS headers
(algorithm, optional key-id, optional context)
such that the object gets encrypted using the
SSE-KMS method. Further, auto-encryption now
defaults to SSE-KMS.

This commit does not try to do any refactoring
and instead tries to implement SSE-KMS as a minimal
change to the code base. However, refactoring the entire
crypto-related code is planned - but needs a separate
effort.

Signed-off-by: Andreas Auernhammer <aead@mail.de>
Co-authored-by: Klaus Post <klauspost@gmail.com>
2021-05-05 11:24:14 -07:00
Harshavardhana
3a0e7347ca
support startTLS with serverName TLSConfig (#12219)
fixes #12216
2021-05-04 20:13:24 -07:00
Harshavardhana
67001e3ce9
fix: allow root credentials to generate STS, service accounts (#12210) 2021-05-04 11:58:19 -07:00
Harshavardhana
804a23a06d update docs to remove _OLD credential references
also update the docs about config, IAM on encryption.
2021-05-04 10:27:51 -07:00
Nitish Tiwari
c8aa56ccd7
Add node cpu & memory metrics to Prometheus cluster endpoint (#12214) 2021-05-04 10:17:10 -07:00
Harshavardhana
ff36baeaa7
fix: attempt to drain the ReadFileStream for connection pooling (#12208)
avoid time_wait build up with getObject requests if there are
pending callers and they timeout, can lead to time_wait states

Bonus share the same buffer pool with erasure healing logic,
additionally also fixes a race where parallel readers were
never cleanup during Encode() phase, because pipe.Reader end
was never closed().

Added closer right away upon an error during Encode to make
sure to avoid racy Close() while stream was still being
Read().
2021-05-04 10:12:08 -07:00
Krishnan Parthasarathi
860bf1bab2
Add IsRemote method on FileInfo, ObjectInfo (#12209)
Provides a convenient method to know if an object's contents are in its remote
tier.
2021-05-04 08:40:42 -07:00
Andreas Auernhammer
4815f92fa8
fix MINIO_KMS_SECRET_KEY env. variable parsing (#12200)
This commit fixes a bug when parsing the env. variable
`MINIO_KMS_SECRET_KEY`. Before, the env. variable
name - instead of its value - was parsed. This (obviously)
did not work properly.

This commit fixes this.

Signed-off-by: Andreas Auernhammer <aead@mail.de>
2021-04-30 18:47:30 -07:00
Harshavardhana
0d3ddf7286
fix: improve NewObjectReader implementation for careful cleanup usage (#12199)
cleanup functions should never be cleaned before the reader is
instantiated, this type of design leads to situations where order
of lockers and places for them to use becomes confusing.

Allow WithCleanupFuncs() if the caller wishes to add cleanupFns
to be run upon close() or an error during initialization of the
reader.

Also make sure streams are closed before we unlock the resources,
this allows for ordered cleanup of resources.
2021-04-30 18:37:58 -07:00
Harshavardhana
3524c00090 fix: nats testdata relocation fix 2021-04-30 17:22:56 -07:00
Harshavardhana
f7a87b30bf Revert "deprecate embedded browser (#12163)"
This reverts commit 736d8cbac4.

Bring contrib files for older contributions
2021-04-30 08:50:39 -07:00
Harshavardhana
64f6020854
fix: cleanup locking, cancel context upon lock timeout (#12183)
upon errors to acquire lock context would still leak,
since the cancel would never be called. since the lock
is never acquired - proactively clear it before returning.
2021-04-29 20:55:21 -07:00
Harshavardhana
0faa4e6187
fix: make sure failed requests only to failed queue (#12196)
failed queue should be used for retried requests to
avoid cascading the failures into incoming queue, this
would allow for a more fair retry for failed replicas.

Additionally also avoid taking context in queue task
to avoid confusion, simplifies its usage.
2021-04-29 18:20:39 -07:00
Poorna Krishnamoorthy
90112b5644
Update ReplicationStatus if metadata not updated correctly (#12191)
There can be situations where replication completed but the
`X-Amz-Replication-Status` metadata update failed such as
when the server returns 503 under high load. This object version will
continue to be picked up by the scanner and replicateObject would perform
no action since the versions match between source and target.
The metadata would never reflect that replication was successful
without this fix, leading to repeated re-queuing.
2021-04-29 16:46:26 -07:00
Harshavardhana
c4b21ac7fa
fix: remove healthcheck routine for replication targets (#12192)
Bonus also fix a racy lookup on arnsMap() without a
read lock, hold read locks to avoid such race.

moving the healthcheck logic to minio-go
2021-04-29 16:41:28 -07:00
Andreas Auernhammer
e5ec1325fc
docs: add QuickStart section to KMS encryption of IAM data (#12190)
This commit enhances the docs about IAM encryption.
It adds a quick-start section that explains how to
get started quickly with `MINIO_KMS_SECRET_KEY`
instead of setting up KES.

It also removes the startup message that gets printed
when the server migrates IAM data to plaintext.
We will point this out in the release notes.

Signed-off-by: Andreas Auernhammer <aead@mail.de>
2021-04-29 14:20:28 -07:00
Harshavardhana
c5a80ca5d5
support service accounts for OpenID connect properly (#12178)
OpenID connect generated service accounts do not work
properly after console logout, since the parentUser state
is lost - instead use sub+iss claims for parentUser, according
to OIDC spec both the claims provide the necessary stability
across logins etc.
2021-04-29 13:01:42 -07:00
Harshavardhana
8cd89e10ea Revert "fix: remove deprecated MINIO_ACCESS_KEY, MINIO_SECRET_KEY envs (#12173)"
This reverts commit b0baaeaa3d.
2021-04-29 10:56:53 -07:00
Harshavardhana
091845df39
fix: return quorum error upon decode failures (#12184) 2021-04-29 10:00:03 -07:00
Harshavardhana
336c8ac99f
fix: do not heal when disks are down (#12186)
HeadObject() was erroneously attempting
a heal when disks are down, avoid it.
2021-04-29 09:54:16 -07:00
Harshavardhana
b3c8a1864f
fix: optimize ListBuckets for anonymous users (#12182)
anonymous users are never allowed to listBuckets(),
we do not need to further validate the policy, we can
simply reject if credentials are empty.
2021-04-28 21:37:02 -07:00
Poorna Krishnamoorthy
632252ff1d
fix: change SetRemoteTarget API to allow editing remote target granularly (#12175)
Currently, only credentials could be updated with
`mc admin bucket remote edit`. 

Allow updating synchronous replication flag, path, 
bandwidth and healthcheck duration on buckets, and
a flag to disable proxying in active-active replication.
2021-04-28 15:26:20 -07:00
Harshavardhana
77f9c71133 Revert "redirect to console project for browser (#12172)"
This reverts commit 301669cf7b.

fixes #12179
2021-04-28 12:22:15 -07:00
Krishnan Parthasarathi
0c9d095deb
ilm: Close warmBackend GetObject reader (#12174) 2021-04-27 22:42:18 -07:00
Harshavardhana
b0baaeaa3d
fix: remove deprecated MINIO_ACCESS_KEY, MINIO_SECRET_KEY envs (#12173) 2021-04-27 22:41:24 -07:00
Harshavardhana
301669cf7b
redirect to console project for browser (#12172) 2021-04-27 16:39:41 -07:00
Anis Elleuch
9e797532dc
lock: Always cancel the returned Get(R)Lock context (#12162)
* lock: Always cancel the returned Get(R)Lock context

There is a leak with cancel created inside the locking mechanism. The
cancel purpose was to cancel operations such erasure get/put that are
holding non-refreshable locks.

This PR will ensure the created context.Cancel is passed to the unlock
API so it will cleanup and avoid leaks.

* locks: Avoid returning nil cancel in local lockers

Since there is no Refresh mechanism in the local locking mechanism, we
do not generate a new context or cancel. Currently, a nil cancel
function is returned but this can cause a crash. Return a dummy function
instead.
2021-04-27 16:12:50 -07:00
Harshavardhana
736d8cbac4
deprecate embedded browser (#12163)
https://github.com/minio/console takes over the functionality for the
future object browser development

Signed-off-by: Harshavardhana <harsha@minio.io>
2021-04-27 10:52:12 -07:00
Harshavardhana
cf335f6c63
service accounts should use LDAP user DN to assign credentials (#12166)
LDAP DN should be used when allowing setting service accounts
for LDAP users instead of just simple user,

Bonus root owner should be allowed full access
to all service account APIs.

Signed-off-by: Harshavardhana <harsha@minio.io>
2021-04-27 10:04:08 -07:00
Harshavardhana
c8050bc079
fix: sleeper behavior in data scanner (#12164)
do not apply healReplication() for ILM
expired, transitioned objects
2021-04-27 08:24:44 -07:00
Harshavardhana
edda244066 move pkg/rpc, pkg/csvparser, pkg/argon2 to contrib
Signed-off-by: Harshavardhana <harsha@minio.io>
2021-04-26 18:24:40 -07:00
Poorna Krishnamoorthy
4be0f92067
Fix multipart restore to remove part match (#12161)
Part ETags are not available after multipart finalizes, removing this
check as not useful.

Signed-off-by: Poorna Krishnamoorthy <poorna@minio.io>
Co-authored-by: Harshavardhana <harsha@minio.io>
2021-04-26 18:24:06 -07:00
Harshavardhana
26544848ea
remove legacy master_key support by June (#12153)
Signed-off-by: Harshavardhana <harsha@minio.io>
2021-04-26 16:02:05 -07:00
Harshavardhana
2966823818
use jsoniter for json marshal/unmarshal in KMS (#12146)
Signed-off-by: Harshavardhana <harsha@minio.io>
2021-04-26 16:01:52 -07:00
Harshavardhana
d501c5e38b
add missing responseBody drain (#12147)
Signed-off-by: Harshavardhana <harsha@minio.io>
2021-04-26 08:59:54 -07:00
Harshavardhana
d825d92499 rename production to release directory, rebuild assets 2021-04-25 16:51:29 -07:00
Andreas Auernhammer
f7feff8665
avoid parsing MINIO_KMS_MASTER_KEY as base64 (#12149)
This commit reverts a change that added support for
parsing base64-encoded keys set via `MINIO_KMS_MASTER_KEY`.

The env. variable `MINIO_KMS_MASTER_KEY` is deprecated and
should ONLY support parsing existing keys - not the new format.

Any new deployment should use `MINIO_KMS_SECRET_KEY`. The legacy
env. variable `MINIO_KMS_MASTER_KEY` will be removed at some point
in time.

Signed-off-by: Andreas Auernhammer <aead@mail.de>
2021-04-25 11:04:31 -07:00