Commit Graph

8736 Commits

Author SHA1 Message Date
Krishnan Parthasarathi 32df742b85
fix: nil pointer access in warmBackendAzure.Put (#13263) 2021-09-21 15:47:30 -07:00
Harshavardhana 8392765213
healObjects() should cancel() context before writing to errCh (#13262)
also remove HealObjects() code from dataScanner running another
listing from the data-scanner is super in-efficient and in-fact
this code is redundant since we already attempt to heal all
dangling objects anyways.
2021-09-21 14:55:17 -07:00
Poorna Krishnamoorthy 806b10b934
fix: improve error messages returned during replication setup (#13261) 2021-09-21 13:03:20 -07:00
Aditya Manthramurthy 1fa0553c71
Remove support for elasticsearch versions < 7.x (#13260) 2021-09-21 12:57:10 -07:00
Harshavardhana 50a68a1791
allow S3 gateway to support object locked buckets (#13257)
- Supports object locked buckets that require
  PutObject() to set content-md5 always.
- Use SSE-S3 when S3 gateway is being used instead
  of SSE-KMS for auto-encryption.
2021-09-21 09:02:15 -07:00
Poorna Krishnamoorthy 0b55a0423e
fix: cache usage deserialization from v5 to v6 (#13258) 2021-09-21 09:01:51 -07:00
Anis Elleuch 565d95a377
Enable console logging when server debug is enabled (#13259)
_MINIO_SERVER_DEBUG will enable console logging.
2021-09-21 09:01:29 -07:00
Harshavardhana f492f72154
add 3site replication script (#13256) 2021-09-20 18:24:24 -07:00
Harshavardhana 4d84f0f6f0
fix: support existing folders in single drive mode (#13254)
This PR however also proceeds to simplify the loading
of various subsystems such as

- globalNotificationSys
- globalTargetSys

converge them directly into single bucket metadata sys
loader, once that is loaded automatically every other
target should be loaded and configured properly.

fixes #13252
2021-09-20 17:41:01 -07:00
Aditya Manthramurthy a0d0c8e4af
Allow root user to create service accounts in LDAP (#13221)
Additionally, fix a bug in service account creation for LDAP users: the
LDAP short username was not associated with the service account.
2021-09-20 14:28:19 -07:00
Harshavardhana bef748abbd fix: labels are strings in helm 2021-09-19 22:30:22 -07:00
Poorna Krishnamoorthy c4373ef290
Add support for multi site replication (#12880) 2021-09-18 13:31:35 -07:00
Minio Trusted 0b8c5a6872 Update yaml files to latest version RELEASE.2021-09-18T18-09-59Z 2021-09-18 20:28:07 +00:00
Harshavardhana 829ecb2086 update helm to 3.1.3 2021-09-18 11:09:59 -07:00
Goncharov Sergey d3564a4b09
Split toleration, nodeSelector and affinity from main pods and jobs (#13247) 2021-09-18 11:09:09 -07:00
Harshavardhana a244753f47 update to helm v3.1.2 2021-09-17 21:26:47 -07:00
Goncharov Sergey 745782a77a
fix: service monitoring labels in helm release (#13238)
fixes #13236
2021-09-17 21:18:27 -07:00
Poorna Krishnamoorthy 246cbe1312
update minio-go with healthcheck fixes (#13244) 2021-09-17 21:14:13 -07:00
Poorna Krishnamoorthy 6c941122eb
cancel active goroutine when remote target is edited (#13243) 2021-09-17 20:05:38 -07:00
Harshavardhana 1a884cd8e1
fix: deleting objects was not working after upgrades (#13242)
DeleteObject() on existing objects before `xl.json` to
`xl.meta` change were not working, not sure when this
regression was added. This PR fixes this properly.

Also this PR ensures that we perform rename of xl.json
to xl.meta only during "write" phase of the call i.e
either during Healing or PutObject() overwrites.

Also handles few other scenarios during migration where
`backendEncryptedFile` was missing deleteConfig() will
fail with `configNotFound` this case was not ignored,
which can lead to failure during upgrades.
2021-09-17 19:34:48 -07:00
Poorna Krishnamoorthy 18f008f7c7
Fix retention enforcement check for deleted object versions (#13240)
if an object is pending version purge, it should be treated
as ErrNone in retention enforcement check
2021-09-17 15:21:24 -07:00
Harshavardhana 6d42569ade
remove ListBucketsMetadata instead add them to AccountInfo() (#13241) 2021-09-17 15:02:21 -07:00
Harshavardhana 5ed781a330
check for context canceled after competing for locks (#13239)
once we have competed for locks, verify if the
context is still valid - this is to ensure that
we do not start readdir() or read() calls on the
drives on canceled connections.
2021-09-17 14:11:01 -07:00
Harshavardhana 66fcd02aa2
de-couple walkMu and walkReadMu for some granularity (#13231)
This commit brings two locks instead of single lock for
WalkDir() calls on top of c25816eabc.

The main reason is to avoid contention between readMetadata()
and ListDir() calls, ListDir() can take time on prefixes that
are huge for readdir() but this shouldn't end up blocking
all readMetadata() operations, this allows for more room for
I/O while not overly penalizing all listing operations.
2021-09-17 12:14:12 -07:00
Andreas Auernhammer 1fc0e9a6aa
sts: allow clients to send certificate chain (#13235)
This commit fixes an issue in the `AssumeRoleWithCertificate`
handler.

Before clients received an error when they send
a chain of X.509 certificates (their client certificate as
well as intermediate / root CAs).

Now, client can send a certificate chain and the server
will only consider non-CA / leaf certificates as possible
client certificate candidates. However, the client still
can only send one certificate.

Signed-off-by: Andreas Auernhammer <hi@aead.dev>
2021-09-17 09:37:01 -07:00
Anis Elleuch 91567ba916
s3: Put bucket tagging to return an error when bucket is not found (#13232)
instead of creating new metadata in .minio.sys directory
2021-09-17 08:32:32 -07:00
Klaus Post d80826b05d
Clean up metacache saver (#13225)
Don't report success before the listing has actually finished. 
This will make stop conditions more clear.
2021-09-16 13:35:25 -07:00
Harshavardhana 45bcf73185
feat: Add ListBucketsWithMetadata extension API (#13219) 2021-09-16 09:52:41 -07:00
Poorna Krishnamoorthy 78dc08bdc2
remove s3:ReplicateDelete permission check from DeleteObject APIs (#13220) 2021-09-15 23:02:16 -07:00
Klaus Post f98f115ac2
fs: Fix non-progressing scanner (#13218)
Scanner would keep doing the same cycle in FS mode leading to missed updates.

Add a few sanity checks and handle errors better.
2021-09-15 09:24:41 -07:00
Minio Trusted bf409936e7 Update yaml files to latest version RELEASE.2021-09-15T04-54-25Z 2021-09-15 08:04:57 +00:00
Shireesh Anjal b4364723ef
Add config to store subnet license (#13194)
Command to set subnet license:

`mc admin config set {alias} subnet license={token}`

Signed-off-by: Shireesh Anjal <shireesh@minio.io>
Co-authored-by: Harshavardhana <harsha@minio.io>
2021-09-14 21:54:25 -07:00
Harshavardhana bcc6359dec support Console UI with userInfo claims for OpenID 2021-09-14 17:09:18 -07:00
Harshavardhana 787a72a993
make sure to ignore the rootDisk when healing drives (#13209)
fixes #13208
2021-09-14 15:10:00 -07:00
Harshavardhana d9eb962969
allow admin API to support UNSIGNED-PAYLOAD (#13207)
admin API requests do not support x-amz-content-sha256
set with UNSIGNED-PAYLOAD, keep this consistent and
support it properly.
2021-09-14 13:55:24 -07:00
Anis Elleuch f221153776
s3-gateway: Allow encryption S3 passthrough for SSE-S3 (#13204)
This reverts commit 35cbe43b6d.
2021-09-14 12:55:32 -07:00
Harshavardhana 67596ef0cc
fix sse-kms context unmarshal failure (#13206)
json.Unmarshal expects a pointer receiver, otherwise
kms.Context unmarshal fails with lack of pointer receiver,
this becomes complicated due to type aliasing over
map[string]string - fix it properly.
2021-09-14 12:52:46 -07:00
Klaus Post bf5bfe589f
xlmeta: Recover corrupted metadata (#13205)
When unable to load existing metadata new versions 
would not be written. This would leave objects in a 
permanently unrecoverable state

Instead, start with clean metadata and write the incoming data.
2021-09-14 11:34:25 -07:00
Harshavardhana af78c3925a
add userinfo support for OpenID (#12469)
Some identity providers like GitLab do not provide
information about group membership as part of the
identity token claims. They only expose it via OIDC compatible
'/oauth/userinfo' endpoint, as described in the OpenID
Connect 1.0 sepcification.

But this of course requires application to make sure to add
additional accessToken, since idToken cannot be re-used to
perform the same 'userinfo' call. This is why this is specialized
requirement. Gitlab seems to be the only OpenID vendor that requires
this support for the time being.

fixes #12367
2021-09-13 16:22:14 -07:00
Harshavardhana d144958669 update helm to v3.1.1 - saAccount fix 2021-09-13 09:43:10 -07:00
Klaus Post 5a64003f6f
select: Return null for non-exiting column indexes (#13196)
Fixes #13186
2021-09-13 09:13:25 -07:00
Kanagaraj M 311718309c
fix ServiceAccount creation in helm chart (#13197)
Fixed the variable name on the templated and added name.
2021-09-13 09:12:20 -07:00
Anis Elleuch 98479d7ffd
Fix deadlock when error during metacache generation (#13201)
A typo forgot to release a lock after acquiring it.
2021-09-13 09:11:39 -07:00
Harshavardhana 90e505e58f calculate API requests/error as increase() intervals not as rate() 2021-09-12 11:28:28 -07:00
Harshavardhana 410b8dd0fd add service monitor support and service account support 2021-09-12 11:19:27 -07:00
Anis Elleuch c2f25b6f62
gateway/s3: allow tracing requests to backend service (#13189)
fixes #13089
fixes #13133

Co-authored-by: Anis Elleuch <anis@min.io>
Co-authored-by: Harshavardhana <harsha@minio.io>
2021-09-11 09:20:01 -07:00
Krishna Srinivas 03a2a74697
Support speedtest autotune on the server side (#13086) 2021-09-10 17:43:34 -07:00
ArthurMa 2807c11410
http hook should accept more than 200 statusCode (#13180)
Co-authored-by: Klaus Post <klauspost@gmail.com>
2021-09-10 14:27:37 -07:00
Harshavardhana 39d51ce845 fix: add Dockerfile.release* /opt/bin writable 2021-09-09 22:27:33 -07:00
Minio Trusted a216583d95 Update yaml files to latest version RELEASE.2021-09-09T21-37-07Z 2021-09-09 23:50:26 +00:00