Commit Graph

9030 Commits

Author SHA1 Message Date
Anis Elleuch 565d95a377
Enable console logging when server debug is enabled (#13259)
_MINIO_SERVER_DEBUG will enable console logging.
2021-09-21 09:01:29 -07:00
Harshavardhana f492f72154
add 3site replication script (#13256) 2021-09-20 18:24:24 -07:00
Harshavardhana 4d84f0f6f0
fix: support existing folders in single drive mode (#13254)
This PR however also proceeds to simplify the loading
of various subsystems such as

- globalNotificationSys
- globalTargetSys

converge them directly into single bucket metadata sys
loader, once that is loaded automatically every other
target should be loaded and configured properly.

fixes #13252
2021-09-20 17:41:01 -07:00
Aditya Manthramurthy a0d0c8e4af
Allow root user to create service accounts in LDAP (#13221)
Additionally, fix a bug in service account creation for LDAP users: the
LDAP short username was not associated with the service account.
2021-09-20 14:28:19 -07:00
Harshavardhana bef748abbd fix: labels are strings in helm 2021-09-19 22:30:22 -07:00
Poorna Krishnamoorthy c4373ef290
Add support for multi site replication (#12880) 2021-09-18 13:31:35 -07:00
Minio Trusted 0b8c5a6872 Update yaml files to latest version RELEASE.2021-09-18T18-09-59Z 2021-09-18 20:28:07 +00:00
Harshavardhana 829ecb2086 update helm to 3.1.3 2021-09-18 11:09:59 -07:00
Goncharov Sergey d3564a4b09
Split toleration, nodeSelector and affinity from main pods and jobs (#13247) 2021-09-18 11:09:09 -07:00
Harshavardhana a244753f47 update to helm v3.1.2 2021-09-17 21:26:47 -07:00
Goncharov Sergey 745782a77a
fix: service monitoring labels in helm release (#13238)
fixes #13236
2021-09-17 21:18:27 -07:00
Poorna Krishnamoorthy 246cbe1312
update minio-go with healthcheck fixes (#13244) 2021-09-17 21:14:13 -07:00
Poorna Krishnamoorthy 6c941122eb
cancel active goroutine when remote target is edited (#13243) 2021-09-17 20:05:38 -07:00
Harshavardhana 1a884cd8e1
fix: deleting objects was not working after upgrades (#13242)
DeleteObject() on existing objects before `xl.json` to
`xl.meta` change were not working, not sure when this
regression was added. This PR fixes this properly.

Also this PR ensures that we perform rename of xl.json
to xl.meta only during "write" phase of the call i.e
either during Healing or PutObject() overwrites.

Also handles few other scenarios during migration where
`backendEncryptedFile` was missing deleteConfig() will
fail with `configNotFound` this case was not ignored,
which can lead to failure during upgrades.
2021-09-17 19:34:48 -07:00
Poorna Krishnamoorthy 18f008f7c7
Fix retention enforcement check for deleted object versions (#13240)
if an object is pending version purge, it should be treated
as ErrNone in retention enforcement check
2021-09-17 15:21:24 -07:00
Harshavardhana 6d42569ade
remove ListBucketsMetadata instead add them to AccountInfo() (#13241) 2021-09-17 15:02:21 -07:00
Harshavardhana 5ed781a330
check for context canceled after competing for locks (#13239)
once we have competed for locks, verify if the
context is still valid - this is to ensure that
we do not start readdir() or read() calls on the
drives on canceled connections.
2021-09-17 14:11:01 -07:00
Harshavardhana 66fcd02aa2
de-couple walkMu and walkReadMu for some granularity (#13231)
This commit brings two locks instead of single lock for
WalkDir() calls on top of c25816eabc.

The main reason is to avoid contention between readMetadata()
and ListDir() calls, ListDir() can take time on prefixes that
are huge for readdir() but this shouldn't end up blocking
all readMetadata() operations, this allows for more room for
I/O while not overly penalizing all listing operations.
2021-09-17 12:14:12 -07:00
Andreas Auernhammer 1fc0e9a6aa
sts: allow clients to send certificate chain (#13235)
This commit fixes an issue in the `AssumeRoleWithCertificate`
handler.

Before clients received an error when they send
a chain of X.509 certificates (their client certificate as
well as intermediate / root CAs).

Now, client can send a certificate chain and the server
will only consider non-CA / leaf certificates as possible
client certificate candidates. However, the client still
can only send one certificate.

Signed-off-by: Andreas Auernhammer <hi@aead.dev>
2021-09-17 09:37:01 -07:00
Anis Elleuch 91567ba916
s3: Put bucket tagging to return an error when bucket is not found (#13232)
instead of creating new metadata in .minio.sys directory
2021-09-17 08:32:32 -07:00
Klaus Post d80826b05d
Clean up metacache saver (#13225)
Don't report success before the listing has actually finished. 
This will make stop conditions more clear.
2021-09-16 13:35:25 -07:00
Harshavardhana 45bcf73185
feat: Add ListBucketsWithMetadata extension API (#13219) 2021-09-16 09:52:41 -07:00
Poorna Krishnamoorthy 78dc08bdc2
remove s3:ReplicateDelete permission check from DeleteObject APIs (#13220) 2021-09-15 23:02:16 -07:00
Klaus Post f98f115ac2
fs: Fix non-progressing scanner (#13218)
Scanner would keep doing the same cycle in FS mode leading to missed updates.

Add a few sanity checks and handle errors better.
2021-09-15 09:24:41 -07:00
Minio Trusted bf409936e7 Update yaml files to latest version RELEASE.2021-09-15T04-54-25Z 2021-09-15 08:04:57 +00:00
Shireesh Anjal b4364723ef
Add config to store subnet license (#13194)
Command to set subnet license:

`mc admin config set {alias} subnet license={token}`

Signed-off-by: Shireesh Anjal <shireesh@minio.io>
Co-authored-by: Harshavardhana <harsha@minio.io>
2021-09-14 21:54:25 -07:00
Harshavardhana bcc6359dec support Console UI with userInfo claims for OpenID 2021-09-14 17:09:18 -07:00
Harshavardhana 787a72a993
make sure to ignore the rootDisk when healing drives (#13209)
fixes #13208
2021-09-14 15:10:00 -07:00
Harshavardhana d9eb962969
allow admin API to support UNSIGNED-PAYLOAD (#13207)
admin API requests do not support x-amz-content-sha256
set with UNSIGNED-PAYLOAD, keep this consistent and
support it properly.
2021-09-14 13:55:24 -07:00
Anis Elleuch f221153776
s3-gateway: Allow encryption S3 passthrough for SSE-S3 (#13204)
This reverts commit 35cbe43b6d.
2021-09-14 12:55:32 -07:00
Harshavardhana 67596ef0cc
fix sse-kms context unmarshal failure (#13206)
json.Unmarshal expects a pointer receiver, otherwise
kms.Context unmarshal fails with lack of pointer receiver,
this becomes complicated due to type aliasing over
map[string]string - fix it properly.
2021-09-14 12:52:46 -07:00
Klaus Post bf5bfe589f
xlmeta: Recover corrupted metadata (#13205)
When unable to load existing metadata new versions 
would not be written. This would leave objects in a 
permanently unrecoverable state

Instead, start with clean metadata and write the incoming data.
2021-09-14 11:34:25 -07:00
Harshavardhana af78c3925a
add userinfo support for OpenID (#12469)
Some identity providers like GitLab do not provide
information about group membership as part of the
identity token claims. They only expose it via OIDC compatible
'/oauth/userinfo' endpoint, as described in the OpenID
Connect 1.0 sepcification.

But this of course requires application to make sure to add
additional accessToken, since idToken cannot be re-used to
perform the same 'userinfo' call. This is why this is specialized
requirement. Gitlab seems to be the only OpenID vendor that requires
this support for the time being.

fixes #12367
2021-09-13 16:22:14 -07:00
Harshavardhana d144958669 update helm to v3.1.1 - saAccount fix 2021-09-13 09:43:10 -07:00
Klaus Post 5a64003f6f
select: Return null for non-exiting column indexes (#13196)
Fixes #13186
2021-09-13 09:13:25 -07:00
Kanagaraj M 311718309c
fix ServiceAccount creation in helm chart (#13197)
Fixed the variable name on the templated and added name.
2021-09-13 09:12:20 -07:00
Anis Elleuch 98479d7ffd
Fix deadlock when error during metacache generation (#13201)
A typo forgot to release a lock after acquiring it.
2021-09-13 09:11:39 -07:00
Harshavardhana 90e505e58f calculate API requests/error as increase() intervals not as rate() 2021-09-12 11:28:28 -07:00
Harshavardhana 410b8dd0fd add service monitor support and service account support 2021-09-12 11:19:27 -07:00
Anis Elleuch c2f25b6f62
gateway/s3: allow tracing requests to backend service (#13189)
fixes #13089
fixes #13133

Co-authored-by: Anis Elleuch <anis@min.io>
Co-authored-by: Harshavardhana <harsha@minio.io>
2021-09-11 09:20:01 -07:00
Krishna Srinivas 03a2a74697
Support speedtest autotune on the server side (#13086) 2021-09-10 17:43:34 -07:00
ArthurMa 2807c11410
http hook should accept more than 200 statusCode (#13180)
Co-authored-by: Klaus Post <klauspost@gmail.com>
2021-09-10 14:27:37 -07:00
Harshavardhana 39d51ce845 fix: add Dockerfile.release* /opt/bin writable 2021-09-09 22:27:33 -07:00
Minio Trusted a216583d95 Update yaml files to latest version RELEASE.2021-09-09T21-37-07Z 2021-09-09 23:50:26 +00:00
Harshavardhana 5c448b1b97
fix: update allowed max chunk payloadsize to 16MiB (#13169)
fixes #13163
2021-09-09 14:37:07 -07:00
Klaus Post 7f49c38e2d
Recover corrupted usage files if any (#13179) 2021-09-09 11:24:22 -07:00
Klaus Post 0e7fdcee30
Healing: Decide healing inlining based on metadata (#13178)
Don't perform an independent evaluation of inlining, but mirror the decision made when uploading the object.

Leads to some objects being inlined or not based on new metrics. Instead respect previous decision.
2021-09-09 08:55:43 -07:00
Poorna Krishnamoorthy 418f8bed6a
Detect multipart uploads correctly in unencrypted case (#13176)
This is a fix building on #13171 to ensure objects uploaded using multipart are 
replicated as multipart
2021-09-09 07:52:49 -07:00
soudhaf 950fe73c4f
fix: minio console nginx config (#13156) 2021-09-08 23:41:06 -07:00
Harshavardhana 0892f1e406
fix: multipart replication and encrypted etag for sse-s3 (#13171)
Replication was not working properly for encrypted
objects in single PUT object for preserving etag,

We need to make sure to preserve etag such that replication
works properly and not gets into infinite loops of copying
due to ETag mismatches.
2021-09-08 22:25:23 -07:00