Commit Graph

8705 Commits

Author SHA1 Message Date
Harshavardhana
3c70eca758
enable SO_REUSEPORT sockets, allow cleaner reuse of time_waits (#13307)
Refer here https://lwn.net/Articles/542629/
2021-09-27 09:27:16 -07:00
Anis Elleuch
68a2d6fc40
xl: Avoid empty endpoints (#13299)
An endpoint can be empty when a disk is offline or something 
wrong with it. Avoid it by filling erasureSets.endpointStrings 
with values from arguments.
2021-09-25 10:51:03 -07:00
Minio Trusted
a5923a5d51 Update yaml files to latest version RELEASE.2021-09-24T00-24-24Z 2021-09-24 19:20:44 +00:00
Harshavardhana
769f0b1e24
fix: fallback listing on drives that are unformatted, disconnected (#13249) 2021-09-23 17:24:24 -07:00
Harshavardhana
a1271d984f
add missing notification subsystem targets (#13294)
fixes #13293
2021-09-23 17:23:50 -07:00
Anis Elleuch
db65ec4674
update: Add permission check before starting to update (#13291) 2021-09-23 12:57:21 -07:00
Harshavardhana
a984c55cf9 add namespace support with helm v3.1.6 release 2021-09-23 12:56:39 -07:00
Mitchell Maler
cdd0828c4a
add namespace: {{ .Release.Namespace | quote }} to each namespaced resource (#13285) 2021-09-23 12:56:08 -07:00
Harshavardhana
1984d0671b
update reedsolomon package to v1.9.13 (#13288) 2021-09-23 10:46:24 -07:00
Krishnan Parthasarathi
3e4efff73d
Allow only account-key updates for azure tier (#13276) 2021-09-23 02:34:31 -07:00
Minio Trusted
f4d1b7c603 Update yaml files to latest version RELEASE.2021-09-23T04-46-24Z 2021-09-23 07:42:55 +00:00
Harshavardhana
200caab82b
fix: multi-pool setup make sure acquire locks properly (#13280)
This was a regression introduced in '14bb969782'
this has the potential to cause corruption when
there are concurrent overwrites attempting to update
the content on the namespace.

This PR adds a situation where PutObject(), CopyObject()
compete properly for the same locks with NewMultipartUpload()
however it ends up turning off competing locks for the actual
object with GetObject() and DeleteObject() - since they do not
compete due to concurrent I/O on a versioned bucket it can lead
to loss of versions.

This PR fixes this bug with multi-pool setup with replication
that causes corruption of inlined data due to lack of competing
locks in a multi-pool setup.

Instead CompleteMultipartUpload holds the necessary
locks when finishing the transaction, knowing the exact
location of an object to schedule the multipart upload
doesn't need to compete in this manner, a pool id location
for existing object.
2021-09-22 21:46:24 -07:00
Harshavardhana
f9b104f37b Revert "s3: Put bucket tagging to return an error when bucket is not found (#13232)"
This reverts commit 91567ba916.

Revert because the error was incorrectly converted, there are
callers that rely on errConfigNotFound and it also took away
the migration code.

Instead the correct fix is PutBucketTaggingHandler() which
is already added.
2021-09-22 20:06:25 -07:00
Harshavardhana
c25b482301 update console to v0.10.1 2021-09-22 19:42:31 -07:00
Krishnan Parthasarathi
31d7cc2cd4
erasure: Set fi.IsLatest when adding a new version (#13277) 2021-09-22 19:17:09 -07:00
Poorna Krishnamoorthy
19ecdc75a8
replication: Simplify metrics calculation (#13274)
Also doing some code cleanup
2021-09-22 10:48:45 -07:00
Harshavardhana
46724508f8 update helm to v3.1.5 2021-09-22 09:52:01 -07:00
Kanagaraj M
51b0194b8a
fix servicemonitor scheme in helm chart (#13271)
Added `scheme` to servicemonitor.yaml
2021-09-22 09:16:41 -07:00
Harshavardhana
9a27c4a2f0
do not panic if DNS_WEBHOOK_ENDPOINT is not reachable (#13265) 2021-09-22 09:16:12 -07:00
Krishnan Parthasarathi
32df742b85
fix: nil pointer access in warmBackendAzure.Put (#13263) 2021-09-21 15:47:30 -07:00
Harshavardhana
8392765213
healObjects() should cancel() context before writing to errCh (#13262)
also remove HealObjects() code from dataScanner running another
listing from the data-scanner is super in-efficient and in-fact
this code is redundant since we already attempt to heal all
dangling objects anyways.
2021-09-21 14:55:17 -07:00
Poorna Krishnamoorthy
806b10b934
fix: improve error messages returned during replication setup (#13261) 2021-09-21 13:03:20 -07:00
Aditya Manthramurthy
1fa0553c71
Remove support for elasticsearch versions < 7.x (#13260) 2021-09-21 12:57:10 -07:00
Harshavardhana
50a68a1791
allow S3 gateway to support object locked buckets (#13257)
- Supports object locked buckets that require
  PutObject() to set content-md5 always.
- Use SSE-S3 when S3 gateway is being used instead
  of SSE-KMS for auto-encryption.
2021-09-21 09:02:15 -07:00
Poorna Krishnamoorthy
0b55a0423e
fix: cache usage deserialization from v5 to v6 (#13258) 2021-09-21 09:01:51 -07:00
Anis Elleuch
565d95a377
Enable console logging when server debug is enabled (#13259)
_MINIO_SERVER_DEBUG will enable console logging.
2021-09-21 09:01:29 -07:00
Harshavardhana
f492f72154
add 3site replication script (#13256) 2021-09-20 18:24:24 -07:00
Harshavardhana
4d84f0f6f0
fix: support existing folders in single drive mode (#13254)
This PR however also proceeds to simplify the loading
of various subsystems such as

- globalNotificationSys
- globalTargetSys

converge them directly into single bucket metadata sys
loader, once that is loaded automatically every other
target should be loaded and configured properly.

fixes #13252
2021-09-20 17:41:01 -07:00
Aditya Manthramurthy
a0d0c8e4af
Allow root user to create service accounts in LDAP (#13221)
Additionally, fix a bug in service account creation for LDAP users: the
LDAP short username was not associated with the service account.
2021-09-20 14:28:19 -07:00
Harshavardhana
bef748abbd fix: labels are strings in helm 2021-09-19 22:30:22 -07:00
Poorna Krishnamoorthy
c4373ef290
Add support for multi site replication (#12880) 2021-09-18 13:31:35 -07:00
Minio Trusted
0b8c5a6872 Update yaml files to latest version RELEASE.2021-09-18T18-09-59Z 2021-09-18 20:28:07 +00:00
Harshavardhana
829ecb2086 update helm to 3.1.3 2021-09-18 11:09:59 -07:00
Goncharov Sergey
d3564a4b09
Split toleration, nodeSelector and affinity from main pods and jobs (#13247) 2021-09-18 11:09:09 -07:00
Harshavardhana
a244753f47 update to helm v3.1.2 2021-09-17 21:26:47 -07:00
Goncharov Sergey
745782a77a
fix: service monitoring labels in helm release (#13238)
fixes #13236
2021-09-17 21:18:27 -07:00
Poorna Krishnamoorthy
246cbe1312
update minio-go with healthcheck fixes (#13244) 2021-09-17 21:14:13 -07:00
Poorna Krishnamoorthy
6c941122eb
cancel active goroutine when remote target is edited (#13243) 2021-09-17 20:05:38 -07:00
Harshavardhana
1a884cd8e1
fix: deleting objects was not working after upgrades (#13242)
DeleteObject() on existing objects before `xl.json` to
`xl.meta` change were not working, not sure when this
regression was added. This PR fixes this properly.

Also this PR ensures that we perform rename of xl.json
to xl.meta only during "write" phase of the call i.e
either during Healing or PutObject() overwrites.

Also handles few other scenarios during migration where
`backendEncryptedFile` was missing deleteConfig() will
fail with `configNotFound` this case was not ignored,
which can lead to failure during upgrades.
2021-09-17 19:34:48 -07:00
Poorna Krishnamoorthy
18f008f7c7
Fix retention enforcement check for deleted object versions (#13240)
if an object is pending version purge, it should be treated
as ErrNone in retention enforcement check
2021-09-17 15:21:24 -07:00
Harshavardhana
6d42569ade
remove ListBucketsMetadata instead add them to AccountInfo() (#13241) 2021-09-17 15:02:21 -07:00
Harshavardhana
5ed781a330
check for context canceled after competing for locks (#13239)
once we have competed for locks, verify if the
context is still valid - this is to ensure that
we do not start readdir() or read() calls on the
drives on canceled connections.
2021-09-17 14:11:01 -07:00
Harshavardhana
66fcd02aa2
de-couple walkMu and walkReadMu for some granularity (#13231)
This commit brings two locks instead of single lock for
WalkDir() calls on top of c25816eabc.

The main reason is to avoid contention between readMetadata()
and ListDir() calls, ListDir() can take time on prefixes that
are huge for readdir() but this shouldn't end up blocking
all readMetadata() operations, this allows for more room for
I/O while not overly penalizing all listing operations.
2021-09-17 12:14:12 -07:00
Andreas Auernhammer
1fc0e9a6aa
sts: allow clients to send certificate chain (#13235)
This commit fixes an issue in the `AssumeRoleWithCertificate`
handler.

Before clients received an error when they send
a chain of X.509 certificates (their client certificate as
well as intermediate / root CAs).

Now, client can send a certificate chain and the server
will only consider non-CA / leaf certificates as possible
client certificate candidates. However, the client still
can only send one certificate.

Signed-off-by: Andreas Auernhammer <hi@aead.dev>
2021-09-17 09:37:01 -07:00
Anis Elleuch
91567ba916
s3: Put bucket tagging to return an error when bucket is not found (#13232)
instead of creating new metadata in .minio.sys directory
2021-09-17 08:32:32 -07:00
Klaus Post
d80826b05d
Clean up metacache saver (#13225)
Don't report success before the listing has actually finished. 
This will make stop conditions more clear.
2021-09-16 13:35:25 -07:00
Harshavardhana
45bcf73185
feat: Add ListBucketsWithMetadata extension API (#13219) 2021-09-16 09:52:41 -07:00
Poorna Krishnamoorthy
78dc08bdc2
remove s3:ReplicateDelete permission check from DeleteObject APIs (#13220) 2021-09-15 23:02:16 -07:00
Klaus Post
f98f115ac2
fs: Fix non-progressing scanner (#13218)
Scanner would keep doing the same cycle in FS mode leading to missed updates.

Add a few sanity checks and handle errors better.
2021-09-15 09:24:41 -07:00
Minio Trusted
bf409936e7 Update yaml files to latest version RELEASE.2021-09-15T04-54-25Z 2021-09-15 08:04:57 +00:00