Commit Graph

9987 Commits

Author SHA1 Message Date
Aditya Manthramurthy ddf84f8257
fix: concurrency bug in site-replication (#14786)
The site replication status call was using a loop iteration variable sent
directly into go-routines instead of being passed as an argument. As the
variable is being updated in the loop, previously launched go routines do not
necessarily use the value at the time they were launched.
2022-04-20 16:20:07 -07:00
Harshavardhana 507f993075
attempt to real resolve when there is a quorum failure on reads (#14613) 2022-04-20 12:49:05 -07:00
Harshavardhana 73a6a60785
fix: replication deleteObject() regression and CopyObject() behavior (#14780)
This PR fixes two issues

- The first fix is a regression from #14555, the fix itself in #14555
  is correct but the interpretation of that information by the
  object layer code for "replication" was not correct. This PR
  tries to fix this situation by making sure the "Delete" replication
  works as expected when "VersionPurgeStatus" is already set.

  Without this fix, there is a DELETE marker created incorrectly on
  the source where the "DELETE" was triggered.

- The second fix is perhaps an older problem started since we inlined-data
  on the disk for small objects, CopyObject() incorrectly inline's
  a non-inlined data. This is due to the fact that we have code where
  we read the `part.1` under certain conditions where the size of the
  `part.1` is less than the specific "threshold".

  This eventually causes problems when we are "deleting" the data that
  is only inlined, which means dataDir is ignored leaving such
  dataDir on the disk, that looks like an inconsistent content on
  the namespace.

fixes #14767
2022-04-20 10:22:05 -07:00
Anis Elleuch cf4cf58faf
Do not allow parallel upgrade in one server (#14782)
It is wasteful to allow parallel upgrades of MinIO server. This also generates
 weird error invoked by selfupdate module when it happens such as:

'rename /opt/bin/.minio.old /opt/bin/..minio.old.old'
2022-04-20 06:18:21 -07:00
polaris-megrez 6bc3c74c0c
honor client context in IAM user/policy listing calls (#14682) 2022-04-19 09:00:19 -07:00
Harshavardhana 598ce1e354
supply prefix filtering when necessary (#14772)
currently filterPefix was never used and set
that would filter out entries when needed
when `prefix` doesn't end with `/` - this
often leads to objects getting Walked(), Healed()
that were never requested by the caller.
2022-04-19 08:20:48 -07:00
Aditya Manthramurthy 4685b76e08
Update dperf v0.3.6 (#14773) 2022-04-19 02:40:36 -07:00
Minio Trusted 78c9109f6c update console to v0.15.14 2022-04-18 17:29:56 -07:00
Harshavardhana 7e248fc0ba
wait on parallel decom to complete before returning (#14764)
without this wait there is a potential for some objects
that are in actively being decommissioned would cancel,
however the decommission status might wrongly conclude
this as "Complete".

To avoid this make sure to add waitgroups on the parallel
workers, allowing parallel copies to complete fully before
we return.
2022-04-18 13:26:29 -07:00
Daniel Valdivia c526fa9119
Support console UI access at a subpath on a subdomain (#14761)
fixes #14285 

Signed-off-by: Daniel Valdivia <18384552+dvaldivia@users.noreply.github.com>
2022-04-17 16:01:49 -07:00
Harshavardhana 520e0fd985 update helm to v3.6.6 2022-04-17 14:46:44 -07:00
Yi Siqi 54a7eba358
Support overriding existing secrets (#14690) 2022-04-16 07:36:50 -07:00
Minio Trusted 1494ba2e6e Update yaml files to latest version RELEASE.2022-04-16T04-26-02Z 2022-04-16 05:03:00 +00:00
Anis Elleuch a5b3548ede
Bring back listing LDAP users temporarly (#14760)
In previous releases, mc admin user list would return the list of users
that have policies mapped in IAM database. However, this was removed but
this commit will bring it back until we revamp this.
2022-04-15 21:26:02 -07:00
Harshavardhana 8318aa0113
cancel active routine only after metadata has been saved (#14757)
currently updated pool.bin was not saved properly, that would
lead to unable to remove a pool upon a successful decommission.

fixes #14756
2022-04-15 13:16:15 -07:00
Harshavardhana e69c42956b
fix: IAM reload should only list at config/iam/ precisely (#14753) 2022-04-15 12:12:45 -07:00
Harshavardhana 53ca589c11 update deps for minio-go/v7 and jwt/v4 2022-04-15 00:50:22 -07:00
Daniel Valdivia ca8ff8718e
Update Console v0.15.13 (#14751)
Signed-off-by: Daniel Valdivia <18384552+dvaldivia@users.noreply.github.com>
2022-04-14 18:35:00 -07:00
Aditya Manthramurthy e8e48e4c4a
S3 select switch to new parquet library and reduce locking (#14731)
- This change switches to a new parquet library
- SelectObjectContent now takes a single lock at the beginning and holds it
during the operation. Previously the operation took a lock every time the
parquet library performed a Seek on the underlying object stream.
- Add basic support for LogicalType annotations for timestamps.
2022-04-14 06:54:47 -07:00
Minio Trusted 67e17ed3f8 update helm v3.6.5
Signed-off-by: Minio Trusted <trusted@minio.io>
2022-04-13 15:45:54 -07:00
Harshavardhana 2a6a40e93b
enable go1.18.x builds (#14746) 2022-04-13 14:21:55 -07:00
Harshavardhana eda34423d7 update gofumpt -w - new changes 2022-04-13 12:00:11 -07:00
Yi Siqi 7ce1f6e736
Support templating accessKey existingSecret and bucket name (#14643) 2022-04-13 11:58:29 -07:00
Shireesh Anjal 5c53620a72
Include speedtest as part of healthinfo api (#14696)
Execute the object, drive and net speedtests as part of the healthinfo
(if requested by the client), and include their result in the response.

The options for the speedtests have been picked from the default values
used by `mc support perf` command.
2022-04-12 13:17:44 -07:00
Krishna Srinivas 5f94cec1e2
Allow parallel decom migration threads to be more than erasure sets (#14733) 2022-04-12 10:49:53 -07:00
Minio Trusted 646350fa7f Update yaml files to latest version RELEASE.2022-04-12T06-55-35Z 2022-04-12 07:23:20 +00:00
Aditya Manthramurthy e162a055cc
Bump up console to v0.15.11 (#14734) 2022-04-11 23:55:35 -07:00
Krishnan Parthasarathi 28d3ad3ada
Honor object retention when applying ILM policies (#14732) 2022-04-11 21:55:56 -07:00
Harshavardhana 0bd44a7764 update helm v3.6.4 2022-04-11 18:30:28 -07:00
Aditya Manthramurthy 8be6d887e2
Bump up dperf to 0.3.5 (#14730) 2022-04-11 15:50:15 -07:00
Aditya Manthramurthy 66b14a0d32
Fix service account privilege escalation (#14729)
Ensure that a regular unprivileged user is unable to create service accounts for other users/root.
2022-04-11 15:30:28 -07:00
Harshavardhana 153a612253
fetch bucket retention config once for ILM evalAction (#14727)
This is mainly an optimization, does not change any
existing functionality.
2022-04-11 13:25:32 -07:00
Krishnan Parthasarathi 1a1b55e133
Add support for minio tier type (#14468) 2022-04-11 13:24:40 -07:00
Naveen 879de20edf
Set permissions for GitHub actions (#14693)
- Included permissions for the action. https://github.com/ossf/scorecard/blob/main/docs/checks.md#token-permissions

https://docs.github.com/en/actions/using-jobs/assigning-permissions-to-jobs

[Keeping your GitHub Actions and workflows secure Part 1: Preventing pwn requests](https://securitylab.github.com/research/github-actions-preventing-pwn-requests/)

> Restrict the GitHub token permissions only to the required ones; this way, even if the attackers will succeed in compromising your workflow, they won’t be able to do much.

https://www.legitsecurity.com/blog/github-privilege-escalation-vulnerability

Signed-off-by: naveensrinivasan <172697+naveensrinivasan@users.noreply.github.com>
2022-04-11 02:45:59 -07:00
Harshavardhana e77ad3f9bb
make sure to pass Lifecycle if set for List filtering (#14722)
PR #14606 never really passed the Lifecycle filter
down to the listing callers to ensure skipping the
entries.
2022-04-10 11:14:52 -07:00
Harshavardhana 4ce86ff5fa
align atomic variables once more for 32bit (#14721) 2022-04-09 22:19:44 -07:00
Daniel Valdivia e290c010e6
Console v0.15.10 (#14723)
Signed-off-by: Daniel Valdivia <hola@danielvaldivia.com>
2022-04-09 20:55:36 -07:00
Minio Trusted 33d267fa1b Update yaml files to latest version RELEASE.2022-04-09T15-09-52Z 2022-04-09 20:23:18 +00:00
Harshavardhana 601a744159
pass the necessary query params for remote NSSCanner (#14719)
fixes a regression from #14464
2022-04-09 08:09:52 -07:00
Minio Trusted f630d7c3fa Update yaml files to latest version RELEASE.2022-04-08T19-44-35Z 2022-04-08 23:35:38 +00:00
Harshavardhana 91bfefcf8c move back go.mod to 1.17 2022-04-08 16:25:20 -07:00
Poorna a1b01e6d5f
Combine profiling start/stop APIs into one (#14662)
Take profile duration as a query parameter for profile API
2022-04-08 12:44:35 -07:00
Krishna Srinivas 48594617b5
Parallelize decommissioning process (#14704) 2022-04-07 23:19:13 -07:00
Krishna Srinivas b35b9dcff7
Use S3 client for uplooads/downloads during perf test (#14570) 2022-04-07 21:20:40 -07:00
Lenin Alevski a3e317773a
Skip commented lines when parsing MinIO configuration file (#14710)
Signed-off-by: Lenin Alevski <alevsk.8772@gmail.com>
2022-04-07 16:02:51 -07:00
Anis Elleuch 16431d222c
heal: Enable periodic bitrot scan configuration (#14464) 2022-04-07 08:10:40 -07:00
Harshavardhana ee49a23220
resume/start decommission on the first node of the pool under decommission (#14705)
Additionally fixes

- IsSuspended() can use read locks
- Avoid double cancels panic on canceler
2022-04-06 23:42:05 -07:00
Harshavardhana a9eef521ec skip config/history/ during IAM load (#14698) 2022-04-06 21:03:41 -07:00
Klaus Post 901d33b59c
Tweak listing quorum (#14703)
Always go for 50% quorum, and only use non-healing disks.

Fixes #14635
2022-04-06 12:24:21 -07:00
Daniel Valdivia 255116fde7
Update Console Dependency to v0.15.9 (#14699)
Signed-off-by: Daniel Valdivia <hola@danielvaldivia.com>
2022-04-05 20:46:17 -07:00