Commit Graph

120 Commits

Author SHA1 Message Date
Harshavardhana
e12ab486a2
avoid using os.Getenv for internal code, use env.Get() instead (#17688) 2023-07-20 07:52:49 -07:00
Harshavardhana
3e196fa7b3
fix: ILM newer noncurrent version limit must return correct versions (#17652)
objects/versions that are not expired via NewerNoncurrentVersions
must be properly returned to be applied under further ILM actions.

this would cause legitimately expired objects to be missed
from expiration.
2023-07-14 16:42:35 -07:00
Harshavardhana
a566bcf613
treat 0-byte objects to honor same quorum as delete marker (#17633)
on unversioned buckets its possible that 0-byte objects
might lose quorum on flaky systems, allow them to be same
as DELETE markers. Since practically speak they have no
content.
2023-07-11 21:53:49 -07:00
Harshavardhana
82075e8e3a
use strconv variants to improve on performance per 'op' (#17626)
```
BenchmarkItoa
BenchmarkItoa-8         	673628088	         1.946 ns/op	       0 B/op	       0 allocs/op
BenchmarkFormatInt
BenchmarkFormatInt-8    	592919769	         2.012 ns/op	       0 B/op	       0 allocs/op
BenchmarkSprint
BenchmarkSprint-8       	26149144	        49.06 ns/op	       2 B/op	       1 allocs/op
BenchmarkSprintBool
BenchmarkSprintBool-8   	26440180	        45.92 ns/op	       4 B/op	       1 allocs/op
BenchmarkFormatBool
BenchmarkFormatBool-8   	1000000000	         0.2558 ns/op	       0 B/op	       0 allocs/op
```
2023-07-11 07:46:58 -07:00
Harshavardhana
8522905d97 update helm linting workflow 2023-07-10 20:11:51 -07:00
Ian Martin
90c2129f44
Add helm chart linting to CI workflow (#17606) 2023-07-08 15:41:12 -07:00
Anis Eleuch
66bea3942a
CI/CD to stop one node per pool in the two pools mint test (#17518)
This is to make sure that all S3 ops work when there is enough quorum
2023-07-07 00:10:13 -07:00
Harshavardhana
6b7c98bd0f
make sure we pick up the right Go version in vulncheck (#17388) 2023-06-09 00:32:12 -07:00
Harshavardhana
75c6fc4f02
only allow decryption of etag for only sse-s3 (#17335) 2023-06-05 13:08:51 -07:00
Harshavardhana
4a425cbac1
cleanup scripts and apply shfmt (#17284) 2023-05-25 22:07:25 -07:00
Harshavardhana
876f51a708 remove minio-js from mint tests until next minio-js release 2023-05-17 09:09:54 -07:00
Harshavardhana
06557fe8be
allow decommissioned pools to be removed while others are finishing (#17221) 2023-05-16 16:00:57 -07:00
Harshavardhana
b53376a3a4
change directory objects to never create new versions (#17109) 2023-05-02 16:09:33 -07:00
Harshavardhana
7ae69accc0
allow root user to be disabled via config settings (#17089) 2023-04-28 12:24:14 -07:00
Klaus Post
62c3df0ca3
fix: directory listing on Go 1.20 windows (#16976) 2023-04-05 14:36:49 -07:00
Klaus Post
ae011663e8
mint: Ignore teardown errors (#16979) 2023-04-05 11:10:24 -07:00
Harshavardhana
f65cce4317
move mint tests to separate folders to not confuse GitHub (#16940) 2023-03-31 14:38:10 -07:00
Harshavardhana
13151cbb2b
[testing] add mint runner test (#16868) 2023-03-29 11:38:43 -07:00
Aditya Manthramurthy
c95ede35c1
Switch windows CI back to go 1.19.x (#16755) 2023-03-03 15:19:28 -08:00
Aditya Manthramurthy
7415e1aa56
Switch to go1.20 in CI (#16743) 2023-03-03 10:15:03 -08:00
Cesar Celis Hernandez
422c396d73
Removing old action that is no longer needed (#16550) 2023-02-07 07:06:29 +05:30
Cesar Celis Hernandez
8ca14e6267
Updating enterprise action (#16518) 2023-02-02 19:23:31 +05:30
Alex
1a17fc17bb
GitHub Workflows security hardening (#15708) 2023-01-21 09:55:17 -08:00
Harshavardhana
75faef888e
disable builds for go1.18 (#16332) 2022-12-30 11:37:07 -08:00
Harshavardhana
dfe73629a3
fix: delete marker discrepancies via DeleteObject() API (#16195) 2022-12-08 18:15:16 -08:00
Anis Elleuch
52f4124678
Remove go1.18 from Github workflow tests (#16180) 2022-12-06 09:11:20 -08:00
Cesar Celis Hernandez
90e37a8745
Start PR on Enterprise when there is new MinIO Version (#16121) 2022-12-04 02:29:25 -08:00
yanggang
d2c9a9e395
add windows port allot by "netsh dynamicport" (#15986) 2022-11-02 09:10:26 -07:00
Harshavardhana
07b6dce1a5
remove nancy, we rely on vulncheck from now on (#15893) 2022-10-18 10:45:44 -07:00
Minio Trusted
5c034e26bd fix: govulncheck must use go1.19.2 2022-10-06 12:42:59 -07:00
Harshavardhana
cef0fb1434
remove build asset caching (#15807) 2022-10-06 11:36:07 -07:00
Harshavardhana
b04c0697e1
validate correct ETag for the parts sent during CompleteMultipart (#15751) 2022-09-23 21:17:08 -07:00
Klaus Post
5c61c3ccdc
Fix flaky TestGetObjectWithOutdatedDisks (#15687)
On occasion this test fails:

```
2022-09-12T17:22:44.6562737Z === RUN   TestGetObjectWithOutdatedDisks
2022-09-12T17:22:44.6563751Z     erasure-object_test.go:1214: Test 2: Expected data to have md5sum = `c946b71bb69c07daf25470742c967e7c`, found `7d16d23f07072af1a809707ba101ae07`
2
```

Theory: Both objects are written with the same timestamp due to lower timer resolution on Windows. This results in secondary resolution, which is deterministic, but random.

Solution: Instead of hacking in a wait we request the specific version we want. Should still keep the test relevant.

Bonus: Remote action dependency for vulncheck
2022-09-14 08:17:39 -07:00
Harshavardhana
a71629d4dd
use go1.19.1 for vulncheck (#15680) 2022-09-12 20:16:39 -07:00
Andreas Auernhammer
d6132b854f
add vulncheck action to CI (#15676)
Signed-off-by: Andreas Auernhammer <hi@aead.dev>
2022-09-12 10:10:30 -07:00
Anis Elleuch
bb855499e1
Update PR template to make it easy to request for doc changes (#15665)
A link can be clicked to open a new request in minio/docs if something
in the documentation needs to be changed.
2022-09-07 15:25:02 -07:00
Harshavardhana
433b6fa8fe
upgrade golang-lint to the latest (#15600) 2022-08-26 12:52:29 -07:00
Harshavardhana
d350b666ff
feat: add idempotent delete marker support (#15521)
The bottom line is delete markers are a nuisance,
most applications are not version aware and this
has simply complicated the version management.

AWS S3 gave an unnecessary complication overhead
for customers, they need to now manage these
markers by applying ILM settings and clean
them up on a regular basis.

To make matters worse all these delete markers
get replicated as well in a replicated setup,
requiring two ILM settings on each site.

This PR is an attempt to address this inferior
implementation by deviating MinIO towards an
idempotent delete marker implementation i.e
MinIO will never create any more than single
consecutive delete markers.

This significantly reduces operational overhead
by making versioning more useful for real data.

This is an S3 spec deviation for pragmatic reasons.
2022-08-18 16:41:59 -07:00
Harshavardhana
a6e0ec4e6f
Add support converting non-inlined to inlined (#15444)
This is a feature to allow for inode compaction on
large clusters that use a lot of small files spread
across a large heirarchy.
2022-08-02 23:10:22 -07:00
Naveen
d46cf50760
chore(deps): Included dependency review (#14958)
> Dependency Review GitHub Action in your repository to enforce dependency
> reviews on your pull requests.
> The action scans for vulnerable versions of dependencies introduced by package version
> changes in pull requests,
> and warns you about the associated security vulnerabilities.
> This gives you better visibility of what's changing in a pull request,
> and helps prevent vulnerabilities from being added to your repository.

https://docs.github.com/en/code-security/supply-chain-security/understanding-your-software-supply-chain/about-dependency-review#dependency-review-enforcement

Signed-off-by: Naveen <172697+naveensrinivasan@users.noreply.github.com>
2022-07-27 20:53:26 -07:00
Harshavardhana
4c6498d726
move all CI/CD to go1.18 (#15401) 2022-07-25 15:27:20 -07:00
Harshavardhana
b6eb8dff64
Add decommission compression+encryption enabled tests (#15322)
update compression environment variables to follow
the expected sub-system style, however support fallback
mode.
2022-07-17 08:43:14 -07:00
Anis Elleuch
1cfa2e04bc
Add a github workflow test for root disk detection (#15267)
Use losetup to create fake disks, start a MinIO cluster, umount 
one disk, and fails if the mount point directory will have format.json
recreated. It should fail because the mount point directory will belong
to the root disk after unmount.
2022-07-13 16:29:44 -07:00
Harshavardhana
b311abed31
decom IAM, Bucket metadata properly (#15220)
Current code incorrectly passed the
config asset object name while decommissioning,
make sure that we pass the right object name
to be hashed on the newer set of pools.

This PR fixes situations after a successful
decommission, the users and policies might go
missing due to wrong hashed set.
2022-07-04 14:02:54 -07:00
Aditya Manthramurthy
addfa35d93
Add FIPS build to CI and add README.fips.md (#15038) 2022-06-04 18:25:37 -07:00
Aditya Manthramurthy
4629abd5a2
Add tests for Access Management Plugin (#14909) 2022-05-12 15:24:19 -07:00
Aditya Manthramurthy
0e502899a8
Add support for multiple OpenID providers with role policies (#14223)
- When using multiple providers, claim-based providers are not allowed. All
providers must use role policies.

- Update markdown config to allow `details` HTML element
2022-04-28 18:27:09 -07:00
Harshavardhana
507f993075
attempt to real resolve when there is a quorum failure on reads (#14613) 2022-04-20 12:49:05 -07:00
Harshavardhana
2a6a40e93b
enable go1.18.x builds (#14746) 2022-04-13 14:21:55 -07:00
Naveen
879de20edf
Set permissions for GitHub actions (#14693)
- Included permissions for the action. https://github.com/ossf/scorecard/blob/main/docs/checks.md#token-permissions

https://docs.github.com/en/actions/using-jobs/assigning-permissions-to-jobs

[Keeping your GitHub Actions and workflows secure Part 1: Preventing pwn requests](https://securitylab.github.com/research/github-actions-preventing-pwn-requests/)

> Restrict the GitHub token permissions only to the required ones; this way, even if the attackers will succeed in compromising your workflow, they won’t be able to do much.

https://www.legitsecurity.com/blog/github-privilege-escalation-vulnerability

Signed-off-by: naveensrinivasan <172697+naveensrinivasan@users.noreply.github.com>
2022-04-11 02:45:59 -07:00