Commit Graph

9714 Commits

Author SHA1 Message Date
Harshavardhana
16bc11e72e
fix: disallow newer policies, users & groups with space characters (#14845)
space characters at the beginning or at the end can lead to
confusion under various UI elements in differentiating the
actual name of "policy, user or group" - to avoid this behavior
this PR onwards we shall reject such inputs for newer entries.

existing saved entries will behave as is and are going to be
operable until they are removed/renamed to something more
meaningful.
2022-05-02 09:27:35 -07:00
Harshavardhana
2719f1efaa
fix: reject invalid r.Host headers (#14846)
r.Host headers can come in unparsed that may contain
invalid hostnames, reject such requests as invalid.

This is a continuation fix from #14844
2022-05-02 04:42:41 -07:00
Minio Trusted
cff1be0ae8 update helm release to v4.0.1 2022-05-01 23:10:34 -07:00
Harshavardhana
39ac62a1a1
fix: panic in browser redirect handler for unexpected r.Host (#14844)
```
panic: "GET /": invalid hostname
goroutine 148 [running]:
runtime/debug.Stack()
	runtime/debug/stack.go:24 +0x65
github.com/minio/minio/cmd.setCriticalErrorHandler.func1.1()
	github.com/minio/minio/cmd/generic-handlers.go:469 +0x8e
panic({0x2201f00, 0xc001f1ddd0})
	runtime/panic.go:1038 +0x215
github.com/minio/pkg/net.URL.String({{0x25aa417, 0x5}, {0x0, 0x0}, 0x0, {0xc000174380, 0xd7}, {0x0, 0x0}, {0x0, ...}, ...})
	github.com/minio/pkg@v1.1.23/net/url.go:97 +0xfe
github.com/minio/minio/cmd.setBrowserRedirectHandler.func1({0x49af080, 0xc0003c20e0}, 0xc00002ea00)
	github.com/minio/minio/cmd/generic-handlers.go:136 +0x118
net/http.HandlerFunc.ServeHTTP(0xc00002ea00, {0x49af080, 0xc0003c20e0}, 0xa)
	net/http/server.go:2047 +0x2f
github.com/minio/minio/cmd.setAuthHandler.func1({0x49af080, 0xc0003c20e0}, 0xc00002ea00)
	github.com/minio/minio/cmd/auth-handler.go:525 +0x3d8
net/http.HandlerFunc.ServeHTTP(0xc00002e900, {0x49af080, 0xc0003c20e0}, 0xc001f33701)
	net/http/server.go:2047 +0x2f
github.com/gorilla/mux.(*Router).ServeHTTP(0xc0025d0780, {0x49af080, 0xc0003c20e0}, 0xc00002e800)
	github.com/gorilla/mux@v1.8.0/mux.go:210 +0x1cf
github.com/rs/cors.(*Cors).Handler.func1({0x49af080, 0xc0003c20e0}, 0xc00002e800)
	github.com/rs/cors@v1.7.0/cors.go:219 +0x1bd
net/http.HandlerFunc.ServeHTTP(0x0, {0x49af080, 0xc0003c20e0}, 0xc00068d9f8)
	net/http/server.go:2047 +0x2f
github.com/minio/minio/cmd.setCriticalErrorHandler.func1({0x49af080, 0xc0003c20e0}, 0x4a5cd3)
	github.com/minio/minio/cmd/generic-handlers.go:476 +0x83
net/http.HandlerFunc.ServeHTTP(0x72, {0x49af080, 0xc0003c20e0}, 0x0)
	net/http/server.go:2047 +0x2f
github.com/minio/minio/internal/http.(*Server).Start.func1({0x49af080, 0xc0003c20e0}, 0x10000c001f1dda0)
	github.com/minio/minio/internal/http/server.go:105 +0x1b6
net/http.HandlerFunc.ServeHTTP(0x0, {0x49af080, 0xc0003c20e0}, 0x46982e)
	net/http/server.go:2047 +0x2f
net/http.serverHandler.ServeHTTP({0xc003dc1950}, {0x49af080, 0xc0003c20e0}, 0xc00002e800)
	net/http/server.go:2879 +0x43b
net/http.(*conn).serve(0xc000514d20, {0x49cfc38, 0xc0010c0e70})
	net/http/server.go:1930 +0xb08
created by net/http.(*Server).Serve
	net/http/server.go:3034 +0x4e8
```
2022-05-01 13:45:45 -07:00
Minio Trusted
f427dbbd60 Update yaml files to latest version RELEASE.2022-04-30T22-23-53Z 2022-05-01 01:33:28 +00:00
Harshavardhana
c3f689a7d9
JWKS should be parsed before usage (#14842)
fixes #14811
2022-04-30 15:23:53 -07:00
Harshavardhana
85f3a9f3b0 Remove Azure gateway implementation (#14418)
refer #14331
2022-04-29 12:51:23 -07:00
Klaus Post
13ba4b433d
Clean up cpuio profiling (#14838)
Don't start regular cpu profile as well. Use bed madmin const.
2022-04-29 09:35:42 -07:00
Minio Trusted
96f27a4965 Update yaml files to latest version RELEASE.2022-04-29T01-27-09Z 2022-04-29 06:32:50 +00:00
Aditya Manthramurthy
0e502899a8
Add support for multiple OpenID providers with role policies (#14223)
- When using multiple providers, claim-based providers are not allowed. All
providers must use role policies.

- Update markdown config to allow `details` HTML element
2022-04-28 18:27:09 -07:00
Harshavardhana
424b44c247
allow changing server command line from http->https (#14832)
this is allowed as long as order is preserved as is
on an existing setup, the new command line is updated
in `pool.bin` to facilitate future decommission's on
these pools.
2022-04-28 16:27:53 -07:00
Harshavardhana
01a71c366d
allow service accounts and temp credentials site-level healing (#14829)
This PR introduces support for site level

- service account healing
- temporary credentials healing
2022-04-28 02:39:00 -07:00
Harshavardhana
990fbeb3a4 rename true/false to on/off in bucket notification docs 2022-04-27 23:51:31 -07:00
Harshavardhana
5a9a898ba2
allow forcibly creating metadata on buckets (#14820)
introduce x-minio-force-create environment variable
to force create a bucket and its metadata as required,
it is useful in some situations when bucket metadata
needs recovery.
2022-04-27 04:44:07 -07:00
Sidhartha Mani
fe1fbe0005
standardize config help defaults (#14788) 2022-04-26 20:11:37 -07:00
Harshavardhana
c56a139fdc
fix: support decommissioning directory objects (#14822)
improvements in this PR include

- decommission objects that have __XLDIR__ suffix
- decommission objects that have `null` version on
  a versioned bucket.
- make sure to look for any "decom" failures to ensure
  that we do not wrong conclude decom as complete without
  all files getting copied over.
- break out eagerly upon first error for objects with
  multiple versions, leave the object as is for support
  debugging and analysis.
2022-04-26 20:06:41 -07:00
Anis Elleuch
df50eda811
Add number of versions in server info API (#14812)
The goal is to show the number of versions in the server info API.
2022-04-25 22:04:10 -07:00
Aditya Manthramurthy
f5d3313210
Increase context timeout for IAM concurrency test (#14817)
- This should reduce failures in Windows CI
2022-04-25 20:14:20 -07:00
Minio Trusted
97fcc9ff99 update helm release to v4.0.0 removes gcs gateway support
newer MinIO server removes "gcs" gateway support as per #14331
2022-04-25 19:41:39 -07:00
Minio Trusted
8a6b2b4447 Update yaml files to latest version RELEASE.2022-04-26T01-20-24Z 2022-04-26 02:08:20 +00:00
Aditya Manthramurthy
757eaeae92
Update console to v0.16.0 (#14816) 2022-04-25 18:20:24 -07:00
Daniel Valdivia
b7dd61f6bc
Fix double slash subpath for console (#14815)
Signed-off-by: Daniel Valdivia <18384552+dvaldivia@users.noreply.github.com>
2022-04-25 13:05:56 -07:00
Minio Trusted
d2a95a04a4 update pkg v1.1.22 2022-04-25 10:33:38 -07:00
Harshavardhana
0cc993f403 Remove GCS, HDFS gateway implementations #14418
refer #14331
2022-04-24 10:19:17 -07:00
Poorna
3a64580663
Add support for site replication healing (#14572)
heal bucket metadata and IAM entries for
sites participating in site replication from
the site with the most updated entry.

Co-authored-by: Harshavardhana <harsha@minio.io>
Co-authored-by: Aditya Manthramurthy <aditya@minio.io>
2022-04-24 02:36:31 -07:00
Harshavardhana
d087e28dce
start using t.SetEnv instead of os.Setenv (#14787) 2022-04-23 15:33:45 -07:00
Klaus Post
96adfaebe1
Make storage class config dynamic (#14791)
Updating the storage class is already thread safe, so we can do this safely.
2022-04-21 12:07:33 -07:00
Aditya Manthramurthy
ddf84f8257
fix: concurrency bug in site-replication (#14786)
The site replication status call was using a loop iteration variable sent
directly into go-routines instead of being passed as an argument. As the
variable is being updated in the loop, previously launched go routines do not
necessarily use the value at the time they were launched.
2022-04-20 16:20:07 -07:00
Harshavardhana
507f993075
attempt to real resolve when there is a quorum failure on reads (#14613) 2022-04-20 12:49:05 -07:00
Harshavardhana
73a6a60785
fix: replication deleteObject() regression and CopyObject() behavior (#14780)
This PR fixes two issues

- The first fix is a regression from #14555, the fix itself in #14555
  is correct but the interpretation of that information by the
  object layer code for "replication" was not correct. This PR
  tries to fix this situation by making sure the "Delete" replication
  works as expected when "VersionPurgeStatus" is already set.

  Without this fix, there is a DELETE marker created incorrectly on
  the source where the "DELETE" was triggered.

- The second fix is perhaps an older problem started since we inlined-data
  on the disk for small objects, CopyObject() incorrectly inline's
  a non-inlined data. This is due to the fact that we have code where
  we read the `part.1` under certain conditions where the size of the
  `part.1` is less than the specific "threshold".

  This eventually causes problems when we are "deleting" the data that
  is only inlined, which means dataDir is ignored leaving such
  dataDir on the disk, that looks like an inconsistent content on
  the namespace.

fixes #14767
2022-04-20 10:22:05 -07:00
Anis Elleuch
cf4cf58faf
Do not allow parallel upgrade in one server (#14782)
It is wasteful to allow parallel upgrades of MinIO server. This also generates
 weird error invoked by selfupdate module when it happens such as:

'rename /opt/bin/.minio.old /opt/bin/..minio.old.old'
2022-04-20 06:18:21 -07:00
polaris-megrez
6bc3c74c0c
honor client context in IAM user/policy listing calls (#14682) 2022-04-19 09:00:19 -07:00
Harshavardhana
598ce1e354
supply prefix filtering when necessary (#14772)
currently filterPefix was never used and set
that would filter out entries when needed
when `prefix` doesn't end with `/` - this
often leads to objects getting Walked(), Healed()
that were never requested by the caller.
2022-04-19 08:20:48 -07:00
Aditya Manthramurthy
4685b76e08
Update dperf v0.3.6 (#14773) 2022-04-19 02:40:36 -07:00
Minio Trusted
78c9109f6c update console to v0.15.14 2022-04-18 17:29:56 -07:00
Harshavardhana
7e248fc0ba
wait on parallel decom to complete before returning (#14764)
without this wait there is a potential for some objects
that are in actively being decommissioned would cancel,
however the decommission status might wrongly conclude
this as "Complete".

To avoid this make sure to add waitgroups on the parallel
workers, allowing parallel copies to complete fully before
we return.
2022-04-18 13:26:29 -07:00
Daniel Valdivia
c526fa9119
Support console UI access at a subpath on a subdomain (#14761)
fixes #14285 

Signed-off-by: Daniel Valdivia <18384552+dvaldivia@users.noreply.github.com>
2022-04-17 16:01:49 -07:00
Harshavardhana
520e0fd985 update helm to v3.6.6 2022-04-17 14:46:44 -07:00
Yi Siqi
54a7eba358
Support overriding existing secrets (#14690) 2022-04-16 07:36:50 -07:00
Minio Trusted
1494ba2e6e Update yaml files to latest version RELEASE.2022-04-16T04-26-02Z 2022-04-16 05:03:00 +00:00
Anis Elleuch
a5b3548ede
Bring back listing LDAP users temporarly (#14760)
In previous releases, mc admin user list would return the list of users
that have policies mapped in IAM database. However, this was removed but
this commit will bring it back until we revamp this.
2022-04-15 21:26:02 -07:00
Harshavardhana
8318aa0113
cancel active routine only after metadata has been saved (#14757)
currently updated pool.bin was not saved properly, that would
lead to unable to remove a pool upon a successful decommission.

fixes #14756
2022-04-15 13:16:15 -07:00
Harshavardhana
e69c42956b
fix: IAM reload should only list at config/iam/ precisely (#14753) 2022-04-15 12:12:45 -07:00
Harshavardhana
53ca589c11 update deps for minio-go/v7 and jwt/v4 2022-04-15 00:50:22 -07:00
Daniel Valdivia
ca8ff8718e
Update Console v0.15.13 (#14751)
Signed-off-by: Daniel Valdivia <18384552+dvaldivia@users.noreply.github.com>
2022-04-14 18:35:00 -07:00
Aditya Manthramurthy
e8e48e4c4a
S3 select switch to new parquet library and reduce locking (#14731)
- This change switches to a new parquet library
- SelectObjectContent now takes a single lock at the beginning and holds it
during the operation. Previously the operation took a lock every time the
parquet library performed a Seek on the underlying object stream.
- Add basic support for LogicalType annotations for timestamps.
2022-04-14 06:54:47 -07:00
Minio Trusted
67e17ed3f8 update helm v3.6.5
Signed-off-by: Minio Trusted <trusted@minio.io>
2022-04-13 15:45:54 -07:00
Harshavardhana
2a6a40e93b
enable go1.18.x builds (#14746) 2022-04-13 14:21:55 -07:00
Harshavardhana
eda34423d7 update gofumpt -w - new changes 2022-04-13 12:00:11 -07:00
Yi Siqi
7ce1f6e736
Support templating accessKey existingSecret and bucket name (#14643) 2022-04-13 11:58:29 -07:00