Commit Graph

686 Commits

Author SHA1 Message Date
Harshavardhana
08ff702434
enhance ListSVCs() API to return more info to avoid InfoSvc() (#19642)
ConsoleUI like applications rely on combination of

ListServiceAccounts() and InfoServiceAccount() to populate
UI elements, however individually these calls can be slow
causing the entire UI to load sluggishly.
2024-05-01 05:41:13 -07:00
Harshavardhana
f4f1c42cba
deprecate usage of sha256-simd (#19621)
go1.21 already implements the necessary optimizations
2024-04-25 23:31:35 -07:00
Cesar N
943d815783
Update Console UI to v1.3.0 (#19617) 2024-04-25 12:12:03 -07:00
Harshavardhana
d95e054282
update all deps regular cadence (#19523) 2024-04-16 11:48:56 -07:00
Allan Roger Reid
7c1f9667d1
Use GetDuration() helper for MINIO_KMS_KEY_CACHE_INTERVAL as time.Duration (#19512)
Bonus: Use default duration of 10 seconds if invalid input < time.Second is specified
2024-04-16 08:43:39 -07:00
Alex
8d7d4adb91
Updated Console UI to v1.2.0 (#19467) 2024-04-11 01:31:16 -07:00
Harshavardhana
1bb670ecba
use new generics based LRU from hashicorp (#19409)
we have been using an LRU caching for internode
auth tokens, migrate to using a typed implementation
and also do not cache auth tokens when its an error.
2024-04-04 11:58:48 -07:00
Aditya Manthramurthy
c9e9a8e2b9
fix: ldap: use validated base DNs (#19406)
This fixes a regression from #19358 which prevents policy mappings
created in the latest release from being displayed in policy entity
listing APIs.

This is due to the possibility that the base DNs in the LDAP config are
not in a normalized form and #19358 introduced normalized of mapping
keys (user DNs and group DNs). When listing, we check if the policy
mappings are on entities that parse as valid DNs that are descendants of
the base DNs in the config.

Test added that demonstrates a failure without this fix.
2024-04-04 11:36:18 -07:00
Harshavardhana
e18c0ab9bf update vulncheck to go1.21.9
Signed-off-by: Harshavardhana <harsha@minio.io>
2024-04-03 19:00:59 -07:00
Harshavardhana
d7daae4762
update to latest deps (#19399)
Signed-off-by: Harshavardhana <harsha@minio.io>
2024-04-03 09:28:32 -07:00
Aditya Manthramurthy
7e45d84ace
ldap: improve normalization of DN values (#19358)
Instead of relying on user input values, we use the DN value returned by
the LDAP server.

This handles cases like when a mapping is set on a DN value
`uid=svc.algorithm,OU=swengg,DC=min,DC=io` with a user input value (with
unicode variation) of `uid=svc﹒algorithm,OU=swengg,DC=min,DC=io`. The
LDAP server on lookup of this DN returns the normalized value where the
unicode dot character `SMALL FULL STOP` (in the user input), gets
replaced with regular full stop.
2024-03-27 23:45:26 -07:00
Klaus Post
7ff4164d65
Fix races in IAM cache lazy loading (#19346)
Fix races in IAM cache

Fixes #19344

On the top level we only grab a read lock, but we write to the cache if we manage to fetch it.

a03dac41eb/cmd/iam-store.go (L446) is also flipped to what it should be AFAICT.

Change the internal cache structure to a concurrency safe implementation.

Bonus: Also switch grid implementation.
2024-03-26 11:12:57 -07:00
Harshavardhana
485298b680
update all dependencies (#19235) 2024-03-14 17:41:26 -07:00
dependabot[bot]
1787bcfc91
build(deps): bump github.com/lestrrat-go/jwx from 1.2.28 to 1.2.29 (#19226)
Bumps [github.com/lestrrat-go/jwx](https://github.com/lestrrat-go/jwx) from 1.2.28 to 1.2.29.
- [Release notes](https://github.com/lestrrat-go/jwx/releases)
- [Changelog](https://github.com/lestrrat-go/jwx/blob/v1.2.29/Changes)
- [Commits](https://github.com/lestrrat-go/jwx/compare/v1.2.28...v1.2.29)

---
updated-dependencies:
- dependency-name: github.com/lestrrat-go/jwx
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-08 08:42:47 -08:00
Harshavardhana
48b590e14b
fix: same server to be part of multiple pools (#19216)
our PoolNumber calculation was costly,
while we already had this information per
endpoint, we needed to deduce it appropriately.

This PR addresses this by assigning PoolNumbers
field that carries all the pool numbers that
belong to a server.

properties.PoolNumber still carries a valid value
only when len(properties.PoolNumbers) == 1, otherwise
properties.PoolNumber is set to math.MaxInt (indicating
that this value is undefined) and then one must rely
on properties.PoolNumbers for server participation
in multiple pools.

addresses the issue originating from #11327
2024-03-07 10:24:07 -08:00
Harshavardhana
5dc1ef0b87
fix: go mod update v1.33.0 https://pkg.go.dev/vuln/GO-2024-2611 (#19208) 2024-03-05 14:38:15 -08:00
Harshavardhana
6d08af61a0
for root disks add additional information in the error log (#19177) 2024-03-02 23:45:39 -08:00
Andreas Auernhammer
09626d78ff
automatically generate root credentials with KMS (#19025)
With this commit, MinIO generates root credentials automatically
and deterministically if:

 - No root credentials have been set.
 - A KMS (KES) is configured.
 - API access for the root credentials is disabled (lockdown mode).

Before, MinIO defaults to `minioadmin` for both the access and
secret keys. Now, MinIO generates unique root credentials
automatically on startup using the KMS.

Therefore, it uses the KMS HMAC function to generate pseudo-random
values. These values never change as long as the KMS key remains
the same, and the KMS key must continue to exist since all IAM data
is encrypted with it.

Backward compatibility:

This commit should not cause existing deployments to break. It only
changes the root credentials of deployments that have a KMS configured
(KES, not a static key) but have not set any admin credentials. Such
implementations should be rare or not exist at all.

Even if the worst case would be updating root credentials in mc
or other clients used to administer the cluster. Root credentials
are anyway not intended for regular S3 operations.

Signed-off-by: Andreas Auernhammer <github@aead.dev>
2024-03-01 13:09:42 -08:00
Alex
ef06644799
Updated Console to v1.0.0 (#19164)
Signed-off-by: Benjamin Perez <benjamin@bexsoft.net>
2024-02-29 17:54:07 -08:00
Harshavardhana
d7520f0ae6
fix: make sure maintenance=true is honored properly (#19156)
fixes a regression from #18700
2024-02-29 08:37:57 -08:00
Anis Eleuch
2bdb9511bd
heal: Add skipped objects to the heal summary (#19142)
New disk healing code skips/expires objects that ILM supposed to expire.
Add more visibility to the user about this activity by calculating those
objects and print it at the end of healing activity.
2024-02-28 09:05:40 -08:00
Harshavardhana
b6e98aed01
fix: found races in accessing globalLocalDrives (#19069)
make a copy before accessing globalLocalDrives

Bonus: update console v0.46.0

Signed-off-by: Harshavardhana <harsha@minio.io>
2024-02-16 17:15:57 -08:00
Harshavardhana
6fd0b434e2
upgrade all deps (#19041) 2024-02-14 09:51:34 -08:00
Anis Eleuch
4fa06aefc6
Convert service account add/update expiration to cond values (#19024)
In order to force some users allowed to create or update a service
account to provide an expiration satifying the user policy conditions.
2024-02-12 08:36:16 -08:00
Harshavardhana
7e023f2d50 remove go.mod replace tag 2024-02-06 01:56:57 -08:00
Poorna
27d02ea6f7
metrics: add replication metrics on proxied requests (#18957) 2024-02-05 22:00:45 -08:00
Anis Eleuch
6fd63e920a
log: Use error log type instead of Application/MinIO type (#18930)
* log: Use error log type instead of Application/MinIO type

Also bump github.com/shirou/gopsutil version to address cross
compilation issues.

* Apply suggestions from code review

Co-authored-by: Aditya Manthramurthy <donatello@users.noreply.github.com>

---------

Co-authored-by: Anis Eleuch <anis@min.io>
Co-authored-by: Harshavardhana <harsha@minio.io>
Co-authored-by: Aditya Manthramurthy <donatello@users.noreply.github.com>
2024-02-01 16:13:57 -08:00
Harshavardhana
88837fb753
add new update v2 that updates per node, allows idempotent behavior (#18859)
add new update v2 that updates per node, allows idempotent behavior

new API ensures that

- binary is correct and can be downloaded checksummed verified
- committed to actual path
- restart returns back the relevant waiting drives
2024-01-26 08:40:13 -08:00
Harshavardhana
e11d851aee
add new drive I/O waiting/tokens metric (#18836)
Bonus: add virtual memory used as well part of the system resource metrics.
2024-01-19 14:51:36 -08:00
Harshavardhana
099e88516d
update all dependencies and console v0.45.0 (#18773) 2024-01-10 23:46:16 -08:00
Krishnan Parthasarathi
3a90af0bcd
Add line, col to types used in batch-expire (#18747) 2024-01-08 15:22:28 -08:00
Harshavardhana
a50ea92c64
feat: introduce list_quorum="auto" to prefer quorum drives (#18084)
NOTE: This feature is not retro-active; it will not cater to previous transactions
on existing setups. 

To enable this feature, please set ` _MINIO_DRIVE_QUORUM=on` environment
variable as part of systemd service or k8s configmap. 

Once this has been enabled, you need to also set `list_quorum`. 

```
~ mc admin config set alias/ api list_quorum=auto` 
```

A new debugging tool is available to check for any missing counters.
2023-12-29 15:52:41 -08:00
Daniel Valdivia
5fc7da345d
Upgrade Console to v0.44.0 (#18717)
Signed-off-by: Daniel Valdivia <18384552+dvaldivia@users.noreply.github.com>
2023-12-27 11:19:13 -08:00
Harshavardhana
fd2c38fbef
upgrade Go mod dependencies (#18710) 2023-12-25 01:02:59 -08:00
Anis Eleuch
22f8e39b58
tier: Allow edit of the new Azure and AWS auth params (#18690)
Allow editing for the service principal credentials from Azure
and the web identity token for AWS;

Also, more validation of input parameters.
2023-12-21 16:58:10 -08:00
Cesar N
7a311a3b66
Update Console version to v0.43.1 (#18683) 2023-12-19 17:00:02 -08:00
Harshavardhana
f4389fb322
update dependencies for https://pkg.go.dev/vuln/GO-2023-2402 (#18676) 2023-12-18 18:35:20 -08:00
Harshavardhana
bdf4e386cf
update go dependencies (#18662) 2023-12-15 21:35:07 -08:00
Anis Eleuch
8771617199
tier: Add support of AWS S3 tiering with web identity token file (#18648) 2023-12-14 14:01:49 -08:00
Harshavardhana
8fa2898ff1
update console UI to v0.43.0 release (#18653) 2023-12-14 10:50:54 -08:00
Anis Eleuch
4a21dce2b5
tier: Add support of SP credentials with Azure (#18630)
Co-authored-by: Anis Elleuch <anis@min.io>
2023-12-11 21:51:53 -08:00
Anis Eleuch
2e23e61a45
Add support of conf file to pass arguments and options (#18592) 2023-12-07 01:33:56 -08:00
Alex
cfed671ea3
Update Console version to v0.42.1 (#18606)
Signed-off-by: Benjamin Perez <benjamin@bexsoft.net>
2023-12-06 20:16:00 -08:00
Shireesh Anjal
7350a29fec
Capture percentage of cpu load and memory used (#18596)
By default the cpu load is the cumulative of all cores. Capture the
percentage load (load * 100 / cpu-count)

Also capture the percentage memory used (used * 100 / total)
2023-12-06 13:19:59 -08:00
Harshavardhana
73dde66dbe
stick to go1.19 go.mod (#18600) 2023-12-06 01:09:22 -08:00
Krishnan Parthasarathi
a50f26b7f5
Implement batch-expiration for objects (#17946)
Based on an initial PR from -
https://github.com/minio/minio/pull/17792

But fully completes it with newer finalized YAML spec.
2023-12-02 02:51:33 -08:00
Klaus Post
94fbcd8ebe
Add TLS cert checksum (#18557)
It allows validation of whether all certs match across clusters.
2023-11-30 12:13:50 -08:00
Harshavardhana
e9babf3dac
(chore): update all our deps (#18525) 2023-11-28 14:44:44 -08:00
Harshavardhana
8d9e83fd99
support passing signatureAge conditional (#18529)
this PR allows following policy

```
{
   "Version": "2012-10-17",
   "Statement": [
      {
         "Sid": "Deny a presigned URL request if the signature is more than 10 min old",
         "Effect": "Deny",
         "Action": "s3:*",
         "Resource": "arn:aws:s3:::DOC-EXAMPLE-BUCKET1/*",
         "Condition": {
            "NumericGreaterThan": {
               "s3:signatureAge": 600000
            }
         }
      }
   ]
}
```

This is to basically disable all pre-signed URLs that are older than 10 minutes.
2023-11-27 11:30:19 -08:00
jiuker
be02333529
feat: drive sub-sys to max timeout reload (#18501) 2023-11-27 09:15:06 -08:00