Commit Graph

723 Commits

Author SHA1 Message Date
Klaus Post 51410c9023
Clear omitted fields (#20575)
Searched `msg:"[a-zA-Z0-9]*,omitempty` through the codebase.

Uses latest tinylib master.
2024-10-22 08:30:50 -07:00
Harshavardhana ab7714b01e
upgrade relevant dependencies (#20507) 2024-10-01 23:37:55 -07:00
Harshavardhana 7f1e1713ab
use absolute path for binary checksum verification (#20487) 2024-09-26 08:03:08 -07:00
Klaus Post 974cbb3bb7
Limit jstream parse depth (#20474)
Add https://github.com/bcicen/jstream/pull/15 by vendoring the package.

Sets JSON depth limit to 100 entries in S3 Select.
2024-09-23 12:35:41 -07:00
Harshavardhana 03e996320e
upgrade deps pkg/v3, madmin-go/v3 and lz4/v4 (#20467) 2024-09-21 17:33:43 -07:00
Taran Pelkey 78fcb76294
Add `ListAccessKeysBulk` API for builtin user access keys (#20381) 2024-09-21 04:35:40 -07:00
Ramon de Klein 3d152015eb
Use MinIO console v1.7.1 (#20465) 2024-09-20 18:18:54 -07:00
Anis Eleuch e47d787adb
tier: Add force param to force tiering removal (#20355)
Currently, it is not possible to remove a tier if it is not accessible
or contains some data, add a force flag to make the removal successful
in that case.
2024-09-12 13:44:05 -07:00
Poorna cdd7512a2e
use rename() safety for in-place 'xl.meta' updates (#20414) 2024-09-11 09:08:51 -07:00
Frank Wessels a6d5287310
Reenable SVE support for Graviton 4 (#20410) 2024-09-10 08:35:12 -07:00
Shubhendu 0b7aa6af87
Skip non existent ldap entities while import (#20352)
Dont hard error for nonexisting LDAP entries instead of logging them
report them via `mc`

Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>
2024-09-09 09:59:28 -07:00
Harshavardhana 7a34c88d73
add consistent nonce to make multipart deterministic per part (#20359)
This change adds a consistent nonce to ensure
that multipart uploads are deterministic on a 
per-part basis.

Thanks to @klauspost for the work here minio/sio@3cd3734
2024-08-31 11:25:48 -07:00
Harshavardhana 1c4d28d7af
update pkg/v3, minio-go/v7 and mc (#20327) 2024-08-26 08:33:07 -07:00
Anis Eleuch a8f143298f
heal: Reset healing params when a retry is decided (#20285)
Currently, retry healing of a new drive healing does not reset
HealedBuckets means that the next healing retry will skip those
buckets. The commit will fix this behavior.

Also, the skipped objects counter will include objects uploaded
that are uploaded after the healing is started.
2024-08-22 05:35:43 -07:00
Mark Theunissen 6378ca10a4
kms.ListKeys returns CreatedBy/CreatedAt when information is available (#20223) 2024-08-17 23:43:03 -07:00
Harshavardhana acdb355070
update deps and update azure WARM tier implementation (#20247) 2024-08-13 11:21:34 -07:00
Andreas Auernhammer 14876a4df1
ldap: use custom TLS cipher suites (#20221)
This commit replaces the LDAP client TLS config and
adds a custom list of TLS cipher suites which support
RSA key exchange (RSA kex).

Some LDAP server connections experience a significant slowdown
when these cipher suites are not available. The Go TLS stack
disables them by default. (Can be enabled via GODEBUG=tlsrsakex=1).

fixes https://github.com/minio/minio/issues/20214

With a custom list of TLS ciphers, Go can pick the TLS RSA key-exchange
cipher. Ref:
```
	if c.CipherSuites != nil {
		return c.CipherSuites
	}
	if tlsrsakex.Value() == "1" {
		return defaultCipherSuitesWithRSAKex
	}
```
Ref: https://cs.opensource.google/go/go/+/refs/tags/go1.22.5:src/crypto/tls/common.go;l=1017

Signed-off-by: Andreas Auernhammer <github@aead.dev>
2024-08-07 05:59:47 -07:00
Mark Theunissen 2681219039
Add dummy PutBucketCors for functional test compatibility (#20220) 2024-08-06 08:41:38 -07:00
Cesar N. 6efb56851c
Update console version to 1.7.0 (#20211) 2024-08-02 21:33:23 -07:00
Klaus Post 15b609ecea
Expose RPC reconnections and ping time (#20157)
- Keeps track of reconnection count.
- Keeps track of connection ping roundtrip times. 
  Sends timestamp in ping message.
- Allow ping without payload.
2024-07-25 14:07:21 -07:00
jiuker b3a94c4e85
fix: Use xtime duration to parse batch job (#20117) 2024-07-23 00:05:53 -07:00
Harshavardhana 8e618d45fc
remove unnecessary LRU for internode auth token (#20119)
removes contentious usage of mutexes in LRU, which
were never really reused in any manner; we do not
need it.

To trust hosts, the correct way is TLS certs; this PR completely
removes this dependency, which has never been useful.

```
0  0%  100%  25.83s 26.76%  github.com/hashicorp/golang-lru/v2/expirable.(*LRU[...])
0  0%  100%  28.03s 29.04%  github.com/hashicorp/golang-lru/v2/expirable.(*LRU[...])
```

Bonus: use `x-minio-time` as a nanosecond to avoid unnecessary
parsing logic of time strings instead of using a more
straightforward mechanism.
2024-07-22 00:04:48 -07:00
Frank Wessels 95f076340a
Update reedsolomon dependency with fix for Graviton4 processor (#20102) 2024-07-16 12:27:21 -07:00
Mark Theunissen 698bb93a46
Allow a KMS Action to specify keys in the Resources of a policy (#20079) 2024-07-16 07:03:03 -07:00
Alex 459985f0fa
Update Console to v1.6.3 (#20084)
Signed-off-by: Benjamin Perez <benjamin@bexsoft.net>
2024-07-12 18:46:15 -07:00
dependabot[bot] 4ea6f94ed8
Bump github.com/nats-io/nats-streaming-server from 0.24.3 to 0.24.6 (#20082)
Bumps [github.com/nats-io/nats-streaming-server](https://github.com/nats-io/nats-streaming-server) from 0.24.3 to 0.24.6.
- [Release notes](https://github.com/nats-io/nats-streaming-server/releases)
- [Changelog](https://github.com/nats-io/nats-streaming-server/blob/main/.goreleaser.yml)
- [Commits](https://github.com/nats-io/nats-streaming-server/compare/v0.24.3...v0.24.6)

---
updated-dependencies:
- dependency-name: github.com/nats-io/nats-streaming-server
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-07-12 11:45:09 -07:00
Frank Wessels ef802f2b2c
Updated dependencies for ARM SVE support (#20081) 2024-07-12 07:36:29 -07:00
Taran Pelkey f5d2fbc84c
Add DecodeDN and QuickNormalizeDN functions to LDAP config (#20076) 2024-07-11 18:04:53 -07:00
Harshavardhana 32d04091a2
resume any batch jobs in a goroutine (#20035)
Bonus: move batch job initialization to the last item after all other initialization, 
            allowing for faster startup time for different subsystems.
2024-07-03 00:16:05 -07:00
Anis Eleuch 2ec1f404ac
info: Always refresh the root disk status (#20023)
Add root drive status in the disk info cache function, so unmounting a
drive without restarting a local node reflects the correct value.
2024-07-02 13:41:29 -07:00
Anis Eleuch b35acb3dbc
heal: Add support of healing particular pool/set (#20024) 2024-07-01 15:02:25 -07:00
Harshavardhana 499531f0b5 update minio/console v1.6.1
Signed-off-by: Harshavardhana <harsha@minio.io>
2024-06-25 18:06:18 -07:00
Taran Pelkey 3c2141513f
add `ListAccessKeysLDAPBulk` API to list accessKeys for multiple/all LDAP users (#19835) 2024-06-25 14:21:28 -07:00
Cesar N 20960b6a2d
Update console to v1.6.0 (#19933) 2024-06-13 15:53:53 -07:00
Harshavardhana 5a5046ce45
upgrade all deps and credits (#19930)
Signed-off-by: Harshavardhana <harsha@minio.io>
2024-06-13 08:34:20 -07:00
Klaus Post d2eed44c78
Fix replication checksum transfer (#19906)
Compression will be disabled by default if SSE-C is specified. So we can still honor SSE-C.
2024-06-10 10:40:33 -07:00
Cesar N cbd9efcb43
Update console to v1.5.0 (#19899) 2024-06-08 01:12:00 -07:00
Harshavardhana 2107722829
upgrade go-oidc to fix GO-2024-2631 (#19884) 2024-06-05 15:00:34 -07:00
Harshavardhana 17fe91d6d1
chore: update all deps (#19875) 2024-06-04 12:20:08 -07:00
Klaus Post a8554c4022
Update madmin (#19862)
Make it include https://github.com/minio/madmin-go/pull/285
2024-06-03 05:00:14 -07:00
Klaus Post c5b3f5553f
Add per connection RPC metrics (#19852)
Provides individual and aggregate stats for each RPC connection.

Example:

```
  "rpc": {
   "collectedAt": "2024-05-31T14:33:29.1373103+02:00",
   "connected": 30,
   "disconnected": 0,
   "outgoingStreams": 69,
   "incomingStreams": 0,
   "outgoingBytes": 174822796,
   "incomingBytes": 175821566,
   "outgoingMessages": 768595,
   "incomingMessages": 768589,
   "outQueue": 0,
   "lastPongTime": "2024-05-31T12:33:28Z",
   "byDestination": {
    "http://127.0.0.1:9001": {
     "collectedAt": "2024-05-31T14:33:29.1373103+02:00",
     "connected": 5,
     "disconnected": 0,
     "outgoingStreams": 2,
     "incomingStreams": 0,
     "outgoingBytes": 38432543,
     "incomingBytes": 66604052,
     "outgoingMessages": 229496,
     "incomingMessages": 229575,
     "outQueue": 0,
     "lastPongTime": "2024-05-31T12:33:27Z"
    },
    "http://127.0.0.1:9002": {
     "collectedAt": "2024-05-31T14:33:29.1373103+02:00",
     "connected": 5,
     "disconnected": 0,
     "outgoingStreams": 6,
     "incomingStreams": 0,
     "outgoingBytes": 38215680,
     "incomingBytes": 66121283,
     "outgoingMessages": 228525,
     "incomingMessages": 228510,
     "outQueue": 0,
     "lastPongTime": "2024-05-31T12:33:27Z"
    },
...
```
2024-05-31 22:16:24 -07:00
Harshavardhana e5c83535af
chore: upgrade deps (#19819)
Signed-off-by: Harshavardhana <harsha@minio.io>
2024-05-28 02:27:44 -07:00
Harshavardhana 597a785253
fix: authenticate LDAP via actual DN instead of normalized DN (#19805)
fix: authenticate LDAP via actual DN instead of normalized DN

Normalized DN is only for internal representation, not for
external communication, any communication to LDAP must be
based on actual user DN. LDAP servers do not understand
normalized DN.

fixes #19757
2024-05-25 06:43:06 -07:00
Aditya Manthramurthy 5f78691fcf
ldap: Add user DN attributes list config param (#19758)
This change uses the updated ldap library in minio/pkg (bumped
up to v3). A new config parameter is added for LDAP configuration to
specify extra user attributes to load from the LDAP server and to store
them as additional claims for the user.

A test is added in sts_handlers.go that shows how to access the LDAP
attributes as a claim.

This is in preparation for adding SSH pubkey authentication to MinIO's SFTP
integration.
2024-05-24 16:05:23 -07:00
Shireesh Anjal 2a03a34bde
Upgrade madmin-go to v3.0.52 (#19798)
This will ensure that content of /proc/cmdline from each server is
captured in the health report.
2024-05-24 05:34:57 -07:00
Anis Eleuch 69c9496c71
Upgrade github.com/minio/pkg/v2 and other deps (#19747) 2024-05-15 11:04:40 -07:00
Klaus Post c36eaedb93
Re-add "Fix incorrect merging of slash-suffixed objects (#19729)
Adds regression test for #19699

Failures are a bit luck based, since it requires objects to be placed on different sets.

However this generates a failure prior to #19699

* Revert "Revert "Fix incorrect merging of slash-suffixed objects (#19699)""

This reverts commit f30417d9a8.

* Don't override when suffix doesn't match. Instead rely on quorum for each.
2024-05-13 09:30:24 -07:00
Andreas Auernhammer 8b660e18f2
kms: add support for MinKMS and remove some unused/broken code (#19368)
This commit adds support for MinKMS. Now, there are three KMS
implementations in `internal/kms`: Builtin, MinIO KES and MinIO KMS.

Adding another KMS integration required some cleanup. In particular:
 - Various KMS APIs that haven't been and are not used have been
   removed. A lot of the code was broken anyway.
 - Metrics are now monitored by the `kms.KMS` itself. For basic
   metrics this is simpler than collecting metrics for external
   servers. In particular, each KES server returns its own metrics
   and no cluster-level view.
 - The builtin KMS now uses the same en/decryption implemented by
   MinKMS and KES. It still supports decryption of the previous
   ciphertext format. It's backwards compatible.
 - Data encryption keys now include a master key version since MinKMS
   supports multiple versions (~4 billion in total and 10000 concurrent)
   per key name.

Signed-off-by: Andreas Auernhammer <github@aead.dev>
2024-05-07 16:55:37 -07:00
Alex 1e83f15e2f
Update Console version to v1.4.0 (#19684) 2024-05-06 19:47:37 -07:00
Harshavardhana 08ff702434
enhance ListSVCs() API to return more info to avoid InfoSvc() (#19642)
ConsoleUI like applications rely on combination of

ListServiceAccounts() and InfoServiceAccount() to populate
UI elements, however individually these calls can be slow
causing the entire UI to load sluggishly.
2024-05-01 05:41:13 -07:00