Commit Graph

11534 Commits

Author SHA1 Message Date
Anis Eleuch 414bcb0c73
prom: Add read quorum per erasure set metric (#18736) 2024-01-04 15:05:13 -08:00
Harshavardhana f4710948c4
fix: an odd crash when deleting `null` DEL markers (#18727)
fixes #18724

A regression was introduced in #18547, that attempted
to file adding a missing `null` marker however we
should not skip returning based on versionID instead
it must be based on if we are being asked to create
a DEL marker or not.

The PR also has a side-affect for replicating `null`
marker permanent delete, as it may end up adding a
`null` marker while removing one.

This PR should address both scenarios.
2024-01-02 15:08:18 -08:00
Anis Eleuch 3f4488c589
scanner: Allow full throttle if there is no parallel disk ops (#18109) 2024-01-02 13:51:24 -08:00
Shubhendu 9434fff215
Added list of scanner metrics to document (#18731)
Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>
2024-01-02 10:41:33 -08:00
Minio Trusted 695962fae8 Update yaml files to latest version RELEASE.2024-01-01T16-36-33Z 2024-01-02 01:46:48 +00:00
Pedro Juarez 8f13c8c3bf
Support to store browser config settings (#18631)
* csp_policy
* hsts_seconds
* hsts_include_subdomains
* hsts_preload
* referrer_policy
2024-01-01 08:36:33 -08:00
Tsuzuki Tsuchiya c1cae51fb5
doc: fix the name of RoleArn param in AssumeRoleWithWebIdentity requests (#18711) 2023-12-30 13:25:15 -08:00
Zhou Ting 31d16f6cc2
allow sha256 payload to be configurable for object perf test (#18712)
Signed-off-by: Zhou Ting <ting.z.zhou@intel.com>
2023-12-29 23:56:50 -08:00
Harshavardhana a50ea92c64
feat: introduce list_quorum="auto" to prefer quorum drives (#18084)
NOTE: This feature is not retro-active; it will not cater to previous transactions
on existing setups. 

To enable this feature, please set ` _MINIO_DRIVE_QUORUM=on` environment
variable as part of systemd service or k8s configmap. 

Once this has been enabled, you need to also set `list_quorum`. 

```
~ mc admin config set alias/ api list_quorum=auto` 
```

A new debugging tool is available to check for any missing counters.
2023-12-29 15:52:41 -08:00
Harshavardhana 5b2ced0119
re-use globalLocalDrives properly (#18721) 2023-12-29 09:30:10 -08:00
Anis Eleuch 8a0ba093dd
audit: Fix merrs and derrs object dangling message (#18714)
merrs and derrs are empty when a dangling object is deleted. Fix the bug
and adds invalid-meta data for data blocks
2023-12-27 22:27:04 -08:00
Mario Bros fbd8dfe60f
Adding ~ to match job when multiple jobs (#18706) 2023-12-27 15:39:20 -08:00
Harshavardhana 60aff22931
clarify maximum number of buckets (#18718) 2023-12-27 13:52:19 -08:00
Daniel Valdivia 5fc7da345d
Upgrade Console to v0.44.0 (#18717)
Signed-off-by: Daniel Valdivia <18384552+dvaldivia@users.noreply.github.com>
2023-12-27 11:19:13 -08:00
Harshavardhana fd2c38fbef
upgrade Go mod dependencies (#18710) 2023-12-25 01:02:59 -08:00
Minio Trusted ba245c6c46 Update yaml files to latest version RELEASE.2023-12-23T07-19-11Z 2023-12-23 10:30:11 +00:00
Aditya Manthramurthy 496027b589
Fix precendence bug in S3Select SQL IN clauses (#18708)
Fixes a precendence issue in SQL Select where `a in b and c = 3` was parsed as `a
in (b and c = 3)`.

Fixes #18682
2023-12-22 23:19:11 -08:00
Anis Eleuch 8bd4f6568b
server-info: Avoid initializing audit/log http/kafka targets (#18703)
This can cause unnecessary ServerInfo() call delay.
2023-12-22 10:25:08 -08:00
Shubhendu 9d7660b409
Graph cluster wide where applicable (#18705)
Graph the maximum value reported across nodes at cluster
level for applicable scenarios.

Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>
2023-12-22 08:14:32 -08:00
Harshavardhana da55499db0
fix: reject clients that do not send proper payload (#18701) 2023-12-22 01:26:17 -08:00
Anis Eleuch 22f8e39b58
tier: Allow edit of the new Azure and AWS auth params (#18690)
Allow editing for the service principal credentials from Azure
and the web identity token for AWS;

Also, more validation of input parameters.
2023-12-21 16:58:10 -08:00
Harshavardhana eba23bbac4
rename object_size -> block_size for cache subsystem (#18694) 2023-12-21 16:57:13 -08:00
Harshavardhana 4550535cbb
send proper IPv6 names avoid bracketing notation (#18699)
Following policies if present

```
       "Condition": {
         "IpAddress": {
            "aws:SourceIp": [
              "54.240.143.0/24",
               "2001:DB8:1234:5678::/64"
             ]
          }
        }
```

And client is making a request to MinIO via IPv6 can
potentially crash the server.

Workarounds are turn-off IPv6 and use only IPv4
2023-12-21 16:56:55 -08:00
Anis Eleuch 8432fd5ac2
prom: Add online and healing drives metrics per erasure set (#18700) 2023-12-21 16:56:43 -08:00
Harshavardhana 7c948adf88
allow pre-allocating buffers to reduce frequent GCs during growth (#18686)
This PR also increases per node bpool memory from 1024 entries
to 2048 entries; along with that, it also moves the byte pool
centrally instead of being per pool.
2023-12-21 08:59:38 -08:00
Krishnan Parthasarathi 56b7045c20
Export tier metrics (#18678)
minio_node_tier_ttlb_seconds - Distribution of time to last byte for streaming objects from warm tier
minio_node_tier_requests_success - Number of requests to download object from warm tier that were successful
minio_node_tier_requests_failure - Number of requests to download object from warm tier that failed
2023-12-20 20:13:40 -08:00
Minio Trusted b1a109a611 Update yaml files to latest version RELEASE.2023-12-20T01-00-02Z 2023-12-20 08:43:46 +00:00
Cesar N 7a311a3b66
Update Console version to v0.43.1 (#18683) 2023-12-19 17:00:02 -08:00
Poorna d55b6b9909
Fix quota config replication for SR (#18684)
Fixing regression introduced by PR #17988
2023-12-19 13:22:47 -08:00
Harshavardhana f4389fb322
update dependencies for https://pkg.go.dev/vuln/GO-2023-2402 (#18676) 2023-12-18 18:35:20 -08:00
Daniel Jakots 331208bec1
Enable cross compile for openbsd/amd64 (#18666) 2023-12-18 17:08:22 -08:00
Shireesh Anjal 7680e5f81d
Read new key license_v2 from SUBNET response (#18669)
SUBNET now has a v2 of license that is returned in the new key
`license_v2`. mc will start reading and storing the same. (The old key
`license` is deprecated but is still available in SUBNET response to
ensure that the current released version of minio doesn't break)
2023-12-18 08:21:44 -08:00
Anugrah Vijay 6acf038a84
docs: fix bucket metrics API\ path in docs (#18661) 2023-12-18 08:21:08 -08:00
Harshavardhana bdf4e386cf
update go dependencies (#18662) 2023-12-15 21:35:07 -08:00
Taran Pelkey ad8a34858f
Add APIs to create and list access keys for LDAP (#18402) 2023-12-15 13:00:43 -08:00
Krishnan Parthasarathi 162eced7d2
Fix incorrect metric desc for bucketRequestsDuration (#18657) 2023-12-14 19:02:11 -08:00
Krishnan Parthasarathi bec1f7c26a
metrics: Refactor handling of histogram vectors (#18632) 2023-12-14 14:02:52 -08:00
Anis Eleuch 8771617199
tier: Add support of AWS S3 tiering with web identity token file (#18648) 2023-12-14 14:01:49 -08:00
Minio Trusted 54bc995f0a Update yaml files to latest version RELEASE.2023-12-14T18-51-57Z 2023-12-14 19:54:40 +00:00
Klaus Post 6c89a81af4
Fix CreateFile shared buffer corruption. (#18652)
`(*xlStorageDiskIDCheck).CreateFile` wraps the incoming reader in `xioutil.NewDeadlineReader`.

The wrapped reader is handed to `(*xlStorage).CreateFile`. This performs a Read call via `writeAllDirect`, 
which reads into an `ODirectPool` buffer.

`(*DeadlineReader).Read` spawns an async read into the buffer. If a timeout is hit while reading, 
the read operation returns to `writeAllDirect`. The operation returns an error and the buffer is reused.

However, if the async `Read` call unblocks, it will write to the now recycled buffer.

Fix: Remove the `DeadlineReader` - it is inherently unsafe. Instead, rely on the network timeouts. 
This is not a disk timeout, anyway.

Regression in https://github.com/minio/minio/pull/17745
2023-12-14 10:51:57 -08:00
Harshavardhana 8fa2898ff1
update console UI to v0.43.0 release (#18653) 2023-12-14 10:50:54 -08:00
Praveen raj Mani 10ca0a6936
Label the notification target metrics by their target IDs (#18633)
This patch adds the targetID to the existing notification target metrics
and deprecates the current target metrics which points to the overall
event notification subsystem
2023-12-14 09:09:26 -08:00
Harshavardhana b3314e97a6
re-use the same local drive used by remote-peer (#18645)
historically, we have always kept storage-rest-server
and a local storage API separate without much trouble,
since they both can independently operate due to no
special state() between them.

however, over some time, we have added state()
such as

- drive monitoring threads now there will be "2" of
  them per drive instead of just 1.

- concurrent tokens available per drive are now twice
  instead of just single shared, allowing unexpectedly
  high amount of I/O to go through.

- applying serialization by using walkMutexes can now
  be adequately honored for both remote callers and local
  callers.
2023-12-13 19:27:55 -08:00
Minio Trusted 3b9a948045 Update yaml files to latest version RELEASE.2023-12-13T23-28-55Z 2023-12-14 03:04:03 +00:00
Poorna 3781a0f9ad
replication: Pass metadata timestamps in CopyObject call (#18647)
Regression from #18285. CopyObject options were inheriting source MTime
for metadata timestamps if unspecified, removing this prevented metadata
updates from being applied on target.
2023-12-13 15:28:55 -08:00
Poorna e79b289325
fix datadir missing check on HeadObject (#18646)
versions pending purge in replication were seeing a errFileCorrupt
that prevents permanent deletion after replication.

Regression from PR#18477
2023-12-13 14:54:01 -08:00
Shubhendu 6d4c1156d6
Changed the expression to render the value (#18627)
The metrics `minio_bucket_replication_received_bytes` and
`minio_bucket_replication_sent_bytes` are additive in nature
and rendering the value as is looks fine.

Also added sort order for few graphs for better reading of tool
tips as keeping ones with highest value at top helps.

Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>
2023-12-13 10:05:47 -08:00
Harshavardhana 3f72c7fcc7
healthcheck requests with user-agent mozilla do not need redirects (#18642)
apparently, windows powershell curl has this abhorrent behavior
2023-12-12 16:16:26 -08:00
Harshavardhana d521c84d55
reduce logging during permission denied errors (#18641)
log them if any only once
2023-12-12 16:11:17 -08:00
opencmit2 946b070744
resolve the absence of the sed command in ubi-micro image (#18459)
fix #18315
2023-12-12 15:43:32 -08:00