Commit Graph

1562 Commits

Author SHA1 Message Date
jiuker 9496c17e13
doc: add Content-Type to s3zip (#19455) 2024-04-10 09:28:27 -07:00
Minio Trusted 51fc145161 Update yaml files to latest version RELEASE.2024-04-06T05-26-02Z 2024-04-06 06:44:30 +00:00
Shubhendu d96d696841
Dont use deprecated angular (#19396)
Support for Angular would be stopped with newer versions of grafana

Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>
2024-04-03 19:01:53 -07:00
Minio Trusted 06929258bc Update yaml files to latest version RELEASE.2024-03-30T09-41-56Z 2024-03-30 18:32:38 +00:00
Shubhendu 468a9fae83
Enable replication of SSE-C objects (#19107)
If site replication enabled across sites, replicate the SSE-C
objects as well. These objects could be read from target sites
using the same client encryption keys.

Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>
2024-03-28 10:44:56 -07:00
Shubhendu d87f91720b
Split the replication dashboard in cluster and node level (#19374)
Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>
2024-03-28 10:15:39 -07:00
Shubhendu d63e603040
Pre populate the server names using a query (#19367)
User doesn't need to remember and enter the server values,
rather they can select from the pre populated list.

Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>
2024-03-28 08:14:26 -07:00
Shubhendu 3d4fc28ec9
Render node graphs by node (#19356)
As total drives count, online vs offline are per node basis, its
corect to select node for which graphs need to be rendered.

Set prometheus scrape jobs to fetch metrics from all nodes. A sample
scrape job for node metrics could be as below

```
- job_name: minio-job-node
  bearer_token: <token>
  metrics_path: /minio/v2/metrics/node
  scheme: https
  tls_config:
    insecure_skip_verify: true
  static_configs:
  - targets: [tenant1-ss-0-0.tenant1-hl.tenant-ns.svc.cluster.local:9000,tenant1-ss-0-1.tenant1-hl.tenant-ns.svc.cluster.local:9000,tenant1-ss-0-2.tenant1-hl.tenant-ns.svc.cluster.local:9000,tenant1-ss-0-3.tenant1-hl.tenant-ns.svc.cluster.local:9000]
```

Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>
2024-03-27 10:41:08 -07:00
Minio Trusted 428155add9 Update yaml files to latest version RELEASE.2024-03-26T22-10-45Z 2024-03-26 23:12:03 +00:00
Shubhendu 53a14c7301
Adding dashboard for MinIO node metrics (#19329)
Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>
2024-03-26 08:01:28 -07:00
Minio Trusted 15b930be1f Update yaml files to latest version RELEASE.2024-03-21T23-13-43Z 2024-03-22 20:08:28 +00:00
Minio Trusted d2373d5d6c Update yaml files to latest version RELEASE.2024-03-15T01-07-19Z 2024-03-15 02:47:20 +00:00
Harshavardhana 062f0cffad
fix: do not look for non-existent bucket in decom tests (#19261) 2024-03-14 08:54:11 -07:00
Klaus Post a2f6252b2f
xl-meta: Add inline data bitrot check (#19240)
When using `-data` also perform a bitrot check on the data.

Example:

```
λ xl-meta -data net.zip
{
        "minio-1.com:9000/data/minio1/p40/44b6/44b612e9a7294856bd2b5fe6f6cdeb0d.pdf/xl.meta": {"null":{"bitrot_valid":true,"bytes":11710}},
        "minio-1.com:9000/data/minio2/p40/44b6/44b612e9a7294856bd2b5fe6f6cdeb0d.pdf/xl.meta": {"null":{"bitrot_valid":true,"bytes":11710}},
        "minio-1.com:9000/data/minio3/p40/44b6/44b612e9a7294856bd2b5fe6f6cdeb0d.pdf/xl.meta": {"null":{"bitrot_valid":true,"bytes":11710}},
        "minio-1.com:9000/data/minio4/p40/44b6/44b612e9a7294856bd2b5fe6f6cdeb0d.pdf/xl.meta": {"null":{"bitrot_valid":true,"bytes":11710}},
        "minio-2.com:9000/data/minio1/p40/44b6/44b612e9a7294856bd2b5fe6f6cdeb0d.pdf/xl.meta": {"null":{"bitrot_valid":true,"bytes":11710}},
        "minio-2.com:9000/data/minio2/p40/44b6/44b612e9a7294856bd2b5fe6f6cdeb0d.pdf/xl.meta": {"null":{"bitrot_valid":true,"bytes":11710}},
        "minio-2.com:9000/data/minio3/p40/44b6/44b612e9a7294856bd2b5fe6f6cdeb0d.pdf/xl.meta": {"null":{"bitrot_valid":true,"bytes":11710}},
        "minio-2.com:9000/data/minio4/p40/44b6/44b612e9a7294856bd2b5fe6f6cdeb0d.pdf/xl.meta": {"null":{"bitrot_valid":true,"bytes":11710}},
        "minio-3.com:9000/data/minio1/p40/44b6/44b612e9a7294856bd2b5fe6f6cdeb0d.pdf/xl.meta": {"null":{"bitrot_valid":true,"bytes":11710}},
        "minio-3.com:9000/data/minio2/p40/44b6/44b612e9a7294856bd2b5fe6f6cdeb0d.pdf/xl.meta": {"null":{"bitrot_valid":true,"bytes":11710}},
        "minio-3.com:9000/data/minio3/p40/44b6/44b612e9a7294856bd2b5fe6f6cdeb0d.pdf/xl.meta": {"null":{"bitrot_valid":true,"bytes":11710}},
        "minio-3.com:9000/data/minio4/p40/44b6/44b612e9a7294856bd2b5fe6f6cdeb0d.pdf/xl.meta": {"null":{"bitrot_valid":true,"bytes":11710}},
        "minio-4.com:9000/data/minio1/p40/44b6/44b612e9a7294856bd2b5fe6f6cdeb0d.pdf/xl.meta": {"null":{"bitrot_valid":true,"bytes":11710}},
        "minio-4.com:9000/data/minio2/p40/44b6/44b612e9a7294856bd2b5fe6f6cdeb0d.pdf/xl.meta": {"null":{"bitrot_valid":true,"bytes":11710}},
        "minio-4.com:9000/data/minio3/p40/44b6/44b612e9a7294856bd2b5fe6f6cdeb0d.pdf/xl.meta": {"null":{"bitrot_valid":true,"bytes":11710}},
        "minio-4.com:9000/data/minio4/p40/44b6/44b612e9a7294856bd2b5fe6f6cdeb0d.pdf/xl.meta": {"null":{"bitrot_valid":true,"bytes":11710}}
}
```
2024-03-11 10:57:11 -07:00
Dennis Marttinen 6c964fede5
Improve handling of compression inclusion for objects (#19234) 2024-03-11 04:55:34 -07:00
Aditya Manthramurthy b2c5b75efa
feat: Add Metrics V3 API (#19068)
Metrics v3 is mainly a reorganization of metrics into smaller groups of
metrics and the removal of internal aggregation of metrics received from
peer nodes in a MinIO cluster.

This change adds the endpoint `/minio/metrics/v3` as the top-level metrics
endpoint and under this, various sub-endpoints are implemented. These
are currently documented in `docs/metrics/v3.md`

The handler will serve metrics at any path
`/minio/metrics/v3/PATH`, as follows:

when PATH is a sub-endpoint listed above => serves the group of
metrics under that path; or when PATH is a (non-empty) parent 
directory of the sub-endpoints listed above => serves metrics
from each child sub-endpoint of PATH. otherwise, returns a no 
resource found error

All available metrics are listed in the `docs/metrics/v3.md`. More will
be added subsequently.
2024-03-10 01:15:15 -08:00
Minio Trusted 2dfa9adc5d Update yaml files to latest version RELEASE.2024-03-10T02-53-48Z 2024-03-10 08:42:35 +00:00
Minio Trusted 8ab3dac4f2 Update yaml files to latest version RELEASE.2024-03-07T00-43-48Z 2024-03-07 06:25:40 +00:00
Poorna 837a2a3d4b
sr: use service account cred for claims check (#19209)
PR #19111 overlaid service account secret with site replicator secret
during token claims check.

Fixes : #19206
2024-03-06 16:19:24 -08:00
Harshavardhana 74ccee6619
avoid too much auditing during decom/rebalance make it more robust (#19174)
there can be a sudden spike in tiny allocations,
due to too much auditing being done, also don't hang
on the

```
h.logCh <- entry
```

after initializing workers if you do not have a way to
dequeue for some reason.
2024-03-06 03:43:16 -08:00
Minio Trusted e3e3d92241 Update yaml files to latest version RELEASE.2024-03-05T04-48-44Z 2024-03-05 06:20:43 +00:00
Aditya Manthramurthy 9a4d003ac7
Add common middleware to S3 API handlers (#19171)
The middleware sets up tracing, throttling, gzipped responses and
collecting API stats.

Additionally, this change updates the names of handler functions in
metric labels to be the same as the name derived from Go lang reflection
on the handler name.

The metric api labels are now stored in memory the same as the handler
name - they will be camelcased, e.g. `GetObject` instead of `getobject`.

For compatibility, we lowercase the metric api label values when emitting the metrics.
2024-03-04 10:05:56 -08:00
Minio Trusted 18b0b7299a Update yaml files to latest version RELEASE.2024-03-03T17-50-39Z 2024-03-03 18:33:43 +00:00
Ravind Kumar f3e7c42425
Update metrics list.md with new metrics from RELEASE.2024-01-05 (#19161) 2024-02-29 14:53:54 -08:00
Shubhendu f46bee242c
Re-organized grafana dashboards (#19157)
Moved different dashboards to their specific directories. Also
mentioned that these dashbards are examples of how to create
graphs using MinIO provided and metrics and customers should
change / add graphs on their specific need basis.

Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>
2024-02-29 10:35:20 -08:00
Anis Eleuch 828d4df6f0
debug: Add --search to print only specific goroutines (#19158)
Easier to filter goroutines belonging to a specific subsystem
2024-02-29 08:28:18 -08:00
Poorna b1351e2dee sr: use site replicator svcacct to sign STS session tokens (#19111)
This change is to decouple need for root credentials to match between
 site replication deployments.

 Also ensuring site replication config initialization is re-tried until
 it succeeds, this deoendency is critical to STS flow in site replication
 scenario.
2024-02-26 13:30:28 -08:00
Minio Trusted 85bcb5874a Update yaml files to latest version RELEASE.2024-02-26T09-33-48Z 2024-02-26 10:28:02 +00:00
Minio Trusted b49ce1713f Update yaml files to latest version RELEASE.2024-02-24T17-11-14Z 2024-02-25 05:23:21 +00:00
schmittey c44f311c4f
Add missing yaml syntax highlighting in prometheus README.md (#19087) 2024-02-20 16:22:37 -08:00
Shubhendu cb7dab17cb
Graph cluster and bucket replication proxied requests (#19078)
Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>
2024-02-20 01:45:00 -08:00
Minio Trusted 23c10350f3 Update yaml files to latest version RELEASE.2024-02-17T01-15-57Z 2024-02-17 01:37:10 +00:00
Harshavardhana b6e98aed01
fix: found races in accessing globalLocalDrives (#19069)
make a copy before accessing globalLocalDrives

Bonus: update console v0.46.0

Signed-off-by: Harshavardhana <harsha@minio.io>
2024-02-16 17:15:57 -08:00
Minio Trusted b5791e6f28 Update yaml files to latest version RELEASE.2024-02-14T21-36-02Z 2024-02-14 21:51:12 +00:00
Harshavardhana 00cb58eaf3
add customer specific hotfixes to 'registry.min.dev' (#19057)
```
REPO="registry.min.dev/<customer>" CRED_DIR=/media/builder/minio make docker-hotfix-push
```
2024-02-14 13:36:02 -08:00
Poorna 912a0031b7
fix sr tests to capture all server logs (#19051) 2024-02-13 20:51:23 -08:00
Minio Trusted f8e15e7d09 Update yaml files to latest version RELEASE.2024-02-13T15-35-11Z 2024-02-13 16:01:38 +00:00
Harshavardhana 0c068b15c7
add missing handler for reloading site replication config on peers (#19042) 2024-02-13 06:55:54 -08:00
Minio Trusted 6b63123ca9 Update yaml files to latest version RELEASE.2024-02-12T21-02-27Z 2024-02-12 21:40:49 +00:00
Harshavardhana eac4e4b279
honor replaced disk properly by updating globalLocalDrives (#19038)
globalLocalDrives seem to be not updated during the
HealFormat() leads to a requirement where the server
needs to be restarted for the healing to continue.
2024-02-12 13:00:20 -08:00
Minio Trusted 8e69f3cb89 Update yaml files to latest version RELEASE.2024-02-09T21-25-16Z 2024-02-09 22:48:07 +00:00
Harshavardhana 997ba3a574
introduce reader deadlines for net.Conn (#19023)
Bonus: set "retry-after" header for AWS SDKs if possible to honor them.
2024-02-09 13:25:16 -08:00
Aditya Manthramurthy a29c66ed74
Update IAM access manager plugin demo (#19007)
Now prints the JSON payload for easier debugging.
2024-02-08 09:15:20 -08:00
Minio Trusted eead4db1d2 Update yaml files to latest version RELEASE.2024-02-06T21-36-22Z 2024-02-07 12:11:36 +00:00
Poorna 27d02ea6f7
metrics: add replication metrics on proxied requests (#18957) 2024-02-05 22:00:45 -08:00
Minio Trusted 8414aff424 Update yaml files to latest version RELEASE.2024-02-04T22-36-13Z 2024-02-05 00:46:41 +00:00
Minio Trusted ee0055b929 Update yaml files to latest version RELEASE.2024-01-31T20-20-33Z 2024-01-31 20:35:33 +00:00
Daniel Valdivia 403ec7cf21
fix: metrics URI path in prometheus docs (#18907) 2024-01-29 14:34:21 -08:00
Minio Trusted ff4f4d4649 Update yaml files to latest version RELEASE.2024-01-29T03-56-32Z 2024-01-29 05:33:23 +00:00
Minio Trusted ff8269575a Update yaml files to latest version RELEASE.2024-01-28T22-35-53Z 2024-01-29 01:22:56 +00:00