Commit Graph

11438 Commits

Author SHA1 Message Date
Shubhendu
6d4c1156d6
Changed the expression to render the value (#18627)
The metrics `minio_bucket_replication_received_bytes` and
`minio_bucket_replication_sent_bytes` are additive in nature
and rendering the value as is looks fine.

Also added sort order for few graphs for better reading of tool
tips as keeping ones with highest value at top helps.

Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>
2023-12-13 10:05:47 -08:00
Harshavardhana
3f72c7fcc7
healthcheck requests with user-agent mozilla do not need redirects (#18642)
apparently, windows powershell curl has this abhorrent behavior
2023-12-12 16:16:26 -08:00
Harshavardhana
d521c84d55
reduce logging during permission denied errors (#18641)
log them if any only once
2023-12-12 16:11:17 -08:00
opencmit2
946b070744
resolve the absence of the sed command in ubi-micro image (#18459)
fix #18315
2023-12-12 15:43:32 -08:00
Anis Eleuch
4a21dce2b5
tier: Add support of SP credentials with Azure (#18630)
Co-authored-by: Anis Elleuch <anis@min.io>
2023-12-11 21:51:53 -08:00
Minio Trusted
5fe7f9fa93 Update yaml files to latest version RELEASE.2023-12-09T18-17-51Z 2023-12-09 21:03:28 +00:00
Harshavardhana
65f34cd823
fix: remove ODirectReader entirely since we do not need it anymore (#18619) 2023-12-09 10:17:51 -08:00
Harshavardhana
196e7e072b
allow bitrot files to be healed in MRF (#18618)
bitrot scanMode was ignored in MRF,
allow it to heal relevant content if
needed when seen as an error.
2023-12-08 12:26:01 -08:00
Anis Eleuch
6f97663174
yml-config: Add support of rootUser and rootPassword (#18615)
Users can define the root user and password in the yaml configuration
file; Root credentials defined in the environment variable still take
precedence
2023-12-08 12:04:54 -08:00
Anis Eleuch
aed7a1818a
info: Populate pool/set/disk indexes for offline disks (#18613)
This can be calculated from the disk layout and some external
applications would like to know the location of the offline
disks.
2023-12-08 08:13:04 -08:00
Harshavardhana
b50d90183e
use latest minisign release (#18614) 2023-12-08 00:22:22 -08:00
Poorna
6b06da76cb
add configuration to limit replication workers (#18601) 2023-12-07 16:22:00 -08:00
jiuker
6ca6788bb7
feat: add events_errors_total metric (#18610) 2023-12-07 16:21:17 -08:00
Anis Eleuch
2e23e61a45
Add support of conf file to pass arguments and options (#18592) 2023-12-07 01:33:56 -08:00
Minio Trusted
9cdf490bc5 Update yaml files to latest version RELEASE.2023-12-07T04-16-00Z 2023-12-07 06:28:15 +00:00
Alex
cfed671ea3
Update Console version to v0.42.1 (#18606)
Signed-off-by: Benjamin Perez <benjamin@bexsoft.net>
2023-12-06 20:16:00 -08:00
Harshavardhana
53ce92b9ca
fix: use the right channel to feed the data in (#18605)
this PR fixes a regression in batch replication
where we weren't sending any data from the Walk()
results due to incorrect channels being used.
2023-12-06 18:17:03 -08:00
Shireesh Anjal
7350a29fec
Capture percentage of cpu load and memory used (#18596)
By default the cpu load is the cumulative of all cores. Capture the
percentage load (load * 100 / cpu-count)

Also capture the percentage memory used (used * 100 / total)
2023-12-06 13:19:59 -08:00
jiuker
5cc2c62c66
fix: GetFreePort() will get the same port (#18604) 2023-12-06 10:36:42 -08:00
Harshavardhana
4bc5ed6c76
support LDAP service accounts via SFTP, FTP logins (#18599) 2023-12-06 04:31:35 -08:00
Minio Trusted
e99a597899 Update yaml files to latest version RELEASE.2023-12-06T09-09-22Z 2023-12-06 10:16:37 +00:00
Harshavardhana
73dde66dbe
stick to go1.19 go.mod (#18600) 2023-12-06 01:09:22 -08:00
Harshavardhana
e30c0e7ca3 Revert "Heal buckets at node level (#18504)"
This reverts commit 708296ae1b.
2023-12-05 22:34:46 -08:00
Klaus Post
8fc200c0cc
Truncate long traces for internode communication (#18593)
Prevent excessively long request traces.
2023-12-05 12:16:48 -08:00
Shubhendu
708296ae1b
Heal buckets at node level (#18504) 2023-12-05 02:17:35 -08:00
Harshavardhana
fbb5e75e01
avoid run-away goroutine build-up in notification send, use channels (#18533)
use memory for async events when necessary and dequeue them as
needed, for all synchronous events customers must enable

```
MINIO_API_SYNC_EVENTS=on
```

Async events can be lost but is upto to the admin to
decide what they want, we will not create run-away number
of goroutines per event instead we will queue them properly.

Currently the max async workers is set to runtime.GOMAXPROCS(0)
which is more than sufficient in general, but it can be made
configurable in future but may not be needed.
2023-12-05 02:16:33 -08:00
Harshavardhana
f327b21557
handle crashes with ILM expiry changes (#18590) 2023-12-05 01:14:36 -08:00
Harshavardhana
45b7253f39
parallelize renameData() cleanup upon error (#18591) 2023-12-04 14:54:34 -08:00
Harshavardhana
05bb655efc
avoid caching metrics for timeout errors per drive (#18584)
Bonus: combine the loop for drive/REST registration.
2023-12-04 11:54:13 -08:00
Harshavardhana
8fdfcfb562
upon RenameData() quorum error delete any partial success (#18586)
there is potential for danglingWrites when quorum failed, where
only some drives took a successful write, generally this is left
to the healing routine to pick it up. However it is better that
we delete it right away to avoid potential for quorum issues on
version signature when there are many versions of an object.
2023-12-04 11:33:39 -08:00
Harshavardhana
e7c144eeac
avoid double MRF heal when there is versions disparity (#18585) 2023-12-04 11:13:50 -08:00
Harshavardhana
e98172d72d
avoid hot-tier SLA to be tied to warm-tier SLA (#18581)
it is okay if the warm-tier cannot keep up, we should continue
to take I/O at hot-tier, only fail hot-tier or block it when
we are disk full.

Bonus: add metrics counter for these missed tasks, we will
know for sure if one of the node is lagging behind or is
losing too many tasks during transitioning.
2023-12-02 13:02:12 -08:00
Minio Trusted
f2d063e7b9 Update yaml files to latest version RELEASE.2023-12-02T10-51-33Z 2023-12-02 11:14:35 +00:00
Krishnan Parthasarathi
a50f26b7f5
Implement batch-expiration for objects (#17946)
Based on an initial PR from -
https://github.com/minio/minio/pull/17792

But fully completes it with newer finalized YAML spec.
2023-12-02 02:51:33 -08:00
Klaus Post
69294cf98a
Disable DMA optimization on windows (#18575)
It appears that Windows can lock up when errors occur. Use regular copy here.
2023-12-01 16:13:19 -08:00
Krishnan Parthasarathi
c397fb6c7a
Minor fixes to bucket replication (#18578) 2023-12-01 16:13:08 -08:00
Klaus Post
961b0b524e
Do not require restart when a disk is unreachable during node boot (#18576)
A disk that is not able to initialize when an instance is started
will never have a handler registered, which means a user will
need to restart the node after fixing the disk;

This will also prevent showing the wrong 'upgrade is needed.'
error message in that case.

When the disk is still failing, print an error every 30 minutes;
Disk reconnection will be retried every 30 seconds.

Co-authored-by: Anis Elleuch <anis@min.io>
2023-12-01 12:01:14 -08:00
Klaus Post
860fc200b0
Local and Remote hosts swapped in grid traces (#18574)
Local and Remote hosts swapped in grid trace

A bit counter-intuitive, but simple fix.
2023-12-01 08:04:08 -08:00
Harshavardhana
109a9e3f35
skip ILM expired objects from healing (#18569) 2023-12-01 07:56:24 -08:00
Klaus Post
5f971fea6e
Fix Mux Connect Error (#18567)
`OpMuxConnectError` was not handled correctly.

Remove local checks for single request handlers so they can 
run before being registered locally.

Bonus: Only log IAM bootstrap on startup.
2023-12-01 00:18:04 -08:00
Harshavardhana
0d7abe3b9f
allow hotfixes to generate deb, rpm packages (#18568)
```
using deb packager...
created package: minio-release/linux-amd64/minio_20231120224007.0.0.hotfix.e96ac7272_amd64.deb
using rpm packager...
created package: minio-release/linux-amd64/minio-20231120224007.0.0.hotfix.e96ac7272-1.x86_64.rpm
```
2023-11-30 15:25:51 -08:00
Klaus Post
94fbcd8ebe
Add TLS cert checksum (#18557)
It allows validation of whether all certs match across clusters.
2023-11-30 12:13:50 -08:00
Harshavardhana
879d5dd236
site replication must heal policy mappings with correct userType (#18563) 2023-11-30 10:34:18 -08:00
jiuker
34187e047d
feat: support elasticsearch notification endpoint compression codec (#18562) 2023-11-30 00:25:03 -08:00
Harshavardhana
0ee722f8c3
cleanup handling of STS isAllowed and simplifies the PolicyDBGet() (#18554) 2023-11-29 16:07:35 -08:00
Anis Eleuch
b7d11141e1
rename Force to Immediate for clarity (#18540) 2023-11-28 22:35:16 -08:00
Harshavardhana
e9babf3dac
(chore): update all our deps (#18525) 2023-11-28 14:44:44 -08:00
Klaus Post
0bb81f2e9c
Always remove subroute when queuing message on the connection. (#18550) 2023-11-28 11:22:29 -08:00
Klaus Post
bea0b050cd
Improve env var config error reporting (#18549)
Improve env var config error

Env vars that were set on current server but not on remotes were not reported in errors.

Add these.
2023-11-28 10:39:02 -08:00
Shubhendu
ce62980d4e
Fixed transition rules getting overwritten while healing (#18542)
While healing the latest changes of expiry rules across sites
if target had pre existing transition rules, they were getting
overwritten as cloned latest expiry rules from remote site were
getting written as is. Fixed the same and added test cases as
well.

Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>
2023-11-28 10:38:35 -08:00