1
0
mirror of https://github.com/minio/minio.git synced 2025-01-22 12:13:16 -05:00

16 Commits

Author SHA1 Message Date
Krishnan Parthasarathi
b69bcdcdc4
Fix ilm config at startup ()
Remove api.expiration_workers config setting which was inadvertently left behind. Per review comment 

https://github.com/minio/minio/pull/18926, expiration_workers can be configured via ilm.expiration_workers.
2024-03-04 18:50:24 -08:00
Krishnan Parthasarathi
a7577da768
Improve expiration of tiered objects ()
- Use a shared worker pool for all ILM expiry tasks
- Free version cleanup executes in a separate goroutine
- Add a free version only if removing the remote object fails
- Add ILM expiry metrics to the node namespace
- Move tier journal tasks to expiryState
- Remove unused on-disk journal for tiered objects pending deletion
- Distribute expiry tasks across workers such that the expiry of versions of
  the same object serialized
- Ability to resize worker pool without server restart
- Make scaling down of expiryState workers' concurrency safe; Thanks
  @klauspost
- Add error logs when expiryState and transition state are not
  initialized (yet)
* metrics: Add missed tier journal entry tasks
* Initialize the ILM worker pool after the object layer
2024-03-01 21:11:03 -08:00
Harshavardhana
a50ea92c64
feat: introduce list_quorum="auto" to prefer quorum drives ()
NOTE: This feature is not retro-active; it will not cater to previous transactions
on existing setups. 

To enable this feature, please set ` _MINIO_DRIVE_QUORUM=on` environment
variable as part of systemd service or k8s configmap. 

Once this has been enabled, you need to also set `list_quorum`. 

```
~ mc admin config set alias/ api list_quorum=auto` 
```

A new debugging tool is available to check for any missing counters.
2023-12-29 15:52:41 -08:00
Poorna
6b06da76cb
add configuration to limit replication workers () 2023-12-07 16:22:00 -08:00
Harshavardhana
91d8bddbd1
use sendfile/splice implementation to perform DMA ()
sendfile implementation to perform DMA on all platforms

Go stdlib already supports sendfile/splice implementations
for

- Linux
- Windows
- *BSD
- Solaris

Along with this change however O_DIRECT for reads() must be
removed as well since we need to use sendfile() implementation

The main reason to add O_DIRECT for reads was to reduce the
chances of page-cache causing OOMs for MinIO, however it would
seem that avoiding buffer copies from user-space to kernel space
this issue is not a problem anymore.

There is no Go based memory allocation required, and neither
the page-cache is referenced back to MinIO. This page-
cache reference is fully owned by kernel at this point, this
essentially should solve the problem of page-cache build up.

With this now we also support SG - when NIC supports Scatter/Gather
https://en.wikipedia.org/wiki/Gather/scatter_(vector_addressing)
2023-11-10 10:10:14 -08:00
Harshavardhana
73edd5b8fd
introduce 'mc admin config set alias/ api odirect=on' ()
change disable_odirect=off -> odirect=on to make it
easier to understand, instead of making it double
negative.
2023-07-31 00:12:53 -07:00
Harshavardhana
f13cfcb83e
allow disabling O_DIRECT for write ops ()
on really slow systems, O_DIRECT simply kills the drives
allow for a way to disable them.
2023-07-29 15:17:56 -07:00
Praveen raj Mani
7c72b25ef0
Add an option to make bucket notifications synchronous ()
With the current asynchronous behaviour in sending notification events
to the targets, we can't provide guaranteed delivery as the systems
might go for restarts.

For such event-driven use-cases, we can provide an option to enable
synchronous events where the APIs wait until the event is successfully
sent or persisted.

This commit adds 'MINIO_API_SYNC_EVENTS' env which when set to 'on'
will enable sending/persisting events to targets synchronously.
2023-06-20 17:38:59 -07:00
Harshavardhana
7ae69accc0
allow root user to be disabled via config settings () 2023-04-28 12:24:14 -07:00
Poorna
8ea6fb368d
Add auto configuration of replication workers () 2022-09-24 16:20:28 -07:00
Sidhartha Mani
fe1fbe0005
standardize config help defaults () 2022-04-26 20:11:37 -07:00
Harshavardhana
70e1cbda21
allow disabling O_DIRECT in certain environments for reads ()
repeated reads on single large objects in HPC like
workloads, need the following option to disable
O_DIRECT for a more effective usage of the kernel
page-cache.

However this optional should be used in very specific
situations only, and shouldn't be enabled on all
servers.

NVMe servers benefit always from keeping O_DIRECT on.
2022-01-17 08:34:14 -08:00
David Regla
a188554fe1
Add missing keys to API config help ()
Added missing `apiClusterDeadline` and `apiListQuorum` to API config.HelpKVS structure
2021-10-10 09:52:21 -07:00
Harshavardhana
fabf60bc4c
fix: allow configuring cleanup of stale multipart uploads ()
allow dynamically changing cleanup of stale multipart
uploads, their expiry and how frequently its checked.

Improves 
2021-10-04 10:52:28 -07:00
Krishnan Parthasarathi
65b6f4aa31
Add dynamic reconfiguration of number of transition workers () 2021-08-11 22:23:56 -07:00
Harshavardhana
1f262daf6f
rename all remaining packages to internal/ ()
This is to ensure that there are no projects
that try to import `minio/minio/pkg` into
their own repo. Any such common packages should
go to `https://github.com/minio/pkg`
2021-06-01 14:59:40 -07:00