mirror of
https://github.com/minio/minio.git
synced 2024-12-23 21:55:53 -05:00
Added kernel tuning docs (#3921)
This commit is contained in:
parent
a099319e66
commit
a9c0f1e0a4
71
docs/deployment/kernel-tuning/README.md
Normal file
71
docs/deployment/kernel-tuning/README.md
Normal file
@ -0,0 +1,71 @@
|
||||
# Kernel Tuning for Minio Production Deployment on Linux Servers [![Slack](https://slack.minio.io/slack?type=svg)](https://slack.minio.io) [![Go Report Card](https://goreportcard.com/badge/minio/minio)](https://goreportcard.com/report/minio/minio) [![Docker Pulls](https://img.shields.io/docker/pulls/minio/minio.svg?maxAge=604800)](https://hub.docker.com/r/minio/minio/) [![codecov](https://codecov.io/gh/minio/minio/branch/master/graph/badge.svg)](https://codecov.io/gh/minio/minio)
|
||||
|
||||
## Tuning Network Parameters
|
||||
|
||||
Following network parameter settings can help ensure optimal Minio server performance on production workloads.
|
||||
|
||||
- *`tcp_fin_timeout`* : A socket left in memory takes approximately 1.5Kb of memory. It makes sense to close the unused sockets preemptively to ensure no memory leakage. This way, even if a peer doesn't close the socket due to some reason, the system itself closes it after a timeout. `tcp_fin_timeout` variable defines this timeout and tells kernel how long to keep sockets in the state FIN-WAIT-2. We recommend setting it to 30. You can set it as shown below
|
||||
|
||||
```sh
|
||||
`sysctl -w net.ipv4.tcp_fin_timeout=30`
|
||||
```
|
||||
|
||||
- *`tcp_keepalive_probes`* : This variable defines the number of unacknowledged probes to be sent before considering a connection dead. You can set it as shown below
|
||||
|
||||
```sh
|
||||
sysctl -w net.ipv4.tcp_keepalive_probes=5
|
||||
```
|
||||
|
||||
- *`wmem_max`*: This parameter sets the max OS send buffer size for all types of connections.
|
||||
|
||||
```sh
|
||||
sysctl -w net.core.wmem_max=540000
|
||||
```
|
||||
|
||||
- *`rmem_max`*: This parameter sets the max OS receive buffer size for all types of connections.
|
||||
|
||||
```sh
|
||||
sysctl -w net.core.rmem_max=540000
|
||||
```
|
||||
|
||||
## Tuning Virtual Memory
|
||||
|
||||
Recommended virtual memory settings are as follows.
|
||||
|
||||
- *`swappiness`* : This parameter controls the relative weight given to swapping out runtime memory, as opposed to dropping pages from the system page cache. It takes values from 0 to 100, both inclusive. We recommend setting it to 10.
|
||||
|
||||
```sh
|
||||
sysctl -w vm.swappiness=10
|
||||
```
|
||||
|
||||
- *`dirty_background_ratio`*: This is the percentage of system memory that can be filled with `dirty` pages, i.e. memory pages that still need to be written to disk. We recommend writing the data to the disk as soon as possible. To do this, set the `dirty_background_ratio` to 1.
|
||||
|
||||
```sh
|
||||
sysctl -w vm.dirty_background_ratio=1
|
||||
```
|
||||
|
||||
- *`dirty_ratio`*: This defines is the absolute maximum amount of system memory that can be filled with dirty pages before everything must get committed to disk.
|
||||
|
||||
```sh
|
||||
sysctl -w vm.dirty_ratio=5
|
||||
```
|
||||
|
||||
## Tuning Scheduler
|
||||
|
||||
Proper scheduler configuration makes sure Minio process gets adequate CPU time. Here are the recommended scheduler settings
|
||||
|
||||
- *`sched_min_granularity_ns`*: This parameter decides the minimum time a task will be be allowed to run on CPU before being pre-empted out. We recommend setting it to 10ms.
|
||||
|
||||
```sh
|
||||
sysctl -w kernel.sched_min_granularity_ns=10
|
||||
```
|
||||
|
||||
- *`sched_wakeup_granularity_ns`*: Lowering this parameter improves wake-up latency and throughput for latency critical tasks, particularly when a short duty cycle load component must compete with CPU bound components.
|
||||
|
||||
```sh
|
||||
sysctl -w kernel.sched_wakeup_granularity_ns=15
|
||||
```
|
||||
|
||||
## Tuning Disks
|
||||
|
||||
The recommendations for disk tuning are conveniently packaged in a well commented [shell script](https://github.com/minio/minio/blob/master/docs/deployment/kernel-tuning/disk-tuning.sh). Please review the shell script for our recommendations.
|
48
docs/deployment/kernel-tuning/disk-tuning.sh
Normal file
48
docs/deployment/kernel-tuning/disk-tuning.sh
Normal file
@ -0,0 +1,48 @@
|
||||
#!/bin/bash
|
||||
|
||||
## Minio Cloud Storage, (C) 2017 Minio, Inc.
|
||||
##
|
||||
## Licensed under the Apache License, Version 2.0 (the "License");
|
||||
## you may not use this file except in compliance with the License.
|
||||
## You may obtain a copy of the License at
|
||||
##
|
||||
## http://www.apache.org/licenses/LICENSE-2.0
|
||||
##
|
||||
## Unless required by applicable law or agreed to in writing, software
|
||||
## distributed under the License is distributed on an "AS IS" BASIS,
|
||||
## WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
## See the License for the specific language governing permissions and
|
||||
## limitations under the License.
|
||||
|
||||
for i in $(ls -d /sys/block/*/queue/iosched 2>/dev/null); do
|
||||
iosched_dir=$(echo $i | awk '/iosched/ {print $1}')
|
||||
[ -z $iosched_dir ] && {
|
||||
continue
|
||||
}
|
||||
## Change each disk ioscheduler to be "deadline"
|
||||
## Deadline dispatches I/Os in batches. A batch is a
|
||||
## sequence of either read or write I/Os which are in
|
||||
## increasing LBA order (the one-way elevator). After
|
||||
## processing each batch, the I/O scheduler checks to
|
||||
## see whether write requests have been starved for too
|
||||
## long, and then decides whether to start a new batch
|
||||
## of reads or writes
|
||||
path=$(dirname $iosched_dir)
|
||||
[ -f $path/scheduler ] && {
|
||||
echo "deadline" > $path/scheduler
|
||||
}
|
||||
## This controls how many requests may be allocated
|
||||
## in the block layer for read or write requests.
|
||||
## Note that the total allocated number may be twice
|
||||
## this amount, since it applies only to reads or
|
||||
## writes (not the accumulate sum).
|
||||
[ -f $path/nr_requests ] && {
|
||||
echo "256" > $path/nr_requests
|
||||
}
|
||||
## This is the maximum number of kilobytes
|
||||
## supported in a single data transfer at
|
||||
## block layer.
|
||||
[ -f $path/max_hw_sectors_kb ] && {
|
||||
echo "1024" > $path/max_hw_sectors_kb
|
||||
}
|
||||
done
|
Loading…
Reference in New Issue
Block a user