MyFavMirrors/minio
1
0
Fork 0
mirror of https://github.com/minio/minio.git synced 2025-01-11 15:03:22 -05:00
Code Issues Packages Projects Releases Wiki Activity

Bump readiness check upto 10000 go-routines (#8057)

Browse Source 
Most of our current workloads reach this value
regularly, it doesn't make sense to keep 1000
go-routine limit.
This commit is contained in:
Harshavardhana 2019-08-10 05:43:14 -07:00 committed by Nitish Tiwari
parent 3385bf3da8
commit 5a28ef0d47
2 changed files with 11 additions and 9 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

18
cmd/healthcheck-handler.go
View File

@ -26,14 +26,16 @@ import (
)
const (
minioHealthGoroutineThreshold = 1000
minioHealthGoroutineThreshold = 10000
)
// ReadinessCheckHandler -- checks if there are more than threshold number of goroutines running,
// returns service unavailable.
// Readiness probes are used to detect situations where application is under heavy load
// and temporarily unable to serve. In a orchestrated setup like Kubernetes, containers reporting
// that they are not ready do not receive traffic through Kubernetes Services.
// ReadinessCheckHandler -- checks if there are more than threshold
// number of goroutines running, returns service unavailable.
//
// Readiness probes are used to detect situations where application
// is under heavy load and temporarily unable to serve. In a orchestrated
// setup like Kubernetes, containers reporting that they are not ready do
// not receive traffic through Kubernetes Services.
func ReadinessCheckHandler(w http.ResponseWriter, r *http.Request) {
if err := goroutineCountCheck(minioHealthGoroutineThreshold); err != nil {
writeResponse(w, http.StatusServiceUnavailable, nil, mimeNone)
@ -98,8 +100,8 @@ func LivenessCheckHandler(w http.ResponseWriter, r *http.Request) {
writeResponse(w, http.StatusOK, nil, mimeNone)
}
// checks threshold against total number of go-routines in the system and throws error if
// more than threshold go-routines are running.
// checks threshold against total number of go-routines in the system and
// throws error if more than threshold go-routines are running.
func goroutineCountCheck(threshold int) error {
count := runtime.NumGoroutine()
if count > threshold {

2
docs/metrics/healthcheck/README.md
View File

@ -14,7 +14,7 @@ When liveness probe fails, Kubernetes like platforms restart the container.
This probe is used to identify situations where the server is not ready to accept requests yet. In most cases, such conditions recover in some time.
Internally, MinIO readiness probe handler checks for total go-routines. If the number of go-routines is less than 1000 (threshold), the server returns 200 OK, otherwise 503 Service Unavailable.
Internally, MinIO readiness probe handler checks for total go-routines. If the number of go-routines is less than 10000 (threshold), the server returns 200 OK, otherwise 503 Service Unavailable.
Platforms like Kubernetes *do not* forward traffic to a pod until its readiness probe is successful.