Commit Graph

10 Commits

Author SHA1 Message Date
poornas 224b4f13b8
Add cache eviction low and high watermarks (#8958)
To allow better control the cache eviction process.

Introduce MINIO_CACHE_WATERMARK_LOW and 
MINIO_CACHE_WATERMARK_HIGH env. variables to specify 
when to stop/start cache eviction process. 

Deprecate MINIO_CACHE_EXPIRY environment variable. Cache 
gc sweeps at 30 minute intervals whenever high watermark is
reached to clear least recently accessed entries in the cache
until sufficient space is cleared to reach the low watermark.

Garbage collection uses an adaptive file scoring approach based
on last access time, with greater weights assigned to larger
objects and those with more hits to find the candidates for eviction.

Thanks to @klauspost for this file scoring algorithm

Co-authored-by: Klaus Post <klauspost@minio.io>
2020-02-23 19:03:39 +05:30
poornas 278a165674
Allow caching based on a configurable number of hits. (#8891)
Co-authored-by: Harshavardhana <harsha@minio.io>
2020-02-04 09:10:01 +05:30
Harshavardhana 686d4656de
fix: set appropriate defaults when new keys added (#8795)
A new key was added in identity_openid recently
required explicitly for client to set the optional
value without that it would be empty, handle this
appropriately.

Fixes #8787
2020-01-10 16:57:18 -08:00
Harshavardhana c9940d8c3f Final changes to config sub-system (#8600)
- Introduces changes such as certain types of
  errors that can be ignored or which need to 
  go into safe mode.
- Update help text as per the review
2019-12-04 15:32:37 -08:00
Harshavardhana 794eb54da8 Export command prints turned-off sub-sys as comments (#8594)
This PR also tries to

- Preserve the order of keys printed in export command
- Fix cache to be enabled with _STATE env to keep
  backward compatibility
2019-12-03 10:50:20 -08:00
Harshavardhana 5ac4b517c9
Order all keys in config (#8541)
New changes

- return default values when sub-sys is
  not configured.
- state is hidden parameter now
- remove worm mode to be saved in config
2019-11-20 15:10:24 -08:00
Harshavardhana 26a866a202
Fix review comments and new changes in config (#8515)
- Migrate and save only settings which are enabled
- Rename logger_http to logger_webhook and
  logger_http_audit to audit_webhook
- No more pretty printing comments, comment
  is a key=value pair now.
- Avoid quotes on values which do not have space in them
- `state="on"` is implicit for all SetConfigKV unless
  specified explicitly as `state="off"`
- Disabled IAM users should be disabled always
2019-11-13 17:38:05 -08:00
Harshavardhana 47b13cdb80 Add etcd part of config support, add noColor/json support (#8439)
- Add color/json mode support for get/help commands
- Support ENV help for all sub-systems
- Add support for etcd as part of config
2019-10-30 00:04:39 -07:00
Harshavardhana ee4a6a823d Migrate config to KV data format (#8392)
- adding oauth support to MinIO browser (#8400) by @kanagaraj
- supports multi-line get/set/del for all config fields
- add support for comments, allow toggle
- add extensive validation of config before saving
- support MinIO browser to support proper claims, using STS tokens
- env support for all config parameters, legacy envs are also
  supported with all documentation now pointing to latest ENVs
- preserve accessKey/secretKey from FS mode setups
- add history support implements three APIs
  - ClearHistory
  - RestoreHistory
  - ListHistory
- add help command support for each config parameters
- all the bug fixes after migration to KV, and other bug
  fixes encountered during testing.
2019-10-22 22:59:13 -07:00
Harshavardhana 589e32a4ed Refactor config and split them in packages (#8351)
This change is related to larger config migration PR
change, this is a first stage change to move our
configs to `cmd/config/` - divided into its subsystems
2019-10-04 23:05:33 +05:30