Currently, one node in a cluster can fail to boot with the following error message:
```
ERROR Unable to initialize config system: Storage resources are insufficient for the write operation
```
This happens when disks are formatted, read quorum is met but write
quorum is not met. In checkServerConfig(), a insufficient read quorum
error is replaced by errConfigNotFound, the code will generate a
new config json and try to save it, but it will fail because write
quorum is not met.
Replacing read quorum with errConfigNotFound is also wrong because it
can lead, in rare cases, to overwrite the config set by the user.
So, this commit adds a retry mechanism in configuration initialization
to retry only with read or write quorum errors.
This commit will also fix the following cases:
- Read quorum is lost just after the initialization of the object layer.
- Write quorum not met when upgrading configuration version.
This PR adds two new admin APIs in Minio server and madmin package:
- GetConfigKeys(keys []string) ([]byte, error)
- SetConfigKeys(params map[string]string) (err error)
A key is a path in Minio configuration file, (e.g. notify.webhook.1)
The user will always send a string value when setting it in the config file,
the API will know how to convert the value to the appropriate type. The user
is also able to set a raw json.
Before setting a new config, Minio will validate all fields and try to connect
to notification targets if available.
This PR is the first set of changes to move the config
to the backend, the changes use the existing `config.json`
allows it to be migrated such that we can save it in on
backend disks.
In future releases, we will slowly migrate out of the
current architecture.
Fixes#6182
These messages based on our prep stage during XL
and prints more informative message regarding
drive information.
This change also does a much needed refactoring.
