Commit Graph

35 Commits

Author SHA1 Message Date
Aditya Manthramurthy
18dffb26e7
Allow querying a single target in config get API (#15587) 2022-08-25 00:17:05 -07:00
Aditya Manthramurthy
c54c13831a
fix: error types in internal config package (#15559) 2022-08-20 15:11:49 -07:00
Aditya Manthramurthy
21831b3fe2
fix: cleanup config KV parsing using madmin helpers (#15552) 2022-08-18 15:55:17 -07:00
Aditya Manthramurthy
701a82642b
Ensure subnet+callhome configs are single sub-sys (#15542) 2022-08-16 17:46:46 -07:00
Aditya Manthramurthy
ee2afcf70b
Move some config constants to madmin (#15529) 2022-08-13 14:50:16 -07:00
Aditya Manthramurthy
9986e103cf
Fix env var output in config get/export APIs (#15528)
Fix a bug where env vars are not output when the config for the
subsystem is specified solely via env vars.
2022-08-13 10:39:01 -07:00
Aditya Manthramurthy
3d94c38ec4
Add env variables to configuration APIs output (#15465)
Config export and config get APIs now include environment 
variables set on the server
2022-08-04 22:21:52 -07:00
Anis Elleuch
10b49eb4fb
Fix resetting a config with a non default target name (#15448)
mc admin config reset <alias> notify_webhook:something was not working
properly.

The reason is that GetSubSys() was not calculating the target
name properly because it is quitting early when the number of config
inputs ('notify_webhook:something' in this case) is equal to 1.

This commit will make the code calculates always calculate the target
name if found.
2022-08-01 07:52:23 -07:00
Taran Pelkey
e83930333b
Allow DelKVS to delete specific sub-system fields. (#15354) 2022-07-22 14:48:23 -07:00
Aditya Manthramurthy
af9bc7ea7d
Add external IDP management Admin API for OpenID (#15152) 2022-07-05 18:18:04 -07:00
Harshavardhana
8856a2d77b
finalize startup-banner and remove unnecessary logs (#15202) 2022-06-29 16:32:04 -07:00
Aditya Manthramurthy
7f629df4d5
Add generic function to retrieve config value with metadata (#15083)
`config.ResolveConfigParam` returns the value of a configuration for any
subsystem based on checking env, config store, and default value. Also returns info
about which config source returned the value.

This is useful to return info about config params overridden via env in the user
APIs. Currently implemented only for OpenID subsystem, but will be extended for
others subsequently.
2022-06-17 11:39:21 -07:00
Shireesh Anjal
4ce81fd07f
Add periodic callhome functionality (#14918)
* Add periodic callhome functionality

Periodically (every 24hrs by default), fetch callhome information and
upload it to SUBNET.

New config keys under the `callhome` subsystem:

enable - Set to `on` for enabling callhome. Default `off`
frequency - Interval between callhome cycles. Default `24h`

* Improvements based on review comments

- Update `enableCallhome` safely
- Rename pctx to ctx
- Block during execution of callhome
- Store parsed proxy URL in global subnet config
- Store callhome URL(s) in constants
- Use existing global transport
- Pass auth token to subnetPostReq
- Use `config.EnableOn` instead of `"on"`

* Use atomic package instead of lock

* Use uber atomic package

* Use `Cancel` instead of `cancel`

Co-authored-by: Harshavardhana <harsha@minio.io>

Co-authored-by: Harshavardhana <harsha@minio.io>
Co-authored-by: Aditya Manthramurthy <donatello@users.noreply.github.com>
2022-06-06 16:14:52 -07:00
Aditya Manthramurthy
464b9d7c80
Add support for Identity Management Plugin (#14913)
- Adds an STS API `AssumeRoleWithCustomToken` that can be used to 
  authenticate via the Id. Mgmt. Plugin.
- Adds a sample identity manager plugin implementation
- Add doc for plugin and STS API
- Add an example program using go SDK for AssumeRoleWithCustomToken
2022-05-26 17:58:09 -07:00
Aditya Manthramurthy
83071a3459
Add support for Access Management Plugin (#14875)
- This change renames the OPA integration as Access Management Plugin - there is
nothing specific to OPA in the integration, it is just a webhook.

- OPA configuration is automatically migrated to Access Management Plugin and
OPA specific configuration is marked as deprecated.

- OPA doc is updated and moved.
2022-05-10 17:14:55 -07:00
Aditya Manthramurthy
0e502899a8
Add support for multiple OpenID providers with role policies (#14223)
- When using multiple providers, claim-based providers are not allowed. All
providers must use role policies.

- Update markdown config to allow `details` HTML element
2022-04-28 18:27:09 -07:00
Klaus Post
96adfaebe1
Make storage class config dynamic (#14791)
Updating the storage class is already thread safe, so we can do this safely.
2022-04-21 12:07:33 -07:00
Shireesh Anjal
3934700a08
Make audit webhook and kafka config dynamic (#14390) 2022-02-24 09:05:33 -08:00
Shireesh Anjal
94d37d05e5
Apply dynamic config at sub-system level (#14369)
Currently, when applying any dynamic config, the system reloads and
re-applies the config of all the dynamic sub-systems.

This PR refactors the code in such a way that changing config of a given
dynamic sub-system will work on only that sub-system.
2022-02-22 10:59:28 -08:00
Shireesh Anjal
28f188e3ef
Make logger webhook config dynamic (#14289)
It should not be required to restart the 
server after setting the logger webhook config.
2022-02-17 11:11:15 -08:00
Shireesh Anjal
9890f579f8
Add subsystem level validation on config set (#14269)
When setting a config of a particular sub-system, validate the existing
config and notification targets of only that sub-system, so that
existing errors related to one sub-system (e.g. notification target
offline) do not result in errors for other sub-systems.
2022-02-08 10:36:41 -08:00
Shireesh Anjal
3882da6ac5
Add subnet proxy config (#14225)
Will store the HTTP(S) proxy URL to use for connecting to SUBNET.
2022-02-01 09:52:38 -08:00
Harshavardhana
f527c708f2
run gofumpt cleanup across code-base (#14015) 2022-01-02 09:15:06 -08:00
Shireesh Anjal
d29df6714a
Introduce new config subnet api_key (#13793)
The earlier approach of using a license token for 
communicating with SUBNET is being replaced 
with a simpler mechanism of API keys. Unlike the 
license which is a JWT token, these API keys will 
be simple UUID tokens and don't have any embedded 
information in them. SUBNET would generate the 
API key on cluster registration, and then it would 
be saved in this config, to be used for subsequent 
communication with SUBNET.
2021-12-03 09:32:11 -08:00
Aditya Manthramurthy
4ce6d35e30
Add new site config sub-system intended to replace region (#13672)
- New sub-system has "region" and "name" fields.

- `region` subsystem is marked as deprecated, however still works, unless the
new region parameter under `site` is set - in this case, the region subsystem is
ignored. `region` subsystem is hidden from top-level help (i.e. from `mc admin
config set myminio`), but appears when specifically requested (i.e. with `mc
admin config set myminio region`).

- MINIO_REGION, MINIO_REGION_NAME are supported as legacy environment variables for server region.

- Adds MINIO_SITE_REGION as the current environment variable to configure the
server region and MINIO_SITE_NAME for the site name.
2021-11-25 13:06:25 -08:00
Shireesh Anjal
d008e90d50
Support dynamic reset of minio config (#13626)
If a given MinIO config is dynamic (can be changed without restart),
ensure that it can be reset also without restart.

Signed-off-by: Shireesh Anjal <shireesh@minio.io>
2021-11-10 10:01:32 -08:00
Shireesh Anjal
b4364723ef
Add config to store subnet license (#13194)
Command to set subnet license:

`mc admin config set {alias} subnet license={token}`

Signed-off-by: Shireesh Anjal <shireesh@minio.io>
Co-authored-by: Harshavardhana <harsha@minio.io>
2021-09-14 21:54:25 -07:00
Andreas Auernhammer
e438dccf19
sts: add support for certificate-based authentication (#12748)
This commit adds a new STS API for X.509 certificate
authentication.

A client can make an HTTP POST request over a TLS connection
and MinIO will verify the provided client certificate, map it to an 
S3 policy and return temp. S3 credentials to the client.

So, this STS API allows clients to authenticate with X.509
certificates over TLS and obtain temp. S3 credentials.

For more details and examples refer to the docs/sts/tls.md
documentation.

Signed-off-by: Andreas Auernhammer <hi@aead.dev>
2021-09-07 19:03:48 -07:00
Shireesh Anjal
4d6d4244f1
Anonymize hosts in subnet health report (#12627)
Ensure that hostnames / ip addresses are not printed in the subnet
health report. Anonymize them by replacing them with `servern` where `n`
represents the position of the server in the pool.

This is done by building a `host anonymizer` map that maps every
possible value containing the host e.g. host, host:port,
http://host:port, etc to the corresponding anonymized name and using
this map to replace the values at the time of health report generation.

A different logic is used to anonymize host names in the `procinfo`
data, as the host names are part of an ellipses pattern in the process
start command. Here we just replace the prefix/suffix of the ellipses
pattern with their hashes.
2021-07-14 00:23:22 -07:00
Harshavardhana
e316873f84
feat: Add support for kakfa audit logger target (#12678) 2021-07-13 09:39:13 -07:00
Harshavardhana
8d1bc65757
allow resetting and reapply config on broken clusters (#12554)
Bonus: remove kms_kes as sub-system, since its ENV only.
- also fixes a crash with etcd cluster without KMS
configured and also if KMS decryption is missing.
2021-06-24 16:24:12 -07:00
Harshavardhana
ed6cc66cf4
fix: remove unnecessary limit for queueStore (#12491)
There is no good reason to limit ourselves
to max_open_fd for queue_store

Bonus: Support for publisher confirms
2021-06-14 13:28:44 -07:00
Shireesh Anjal
fb140c146b
Redact sensitive values from config in health data (#12421)
The health api returns the server configuration details. Redact
sensitive values from the config values like URLs and credentials.
2021-06-03 08:15:44 -07:00
Harshavardhana
7a3b5235bf remove deprecated kms_vault unused key name 2021-06-03 00:10:11 -07:00
Harshavardhana
1f262daf6f
rename all remaining packages to internal/ (#12418)
This is to ensure that there are no projects
that try to import `minio/minio/pkg` into
their own repo. Any such common packages should
go to `https://github.com/minio/pkg`
2021-06-01 14:59:40 -07:00