Commit Graph

47 Commits

Author SHA1 Message Date
Aditya Manthramurthy
f3248a4b37 Redact all secrets from config viewing APIs (#17380)
This change adds a `Secret` property to `HelpKV` to identify secrets
like passwords and auth tokens that should not be revealed by the server
in its configuration fetching APIs. Configuration reporting APIs now do
not return secrets.
2023-06-23 07:45:27 -07:00
Harshavardhana
74759b05a5 make sure to set relevant config entries correctly (#17485)
Bonus: also allow skipping keys properly.
2023-06-22 10:04:02 -07:00
Aditya Manthramurthy
5a1612fe32 Bump up madmin-go and pkg deps (#17469) 2023-06-19 17:53:08 -07:00
jiuker
d749aaab69 fix: ignore existing target status when adding new targets (#17250) 2023-05-24 22:57:37 -07:00
Poorna
d350654aee config: fix duplication of replication priority key (#17014) 2023-04-11 19:22:10 -07:00
Harshavardhana
901887e6bf feat: add lambda transformation functions target (#16507) 2023-03-07 08:12:41 -08:00
Anis Elleuch
e57e946206 Do not save credentials in config.json (#16275) 2022-12-19 12:27:06 -08:00
Aditya Manthramurthy
9e6cc847f8 Add HTTP2 config option for policy plugin (#16225) 2022-12-13 14:28:48 -08:00
Aditya Manthramurthy
a30cfdd88f Bump up madmin-go to v2 (#16162) 2022-12-06 13:46:50 -08:00
Anis Elleuch
993e586855 config: return XMinioConfigNotFound code for non existing config (#16065) 2022-11-18 10:28:14 -08:00
Anis Elleuch
7721595aa9 config: Deprecated delay/max_wait/scanner and introduce speed (#15941) 2022-11-01 08:04:07 -07:00
Aditya Manthramurthy
64cf887b28 use LDAP config from minio/pkg to share with console (#15810) 2022-10-07 22:12:36 -07:00
Aditya Manthramurthy
18dffb26e7 Allow querying a single target in config get API (#15587) 2022-08-25 00:17:05 -07:00
Aditya Manthramurthy
c54c13831a fix: error types in internal config package (#15559) 2022-08-20 15:11:49 -07:00
Aditya Manthramurthy
21831b3fe2 fix: cleanup config KV parsing using madmin helpers (#15552) 2022-08-18 15:55:17 -07:00
Aditya Manthramurthy
701a82642b Ensure subnet+callhome configs are single sub-sys (#15542) 2022-08-16 17:46:46 -07:00
Aditya Manthramurthy
ee2afcf70b Move some config constants to madmin (#15529) 2022-08-13 14:50:16 -07:00
Aditya Manthramurthy
9986e103cf Fix env var output in config get/export APIs (#15528)
Fix a bug where env vars are not output when the config for the
subsystem is specified solely via env vars.
2022-08-13 10:39:01 -07:00
Aditya Manthramurthy
3d94c38ec4 Add env variables to configuration APIs output (#15465)
Config export and config get APIs now include environment 
variables set on the server
2022-08-04 22:21:52 -07:00
Anis Elleuch
10b49eb4fb Fix resetting a config with a non default target name (#15448)
mc admin config reset <alias> notify_webhook:something was not working
properly.

The reason is that GetSubSys() was not calculating the target
name properly because it is quitting early when the number of config
inputs ('notify_webhook:something' in this case) is equal to 1.

This commit will make the code calculates always calculate the target
name if found.
2022-08-01 07:52:23 -07:00
Taran Pelkey
e83930333b Allow DelKVS to delete specific sub-system fields. (#15354) 2022-07-22 14:48:23 -07:00
Aditya Manthramurthy
af9bc7ea7d Add external IDP management Admin API for OpenID (#15152) 2022-07-05 18:18:04 -07:00
Harshavardhana
8856a2d77b finalize startup-banner and remove unnecessary logs (#15202) 2022-06-29 16:32:04 -07:00
Aditya Manthramurthy
7f629df4d5 Add generic function to retrieve config value with metadata (#15083)
`config.ResolveConfigParam` returns the value of a configuration for any
subsystem based on checking env, config store, and default value. Also returns info
about which config source returned the value.

This is useful to return info about config params overridden via env in the user
APIs. Currently implemented only for OpenID subsystem, but will be extended for
others subsequently.
2022-06-17 11:39:21 -07:00
Shireesh Anjal
4ce81fd07f Add periodic callhome functionality (#14918)
* Add periodic callhome functionality

Periodically (every 24hrs by default), fetch callhome information and
upload it to SUBNET.

New config keys under the `callhome` subsystem:

enable - Set to `on` for enabling callhome. Default `off`
frequency - Interval between callhome cycles. Default `24h`

* Improvements based on review comments

- Update `enableCallhome` safely
- Rename pctx to ctx
- Block during execution of callhome
- Store parsed proxy URL in global subnet config
- Store callhome URL(s) in constants
- Use existing global transport
- Pass auth token to subnetPostReq
- Use `config.EnableOn` instead of `"on"`

* Use atomic package instead of lock

* Use uber atomic package

* Use `Cancel` instead of `cancel`

Co-authored-by: Harshavardhana <harsha@minio.io>

Co-authored-by: Harshavardhana <harsha@minio.io>
Co-authored-by: Aditya Manthramurthy <donatello@users.noreply.github.com>
2022-06-06 16:14:52 -07:00
Aditya Manthramurthy
464b9d7c80 Add support for Identity Management Plugin (#14913)
- Adds an STS API `AssumeRoleWithCustomToken` that can be used to 
  authenticate via the Id. Mgmt. Plugin.
- Adds a sample identity manager plugin implementation
- Add doc for plugin and STS API
- Add an example program using go SDK for AssumeRoleWithCustomToken
2022-05-26 17:58:09 -07:00
Aditya Manthramurthy
83071a3459 Add support for Access Management Plugin (#14875)
- This change renames the OPA integration as Access Management Plugin - there is
nothing specific to OPA in the integration, it is just a webhook.

- OPA configuration is automatically migrated to Access Management Plugin and
OPA specific configuration is marked as deprecated.

- OPA doc is updated and moved.
2022-05-10 17:14:55 -07:00
Aditya Manthramurthy
0e502899a8 Add support for multiple OpenID providers with role policies (#14223)
- When using multiple providers, claim-based providers are not allowed. All
providers must use role policies.

- Update markdown config to allow `details` HTML element
2022-04-28 18:27:09 -07:00
Klaus Post
96adfaebe1 Make storage class config dynamic (#14791)
Updating the storage class is already thread safe, so we can do this safely.
2022-04-21 12:07:33 -07:00
Shireesh Anjal
3934700a08 Make audit webhook and kafka config dynamic (#14390) 2022-02-24 09:05:33 -08:00
Shireesh Anjal
94d37d05e5 Apply dynamic config at sub-system level (#14369)
Currently, when applying any dynamic config, the system reloads and
re-applies the config of all the dynamic sub-systems.

This PR refactors the code in such a way that changing config of a given
dynamic sub-system will work on only that sub-system.
2022-02-22 10:59:28 -08:00
Shireesh Anjal
28f188e3ef Make logger webhook config dynamic (#14289)
It should not be required to restart the 
server after setting the logger webhook config.
2022-02-17 11:11:15 -08:00
Shireesh Anjal
9890f579f8 Add subsystem level validation on config set (#14269)
When setting a config of a particular sub-system, validate the existing
config and notification targets of only that sub-system, so that
existing errors related to one sub-system (e.g. notification target
offline) do not result in errors for other sub-systems.
2022-02-08 10:36:41 -08:00
Shireesh Anjal
3882da6ac5 Add subnet proxy config (#14225)
Will store the HTTP(S) proxy URL to use for connecting to SUBNET.
2022-02-01 09:52:38 -08:00
Harshavardhana
f527c708f2 run gofumpt cleanup across code-base (#14015) 2022-01-02 09:15:06 -08:00
Shireesh Anjal
d29df6714a Introduce new config subnet api_key (#13793)
The earlier approach of using a license token for 
communicating with SUBNET is being replaced 
with a simpler mechanism of API keys. Unlike the 
license which is a JWT token, these API keys will 
be simple UUID tokens and don't have any embedded 
information in them. SUBNET would generate the 
API key on cluster registration, and then it would 
be saved in this config, to be used for subsequent 
communication with SUBNET.
2021-12-03 09:32:11 -08:00
Aditya Manthramurthy
4ce6d35e30 Add new site config sub-system intended to replace region (#13672)
- New sub-system has "region" and "name" fields.

- `region` subsystem is marked as deprecated, however still works, unless the
new region parameter under `site` is set - in this case, the region subsystem is
ignored. `region` subsystem is hidden from top-level help (i.e. from `mc admin
config set myminio`), but appears when specifically requested (i.e. with `mc
admin config set myminio region`).

- MINIO_REGION, MINIO_REGION_NAME are supported as legacy environment variables for server region.

- Adds MINIO_SITE_REGION as the current environment variable to configure the
server region and MINIO_SITE_NAME for the site name.
2021-11-25 13:06:25 -08:00
Shireesh Anjal
d008e90d50 Support dynamic reset of minio config (#13626)
If a given MinIO config is dynamic (can be changed without restart),
ensure that it can be reset also without restart.

Signed-off-by: Shireesh Anjal <shireesh@minio.io>
2021-11-10 10:01:32 -08:00
Shireesh Anjal
b4364723ef Add config to store subnet license (#13194)
Command to set subnet license:

`mc admin config set {alias} subnet license={token}`

Signed-off-by: Shireesh Anjal <shireesh@minio.io>
Co-authored-by: Harshavardhana <harsha@minio.io>
2021-09-14 21:54:25 -07:00
Andreas Auernhammer
e438dccf19 sts: add support for certificate-based authentication (#12748)
This commit adds a new STS API for X.509 certificate
authentication.

A client can make an HTTP POST request over a TLS connection
and MinIO will verify the provided client certificate, map it to an 
S3 policy and return temp. S3 credentials to the client.

So, this STS API allows clients to authenticate with X.509
certificates over TLS and obtain temp. S3 credentials.

For more details and examples refer to the docs/sts/tls.md
documentation.

Signed-off-by: Andreas Auernhammer <hi@aead.dev>
2021-09-07 19:03:48 -07:00
Shireesh Anjal
4d6d4244f1 Anonymize hosts in subnet health report (#12627)
Ensure that hostnames / ip addresses are not printed in the subnet
health report. Anonymize them by replacing them with `servern` where `n`
represents the position of the server in the pool.

This is done by building a `host anonymizer` map that maps every
possible value containing the host e.g. host, host:port,
http://host:port, etc to the corresponding anonymized name and using
this map to replace the values at the time of health report generation.

A different logic is used to anonymize host names in the `procinfo`
data, as the host names are part of an ellipses pattern in the process
start command. Here we just replace the prefix/suffix of the ellipses
pattern with their hashes.
2021-07-14 00:23:22 -07:00
Harshavardhana
e316873f84 feat: Add support for kakfa audit logger target (#12678) 2021-07-13 09:39:13 -07:00
Harshavardhana
8d1bc65757 allow resetting and reapply config on broken clusters (#12554)
Bonus: remove kms_kes as sub-system, since its ENV only.
- also fixes a crash with etcd cluster without KMS
configured and also if KMS decryption is missing.
2021-06-24 16:24:12 -07:00
Harshavardhana
ed6cc66cf4 fix: remove unnecessary limit for queueStore (#12491)
There is no good reason to limit ourselves
to max_open_fd for queue_store

Bonus: Support for publisher confirms
2021-06-14 13:28:44 -07:00
Shireesh Anjal
fb140c146b Redact sensitive values from config in health data (#12421)
The health api returns the server configuration details. Redact
sensitive values from the config values like URLs and credentials.
2021-06-03 08:15:44 -07:00
Harshavardhana
7a3b5235bf remove deprecated kms_vault unused key name 2021-06-03 00:10:11 -07:00
Harshavardhana
1f262daf6f rename all remaining packages to internal/ (#12418)
This is to ensure that there are no projects
that try to import `minio/minio/pkg` into
their own repo. Any such common packages should
go to `https://github.com/minio/pkg`
2021-06-01 14:59:40 -07:00