MyFavMirrors
/
minio
mirror of https://github.com/minio/minio.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
minio/cmd/admin-router.go

257 lines
16 KiB
Go
Raw Normal View History

update license change for MinIO Signed-off-by: Harshavardhana <harsha@minio.io>
2021-04-18 15:41:13 -04:00
// Copyright (c) 2015-2021 MinIO, Inc.
//
// This file is part of MinIO Object Storage stack
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
Add service API handler stubs for status, stop and restart (#3417)
2016-12-16 01:26:15 -05:00
package cmd
Move admin APIs to new path and add redesigned heal APIs (#5351) - Changes related to moving admin APIs - admin APIs now have an endpoint under /minio/admin - admin APIs are now versioned - a new API to server the version is added at "GET /minio/admin/version" and all API operations have the path prefix /minio/admin/v1/<operation> - new service stop API added - credentials change API is moved to /minio/admin/v1/config/credential - credentials change API and configuration get/set API now require TLS so that credentials are protected - all API requests now receive JSON - heal APIs are disabled as they will be changed substantially - Heal API changes Heal API is now provided at a single endpoint with the ability for a client to start a heal sequence on all the data in the server, a single bucket, or under a prefix within a bucket. When a heal sequence is started, the server returns a unique token that needs to be used for subsequent 'status' requests to fetch heal results. On each status request from the client, the server returns heal result records that it has accumulated since the previous status request. The server accumulates upto 1000 records and pauses healing further objects until the client requests for status. If the client does not request any further records for a long time, the server aborts the heal sequence automatically. A heal result record is returned for each entity healed on the server, such as system metadata, object metadata, buckets and objects, and has information about the before and after states on each disk. A client may request to force restart a heal sequence - this causes the running heal sequence to be aborted at the next safe spot and starts a new heal sequence.
2018-01-22 17:54:55 -05:00
import (
"net/http"
use package name correctly (#5827)
2018-04-21 22:23:54 -04:00
"github.com/gorilla/mux"
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-12 13:49:50 -04:00
"github.com/klauspost/compress/gzhttp"
"github.com/klauspost/compress/gzip"
move madmin to github.com/minio/madmin-go (#12239)
2021-05-06 11:52:02 -04:00
"github.com/minio/madmin-go"
Simplify gzhttp wrapper (#12912) The wrapper now accepts interfaces so we don't need a wrapper for that any more.
2021-08-09 15:45:59 -04:00
"github.com/minio/minio/internal/logger"
Move admin APIs to new path and add redesigned heal APIs (#5351) - Changes related to moving admin APIs - admin APIs now have an endpoint under /minio/admin - admin APIs are now versioned - a new API to server the version is added at "GET /minio/admin/version" and all API operations have the path prefix /minio/admin/v1/<operation> - new service stop API added - credentials change API is moved to /minio/admin/v1/config/credential - credentials change API and configuration get/set API now require TLS so that credentials are protected - all API requests now receive JSON - heal APIs are disabled as they will be changed substantially - Heal API changes Heal API is now provided at a single endpoint with the ability for a client to start a heal sequence on all the data in the server, a single bucket, or under a prefix within a bucket. When a heal sequence is started, the server returns a unique token that needs to be used for subsequent 'status' requests to fetch heal results. On each status request from the client, the server returns heal result records that it has accumulated since the previous status request. The server accumulates upto 1000 records and pauses healing further objects until the client requests for status. If the client does not request any further records for a long time, the server aborts the heal sequence automatically. A heal result record is returned for each entity healed on the server, such as system metadata, object metadata, buckets and objects, and has information about the before and after states on each disk. A client may request to force restart a heal sequence - this causes the running heal sequence to be aborted at the next safe spot and starts a new heal sequence.
2018-01-22 17:54:55 -05:00
)
const (
feat: introduce listUsers, listPolicies for any bucket (#12372) Bonus change LDAP settings such as user, group mappings are now listed as part of `mc admin user list` and `mc admin group list` Additionally this PR also deprecates the `/v2` API that is no longer in use.
2021-05-27 13:15:02 -04:00
adminPathPrefix = minioReservedBucketPath + "/admin"
adminAPIVersion = madmin.AdminAPIVersion
adminAPIVersionPrefix = SlashSeparator + adminAPIVersion
Move admin APIs to new path and add redesigned heal APIs (#5351) - Changes related to moving admin APIs - admin APIs now have an endpoint under /minio/admin - admin APIs are now versioned - a new API to server the version is added at "GET /minio/admin/version" and all API operations have the path prefix /minio/admin/v1/<operation> - new service stop API added - credentials change API is moved to /minio/admin/v1/config/credential - credentials change API and configuration get/set API now require TLS so that credentials are protected - all API requests now receive JSON - heal APIs are disabled as they will be changed substantially - Heal API changes Heal API is now provided at a single endpoint with the ability for a client to start a heal sequence on all the data in the server, a single bucket, or under a prefix within a bucket. When a heal sequence is started, the server returns a unique token that needs to be used for subsequent 'status' requests to fetch heal results. On each status request from the client, the server returns heal result records that it has accumulated since the previous status request. The server accumulates upto 1000 records and pauses healing further objects until the client requests for status. If the client does not request any further records for a long time, the server aborts the heal sequence automatically. A heal result record is returned for each entity healed on the server, such as system metadata, object metadata, buckets and objects, and has information about the before and after states on each disk. A client may request to force restart a heal sequence - this causes the running heal sequence to be aborted at the next safe spot and starts a new heal sequence.
2018-01-22 17:54:55 -05:00
)
Add service API handler stubs for status, stop and restart (#3417)
2016-12-16 01:26:15 -05:00
Replace Minio refs in docs with MinIO and links (#7494)
2019-04-09 14:39:42 -04:00
// adminAPIHandlers provides HTTP handlers for MinIO admin API.
Return appropriate errors API versions changes across REST APIs (#8480) This PR adds code to appropriately handle versioning issues that come up quite constantly across our API changes. Currently we were also routing our requests wrong which sort of made it harder to write a consistent error handling code to appropriately reject or honor requests. This PR potentially fixes issues - old mc is used against new minio release which is incompatible returns an appropriate for client action. - any older servers talking to each other, report appropriate error - incompatible peer servers should report error and reject the calls with appropriate error
2019-11-04 12:30:59 -05:00
type adminAPIHandlers struct{}
Add service API handler stubs for status, stop and restart (#3417)
2016-12-16 01:26:15 -05:00
// registerAdminRouter - Add handler functions for each service REST API routes.
fix: add IAM dummy store for gateway operations (#12670) with console addition users cannot login with root credentials without etcd persistent layer, allow a dummy store such that such functionalities can be supported when running as non-persistent manner, this enables all calls and operations.
2021-07-10 11:32:52 -04:00
func registerAdminRouter(router *mux.Router, enableConfigOps bool) {
Add service API handler stubs for status, stop and restart (#3417)
2016-12-16 01:26:15 -05:00
adminAPI := adminAPIHandlers{}
// Admin router
Return appropriate errors API versions changes across REST APIs (#8480) This PR adds code to appropriately handle versioning issues that come up quite constantly across our API changes. Currently we were also routing our requests wrong which sort of made it harder to write a consistent error handling code to appropriately reject or honor requests. This PR potentially fixes issues - old mc is used against new minio release which is incompatible returns an appropriate for client action. - any older servers talking to each other, report appropriate error - incompatible peer servers should report error and reject the calls with appropriate error
2019-11-04 12:30:59 -05:00
adminRouter := router.PathPrefix(adminPathPrefix).Subrouter()
Move admin APIs to new path and add redesigned heal APIs (#5351) - Changes related to moving admin APIs - admin APIs now have an endpoint under /minio/admin - admin APIs are now versioned - a new API to server the version is added at "GET /minio/admin/version" and all API operations have the path prefix /minio/admin/v1/<operation> - new service stop API added - credentials change API is moved to /minio/admin/v1/config/credential - credentials change API and configuration get/set API now require TLS so that credentials are protected - all API requests now receive JSON - heal APIs are disabled as they will be changed substantially - Heal API changes Heal API is now provided at a single endpoint with the ability for a client to start a heal sequence on all the data in the server, a single bucket, or under a prefix within a bucket. When a heal sequence is started, the server returns a unique token that needs to be used for subsequent 'status' requests to fetch heal results. On each status request from the client, the server returns heal result records that it has accumulated since the previous status request. The server accumulates upto 1000 records and pauses healing further objects until the client requests for status. If the client does not request any further records for a long time, the server aborts the heal sequence automatically. A heal result record is returned for each entity healed on the server, such as system metadata, object metadata, buckets and objects, and has information about the before and after states on each disk. A client may request to force restart a heal sequence - this causes the running heal sequence to be aborted at the next safe spot and starts a new heal sequence.
2018-01-22 17:54:55 -05:00
fix: change policies API to return and take struct (#9181) This allows for order guarantees in returned values can be consumed safely by the caller to avoid any additional parsing and validation. Fixes #9171
2020-04-07 22:30:59 -04:00
adminVersions := []string{
adminAPIVersionPrefix,
}
Implement list, clear locks REST API w/ pkg/madmin support (#3491) * Filter lock info based on bucket, prefix and time since lock was held * Implement list and clear locks REST API * madmin: Add list and clear locks API * locks: Clear locks matching bucket, prefix, relTime. * Gather lock information across nodes for both list and clear locks admin REST API. * docs: Add lock API to management APIs
2017-01-04 02:39:22 -05:00
Simplify gzhttp wrapper (#12912) The wrapper now accepts interfaces so we don't need a wrapper for that any more.
2021-08-09 15:45:59 -04:00
gz, err := gzhttp.NewWrapper(gzhttp.MinSize(1000), gzhttp.CompressionLevel(gzip.BestSpeed))
if err != nil {
// Static params, so this is very unlikely.
logger.Fatal(err, "Unable to initialize server")
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-12 13:49:50 -04:00
}
fix: change policies API to return and take struct (#9181) This allows for order guarantees in returned values can be consumed safely by the caller to avoid any additional parsing and validation. Fixes #9171
2020-04-07 22:30:59 -04:00
for _, adminVersion := range adminVersions {
// Restart and stop MinIO service.
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-12 13:49:50 -04:00
adminRouter.Methods(http.MethodPost).Path(adminVersion+"/service").HandlerFunc(gz(httpTraceAll(adminAPI.ServiceHandler))).Queries("action", "{action:.*}")
fix: change policies API to return and take struct (#9181) This allows for order guarantees in returned values can be consumed safely by the caller to avoid any additional parsing and validation. Fixes #9171
2020-04-07 22:30:59 -04:00
// Update MinIO servers.
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-12 13:49:50 -04:00
adminRouter.Methods(http.MethodPost).Path(adminVersion+"/update").HandlerFunc(gz(httpTraceAll(adminAPI.ServerUpdateHandler))).Queries("updateURL", "{updateURL:.*}")
The prometheus metrics refractoring (#8003) The measures are consolidated to the following metrics - `disk_storage_used` : Disk space used by the disk. - `disk_storage_available`: Available disk space left on the disk. - `disk_storage_total`: Total disk space on the disk. - `disks_offline`: Total number of offline disks in current MinIO instance. - `disks_total`: Total number of disks in current MinIO instance. - `s3_requests_total`: Total number of s3 requests in current MinIO instance. - `s3_errors_total`: Total number of errors in s3 requests in current MinIO instance. - `s3_requests_current`: Total number of active s3 requests in current MinIO instance. - `internode_rx_bytes_total`: Total number of internode bytes received by current MinIO server instance. - `internode_tx_bytes_total`: Total number of bytes sent to the other nodes by current MinIO server instance. - `s3_rx_bytes_total`: Total number of s3 bytes received by current MinIO server instance. - `s3_tx_bytes_total`: Total number of s3 bytes sent by current MinIO server instance. - `minio_version_info`: Current MinIO version with commit-id. - `s3_ttfb_seconds_bucket`: Histogram that holds the latency information of the requests. And this PR also modifies the current StorageInfo queries - Decouples StorageInfo from ServerInfo . - StorageInfo is enhanced to give endpoint information. NOTE: ADMIN API VERSION IS BUMPED UP IN THIS PR Fixes #7873
2019-10-23 00:01:14 -04:00
fix: change policies API to return and take struct (#9181) This allows for order guarantees in returned values can be consumed safely by the caller to avoid any additional parsing and validation. Fixes #9171
2020-04-07 22:30:59 -04:00
// Info operations
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-12 13:49:50 -04:00
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/info").HandlerFunc(gz(httpTraceAll(adminAPI.ServerInfoHandler)))
Add admin file inspector (#12635) Download files from *any* bucket/path as an encrypted zip file. The key is included in the response but can be separated so zip and the key doesn't have to be sent on the same channel. Requires https://github.com/minio/pkg/pull/6
2021-07-09 14:29:16 -04:00
adminRouter.Methods(http.MethodGet).Path(adminVersion+"/inspect-data").HandlerFunc(httpTraceHdrs(adminAPI.InspectDataHandler)).Queries("volume", "{volume:.*}", "file", "{file:.*}")
admin: Add ServerInfo API() (#3743)
2017-02-15 13:45:45 -05:00
fix: change policies API to return and take struct (#9181) This allows for order guarantees in returned values can be consumed safely by the caller to avoid any additional parsing and validation. Fixes #9171
2020-04-07 22:30:59 -04:00
// StorageInfo operations
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-12 13:49:50 -04:00
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/storageinfo").HandlerFunc(gz(httpTraceAll(adminAPI.StorageInfoHandler)))
fix: change policies API to return and take struct (#9181) This allows for order guarantees in returned values can be consumed safely by the caller to avoid any additional parsing and validation. Fixes #9171
2020-04-07 22:30:59 -04:00
// DataUsageInfo operations
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-12 13:49:50 -04:00
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/datausageinfo").HandlerFunc(gz(httpTraceAll(adminAPI.DataUsageInfoHandler)))
Add new admin API to return Accounting Usage (#8689)
2020-02-04 21:20:39 -05:00
Support bucket versioning (#9377) - Implement a new xl.json 2.0.0 format to support, this moves the entire marshaling logic to POSIX layer, top layer always consumes a common FileInfo construct which simplifies the metadata reads. - Implement list object versions - Migrate to siphash from crchash for new deployments for object placements. Fixes #2111
2020-06-12 23:04:01 -04:00
if globalIsDistErasure || globalIsErasure {
add gocritic/ruleguard checks back again, cleanup code. (#13665) - remove some duplicated code - reported a bug, separately fixed in #13664 - using strings.ReplaceAll() when needed - using filepath.ToSlash() use when needed - remove all non-Go style comments from the codebase Co-authored-by: Aditya Manthramurthy <donatello@users.noreply.github.com>
2021-11-16 12:28:29 -05:00
// Heal operations
Add ServerDrivesPerfInfo() admin API (#6969) This is part of implementation for mc admin health command. The ServerDrivesPerfInfo() admin API returns read and write speed information for all the drives (local and remote) in a given Minio server deployment. Part of minio/mc#2606
2018-12-31 12:46:44 -05:00
fix: change policies API to return and take struct (#9181) This allows for order guarantees in returned values can be consumed safely by the caller to avoid any additional parsing and validation. Fixes #9171
2020-04-07 22:30:59 -04:00
// Heal processing endpoint.
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-12 13:49:50 -04:00
adminRouter.Methods(http.MethodPost).Path(adminVersion + "/heal/").HandlerFunc(gz(httpTraceAll(adminAPI.HealHandler)))
adminRouter.Methods(http.MethodPost).Path(adminVersion + "/heal/{bucket}").HandlerFunc(gz(httpTraceAll(adminAPI.HealHandler)))
adminRouter.Methods(http.MethodPost).Path(adminVersion + "/heal/{bucket}/{prefix:.*}").HandlerFunc(gz(httpTraceAll(adminAPI.HealHandler)))
adminRouter.Methods(http.MethodPost).Path(adminVersion + "/background-heal/status").HandlerFunc(gz(httpTraceAll(adminAPI.BackgroundHealStatusHandler)))
feat: decommission feature for pools (#14012) ``` λ mc admin decommission start alias/ http://minio{1...2}/data{1...4} ``` ``` λ mc admin decommission status alias/ ┌─────┬─────────────────────────────────┬──────────────────────────────────┬────────┐ │ ID │ Pools │ Capacity │ Status │ │ 1st │ http://minio{1...2}/data{1...4} │ 439 GiB (used) / 561 GiB (total) │ Active │ │ 2nd │ http://minio{3...4}/data{1...4} │ 329 GiB (used) / 421 GiB (total) │ Active │ └─────┴─────────────────────────────────┴──────────────────────────────────┴────────┘ ``` ``` λ mc admin decommission status alias/ http://minio{1...2}/data{1...4} Progress: ===================> [1GiB/sec] [15%] [4TiB/50TiB] Time Remaining: 4 hours (started 3 hours ago) ``` ``` λ mc admin decommission status alias/ http://minio{1...2}/data{1...4} ERROR: This pool is not scheduled for decommissioning currently. ``` ``` λ mc admin decommission cancel alias/ ┌─────┬─────────────────────────────────┬──────────────────────────────────┬──────────┐ │ ID │ Pools │ Capacity │ Status │ │ 1st │ http://minio{1...2}/data{1...4} │ 439 GiB (used) / 561 GiB (total) │ Draining │ └─────┴─────────────────────────────────┴──────────────────────────────────┴──────────┘ ``` > NOTE: Canceled decommission will not make the pool active again, since we might have > Potentially partial duplicate content on the other pools, to avoid this scenario be > very sure to start decommissioning as a planned activity. ``` λ mc admin decommission cancel alias/ http://minio{1...2}/data{1...4} ┌─────┬─────────────────────────────────┬──────────────────────────────────┬────────────────────┐ │ ID │ Pools │ Capacity │ Status │ │ 1st │ http://minio{1...2}/data{1...4} │ 439 GiB (used) / 561 GiB (total) │ Draining(Canceled) │ └─────┴─────────────────────────────────┴──────────────────────────────────┴────────────────────┘ ```
2022-01-10 12:07:49 -05:00
// Pool operations
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/pools/list").HandlerFunc(gz(httpTraceAll(adminAPI.ListPools)))
adminRouter.Methods(http.MethodGet).Path(adminVersion+"/pools/status").HandlerFunc(gz(httpTraceAll(adminAPI.StatusPool))).Queries("pool", "{pool:.*}")
adminRouter.Methods(http.MethodPost).Path(adminVersion+"/pools/decommission").HandlerFunc(gz(httpTraceAll(adminAPI.StartDecommission))).Queries("pool", "{pool:.*}")
adminRouter.Methods(http.MethodPost).Path(adminVersion+"/pools/cancel").HandlerFunc(gz(httpTraceAll(adminAPI.CancelDecommission))).Queries("pool", "{pool:.*}")
fix: change policies API to return and take struct (#9181) This allows for order guarantees in returned values can be consumed safely by the caller to avoid any additional parsing and validation. Fixes #9171
2020-04-07 22:30:59 -04:00
}
Introduce STS client grants API and OPA policy integration (#6168) This PR introduces two new features - AWS STS compatible STS API named AssumeRoleWithClientGrants ``` POST /?Action=AssumeRoleWithClientGrants&Token=<jwt> ``` This API endpoint returns temporary access credentials, access tokens signature types supported by this API - RSA keys - ECDSA keys Fetches the required public key from the JWKS endpoints, provides them as rsa or ecdsa public keys. - External policy engine support, in this case OPA policy engine - Credentials are stored on disks
2018-10-09 17:00:01 -04:00
fix: change policies API to return and take struct (#9181) This allows for order guarantees in returned values can be consumed safely by the caller to avoid any additional parsing and validation. Fixes #9171
2020-04-07 22:30:59 -04:00
// Profiling operations
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-12 13:49:50 -04:00
adminRouter.Methods(http.MethodPost).Path(adminVersion+"/profiling/start").HandlerFunc(gz(httpTraceAll(adminAPI.StartProfilingHandler))).
fix: change policies API to return and take struct (#9181) This allows for order guarantees in returned values can be consumed safely by the caller to avoid any additional parsing and validation. Fixes #9171
2020-04-07 22:30:59 -04:00
Queries("profilerType", "{profilerType:.*}")
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-12 13:49:50 -04:00
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/profiling/download").HandlerFunc(gz(httpTraceAll(adminAPI.DownloadProfilingHandler)))
Add ListUsers API to list all configured users in IAM (#6619)
2018-10-13 03:18:43 -04:00
fix: change policies API to return and take struct (#9181) This allows for order guarantees in returned values can be consumed safely by the caller to avoid any additional parsing and validation. Fixes #9171
2020-04-07 22:30:59 -04:00
// Config KV operations.
if enableConfigOps {
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-12 13:49:50 -04:00
adminRouter.Methods(http.MethodGet).Path(adminVersion+"/get-config-kv").HandlerFunc(gz(httpTraceHdrs(adminAPI.GetConfigKVHandler))).Queries("key", "{key:.*}")
adminRouter.Methods(http.MethodPut).Path(adminVersion + "/set-config-kv").HandlerFunc(gz(httpTraceHdrs(adminAPI.SetConfigKVHandler)))
adminRouter.Methods(http.MethodDelete).Path(adminVersion + "/del-config-kv").HandlerFunc(gz(httpTraceHdrs(adminAPI.DelConfigKVHandler)))
allow config help in gateway mode (#9356) allow `mc admin config set mygateway/ audit_webhook --env` to fetch the documentation as needed, this is just to ensure that our users can still access the relevant ENV docs while running in gateway mode.
2020-04-16 17:49:12 -04:00
}
// Enable config help in all modes.
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-12 13:49:50 -04:00
adminRouter.Methods(http.MethodGet).Path(adminVersion+"/help-config-kv").HandlerFunc(gz(httpTraceAll(adminAPI.HelpConfigKVHandler))).Queries("subSys", "{subSys:.*}", "key", "{key:.*}")
allow config help in gateway mode (#9356) allow `mc admin config set mygateway/ audit_webhook --env` to fetch the documentation as needed, this is just to ensure that our users can still access the relevant ENV docs while running in gateway mode.
2020-04-16 17:49:12 -04:00
// Config KV history operations.
if enableConfigOps {
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-12 13:49:50 -04:00
adminRouter.Methods(http.MethodGet).Path(adminVersion+"/list-config-history-kv").HandlerFunc(gz(httpTraceAll(adminAPI.ListConfigHistoryKVHandler))).Queries("count", "{count:[0-9]+}")
adminRouter.Methods(http.MethodDelete).Path(adminVersion+"/clear-config-history-kv").HandlerFunc(gz(httpTraceHdrs(adminAPI.ClearConfigHistoryKVHandler))).Queries("restoreId", "{restoreId:.*}")
adminRouter.Methods(http.MethodPut).Path(adminVersion+"/restore-config-history-kv").HandlerFunc(gz(httpTraceHdrs(adminAPI.RestoreConfigHistoryKVHandler))).Queries("restoreId", "{restoreId:.*}")
fix: change policies API to return and take struct (#9181) This allows for order guarantees in returned values can be consumed safely by the caller to avoid any additional parsing and validation. Fixes #9171
2020-04-07 22:30:59 -04:00
}
Add canned policy support (#6637) This PR adds an additional API where we can create a new set of canned policies which can be used with one or many users.
2018-10-16 15:48:19 -04:00
add gocritic/ruleguard checks back again, cleanup code. (#13665) - remove some duplicated code - reported a bug, separately fixed in #13664 - using strings.ReplaceAll() when needed - using filepath.ToSlash() use when needed - remove all non-Go style comments from the codebase Co-authored-by: Aditya Manthramurthy <donatello@users.noreply.github.com>
2021-11-16 12:28:29 -05:00
// Config import/export bulk operations
fix: change policies API to return and take struct (#9181) This allows for order guarantees in returned values can be consumed safely by the caller to avoid any additional parsing and validation. Fixes #9171
2020-04-07 22:30:59 -04:00
if enableConfigOps {
// Get config
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-12 13:49:50 -04:00
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/config").HandlerFunc(gz(httpTraceHdrs(adminAPI.GetConfigHandler)))
fix: change policies API to return and take struct (#9181) This allows for order guarantees in returned values can be consumed safely by the caller to avoid any additional parsing and validation. Fixes #9171
2020-04-07 22:30:59 -04:00
// Set config
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-12 13:49:50 -04:00
adminRouter.Methods(http.MethodPut).Path(adminVersion + "/config").HandlerFunc(gz(httpTraceHdrs(adminAPI.SetConfigHandler)))
fix: change policies API to return and take struct (#9181) This allows for order guarantees in returned values can be consumed safely by the caller to avoid any additional parsing and validation. Fixes #9171
2020-04-07 22:30:59 -04:00
}
Add canned policy support (#6637) This PR adds an additional API where we can create a new set of canned policies which can be used with one or many users.
2018-10-16 15:48:19 -04:00
fix: add IAM dummy store for gateway operations (#12670) with console addition users cannot login with root credentials without etcd persistent layer, allow a dummy store such that such functionalities can be supported when running as non-persistent manner, this enables all calls and operations.
2021-07-10 11:32:52 -04:00
// -- IAM APIs --
Add service account type in IAM (#9029)
2020-03-17 13:36:13 -04:00
fix: add IAM dummy store for gateway operations (#12670) with console addition users cannot login with root credentials without etcd persistent layer, allow a dummy store such that such functionalities can be supported when running as non-persistent manner, this enables all calls and operations.
2021-07-10 11:32:52 -04:00
// Add policy IAM
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-12 13:49:50 -04:00
adminRouter.Methods(http.MethodPut).Path(adminVersion+"/add-canned-policy").HandlerFunc(gz(httpTraceAll(adminAPI.AddCannedPolicy))).Queries("name", "{name:.*}")
Add InfoCannedPolicy API to fetch only necessary policy (#8307) This PR adds - InfoCannedPolicy() API for efficiency in fetching policies - Send group memberships for LDAPUser if available
2019-09-26 14:23:13 -04:00
fix: add IAM dummy store for gateway operations (#12670) with console addition users cannot login with root credentials without etcd persistent layer, allow a dummy store such that such functionalities can be supported when running as non-persistent manner, this enables all calls and operations.
2021-07-10 11:32:52 -04:00
// Add user IAM
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-12 13:49:50 -04:00
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/accountinfo").HandlerFunc(gz(httpTraceAll(adminAPI.AccountInfoHandler)))
add missing admin actions, enhance AccountUsageInfo (#9607)
2020-05-15 21:16:45 -04:00
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-12 13:49:50 -04:00
adminRouter.Methods(http.MethodPut).Path(adminVersion+"/add-user").HandlerFunc(gz(httpTraceHdrs(adminAPI.AddUser))).Queries("accessKey", "{accessKey:.*}")
Introduce STS client grants API and OPA policy integration (#6168) This PR introduces two new features - AWS STS compatible STS API named AssumeRoleWithClientGrants ``` POST /?Action=AssumeRoleWithClientGrants&Token=<jwt> ``` This API endpoint returns temporary access credentials, access tokens signature types supported by this API - RSA keys - ECDSA keys Fetches the required public key from the JWKS endpoints, provides them as rsa or ecdsa public keys. - External policy engine support, in this case OPA policy engine - Credentials are stored on disks
2018-10-09 17:00:01 -04:00
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-12 13:49:50 -04:00
adminRouter.Methods(http.MethodPut).Path(adminVersion+"/set-user-status").HandlerFunc(gz(httpTraceHdrs(adminAPI.SetUserStatus))).Queries("accessKey", "{accessKey:.*}").Queries("status", "{status:.*}")
Set the policy mapping for a user or group (#8036) Add API to set policy mapping for a user or group Contains a breaking Admin APIs change. - Also enforce all applicable policies - Removes the previous /set-user-policy API Bump up peerRESTVersion Add get user info API to show groups of a user
2019-08-13 16:41:06 -04:00
fix: add IAM dummy store for gateway operations (#12670) with console addition users cannot login with root credentials without etcd persistent layer, allow a dummy store such that such functionalities can be supported when running as non-persistent manner, this enables all calls and operations.
2021-07-10 11:32:52 -04:00
// Service accounts ops
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-12 13:49:50 -04:00
adminRouter.Methods(http.MethodPut).Path(adminVersion + "/add-service-account").HandlerFunc(gz(httpTraceHdrs(adminAPI.AddServiceAccount)))
adminRouter.Methods(http.MethodPost).Path(adminVersion+"/update-service-account").HandlerFunc(gz(httpTraceHdrs(adminAPI.UpdateServiceAccount))).Queries("accessKey", "{accessKey:.*}")
adminRouter.Methods(http.MethodGet).Path(adminVersion+"/info-service-account").HandlerFunc(gz(httpTraceHdrs(adminAPI.InfoServiceAccount))).Queries("accessKey", "{accessKey:.*}")
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/list-service-accounts").HandlerFunc(gz(httpTraceHdrs(adminAPI.ListServiceAccounts)))
adminRouter.Methods(http.MethodDelete).Path(adminVersion+"/delete-service-account").HandlerFunc(gz(httpTraceHdrs(adminAPI.DeleteServiceAccount))).Queries("accessKey", "{accessKey:.*}")
Introduce STS client grants API and OPA policy integration (#6168) This PR introduces two new features - AWS STS compatible STS API named AssumeRoleWithClientGrants ``` POST /?Action=AssumeRoleWithClientGrants&Token=<jwt> ``` This API endpoint returns temporary access credentials, access tokens signature types supported by this API - RSA keys - ECDSA keys Fetches the required public key from the JWKS endpoints, provides them as rsa or ecdsa public keys. - External policy engine support, in this case OPA policy engine - Credentials are stored on disks
2018-10-09 17:00:01 -04:00
fix: add IAM dummy store for gateway operations (#12670) with console addition users cannot login with root credentials without etcd persistent layer, allow a dummy store such that such functionalities can be supported when running as non-persistent manner, this enables all calls and operations.
2021-07-10 11:32:52 -04:00
// Info policy IAM latest
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-12 13:49:50 -04:00
adminRouter.Methods(http.MethodGet).Path(adminVersion+"/info-canned-policy").HandlerFunc(gz(httpTraceHdrs(adminAPI.InfoCannedPolicy))).Queries("name", "{name:.*}")
fix: add IAM dummy store for gateway operations (#12670) with console addition users cannot login with root credentials without etcd persistent layer, allow a dummy store such that such functionalities can be supported when running as non-persistent manner, this enables all calls and operations.
2021-07-10 11:32:52 -04:00
// List policies latest
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-12 13:49:50 -04:00
adminRouter.Methods(http.MethodGet).Path(adminVersion+"/list-canned-policies").HandlerFunc(gz(httpTraceHdrs(adminAPI.ListBucketPolicies))).Queries("bucket", "{bucket:.*}")
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/list-canned-policies").HandlerFunc(gz(httpTraceHdrs(adminAPI.ListCannedPolicies)))
Add IAM groups support (#7981) This change adds admin APIs and IAM subsystem APIs to: - add or remove members to a group (group addition and deletion is implicit on add and remove) - enable/disable a group - list and fetch group info
2019-08-02 17:25:00 -04:00
fix: add IAM dummy store for gateway operations (#12670) with console addition users cannot login with root credentials without etcd persistent layer, allow a dummy store such that such functionalities can be supported when running as non-persistent manner, this enables all calls and operations.
2021-07-10 11:32:52 -04:00
// Remove policy IAM
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-12 13:49:50 -04:00
adminRouter.Methods(http.MethodDelete).Path(adminVersion+"/remove-canned-policy").HandlerFunc(gz(httpTraceHdrs(adminAPI.RemoveCannedPolicy))).Queries("name", "{name:.*}")
Add IAM groups support (#7981) This change adds admin APIs and IAM subsystem APIs to: - add or remove members to a group (group addition and deletion is implicit on add and remove) - enable/disable a group - list and fetch group info
2019-08-02 17:25:00 -04:00
fix: add IAM dummy store for gateway operations (#12670) with console addition users cannot login with root credentials without etcd persistent layer, allow a dummy store such that such functionalities can be supported when running as non-persistent manner, this enables all calls and operations.
2021-07-10 11:32:52 -04:00
// Set user or group policy
adminRouter.Methods(http.MethodPut).Path(adminVersion+"/set-user-or-group-policy").
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-12 13:49:50 -04:00
HandlerFunc(gz(httpTraceHdrs(adminAPI.SetPolicyForUserOrGroup))).
fix: add IAM dummy store for gateway operations (#12670) with console addition users cannot login with root credentials without etcd persistent layer, allow a dummy store such that such functionalities can be supported when running as non-persistent manner, this enables all calls and operations.
2021-07-10 11:32:52 -04:00
Queries("policyName", "{policyName:.*}", "userOrGroup", "{userOrGroup:.*}", "isGroup", "{isGroup:true|false}")
Add IAM groups support (#7981) This change adds admin APIs and IAM subsystem APIs to: - add or remove members to a group (group addition and deletion is implicit on add and remove) - enable/disable a group - list and fetch group info
2019-08-02 17:25:00 -04:00
fix: add IAM dummy store for gateway operations (#12670) with console addition users cannot login with root credentials without etcd persistent layer, allow a dummy store such that such functionalities can be supported when running as non-persistent manner, this enables all calls and operations.
2021-07-10 11:32:52 -04:00
// Remove user IAM
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-12 13:49:50 -04:00
adminRouter.Methods(http.MethodDelete).Path(adminVersion+"/remove-user").HandlerFunc(gz(httpTraceHdrs(adminAPI.RemoveUser))).Queries("accessKey", "{accessKey:.*}")
Add IAM groups support (#7981) This change adds admin APIs and IAM subsystem APIs to: - add or remove members to a group (group addition and deletion is implicit on add and remove) - enable/disable a group - list and fetch group info
2019-08-02 17:25:00 -04:00
fix: add IAM dummy store for gateway operations (#12670) with console addition users cannot login with root credentials without etcd persistent layer, allow a dummy store such that such functionalities can be supported when running as non-persistent manner, this enables all calls and operations.
2021-07-10 11:32:52 -04:00
// List users
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-12 13:49:50 -04:00
adminRouter.Methods(http.MethodGet).Path(adminVersion+"/list-users").HandlerFunc(gz(httpTraceHdrs(adminAPI.ListBucketUsers))).Queries("bucket", "{bucket:.*}")
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/list-users").HandlerFunc(gz(httpTraceHdrs(adminAPI.ListUsers)))
Enable admin users API on gateway (#6659) This is only enabled when etcd is enabled, healing is only enabled for erasure coded backend.
2018-10-17 20:25:16 -04:00
fix: add IAM dummy store for gateway operations (#12670) with console addition users cannot login with root credentials without etcd persistent layer, allow a dummy store such that such functionalities can be supported when running as non-persistent manner, this enables all calls and operations.
2021-07-10 11:32:52 -04:00
// User info
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-12 13:49:50 -04:00
adminRouter.Methods(http.MethodGet).Path(adminVersion+"/user-info").HandlerFunc(gz(httpTraceHdrs(adminAPI.GetUserInfo))).Queries("accessKey", "{accessKey:.*}")
fix: add IAM dummy store for gateway operations (#12670) with console addition users cannot login with root credentials without etcd persistent layer, allow a dummy store such that such functionalities can be supported when running as non-persistent manner, this enables all calls and operations.
2021-07-10 11:32:52 -04:00
// Add/Remove members from group
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-12 13:49:50 -04:00
adminRouter.Methods(http.MethodPut).Path(adminVersion + "/update-group-members").HandlerFunc(gz(httpTraceHdrs(adminAPI.UpdateGroupMembers)))
fix: change policies API to return and take struct (#9181) This allows for order guarantees in returned values can be consumed safely by the caller to avoid any additional parsing and validation. Fixes #9171
2020-04-07 22:30:59 -04:00
fix: add IAM dummy store for gateway operations (#12670) with console addition users cannot login with root credentials without etcd persistent layer, allow a dummy store such that such functionalities can be supported when running as non-persistent manner, this enables all calls and operations.
2021-07-10 11:32:52 -04:00
// Get Group
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-12 13:49:50 -04:00
adminRouter.Methods(http.MethodGet).Path(adminVersion+"/group").HandlerFunc(gz(httpTraceHdrs(adminAPI.GetGroup))).Queries("group", "{group:.*}")
fix: change policies API to return and take struct (#9181) This allows for order guarantees in returned values can be consumed safely by the caller to avoid any additional parsing and validation. Fixes #9171
2020-04-07 22:30:59 -04:00
fix: add IAM dummy store for gateway operations (#12670) with console addition users cannot login with root credentials without etcd persistent layer, allow a dummy store such that such functionalities can be supported when running as non-persistent manner, this enables all calls and operations.
2021-07-10 11:32:52 -04:00
// List Groups
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-12 13:49:50 -04:00
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/groups").HandlerFunc(gz(httpTraceHdrs(adminAPI.ListGroups)))
fix: change policies API to return and take struct (#9181) This allows for order guarantees in returned values can be consumed safely by the caller to avoid any additional parsing and validation. Fixes #9171
2020-04-07 22:30:59 -04:00
fix: add IAM dummy store for gateway operations (#12670) with console addition users cannot login with root credentials without etcd persistent layer, allow a dummy store such that such functionalities can be supported when running as non-persistent manner, this enables all calls and operations.
2021-07-10 11:32:52 -04:00
// Set Group Status
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-12 13:49:50 -04:00
adminRouter.Methods(http.MethodPut).Path(adminVersion+"/set-group-status").HandlerFunc(gz(httpTraceHdrs(adminAPI.SetGroupStatus))).Queries("group", "{group:.*}").Queries("status", "{status:.*}")
fix: change policies API to return and take struct (#9181) This allows for order guarantees in returned values can be consumed safely by the caller to avoid any additional parsing and validation. Fixes #9171
2020-04-07 22:30:59 -04:00
Support bucket versioning (#9377) - Implement a new xl.json 2.0.0 format to support, this moves the entire marshaling logic to POSIX layer, top layer always consumes a common FileInfo construct which simplifies the metadata reads. - Implement list object versions - Migrate to siphash from crchash for new deployments for object placements. Fixes #2111
2020-06-12 23:04:01 -04:00
if globalIsDistErasure || globalIsErasure {
remove/deprecate crawler disable environment (#11214) with changes present to automatically throttle crawler at runtime, there is no need to have an environment value to disable crawling. crawling is a fundamental piece for healing, lifecycle and many other features there is no good reason anyone would need to disable this on a production system. * Apply suggestions from code review
2021-01-04 12:43:31 -05:00
// GetBucketQuotaConfig
adminRouter.Methods(http.MethodGet).Path(adminVersion+"/get-bucket-quota").HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-12 13:49:50 -04:00
gz(httpTraceHdrs(adminAPI.GetBucketQuotaConfigHandler))).Queries("bucket", "{bucket:.*}")
remove/deprecate crawler disable environment (#11214) with changes present to automatically throttle crawler at runtime, there is no need to have an environment value to disable crawling. crawling is a fundamental piece for healing, lifecycle and many other features there is no good reason anyone would need to disable this on a production system. * Apply suggestions from code review
2021-01-04 12:43:31 -05:00
// PutBucketQuotaConfig
adminRouter.Methods(http.MethodPut).Path(adminVersion+"/set-bucket-quota").HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-12 13:49:50 -04:00
gz(httpTraceHdrs(adminAPI.PutBucketQuotaConfigHandler))).Queries("bucket", "{bucket:.*}")
remove/deprecate crawler disable environment (#11214) with changes present to automatically throttle crawler at runtime, there is no need to have an environment value to disable crawling. crawling is a fundamental piece for healing, lifecycle and many other features there is no good reason anyone would need to disable this on a production system. * Apply suggestions from code review
2021-01-04 12:43:31 -05:00
// Bucket replication operations
// GetBucketTargetHandler
adminRouter.Methods(http.MethodGet).Path(adminVersion+"/list-remote-targets").HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-12 13:49:50 -04:00
gz(httpTraceHdrs(adminAPI.ListRemoteTargetsHandler))).Queries("bucket", "{bucket:.*}", "type", "{type:.*}")
remove/deprecate crawler disable environment (#11214) with changes present to automatically throttle crawler at runtime, there is no need to have an environment value to disable crawling. crawling is a fundamental piece for healing, lifecycle and many other features there is no good reason anyone would need to disable this on a production system. * Apply suggestions from code review
2021-01-04 12:43:31 -05:00
// SetRemoteTargetHandler
adminRouter.Methods(http.MethodPut).Path(adminVersion+"/set-remote-target").HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-12 13:49:50 -04:00
gz(httpTraceHdrs(adminAPI.SetRemoteTargetHandler))).Queries("bucket", "{bucket:.*}")
remove/deprecate crawler disable environment (#11214) with changes present to automatically throttle crawler at runtime, there is no need to have an environment value to disable crawling. crawling is a fundamental piece for healing, lifecycle and many other features there is no good reason anyone would need to disable this on a production system. * Apply suggestions from code review
2021-01-04 12:43:31 -05:00
// RemoveRemoteTargetHandler
adminRouter.Methods(http.MethodDelete).Path(adminVersion+"/remove-remote-target").HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-12 13:49:50 -04:00
gz(httpTraceHdrs(adminAPI.RemoveRemoteTargetHandler))).Queries("bucket", "{bucket:.*}", "arn", "{arn:.*}")
Support for remote tier management (#12090) With this change, MinIO's ILM supports transitioning objects to a remote tier. This change includes support for Azure Blob Storage, AWS S3 compatible object storage incl. MinIO and Google Cloud Storage as remote tier storage backends. Some new additions include: - Admin APIs remote tier configuration management - Simple journal to track remote objects to be 'collected' This is used by object API handlers which 'mutate' object versions by overwriting/replacing content (Put/CopyObject) or removing the version itself (e.g DeleteObjectVersion). - Rework of previous ILM transition to fit the new model In the new model, a storage class (a.k.a remote tier) is defined by the 'remote' object storage type (one of s3, azure, GCS), bucket name and a prefix. * Fixed bugs, review comments, and more unit-tests - Leverage inline small object feature - Migrate legacy objects to the latest object format before transitioning - Fix restore to particular version if specified - Extend SharedDataDirCount to handle transitioned and restored objects - Restore-object should accept version-id for version-suspended bucket (#12091) - Check if remote tier creds have sufficient permissions - Bonus minor fixes to existing error messages Co-authored-by: Poorna Krishnamoorthy <poorna@minio.io> Co-authored-by: Krishna Srinivas <krishna@minio.io> Signed-off-by: Harshavardhana <harsha@minio.io>
2021-04-19 13:30:42 -04:00
// Remote Tier management operations
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-12 13:49:50 -04:00
adminRouter.Methods(http.MethodPut).Path(adminVersion + "/tier").HandlerFunc(gz(httpTraceHdrs(adminAPI.AddTierHandler)))
adminRouter.Methods(http.MethodPost).Path(adminVersion + "/tier/{tier}").HandlerFunc(gz(httpTraceHdrs(adminAPI.EditTierHandler)))
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/tier").HandlerFunc(gz(httpTraceHdrs(adminAPI.ListTierHandler)))
Add support for tier-remove and tier-verify (#14382) * Add tier remove support only if it's empty * Add support for tier verify
2022-02-23 16:34:25 -05:00
adminRouter.Methods(http.MethodDelete).Path(adminVersion + "/tier/{tier}").HandlerFunc(gz(httpTraceHdrs(adminAPI.RemoveTierHandler)))
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/tier/{tier}").HandlerFunc(gz(httpTraceHdrs(adminAPI.VerifyTierHandler)))
ilm: Make per-tier stats available via admin-tier-info (#13381)
2021-10-23 21:38:33 -04:00
// Tier stats
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/tier-stats").HandlerFunc(gz(httpTraceHdrs(adminAPI.TierStatsHandler)))
Add new site replication feature (#13311) This change allows a set of MinIO sites (clusters) to be configured for mutual replication of all buckets (including bucket policies, tags, object-lock configuration and bucket encryption), IAM policies, LDAP service accounts and LDAP STS accounts.
2021-10-06 19:36:31 -04:00
// Cluster Replication APIs
adminRouter.Methods(http.MethodPut).Path(adminVersion + "/site-replication/add").HandlerFunc(gz(httpTraceHdrs(adminAPI.SiteReplicationAdd)))
Add API for removing site(s) from site replication (#14104)
2022-02-01 20:26:09 -05:00
adminRouter.Methods(http.MethodPut).Path(adminVersion + "/site-replication/remove").HandlerFunc(gz(httpTraceHdrs(adminAPI.SiteReplicationRemove)))
Add new site replication feature (#13311) This change allows a set of MinIO sites (clusters) to be configured for mutual replication of all buckets (including bucket policies, tags, object-lock configuration and bucket encryption), IAM policies, LDAP service accounts and LDAP STS accounts.
2021-10-06 19:36:31 -04:00
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/site-replication/info").HandlerFunc(gz(httpTraceHdrs(adminAPI.SiteReplicationInfo)))
Add API for cluster replication status visibility (#13885)
2022-01-05 05:44:08 -05:00
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/site-replication/metainfo").HandlerFunc(gz(httpTraceHdrs(adminAPI.SiteReplicationMetaInfo)))
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/site-replication/status").HandlerFunc(gz(httpTraceHdrs(adminAPI.SiteReplicationStatus)))
madmin-go v1.1.23 and pkg v1.1.11
2021-12-27 02:10:34 -05:00
adminRouter.Methods(http.MethodPut).Path(adminVersion + "/site-replication/peer/join").HandlerFunc(gz(httpTraceHdrs(adminAPI.SRPeerJoin)))
adminRouter.Methods(http.MethodPut).Path(adminVersion+"/site-replication/peer/bucket-ops").HandlerFunc(gz(httpTraceHdrs(adminAPI.SRPeerBucketOps))).Queries("bucket", "{bucket:.*}").Queries("operation", "{operation:.*}")
adminRouter.Methods(http.MethodPut).Path(adminVersion + "/site-replication/peer/iam-item").HandlerFunc(gz(httpTraceHdrs(adminAPI.SRPeerReplicateIAMItem)))
adminRouter.Methods(http.MethodPut).Path(adminVersion + "/site-replication/peer/bucket-meta").HandlerFunc(gz(httpTraceHdrs(adminAPI.SRPeerReplicateBucketItem)))
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/site-replication/peer/idp-settings").HandlerFunc(gz(httpTraceHdrs(adminAPI.SRPeerGetIDPSettings)))
Add API for removing site(s) from site replication (#14022)
2022-01-21 11:48:21 -05:00
adminRouter.Methods(http.MethodPut).Path(adminVersion + "/site-replication/edit").HandlerFunc(gz(httpTraceHdrs(adminAPI.SiteReplicationEdit)))
adminRouter.Methods(http.MethodPut).Path(adminVersion + "/site-replication/peer/edit").HandlerFunc(gz(httpTraceHdrs(adminAPI.SRPeerEdit)))
Add API for removing site(s) from site replication (#14104)
2022-02-01 20:26:09 -05:00
adminRouter.Methods(http.MethodPut).Path(adminVersion + "/site-replication/peer/remove").HandlerFunc(gz(httpTraceHdrs(adminAPI.SRPeerRemove)))
fix: change policies API to return and take struct (#9181) This allows for order guarantees in returned values can be consumed safely by the caller to avoid any additional parsing and validation. Fixes #9171
2020-04-07 22:30:59 -04:00
}
remove/deprecate crawler disable environment (#11214) with changes present to automatically throttle crawler at runtime, there is no need to have an environment value to disable crawling. crawling is a fundamental piece for healing, lifecycle and many other features there is no good reason anyone would need to disable this on a production system. * Apply suggestions from code review
2021-01-04 12:43:31 -05:00
Support bucket versioning (#9377) - Implement a new xl.json 2.0.0 format to support, this moves the entire marshaling logic to POSIX layer, top layer always consumes a common FileInfo construct which simplifies the metadata reads. - Implement list object versions - Migrate to siphash from crchash for new deployments for object placements. Fixes #2111
2020-06-12 23:04:01 -04:00
if globalIsDistErasure {
fix: expire locks only on participating lockers (#11335) additionally also add a new ForceUnlock API, to allow forcibly unlocking locks if possible.
2021-01-25 13:01:27 -05:00
// Top locks
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-12 13:49:50 -04:00
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/top/locks").HandlerFunc(gz(httpTraceHdrs(adminAPI.TopLocksHandler)))
fix: expire locks only on participating lockers (#11335) additionally also add a new ForceUnlock API, to allow forcibly unlocking locks if possible.
2021-01-25 13:01:27 -05:00
// Force unlocks paths
adminRouter.Methods(http.MethodPost).Path(adminVersion+"/force-unlock").
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-12 13:49:50 -04:00
Queries("paths", "{paths:.*}").HandlerFunc(gz(httpTraceHdrs(adminAPI.ForceUnlockHandler)))
fix: change policies API to return and take struct (#9181) This allows for order guarantees in returned values can be consumed safely by the caller to avoid any additional parsing and validation. Fixes #9171
2020-04-07 22:30:59 -04:00
}
Top Locks command implementation (#7052) API to list locks used in distributed XL mode
2019-01-24 10:22:14 -05:00
Server side speedtest implementation (#12750)
2021-07-27 15:55:56 -04:00
adminRouter.Methods(http.MethodPost).Path(adminVersion + "/speedtest").HandlerFunc(httpTraceHdrs(adminAPI.SpeedtestHandler))
add support for speedtest drive (#14182)
2022-02-02 01:38:05 -05:00
adminRouter.Methods(http.MethodPost).Path(adminVersion + "/speedtest/object").HandlerFunc(httpTraceHdrs(adminAPI.ObjectSpeedtestHandler))
adminRouter.Methods(http.MethodPost).Path(adminVersion + "/speedtest/drive").HandlerFunc(httpTraceHdrs(adminAPI.DriveSpeedtestHandler))
adminRouter.Methods(http.MethodPost).Path(adminVersion + "/speedtest/net").HandlerFunc(httpTraceHdrs(adminAPI.NetSpeedtestHandler))
Server side speedtest implementation (#12750)
2021-07-27 15:55:56 -04:00
fix: change policies API to return and take struct (#9181) This allows for order guarantees in returned values can be consumed safely by the caller to avoid any additional parsing and validation. Fixes #9171
2020-04-07 22:30:59 -04:00
// HTTP Trace
Simplify gzhttp wrapper (#12912) The wrapper now accepts interfaces so we don't need a wrapper for that any more.
2021-08-09 15:45:59 -04:00
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/trace").HandlerFunc(gz(http.HandlerFunc(adminAPI.TraceHandler)))
KMS Admin-API: add route and handler for KMS key info (#7955) This commit adds an admin API route and handler for requesting status information about a KMS key. Therefore, the client specifies the KMS key ID (when empty / not set the server takes the currently configured default key-ID) and the server tries to perform a dummy encryption, re-wrap and decryption operation. If all three succeed we know that the server can access the KMS and has permissions to generate, re-wrap and decrypt data keys (policy is set correctly).
2019-09-04 16:19:44 -04:00
fix: change policies API to return and take struct (#9181) This allows for order guarantees in returned values can be consumed safely by the caller to avoid any additional parsing and validation. Fixes #9171
2020-04-07 22:30:59 -04:00
// Console Logs
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-12 13:49:50 -04:00
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/log").HandlerFunc(gz(httpTraceAll(adminAPI.ConsoleLogHandler)))
Add admin API to send console log messages (#7784) Utilized by mc admin console command.
2019-09-03 14:10:48 -04:00
fix: change policies API to return and take struct (#9181) This allows for order guarantees in returned values can be consumed safely by the caller to avoid any additional parsing and validation. Fixes #9171
2020-04-07 22:30:59 -04:00
// -- KMS APIs --
//
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-12 13:49:50 -04:00
adminRouter.Methods(http.MethodPost).Path(adminVersion + "/kms/status").HandlerFunc(gz(httpTraceAll(adminAPI.KMSStatusHandler)))
adminRouter.Methods(http.MethodPost).Path(adminVersion+"/kms/key/create").HandlerFunc(gz(httpTraceAll(adminAPI.KMSCreateKeyHandler))).Queries("key-id", "{key-id:.*}")
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/kms/key/status").HandlerFunc(gz(httpTraceAll(adminAPI.KMSKeyStatusHandler)))
KMS Admin-API: add route and handler for KMS key info (#7955) This commit adds an admin API route and handler for requesting status information about a KMS key. Therefore, the client specifies the KMS key ID (when empty / not set the server takes the currently configured default key-ID) and the server tries to perform a dummy encryption, re-wrap and decryption operation. If all three succeed we know that the server can access the KMS and has permissions to generate, re-wrap and decrypt data keys (policy is set correctly).
2019-09-04 16:19:44 -04:00
fix: change policies API to return and take struct (#9181) This allows for order guarantees in returned values can be consumed safely by the caller to avoid any additional parsing and validation. Fixes #9171
2020-04-07 22:30:59 -04:00
if !globalIsGateway {
Rename OBD to Health (#10842) Also, Remove thread stats and openfds from the health report as we already have process stats and numfds
2020-11-20 15:52:53 -05:00
// Keep obdinfo for backward compatibility with mc
obd: Add console log to OBD output (#10372)
2020-09-15 21:02:54 -04:00
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/obdinfo").
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-12 13:49:50 -04:00
HandlerFunc(gz(httpTraceHdrs(adminAPI.HealthInfoHandler)))
Rename OBD to Health (#10842) Also, Remove thread stats and openfds from the health report as we already have process stats and numfds
2020-11-20 15:52:53 -05:00
// -- Health API --
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/healthinfo").
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-12 13:49:50 -04:00
HandlerFunc(gz(httpTraceHdrs(adminAPI.HealthInfoHandler)))
Add basic bandwidth monitoring for replication. (#10501) This change tracks bandwidth for a bucket and object - [x] Add Admin API - [x] Add Peer API - [x] Add BW throttling - [x] Admin APIs to set replication limit - [x] Admin APIs for fetch bandwidth
2020-10-09 23:36:00 -04:00
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/bandwidth").
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-12 13:49:50 -04:00
HandlerFunc(gz(httpTraceHdrs(adminAPI.BandwidthMonitorHandler)))
fix: change policies API to return and take struct (#9181) This allows for order guarantees in returned values can be consumed safely by the caller to avoid any additional parsing and validation. Fixes #9171
2020-04-07 22:30:59 -04:00
}
Implement oboard diagnostics admin API (#9024) - Implement a graph algorithm to test network bandwidth from every node to every other node - Saturate any network bandwidth adaptively, accounting for slow and fast network capacity - Implement parallel drive OBD tests - Implement a paging mechanism for OBD test to provide periodic updates to client - Implement Sys, Process, Host, Mem OBD Infos
2020-03-27 00:07:39 -04:00
}
Return appropriate errors API versions changes across REST APIs (#8480) This PR adds code to appropriately handle versioning issues that come up quite constantly across our API changes. Currently we were also routing our requests wrong which sort of made it harder to write a consistent error handling code to appropriately reject or honor requests. This PR potentially fixes issues - old mc is used against new minio release which is incompatible returns an appropriate for client action. - any older servers talking to each other, report appropriate error - incompatible peer servers should report error and reject the calls with appropriate error
2019-11-04 12:30:59 -05:00
// If none of the routes match add default error handler routes
Remove redundant http.HandlerFunc type conversion. (#10576)
2020-09-28 16:33:49 -04:00
adminRouter.NotFoundHandler = httpTraceAll(errorResponseHandler)
ListObjects Metadata Caching (#10648) Design: https://gist.github.com/klauspost/025c09b48ed4a1293c917cecfabdf21c Gist of improvements: * Cross-server caching and listing will use the same data across servers and requests. * Lists can be arbitrarily resumed at a constant speed. * Metadata for all files scanned is stored for streaming retrieval. * The existing bloom filters controlled by the crawler is used for validating caches. * Concurrent requests for the same data (or parts of it) will not spawn additional walkers. * Listing a subdirectory of an existing recursive cache will use the cache. * All listing operations are fully streamable so the number of objects in a bucket no longer dictates the amount of memory. * Listings can be handled by any server within the cluster. * Caches are cleaned up when out of date or superseded by a more recent one.
2020-10-28 12:18:35 -04:00
adminRouter.MethodNotAllowedHandler = httpTraceAll(methodNotAllowedHandler("Admin"))
Add service API handler stubs for status, stop and restart (#3417)
2016-12-16 01:26:15 -05:00
}