MyFavMirrors/minio
1
0
Fork 0
mirror of https://github.com/minio/minio.git synced 2024-12-26 23:25:54 -05:00
Code Issues Packages Projects Releases Wiki Activity
757cf413cb
minio/cmd/admin-router.go

429 lines
26 KiB
Go
Raw Normal View History

update license change for MinIO Signed-off-by: Harshavardhana <harsha@minio.io>
2021-04-18 15:41:13 -04:00
// Copyright (c) 2015-2021 MinIO, Inc.
//
// This file is part of MinIO Object Storage stack
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
Add service API handler stubs for status, stop and restart (#3417)
2016-12-16 01:26:15 -05:00
package cmd
Move admin APIs to new path and add redesigned heal APIs (#5351) - Changes related to moving admin APIs - admin APIs now have an endpoint under /minio/admin - admin APIs are now versioned - a new API to server the version is added at "GET /minio/admin/version" and all API operations have the path prefix /minio/admin/v1/<operation> - new service stop API added - credentials change API is moved to /minio/admin/v1/config/credential - credentials change API and configuration get/set API now require TLS so that credentials are protected - all API requests now receive JSON - heal APIs are disabled as they will be changed substantially - Heal API changes Heal API is now provided at a single endpoint with the ability for a client to start a heal sequence on all the data in the server, a single bucket, or under a prefix within a bucket. When a heal sequence is started, the server returns a unique token that needs to be used for subsequent 'status' requests to fetch heal results. On each status request from the client, the server returns heal result records that it has accumulated since the previous status request. The server accumulates upto 1000 records and pauses healing further objects until the client requests for status. If the client does not request any further records for a long time, the server aborts the heal sequence automatically. A heal result record is returned for each entity healed on the server, such as system metadata, object metadata, buckets and objects, and has information about the before and after states on each disk. A client may request to force restart a heal sequence - this causes the running heal sequence to be aborted at the next safe spot and starts a new heal sequence.
2018-01-22 17:54:55 -05:00
import (
"net/http"
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-12 13:49:50 -04:00
"github.com/klauspost/compress/gzhttp"
"github.com/klauspost/compress/gzip"
Bump up madmin-go and pkg deps (#17469)
2023-06-19 20:53:08 -04:00
"github.com/minio/madmin-go/v3"
Simplify gzhttp wrapper (#12912) The wrapper now accepts interfaces so we don't need a wrapper for that any more.
2021-08-09 15:45:59 -04:00
"github.com/minio/minio/internal/logger"
migrate to minio/mux from gorilla/mux (#16456)
2023-01-23 06:12:47 -05:00
"github.com/minio/mux"
Move admin APIs to new path and add redesigned heal APIs (#5351) - Changes related to moving admin APIs - admin APIs now have an endpoint under /minio/admin - admin APIs are now versioned - a new API to server the version is added at "GET /minio/admin/version" and all API operations have the path prefix /minio/admin/v1/<operation> - new service stop API added - credentials change API is moved to /minio/admin/v1/config/credential - credentials change API and configuration get/set API now require TLS so that credentials are protected - all API requests now receive JSON - heal APIs are disabled as they will be changed substantially - Heal API changes Heal API is now provided at a single endpoint with the ability for a client to start a heal sequence on all the data in the server, a single bucket, or under a prefix within a bucket. When a heal sequence is started, the server returns a unique token that needs to be used for subsequent 'status' requests to fetch heal results. On each status request from the client, the server returns heal result records that it has accumulated since the previous status request. The server accumulates upto 1000 records and pauses healing further objects until the client requests for status. If the client does not request any further records for a long time, the server aborts the heal sequence automatically. A heal result record is returned for each entity healed on the server, such as system metadata, object metadata, buckets and objects, and has information about the before and after states on each disk. A client may request to force restart a heal sequence - this causes the running heal sequence to be aborted at the next safe spot and starts a new heal sequence.
2018-01-22 17:54:55 -05:00
)
const (
feat: support perf site replication (#17477)
2023-07-06 01:28:26 -04:00
adminPathPrefix = minioReservedBucketPath + "/admin"
adminAPIVersion = madmin.AdminAPIVersion
adminAPIVersionPrefix = SlashSeparator + adminAPIVersion
adminAPISiteReplicationDevNull = "/site-replication/devnull"
adminAPISiteReplicationNetPerf = "/site-replication/netperf"
feat: support perf client to show `TX` from client to server (#17718)
2023-08-10 10:14:46 -04:00
adminAPIClientDevNull = "/speedtest/client/devnull"
adminAPIClientDevExtraTime = "/speedtest/client/devnull/extratime"
Move admin APIs to new path and add redesigned heal APIs (#5351) - Changes related to moving admin APIs - admin APIs now have an endpoint under /minio/admin - admin APIs are now versioned - a new API to server the version is added at "GET /minio/admin/version" and all API operations have the path prefix /minio/admin/v1/<operation> - new service stop API added - credentials change API is moved to /minio/admin/v1/config/credential - credentials change API and configuration get/set API now require TLS so that credentials are protected - all API requests now receive JSON - heal APIs are disabled as they will be changed substantially - Heal API changes Heal API is now provided at a single endpoint with the ability for a client to start a heal sequence on all the data in the server, a single bucket, or under a prefix within a bucket. When a heal sequence is started, the server returns a unique token that needs to be used for subsequent 'status' requests to fetch heal results. On each status request from the client, the server returns heal result records that it has accumulated since the previous status request. The server accumulates upto 1000 records and pauses healing further objects until the client requests for status. If the client does not request any further records for a long time, the server aborts the heal sequence automatically. A heal result record is returned for each entity healed on the server, such as system metadata, object metadata, buckets and objects, and has information about the before and after states on each disk. A client may request to force restart a heal sequence - this causes the running heal sequence to be aborted at the next safe spot and starts a new heal sequence.
2018-01-22 17:54:55 -05:00
)
Add service API handler stubs for status, stop and restart (#3417)
2016-12-16 01:26:15 -05:00
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
var gzipHandler = func() func(http.Handler) http.HandlerFunc {
gz, err := gzhttp.NewWrapper(gzhttp.MinSize(1000), gzhttp.CompressionLevel(gzip.BestSpeed))
if err != nil {
// Static params, so this is very unlikely.
logger.Fatal(err, "Unable to initialize server")
}
return gz
}()
// Set of handler options as bit flags
type hFlag uint8
const (
// this flag disables gzip compression of responses
noGZFlag = 1 << iota
// this flag enables tracing body and headers instead of just headers
traceAllFlag
// pass this flag to skip checking if object layer is available
noObjLayerFlag
)
fix: some flyby typos in the code (#19212) Signed-off-by: thirdkeyword <fliterdashen@gmail.com>
2024-03-10 17:09:36 -04:00
// Has checks if the given flag is enabled in `h`.
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
func (h hFlag) Has(flag hFlag) bool {
// Use bitwise-AND and check if the result is non-zero.
return h&flag != 0
}
// adminMiddleware performs some common admin handler functionality for all
// handlers:
//
// - updates request context with `logger.ReqInfo` and api name based on the
// name of the function handler passed (this handler must be a method of
// `adminAPIHandlers`).
//
// - sets up call to send AuditLog
//
Add common middleware to S3 API handlers (#19171) The middleware sets up tracing, throttling, gzipped responses and collecting API stats. Additionally, this change updates the names of handler functions in metric labels to be the same as the name derived from Go lang reflection on the handler name. The metric api labels are now stored in memory the same as the handler name - they will be camelcased, e.g. `GetObject` instead of `getobject`. For compatibility, we lowercase the metric api label values when emitting the metrics.
2024-03-04 13:05:56 -05:00
// While this is a middleware function (i.e. it takes a handler function and
// returns one), due to flags being passed based on required conditions, it is
// done per-"handler function registration" in the router.
//
// The passed in handler function must be a method of `adminAPIHandlers` for the
// name displayed in logs and trace to be accurate. The name is extracted via
// reflection.
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
//
// When no flags are passed, gzip compression, http tracing of headers and
// checking of object layer availability are all enabled. Use flags to modify
// this behavior.
func adminMiddleware(f http.HandlerFunc, flags ...hFlag) http.HandlerFunc {
// Collect all flags with bitwise-OR and assign operator
var handlerFlags hFlag
for _, flag := range flags {
handlerFlags |= flag
}
Add common middleware to S3 API handlers (#19171) The middleware sets up tracing, throttling, gzipped responses and collecting API stats. Additionally, this change updates the names of handler functions in metric labels to be the same as the name derived from Go lang reflection on the handler name. The metric api labels are now stored in memory the same as the handler name - they will be camelcased, e.g. `GetObject` instead of `getobject`. For compatibility, we lowercase the metric api label values when emitting the metrics.
2024-03-04 13:05:56 -05:00
// Get name of the handler using reflection.
handlerName := getHandlerName(f, "adminAPIHandlers")
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
var handler http.HandlerFunc = func(w http.ResponseWriter, r *http.Request) {
// Update request context with `logger.ReqInfo`.
r = r.WithContext(newContext(r, w, handlerName))
defer logger.AuditLog(r.Context(), w, r, mustGetClaimsFromToken(r))
// Check if object layer is available, if not return error early.
if !handlerFlags.Has(noObjLayerFlag) {
objectAPI := newObjectLayerFn()
if objectAPI == nil || globalNotificationSys == nil {
writeErrorResponseJSON(r.Context(), w, errorCodes.ToAPIErr(ErrServerNotInitialized), r.URL)
return
}
}
// Apply http tracing "middleware" based on presence of flag.
var f2 http.HandlerFunc
if handlerFlags.Has(traceAllFlag) {
f2 = httpTraceAll(f)
} else {
f2 = httpTraceHdrs(f)
}
// call the final handler
f2(w, r)
}
// Enable compression of responses based on presence of flag.
if !handlerFlags.Has(noGZFlag) {
handler = gzipHandler(handler)
}
return handler
}
Replace Minio refs in docs with MinIO and links (#7494)
2019-04-09 14:39:42 -04:00
// adminAPIHandlers provides HTTP handlers for MinIO admin API.
Return appropriate errors API versions changes across REST APIs (#8480) This PR adds code to appropriately handle versioning issues that come up quite constantly across our API changes. Currently we were also routing our requests wrong which sort of made it harder to write a consistent error handling code to appropriately reject or honor requests. This PR potentially fixes issues - old mc is used against new minio release which is incompatible returns an appropriate for client action. - any older servers talking to each other, report appropriate error - incompatible peer servers should report error and reject the calls with appropriate error
2019-11-04 12:30:59 -05:00
type adminAPIHandlers struct{}
Add service API handler stubs for status, stop and restart (#3417)
2016-12-16 01:26:15 -05:00
// registerAdminRouter - Add handler functions for each service REST API routes.
fix: add IAM dummy store for gateway operations (#12670) with console addition users cannot login with root credentials without etcd persistent layer, allow a dummy store such that such functionalities can be supported when running as non-persistent manner, this enables all calls and operations.
2021-07-10 11:32:52 -04:00
func registerAdminRouter(router *mux.Router, enableConfigOps bool) {
Add service API handler stubs for status, stop and restart (#3417)
2016-12-16 01:26:15 -05:00
adminAPI := adminAPIHandlers{}
// Admin router
Return appropriate errors API versions changes across REST APIs (#8480) This PR adds code to appropriately handle versioning issues that come up quite constantly across our API changes. Currently we were also routing our requests wrong which sort of made it harder to write a consistent error handling code to appropriately reject or honor requests. This PR potentially fixes issues - old mc is used against new minio release which is incompatible returns an appropriate for client action. - any older servers talking to each other, report appropriate error - incompatible peer servers should report error and reject the calls with appropriate error
2019-11-04 12:30:59 -05:00
adminRouter := router.PathPrefix(adminPathPrefix).Subrouter()
Move admin APIs to new path and add redesigned heal APIs (#5351) - Changes related to moving admin APIs - admin APIs now have an endpoint under /minio/admin - admin APIs are now versioned - a new API to server the version is added at "GET /minio/admin/version" and all API operations have the path prefix /minio/admin/v1/<operation> - new service stop API added - credentials change API is moved to /minio/admin/v1/config/credential - credentials change API and configuration get/set API now require TLS so that credentials are protected - all API requests now receive JSON - heal APIs are disabled as they will be changed substantially - Heal API changes Heal API is now provided at a single endpoint with the ability for a client to start a heal sequence on all the data in the server, a single bucket, or under a prefix within a bucket. When a heal sequence is started, the server returns a unique token that needs to be used for subsequent 'status' requests to fetch heal results. On each status request from the client, the server returns heal result records that it has accumulated since the previous status request. The server accumulates upto 1000 records and pauses healing further objects until the client requests for status. If the client does not request any further records for a long time, the server aborts the heal sequence automatically. A heal result record is returned for each entity healed on the server, such as system metadata, object metadata, buckets and objects, and has information about the before and after states on each disk. A client may request to force restart a heal sequence - this causes the running heal sequence to be aborted at the next safe spot and starts a new heal sequence.
2018-01-22 17:54:55 -05:00
fix: change policies API to return and take struct (#9181) This allows for order guarantees in returned values can be consumed safely by the caller to avoid any additional parsing and validation. Fixes #9171
2020-04-07 22:30:59 -04:00
adminVersions := []string{
adminAPIVersionPrefix,
}
Implement list, clear locks REST API w/ pkg/madmin support (#3491) * Filter lock info based on bucket, prefix and time since lock was held * Implement list and clear locks REST API * madmin: Add list and clear locks API * locks: Clear locks matching bucket, prefix, relTime. * Gather lock information across nodes for both list and clear locks admin REST API. * docs: Add lock API to management APIs
2017-01-04 02:39:22 -05:00
fix: change policies API to return and take struct (#9181) This allows for order guarantees in returned values can be consumed safely by the caller to avoid any additional parsing and validation. Fixes #9171
2020-04-07 22:30:59 -04:00
for _, adminVersion := range adminVersions {
introduce new ServiceV2 API to handle guided restarts (#18826) New API now verifies any hung disks before restart/stop, provides a 'per node' break down of the restart/stop results. Provides also how many blocked syscalls are present on the drives and what users must do about them. Adds options to do pre-flight checks to provide information to the user regarding any hung disks. Provides 'force' option to forcibly attempt a restart() even with waiting syscalls on the drives.
2024-01-19 17:22:36 -05:00
// Restart and stop MinIO service type=2
adminRouter.Methods(http.MethodPost).Path(adminVersion+"/service").HandlerFunc(adminMiddleware(adminAPI.ServiceV2Handler, traceAllFlag)).Queries("action", "{action:.*}", "type", "2")
add new update v2 that updates per node, allows idempotent behavior (#18859) add new update v2 that updates per node, allows idempotent behavior new API ensures that - binary is correct and can be downloaded checksummed verified - committed to actual path - restart returns back the relevant waiting drives
2024-01-26 11:40:13 -05:00
// Deprecated: Restart and stop MinIO service.
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminRouter.Methods(http.MethodPost).Path(adminVersion+"/service").HandlerFunc(adminMiddleware(adminAPI.ServiceHandler, traceAllFlag)).Queries("action", "{action:.*}")
introduce new ServiceV2 API to handle guided restarts (#18826) New API now verifies any hung disks before restart/stop, provides a 'per node' break down of the restart/stop results. Provides also how many blocked syscalls are present on the drives and what users must do about them. Adds options to do pre-flight checks to provide information to the user regarding any hung disks. Provides 'force' option to forcibly attempt a restart() even with waiting syscalls on the drives.
2024-01-19 17:22:36 -05:00
add new update v2 that updates per node, allows idempotent behavior (#18859) add new update v2 that updates per node, allows idempotent behavior new API ensures that - binary is correct and can be downloaded checksummed verified - committed to actual path - restart returns back the relevant waiting drives
2024-01-26 11:40:13 -05:00
// Update all MinIO servers type=2
adminRouter.Methods(http.MethodPost).Path(adminVersion+"/update").HandlerFunc(adminMiddleware(adminAPI.ServerUpdateV2Handler, traceAllFlag)).Queries("updateURL", "{updateURL:.*}", "type", "2")
// Deprecated: Update MinIO servers.
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminRouter.Methods(http.MethodPost).Path(adminVersion+"/update").HandlerFunc(adminMiddleware(adminAPI.ServerUpdateHandler, traceAllFlag)).Queries("updateURL", "{updateURL:.*}")
The prometheus metrics refractoring (#8003) The measures are consolidated to the following metrics - `disk_storage_used` : Disk space used by the disk. - `disk_storage_available`: Available disk space left on the disk. - `disk_storage_total`: Total disk space on the disk. - `disks_offline`: Total number of offline disks in current MinIO instance. - `disks_total`: Total number of disks in current MinIO instance. - `s3_requests_total`: Total number of s3 requests in current MinIO instance. - `s3_errors_total`: Total number of errors in s3 requests in current MinIO instance. - `s3_requests_current`: Total number of active s3 requests in current MinIO instance. - `internode_rx_bytes_total`: Total number of internode bytes received by current MinIO server instance. - `internode_tx_bytes_total`: Total number of bytes sent to the other nodes by current MinIO server instance. - `s3_rx_bytes_total`: Total number of s3 bytes received by current MinIO server instance. - `s3_tx_bytes_total`: Total number of s3 bytes sent by current MinIO server instance. - `minio_version_info`: Current MinIO version with commit-id. - `s3_ttfb_seconds_bucket`: Histogram that holds the latency information of the requests. And this PR also modifies the current StorageInfo queries - Decouples StorageInfo from ServerInfo . - StorageInfo is enhanced to give endpoint information. NOTE: ADMIN API VERSION IS BUMPED UP IN THIS PR Fixes #7873
2019-10-23 00:01:14 -04:00
fix: change policies API to return and take struct (#9181) This allows for order guarantees in returned values can be consumed safely by the caller to avoid any additional parsing and validation. Fixes #9171
2020-04-07 22:30:59 -04:00
// Info operations
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/info").HandlerFunc(adminMiddleware(adminAPI.ServerInfoHandler, traceAllFlag, noObjLayerFlag))
adminRouter.Methods(http.MethodGet, http.MethodPost).Path(adminVersion + "/inspect-data").HandlerFunc(adminMiddleware(adminAPI.InspectDataHandler, noGZFlag, traceAllFlag))
admin: Add ServerInfo API() (#3743)
2017-02-15 13:45:45 -05:00
fix: change policies API to return and take struct (#9181) This allows for order guarantees in returned values can be consumed safely by the caller to avoid any additional parsing and validation. Fixes #9171
2020-04-07 22:30:59 -04:00
// StorageInfo operations
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/storageinfo").HandlerFunc(adminMiddleware(adminAPI.StorageInfoHandler, traceAllFlag))
fix: change policies API to return and take struct (#9181) This allows for order guarantees in returned values can be consumed safely by the caller to avoid any additional parsing and validation. Fixes #9171
2020-04-07 22:30:59 -04:00
// DataUsageInfo operations
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/datausageinfo").HandlerFunc(adminMiddleware(adminAPI.DataUsageInfoHandler, traceAllFlag))
Add detailed scanner metrics (#15161)
2022-07-05 17:45:49 -04:00
// Metrics operation
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/metrics").HandlerFunc(adminMiddleware(adminAPI.MetricsHandler, traceAllFlag))
Add new admin API to return Accounting Usage (#8689)
2020-02-04 21:20:39 -05:00
Support bucket versioning (#9377) - Implement a new xl.json 2.0.0 format to support, this moves the entire marshaling logic to POSIX layer, top layer always consumes a common FileInfo construct which simplifies the metadata reads. - Implement list object versions - Migrate to siphash from crchash for new deployments for object placements. Fixes #2111
2020-06-12 23:04:01 -04:00
if globalIsDistErasure || globalIsErasure {
add gocritic/ruleguard checks back again, cleanup code. (#13665) - remove some duplicated code - reported a bug, separately fixed in #13664 - using strings.ReplaceAll() when needed - using filepath.ToSlash() use when needed - remove all non-Go style comments from the codebase Co-authored-by: Aditya Manthramurthy <donatello@users.noreply.github.com>
2021-11-16 12:28:29 -05:00
// Heal operations
Add ServerDrivesPerfInfo() admin API (#6969) This is part of implementation for mc admin health command. The ServerDrivesPerfInfo() admin API returns read and write speed information for all the drives (local and remote) in a given Minio server deployment. Part of minio/mc#2606
2018-12-31 12:46:44 -05:00
fix: change policies API to return and take struct (#9181) This allows for order guarantees in returned values can be consumed safely by the caller to avoid any additional parsing and validation. Fixes #9171
2020-04-07 22:30:59 -04:00
// Heal processing endpoint.
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminRouter.Methods(http.MethodPost).Path(adminVersion + "/heal/").HandlerFunc(adminMiddleware(adminAPI.HealHandler, traceAllFlag))
adminRouter.Methods(http.MethodPost).Path(adminVersion + "/heal/{bucket}").HandlerFunc(adminMiddleware(adminAPI.HealHandler, traceAllFlag))
adminRouter.Methods(http.MethodPost).Path(adminVersion + "/heal/{bucket}/{prefix:.*}").HandlerFunc(adminMiddleware(adminAPI.HealHandler, traceAllFlag))
adminRouter.Methods(http.MethodPost).Path(adminVersion + "/background-heal/status").HandlerFunc(adminMiddleware(adminAPI.BackgroundHealStatusHandler, traceAllFlag))
feat: decommission feature for pools (#14012) ``` λ mc admin decommission start alias/ http://minio{1...2}/data{1...4} ``` ``` λ mc admin decommission status alias/ ┌─────┬─────────────────────────────────┬──────────────────────────────────┬────────┐ │ ID │ Pools │ Capacity │ Status │ │ 1st │ http://minio{1...2}/data{1...4} │ 439 GiB (used) / 561 GiB (total) │ Active │ │ 2nd │ http://minio{3...4}/data{1...4} │ 329 GiB (used) / 421 GiB (total) │ Active │ └─────┴─────────────────────────────────┴──────────────────────────────────┴────────┘ ``` ``` λ mc admin decommission status alias/ http://minio{1...2}/data{1...4} Progress: ===================> [1GiB/sec] [15%] [4TiB/50TiB] Time Remaining: 4 hours (started 3 hours ago) ``` ``` λ mc admin decommission status alias/ http://minio{1...2}/data{1...4} ERROR: This pool is not scheduled for decommissioning currently. ``` ``` λ mc admin decommission cancel alias/ ┌─────┬─────────────────────────────────┬──────────────────────────────────┬──────────┐ │ ID │ Pools │ Capacity │ Status │ │ 1st │ http://minio{1...2}/data{1...4} │ 439 GiB (used) / 561 GiB (total) │ Draining │ └─────┴─────────────────────────────────┴──────────────────────────────────┴──────────┘ ``` > NOTE: Canceled decommission will not make the pool active again, since we might have > Potentially partial duplicate content on the other pools, to avoid this scenario be > very sure to start decommissioning as a planned activity. ``` λ mc admin decommission cancel alias/ http://minio{1...2}/data{1...4} ┌─────┬─────────────────────────────────┬──────────────────────────────────┬────────────────────┐ │ ID │ Pools │ Capacity │ Status │ │ 1st │ http://minio{1...2}/data{1...4} │ 439 GiB (used) / 561 GiB (total) │ Draining(Canceled) │ └─────┴─────────────────────────────────┴──────────────────────────────────┴────────────────────┘ ```
2022-01-10 12:07:49 -05:00
// Pool operations
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/pools/list").HandlerFunc(adminMiddleware(adminAPI.ListPools, traceAllFlag))
adminRouter.Methods(http.MethodGet).Path(adminVersion+"/pools/status").HandlerFunc(adminMiddleware(adminAPI.StatusPool, traceAllFlag)).Queries("pool", "{pool:.*}")
feat: decommission feature for pools (#14012) ``` λ mc admin decommission start alias/ http://minio{1...2}/data{1...4} ``` ``` λ mc admin decommission status alias/ ┌─────┬─────────────────────────────────┬──────────────────────────────────┬────────┐ │ ID │ Pools │ Capacity │ Status │ │ 1st │ http://minio{1...2}/data{1...4} │ 439 GiB (used) / 561 GiB (total) │ Active │ │ 2nd │ http://minio{3...4}/data{1...4} │ 329 GiB (used) / 421 GiB (total) │ Active │ └─────┴─────────────────────────────────┴──────────────────────────────────┴────────┘ ``` ``` λ mc admin decommission status alias/ http://minio{1...2}/data{1...4} Progress: ===================> [1GiB/sec] [15%] [4TiB/50TiB] Time Remaining: 4 hours (started 3 hours ago) ``` ``` λ mc admin decommission status alias/ http://minio{1...2}/data{1...4} ERROR: This pool is not scheduled for decommissioning currently. ``` ``` λ mc admin decommission cancel alias/ ┌─────┬─────────────────────────────────┬──────────────────────────────────┬──────────┐ │ ID │ Pools │ Capacity │ Status │ │ 1st │ http://minio{1...2}/data{1...4} │ 439 GiB (used) / 561 GiB (total) │ Draining │ └─────┴─────────────────────────────────┴──────────────────────────────────┴──────────┘ ``` > NOTE: Canceled decommission will not make the pool active again, since we might have > Potentially partial duplicate content on the other pools, to avoid this scenario be > very sure to start decommissioning as a planned activity. ``` λ mc admin decommission cancel alias/ http://minio{1...2}/data{1...4} ┌─────┬─────────────────────────────────┬──────────────────────────────────┬────────────────────┐ │ ID │ Pools │ Capacity │ Status │ │ 1st │ http://minio{1...2}/data{1...4} │ 439 GiB (used) / 561 GiB (total) │ Draining(Canceled) │ └─────┴─────────────────────────────────┴──────────────────────────────────┴────────────────────┘ ```
2022-01-10 12:07:49 -05:00
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminRouter.Methods(http.MethodPost).Path(adminVersion+"/pools/decommission").HandlerFunc(adminMiddleware(adminAPI.StartDecommission, traceAllFlag)).Queries("pool", "{pool:.*}")
adminRouter.Methods(http.MethodPost).Path(adminVersion+"/pools/cancel").HandlerFunc(adminMiddleware(adminAPI.CancelDecommission, traceAllFlag)).Queries("pool", "{pool:.*}")
feat: introduce pool-level rebalance (#15483)
2022-10-25 15:36:57 -04:00
// Rebalance operations
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminRouter.Methods(http.MethodPost).Path(adminVersion + "/rebalance/start").HandlerFunc(adminMiddleware(adminAPI.RebalanceStart, traceAllFlag))
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/rebalance/status").HandlerFunc(adminMiddleware(adminAPI.RebalanceStatus, traceAllFlag))
adminRouter.Methods(http.MethodPost).Path(adminVersion + "/rebalance/stop").HandlerFunc(adminMiddleware(adminAPI.RebalanceStop, traceAllFlag))
fix: change policies API to return and take struct (#9181) This allows for order guarantees in returned values can be consumed safely by the caller to avoid any additional parsing and validation. Fixes #9171
2020-04-07 22:30:59 -04:00
}
Introduce STS client grants API and OPA policy integration (#6168) This PR introduces two new features - AWS STS compatible STS API named AssumeRoleWithClientGrants ``` POST /?Action=AssumeRoleWithClientGrants&Token=<jwt> ``` This API endpoint returns temporary access credentials, access tokens signature types supported by this API - RSA keys - ECDSA keys Fetches the required public key from the JWKS endpoints, provides them as rsa or ecdsa public keys. - External policy engine support, in this case OPA policy engine - Credentials are stored on disks
2018-10-09 17:00:01 -04:00
Combine profiling start/stop APIs into one (#14662) Take profile duration as a query parameter for profile API
2022-04-08 15:44:35 -04:00
// Profiling operations - deprecated API
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminRouter.Methods(http.MethodPost).Path(adminVersion+"/profiling/start").HandlerFunc(adminMiddleware(adminAPI.StartProfilingHandler, traceAllFlag, noObjLayerFlag)).
fix: change policies API to return and take struct (#9181) This allows for order guarantees in returned values can be consumed safely by the caller to avoid any additional parsing and validation. Fixes #9171
2020-04-07 22:30:59 -04:00
Queries("profilerType", "{profilerType:.*}")
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/profiling/download").HandlerFunc(adminMiddleware(adminAPI.DownloadProfilingHandler, traceAllFlag, noObjLayerFlag))
Combine profiling start/stop APIs into one (#14662) Take profile duration as a query parameter for profile API
2022-04-08 15:44:35 -04:00
// Profiling operations
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminRouter.Methods(http.MethodPost).Path(adminVersion + "/profile").HandlerFunc(adminMiddleware(adminAPI.ProfileHandler, traceAllFlag, noObjLayerFlag))
Add ListUsers API to list all configured users in IAM (#6619)
2018-10-13 03:18:43 -04:00
fix: change policies API to return and take struct (#9181) This allows for order guarantees in returned values can be consumed safely by the caller to avoid any additional parsing and validation. Fixes #9171
2020-04-07 22:30:59 -04:00
// Config KV operations.
if enableConfigOps {
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminRouter.Methods(http.MethodGet).Path(adminVersion+"/get-config-kv").HandlerFunc(adminMiddleware(adminAPI.GetConfigKVHandler)).Queries("key", "{key:.*}")
adminRouter.Methods(http.MethodPut).Path(adminVersion + "/set-config-kv").HandlerFunc(adminMiddleware(adminAPI.SetConfigKVHandler))
adminRouter.Methods(http.MethodDelete).Path(adminVersion + "/del-config-kv").HandlerFunc(adminMiddleware(adminAPI.DelConfigKVHandler))
allow config help in gateway mode (#9356) allow `mc admin config set mygateway/ audit_webhook --env` to fetch the documentation as needed, this is just to ensure that our users can still access the relevant ENV docs while running in gateway mode.
2020-04-16 17:49:12 -04:00
}
// Enable config help in all modes.
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminRouter.Methods(http.MethodGet).Path(adminVersion+"/help-config-kv").HandlerFunc(adminMiddleware(adminAPI.HelpConfigKVHandler, traceAllFlag)).Queries("subSys", "{subSys:.*}", "key", "{key:.*}")
allow config help in gateway mode (#9356) allow `mc admin config set mygateway/ audit_webhook --env` to fetch the documentation as needed, this is just to ensure that our users can still access the relevant ENV docs while running in gateway mode.
2020-04-16 17:49:12 -04:00
// Config KV history operations.
if enableConfigOps {
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminRouter.Methods(http.MethodGet).Path(adminVersion+"/list-config-history-kv").HandlerFunc(adminMiddleware(adminAPI.ListConfigHistoryKVHandler, traceAllFlag)).Queries("count", "{count:[0-9]+}")
adminRouter.Methods(http.MethodDelete).Path(adminVersion+"/clear-config-history-kv").HandlerFunc(adminMiddleware(adminAPI.ClearConfigHistoryKVHandler)).Queries("restoreId", "{restoreId:.*}")
adminRouter.Methods(http.MethodPut).Path(adminVersion+"/restore-config-history-kv").HandlerFunc(adminMiddleware(adminAPI.RestoreConfigHistoryKVHandler)).Queries("restoreId", "{restoreId:.*}")
fix: change policies API to return and take struct (#9181) This allows for order guarantees in returned values can be consumed safely by the caller to avoid any additional parsing and validation. Fixes #9171
2020-04-07 22:30:59 -04:00
}
Add canned policy support (#6637) This PR adds an additional API where we can create a new set of canned policies which can be used with one or many users.
2018-10-16 15:48:19 -04:00
add gocritic/ruleguard checks back again, cleanup code. (#13665) - remove some duplicated code - reported a bug, separately fixed in #13664 - using strings.ReplaceAll() when needed - using filepath.ToSlash() use when needed - remove all non-Go style comments from the codebase Co-authored-by: Aditya Manthramurthy <donatello@users.noreply.github.com>
2021-11-16 12:28:29 -05:00
// Config import/export bulk operations
fix: change policies API to return and take struct (#9181) This allows for order guarantees in returned values can be consumed safely by the caller to avoid any additional parsing and validation. Fixes #9171
2020-04-07 22:30:59 -04:00
if enableConfigOps {
// Get config
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/config").HandlerFunc(adminMiddleware(adminAPI.GetConfigHandler))
fix: change policies API to return and take struct (#9181) This allows for order guarantees in returned values can be consumed safely by the caller to avoid any additional parsing and validation. Fixes #9171
2020-04-07 22:30:59 -04:00
// Set config
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminRouter.Methods(http.MethodPut).Path(adminVersion + "/config").HandlerFunc(adminMiddleware(adminAPI.SetConfigHandler))
fix: change policies API to return and take struct (#9181) This allows for order guarantees in returned values can be consumed safely by the caller to avoid any additional parsing and validation. Fixes #9171
2020-04-07 22:30:59 -04:00
}
Add canned policy support (#6637) This PR adds an additional API where we can create a new set of canned policies which can be used with one or many users.
2018-10-16 15:48:19 -04:00
fix: add IAM dummy store for gateway operations (#12670) with console addition users cannot login with root credentials without etcd persistent layer, allow a dummy store such that such functionalities can be supported when running as non-persistent manner, this enables all calls and operations.
2021-07-10 11:32:52 -04:00
// -- IAM APIs --
Add service account type in IAM (#9029)
2020-03-17 13:36:13 -04:00
fix: add IAM dummy store for gateway operations (#12670) with console addition users cannot login with root credentials without etcd persistent layer, allow a dummy store such that such functionalities can be supported when running as non-persistent manner, this enables all calls and operations.
2021-07-10 11:32:52 -04:00
// Add policy IAM
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminRouter.Methods(http.MethodPut).Path(adminVersion+"/add-canned-policy").HandlerFunc(adminMiddleware(adminAPI.AddCannedPolicy, traceAllFlag)).Queries("name", "{name:.*}")
Add InfoCannedPolicy API to fetch only necessary policy (#8307) This PR adds - InfoCannedPolicy() API for efficiency in fetching policies - Send group memberships for LDAPUser if available
2019-09-26 14:23:13 -04:00
fix: add IAM dummy store for gateway operations (#12670) with console addition users cannot login with root credentials without etcd persistent layer, allow a dummy store such that such functionalities can be supported when running as non-persistent manner, this enables all calls and operations.
2021-07-10 11:32:52 -04:00
// Add user IAM
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/accountinfo").HandlerFunc(adminMiddleware(adminAPI.AccountInfoHandler, traceAllFlag))
add missing admin actions, enhance AccountUsageInfo (#9607)
2020-05-15 21:16:45 -04:00
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminRouter.Methods(http.MethodPut).Path(adminVersion+"/add-user").HandlerFunc(adminMiddleware(adminAPI.AddUser)).Queries("accessKey", "{accessKey:.*}")
Introduce STS client grants API and OPA policy integration (#6168) This PR introduces two new features - AWS STS compatible STS API named AssumeRoleWithClientGrants ``` POST /?Action=AssumeRoleWithClientGrants&Token=<jwt> ``` This API endpoint returns temporary access credentials, access tokens signature types supported by this API - RSA keys - ECDSA keys Fetches the required public key from the JWKS endpoints, provides them as rsa or ecdsa public keys. - External policy engine support, in this case OPA policy engine - Credentials are stored on disks
2018-10-09 17:00:01 -04:00
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminRouter.Methods(http.MethodPut).Path(adminVersion+"/set-user-status").HandlerFunc(adminMiddleware(adminAPI.SetUserStatus)).Queries("accessKey", "{accessKey:.*}").Queries("status", "{status:.*}")
Set the policy mapping for a user or group (#8036) Add API to set policy mapping for a user or group Contains a breaking Admin APIs change. - Also enforce all applicable policies - Removes the previous /set-user-policy API Bump up peerRESTVersion Add get user info API to show groups of a user
2019-08-13 16:41:06 -04:00
fix: add IAM dummy store for gateway operations (#12670) with console addition users cannot login with root credentials without etcd persistent layer, allow a dummy store such that such functionalities can be supported when running as non-persistent manner, this enables all calls and operations.
2021-07-10 11:32:52 -04:00
// Service accounts ops
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminRouter.Methods(http.MethodPut).Path(adminVersion + "/add-service-account").HandlerFunc(adminMiddleware(adminAPI.AddServiceAccount))
adminRouter.Methods(http.MethodPost).Path(adminVersion+"/update-service-account").HandlerFunc(adminMiddleware(adminAPI.UpdateServiceAccount)).Queries("accessKey", "{accessKey:.*}")
adminRouter.Methods(http.MethodGet).Path(adminVersion+"/info-service-account").HandlerFunc(adminMiddleware(adminAPI.InfoServiceAccount)).Queries("accessKey", "{accessKey:.*}")
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/list-service-accounts").HandlerFunc(adminMiddleware(adminAPI.ListServiceAccounts))
adminRouter.Methods(http.MethodDelete).Path(adminVersion+"/delete-service-account").HandlerFunc(adminMiddleware(adminAPI.DeleteServiceAccount)).Queries("accessKey", "{accessKey:.*}")
Introduce STS client grants API and OPA policy integration (#6168) This PR introduces two new features - AWS STS compatible STS API named AssumeRoleWithClientGrants ``` POST /?Action=AssumeRoleWithClientGrants&Token=<jwt> ``` This API endpoint returns temporary access credentials, access tokens signature types supported by this API - RSA keys - ECDSA keys Fetches the required public key from the JWKS endpoints, provides them as rsa or ecdsa public keys. - External policy engine support, in this case OPA policy engine - Credentials are stored on disks
2018-10-09 17:00:01 -04:00
Implement STS account info API (#16115)
2022-12-13 11:38:50 -05:00
// STS accounts ops
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminRouter.Methods(http.MethodGet).Path(adminVersion+"/temporary-account-info").HandlerFunc(adminMiddleware(adminAPI.TemporaryAccountInfo)).Queries("accessKey", "{accessKey:.*}")
Implement STS account info API (#16115)
2022-12-13 11:38:50 -05:00
fix: add IAM dummy store for gateway operations (#12670) with console addition users cannot login with root credentials without etcd persistent layer, allow a dummy store such that such functionalities can be supported when running as non-persistent manner, this enables all calls and operations.
2021-07-10 11:32:52 -04:00
// Info policy IAM latest
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminRouter.Methods(http.MethodGet).Path(adminVersion+"/info-canned-policy").HandlerFunc(adminMiddleware(adminAPI.InfoCannedPolicy)).Queries("name", "{name:.*}")
fix: add IAM dummy store for gateway operations (#12670) with console addition users cannot login with root credentials without etcd persistent layer, allow a dummy store such that such functionalities can be supported when running as non-persistent manner, this enables all calls and operations.
2021-07-10 11:32:52 -04:00
// List policies latest
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminRouter.Methods(http.MethodGet).Path(adminVersion+"/list-canned-policies").HandlerFunc(adminMiddleware(adminAPI.ListBucketPolicies)).Queries("bucket", "{bucket:.*}")
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/list-canned-policies").HandlerFunc(adminMiddleware(adminAPI.ListCannedPolicies))
Add IAM groups support (#7981) This change adds admin APIs and IAM subsystem APIs to: - add or remove members to a group (group addition and deletion is implicit on add and remove) - enable/disable a group - list and fetch group info
2019-08-02 17:25:00 -04:00
Add API to fetch policy user/group associations (#16239)
2022-12-19 13:37:03 -05:00
// Builtin IAM policy associations
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/idp/builtin/policy-entities").HandlerFunc(adminMiddleware(adminAPI.ListPolicyMappingEntities))
Add API to fetch policy user/group associations (#16239)
2022-12-19 13:37:03 -05:00
fix: add IAM dummy store for gateway operations (#12670) with console addition users cannot login with root credentials without etcd persistent layer, allow a dummy store such that such functionalities can be supported when running as non-persistent manner, this enables all calls and operations.
2021-07-10 11:32:52 -04:00
// Remove policy IAM
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminRouter.Methods(http.MethodDelete).Path(adminVersion+"/remove-canned-policy").HandlerFunc(adminMiddleware(adminAPI.RemoveCannedPolicy)).Queries("name", "{name:.*}")
Add IAM groups support (#7981) This change adds admin APIs and IAM subsystem APIs to: - add or remove members to a group (group addition and deletion is implicit on add and remove) - enable/disable a group - list and fetch group info
2019-08-02 17:25:00 -04:00
fix: add IAM dummy store for gateway operations (#12670) with console addition users cannot login with root credentials without etcd persistent layer, allow a dummy store such that such functionalities can be supported when running as non-persistent manner, this enables all calls and operations.
2021-07-10 11:32:52 -04:00
// Set user or group policy
adminRouter.Methods(http.MethodPut).Path(adminVersion+"/set-user-or-group-policy").
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
HandlerFunc(adminMiddleware(adminAPI.SetPolicyForUserOrGroup)).
fix: add IAM dummy store for gateway operations (#12670) with console addition users cannot login with root credentials without etcd persistent layer, allow a dummy store such that such functionalities can be supported when running as non-persistent manner, this enables all calls and operations.
2021-07-10 11:32:52 -04:00
Queries("policyName", "{policyName:.*}", "userOrGroup", "{userOrGroup:.*}", "isGroup", "{isGroup:true|false}")
Add IAM groups support (#7981) This change adds admin APIs and IAM subsystem APIs to: - add or remove members to a group (group addition and deletion is implicit on add and remove) - enable/disable a group - list and fetch group info
2019-08-02 17:25:00 -04:00
Fix locking in policy attach API (#17426) For policy attach/detach API to work correctly the server should hold a lock before reading existing policy mapping and until after writing the updated policy mapping. This is fixed in this change. A site replication bug, where LDAP policy attach/detach were not correctly propagated is also fixed in this change. Bonus: Additionally, the server responds with the actual (or net) changes performed in the attach/detach API call. For e.g. if a user already has policy A applied, and a call to attach policies A and B is performed, the server will respond that B was attached successfully.
2023-06-22 01:44:50 -04:00
// Attach/Detach policies to/from user or group
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminRouter.Methods(http.MethodPost).Path(adminVersion + "/idp/builtin/policy/{operation}").HandlerFunc(adminMiddleware(adminAPI.AttachDetachPolicyBuiltin))
Add endpoints for managing IAM policies (#15897) Co-authored-by: Taran <taran@minio.io> Co-authored-by: ¨taran-p¨ <¨taran@minio.io¨> Co-authored-by: Aditya Manthramurthy <donatello@users.noreply.github.com>
2022-12-13 15:13:23 -05:00
fix: add IAM dummy store for gateway operations (#12670) with console addition users cannot login with root credentials without etcd persistent layer, allow a dummy store such that such functionalities can be supported when running as non-persistent manner, this enables all calls and operations.
2021-07-10 11:32:52 -04:00
// Remove user IAM
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminRouter.Methods(http.MethodDelete).Path(adminVersion+"/remove-user").HandlerFunc(adminMiddleware(adminAPI.RemoveUser)).Queries("accessKey", "{accessKey:.*}")
Add IAM groups support (#7981) This change adds admin APIs and IAM subsystem APIs to: - add or remove members to a group (group addition and deletion is implicit on add and remove) - enable/disable a group - list and fetch group info
2019-08-02 17:25:00 -04:00
fix: add IAM dummy store for gateway operations (#12670) with console addition users cannot login with root credentials without etcd persistent layer, allow a dummy store such that such functionalities can be supported when running as non-persistent manner, this enables all calls and operations.
2021-07-10 11:32:52 -04:00
// List users
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminRouter.Methods(http.MethodGet).Path(adminVersion+"/list-users").HandlerFunc(adminMiddleware(adminAPI.ListBucketUsers)).Queries("bucket", "{bucket:.*}")
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/list-users").HandlerFunc(adminMiddleware(adminAPI.ListUsers))
Enable admin users API on gateway (#6659) This is only enabled when etcd is enabled, healing is only enabled for erasure coded backend.
2018-10-17 20:25:16 -04:00
fix: add IAM dummy store for gateway operations (#12670) with console addition users cannot login with root credentials without etcd persistent layer, allow a dummy store such that such functionalities can be supported when running as non-persistent manner, this enables all calls and operations.
2021-07-10 11:32:52 -04:00
// User info
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminRouter.Methods(http.MethodGet).Path(adminVersion+"/user-info").HandlerFunc(adminMiddleware(adminAPI.GetUserInfo)).Queries("accessKey", "{accessKey:.*}")
fix: add IAM dummy store for gateway operations (#12670) with console addition users cannot login with root credentials without etcd persistent layer, allow a dummy store such that such functionalities can be supported when running as non-persistent manner, this enables all calls and operations.
2021-07-10 11:32:52 -04:00
// Add/Remove members from group
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminRouter.Methods(http.MethodPut).Path(adminVersion + "/update-group-members").HandlerFunc(adminMiddleware(adminAPI.UpdateGroupMembers))
fix: change policies API to return and take struct (#9181) This allows for order guarantees in returned values can be consumed safely by the caller to avoid any additional parsing and validation. Fixes #9171
2020-04-07 22:30:59 -04:00
fix: add IAM dummy store for gateway operations (#12670) with console addition users cannot login with root credentials without etcd persistent layer, allow a dummy store such that such functionalities can be supported when running as non-persistent manner, this enables all calls and operations.
2021-07-10 11:32:52 -04:00
// Get Group
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminRouter.Methods(http.MethodGet).Path(adminVersion+"/group").HandlerFunc(adminMiddleware(adminAPI.GetGroup)).Queries("group", "{group:.*}")
fix: change policies API to return and take struct (#9181) This allows for order guarantees in returned values can be consumed safely by the caller to avoid any additional parsing and validation. Fixes #9171
2020-04-07 22:30:59 -04:00
fix: add IAM dummy store for gateway operations (#12670) with console addition users cannot login with root credentials without etcd persistent layer, allow a dummy store such that such functionalities can be supported when running as non-persistent manner, this enables all calls and operations.
2021-07-10 11:32:52 -04:00
// List Groups
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/groups").HandlerFunc(adminMiddleware(adminAPI.ListGroups))
fix: change policies API to return and take struct (#9181) This allows for order guarantees in returned values can be consumed safely by the caller to avoid any additional parsing and validation. Fixes #9171
2020-04-07 22:30:59 -04:00
fix: add IAM dummy store for gateway operations (#12670) with console addition users cannot login with root credentials without etcd persistent layer, allow a dummy store such that such functionalities can be supported when running as non-persistent manner, this enables all calls and operations.
2021-07-10 11:32:52 -04:00
// Set Group Status
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminRouter.Methods(http.MethodPut).Path(adminVersion+"/set-group-status").HandlerFunc(adminMiddleware(adminAPI.SetGroupStatus)).Queries("group", "{group:.*}").Queries("status", "{status:.*}")
fix: change policies API to return and take struct (#9181) This allows for order guarantees in returned values can be consumed safely by the caller to avoid any additional parsing and validation. Fixes #9171
2020-04-07 22:30:59 -04:00
Add APIs to import/export IAM data (#15014)
2022-06-23 12:25:15 -04:00
// Export IAM info to zipped file
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/export-iam").HandlerFunc(adminMiddleware(adminAPI.ExportIAM, noGZFlag))
Add APIs to import/export IAM data (#15014)
2022-06-23 12:25:15 -04:00
// Import IAM info
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminRouter.Methods(http.MethodPut).Path(adminVersion + "/import-iam").HandlerFunc(adminMiddleware(adminAPI.ImportIAM, noGZFlag))
Add APIs to import/export IAM data (#15014)
2022-06-23 12:25:15 -04:00
Add external IDP management Admin API for OpenID (#15152)
2022-07-05 21:18:04 -04:00
// IDentity Provider configuration APIs
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminRouter.Methods(http.MethodPut).Path(adminVersion + "/idp-config/{type}/{name}").HandlerFunc(adminMiddleware(adminAPI.AddIdentityProviderCfg))
adminRouter.Methods(http.MethodPost).Path(adminVersion + "/idp-config/{type}/{name}").HandlerFunc(adminMiddleware(adminAPI.UpdateIdentityProviderCfg))
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/idp-config/{type}").HandlerFunc(adminMiddleware(adminAPI.ListIdentityProviderCfg))
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/idp-config/{type}/{name}").HandlerFunc(adminMiddleware(adminAPI.GetIdentityProviderCfg))
adminRouter.Methods(http.MethodDelete).Path(adminVersion + "/idp-config/{type}/{name}").HandlerFunc(adminMiddleware(adminAPI.DeleteIdentityProviderCfg))
Add external IDP management Admin API for OpenID (#15152)
2022-07-05 21:18:04 -04:00
Add APIs to create and list access keys for LDAP (#18402)
2023-12-15 16:00:43 -05:00
// LDAP specific service accounts ops
adminRouter.Methods(http.MethodPut).Path(adminVersion + "/idp/ldap/add-service-account").HandlerFunc(adminMiddleware(adminAPI.AddServiceAccountLDAP))
adminRouter.Methods(http.MethodGet).Path(adminVersion+"/idp/ldap/list-access-keys").
add `ListAccessKeysLDAPBulk` API to list accessKeys for multiple/all LDAP users (#19835)
2024-06-25 17:21:28 -04:00
HandlerFunc(adminMiddleware(adminAPI.ListAccessKeysLDAP)).Queries("userDN", "{userDN:.*}", "listType", "{listType:.*}")
adminRouter.Methods(http.MethodGet).Path(adminVersion+"/idp/ldap/list-access-keys-bulk").
HandlerFunc(adminMiddleware(adminAPI.ListAccessKeysLDAPBulk)).Queries("listType", "{listType:.*}")
Add APIs to create and list access keys for LDAP (#18402)
2023-12-15 16:00:43 -05:00
Add LDAP policy entities API (#15908)
2022-11-07 17:35:09 -05:00
// LDAP IAM operations
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/idp/ldap/policy-entities").HandlerFunc(adminMiddleware(adminAPI.ListLDAPPolicyMappingEntities))
adminRouter.Methods(http.MethodPost).Path(adminVersion + "/idp/ldap/policy/{operation}").HandlerFunc(adminMiddleware(adminAPI.AttachDetachPolicyLDAP))
Add external IDP management Admin API for OpenID (#15152)
2022-07-05 21:18:04 -04:00
// -- END IAM APIs --
feat: Single drive XL implementation (#14970) Main motivation is move towards a common backend format for all different types of modes in MinIO, allowing for a simpler code and predictable behavior across all features. This PR also brings features such as versioning, replication, transitioning to single drive setups.
2022-05-30 13:58:37 -04:00
// GetBucketQuotaConfig
adminRouter.Methods(http.MethodGet).Path(adminVersion+"/get-bucket-quota").HandlerFunc(
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminMiddleware(adminAPI.GetBucketQuotaConfigHandler)).Queries("bucket", "{bucket:.*}")
feat: Single drive XL implementation (#14970) Main motivation is move towards a common backend format for all different types of modes in MinIO, allowing for a simpler code and predictable behavior across all features. This PR also brings features such as versioning, replication, transitioning to single drive setups.
2022-05-30 13:58:37 -04:00
// PutBucketQuotaConfig
adminRouter.Methods(http.MethodPut).Path(adminVersion+"/set-bucket-quota").HandlerFunc(
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminMiddleware(adminAPI.PutBucketQuotaConfigHandler)).Queries("bucket", "{bucket:.*}")
feat: Single drive XL implementation (#14970) Main motivation is move towards a common backend format for all different types of modes in MinIO, allowing for a simpler code and predictable behavior across all features. This PR also brings features such as versioning, replication, transitioning to single drive setups.
2022-05-30 13:58:37 -04:00
// Bucket replication operations
// GetBucketTargetHandler
adminRouter.Methods(http.MethodGet).Path(adminVersion+"/list-remote-targets").HandlerFunc(
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminMiddleware(adminAPI.ListRemoteTargetsHandler)).Queries("bucket", "{bucket:.*}", "type", "{type:.*}")
feat: Single drive XL implementation (#14970) Main motivation is move towards a common backend format for all different types of modes in MinIO, allowing for a simpler code and predictable behavior across all features. This PR also brings features such as versioning, replication, transitioning to single drive setups.
2022-05-30 13:58:37 -04:00
// SetRemoteTargetHandler
adminRouter.Methods(http.MethodPut).Path(adminVersion+"/set-remote-target").HandlerFunc(
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminMiddleware(adminAPI.SetRemoteTargetHandler)).Queries("bucket", "{bucket:.*}")
feat: Single drive XL implementation (#14970) Main motivation is move towards a common backend format for all different types of modes in MinIO, allowing for a simpler code and predictable behavior across all features. This PR also brings features such as versioning, replication, transitioning to single drive setups.
2022-05-30 13:58:37 -04:00
// RemoveRemoteTargetHandler
adminRouter.Methods(http.MethodDelete).Path(adminVersion+"/remove-remote-target").HandlerFunc(
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminMiddleware(adminAPI.RemoveRemoteTargetHandler)).Queries("bucket", "{bucket:.*}", "arn", "{arn:.*}")
feat: add API to return list of objects waiting to be replicated (#15091)
2022-07-21 14:05:44 -04:00
// ReplicationDiff - MinIO extension API
adminRouter.Methods(http.MethodPost).Path(adminVersion+"/replication/diff").HandlerFunc(
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminMiddleware(adminAPI.ReplicationDiffHandler)).Queries("bucket", "{bucket:.*}")
replication: Simplify mrf requeueing and add backlog handler (#17171) Simplify MRF queueing and add backlog handler - Limit re-tries to 3 to avoid repeated re-queueing. Fall offs to be re-tried when the scanner revisits this object or upon access. - Change MRF to have each node process only its MRF entries. - Collect MRF backlog by the node to allow for current backlog visibility
2023-07-13 02:51:33 -04:00
// ReplicationMRFHandler - MinIO extension API
adminRouter.Methods(http.MethodGet).Path(adminVersion+"/replication/mrf").HandlerFunc(
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminMiddleware(adminAPI.ReplicationMRFHandler)).Queries("bucket", "{bucket:.*}")
feat: Single drive XL implementation (#14970) Main motivation is move towards a common backend format for all different types of modes in MinIO, allowing for a simpler code and predictable behavior across all features. This PR also brings features such as versioning, replication, transitioning to single drive setups.
2022-05-30 13:58:37 -04:00
feat: implement support batch replication (#15554)
2022-10-03 05:10:15 -04:00
// Batch job operations
adminRouter.Methods(http.MethodPost).Path(adminVersion + "/start-job").HandlerFunc(
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminMiddleware(adminAPI.StartBatchJob))
feat: implement support batch replication (#15554)
2022-10-03 05:10:15 -04:00
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/list-jobs").HandlerFunc(
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminMiddleware(adminAPI.ListBatchJobs))
feat: implement support batch replication (#15554)
2022-10-03 05:10:15 -04:00
Add batch status API (#19679) Currently the status of a completed or failed batch is held in the memory, a simple restart will lose the status and the user will not have any visibility of the job that was long running. In addition to the metrics, add a new API that reads the batch status from the drives. A batch job will be cleaned up three days after completion. Also add the batch type in the batch id, the reason is that the batch job request is removed immediately when the job is finished, then we do not know the type of batch job anymore, hence a difficulty to locate the job report
2024-07-02 04:17:52 -04:00
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/status-job").HandlerFunc(
adminMiddleware(adminAPI.BatchJobStatus))
feat: implement support batch replication (#15554)
2022-10-03 05:10:15 -04:00
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/describe-job").HandlerFunc(
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminMiddleware(adminAPI.DescribeBatchJob))
Add support for batch job cancellation (#16843)
2023-03-18 02:42:43 -04:00
adminRouter.Methods(http.MethodDelete).Path(adminVersion + "/cancel-job").HandlerFunc(
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminMiddleware(adminAPI.CancelBatchJob))
feat: implement support batch replication (#15554)
2022-10-03 05:10:15 -04:00
Add import/export APIs to migrate bucket metadata (#14929)
2022-06-18 09:55:39 -04:00
// Bucket migration operations
// ExportBucketMetaHandler
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/export-bucket-metadata").HandlerFunc(
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminMiddleware(adminAPI.ExportBucketMetadataHandler))
Add import/export APIs to migrate bucket metadata (#14929)
2022-06-18 09:55:39 -04:00
// ImportBucketMetaHandler
adminRouter.Methods(http.MethodPut).Path(adminVersion + "/import-bucket-metadata").HandlerFunc(
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminMiddleware(adminAPI.ImportBucketMetadataHandler))
Add import/export APIs to migrate bucket metadata (#14929)
2022-06-18 09:55:39 -04:00
feat: Single drive XL implementation (#14970) Main motivation is move towards a common backend format for all different types of modes in MinIO, allowing for a simpler code and predictable behavior across all features. This PR also brings features such as versioning, replication, transitioning to single drive setups.
2022-05-30 13:58:37 -04:00
// Remote Tier management operations
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminRouter.Methods(http.MethodPut).Path(adminVersion + "/tier").HandlerFunc(adminMiddleware(adminAPI.AddTierHandler))
adminRouter.Methods(http.MethodPost).Path(adminVersion + "/tier/{tier}").HandlerFunc(adminMiddleware(adminAPI.EditTierHandler))
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/tier").HandlerFunc(adminMiddleware(adminAPI.ListTierHandler))
adminRouter.Methods(http.MethodDelete).Path(adminVersion + "/tier/{tier}").HandlerFunc(adminMiddleware(adminAPI.RemoveTierHandler))
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/tier/{tier}").HandlerFunc(adminMiddleware(adminAPI.VerifyTierHandler))
feat: Single drive XL implementation (#14970) Main motivation is move towards a common backend format for all different types of modes in MinIO, allowing for a simpler code and predictable behavior across all features. This PR also brings features such as versioning, replication, transitioning to single drive setups.
2022-05-30 13:58:37 -04:00
// Tier stats
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/tier-stats").HandlerFunc(adminMiddleware(adminAPI.TierStatsHandler))
feat: Single drive XL implementation (#14970) Main motivation is move towards a common backend format for all different types of modes in MinIO, allowing for a simpler code and predictable behavior across all features. This PR also brings features such as versioning, replication, transitioning to single drive setups.
2022-05-30 13:58:37 -04:00
// Cluster Replication APIs
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminRouter.Methods(http.MethodPut).Path(adminVersion + "/site-replication/add").HandlerFunc(adminMiddleware(adminAPI.SiteReplicationAdd))
adminRouter.Methods(http.MethodPut).Path(adminVersion + "/site-replication/remove").HandlerFunc(adminMiddleware(adminAPI.SiteReplicationRemove))
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/site-replication/info").HandlerFunc(adminMiddleware(adminAPI.SiteReplicationInfo))
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/site-replication/metainfo").HandlerFunc(adminMiddleware(adminAPI.SiteReplicationMetaInfo))
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/site-replication/status").HandlerFunc(adminMiddleware(adminAPI.SiteReplicationStatus))
adminRouter.Methods(http.MethodPost).Path(adminVersion + adminAPISiteReplicationDevNull).HandlerFunc(adminMiddleware(adminAPI.SiteReplicationDevNull, noObjLayerFlag))
adminRouter.Methods(http.MethodPost).Path(adminVersion + adminAPISiteReplicationNetPerf).HandlerFunc(adminMiddleware(adminAPI.SiteReplicationNetPerf, noObjLayerFlag))
adminRouter.Methods(http.MethodPut).Path(adminVersion + "/site-replication/peer/join").HandlerFunc(adminMiddleware(adminAPI.SRPeerJoin))
adminRouter.Methods(http.MethodPut).Path(adminVersion+"/site-replication/peer/bucket-ops").HandlerFunc(adminMiddleware(adminAPI.SRPeerBucketOps)).Queries("bucket", "{bucket:.*}").Queries("operation", "{operation:.*}")
adminRouter.Methods(http.MethodPut).Path(adminVersion + "/site-replication/peer/iam-item").HandlerFunc(adminMiddleware(adminAPI.SRPeerReplicateIAMItem))
adminRouter.Methods(http.MethodPut).Path(adminVersion + "/site-replication/peer/bucket-meta").HandlerFunc(adminMiddleware(adminAPI.SRPeerReplicateBucketItem))
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/site-replication/peer/idp-settings").HandlerFunc(adminMiddleware(adminAPI.SRPeerGetIDPSettings))
adminRouter.Methods(http.MethodPut).Path(adminVersion + "/site-replication/edit").HandlerFunc(adminMiddleware(adminAPI.SiteReplicationEdit))
adminRouter.Methods(http.MethodPut).Path(adminVersion + "/site-replication/peer/edit").HandlerFunc(adminMiddleware(adminAPI.SRPeerEdit))
adminRouter.Methods(http.MethodPut).Path(adminVersion + "/site-replication/peer/remove").HandlerFunc(adminMiddleware(adminAPI.SRPeerRemove))
adminRouter.Methods(http.MethodPut).Path(adminVersion+"/site-replication/resync/op").HandlerFunc(adminMiddleware(adminAPI.SiteReplicationResyncOp)).Queries("operation", "{operation:.*}")
Replicate Expiry ILM configs while site replication (#18130) Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>
2023-11-21 12:48:06 -05:00
adminRouter.Methods(http.MethodPut).Path(adminVersion + "/site-replication/state/edit").HandlerFunc(adminMiddleware(adminAPI.SRStateEdit))
remove/deprecate crawler disable environment (#11214) with changes present to automatically throttle crawler at runtime, there is no need to have an environment value to disable crawling. crawling is a fundamental piece for healing, lifecycle and many other features there is no good reason anyone would need to disable this on a production system. * Apply suggestions from code review
2021-01-04 12:43:31 -05:00
Support bucket versioning (#9377) - Implement a new xl.json 2.0.0 format to support, this moves the entire marshaling logic to POSIX layer, top layer always consumes a common FileInfo construct which simplifies the metadata reads. - Implement list object versions - Migrate to siphash from crchash for new deployments for object placements. Fixes #2111
2020-06-12 23:04:01 -04:00
if globalIsDistErasure {
fix: expire locks only on participating lockers (#11335) additionally also add a new ForceUnlock API, to allow forcibly unlocking locks if possible.
2021-01-25 13:01:27 -05:00
// Top locks
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/top/locks").HandlerFunc(adminMiddleware(adminAPI.TopLocksHandler))
fix: expire locks only on participating lockers (#11335) additionally also add a new ForceUnlock API, to allow forcibly unlocking locks if possible.
2021-01-25 13:01:27 -05:00
// Force unlocks paths
adminRouter.Methods(http.MethodPost).Path(adminVersion+"/force-unlock").
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
Queries("paths", "{paths:.*}").HandlerFunc(adminMiddleware(adminAPI.ForceUnlockHandler))
fix: change policies API to return and take struct (#9181) This allows for order guarantees in returned values can be consumed safely by the caller to avoid any additional parsing and validation. Fixes #9171
2020-04-07 22:30:59 -04:00
}
Top Locks command implementation (#7052) API to list locks used in distributed XL mode
2019-01-24 10:22:14 -05:00
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminRouter.Methods(http.MethodPost).Path(adminVersion + "/speedtest").HandlerFunc(adminMiddleware(adminAPI.ObjectSpeedTestHandler, noGZFlag))
adminRouter.Methods(http.MethodPost).Path(adminVersion + "/speedtest/object").HandlerFunc(adminMiddleware(adminAPI.ObjectSpeedTestHandler, noGZFlag))
adminRouter.Methods(http.MethodPost).Path(adminVersion + "/speedtest/drive").HandlerFunc(adminMiddleware(adminAPI.DriveSpeedtestHandler, noGZFlag))
adminRouter.Methods(http.MethodPost).Path(adminVersion + "/speedtest/net").HandlerFunc(adminMiddleware(adminAPI.NetperfHandler, noGZFlag))
adminRouter.Methods(http.MethodPost).Path(adminVersion + "/speedtest/site").HandlerFunc(adminMiddleware(adminAPI.SitePerfHandler, noGZFlag))
feat: support perf client to show `TX` from client to server (#17718)
2023-08-10 10:14:46 -04:00
adminRouter.Methods(http.MethodPost).Path(adminVersion + adminAPIClientDevNull).HandlerFunc(adminMiddleware(adminAPI.ClientDevNull, noGZFlag))
adminRouter.Methods(http.MethodPost).Path(adminVersion + adminAPIClientDevExtraTime).HandlerFunc(adminMiddleware(adminAPI.ClientDevNullExtraTime, noGZFlag))
Server side speedtest implementation (#12750)
2021-07-27 15:55:56 -04:00
fix: change policies API to return and take struct (#9181) This allows for order guarantees in returned values can be consumed safely by the caller to avoid any additional parsing and validation. Fixes #9171
2020-04-07 22:30:59 -04:00
// HTTP Trace
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/trace").HandlerFunc(adminMiddleware(adminAPI.TraceHandler, noObjLayerFlag))
KMS Admin-API: add route and handler for KMS key info (#7955) This commit adds an admin API route and handler for requesting status information about a KMS key. Therefore, the client specifies the KMS key ID (when empty / not set the server takes the currently configured default key-ID) and the server tries to perform a dummy encryption, re-wrap and decryption operation. If all three succeed we know that the server can access the KMS and has permissions to generate, re-wrap and decrypt data keys (policy is set correctly).
2019-09-04 16:19:44 -04:00
fix: change policies API to return and take struct (#9181) This allows for order guarantees in returned values can be consumed safely by the caller to avoid any additional parsing and validation. Fixes #9171
2020-04-07 22:30:59 -04:00
// Console Logs
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/log").HandlerFunc(adminMiddleware(adminAPI.ConsoleLogHandler, traceAllFlag))
Add admin API to send console log messages (#7784) Utilized by mc admin console command.
2019-09-03 14:10:48 -04:00
fix: change policies API to return and take struct (#9181) This allows for order guarantees in returned values can be consumed safely by the caller to avoid any additional parsing and validation. Fixes #9171
2020-04-07 22:30:59 -04:00
// -- KMS APIs --
//
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
adminRouter.Methods(http.MethodPost).Path(adminVersion + "/kms/status").HandlerFunc(adminMiddleware(adminAPI.KMSStatusHandler, traceAllFlag))
adminRouter.Methods(http.MethodPost).Path(adminVersion+"/kms/key/create").HandlerFunc(adminMiddleware(adminAPI.KMSCreateKeyHandler, traceAllFlag)).Queries("key-id", "{key-id:.*}")
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/kms/key/status").HandlerFunc(adminMiddleware(adminAPI.KMSKeyStatusHandler, traceAllFlag))
KMS Admin-API: add route and handler for KMS key info (#7955) This commit adds an admin API route and handler for requesting status information about a KMS key. Therefore, the client specifies the KMS key ID (when empty / not set the server takes the currently configured default key-ID) and the server tries to perform a dummy encryption, re-wrap and decryption operation. If all three succeed we know that the server can access the KMS and has permissions to generate, re-wrap and decrypt data keys (policy is set correctly).
2019-09-04 16:19:44 -04:00
remove gateway completely (#15929)
2022-10-24 20:44:15 -04:00
// Keep obdinfo for backward compatibility with mc
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/obdinfo").
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
HandlerFunc(adminMiddleware(adminAPI.HealthInfoHandler))
remove gateway completely (#15929)
2022-10-24 20:44:15 -04:00
// -- Health API --
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/healthinfo").
Send AuditLog via new middleware fn for admin APIs (#17632) A new middleware function is added for admin handlers, including options for modifying certain behaviors. This admin middleware: - sets the handler context via reflection in the request and sends AuditLog - checks for object API availability (skipping it if a flag is passed) - enables gzip compression (skipping it if a flag is passed) - enables header tracing (adding body tracing if a flag is passed) While the new function is a middleware, due to the flags used for conditional behavior modification, which is used in each route registration call. To try to ensure that no regressions are introduced, the following changes were done mechanically mostly with `sed` and regexp: - Remove defer logger.AuditLog in admin handlers - Replace newContext() calls with r.Context() - Update admin routes registration calls Bonus: remove unused NetSpeedtestHandler Since the new adminMiddleware function checks for object layer presence by default, we need to pass the `noObjLayerFlag` explicitly to admin handlers that should work even when it is not available. The following admin handlers do not require it: - ServerInfoHandler - StartProfilingHandler - DownloadProfilingHandler - ProfileHandler - SiteReplicationDevNull - SiteReplicationNetPerf - TraceHandler For these handlers adminMiddleware does not check for the object layer presence (disabled by passing the `noObjLayerFlag`), and for all other handlers, the pre-check ensures that the handler is not called when the object layer is not available - the client would get a ErrServerNotInitialized and can retry later. This `noObjLayerFlag` is added based on existing behavior for these handlers only.
2023-07-13 17:52:21 -04:00
HandlerFunc(adminMiddleware(adminAPI.HealthInfoHandler))
Implement oboard diagnostics admin API (#9024) - Implement a graph algorithm to test network bandwidth from every node to every other node - Saturate any network bandwidth adaptively, accounting for slow and fast network capacity - Implement parallel drive OBD tests - Implement a paging mechanism for OBD test to provide periodic updates to client - Implement Sys, Process, Host, Mem OBD Infos
2020-03-27 00:07:39 -04:00
}
Return appropriate errors API versions changes across REST APIs (#8480) This PR adds code to appropriately handle versioning issues that come up quite constantly across our API changes. Currently we were also routing our requests wrong which sort of made it harder to write a consistent error handling code to appropriately reject or honor requests. This PR potentially fixes issues - old mc is used against new minio release which is incompatible returns an appropriate for client action. - any older servers talking to each other, report appropriate error - incompatible peer servers should report error and reject the calls with appropriate error
2019-11-04 12:30:59 -05:00
// If none of the routes match add default error handler routes
Remove redundant http.HandlerFunc type conversion. (#10576)
2020-09-28 16:33:49 -04:00
adminRouter.NotFoundHandler = httpTraceAll(errorResponseHandler)
ListObjects Metadata Caching (#10648) Design: https://gist.github.com/klauspost/025c09b48ed4a1293c917cecfabdf21c Gist of improvements: * Cross-server caching and listing will use the same data across servers and requests. * Lists can be arbitrarily resumed at a constant speed. * Metadata for all files scanned is stored for streaming retrieval. * The existing bloom filters controlled by the crawler is used for validating caches. * Concurrent requests for the same data (or parts of it) will not spawn additional walkers. * Listing a subdirectory of an existing recursive cache will use the cache. * All listing operations are fully streamable so the number of objects in a bucket no longer dictates the amount of memory. * Listings can be handled by any server within the cluster. * Caches are cleaned up when out of date or superseded by a more recent one.
2020-10-28 12:18:35 -04:00
adminRouter.MethodNotAllowedHandler = httpTraceAll(methodNotAllowedHandler("Admin"))
Add service API handler stubs for status, stop and restart (#3417)
2016-12-16 01:26:15 -05:00
}
Reference in New Issue Copy Permalink