Commit Graph

103 Commits

Author SHA1 Message Date
kannappanr 70b350c383
Remove DeploymentID from response headers (#7815)
Response headers need not contain deployment ID.
2019-07-01 12:22:01 -07:00
Krishna Srinivas 183ec094c4 Simplify HTTP trace related code (#7833) 2019-06-26 22:41:12 -07:00
Anis Elleuch 48f2c98052 admin: Add Background heal status info API (#7774)
This API returns the information related to the self healing routine.

For the moment, it returns:
- The total number of objects that are scanned
- The last time when an item was scanned
2019-06-25 16:42:24 -07:00
Harshavardhana 90ca73af13 Allow trace even if server is not initialized (#7822) 2019-06-21 16:47:51 -07:00
Harshavardhana cd7d5b59e5
Add DeleteUser() to generate events in etcd (#7804)
Fixes a regression introduced in 6d89435356

Fixes #7797
2019-06-18 15:44:23 -07:00
poornas 97090aa16c Add admin API to send trace notifications to registered (#7128)
Remove current functionality to log trace to file
using MINIO_HTTP_TRACE env, and replace it with
mc admin trace command on mc client.
2019-06-08 15:54:41 -07:00
Harshavardhana 6d89435356 Reload a specific user or policy on peers (#7705)
Fixes #7587
2019-06-06 17:46:22 -07:00
Harshavardhana ae002aa724 Deprecate updating admin credentials using API calls (#7570)
Root credentials are not allowed to change in all of our
distributed setup deployments, this PR simply removes
that behavior.
2019-04-24 12:54:44 -07:00
kannappanr 5ecac91a55
Replace Minio refs in docs with MinIO and links (#7494) 2019-04-09 11:39:42 -07:00
Harshavardhana c90999df98 Valid if bucket names are internal (#7476)
This commit fixes a privilege escalation issue against
the S3 and web handlers. An authenticated IAM user
can:

- Read from or write to the internal '.minio.sys'
bucket by simply sending a properly signed
S3 GET or PUT request. Further, the user can
- Read from or write to the internal '.minio.sys'
bucket using the 'Upload'/'Download'/'DownloadZIP'
API by sending a "browser" request authenticated
with its JWT token.
2019-04-03 23:10:37 -07:00
kannappanr 87cf51d5ab
unused code: Remove LoadCredentials function (#7369)
It is required to set the environment variable in the case of distributed
minio. LoadCredentials is used to notify peers of the change and will not work if
environment variable is set. so, this function will never be called.
2019-03-20 18:09:57 -07:00
Krishnan Parthasarathi 8a77a298f2 Add deploymentID to ServerInfo (#7372) 2019-03-19 16:12:24 +05:30
Kirill Motkov 3d29ab4059 Rewrite if-else chains to switch statements (#7382) 2019-03-18 07:46:20 -07:00
Harshavardhana a51781e5cf Use context to fill in more details about error XML (#7232) 2019-02-13 16:07:21 -08:00
Harshavardhana df35d7db9d Introduce staticcheck for stricter builds (#7035) 2019-02-13 18:29:36 +05:30
Harshavardhana fef5416b3c Support unknown gateway errors and convert at handler layer (#7219)
Different gateway implementations due to different backend
API errors, might return different unsupported errors at
our handler layer. Current code posed a problem for us because
this information was lost and we would convert it to InternalError
in this situation all S3 clients end up retrying the request.

To avoid this unexpected situation implement a way to support
this cleanly such that the underlying information is not lost
which is returned by gateway.
2019-02-12 14:55:52 +05:30
Praveen raj Mani 8af1f0cc7b Improved error message for user and access key conflict (#7190) 2019-02-07 17:25:58 -08:00
Sidhartha Mani 34e7259f95 Add Historic CPU and memory stats (#7136)
Collect historic cpu and mem stats.  Also, use actual values 
instead of formatted strings while returning to the client. The string 
formatting prevents values from being processed by the server or 
by the client without parsing it. 

This change will allow the values to be processed (eg. 
compute rolling-average over the lifetime of the minio server)
and offloads the formatting to the client.
2019-01-30 12:47:32 +05:30
kannappanr ce870466ff
Top Locks command implementation (#7052)
API to list locks used in distributed XL mode
2019-01-24 07:22:14 -08:00
kannappanr e0d22359e7
Fix lint warnings (#7099) 2019-01-16 12:49:20 -08:00
Harshavardhana 8757c963ba
Migrate all Peer communication to common Notification subsystem (#7031)
Deprecate the use of Admin Peers concept and migrate all peer
communication to Notification subsystem. This finally allows
for a common subsystem for all peer notification in case of
distributed server deployments.
2019-01-14 12:14:20 +05:30
Sidhartha Mani f3f47d8cd3 Add ServerCPULoadInfo() and ServerMemUsageInfo() admin API (#7038) 2019-01-09 19:04:19 -08:00
Nitish Tiwari fcb56d864c Add ServerDrivesPerfInfo() admin API (#6969)
This is part of implementation for mc admin health command. The
ServerDrivesPerfInfo() admin API returns read and write speed
information for all the drives (local and remote) in a given Minio
server deployment.

Part of minio/mc#2606
2018-12-31 09:46:44 -08:00
Harshavardhana 4f31a9a33b Reload users upon AddUser on peers (#6975)
Also migrate ReloadFormat to notification subsystem,
remove GetConfig() we do not use this API anymore
2018-12-18 14:39:21 -08:00
Harshavardhana dba61867e8 Redirect browser requests returning AccessDenied (#6848)
Anonymous requests from S3 resources returning
AccessDenied should be auto redirected to browser
for login.
2018-11-26 12:15:12 -08:00
Harshavardhana a55a298e00 Make sure to log unhandled errors always (#6784)
In many situations, while testing we encounter
ErrInternalError, to reduce logging we have
removed logging from quite a few places which
is acceptable but when ErrInternalError occurs
we should have a facility to log the corresponding
error, this helps to debug Minio server.
2018-11-12 11:07:43 -08:00
Harshavardhana a9cda850ca Add forceStop flag to provide facility to stop healing (#6718)
This PR also makes sure that we deal with HTTP request
count by ignoring the on-going heal operation, i.e
do not wait on itself.
2018-11-04 19:24:16 -08:00
Praveen raj Mani acf46cc3b5 SetConfigHandler should avoid setting an invalid notification config (#6679)
Fixes #6642 
Fixes #6641
2018-10-22 11:51:26 -07:00
Harshavardhana b251454dd6 Fix toggling users status (#6640) 2018-10-16 14:55:23 -07:00
Harshavardhana 1e7e5e297c
Add canned policy support (#6637)
This PR adds an additional API where we can create
a new set of canned policies which can be used with one
or many users.
2018-10-16 12:48:19 -07:00
Harshavardhana 3ef3fefd54 Add ListUsers API to list all configured users in IAM (#6619) 2018-10-13 12:48:43 +05:30
Harshavardhana 54ae364def Introduce STS client grants API and OPA policy integration (#6168)
This PR introduces two new features

- AWS STS compatible STS API named AssumeRoleWithClientGrants

```
POST /?Action=AssumeRoleWithClientGrants&Token=<jwt>
```

This API endpoint returns temporary access credentials, access
tokens signature types supported by this API

  - RSA keys
  - ECDSA keys

Fetches the required public key from the JWKS endpoints, provides
them as rsa or ecdsa public keys.

- External policy engine support, in this case OPA policy engine

- Credentials are stored on disks
2018-10-09 14:00:01 -07:00
Pontus Leitzler 274b35154c Fix go1.11 vet shadow errors (#6555) 2018-10-02 15:21:34 -07:00
Anis Elleuch 83d7ec09c1 Disable restarting server after setting a new config (#6521)
Also disable listening to service restart event in tests since
we don't do this anymore.
2018-09-28 12:10:51 -07:00
Anis Elleuch 6c7c6bec91 profiler: Download API returns error when all nodes fail (#6525)
When download profiling data API fails to gather profiling data
from all nodes for any reason (including profiler not enabled),
return 400 http code with the appropriate json message.
2018-09-27 10:34:37 -07:00
Harshavardhana 1111419d4a Add debugging for mutex, tracing (#6522) 2018-09-27 09:32:05 +05:30
Anis Elleuch 9531cddb06 Add Profiler Admin API (#6463)
Two handlers are added to admin API to enable profiling and disable
profiling of a server in a standalone mode, or all nodes in the
distributed mode.

/minio/admin/profiling/start/{cpu,block,mem}:
  - Start profiling and return starting JSON results, e.g. one
    node is offline.

/minio/admin/profiling/download:
  - Stop the on-going profiling task
  - Stream a zip file which contains all profiling files that can
    be later inspected by go tool pprof
2018-09-18 16:46:35 -07:00
Anis Elleuch 3099af70a3 Add admin get/set config keys API (#6113)
This PR adds two new admin APIs in Minio server and madmin package:
- GetConfigKeys(keys []string) ([]byte, error)
- SetConfigKeys(params map[string]string) (err error)

A key is a path in Minio configuration file, (e.g. notify.webhook.1)

The user will always send a string value when setting it in the config file,
the API will know how to convert the value to the appropriate type. The user
is also able to set a raw json.

Before setting a new config, Minio will validate all fields and try to connect
to notification targets if available.
2018-09-06 20:33:18 +05:30
Harshavardhana 9f14433cbd Ensure that setConfig uses latest functionality (#6302) 2018-08-17 18:51:34 -07:00
Harshavardhana 0e02328c98 Migrate config.json from config-dir to backend (#6195)
This PR is the first set of changes to move the config
to the backend, the changes use the existing `config.json`
allows it to be migrated such that we can save it in on
backend disks.

In future releases, we will slowly migrate out of the
current architecture.

Fixes #6182
2018-08-15 10:11:47 +05:30
Oleg Kovalov 37de2dbd3b simplifying if-else chains to switches (#6208) 2018-08-06 10:26:40 -07:00
Harshavardhana 556a51120c Deprecate ListLocks and ClearLocks (#6233)
No locks are ever left in memory, we also
have a periodic interval of clearing stale locks
anyways. The lock instrumentation was not complete
and was seldom used.

Deprecate this for now and bring it back later if
it is really needed. This also in-turn seems to improve
performance slightly.
2018-08-02 23:09:42 +05:30
kannappanr 76ddf4d32f Log x-amz-request-id as log and XML error response (#6173)
Currently, requestid field in logEntry is not populated, as the
requestid field gets set at the very end.
It is now set before regular handler functions. This is also
useful in setting it as part of the XML error response.

Travis build for ppc64le has been quite inconsistent and stays queued
for most of the time. Removing this build as part of Travis.yml for
the time being.
2018-07-20 18:46:32 -07:00
Anis Elleuch e8a008f5b5 Better validation of all config file fields (#6090)
Add Validate() to serverConfig to call it at server
startup and in Admin SetConfig handler to minimize
errors scenario after server restart.
2018-07-18 11:22:29 -07:00
Harshavardhana 371349787f Remove region requirement for Healing (#6031) 2018-06-08 17:54:57 -07:00
Nitish Tiwari 3dc13323e5 Use random host from among multiple hosts to create requests
Also use hosts passed to Minio startup command to populate IP
addresses if MINIO_PUBLIC_IPS is not set.
2018-06-08 10:22:01 -07:00
Harshavardhana 853ea371ce Bring etcd support for bucket DNS federation
- Supports centralized `config.json`
- Supports centralized `bucket` service records
  for client lookups
- implement a new proxy forwarder
2018-06-08 10:22:01 -07:00
Praveen raj Mani c0cfe21c00 Ignore region in the case of admin API (#5919)
Admin API is not an S3 API and hence it is not required
to honor server region while validating admin API calls.

Fixes #2411
2018-06-07 10:37:31 -07:00
Harshavardhana 6138cae8e7 Persist MINIO_WORM as part of config.json (#6022) 2018-06-06 18:10:51 -07:00
Bala FA 6a53dd1701 Implement HTTP POST based RPC (#5840)
Added support for new RPC support using HTTP POST.  RPC's 
arguments and reply are Gob encoded and sent as HTTP 
request/response body.

This patch also removes Go RPC based implementation.
2018-06-06 14:21:56 +05:30