Commit Graph

98 Commits

Author SHA1 Message Date
Harshavardhana e82dcd195c Deprecate config-dir bring in certs-dir for TLS configuration (#7033)
This PR is to provide indication that config-dir will be removed
in future and all users should migrate to new --certs-dir option

Fixes #7016
Fixes #7032
2019-01-02 10:05:16 -08:00
Anis Elleuch 99b843a64e Add anonymous flag to prevent logging sensitive information (#6899) 2018-12-18 16:08:11 -08:00
Harshavardhana e7c902bbbc
Return proper errors when admin API is not initialized (#6988)
Especially in gateway IAM admin APIs are not enabled
if etcd is not enabled, we should enable admin API though
but only enable IAM and Config APIs with etcd configured.
2018-12-18 13:03:26 -08:00
poornas 7da0336ac8 Make sure env are loaded before gateway layer initialization (#6989) 2018-12-18 10:42:09 -08:00
Andreas Auernhammer d264d2c899 add auto-encryption feature (#6523)
This commit adds an auto-encryption feature which allows
the Minio operator to ensure that uploaded objects are
always encrypted.

This change adds the `autoEncryption` configuration option
as part of the KMS conifguration and the ENV. variable
`MINIO_SSE_AUTO_ENCRYPTION:{on,off}`.

It also updates the KMS documentation according to the
changes.

Fixes #6502
2018-12-14 13:35:48 -08:00
Harshavardhana bebaff269c Support IPv6 in minio command line (#6947)
Fixes #6946
2018-12-14 13:07:46 +05:30
poornas f6980c4630 fix ConfigSys and NotificationSys initialization for NAS (#6920) 2018-12-05 14:03:42 -08:00
Harshavardhana d6af3c1237 Add bucket notification support for NAS gateway (#6908)
Fixes #6885
2018-12-03 14:02:14 +05:30
Anis Elleuch dd092f6c2b gateway: Properly set globalMinioPort (#6859)
globalMinioPort is used in federation which stores the address
and the port number of the server hosting the specified bucket,
this latter uses globalMinioPort but this latter is not set in
startup of the gateway mode.

This commit fixes the behavior.
2018-11-26 23:19:38 +05:30
Harshavardhana a9de303d8b
Update command line docs (#6839) 2018-11-20 17:35:33 -08:00
Harshavardhana bfb505aa8e Refactor logging in more Go idiomatic style (#6816)
This refactor brings a change which allows
targets to be added in a cleaner way and also
audit is now moved out.

This PR also simplifies logger dependency for auditing
2018-11-19 14:47:03 -08:00
Harshavardhana bef0318c36 Support audit logs with additional fields (#6738)
This PR adds support

- Request query params
- Request headers
- Response headers

AuditLogEntry is exported and versioned as well
starting with this PR.
2018-11-02 18:40:08 -07:00
Harshavardhana 9fe51e392b Support etcd TLS certficates (#6719)
This PR supports two models for etcd certs

- Client-to-server transport security with HTTPS
- Client-to-server authentication with HTTPS client certificates
2018-10-29 11:14:12 -07:00
Pontus Leitzler 81d21850ec Root CAs can be used for backend without TLS (#6711) 2018-10-28 06:21:00 +05:30
Harshavardhana 18d9a20ff6 Enable admin users API on gateway (#6659)
This is only enabled when etcd is enabled, healing is only
enabled for erasure coded backend.
2018-10-17 17:25:16 -07:00
Harshavardhana b0c9ae7490 Add audit logging for S3 and Web handlers (#6571)
This PR brings an additional logger implementation
called AuditLog which logs to http targets

The intention is to use AuditLog to log all incoming
requests, this is used as a mechanism by external log
collection entities for processing Minio requests.
2018-10-12 12:25:59 -07:00
Harshavardhana 143e7fe300 Add etcd support to support STS on gateway mode (#6531) 2018-10-12 11:32:18 -07:00
Harshavardhana 54ae364def Introduce STS client grants API and OPA policy integration (#6168)
This PR introduces two new features

- AWS STS compatible STS API named AssumeRoleWithClientGrants

```
POST /?Action=AssumeRoleWithClientGrants&Token=<jwt>
```

This API endpoint returns temporary access credentials, access
tokens signature types supported by this API

  - RSA keys
  - ECDSA keys

Fetches the required public key from the JWKS endpoints, provides
them as rsa or ecdsa public keys.

- External policy engine support, in this case OPA policy engine

- Credentials are stored on disks
2018-10-09 14:00:01 -07:00
Krishnan Parthasarathi 9e5c4df106 Gateway should honour --address flag while checking port availability (#6428)
Fixes #6426
2018-09-06 16:42:33 -07:00
Harshavardhana 7d7e21aebb Merge initConfig logic to ConfigSys (#6312) 2018-08-19 13:57:18 -07:00
Harshavardhana a6b8a5487a Fail gateway properly with an error on port conflicts (#6303) 2018-08-17 21:06:36 -07:00
Harshavardhana 0e02328c98 Migrate config.json from config-dir to backend (#6195)
This PR is the first set of changes to move the config
to the backend, the changes use the existing `config.json`
allows it to be migrated such that we can save it in on
backend disks.

In future releases, we will slowly migrate out of the
current architecture.

Fixes #6182
2018-08-15 10:11:47 +05:30
poornas 5f69f04909 nas gateway: fix regression in global bucket policy initialization (#6243)
Fixes #6238
globalPolicySys used to be initialized in fs/xl layer. The referenced
commit moved this logic to server/gateway initialization,but a check
to avoid double initialization prevented globalPolicySys to be loaded
from disk for NAS.

fixes regression from commit be1700f595
2018-08-03 15:12:18 -07:00
Anis Elleuch 9c5e971a58 Add new console/http loggers (#6066)
- Add console target logging, enabled by default.
- Add http target logging, which supports an endpoint
  with basic authentication (username/password are passed
  in the endpoint url itself)
- HTTP target logging is asynchronous and some logs can be
  dropped if channel buffer (10000) is full
2018-07-19 15:55:06 -07:00
Anis Elleuch be1700f595 Avoid startup abort when a notify target is down (#6126)
Minio server was preventing itself to start when any notification
target is down and not running. The PR changes the behavior by
avoiding startup abort in that case, so the user will still
be able to access Minio server using mc admin commands after
a restart or set config commands.
2018-07-10 07:20:31 +05:30
Harshavardhana abf209b1dd load bucket policies using object layer API (#6084)
This PR fixes an issue during gateway mode
where underlying policies were not translated
into meaningful policies.
2018-06-27 12:29:48 +05:30
Andreas Auernhammer cd152f404a replace os.Exit with panic for logger.CriticalIf (#6065)
This commit prevents complete server failures caused by
`logger.CriticalIf` calls. Instead of calling `os.Exit(1)`
the function now executes a panic with a special value
indicating that a critical error happend. At the top HTTP
handler layer panics are recovered and if its a critical
error the client gets an InternalServerError status code.

Further this allows unit tests to cover critical-error code
paths.
2018-06-25 13:51:49 -07:00
kannappanr 577d10674d
Cleanup stack trace in error logs (#6045)
Add compile time GOROOT path to the list of prefix
of file paths to be removed.

Add webhandler function names to the slice that
stores function names to terminate logging.
2018-06-18 12:04:46 -07:00
Bala FA 6a53dd1701 Implement HTTP POST based RPC (#5840)
Added support for new RPC support using HTTP POST.  RPC's 
arguments and reply are Gob encoded and sent as HTTP 
request/response body.

This patch also removes Go RPC based implementation.
2018-06-06 14:21:56 +05:30
Harshavardhana 537fd8c821 Bring nicer ui error if creds are missing in Gateway (#6003) 2018-06-01 15:45:11 -07:00
Harshavardhana 74328c3061 Implement auto cert reloading (#5963) 2018-05-31 12:30:15 -07:00
Yaroslav Skopets a50cc7e937 Add Prometheus metrics for Minio gateway (#5987) 2018-05-30 10:13:46 +05:30
Harshavardhana 71c4ff9d10 Fix tabs/indents in help message (#5962) 2018-05-21 11:11:57 -07:00
Anis Elleuch c525424179 Return detailed msg for notification errors (#5913) 2018-05-09 15:11:24 -07:00
Anis Elleuch 32700fca52 Enhance fatal errors printing of common issues seen by users (#5878) 2018-05-08 19:04:36 -07:00
Bala FA 0d52126023 Enhance policy handling to support SSE and WORM (#5790)
- remove old bucket policy handling
- add new policy handling
- add new policy handling unit tests

This patch brings support to bucket policy to have more control not
limiting to anonymous.  Bucket owner controls to allow/deny any rest
API.

For example server side encryption can be controlled by allowing
PUT/GET objects with encryptions including bucket owner.
2018-04-24 15:53:30 -07:00
Bala FA 76cc65531c use package name correctly (#5827) 2018-04-21 19:23:54 -07:00
Harshavardhana 57b8db2088 Cleanup help to bring concise self-help message (#5788) 2018-04-13 11:57:05 -07:00
kannappanr 57a3d9c16c
Modify fatalIf, startup and update message logging code (#5780)
Use a common logging framework to log fatalIf, startup, Info and Update
messages.
2018-04-10 09:37:14 -07:00
kannappanr f8a3fd0c2a
Create logger package and rename errorIf to LogIf (#5678)
Removing message from error logging
Replace errors.Trace with LogIf
2018-04-05 15:04:40 -07:00
kannappanr 91fd8ffeb7
Move http package inside cmd (#5776) 2018-04-05 14:19:27 -07:00
Harshavardhana 30ab71c9d9 Add missing healthcheck router for gateway (#5764) 2018-04-05 07:37:54 +05:30
Bala FA 0e4431725c make notification as separate package (#5294)
* Remove old notification files

* Add net package

* Add event package

* Modify minio to take new notification system
2018-03-15 13:03:41 -07:00
kannappanr d32f90fe95 Honor global flags irrespective of the position. (#5486)
Flags like `json, config-dir, quiet` are now honored even if they are
between minio and gateway in the cli, like, `minio --json gateway s3`.

Fixes #5403
2018-02-28 20:13:33 -08:00
poornas 4f73fd9487 Unify gateway and object layer. (#5487)
* Unify gateway and object layer. Bring bucket policies into
object layer.
2018-02-09 15:19:30 -08:00
ebozduman 24d9d7e5fa Removes logrus package and refactors logging messages (#5293)
This fix removes logrus package dependency and refactors the console
logging as the only logging mechanism by removing file logging support.
It rearranges the log message format and adds stack trace information
whenever trace information is not available in the error structure.
It also adds `--json` flag support for server logging.
When minio server is started with `--json` flag, all log messages are
displayed in json format, with no start-up and informational log
messages.
Fixes #5265 #5220 #5197
2018-01-17 07:24:46 -08:00
Harshavardhana f25ec31565 Set maxResources appropriately for gateway like server (#5321) 2017-12-24 20:09:30 +05:30
kannappanr a1c1a18dc5 Remove "logger" field from config.json (#5268)
File logging removed as part of improvement to server logging.

config.json format updated to version 21.

Fixes #5176
2017-12-06 12:48:29 +05:30
Harshavardhana eb2894233c Convert gateways into respective packages (#5200)
- Make azure gateway a package
- Make b2 gateway a package
- Make gcs gateway a package
- Make s3 gateway a package
- Make sia gateway a package
2017-12-05 17:58:09 -08:00
Harshavardhana 8efa82126b
Convert errors tracer into a separate package (#5221) 2017-11-25 11:58:29 -08:00