Commit Graph

65 Commits

Author SHA1 Message Date
Praveen raj Mani 8836d57e3c The prometheus metrics refractoring (#8003)
The measures are consolidated to the following metrics

- `disk_storage_used` : Disk space used by the disk.
- `disk_storage_available`: Available disk space left on the disk.
- `disk_storage_total`: Total disk space on the disk.
- `disks_offline`: Total number of offline disks in current MinIO instance.
- `disks_total`: Total number of disks in current MinIO instance.
- `s3_requests_total`: Total number of s3 requests in current MinIO instance.
- `s3_errors_total`: Total number of errors in s3 requests in current MinIO instance.
- `s3_requests_current`: Total number of active s3 requests in current MinIO instance.
- `internode_rx_bytes_total`: Total number of internode bytes received by current MinIO server instance.
- `internode_tx_bytes_total`: Total number of bytes sent to the other nodes by current MinIO server instance.
- `s3_rx_bytes_total`: Total number of s3 bytes received by current MinIO server instance.
- `s3_tx_bytes_total`: Total number of s3 bytes sent by current MinIO server instance.
- `minio_version_info`: Current MinIO version with commit-id.
- `s3_ttfb_seconds_bucket`: Histogram that holds the latency information of the requests.

And this PR also modifies the current StorageInfo queries

- Decouples StorageInfo from ServerInfo .
- StorageInfo is enhanced to give endpoint information.

NOTE: ADMIN API VERSION IS BUMPED UP IN THIS PR

Fixes #7873
2019-10-22 21:01:14 -07:00
Harshavardhana f2cc97a44c
Improve MQTT token registration retry (#8397) 2019-10-15 19:39:14 -07:00
Andreas Auernhammer b7ee0bbbc9 madmin: Migrate to provable secure channel construction (#8395)
This commit replaces the currently used channel construction
for en/decrypting config data with a provable secure scheme.
2019-10-15 02:36:04 +05:30
Minio Trusted ae752ed1fa Update yaml files to latest version RELEASE.2019-10-11T00-38-09Z 2019-10-11 00:45:13 +00:00
Harshavardhana 6a4ef2e48e Initialize configs correctly, move notification config (#8367)
This PR also removes deprecated tests, adds checks
to avoid races reproduced on CI/CD.
2019-10-09 11:41:15 +05:30
Ashish Kumar Sinha 74008446fe CPU hardware info (#8187) 2019-10-03 20:18:38 +05:30
Harshavardhana 90bfa6260a Fix LDAP TLS support to use custom CAs (#8352) 2019-10-03 01:44:57 +05:30
Harshavardhana 4ec9b349d0
azure: Fix upload corruption with PutObject() on certain sizes (#8330)
On objects bigger than 100MiB can have a corrupted object
stored due to partial blockListing attempted right after
each blocks uploaded. Simplify this code to ensure that
all the blocks successfully uploaded are committed right
away.

This PR also updates the azure-sdk-go to latest release.
2019-09-30 18:42:18 -07:00
Andreas Auernhammer 8771e83545 Update klauspost/reedsolomon dependency to 1.9.3 (#8328)
This commit updates the reedsolomon dependency
since it contains an fix for an unexpected property
of the `Split` function.

See: klauspost/reedsolomon#109
2019-10-01 02:31:00 +05:30
Klaus Post ff726969aa Switch to Snappy -> S2 compression (#8189) 2019-09-25 23:08:24 -07:00
Harshavardhana be70ef59e7
Update minio-go to latest release (#8297)
- Change LDAP XML names to follow server changes
- Fix listObjects hang when listing special characters
- set encoding type to url in listobjects API
2019-09-24 01:01:54 -07:00
Krishnan Parthasarathi 6ba323b009 Add ability to test drive speeds on a MinIO setup (#7664)
- Extends existing Admin API to measure disk performance
2019-09-13 03:22:30 +05:30
Harshavardhana 5512baab21 Fix go mod tidy errors and fix platform independent downloads (#8222)
Refer https://github.com/gin-gonic/gin/issues/1673
2019-09-12 04:57:59 +05:30
Minio Trusted 3d65dc8d94 Update yaml files to latest version RELEASE.2019-09-11T19-53-16Z 2019-09-11 19:59:05 +00:00
Aditya Manthramurthy a0456ce940 LDAP STS API (#8091)
Add LDAP based users-groups system

This change adds support to integrate an LDAP server for user
authentication. This works via a custom STS API for LDAP. Each user
accessing the MinIO who can be authenticated via LDAP receives
temporary credentials to access the MinIO server.

LDAP is enabled only over TLS.

User groups are also supported via LDAP. The administrator may
configure an LDAP search query to find the group attribute of a user -
this may correspond to any attribute in the LDAP tree (that the user
has access to view). One or more groups may be returned by such a
query.

A group is mapped to an IAM policy in the usual way, and the server
enforces a policy corresponding to all the groups and the user's own
mapped policy.

When LDAP is configured, the internal MinIO users system is disabled.
2019-09-10 04:42:29 +05:30
Harshavardhana 94e5cb7576
Migrate to go1.13 to avail all new features (#8203)
Read more https://blog.golang.org/go1.13
2019-09-08 16:44:15 -07:00
Minio Trusted 5bde31d021 Update yaml files to latest version RELEASE.2019-09-05T23-24-38Z 2019-09-05 23:30:40 +00:00
Harshavardhana b52a3e523c Avoid using fastjson parser pool, move back to jsoniter (#8190)
It looks like from implementation point of view fastjson
parser pool doesn't behave the same way as expected
when dealing many `xl.json` from multiple disks.

The fastjson parser pool usage ends up returning incorrect
xl.json entries for checksums, with references pointing
to older entries. This led to the subtle bug where checksum
info is duplicated from a previous xl.json read of a different
file from different disk.
2019-09-06 04:21:27 +05:30
Minio Trusted daa57e8e5a Update yaml files to latest version RELEASE.2019-08-29T00-25-01Z 2019-08-29 00:30:50 +00:00
Minio Trusted de736f2864 Update yaml files to latest version RELEASE.2019-08-21T19-40-07Z 2019-08-21 19:46:01 +00:00
Harshavardhana 9ca7470ccc
Avoid using jsoniter, move to fastjson (#8063)
This is to avoid using unsafe.Pointer type
code dependency for MinIO, this causes
crashes on ARM64 platforms

Refer #8005 collection of runtime crashes due
to unsafe.Pointer usage incorrectly. We have
seen issues like this before when using
jsoniter library in the past.

This PR hopes to fix this using fastjson
2019-08-19 08:35:52 -10:00
Minio Trusted 26640033b0 Update yaml files to latest version RELEASE.2019-08-14T20-37-41Z 2019-08-14 20:42:16 +00:00
Minio Trusted 55b385beee Update yaml files to latest version RELEASE.2019-08-07T01-59-21Z 2019-08-07 02:05:41 +00:00
kannappanr 930943f058
Fix IAM users migration regression in etcd (#8029)
PR #8008 did not migrate user data stored in etcd.
This PR fixes that.
2019-08-06 17:06:31 -07:00
Minio Trusted dfa8835720 Update yaml files to latest version RELEASE.2019-08-01T22-18-54Z 2019-08-01 22:27:41 +00:00
Minio Trusted c301f5882d Update yaml files to latest version RELEASE.2019-07-31T18-57-56Z 2019-07-31 19:06:20 +00:00
Harshavardhana 6f2b4675fa
Add krb5 support for HDFS gateway (#7933) 2019-07-24 18:05:48 -07:00
Minio Trusted 5b71c21330 Update yaml files to latest version RELEASE.2019-07-24T02-02-23Z 2019-07-24 02:09:19 +00:00
Krishnan Parthasarathi 559a59220e Add initial support for bucket lifecycle (#7563)
This PR is based off @sinhaashish's PR for object lifecycle
management, which includes support only for,
- Expiration of object
- Filter using object prefix (_not_ object tags)

N B the code for actual expiration of objects will be included in a
subsequent PR.
2019-07-19 21:20:33 +01:00
Minio Trusted b058e32348 Update yaml files to latest version RELEASE.2019-07-17T22-54-12Z 2019-07-17 22:59:33 +00:00
Harshavardhana 16a45e5aff
Fix dynamic help vars for sub-commands (#7925)
The fix in #7646 introduced a regression which
was left unnoticed, the fix didn't work for
sub-commands unfortunately. This fixes it
by moving v1.21.0 version of the minio/cli
package.

Fixes #7924
2019-07-12 23:32:27 -07:00
Minio Trusted 2d96745156 Update yaml files to latest version RELEASE.2019-07-10T00-34-56Z 2019-07-10 00:39:45 +00:00
mzukowski-reef 9d49688c87 Switch to kurin/blazer from minio/blazer fork for b2 gateway (#7879) 2019-07-09 08:14:02 -07:00
Minio Trusted 22bc15d89b Update yaml files to latest version RELEASE.2019-07-05T21-20-21Z 2019-07-05 21:24:43 +00:00
Krishna Srinivas a2e904b966 Support any string as delimiter for listing (#7882) 2019-07-05 14:06:12 -07:00
Krishna Srinivas 338e9a9be9 Put object client disconnect (#7824)
Fail putObject  and postpolicy in case client prematurely disconnects
Use request's context to cancel lock requests on client disconnects
2019-06-28 22:09:17 -07:00
Minio Trusted 5db60a6c59 Update yaml files to latest version RELEASE.2019-06-27T21-13-50Z 2019-06-27 21:18:52 +00:00
Minio Trusted e43d3a075c Update yaml files to latest version RELEASE.2019-06-19T18-24-42Z 2019-06-19 18:31:37 +00:00
Minio Trusted 59f7266081 Update yaml files to latest version RELEASE.2019-06-15T23-07-18Z 2019-06-15 23:17:28 +00:00
Harshavardhana 99bf4d0c42 [sec] Match ${aws:username} exactly instead of prefix match (#7791)
This PR fixes a security issue where an IAM user based
on his policy is granted more privileges than restricted
by the users IAM policy.

This is due to an issue of prefix based Matcher() function
which was incorrectly matching prefix based on resource
prefixes instead of exact match.
2019-06-15 11:27:17 -07:00
Minio Trusted da2887f914 Update yaml files to latest version RELEASE.2019-06-13T01-41-13Z 2019-06-13 01:46:00 +00:00
Harshavardhana c22439c82e Update minio-go v6.0.29 (#7778)
Bring improved retry logic
2019-06-12 18:09:21 -07:00
Harshavardhana b4ab778cb2 Fix user IAM policy regression, reload policy appropriately (#7770)
Introduce in commit 7e4c9a9e1e

Fixes #7769
2019-06-12 14:49:45 -07:00
Minio Trusted d3a2efbf91 Update yaml files to latest version RELEASE.2019-06-11T00-44-33Z 2019-06-11 00:50:31 +00:00
Harshavardhana cb1566c6e6 S3 Gateway: Handle restricted access credentials (#7757) 2019-06-07 15:49:13 -07:00
Minio Trusted df19546cc8 Update yaml files to latest version RELEASE.2019-06-04T01-15-58Z 2019-06-04 01:22:32 +00:00
Harshavardhana ff7799452b Fix build failure due to nats project migration 2019-06-03 17:49:49 -07:00
Minio Trusted 48ffb7731a Update yaml files to latest version RELEASE.2019-06-01T03-46-14Z 2019-06-01 03:52:47 +00:00
Harshavardhana 2c0b3cadfc Update go mod with sem versions of our libraries (#7687) 2019-05-29 16:35:12 -07:00
Minio Trusted 7a02faab72 Update yaml files to latest version RELEASE.2019-05-23T00-29-34Z 2019-05-23 00:35:13 +00:00