MyFavMirrors
/
minio
mirror of https://github.com/minio/minio.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
9,011 Commits 5 Branches 503 Tags 186 MiB
Commit Graph

1116 Commits

Author SHA1 Message Date
Minio Trusted cfc8b92dff Update yaml files to latest version RELEASE.2021-02-14T04-01-33Z 2021-02-14 04:25:52 +00:00
Minio Trusted 0ef3e359d8 Update yaml files to latest version RELEASE.2021-02-11T08-23-43Z 2021-02-11 08:47:10 +00:00
Aditya Manthramurthy 466e95bb59
Return group DN instead of group name in LDAP STS (#11501) 
- Additionally, check if the user or their groups has a policy attached during
the STS call.

- Remove the group name attribute configuration value.
 2021-02-10 16:52:49 -08:00
Krishna Srinivas 876b79b8d8
read-health check endpoint returns success if cluster can serve read requests (#11310) 2021-02-09 01:00:44 -08:00
Sarasa Kisaragi 152d7cd95b
HDFS support keytab (#11473) 2021-02-07 17:29:47 -08:00
Minio Trusted 647a209c73 Update yaml files to latest version RELEASE.2021-02-07T01-31-02Z 2021-02-07 01:53:27 +00:00
Harshavardhana 9497dfd804 docs: add deprecation notice for federation 2021-02-04 17:18:37 -08:00
Bahram Aghaei f2d49ec21a
Update ldap.md: add a link to ldap.go (#11409) 2021-02-02 15:47:04 -08:00
Ritesh H Shukla c4848f9b4f
Add process start time to cluster metrics. (#11405) 2021-02-01 23:02:18 -08:00
swartz-k 8c663f93f7
fix: typo in chinese docs (#11401) 2021-02-01 18:42:58 -08:00
Minio Trusted b4cb7edf85 Update yaml files to latest version RELEASE.2021-02-01T22-56-52Z 2021-02-01 23:28:23 +00:00
Andreas Auernhammer 33554651e9
crypto: deprecate native Hashicorp Vault support (#11352) 
This commit deprecates the native Hashicorp Vault
support and removes the legacy Vault documentation.

The native Hashicorp Vault documentation is marked as
outdated and deprecated for over a year now. We give
another 6 months before we start removing Hashicorp Vault
support and show a deprecation warning when a MinIO server
starts with a native Vault configuration.
 2021-01-29 17:55:37 -08:00
Minio Trusted 451d9057f3 Update yaml files to latest version RELEASE.2021-01-30T00-20-58Z 2021-01-30 00:45:11 +00:00
Harshavardhana 6717295e18 fix: rename audit log docs and datastructure 2021-01-26 13:39:55 -08:00
Anis Elleuch 00cff1aac5
audit: per object send pool number, set number and servers per operation (#11233) 2021-01-26 13:21:51 -08:00
Aditya Manthramurthy 5f51ef0b40
Add LDAP Lookup-Bind mode (#11318) 
This change allows the MinIO server to be configured with a special (read-only)
LDAP account to perform user DN lookups.

The following configuration parameters are added (along with corresponding
environment variables) to LDAP identity configuration (under `identity_ldap`):

- lookup_bind_dn / MINIO_IDENTITY_LDAP_LOOKUP_BIND_DN
- lookup_bind_password / MINIO_IDENTITY_LDAP_LOOKUP_BIND_PASSWORD
- user_dn_search_base_dn / MINIO_IDENTITY_LDAP_USER_DN_SEARCH_BASE_DN
- user_dn_search_filter / MINIO_IDENTITY_LDAP_USER_DN_SEARCH_FILTER

This lookup-bind account is a service account that is used to lookup the user's
DN from their username provided in the STS API. When configured, searching for
the user DN is enabled and configuration of the base DN and filter for search is
required. In this "lookup-bind" mode, the username format is not checked and must
not be specified. This feature is to support Active Directory setups where the
DN cannot be simply derived from the username.

When the lookup-bind is not configured, the old behavior is enabled: the minio
server performs LDAP lookups as the LDAP user making the STS API request and the
username format is checked and configuring it is required.
 2021-01-25 14:26:10 -08:00
Ritesh H Shukla 0bf2d84f96
update new metrics url docs (#11342) 2021-01-25 01:03:07 -08:00
Ritesh H Shukla 7575c24037
Add open FD and FD limit to cluster metrics (#11328) 2021-01-22 18:30:16 -08:00
Harshavardhana c080f04e66 fix: prometheus metrics link typo update to latest 2021-01-22 01:53:23 -08:00
Ritesh H Shukla b4add82bb6
Updated Prometheus metrics (#11141) 
* Add metrics for nodes online and offline
* Add cluster capacity metrics
* Introduce v2 metrics
 2021-01-18 20:35:38 -08:00
Aditya Manthramurthy 3163a660aa
Fix support for multiple LDAP user formats (#11276) 
Fixes support for using multiple base DNs for user search in the LDAP directory
allowing users from different subtrees in the LDAP hierarchy to request
credentials.

- The username in the produced credentials is now the full DN of the LDAP user
to disambiguate users in different base DNs.
 2021-01-17 21:54:32 -08:00
Harshavardhana 98f76008c7 fix: bucket lifecycle again to remove Days parameter 2021-01-17 01:50:56 -08:00
Harshavardhana 8da0b7cf03 fix: lifecycle documentation for DeleteMarker 2021-01-17 01:37:25 -08:00
Harshavardhana f903cae6ff
Support variable server pools (#11256) 
Current implementation requires server pools to have
same erasure stripe sizes, to facilitate same SLA
and expectations.

This PR allows server pools to be variadic, i.e they
do not have to be same erasure stripe sizes - instead
they should have SLA for parity ratio.

If the parity ratio cannot be guaranteed by the new
server pool, the deployment is rejected i.e server
pool expansion is not allowed.
 2021-01-16 12:08:02 -08:00
Minio Trusted 40d59c1961 Update yaml files to latest version RELEASE.2021-01-16T02-19-44Z 2021-01-16 02:43:53 +00:00
Poorna Krishnamoorthy 7090bcc8e0
fix: doc links and delete replication permissions enforcement (#11285) 2021-01-15 15:22:55 -08:00
Harshavardhana cc2d887e0e fix: whitespace and formatting in replication docs 2021-01-14 22:58:53 -08:00
Poorna Krishnamoorthy c1b4b24236
Update replication docs (#11279) 2021-01-15 10:22:57 +05:30
Poorna Krishnamoorthy feaf8dfb9a
Fix replication status reported on completion (#11273) 
Fixes: #11272
 2021-01-13 11:52:28 -08:00
Poorna Krishnamoorthy b97d53b29c
fix remote target healthcheck (#11267) 2021-01-12 20:48:04 -08:00
Aditya Manthramurthy 00af9881b0
LDAP doc fix: remove repeated paragraph and add emphasis (#11266) 2021-01-12 15:44:31 -08:00
Poorna Krishnamoorthy 7824e19d20
Allow synchronous replication if enabled. (#11165) 
Synchronous replication can be enabled by setting the --sync
flag while adding a remote replication target.

This PR also adds proxying on GET/HEAD to another node in a
active-active replication setup in the event of a 404 on the current node.
 2021-01-11 22:36:51 -08:00
George Tsatsis e8176fe978
Use `-new` during OpenSSL certificate generation (#11199) 
As per https://stackoverflow.com/a/3758443/8156177, OpenSSL expects a certificate via STDIN. 
`-new` will allow a new certificate to be generated instead.
 2021-01-11 02:24:50 -08:00
Minio Trusted d9224fbc65 Update yaml files to latest version RELEASE.2021-01-08T21-18-21Z 2021-01-08 21:37:35 +00:00
Harshavardhana 76e2713ffe
fix: use buffers only when necessary for io.Copy() (#11229) 
Use separate sync.Pool for writes/reads

Avoid passing buffers for io.CopyBuffer()
if the writer or reader implement io.WriteTo or io.ReadFrom
respectively then its useless for sync.Pool to allocate
buffers on its own since that will be completely ignored
by the io.CopyBuffer Go implementation.

Improve this wherever we see this to be optimal.

This allows us to be more efficient on memory usage.
```
   385  // copyBuffer is the actual implementation of Copy and CopyBuffer.
   386  // if buf is nil, one is allocated.
   387  func copyBuffer(dst Writer, src Reader, buf []byte) (written int64, err error) {
   388  	// If the reader has a WriteTo method, use it to do the copy.
   389  	// Avoids an allocation and a copy.
   390  	if wt, ok := src.(WriterTo); ok {
   391  		return wt.WriteTo(dst)
   392  	}
   393  	// Similarly, if the writer has a ReadFrom method, use it to do the copy.
   394  	if rt, ok := dst.(ReaderFrom); ok {
   395  		return rt.ReadFrom(src)
   396  	}
```

From readahead package
```
// WriteTo writes data to w until there's no more data to write or when an error occurs.
// The return value n is the number of bytes written.
// Any error encountered during the write is also returned.
func (a *reader) WriteTo(w io.Writer) (n int64, err error) {
	if a.err != nil {
		return 0, a.err
	}
	n = 0
	for {
		err = a.fill()
		if err != nil {
			return n, err
		}
		n2, err := w.Write(a.cur.buffer())
		a.cur.inc(n2)
		n += int64(n2)
		if err != nil {
			return n, err
		}
```
 2021-01-06 09:36:55 -08:00
Harshavardhana b5d291ea88
fix: rename remaining zone -> pool (#11231) 2021-01-06 09:35:47 -08:00
Klaus Post eb9172eecb
Allow Compression + encryption (#11103) 2021-01-05 20:08:35 -08:00
Harshavardhana cb0eaeaad8
feat: migrate to ROOT_USER/PASSWORD from ACCESS/SECRET_KEY (#11185) 2021-01-05 10:22:57 -08:00
Minio Trusted f3f0041ad0 Update yaml files to latest version RELEASE.2021-01-05T05-22-38Z 2021-01-05 05:42:54 +00:00
Baptiste Mille-Mathias c1f6ca6697
Fix caddy project url (#11198) 2020-12-31 09:44:07 -08:00
0xflotus cde801282d
chore: enabled syntax highlighting in docs (#11182) 2020-12-29 17:38:28 -08:00
Ravind Kumar 6cf0008469
fix: docs typo in object lock docs (#11181) 2020-12-29 16:14:10 -08:00
Minio Trusted 7b0330a98c Update yaml files to latest version RELEASE.2020-12-29T23-29-29Z 2020-12-29 23:46:24 +00:00
Minio Trusted d3c853a3be Update yaml files to latest version RELEASE.2020-12-26T01-35-54Z 2020-12-26 01:53:30 +00:00
Poorna Krishnamoorthy 7b8a456f68
Update lifecycle README docs (#11160) 
Removing reference to transition feature in docs as this
feature is being revamped to provide better extensibility
across different cloud targets.
 2020-12-23 19:56:55 -08:00
Harshavardhana b43906f6ee fix: docs typos and keywords 2020-12-23 11:59:20 -08:00
Minio Trusted bfb92a27b7 Update yaml files to latest version RELEASE.2020-12-23T02-24-12Z 2020-12-23 02:43:25 +00:00
Minio Trusted 386dd56856 Update yaml files to latest version RELEASE.2020-12-18T03-27-42Z 2020-12-18 03:45:13 +00:00
Minio Trusted 26624552be Update yaml files to latest version RELEASE.2020-12-16T05-05-17Z 2020-12-16 05:23:19 +00:00
Minio Trusted 31bf6f0c25 Update yaml files to latest version RELEASE.2020-12-12T08-39-07Z 2020-12-12 08:56:30 +00:00
Harshavardhana 14792cdbc6
docs: fix the metrics formatting (#11081) 2020-12-10 18:15:47 -08:00
Harshavardhana 97856bfebf
fix: grafana double counting for bucket usage, histrogram and objects (#11070) 2020-12-09 20:30:37 -08:00
Minio Trusted 83a67a1d21 Update yaml files to latest version RELEASE.2020-12-10T01-54-29Z 2020-12-10 02:12:01 +00:00
Nitish Tiwari 54d243cd98
fix: grafana dashboard calculating online nodes (#11041) 
Also use a generic name instead of diff names per revision
 2020-12-09 00:26:42 -08:00
Ritesh H Shukla 04848dfa1c
Add documentation for bucket replication related metrics (#11055) 2020-12-08 12:48:10 -08:00
Harshavardhana 4a564336fe Revert "Add metrics for nodes online and offline (#11050)" 
This reverts commit f60bbdf86b.
 2020-12-08 09:23:35 -08:00
Ritesh H Shukla f60bbdf86b
Add metrics for nodes online and offline (#11050) 2020-12-08 01:06:27 -08:00
Harshavardhana 8c79f87f02
add dynamic config docs (#11048) 
Co-authored-by: Eco <41090896+eco-minio@users.noreply.github.com>
 2020-12-07 19:02:20 -08:00
Poorna Krishnamoorthy f3beb1236a
Add cache usage, total capacity to prometheus metrics (#11026) 2020-12-07 16:35:11 -08:00
Poorna Krishnamoorthy 934bed47fa
Add transition event notification (#11047) 
This is a MinIO specific extension to allow monitoring of transition events.
 2020-12-07 13:53:28 -08:00
Nitish Tiwari 6ff12f5f01
Add the dashboard json file (#11028) 
This will allow users to contribute to the dashboard as needed.
 2020-12-04 16:27:41 -08:00
Nitish Tiwari de9b64834e
fix: update grafana dashboard docs (#11023) 
Refer to the official Grafana dashboard
 2020-12-03 15:56:15 -08:00
Minio Trusted d54cf77356 Update yaml files to latest version RELEASE.2020-12-03T05-49-24Z 2020-12-03 06:05:56 +00:00
Minio Trusted 1ab4d6a6aa Update yaml files to latest version RELEASE.2020-12-03T00-03-10Z 2020-12-03 00:24:16 +00:00
Anis Elleuch fe11e9047d
deprecate CommonName from TLS docs (#11017) 
CommonName is not supported anymore in Go 1.15

fix the TLS documentation to use subjAltNames
 2020-12-02 10:18:39 -08:00
Sudarshan (Sid) 3ca046b408
Added set keyword in the command set to enable encryption on buckets (#11010) 2020-12-01 16:00:49 -08:00
Harshavardhana 4ec45753e6 rename server sets to server pools 2020-12-01 13:50:33 -08:00
Harshavardhana 790833f3b2 Revert "Support variable server sets (#10314)" 
This reverts commit aabf053d2f.
 2020-12-01 12:02:29 -08:00
Harshavardhana aabf053d2f
Support variable server sets (#10314) 2020-11-25 16:28:47 -08:00
Minio Trusted f839bb5a0a Update yaml files to latest version RELEASE.2020-11-25T22-36-25Z 2020-11-25 22:53:02 +00:00
Anis Elleuch 75a8e81f8f
azure: Specify different Azure storage in the shell env (#10943) 
AZURE_STORAGE_ACCOUNT and AZURE_STORAGE_KEY are used in 
azure CLI to specify the azure blob storage access & secret keys. With this commit, 
it is possible to set them if you want the gateway's own credentials to be
different from the Azure blob credentials.

Co-authored-by: Harshavardhana <harsha@minio.io>
 2020-11-23 16:45:56 -08:00
Poorna Krishnamoorthy 39f3d5493b
Show Delete replication status header (#10946) 
X-Minio-Replication-Delete-Status header shows the
status of the replication of a permanent delete of a version.

All GETs are disallowed and return 405 on this object version.
In the case of replicating delete markers.

X-Minio-Replication-DeleteMarker-Status shows the status 
of replication, and would similarly return 405.

Additionally, this PR adds reporting of delete marker event completion
and updates documentation
 2020-11-21 23:48:50 -08:00
Shireesh Anjal 7bc47a14cc
Rename OBD to Health (#10842) 
Also, Remove thread stats and openfds from the health report 
as we already have process stats and numfds
 2020-11-20 12:52:53 -08:00
Harshavardhana 9263be8cca
docs: fix missing event types in notifications (#10944) 2020-11-20 11:27:27 -08:00
Poorna Krishnamoorthy 1ebf6f146a Add support for ILM transition (#10565) 
This PR adds transition support for ILM
to transition data to another MinIO target
represented by a storage class ARN. Subsequent
GET or HEAD for that object will be streamed from
the transition tier. If PostRestoreObject API is
invoked, the transitioned object can be restored for
duration specified to the source cluster.
 2020-11-19 18:47:17 -08:00
Harshavardhana 9a34fd5c4a Revert "Revert "Add delete marker replication support (#10396)"" 
This reverts commit 267d7bf0a9.
 2020-11-19 18:43:58 -08:00
Minio Trusted b9e3a8b5ac Update yaml files to latest version RELEASE.2020-11-19T23-48-16Z 2020-11-20 00:06:03 +00:00
Minio Trusted 0784a0c33a Update yaml files to latest version RELEASE.2020-11-13T20-10-18Z 2020-11-13 20:27:01 +00:00
Minio Trusted 3595cb1267 Update yaml files to latest version RELEASE.2020-11-12T22-33-34Z 2020-11-12 22:51:11 +00:00
Omar Alvarez 07859ef48b
docs: clarify notifications support for gateways (#10729) 
Co-authored-by: Harshavardhana <harsha@minio.io>
 2020-11-12 12:19:04 -08:00
Harshavardhana 267d7bf0a9 Revert "Add delete marker replication support (#10396)" 
This reverts commit 50c10a5087.

PR is moved to origin/dev branch
 2020-11-12 11:43:14 -08:00
Anton Melser 2c1e37197b
fix: bad example json for policy in replication docs (#10869) 2020-11-10 17:49:49 -08:00
Poorna Krishnamoorthy 50c10a5087
Add delete marker replication support (#10396) 
Delete marker replication is implemented for V2
configuration specified in AWS spec (though AWS
allows it only in the V1 configuration).

This PR also brings in a MinIO only extension of
replicating permanent deletes, i.e. deletes specifying
version id are replicated to target cluster.
 2020-11-10 15:24:14 -08:00
Minio Trusted 9f4ad873bc Update yaml files to latest version RELEASE.2020-11-10T21-02-24Z 2020-11-10 21:18:35 +00:00
Anis Elleuch 8e8ddf7233
doc: Add definition of 1KB and 1MB in prometheus (#10857) 2020-11-09 10:05:01 -08:00
Minio Trusted 21016265e5 Update yaml files to latest version RELEASE.2020-11-06T23-17-07Z 2020-11-06 23:34:19 +00:00
Minio Trusted 51222cc664 Update yaml files to latest version RELEASE.2020-10-28T08-16-50Z 2020-10-28 08:33:22 +00:00
Minio Trusted ca77ee1c0e Update yaml files to latest version RELEASE.2020-10-27T04-03-55Z 2020-10-27 04:20:10 +00:00
Harshavardhana 4442382c16 update STS examples to use latest v7 APIs 2020-10-24 11:16:59 -07:00
Anis Elleuch 6fd088f448
docs: Update ILM doc with versioning features (#10714) 2020-10-20 09:23:27 -07:00
Minio Trusted ba5215561f Update yaml files to latest version RELEASE.2020-10-18T21-54-12Z 2020-10-18 22:15:30 +00:00
Márk Sági-Kazár 4eb45c9a0f
fix: dex getting started guide URL (#10701) 2020-10-17 16:47:28 -07:00
Harshavardhana ad726b49b4
rename zones to serverSets to avoid terminology conflict (#10679) 
we are bringing in availability zones, we should avoid
zones as per server expansion concept.
 2020-10-15 14:28:50 -07:00
Harshavardhana 9c042a503b
remove deprecate readiness from healthcheck docs (#10659) 2020-10-12 18:56:03 -07:00
Minio Trusted 4a678ad70f Update yaml files to latest version RELEASE.2020-10-12T21-53-21Z 2020-10-12 22:15:06 +00:00
Minio Trusted 071c004f8b Update yaml files to latest version RELEASE.2020-10-09T22-55-05Z 2020-10-09 23:16:30 +00:00
miraculli 1738eb24b1
fix: caching doc README.md missing high watermark (#10646) 2020-10-09 08:32:51 -07:00
Poorna Krishnamoorthy 01498a3e34
fix: add docs for new event types in notification (#10636) 2020-10-06 13:33:23 -07:00
Ravind Kumar 57f0176759
Update KES table to include additional supported KMS providers (#10631) 2020-10-06 11:09:43 -07:00
Minio Trusted 017954e7ea Update yaml files to latest version RELEASE.2020-10-03T02-19-42Z 2020-10-03 02:36:54 +00:00
Minio Trusted 511424a287 Update yaml files to latest version RELEASE.2020-09-26T03-44-56Z 2020-09-26 04:02:42 +00:00
Praveen raj Mani b880796aef
Set the maximum open connections limit in PG and MySQL target configs (#10558) 
As the bulk/recursive delete will require multiple connections to open at an instance,
The default open connections limit will be reached which results in the following error

```FATAL:  sorry, too many clients already```

By setting the open connections to a reasonable value - `2`, We ensure that the max open connections
will not be exhausted and lie under bounds.

The queries are simple inserts/updates/deletes which is operational and sufficient with the
the maximum open connection limit is 2.

Fixes #10553

Allow user configuration for MaxOpenConnections
 2020-09-24 22:20:30 -07:00
poornas e6ab4db6b8
Fix minimum replication workers started (#10560) 
This PR also fixes GetReplicationConfiguration permission
in web-handlers.go to use bucket as resource
 2020-09-24 12:25:41 -07:00
Espen Finnesand df08fd1f03
Update Kubernetes Helm Readme.md (#10559) 
The link to the Minio Helm Chart repository was wrong.
 2020-09-24 08:38:46 -07:00
Minio Trusted ac82f416a4 Update yaml files to latest version RELEASE.2020-09-23T19-18-30Z 2020-09-23 19:38:14 +00:00
飞雪无情 5fdf47b118
Add Chinese documentation for distributed design (#10549) 2020-09-23 09:26:47 -07:00
Harshavardhana 8b74a72b21
fix: rename READY deadline to CLUSTER deadline ENV (#10535) 2020-09-23 09:14:33 -07:00
poornas a4006e23a0
Update replication docs to clarify permissions (#10536) 
Co-authored-by: Klaus Post <klauspost@gmail.com>
 2020-09-22 11:58:04 -07:00
Minio Trusted d73c4f09f3 Update yaml files to latest version RELEASE.2020-09-21T22-31-59Z 2020-09-21 22:58:40 +00:00
飞雪无情 dd4a2d7419
update distributed zh_CN document. (#10531) 2020-09-21 10:54:56 -07:00
Derek Bender 3168e93730
fix typo in healthcheck README.md (#10518) 2020-09-18 09:52:37 -07:00
Aleksey Pogibelev 6f45e303f5
fix tls flag (#10487) 2020-09-18 15:35:54 +05:30
飞雪无情 dff37aa33d
update disk caching zh_CN document. (#10472) 
add disk caching DESIGN zh_CN document.
 2020-09-18 00:07:48 -07:00
Minio Trusted 48d2c03250 Update yaml files to latest version RELEASE.2020-09-17T04-49-20Z 2020-09-17 05:07:58 +00:00
Minio Trusted b2536476c9 Update yaml files to latest version RELEASE.2020-09-16T04-22-35Z 2020-09-16 04:41:00 +00:00
Harshavardhana 9d95937018 update KMS docs indicating deprecation of AUTO_ENCRYPTION env 2020-09-13 16:23:28 -07:00
Minio Trusted eb3ded420e Update yaml files to latest version RELEASE.2020-09-10T22-02-45Z 2020-09-10 22:22:28 +00:00
飞雪无情 a966ccd17d
Support nginx LB with docker-compose example (#10434) 2020-09-10 09:19:17 -07:00
Li Yi 1dce6918c2
fix sysctl.sh quotes which are incompatible with sysctl (#10446) 2020-09-09 17:29:23 -07:00
Minio Trusted 6019628f7d Update yaml files to latest version RELEASE.2020-09-08T23-05-18Z 2020-09-08 23:25:40 +00:00
飞雪无情 a694ba93d9
update kernel tuning zh_CN document (#10433) 2020-09-07 19:21:52 -07:00
Minio Trusted a9aaea0d67 Update yaml files to latest version RELEASE.2020-09-05T07-14-49Z 2020-09-05 07:32:10 +00:00
飞雪无情 746f1585eb
add debugging zh_CN document. (#10401) 2020-09-04 13:03:46 -07:00
Harshavardhana b4e3956e69
update KES docs to talk about 'mc encrypt' command (#10400) 
add a deprecation notice for KMS_AUTO_ENCRYPTION
 2020-09-03 12:43:45 -07:00
Krishna Srinivas c51229493b
azure docs remove default chunk size example (#10406) 2020-09-03 01:31:56 -07:00
Klaus Post 650dccfa9e
cache: Only start at high watermark (#10403) 
Currently, cache purges are triggered as soon as the low watermark is exceeded.
To reduce IO this should only be done when reaching the high watermark.
This simplifies checks and reduces all calls for a GC to go through
`dcache.diskSpaceAvailable(size)`. While a comment claims that 
`dcache.triggerGC <- struct{}{}` was non-blocking I don't see how 
that was possible. Instead, we add a 1 size to the queue channel 
and use channel  semantics to avoid blocking when a GC has 
already been requested.

`bytesToClear` now takes the high watermark into account to it will 
not request any bytes to be cleared until that is reached.
 2020-09-02 17:48:44 -07:00
Minio Trusted d08b4b147d Update yaml files to latest version RELEASE.2020-09-02T18-19-50Z 2020-09-02 18:40:57 +00:00
Harshavardhana 4b6585d249
support 'ldap:user' variable replacement properly (#10391) 
also update `ldap.go` examples with latest
minio-go changes

Fixes #10367
 2020-09-01 12:26:22 +05:30
飞雪无情 cb61e50b51
since its an internal RPC API we do not need to document it. (#10382) 2020-08-31 10:20:07 -07:00
unlimitedbits cd380251b3
Support custom paths for secret files in docker-entrypoint.sh (#10344) 2020-08-28 14:04:29 -07:00
Eco 92cd1eed45
Clarify zone example (#10374) 2020-08-28 14:03:29 -07:00
Harshavardhana db32a24cb6 update kernel tuning docs to indicate sane values 2020-08-28 12:07:55 -07:00
poornas 73a6b4ea11
fix typo in replication docs (#10366) 2020-08-27 12:54:23 -07:00
Minio Trusted c1b88c17cc Update yaml files to latest version RELEASE.2020-08-27T05-16-20Z 2020-08-27 05:36:01 +00:00
Minio Trusted 5a974fb10c Update yaml files to latest version RELEASE.2020-08-26T00-00-49Z 2020-08-26 00:19:30 +00:00
Minio Trusted 7d50a0cfea Update yaml files to latest version RELEASE.2020-08-25T00-21-20Z 2020-08-25 00:44:39 +00:00
飞雪无情 2a8e40f19f
add chroot zh_CN document (#10337) 2020-08-24 13:58:19 -07:00
飞雪无情 21e8440423
add compression zh_CN document (#10330) 2020-08-23 22:07:36 -07:00
Praveen raj Mani d0c910a6f3
Support https and basic-auth for elasticsearch notification target (#10332) 2020-08-23 09:43:48 -07:00
飞雪无情 0ff246653b
CN bucket policy document that is no longer needed can be removed, (#10310) 
which is consistent with the English document.
 2020-08-22 11:05:21 -07:00
飞雪无情 113bcbdb78
update minio limit zh_CN document (#10320) 2020-08-22 11:04:31 -07:00
飞雪无情 331c517a5b
translate bucket versioning design document to zh_CN (#10300) 2020-08-20 13:18:51 -07:00
飞雪无情 2eb5f934d8
add bucket versioning zh_CN document (#10281) 2020-08-18 13:10:26 -07:00
Minio Trusted b43d376a87 Update yaml files to latest version RELEASE.2020-08-18T19-41-00Z 2020-08-18 20:04:22 +00:00
Klaus Post adca28801d
feat: disable Parquet by default (breaking change) (#9920) 
I have built a fuzz test and it crashes heavily in seconds and will OOM shortly after.
It seems like supporting Parquet is basically a completely open way to crash the 
server if you can upload a file and run s3 select on it.

Until Parquet is more hardened it is DISABLED by default since hostile 
crafted input can easily crash the server.

If you are in a controlled environment where it is safe to assume no hostile
content can be uploaded to your cluster you can safely enable Parquet.

To enable Parquet set the environment variable `MINIO_API_SELECT_PARQUET=on`
while starting the MinIO server.

Furthermore, we guard parquet by recover functions.
 2020-08-18 10:23:28 -07:00
Harshavardhana ede86845e5
docs: Add policy variables for resource and conditions (#10278) 
Bonus fix adds LDAP policy variable and clarifies the
usage of policy variables for temporary credentials.

fixes #10197
 2020-08-17 17:39:55 -07:00
Minio Trusted 670724184c Update yaml files to latest version RELEASE.2020-08-16T18-39-38Z 2020-08-16 18:56:41 +00:00
飞雪无情 01a2ccc52f
add bucket retention zh_CN document (#10264) 2020-08-14 22:12:03 -07:00
Minio Trusted 1c865dd119 Update yaml files to latest version RELEASE.2020-08-13T02-39-50Z 2020-08-13 02:57:25 +00:00
飞雪无情 6914b2c99d
Add bucket replication zh_CN document (#10243) 2020-08-11 11:33:01 -07:00
飞雪无情 b4b7a18497
add bucket quota zh_CN document (#10232) 2020-08-10 10:55:34 -07:00
飞雪无情 a49e3647b6
add bucket lifecycle zh_CN document (#10231) 2020-08-09 02:29:25 -07:00
Minio Trusted 954e17c3d0 Update yaml files to latest version RELEASE.2020-08-08T04-50-06Z 2020-08-07 22:32:27 -07:00
Ritesh H Shukla 8049184dcc
fix: documentation changes in replication docs (#10209) 2020-08-07 13:30:52 -07:00
飞雪无情 19c4f3082b
update and improve bucket notifications zh_CN document (#10224) 2020-08-07 12:36:27 -07:00
Minio Trusted 6d64aab420 Update yaml files to latest version RELEASE.2020-08-07T01-23-07Z 2020-08-07 01:43:14 +00:00
poornas adcaa6f9de
fix: Change ListBucketTargets handler (#10217) 
to list all targets across a tenant.
Also fixing some validations.
 2020-08-06 17:10:21 -07:00
飞雪无情 ce129efa09
update and improve config,throttle zh_CN document (#10214) 2020-08-06 11:43:30 -07:00
poornas 121164db56
fix: relax some replication validations (#10210) 
Also inherit storage class from source object
if replication configuration does not have a storage
class specified for destination bucket.
 2020-08-05 20:01:20 -07:00
Minio Trusted 195f95196e Update yaml files to latest version RELEASE.2020-08-05T21-34-13Z 2020-08-05 21:58:17 +00:00
poornas 3acc0ebb81
fix: Change service name in Arn for replication (#10205) 2020-08-05 00:43:18 -07:00
Minio Trusted 1cdaced8b6 Update yaml files to latest version RELEASE.2020-08-04T23-10-51Z 2020-08-04 23:46:06 +00:00
Harshavardhana d90ab904e7
fix: multi-tenant deployment guide to use ellipses format (#10192) 2020-08-04 08:15:37 -07:00
飞雪无情 0cf4539fe8
fix: update and improve zh_CN documentation (#10174) 2020-08-01 11:52:46 -07:00
Minio Trusted 2174a22835 Update yaml files to latest version RELEASE.2020-07-31T03-39-05Z 2020-07-31 03:58:04 +00:00
poornas a8dd7b3eda
Refactor replication target management. (#10154) 
Generalize replication target management so
that remote targets for a bucket can be
managed with ARNs. `mc admin bucket remote`
command will be used to manage targets.
 2020-07-30 19:55:22 -07:00
Harshavardhana 27266f8a54
fix: if OPA set do not enforce policy claim (#10149) 2020-07-28 11:47:57 -07:00
Minio Trusted 98691f75bc Update yaml files to latest version RELEASE.2020-07-27T18-37-02Z 2020-07-27 18:57:10 +00:00
Ritesh H Shukla 416ec316bd
Add clarification for minimum drives per server (#10130) 2020-07-24 22:09:01 -07:00
Minio Trusted 8a23988711 Update yaml files to latest version RELEASE.2020-07-24T22-43-05Z 2020-07-24 23:00:32 +00:00
Harshavardhana 0b5d1bc91d
fix: bucket replication docs (#10104) 
* fix: bucket replication docs

* Update docs/bucket/replication/README.md

Co-authored-by: kannappanr <30541348+kannappanr@users.noreply.github.com>

Co-authored-by: kannappanr <30541348+kannappanr@users.noreply.github.com>
 2020-07-21 22:19:30 -07:00
poornas c43da3005a
Add support for server side bucket replication (#9882) 2020-07-21 17:49:56 -07:00
Minio Trusted ca4c15bc63 Update yaml files to latest version RELEASE.2020-07-22T00-26-33Z 2020-07-22 00:44:03 +00:00
Harshavardhana ec06089eda
fix: re-implement cluster healthcheck (#10101) 2020-07-20 18:31:22 -07:00
Minio Trusted 38f60b3c1d Update yaml files to latest version RELEASE.2020-07-20T02-25-16Z 2020-07-20 02:41:52 +00:00
Harshavardhana 30104cb12b docs: fix veeam document formatting 2020-07-18 18:38:12 -07:00
Minio Trusted 68aaa5bbc3 Update yaml files to latest version RELEASE.2020-07-18T18-48-16Z 2020-07-18 19:04:01 +00:00
Eco 5e8392c8ef
Update Veeam integration doc with immutability references (#10067) 2020-07-16 17:16:53 -07:00
Harshavardhana 07eb24b775
add absolute path for images (#10056) 2020-07-16 00:06:14 -07:00
Harshavardhana cd849bc2ff
update STS docs with new values (#10055) 
Co-authored-by: Poorna <poornas@users.noreply.github.com>
 2020-07-15 14:36:14 -07:00
Harshavardhana ec91fa55db
docs: Add more STS docs with dex and python example (#10047) 2020-07-15 17:25:55 +05:30
Minio Trusted a97ce3c96e Update yaml files to latest version RELEASE.2020-07-14T19-14-30Z 2020-07-14 19:32:27 +00:00
Harshavardhana 369a876ebe
fix: handle array policies in JWT claim (#10041) 
PR #10014 was not complete as only handled
policy claims partially.
 2020-07-14 10:26:47 -07:00
Anis Elleuch 778e9c864f
Move dependency from minio-go v6 to v7 (#10042) 2020-07-14 09:38:05 -07:00
Minio Trusted 8d8f28eae4 Update yaml files to latest version RELEASE.2020-07-13T18-09-56Z 2020-07-13 18:29:35 +00:00
Minio Trusted 3939c6f6e7 Update yaml files to latest version RELEASE.2020-07-12T19-14-17Z 2020-07-12 19:48:42 +00:00
Minio Trusted c2fdf73491 Update yaml files to latest version RELEASE.2020-07-11T21-14-23Z 2020-07-11 21:30:53 +00:00
Harshavardhana 3520e946a2 fix: versioning docs add more examples 2020-07-11 00:57:46 -07:00
Harshavardhana f38adc1865 cleanup security overview guide 2020-07-11 00:34:56 -07:00
Harshavardhana d5ff1c8e3b fix docs image urls to be absolute path 2020-07-11 00:27:30 -07:00
Minio Trusted ad7417bc50 Update yaml files to latest version RELEASE.2020-07-11T06-07-16Z 2020-07-11 06:28:08 +00:00
Harshavardhana ba756cf366
fix: extract array type for policy claim if present (#10014) 2020-07-10 14:48:44 -07:00
Harshavardhana 2e8fc6ebfe
cleanup STS docs (#10003) 2020-07-10 09:07:12 -07:00
Nitish Tiwari 30c251efd3
Add Grafana dashboard (#10000) 2020-07-09 12:01:58 -07:00
Ravind Kumar ee20ebe07a Remove dead link related to DC/OS Deployment Guide (#9996) 2020-07-08 17:38:20 -07:00
Harshavardhana 60417950c7
fix: the versioning/object lock documentation appropriately (#9988) 
- Move the bucket level features into `docs/bucket` directory
- fix issue template and simplify some of them
 2020-07-08 08:44:43 -07:00
Harshavardhana 38eef5ce4c
fix: documentation fixes for docker ENV settings (#9975) 
- update CREDITS file
- fix markdown links
- talk a bit more about upgrades
 2020-07-06 06:42:34 -07:00
Minio Trusted 04de19c870 Update yaml files to latest version RELEASE.2020-07-02T00-15-09Z 2020-07-01 17:36:52 -07:00
Harshavardhana c0ac25bfff
fix: readiness needs to be like liveness (#9941) 
Readiness as no reasoning to be cluster scope
because that is not how the k8s networking works
for pods, all the pods to a deployment are not
sharing the network in a singleton. Instead they
are run as local scopes to themselves, with
readiness failures the pod is potentially taken
out of the network to be resolvable - this
affects the distributed setup in myriad of
different ways.

Instead readiness should behave like liveness
with local scope alone, and should be a dummy
implementation.

This PR all the startup times and overal k8s
startup time dramatically improves.

Added another handler called as `/minio/health/cluster`
to understand the cluster scope health.
 2020-06-30 11:28:27 -07:00
poornas 55a3b071ea
Allow optionally to disable range caching. (#9908) 
The default behavior is to cache each range requested
to cache drive. Add an environment variable
`MINIO_RANGE_CACHE` - when set to off, it disables
range caching and instead downloads entire object
in the background.

Fixes #9870
 2020-06-29 13:25:29 -07:00
Harshavardhana 28a1a17187
docs update versioning images (#9925) 2020-06-27 10:27:23 -07:00
Eco a5e879fbe4
doc: typo fix in versioning README (#9926) 
requets -> requests
 2020-06-27 05:06:31 -07:00
Praveen raj Mani b1705599e1
Fix config leaks and deprecate file-based config setters in NAS gateway (#9884) 
This PR has the following changes

- Removing duplicate lookupConfigs() calls.
- Deprecate admin config APIs for NAS gateways. This will avoid repeated reloads of the config from the disk.
- WatchConfigNASDisk will be removed
- Migration guide for NAS gateways users to migrate to ENV settings.

NOTE: THIS PR HAS A BREAKING CHANGE

Fixes #9875

Co-authored-by: Harshavardhana <harsha@minio.io>
 2020-06-25 15:59:28 +05:30
Ivan Martinez-Ortiz 969b2d2110
Updates the usage documentation of OpenID custom scopes (#9902) 2020-06-24 07:49:09 -07:00
Harshavardhana 21058c34d0
add some description of xl.meta (#9901) 2020-06-22 17:27:54 -07:00
Klaus Post 691dc04fac
Add neater metadata printer (#9898) 
* Just read files from args (more than 1 now supported)
* Pretty print by default. `-ndjson` will disable.
* Check header.
* Support stdin as '-'
* Don't just ignore errors.
 2020-06-22 13:20:22 -07:00
Harshavardhana e92434c2e7
fix: support client customized scopes for OpenID (#9880) 
Fixes #9238
 2020-06-22 12:08:50 -07:00
Minio Trusted 0fff9f9fa6 Update yaml files to latest version RELEASE.2020-06-22T03-12-50Z 2020-06-21 20:26:38 -07:00
Benjamin Sodenkamp 0becf7f03f
Added note to help differentiate MINIO creds from S3 rotating creds (#9887) 2020-06-21 12:39:03 -07:00
Minio Trusted 72743d1590 Update yaml files to latest version RELEASE.2020-06-18T02-23-35Z 2020-06-17 19:33:28 -07:00
Harshavardhana f9aa239973
fix: export prometheus metrics for cache GC triggers (#9815) 
Bonus change to use channel to serialize triggers,
instead of using atomic variables. More efficient
mechanism for synchronization.

Co-authored-by: Nitish Tiwari <nitish@minio.io>
 2020-06-15 09:05:35 -07:00
Minio Trusted 1cd5d7942f Update yaml files to latest version RELEASE.2020-06-14T18-32-17Z 2020-06-14 11:41:18 -07:00
Harshavardhana 4915433bd2
Support bucket versioning (#9377) 
- Implement a new xl.json 2.0.0 format to support,
  this moves the entire marshaling logic to POSIX
  layer, top layer always consumes a common FileInfo
  construct which simplifies the metadata reads.
- Implement list object versions
- Migrate to siphash from crchash for new deployments
  for object placements.

Fixes #2111
 2020-06-12 20:04:01 -07:00
Dan Garrick a53cb236bb
Add enable="on" to set compression command (#9811) 2020-06-11 18:14:13 -07:00
Minio Trusted 761992ca89 Update yaml files to latest version RELEASE.2020-06-12T00-06-19Z 2020-06-12 00:14:09 +00:00
Ritesh H Shukla 920b863955
Fix formatting and wording in distributed docs (#9784) 2020-06-08 18:03:15 -07:00
Harshavardhana c1382c09d9
add reference to distributed doc for clarity (#9783) 2020-06-08 12:30:42 -07:00
Praveen raj Mani 2ce2e88adf
Support mTLS Authentication in Webhooks (#9777) 2020-06-08 05:55:44 -07:00
Harshavardhana 5e529a1c96
simplify context timeout for readiness (#9772) 
additionally also add CORS support to restrict
for specific origin, adds a new config and
updated the documentation as well
 2020-06-04 14:58:34 -07:00
Minio Trusted 7fee96e9de Update yaml files to latest version RELEASE.2020-06-03T22-13-49Z 2020-06-03 22:22:13 +00:00
Minio Trusted 4b68d69188 Update yaml files to latest version RELEASE.2020-06-01T17-28-03Z 2020-06-01 17:36:17 +00:00
Harshavardhana 4301a33371 update docs to mention quota not supported on FS 2020-05-29 07:57:45 -07:00
Minio Trusted f1a03a4ee8 Update yaml files to latest version RELEASE.2020-05-29T14-08-49Z 2020-05-29 14:17:50 +00:00
Minio Trusted 3780ec699f Update yaml files to latest version RELEASE.2020-05-28T23-29-21Z 2020-05-28 23:38:19 +00:00
poornas e5ecd20d44
add docs for caching drives with docker (#9712) 
fixes: #9688
 2020-05-27 09:49:09 -07:00
Harshavardhana 53aaa5d2a5
Export bucket usage counts as part of bucket metrics (#9710) 
Bonus fixes in quota enforcement to use the
new datastructure and use timedValue to cache
a value/reload automatically avoids one less
global variable.
 2020-05-27 06:45:43 -07:00
Justin Clift 5e15b0b844
fix misspelling of Certbot (#9705) 2020-05-26 08:56:50 -07:00
Harshavardhana 6e0575a53d
Revert "Disable crawler in FS/NAS gateway mode (#9695)" (#9702) 
This reverts commit eba423bb9d.

Additionally also address the FS crawler to properly
calculate the sizes for encrypted/compressed content.
 2020-05-25 11:32:53 -07:00
Harshavardhana eba423bb9d
Disable crawler in FS/NAS gateway mode (#9695) 
No one really uses FS for large scale accounting
usage, neither we crawl in NAS gateway mode. It is
worthwhile to simply disable this feature as its
not useful for anyone.

Bonus disable bucket quota ops as well in, FS
and gateway mode
 2020-05-25 00:17:52 -07:00
Kianoosh Hooshmand 1868c7016d
remove additional back-ticks in compression docs (#9685) 2020-05-24 18:34:47 -07:00
谭九鼎 209a3ee7af
fix backtick markdown in chinese docs (#9691) 2020-05-24 09:27:47 -07:00
Harshavardhana e45c90060f
remove references for deprecated dockerfiles and deployment styles (#9675) 2020-05-22 08:40:59 -07:00
Andreas Auernhammer 2164984d2b
fix KMS quickstart guide layout (#9658) 
This commit fixes a layout issue w.r.t. the KMS
Quickstart guide. The problem seems to be caused
by docs server not converting the markdown into html
as expected.

This commit fixes this by converting the ordered list
into subsections.
 2020-05-20 13:55:54 -07:00
Praveen raj Mani 0cc2ed04f5
humanize `timeToFirstByte` and `timeToResponse` upto nanoseconds (#9641) 2020-05-19 18:34:02 -07:00
Andreas Auernhammer b11adfa5cd
simplify the KMS guide and remove unnecessary sections (#9629) 
This commit simplifies the KMS configuration guide by
adding a get started section that uses our KES play instance
at `https://play.min.io:7373`.

Further, it removes sections that we don't recommend for production
anyways (MASTER_KEY).
 2020-05-19 18:33:11 -07:00
poornas 011a2c0b78
Add docs for bucket quota feature (#9503) 
This PR also adds a check to not enforce
bucket quota for server-side metadata copy
of an object onto itself.
 2020-05-16 19:27:33 -07:00
Minio Trusted 0f8df3d340 Update yaml files to latest version RELEASE.2020-05-16T01-33-21Z 2020-05-16 01:40:31 +00:00
Andreas Auernhammer c19ece6921
update KMS guide to reflect latest KES changes (#9591) 
This commit updates the two client env. variables:
```
KES_CLIENT_TLS_KEY_FILE
KES_CLIENT_TLS_CERT_FILE
```

The KES CLI client expects the client key and certificate
as `KES_CLIENT_KEY` resp. `KES_CLIENT_CERT`.
 2020-05-14 14:02:43 -07:00
Harshavardhana 2d735144b9 fix: distributed docs image path 2020-05-11 09:33:55 -07:00
Harshavardhana 0113035237
update distributed setup guide (#9566) 2020-05-11 09:19:10 -07:00
Minio Trusted 5bf3eeaa77 Update yaml files to latest version RELEASE.2020-05-08T02-40-49Z 2020-05-08 02:49:15 +00:00
Harshavardhana 9dda1fd624
Remove B2 gateway implementation (#9547) 
S3 is now natively supported by B2 cloud storage provider
there is no reason to use specialized gateway for B2 anymore,
our current S3 gateway with caching would work with B2.

Resolves #8584
 2020-05-07 19:00:30 -07:00
remche 0674c0075e
add LDAP StartTLS support (#9472) 2020-05-07 15:08:33 -07:00
Harshavardhana 53f4c0fdc0
fix: deprecate old settings and elaborate on tuning (#9542) 
fixes #9538
 2020-05-07 11:08:57 -07:00
Minio Trusted 24f20eb1bd Update yaml files to latest version RELEASE.2020-05-06T23-23-25Z 2020-05-06 23:32:41 +00:00
Minio Trusted 1242dd951a Update yaml files to latest version RELEASE.2020-05-01T22-19-14Z 2020-05-01 22:28:00 +00:00
Andreas Auernhammer d1c8e9f31b
update KMS guide to work with latest KES changes (#9498) 
This commit updates the KMS guide to reflect the
latest changes in KES. Based on internal design
meetings we made some adjustments to the overall
KES configuration.
This commit ensures that the KMS guide contains
a working KES demo-setup with Vault.
 2020-05-01 12:36:30 -07:00
Arthur Lutz da87188ff8
fix: tls doc markdown title (#9487) 2020-04-29 12:28:45 -07:00
Minio Trusted b37a02cddf Update yaml files to latest version RELEASE.2020-04-28T23-56-56Z 2020-04-29 00:05:12 +00:00
Harshavardhana 60d415bb8a
deprecate/remove global WORM mode (#9436) 
global WORM mode is a complex piece for which
the time has passed, with the advent of S3 compatible
object locking and retention implementation global
WORM is sort of deprecated, this has been mentioned
in our documentation for some time, now the time
has come for this to go.
 2020-04-24 16:37:05 -07:00