Commit Graph

250 Commits

Author SHA1 Message Date
Klaus Post ff12080ff5
Remove deprecated io/ioutil (#15707) 2022-09-19 11:05:16 -07:00
Aditya Manthramurthy 3d94c38ec4
Add env variables to configuration APIs output (#15465)
Config export and config get APIs now include environment 
variables set on the server
2022-08-04 22:21:52 -07:00
Andreas Auernhammer d774a3309b
kes: automatically reload KES client certificate (#15450)
This commit adds support for automatically reloading
the MinIO client certificate for authentication to KES.

The client certificate will now be reloaded:
 - when the private key / certificate file changes
 - when a SIGHUP signal is received
 - every 15 minutes

Fixes #14869

Signed-off-by: Andreas Auernhammer <hi@aead.dev>
2022-08-02 16:58:09 -07:00
Harshavardhana 043aaa792d
fix: intrument os.OpenFile differently for Reads and Writes (#15449)
allows us to trace latency for READs or WRITEs
2022-08-01 13:22:43 -07:00
Aditya Manthramurthy 7ac53c07af
fix: passing application configuration to console (#15409)
This is an update to MinIO server after swagger codegen related build
fixes added after issues introduced in 39fd7b0b3b
2022-07-28 18:30:24 -07:00
Aditya Manthramurthy 39fd7b0b3b
Pass multiple IDP config to console (#15270)
This change passes multiple IDP config via a struct 
rather than env variables.
2022-07-22 15:28:02 -07:00
Andreas Auernhammer 242d06274a
kms: add `context.Context` to KMS API calls (#15327)
This commit adds a `context.Context` to the
the KMS `{Stat, CreateKey, GenerateKey}` API
calls.

The context will be used to terminate external calls
as soon as the client requests gets canceled.

A follow-up PR will add a `context.Context` to
the remaining `DecryptKey` API call.

Signed-off-by: Andreas Auernhammer <hi@aead.dev>
2022-07-18 18:54:27 -07:00
Harshavardhana 913e977c8d
remove auto-port warning for console-address (#15260) 2022-07-08 13:36:41 -07:00
Harshavardhana 6722f58668
save MinIO version with each version (8-bytes extra) (#15170)
store MinIO version along with each version in 'xl.meta'
for future purposes, can be used as ways to add specific
code for bug fixes if any.
2022-06-27 03:59:41 -07:00
sota e2e5bd6f19
fix: cant parse comment without '=' in environment file (#15130) 2022-06-21 10:37:15 -07:00
Anis Elleuch 0d00f3a55b
kms: initialize after cli parsing (#15076)
KMS depends on the --certs-dir flag. 

Ensure KMS is initialized after loading the flag.
2022-06-13 13:06:13 -07:00
Harshavardhana af1944f28d
support reading systemctl config automatically on baremetal setups (#15066)
this allows for customers to use `mc admin service restart`
directly even when performing RPM, DEB upgrades. Upon such 'restart'
after upgrade MinIO will re-read the /etc/default/minio for any
newer environment variables.

As long as `MINIO_CONFIG_ENV_FILE=/etc/default/minio` is set, this
is honored.
2022-06-10 09:59:15 -07:00
Shireesh Anjal 4ce81fd07f
Add periodic callhome functionality (#14918)
* Add periodic callhome functionality

Periodically (every 24hrs by default), fetch callhome information and
upload it to SUBNET.

New config keys under the `callhome` subsystem:

enable - Set to `on` for enabling callhome. Default `off`
frequency - Interval between callhome cycles. Default `24h`

* Improvements based on review comments

- Update `enableCallhome` safely
- Rename pctx to ctx
- Block during execution of callhome
- Store parsed proxy URL in global subnet config
- Store callhome URL(s) in constants
- Use existing global transport
- Pass auth token to subnetPostReq
- Use `config.EnableOn` instead of `"on"`

* Use atomic package instead of lock

* Use uber atomic package

* Use `Cancel` instead of `cancel`

Co-authored-by: Harshavardhana <harsha@minio.io>

Co-authored-by: Harshavardhana <harsha@minio.io>
Co-authored-by: Aditya Manthramurthy <donatello@users.noreply.github.com>
2022-06-06 16:14:52 -07:00
Harshavardhana f939222942
add support for extra prometheus labels (#14899)
fixes #14353
2022-05-11 13:04:53 -07:00
Aditya Manthramurthy 0e502899a8
Add support for multiple OpenID providers with role policies (#14223)
- When using multiple providers, claim-based providers are not allowed. All
providers must use role policies.

- Update markdown config to allow `details` HTML element
2022-04-28 18:27:09 -07:00
Daniel Valdivia b7dd61f6bc
Fix double slash subpath for console (#14815)
Signed-off-by: Daniel Valdivia <18384552+dvaldivia@users.noreply.github.com>
2022-04-25 13:05:56 -07:00
Daniel Valdivia c526fa9119
Support console UI access at a subpath on a subdomain (#14761)
fixes #14285 

Signed-off-by: Daniel Valdivia <18384552+dvaldivia@users.noreply.github.com>
2022-04-17 16:01:49 -07:00
Lenin Alevski a3e317773a
Skip commented lines when parsing MinIO configuration file (#14710)
Signed-off-by: Lenin Alevski <alevsk.8772@gmail.com>
2022-04-07 16:02:51 -07:00
Andreas Auernhammer e955aa7f2a
kes: add support for encrypted private keys (#14650)
This commit adds support for encrypted KES
client private keys.

Now, it is possible to encrypt the KES client
private key (`MINIO_KMS_KES_KEY_FILE`) with
a password.

For example, KES CLI already supports the
creation of encrypted private keys:
```
kes identity new --encrypt --key client.key --cert client.crt MinIO
```

To decrypt an encrypted private key, the password
needs to be provided:
```
MINIO_KMS_KES_KEY_PASSWORD=<password>
```

Signed-off-by: Andreas Auernhammer <hi@aead.dev>
2022-03-29 09:53:33 -07:00
Harshavardhana 01ee49045e
fix: handle race in server setup global CI/CD variable (#14579) 2022-03-18 18:21:09 -07:00
Harshavardhana 92a77cc78e
update pkg v1.1.20 to reload certs in k8s always (#14470) 2022-03-04 20:34:39 -08:00
Harshavardhana 0e3bafcc54
improve logs, fix banner formatting (#14456) 2022-03-03 13:21:16 -08:00
Andreas Auernhammer b48f719b8e
kes: remove unnecessary error conversion (#14459)
This commit removes some duplicate code that
converts KES API errors.

This code was added since KES `0.18.0` changed
some exported API errors. However, the KES SDK
handles this error conversion itself.
Therefore, it is not necessary to duplicate this
behavior in MinIO.

See: 21555fa624/error.go (L94)

Signed-off-by: Andreas Auernhammer <hi@aead.dev>
2022-03-03 09:42:37 -08:00
Lenin Alevski 289fcbd08c
KES dependency upgrade (#14454)
- Updating KES dependency to v.0.18.0
- Fixing incompatibility issue when checking for errors during KES key creation

Signed-off-by: Lenin Alevski <alevsk.8772@gmail.com>
2022-03-02 23:03:40 -08:00
Harshavardhana 2d78e20120
enable CI environment additionally for MINIO_CI_CD (#14395)
all CI/CD environments set CI=true this is enough
for MinIO to be run inside CI environments, support
it.
2022-02-23 16:01:59 -08:00
Shireesh Anjal 3882da6ac5
Add subnet proxy config (#14225)
Will store the HTTP(S) proxy URL to use for connecting to SUBNET.
2022-02-01 09:52:38 -08:00
Krishnan Parthasarathi ebc3627c73
further improvements to newXLStorage (#14166)
- create internal erasure volumes only if the disk is unformatted
- return a copy of format data in xlStorage.ReadAll
- parse env vars only once, to be re-used by xl-storage
2022-01-24 17:09:12 -08:00
Harshavardhana 7f214a0e46
use dnscache resolver for resolving command line endpoints (#14135)
this helps in caching the resolved values early on, avoids
causing further resolution for individual nodes when
object layer comes online.

this can speed up our startup time during, upgrades etc by
an order of magnitude.

additional changes in connectLoadInitFormats() and parallelize
all calls that might be potentially blocking.
2022-01-20 13:03:15 -08:00
Klaus Post 0e31cff762
fix: DeleteMultipleObjects to finish even if cancelled + concurrent sets (#14038)
* Process sets concurrently.
* Disconnect context from request.
* Insert context cancellation checks.
* errFileNotFound and errFileVersionNotFound are ok, unless creating delete markers.
2022-01-06 10:47:49 -08:00
Shireesh Anjal c27110e37d
Add timeinfo to health data (#14013)
Capture RoundtripDuration to figure out 
NTP issues in subnet health analyzer.
2022-01-06 01:51:10 -08:00
Harshavardhana f527c708f2
run gofumpt cleanup across code-base (#14015) 2022-01-02 09:15:06 -08:00
Harshavardhana c980804514
trim values from envrionment files (#13991)
trim values to remove any spaces, newlines
from the files while importing credentials
and other values.
2021-12-25 22:02:54 -08:00
Harshavardhana 54ec0a1308
add configurable delta for skipping shards (#13967)
This PR is an attempt to make this configurable
as not all situations have same level of tolerable
delta, i.e disks are replaced days apart or even
hours.

There is also a possibility that nodes have drifted
in time, when NTP is not configured on the system.
2021-12-22 11:43:01 -08:00
Anis Elleuch 5cc16e098c
env: Remove quotes when parsing a config env file (#13953)
The code parsing the config environment file does not remove 
quotes of environment variables values. This commit adds this 
capability.
2021-12-20 13:13:06 -08:00
Harshavardhana 113c7ff49a
add code to parse secrets natively instead of shell scripts (#13883) 2021-12-13 18:23:31 -08:00
Harshavardhana 8144a125ce
check for update in background (#13889) 2021-12-13 09:43:03 -08:00
Shireesh Anjal d29df6714a
Introduce new config `subnet api_key` (#13793)
The earlier approach of using a license token for 
communicating with SUBNET is being replaced 
with a simpler mechanism of API keys. Unlike the 
license which is a JWT token, these API keys will 
be simple UUID tokens and don't have any embedded 
information in them. SUBNET would generate the 
API key on cluster registration, and then it would 
be saved in this config, to be used for subsequent 
communication with SUBNET.
2021-12-03 09:32:11 -08:00
Harshavardhana 24d904d194
reload certs from disk upon SIGHUP (#13792) 2021-12-01 00:38:32 -08:00
Aditya Manthramurthy 4ce6d35e30
Add new `site` config sub-system intended to replace `region` (#13672)
- New sub-system has "region" and "name" fields.

- `region` subsystem is marked as deprecated, however still works, unless the
new region parameter under `site` is set - in this case, the region subsystem is
ignored. `region` subsystem is hidden from top-level help (i.e. from `mc admin
config set myminio`), but appears when specifically requested (i.e. with `mc
admin config set myminio region`).

- MINIO_REGION, MINIO_REGION_NAME are supported as legacy environment variables for server region.

- Adds MINIO_SITE_REGION as the current environment variable to configure the
server region and MINIO_SITE_NAME for the site name.
2021-11-25 13:06:25 -08:00
Harshavardhana 8378bc9958
support dynamic redirect_uri based on incoming 'host' header (#13666)
This feature is useful in situations when console is exposed
over multiple intranent or internet entities when users are
connecting over local IP v/s going through load balancer.

Related console work was merged here

373bfbfe3f
2021-11-16 18:40:39 -08:00
Harshavardhana 661b263e77
add gocritic/ruleguard checks back again, cleanup code. (#13665)
- remove some duplicated code
- reported a bug, separately fixed in #13664
- using strings.ReplaceAll() when needed
- using filepath.ToSlash() use when needed
- remove all non-Go style comments from the codebase

Co-authored-by: Aditya Manthramurthy <donatello@users.noreply.github.com>
2021-11-16 09:28:29 -08:00
Harshavardhana 1016fbb8f9
feat: detect starting from windows explorer (#13570)
Windows users often click on the binary without
knowing MinIO is a command-line tool and should be
run from a terminal. Throw a message to guide them
on what to do.

Co-authored-by: Klaus Post <klauspost@gmail.com>
2021-11-03 14:22:13 -07:00
Harshavardhana 3d5750f31c
update and use rs/dnscache implementation instead of custom (#13348)
additionally optimize for IP only setups, avoid doing
unnecessary lookups if the Dial addr is an IP.

allow support for multiple listeners on same socket,
this is mainly meant for future purposes.
2021-10-05 10:13:04 -07:00
Krishnan Parthasarathi f3aeed77e5
Add immediate inline tiering support (#13298) 2021-10-01 11:58:17 -07:00
Harshavardhana 84dcd25a36 fix: OpenID URL changed in console, adapt to new URL 2021-09-27 19:51:24 -07:00
Anis Elleuch 565d95a377
Enable console logging when server debug is enabled (#13259)
_MINIO_SERVER_DEBUG will enable console logging.
2021-09-21 09:01:29 -07:00
Shireesh Anjal b4364723ef
Add config to store subnet license (#13194)
Command to set subnet license:

`mc admin config set {alias} subnet license={token}`

Signed-off-by: Shireesh Anjal <shireesh@minio.io>
Co-authored-by: Harshavardhana <harsha@minio.io>
2021-09-14 21:54:25 -07:00
Harshavardhana bcc6359dec support Console UI with userInfo claims for OpenID 2021-09-14 17:09:18 -07:00
Harshavardhana a690772cc5
add support to set subnet license for embedded console (#12993) 2021-08-17 11:56:01 -07:00
Krishnan Parthasarathi 65b6f4aa31
Add dynamic reconfiguration of number of transition workers (#12926) 2021-08-11 22:23:56 -07:00
Anis Elleuch 35cbe43b6d
Start gateway when KMS is enabled and encryption is unsupported (#12808)
Before, the gateway will complain that it found KMS configured in the
environment but the gateway mode does not support encryption. This
commit will allow starting of the gateway but ensure that S3 operations
with encryption headers will fail when the gateway doesn't support
encryption. That way, the user can use etcd + KMS and have IAM data
encrypted in the etcd store.

Co-authored-by: Anis Elleuch <anis@min.io>
2021-08-08 12:51:48 -07:00
Harshavardhana 6c0757eea6
fix: simplify APIEndpoints() usage (#12893)
improvements include

- skip IPv6 correctly
- do not set default value for
  MINIO_SERVER_URL, let it be
  configured if not use local IPs

Bonus:

- In healing return error from listPathRaw()
- update console to v0.8.3
2021-08-05 15:01:19 -07:00
Harshavardhana 4d8f81a992
remove replicate set fallback behavior (#12865)
replica set deployment style is an incorrect
style of distributed MinIO deployment, since
statefulsets have been around for many releases
in k8s now.
2021-08-04 15:23:18 -07:00
Harshavardhana e7baf2d7d2 stop all console logging 2021-07-29 23:06:05 -07:00
Harshavardhana 3c34e18a4e
allow multipart uploads for single part multipart (#12821)
its possible that some multipart uploads would have
uploaded only single parts so relying on `len(o.Parts)`
alone is not sufficient, we need to look for ETag
pattern to be absolutely sure.
2021-07-28 22:11:55 -07:00
Poorna Krishnamoorthy b6cd54779c
Increase context timeout for bandwidth throttled reader (#12820)
increase default timeout up to one hour for toy setups.

fixes #12812
2021-07-28 15:20:01 -07:00
Harshavardhana 320e1533c4
use expected MinIO URLs for console (#12770)
when TLS is configured using IPs directly
might interfere and not work properly when
the server is configured with TLS certs but
the certs only have domain certs.

Also additionally allow users to specify
a public accessible URL for console to talk
to MinIO i.e `MINIO_SERVER_URL` this would
allow them to use an external ingress domain
to talk to MinIO. This internally fixes few
problems such as presigned URL generation on
the console UI etc.

This needs to be done additionally for any
MinIO deployments that might have a much more
stricter requirement when running in standalone
mode such as FS or standalone erasure code.
2021-07-21 14:51:16 -07:00
Harshavardhana ab7584eeac update to latest console fs/gateway 2021-07-19 17:49:54 -07:00
Lazzaretti e8cbfa7af2
fix: rename error message for MINIO_BROWSER_REDIRECT_URL (#12700)
rename the error message for MINIO_BROWSER_REDIRECT_URL to match the field name
2021-07-14 12:38:25 -07:00
Harshavardhana a655e7f820
fix: use browser redirect URL for IDP callback (#12689)
if browser_redirect_url is set use that for IDP callback
automatically, if we do not have to set REDIRECT_URI
for OpenID callback URL.
2021-07-12 15:21:07 -07:00
Harshavardhana da0fd5f056
fix: allow customizing console redirection (#12665)
MinIO might be running inside proxies, and
console while being on another port might not be
reachable on a specific port behind such proxies.

For such scenarios customize the redirect URL
such that console can be redirected to correct
proxy endpoint instead.

fixes #12661
2021-07-09 14:27:09 -07:00
Harshavardhana c99d399d09
fix: handle redirects for specific resources (#12629) 2021-07-07 12:04:16 -07:00
Harshavardhana 5a33d3f05a fix: set the correct IDP salt/passphrase 2021-06-30 16:45:52 -07:00
Harshavardhana 4781e7580b
add support for customizing redirect_uri for IDP (#12607) 2021-06-30 16:08:20 -07:00
Harshavardhana 8d1bc65757
allow resetting and reapply config on broken clusters (#12554)
Bonus: remove kms_kes as sub-system, since its ENV only.
- also fixes a crash with etcd cluster without KMS
configured and also if KMS decryption is missing.
2021-06-24 16:24:12 -07:00
Harshavardhana 8f1fe3b761
fix: --console-address when specified endpoints missing (#12534)
Additionally upgrade console dependency for reading
environment variables properly.
2021-06-20 23:04:47 -07:00
Harshavardhana cdeccb5510
feat: Deprecate embedded browser and import console (#12460)
This feature also changes the default port where
the browser is running, now the port has moved
to 9001 and it can be configured with

```
--console-address ":9001"
```
2021-06-17 20:27:04 -07:00
ebozduman b154581b65
fix: partially defined cred env vars cause "minio gateway s3" to fail (#12228)
Both credential env vars not needed to start s3 gateway
2021-06-10 22:28:09 -07:00
Harshavardhana 66d549c05d
remove support for deprecated MINIO_KMS_MASTER_KEY (#12463) 2021-06-08 18:50:14 -07:00
Poorna Krishnamoorthy f199afcd6c
tiering: add aws role support for s3 (#12424)
Signed-off-by: Poorna Krishnamoorthy <poorna@minio.io>
2021-06-04 12:47:00 -07:00
Harshavardhana 1f262daf6f
rename all remaining packages to internal/ (#12418)
This is to ensure that there are no projects
that try to import `minio/minio/pkg` into
their own repo. Any such common packages should
go to `https://github.com/minio/pkg`
2021-06-01 14:59:40 -07:00
Harshavardhana 81d5688d56
move the dependency to minio/pkg for common libraries (#12397) 2021-05-28 15:17:01 -07:00
Harshavardhana 3d9873106d
feat: distributed setup can start now with default credentials (#12303)
In lieu of new changes coming for server command line, this
change is to deprecate strict requirement for distributed setups
to provide root credentials.

Bonus: remove MINIO_WORM warning from April 2020, it is time to
remove this warning.
2021-05-17 08:45:22 -07:00
Andreas Auernhammer 9cd9f5a0b3
check that we can reach KES server and that the default key exists (#12291)
This commit adds a check to the MinIO server setup that verifies
that MinIO can reach KES, if configured, and that the default key
exists. If the default key does not exist it will create it
automatically.

Signed-off-by: Andreas Auernhammer <aead@mail.de>
2021-05-13 11:13:31 -07:00
Andreas Auernhammer d8eb7d3e15
kms: replace KES client implementation with minio/kes (#12207)
This commit replaces the custom KES client implementation
with the KES SDK from https://github.com/minio/kes

The SDK supports multi-server client load-balancing and
requests retry out of the box. Therefore, this change reduces
the overall complexity within the MinIO server and there
is no need to maintain two separate client implementations.

Signed-off-by: Andreas Auernhammer <aead@mail.de>
2021-05-10 18:15:11 -07:00
Andreas Auernhammer 4815f92fa8
fix MINIO_KMS_SECRET_KEY env. variable parsing (#12200)
This commit fixes a bug when parsing the env. variable
`MINIO_KMS_SECRET_KEY`. Before, the env. variable
name - instead of its value - was parsed. This (obviously)
did not work properly.

This commit fixes this.

Signed-off-by: Andreas Auernhammer <aead@mail.de>
2021-04-30 18:47:30 -07:00
Harshavardhana f7a87b30bf Revert "deprecate embedded browser (#12163)"
This reverts commit 736d8cbac4.

Bring contrib files for older contributions
2021-04-30 08:50:39 -07:00
Harshavardhana 8cd89e10ea Revert "fix: remove deprecated MINIO_ACCESS_KEY, MINIO_SECRET_KEY envs (#12173)"
This reverts commit b0baaeaa3d.
2021-04-29 10:56:53 -07:00
Harshavardhana b0baaeaa3d
fix: remove deprecated MINIO_ACCESS_KEY, MINIO_SECRET_KEY envs (#12173) 2021-04-27 22:41:24 -07:00
Harshavardhana 736d8cbac4
deprecate embedded browser (#12163)
https://github.com/minio/console takes over the functionality for the
future object browser development

Signed-off-by: Harshavardhana <harsha@minio.io>
2021-04-27 10:52:12 -07:00
Harshavardhana 26544848ea
remove legacy master_key support by June (#12153)
Signed-off-by: Harshavardhana <harsha@minio.io>
2021-04-26 16:02:05 -07:00
Andreas Auernhammer f7feff8665
avoid parsing MINIO_KMS_MASTER_KEY as base64 (#12149)
This commit reverts a change that added support for
parsing base64-encoded keys set via `MINIO_KMS_MASTER_KEY`.

The env. variable `MINIO_KMS_MASTER_KEY` is deprecated and
should ONLY support parsing existing keys - not the new format.

Any new deployment should use `MINIO_KMS_SECRET_KEY`. The legacy
env. variable `MINIO_KMS_MASTER_KEY` will be removed at some point
in time.

Signed-off-by: Andreas Auernhammer <aead@mail.de>
2021-04-25 11:04:31 -07:00
Harshavardhana f420996dfa
fix: allow parsing keys in both new and old format (#12144)
Bonus fix fallback to decrypt previously
encrypted content as well using older master
key ciphertext format.

Signed-off-by: Harshavardhana <harsha@minio.io>
2021-04-24 19:05:25 -07:00
Krishnan Parthasarathi c829e3a13b Support for remote tier management (#12090)
With this change, MinIO's ILM supports transitioning objects to a remote tier.
This change includes support for Azure Blob Storage, AWS S3 compatible object
storage incl. MinIO and Google Cloud Storage as remote tier storage backends.

Some new additions include:

 - Admin APIs remote tier configuration management

 - Simple journal to track remote objects to be 'collected'
   This is used by object API handlers which 'mutate' object versions by
   overwriting/replacing content (Put/CopyObject) or removing the version
   itself (e.g DeleteObjectVersion).

 - Rework of previous ILM transition to fit the new model
   In the new model, a storage class (a.k.a remote tier) is defined by the
   'remote' object storage type (one of s3, azure, GCS), bucket name and a
   prefix.

* Fixed bugs, review comments, and more unit-tests

- Leverage inline small object feature
- Migrate legacy objects to the latest object format before transitioning
- Fix restore to particular version if specified
- Extend SharedDataDirCount to handle transitioned and restored objects
- Restore-object should accept version-id for version-suspended bucket (#12091)
- Check if remote tier creds have sufficient permissions
- Bonus minor fixes to existing error messages

Co-authored-by: Poorna Krishnamoorthy <poorna@minio.io>
Co-authored-by: Krishna Srinivas <krishna@minio.io>
Signed-off-by: Harshavardhana <harsha@minio.io>
2021-04-23 11:58:53 -07:00
Harshavardhana 069432566f update license change for MinIO
Signed-off-by: Harshavardhana <harsha@minio.io>
2021-04-23 11:58:53 -07:00
Andreas Auernhammer 3455f786fa kms: encrypt IAM/config data with the KMS (#12041)
This commit changes the config/IAM encryption
process. Instead of encrypting config data
(users, policies etc.) with the root credentials
MinIO now encrypts this data with a KMS - if configured.

Therefore, this PR moves the MinIO-KMS configuration (via
env. variables) to a "top-level" configuration.
The KMS configuration cannot be stored in the config file
since it is used to decrypt the config file in the first
place.

As a consequence, this commit also removes support for
Hashicorp Vault - which has been deprecated anyway.

Signed-off-by: Andreas Auernhammer <aead@mail.de>
2021-04-22 09:51:09 -07:00
Harshavardhana 89d58bec16
avoid frequent DNS lookups for baremetal setups (#11972)
bump up the DNS cache for baremetal setups upto 10 minutes
2021-04-08 17:51:59 -07:00
Klaus Post 9efcb9e15c
Fix listPathRaw/WalkDir cancelation (#11905)
In #11888 we observe a lot of running, WalkDir calls.

There doesn't appear to be any listerners for these calls, so they should be aborted.

Ensure that WalkDir aborts when upstream cancels the request.

Fixes #11888
2021-03-26 11:18:30 -07:00
Poorna Krishnamoorthy 2f29719e6b
resize replication worker pool dynamically after config update (#11737) 2021-03-09 02:56:42 -08:00
Harshavardhana ffea6fcf09
fix: rename crawler as scanner in config (#11549) 2021-02-17 12:04:11 -08:00
Harshavardhana 567f7bdd05 fix: verify overlapping domains when > 1 2021-01-28 13:08:53 -08:00
Harshavardhana 6cd255d516
fix: allow updated domain names in federation (#11365)
additionally also disallow overlapping domain names
2021-01-28 11:44:48 -08:00
Harshavardhana 6a95f412c9
avoid double CORS headers in federation (#11334)
CORS proxying adds double headers one
by the receiving server, one by proxied
server. Remove them before proxying
when 'Origin' header is found.
2021-01-23 18:27:23 -08:00
Harshavardhana 1b453728a3
initialize forwarder after init() to avoid crashes (#11330)
DNSCache dialer is a global value initialized in
init(), whereas `go` keeps `var =` before `init()`
, also we don't need to keep proxy routers as
global entities - register the forwarder as
necessary to avoid crashes.
2021-01-22 15:37:41 -08:00
Harshavardhana c222bde14b
fix: use common logging implementation for DNSCache (#11284) 2021-01-15 14:04:56 -08:00
Harshavardhana cb0eaeaad8
feat: migrate to ROOT_USER/PASSWORD from ACCESS/SECRET_KEY (#11185) 2021-01-05 10:22:57 -08:00
Harshavardhana d0027c3c41
do not use large buffers if not necessary (#11220)
without this change, there is a performance
regression for small objects GETs, this makes
the overall speed to go back to pre '59d363'
commit days.
2021-01-04 18:51:52 -08:00
Harshavardhana 3e792ae2a2
fix: change defaults for DNS cache dialer (#11145) 2020-12-21 09:33:29 -08:00
Harshavardhana f714840da7
add _MINIO_SERVER_DEBUG env for enabling debug messages (#11128) 2020-12-17 16:52:47 -08:00
Harshavardhana 2eb52ca5f4
fix: heal bucket metadata right before healing bucket (#11097)
optimization mainly to avoid listing the entire
`.minio.sys/buckets/.minio.sys` directory, this
can get really huge and comes in the way of startup
routines, contents inside `.minio.sys/buckets/.minio.sys`
are rather transient and not necessary to be healed.
2020-12-13 11:57:08 -08:00