MyFavMirrors
/
minio
mirror of https://github.com/minio/minio.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

cleanup markdown docs across multiple files (#14296) 

enable markdown-linter
This commit is contained in:
Harshavardhana 2022-02-11 16:51:25 -08:00 committed by GitHub
parent 2c0f121550
commit e3e0532613
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
71 changed files with 1023 additions and 595 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

25
.github/workflows/markdown-lint.yaml vendored Normal file
View File

@ -0,0 +1,25 @@
name: Markdown Linter
on:
pull_request:
branches:
- master
# This ensures that previous jobs for the PR are canceled when the PR is
# updated.
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref }}
cancel-in-progress: true
jobs:
lint:
name: Lint all docs
runs-on: ubuntu-latest
steps:
- name: Check out code
uses: actions/checkout@v2
- name: Lint all docs
run: |
npm install -g markdownlint-cli
markdownlint --fix '**/*.md' --disable MD013 MD025 MD040 MD024

1
COMPLIANCE.md
View File

@ -1,4 +1,5 @@
# AGPLv3 Compliance
We have designed MinIO as an Open Source software for the Open Source software community. This requires applications to consider whether their usage of MinIO is in compliance with the GNU AGPLv3 [license](https://github.com/minio/minio/blob/master/LICENSE).
MinIO cannot make the determination as to whether your application's usage of MinIO is in compliance with the AGPLv3 license requirements. You should instead rely on your own legal counsel or licensing specialists to audit and ensure your application is in compliance with the licenses of MinIO and all other open-source projects with which your application integrates or interacts. We understand that AGPLv3 licensing is complex and nuanced. It is for that reason we strongly encourage using experts in licensing to make any such determinations around compliance instead of relying on apocryphal or anecdotal advice.

25
CONTRIBUTING.md
View File

@ -7,15 +7,17 @@
Start by forking the MinIO GitHub repository, make changes in a branch and then send a pull request. We encourage pull requests to discuss code changes. Here are the steps in details:
### Setup your MinIO GitHub Repository
Fork [MinIO upstream](https://github.com/minio/minio/fork) source repository to your own personal repository. Copy the URL of your MinIO fork (you will need it for the `git clone` command below).
```sh
$ git clone https://github.com/minio/minio
$ go install -v
$ ls /go/bin/minio
git clone https://github.com/minio/minio
go install -v
ls /go/bin/minio
```
### Set up git remote as ``upstream``
```sh
$ cd minio
$ git remote add upstream https://github.com/minio/minio
@ -25,13 +27,15 @@ $ git merge upstream/master
```
### Create your feature branch
Before making code changes, make sure you create a separate branch for these changes
```
$ git checkout -b my-new-feature
git checkout -b my-new-feature
```
### Test MinIO server changes
After your code changes, make sure
- To add test cases for the new code. If you have questions about how to do it, please ask on our [Slack](https://slack.min.io) channel.
@ -40,29 +44,38 @@ After your code changes, make sure
- To run `make test` and `make build` completes.
### Commit changes
After verification, commit your changes. This is a [great post](https://chris.beams.io/posts/git-commit/) on how to write useful commit messages
```
$ git commit -am 'Add some feature'
git commit -am 'Add some feature'
```
### Push to the branch
Push your locally committed changes to the remote origin (your fork)
```
$ git push origin my-new-feature
git push origin my-new-feature
```
### Create a Pull Request
Pull requests can be created via GitHub. Refer to [this document](https://help.github.com/articles/creating-a-pull-request/) for detailed steps on how to create a pull request. After a Pull Request gets peer reviewed and approved, it will be merged.
## FAQs
### How does ``MinIO`` manage dependencies?
``MinIO`` uses `go mod` to manage its dependencies.
- Run `go get foo/bar` in the source folder to add the dependency to `go.mod` file.
To remove a dependency
- Edit your code and remove the import reference.
- Run `go mod tidy` in the source folder to remove dependency from `go.mod` file.
### What are the coding guidelines for MinIO?
``MinIO`` is fully conformant with Golang style. Refer: [Effective Go](https://github.com/golang/go/wiki/CodeReviewComments) article from Golang project. If you observe offending code, please feel free to send a pull request or ping us on [Slack](https://slack.min.io).

54
README.md
View File

@ -1,4 +1,5 @@
# MinIO Quickstart Guide
[![Slack](https://slack.min.io/slack?type=svg)](https://slack.min.io) [![Docker Pulls](https://img.shields.io/docker/pulls/minio/minio.svg?maxAge=604800)](https://hub.docker.com/r/minio/minio/) [![license](https://img.shields.io/badge/license-AGPL%20V3-blue)](https://github.com/minio/minio/blob/master/LICENSE)
[![MinIO](https://raw.githubusercontent.com/minio/minio/master/.github/logo.svg?sanitize=true)](https://min.io)
@ -26,12 +27,12 @@ podman run -p 9000:9000 -p 9001:9001 \
```
The MinIO deployment starts using default root credentials `minioadmin:minioadmin`. You can test the deployment using the MinIO Console, an embedded
object browser built into MinIO Server. Point a web browser running on the host machine to http://127.0.0.1:9000 and log in with the
object browser built into MinIO Server. Point a web browser running on the host machine to <http://127.0.0.1:9000> and log in with the
root credentials. You can use the Browser to create buckets, upload objects, and browse the contents of the MinIO server.
You can also connect using any S3-compatible tool, such as the MinIO Client `mc` commandline tool. See
[Test using MinIO Client `mc`](#test-using-minio-client-mc) for more information on using the `mc` commandline tool. For application developers,
see https://docs.min.io/docs/ and click **MinIO SDKs** in the navigation to view MinIO SDKs for supported languages.
see <https://docs.min.io/docs/> and click **MinIO SDKs** in the navigation to view MinIO SDKs for supported languages.
> NOTE: To deploy MinIO on with persistent storage, you must map local persistent directories from the host OS to the container using the `podman -v` option. For example, `-v /mnt/data:/data` maps the host OS drive at `/mnt/data` to `/data` on the container.
@ -57,9 +58,9 @@ brew uninstall minio
brew install minio/stable/minio
```
The MinIO deployment starts using default root credentials `minioadmin:minioadmin`. You can test the deployment using the MinIO Console, an embedded web-based object browser built into MinIO Server. Point a web browser running on the host machine to http://127.0.0.1:9000 and log in with the root credentials. You can use the Browser to create buckets, upload objects, and browse the contents of the MinIO server.
The MinIO deployment starts using default root credentials `minioadmin:minioadmin`. You can test the deployment using the MinIO Console, an embedded web-based object browser built into MinIO Server. Point a web browser running on the host machine to <http://127.0.0.1:9000> and log in with the root credentials. You can use the Browser to create buckets, upload objects, and browse the contents of the MinIO server.
You can also connect using any S3-compatible tool, such as the MinIO Client `mc` commandline tool. See [Test using MinIO Client `mc`](#test-using-minio-client-mc) for more information on using the `mc` commandline tool. For application developers, see https://docs.min.io/docs/ and click **MinIO SDKs** in the navigation to view MinIO SDKs for supported languages.
You can also connect using any S3-compatible tool, such as the MinIO Client `mc` commandline tool. See [Test using MinIO Client `mc`](#test-using-minio-client-mc) for more information on using the `mc` commandline tool. For application developers, see <https://docs.min.io/docs/> and click **MinIO SDKs** in the navigation to view MinIO SDKs for supported languages.
## Binary Download
@ -71,9 +72,9 @@ chmod +x minio
./minio server /data
```
The MinIO deployment starts using default root credentials `minioadmin:minioadmin`. You can test the deployment using the MinIO Console, an embedded web-based object browser built into MinIO Server. Point a web browser running on the host machine to http://127.0.0.1:9000 and log in with the root credentials. You can use the Browser to create buckets, upload objects, and browse the contents of the MinIO server.
The MinIO deployment starts using default root credentials `minioadmin:minioadmin`. You can test the deployment using the MinIO Console, an embedded web-based object browser built into MinIO Server. Point a web browser running on the host machine to <http://127.0.0.1:9000> and log in with the root credentials. You can use the Browser to create buckets, upload objects, and browse the contents of the MinIO server.
You can also connect using any S3-compatible tool, such as the MinIO Client `mc` commandline tool. See [Test using MinIO Client `mc`](#test-using-minio-client-mc) for more information on using the `mc` commandline tool. For application developers, see https://docs.min.io/docs/ and click **MinIO SDKs** in the navigation to view MinIO SDKs for supported languages.
You can also connect using any S3-compatible tool, such as the MinIO Client `mc` commandline tool. See [Test using MinIO Client `mc`](#test-using-minio-client-mc) for more information on using the `mc` commandline tool. For application developers, see <https://docs.min.io/docs/> and click **MinIO SDKs** in the navigation to view MinIO SDKs for supported languages.
# GNU/Linux
@ -91,14 +92,14 @@ The following table lists supported architectures. Replace the `wget` URL with t
| Architecture | URL |
| -------- | ------ |
| 64-bit Intel/AMD | https://dl.min.io/server/minio/release/linux-amd64/minio |
| 64-bit ARM | https://dl.min.io/server/minio/release/linux-arm64/minio |
| 64-bit PowerPC LE (ppc64le) | https://dl.min.io/server/minio/release/linux-ppc64le/minio |
| IBM Z-Series (S390X) | https://dl.min.io/server/minio/release/linux-s390x/minio |
| 64-bit Intel/AMD | <https://dl.min.io/server/minio/release/linux-amd64/minio> |
| 64-bit ARM | <https://dl.min.io/server/minio/release/linux-arm64/minio> |
| 64-bit PowerPC LE (ppc64le) | <https://dl.min.io/server/minio/release/linux-ppc64le/minio> |
| IBM Z-Series (S390X) | <https://dl.min.io/server/minio/release/linux-s390x/minio> |
The MinIO deployment starts using default root credentials `minioadmin:minioadmin`. You can test the deployment using the MinIO Console, an embedded web-based object browser built into MinIO Server. Point a web browser running on the host machine to http://127.0.0.1:9000 and log in with the root credentials. You can use the Browser to create buckets, upload objects, and browse the contents of the MinIO server.
The MinIO deployment starts using default root credentials `minioadmin:minioadmin`. You can test the deployment using the MinIO Console, an embedded web-based object browser built into MinIO Server. Point a web browser running on the host machine to <http://127.0.0.1:9000> and log in with the root credentials. You can use the Browser to create buckets, upload objects, and browse the contents of the MinIO server.
You can also connect using any S3-compatible tool, such as the MinIO Client `mc` commandline tool. See [Test using MinIO Client `mc`](#test-using-minio-client-mc) for more information on using the `mc` commandline tool. For application developers, see https://docs.min.io/docs/ and click **MinIO SDKs** in the navigation to view MinIO SDKs for supported languages.
You can also connect using any S3-compatible tool, such as the MinIO Client `mc` commandline tool. See [Test using MinIO Client `mc`](#test-using-minio-client-mc) for more information on using the `mc` commandline tool. For application developers, see <https://docs.min.io/docs/> and click **MinIO SDKs** in the navigation to view MinIO SDKs for supported languages.
> NOTE: Standalone MinIO servers are best suited for early development and evaluation. Certain features such as versioning, object locking, and bucket replication require distributed deploying MinIO with Erasure Coding. For extended development and production, deploy MinIO with Erasure Coding enabled - specifically, with a *minimum* of 4 drives per MinIO server. See [MinIO Erasure Code Quickstart Guide](https://docs.min.io/docs/minio-erasure-code-quickstart-guide.html) for more complete documentation.
@ -116,9 +117,9 @@ Use the following command to run a standalone MinIO server on the Windows host.
minio.exe server D:\
```
The MinIO deployment starts using default root credentials `minioadmin:minioadmin`. You can test the deployment using the MinIO Console, an embedded web-based object browser built into MinIO Server. Point a web browser running on the host machine to http://127.0.0.1:9000 and log in with the root credentials. You can use the Browser to create buckets, upload objects, and browse the contents of the MinIO server.
The MinIO deployment starts using default root credentials `minioadmin:minioadmin`. You can test the deployment using the MinIO Console, an embedded web-based object browser built into MinIO Server. Point a web browser running on the host machine to <http://127.0.0.1:9000> and log in with the root credentials. You can use the Browser to create buckets, upload objects, and browse the contents of the MinIO server.
You can also connect using any S3-compatible tool, such as the MinIO Client `mc` commandline tool. See [Test using MinIO Client `mc`](#test-using-minio-client-mc) for more information on using the `mc` commandline tool. For application developers, see https://docs.min.io/docs/ and click **MinIO SDKs** in the navigation to view MinIO SDKs for supported languages.
You can also connect using any S3-compatible tool, such as the MinIO Client `mc` commandline tool. See [Test using MinIO Client `mc`](#test-using-minio-client-mc) for more information on using the `mc` commandline tool. For application developers, see <https://docs.min.io/docs/> and click **MinIO SDKs** in the navigation to view MinIO SDKs for supported languages.
> NOTE: Standalone MinIO servers are best suited for early development and evaluation. Certain features such as versioning, object locking, and bucket replication require distributed deploying MinIO with Erasure Coding. For extended development and production, deploy MinIO with Erasure Coding enabled - specifically, with a *minimum* of 4 drives per MinIO server. See [MinIO Erasure Code Quickstart Guide](https://docs.min.io/docs/minio-erasure-code-quickstart-guide.html) for more complete documentation.
@ -130,9 +131,9 @@ Use the following commands to compile and run a standalone MinIO server from sou
GO111MODULE=on go install github.com/minio/minio@latest
```
The MinIO deployment starts using default root credentials `minioadmin:minioadmin`. You can test the deployment using the MinIO Console, an embedded web-based object browser built into MinIO Server. Point a web browser running on the host machine to http://127.0.0.1:9000 and log in with the root credentials. You can use the Browser to create buckets, upload objects, and browse the contents of the MinIO server.
The MinIO deployment starts using default root credentials `minioadmin:minioadmin`. You can test the deployment using the MinIO Console, an embedded web-based object browser built into MinIO Server. Point a web browser running on the host machine to <http://127.0.0.1:9000> and log in with the root credentials. You can use the Browser to create buckets, upload objects, and browse the contents of the MinIO server.
You can also connect using any S3-compatible tool, such as the MinIO Client `mc` commandline tool. See [Test using MinIO Client `mc`](#test-using-minio-client-mc) for more information on using the `mc` commandline tool. For application developers, see https://docs.min.io/docs/ and click **MinIO SDKs** in the navigation to view MinIO SDKs for supported languages.
You can also connect using any S3-compatible tool, such as the MinIO Client `mc` commandline tool. See [Test using MinIO Client `mc`](#test-using-minio-client-mc) for more information on using the `mc` commandline tool. For application developers, see <https://docs.min.io/docs/> and click **MinIO SDKs** in the navigation to view MinIO SDKs for supported languages.
> NOTE: Standalone MinIO servers are best suited for early development and evaluation. Certain features such as versioning, object locking, and bucket replication require distributed deploying MinIO with Erasure Coding. For extended development and production, deploy MinIO with Erasure Coding enabled - specifically, with a *minimum* of 4 drives per MinIO server. See [MinIO Erasure Code Quickstart Guide](https://docs.min.io/docs/minio-erasure-code-quickstart-guide.html) for more complete documentation.
@ -196,6 +197,7 @@ service iptables restart
```
## Pre-existing data
When deployed on a single drive, MinIO server lets clients access any pre-existing data in the data directory. For example, if MinIO is started with the command `minio server /mnt/data`, any pre-existing data in the `/mnt/data` directory would be accessible to the clients.
The above statement is also valid for all gateway backends.
@ -203,11 +205,13 @@ The above statement is also valid for all gateway backends.
# Test MinIO Connectivity
## Test using MinIO Console
MinIO Server comes with an embedded web based object browser. Point your web browser to http://127.0.0.1:9000 to ensure your server has started successfully.
MinIO Server comes with an embedded web based object browser. Point your web browser to <http://127.0.0.1:9000> to ensure your server has started successfully.
> NOTE: MinIO runs console on random port by default if you wish choose a specific port use `--console-address` to pick a specific interface and port.
### Things to consider
MinIO redirects browser access requests to the configured server port (i.e. `127.0.0.1:9000`) to the configured Console port. MinIO uses the hostname or IP address specified in the request when building the redirect URL. The URL and port *must* be accessible by the client for the redirection to work.
For deployments behind a load balancer, proxy, or ingress rule where the MinIO host IP address or port is not public, use the `MINIO_BROWSER_REDIRECT_URL` environment variable to specify the external hostname for the redirect. The LB/Proxy must have rules for directing traffic to the Console port specifically.
@ -218,36 +222,40 @@ Similarly, if your TLS certificates do not have the IP SAN for the MinIO server
For example: `export MINIO_SERVER_URL="https://minio.example.net"`
| Dashboard | Creating a bucket |
| ------------- | ------------- |
| ![Dashboard](https://github.com/minio/minio/blob/master/docs/screenshots/pic1.png?raw=true) | ![Dashboard](https://github.com/minio/minio/blob/master/docs/screenshots/pic2.png?raw=true) |
## Test using MinIO Client `mc`
`mc` provides a modern alternative to UNIX commands like ls, cat, cp, mirror, diff etc. It supports filesystems and Amazon S3 compatible cloud storage services. Follow the MinIO Client [Quickstart Guide](https://docs.min.io/docs/minio-client-quickstart-guide) for further instructions.
# Upgrading MinIO
Upgrades require zero downtime in MinIO, all upgrades are non-disruptive, all transactions on MinIO are atomic. So upgrading all the servers simultaneously is the recommended way to upgrade MinIO.
> NOTE: requires internet access to update directly from https://dl.min.io, optionally you can host any mirrors at https://my-artifactory.example.com/minio/
> NOTE: requires internet access to update directly from <https://dl.min.io>, optionally you can host any mirrors at <https://my-artifactory.example.com/minio/>
- For deployments that installed the MinIO server binary by hand, use [`mc admin update`](https://docs.min.io/minio/baremetal/reference/minio-mc-admin/mc-admin-update.html)
```
```sh
mc admin update <minio alias, e.g., myminio>
```
- For deployments without external internet access (e.g. airgapped environments), download the binary from https://dl.min.io and replace the existing MinIO binary let's say for example `/opt/bin/minio`, apply executable permissions `chmod +x /opt/bin/minio` and do `mc admin service restart alias/`.
- For deployments without external internet access (e.g. airgapped environments), download the binary from <https://dl.min.io> and replace the existing MinIO binary let's say for example `/opt/bin/minio`, apply executable permissions `chmod +x /opt/bin/minio` and do `mc admin service restart alias/`.
- For RPM/DEB installations, upgrade packages **parallelly** on all servers. Once upgraded, perform `systemctl restart minio` across all nodes in **parallel**. RPM/DEB based installations are usually automated using [`ansible`](https://github.com/minio/ansible-minio).
## Upgrade Checklist
- Test all upgrades in a lower environment (DEV, QA, UAT) before applying to production. Performing blind upgrades in production environments carries significant risk.
- Read the release notes for the targeted MinIO release *before* performing any installation, there is no forced requirement to upgrade to latest releases every week. If it has a bug fix you are looking for then yes, else avoid actively upgrading a running production system.
- Make sure MinIO process has write access to `/opt/bin` if you plan to use `mc admin update`. This is needed for MinIO to download the latest binary from https://dl.min.io and save it locally for upgrades.
- Make sure MinIO process has write access to `/opt/bin` if you plan to use `mc admin update`. This is needed for MinIO to download the latest binary from <https://dl.min.io> and save it locally for upgrades.
- `mc admin update` is not supported in kubernetes/container environments, container environments provide their own mechanisms for container updates.
- **We do not recommend upgrading one MinIO server at a time, the product is designed to support parallel upgrades please follow our recommended guidelines.**
# Explore Further
- [MinIO Erasure Code QuickStart Guide](https://docs.min.io/docs/minio-erasure-code-quickstart-guide)
- [Use `mc` with MinIO Server](https://docs.min.io/docs/minio-client-quickstart-guide)
- [Use `aws-cli` with MinIO Server](https://docs.min.io/docs/aws-cli-with-minio)
@ -256,9 +264,11 @@ mc admin update <minio alias, e.g., myminio>
- [The MinIO documentation website](https://docs.min.io)
# Contribute to MinIO Project
Please follow MinIO [Contributor's Guide](https://github.com/minio/minio/blob/master/CONTRIBUTING.md)
# License
- MinIO source is licensed under the GNU AGPLv3 license that can be found in the [LICENSE](https://github.com/minio/minio/blob/master/LICENSE) file.
- MinIO [Documentation](https://github.com/minio/minio/tree/master/docs) © 2021 by MinIO, Inc is licensed under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/).
- [License Compliance](https://github.com/minio/minio/blob/master/COMPLIANCE.md)

9
SECURITY.md
View File

@ -18,9 +18,10 @@ you need access credentials for a successful exploit).
If you have not received a reply to your email within 48 hours or you have not heard from the security team
for the past five days please contact the security team directly:
- Primary security coordinator: aead@min.io
- Secondary coordinator: harsha@min.io
- If you receive no response: dev@min.io
- Primary security coordinator: aead@min.io
- Secondary coordinator: harsha@min.io
- If you receive no response: dev@min.io
### Disclosure Process
@ -32,7 +33,7 @@ MinIO uses the following disclosure process:
If the report is rejected the response explains why.
3. Code is audited to find any potential similar problems.
4. Fixes are prepared for the latest release.
5. On the date that the fixes are applied a security advisory will be published on https://blog.min.io.
5. On the date that the fixes are applied a security advisory will be published on <https://blog.min.io>.
Please inform us in your report email whether MinIO should mention your contribution w.r.t. fixing
the security issue. By default MinIO will **not** publish this information to protect your privacy.

17
VULNERABILITY_REPORT.md
View File

@ -1,11 +1,11 @@
## Vulnerability Management Policy
# Vulnerability Management Policy
This document formally describes the process of addressing and managing a
reported vulnerability that has been found in the MinIO server code base,
any directly connected ecosystem component or a direct / indirect dependency
of the code base.
### Scope
## Scope
The vulnerability management policy described in this document covers the
process of investigating, assessing and resolving a vulnerability report
@ -14,13 +14,13 @@ opened by a MinIO employee or an external third party.
Therefore, it lists pre-conditions and actions that should be performed to
resolve and fix a reported vulnerability.
### Vulnerability Management Process
## Vulnerability Management Process
The vulnerability management process requires that the vulnerability report
contains the following information:
- The project / component that contains the reported vulnerability.
- A description of the vulnerability. In particular, the type of the
- The project / component that contains the reported vulnerability.
- A description of the vulnerability. In particular, the type of the
reported vulnerability and how it might be exploited. Alternatively,
a well-established vulnerability identifier, e.g. CVE number, can be
used instead.
@ -28,12 +28,11 @@ contains the following information:
Based on the description mentioned above, a MinIO engineer or security team
member investigates:
- Whether the reported vulnerability exists.
- The conditions that are required such that the vulnerability can be exploited.
- The steps required to fix the vulnerability.
- Whether the reported vulnerability exists.
- The conditions that are required such that the vulnerability can be exploited.
- The steps required to fix the vulnerability.
In general, if the vulnerability exists in one of the MinIO code bases
itself - not in a code dependency - then MinIO will, if possible, fix
the vulnerability or implement reasonable countermeasures such that the
vulnerability cannot be exploited anymore.

38
docs/bigdata/README.md
View File

@ -12,12 +12,12 @@ MinIO also supports multi-cluster, multi-site federation similar to AWS regions
## **2. Prerequisites**
* Install Hortonworks Distribution using this [guide.](https://docs.hortonworks.com/HDPDocuments/Ambari-2.7.1.0/bk_ambari-installation/content/ch_Installing_Ambari.html)
* [Setup Ambari](https://docs.hortonworks.com/HDPDocuments/Ambari-2.7.1.0/bk_ambari-installation/content/set_up_the_ambari_server.html) which automatically sets up YARN
* [Installing Spark](https://docs.hortonworks.com/HDPDocuments/HDP3/HDP-3.0.1/installing-spark/content/installing_spark.html)
* Install MinIO Distributed Server using one of the guides below.
* [Deployment based on Kubernetes](https://docs.min.io/docs/deploy-minio-on-kubernetes.html#minio-distributed-server-deployment)
* [Deployment based on MinIO Helm Chart](https://github.com/helm/charts/tree/master/stable/minio)
- Install Hortonworks Distribution using this [guide.](https://docs.hortonworks.com/HDPDocuments/Ambari-2.7.1.0/bk_ambari-installation/content/ch_Installing_Ambari.html)
- [Setup Ambari](https://docs.hortonworks.com/HDPDocuments/Ambari-2.7.1.0/bk_ambari-installation/content/set_up_the_ambari_server.html) which automatically sets up YARN
- [Installing Spark](https://docs.hortonworks.com/HDPDocuments/HDP3/HDP-3.0.1/installing-spark/content/installing_spark.html)
- Install MinIO Distributed Server using one of the guides below.
- [Deployment based on Kubernetes](https://docs.min.io/docs/deploy-minio-on-kubernetes.html#minio-distributed-server-deployment)
- [Deployment based on MinIO Helm Chart](https://github.com/helm/charts/tree/master/stable/minio)
## **3. Configure Hadoop, Spark, Hive to use MinIO**
@ -37,10 +37,10 @@ Navigate to **Custom core-site** to configure MinIO parameters for `_s3a_` conne
```
sudo pip install yq
alias kv-pairify='xq ".configuration[]" | jq ".[]" | jq -r ".name + \"=\" + .value"'
alias kv-pairify='yq ".configuration[]" | jq ".[]" | jq -r ".name + \"=\" + .value"'
```
Let's take for example a set of 12 compute nodes with an aggregate memory of *1.2TiB*, we need to do following settings for optimal results. Add the following optimal entries for _core-site.xml_ to configure _s3a_ with **MinIO**. Most important options here are
Let's take for example a set of 12 compute nodes with an aggregate memory of _1.2TiB_, we need to do following settings for optimal results. Add the following optimal entries for _core-site.xml_ to configure _s3a_ with **MinIO**. Most important options here are
```
cat ${HADOOP_CONF_DIR}/core-site.xml | kv-pairify | grep "mapred"
@ -56,7 +56,7 @@ mapreduce.task.io.sort.factor=999 # Threshold before writing to disk
mapreduce.task.sort.spill.percent=0.9 # Minimum % before spilling to disk
```
S3A is the connector to use S3 and other S3-compatible object stores such as MinIO. MapReduce workloads typically interact with object stores in the same way they do with HDFS. These workloads rely on HDFS atomic rename functionality to complete writing data to the datastore. Object storage operations are atomic by nature and they do not require/implement rename API. The default S3A committer emulates renames through copy and delete APIs. This interaction pattern causes significant loss of performance because of the write amplification. *Netflix*, for example, developed two new staging committers - the Directory staging committer and the Partitioned staging committer - to take full advantage of native object storage operations. These committers do not require rename operation. The two staging committers were evaluated, along with another new addition called the Magic committer for benchmarking.
S3A is the connector to use S3 and other S3-compatible object stores such as MinIO. MapReduce workloads typically interact with object stores in the same way they do with HDFS. These workloads rely on HDFS atomic rename functionality to complete writing data to the datastore. Object storage operations are atomic by nature and they do not require/implement rename API. The default S3A committer emulates renames through copy and delete APIs. This interaction pattern causes significant loss of performance because of the write amplification. _Netflix_, for example, developed two new staging committers - the Directory staging committer and the Partitioned staging committer - to take full advantage of native object storage operations. These committers do not require rename operation. The two staging committers were evaluated, along with another new addition called the Magic committer for benchmarking.
It was found that the directory staging committer was the fastest among the three, S3A connector should be configured with the following parameters for optimal results:
@ -95,8 +95,8 @@ fs.s3a.threads.max=2048 # Maximum number of threads for S3A
The rest of the other optimization options are discussed in the links below
* [https://hadoop.apache.org/docs/current/hadoop-aws/tools/hadoop-aws/index.html](https://hadoop.apache.org/docs/current/hadoop-aws/tools/hadoop-aws/index.html)
* [https://hadoop.apache.org/docs/r3.1.1/hadoop-aws/tools/hadoop-aws/committers.html](https://hadoop.apache.org/docs/r3.1.1/hadoop-aws/tools/hadoop-aws/committers.html)
- [https://hadoop.apache.org/docs/current/hadoop-aws/tools/hadoop-aws/index.html](https://hadoop.apache.org/docs/current/hadoop-aws/tools/hadoop-aws/index.html)
- [https://hadoop.apache.org/docs/r3.1.1/hadoop-aws/tools/hadoop-aws/committers.html](https://hadoop.apache.org/docs/r3.1.1/hadoop-aws/tools/hadoop-aws/committers.html)
Once the config changes are applied, proceed to restart **Hadoop** services.
@ -187,16 +187,16 @@ Test the Spark installation by running the following compute intensive example,
Follow these steps to run the Spark Pi example:
* Login as user **‘spark’**.
* When the job runs, the library can now use **MinIO** during intermediate processing.
* Navigate to a node with the Spark client and access the spark2-client directory:
- Login as user **‘spark’**.
- When the job runs, the library can now use **MinIO** during intermediate processing.
- Navigate to a node with the Spark client and access the spark2-client directory:
```
cd /usr/hdp/current/spark2-client
su spark
```
* Run the Apache Spark Pi job in yarn-client mode, using code from **org.apache.spark**:
- Run the Apache Spark Pi job in yarn-client mode, using code from **org.apache.spark**:
```
./bin/spark-submit --class org.apache.spark.examples.SparkPi \
@ -223,9 +223,9 @@ WordCount is a simple program that counts how often a word occurs in a text file
The following example submits WordCount code to the Scala shell. Select an input file for the Spark WordCount example. We can use any text file as input.
* Login as user **‘spark’**.
* When the job runs, the library can now use **MinIO** during intermediate processing.
* Navigate to a node with Spark client and access the spark2-client directory:
- Login as user **‘spark’**.
- When the job runs, the library can now use **MinIO** during intermediate processing.
- Navigate to a node with Spark client and access the spark2-client directory:
```
cd /usr/hdp/current/spark2-client
@ -269,7 +269,7 @@ Type :help for more information.
scala>
```
* At the _scala>_ prompt, submit the job by typing the following commands, Replace node names, file name, and file location with your values:
- At the _scala>_ prompt, submit the job by typing the following commands, Replace node names, file name, and file location with your values:
```
scala> val file = sc.textFile("s3a://testbucket/testdata")

11
docs/bucket/lifecycle/DESIGN.md
View File

@ -6,11 +6,13 @@ Transition tiers can be added to MinIO using `mc admin tier add` command to asso
Lifecycle transition rules can be applied to buckets (both versioned and un-versioned) by specifying the tier name defined above as the transition storage class for the lifecycle rule.
## Implementation
ILM tiering takes place when a object placed in the bucket meets lifecycle transition rules and becomes eligible for tiering. MinIO scanner (which runs at one minute intervals, each time scanning one sixteenth of the namespace), picks up the object for tiering. The data is moved to the remote tier in entirety, leaving only the object metadata on MinIO.
The data on the backend is stored under the `bucket/prefix` specified in the tier configuration with a custom name derived from a randomly generated uuid - e.g. `0b/c4/0bc4fab7-2daf-4d2f-8e39-5c6c6fb7e2d3`. The first two prefixes are characters 1-2,3-4 from the uuid. This format allows tiering to any cloud irrespective of whether the cloud in question supports versioning. The reference to the transitioned object name and transitioned tier is stored as part of the internal metadata for the object (or its version) on MinIO.
Extra metadata maintained internally in `xl.meta` for a transitioned object
```
...
"MetaSys": {
@ -21,6 +23,7 @@ Extra metadata maintained internally in `xl.meta` for a transitioned object
```
When a transitioned object is restored temporarily to local MinIO instance via PostRestoreObject API, the object data is copied back from the remote tier, and additional metadata for the restored object is maintained as referenced below. Once the restore period expires, the local copy of the object is removed by the scanner during its periodic runs.
```
...
"MetaUsr": {
@ -29,16 +32,24 @@ When a transitioned object is restored temporarily to local MinIO instance via P
"x-amz-restore": "ongoing-request=false, expiry-date=Sat, 27 Feb 2021 00:00:00 GMT",
...
```
### Encrypted/Object locked objects
For objects under SSE-S3 or SSE-C encryption, the encrypted content from MinIO cluster is copied as is to the remote tier without any decryption. The content is decrypted as it is streamed from remote tier on `GET/HEAD`. Objects under retention are protected because the metadata present on MinIO server ensures that the object (version) is not deleted until retention period is over. Administrators need to ensure that the remote tier bucket is under proper access control.
### Transition Status
MinIO specific extension header `X-Minio-Transition` is displayed on `HEAD/GET` to predict expected transition date on the object. Once object is transitioned to the remote tier,`x-amz-storage-class` shows the tier name to which object transitioned. Additional headers such as "X-Amz-Restore-Expiry-Days", "x-amz-restore", and "X-Amz-Restore-Request-Date" are displayed when a object is under restore/has been restored to local MinIO cluster.
### Expiry or removal events
An object that is in transition tier will be deleted once the object hits expiry date or if removed via `mc rm` (`mc rm --vid` in the case of delete of a specific object version). Other rules specific to legal hold and object locking precede any lifecycle rules.
### Additional notes
Tiering and lifecycle transition are applicable only to erasure/distributed MinIO.
## Explore Further
- [MinIO | Golang Client API Reference](https://docs.min.io/docs/golang-client-api-reference.html#SetBucketLifecycle)
- [Object Lifecycle Management](https://docs.aws.amazon.com/AmazonS3/latest/dev/object-lifecycle-mgmt.html)

12
docs/bucket/lifecycle/README.md
View File

@ -3,6 +3,7 @@
Enable object lifecycle configuration on buckets to setup automatic deletion of objects after a specified number of days or a specified date.
## 1. Prerequisites
- Install MinIO - [MinIO Quickstart Guide](https://docs.min.io/docs/minio-quickstart-guide).
- Install `mc` - [mc Quickstart Guide](https://docs.minio.io/docs/minio-client-quickstart-guide.html)
@ -45,6 +46,7 @@ Lifecycle configuration imported successfully to `play/testbucket`.
```
- List the current settings
```
$ mc ilm ls play/testbucket
ID | Prefix | Enabled | Expiry | Date/Days | Transition | Date/Days | Storage-Class | Tags
@ -64,6 +66,7 @@ This will only work with a versioned bucket, take a look at [Bucket Versioning G
A non-current object version is a version which is not the latest for a given object. It is possible to set up an automatic removal of non-current versions when a version becomes older than a given number of days.
e.g., To scan objects stored under `user-uploads/` prefix and remove versions older than one year.
```
{
"Rules": [
@ -81,12 +84,12 @@ e.g., To scan objects stored under `user-uploads/` prefix and remove versions ol
}
```
### 3.2 Automatic removal of noncurrent versions keeping only most recent ones after noncurrent days
It is possible to configure automatic removal of older noncurrent versions keeping only the most recent `N` using `NewerNoncurrentVersions`.
e.g, To remove noncurrent versions of all objects keeping the most recent 5 noncurrent versions under the prefix `user-uploads/` 30 days after they become noncurrent ,
```
{
"Rules": [
@ -126,6 +129,7 @@ of objects under the prefix `user-uploads/` as soon as there are more than `N` n
]
}
```
Note: This rule has an implicit zero NoncurrentDays, which makes the expiry of those 'extra' noncurrent versions immediate.
### 3.3 Automatic removal of delete markers with no other versions
@ -145,6 +149,7 @@ When an object has only one version as a delete marker, the latter can be automa
]
}
```
## 4. Enable ILM transition feature
In Erasure mode, MinIO supports tiering to public cloud providers such as GCS, AWS and Azure as well as to other MinIO clusters via the ILM transition feature. This will allow transitioning of older objects to a different cluster or the public cloud by setting up transition rules in the bucket lifecycle configuration. This feature enables applications to optimize storage costs by moving less frequently accessed data to a cheaper storage without compromising accessibility of data.
@ -156,14 +161,17 @@ To transition objects in a bucket to a destination bucket on a different cluster
```
mc admin tier add azure source AZURETIER --endpoint https://blob.core.windows.net --access-key AZURE_ACCOUNT_NAME --secret-key AZURE_ACCOUNT_KEY --bucket azurebucket --prefix testprefix1/
```
> The admin user running this command needs the "admin:SetTier" and "admin:ListTier" permissions if not running as root.
Using above tier, set up a lifecycle rule with transition:
```
mc ilm add --expiry-days 365 --transition-days 45 --storage-class "AZURETIER" myminio/srcbucket
```
Note: In the case of S3, it is possible to create a tier from MinIO running in EC2 to S3 using AWS role attached to EC2 as credentials instead of accesskey/secretkey:
```
mc admin tier add s3 source S3TIER --bucket s3bucket --prefix testprefix/ --use-aws-role
```
@ -177,10 +185,12 @@ aws s3api restore-object --bucket srcbucket \
```
### 4.1 Monitoring transition events
`s3:ObjectTransition:Complete` and `s3:ObjectTransition:Failed` events can be used to monitor transition events between the source cluster and transition tier. To watch lifecycle events, you can enable bucket notification on the source bucket with `mc event add` and specify `--event ilm` flag.
Note that transition event notification is a MinIO extension.
## Explore Further
- [MinIO | Golang Client API Reference](https://docs.min.io/docs/golang-client-api-reference.html#SetBucketLifecycle)
- [Object Lifecycle Management](https://docs.aws.amazon.com/AmazonS3/latest/dev/object-lifecycle-mgmt.html)

180
docs/bucket/notifications/README.md
View File

@ -32,7 +32,6 @@ Various event types supported by MinIO server are
| `s3:BucketCreated` |
| `s3:BucketRemoved` |
Use client tools like `mc` to set and listen for event notifications using the [`event` sub-command](https://docs.min.io/docs/minio-client-complete-guide#events). MinIO SDK's [`BucketNotification` APIs](https://docs.min.io/docs/golang-client-api-reference#SetBucketNotification) can also be used. The notification message MinIO sends to publish an event is a JSON message with the following [structure](https://docs.aws.amazon.com/AmazonS3/latest/dev/notification-content-structure.html).
Bucket events can be published to the following targets:
@ -64,12 +63,11 @@ notify_redis publish bucket notifications to Redis datastores
```
> NOTE:
>
> - '\*' at the end of arg means its mandatory.
> - '\*' at the end of the values, means its the default value for the arg.
> - When configured using environment variables, the `:name` can be specified using this format `MINIO_NOTIFY_WEBHOOK_ENABLE_<name>`.
<a name="AMQP"></a>
## Publish MinIO events via AMQP
Install RabbitMQ from [here](https://www.rabbitmq.com/).
@ -135,7 +133,7 @@ Use `mc admin config set` command to update the configuration for the deployment
An example configuration for RabbitMQ is shown below:
```sh
$ mc admin config set myminio/ notify_amqp:1 exchange="bucketevents" exchange_type="fanout" mandatory="false" no_wait="false" url="amqp://myuser:mypassword@localhost:5672" auto_deleted="false" delivery_mode="0" durable="false" internal="false" routing_key="bucketlogs"
mc admin config set myminio/ notify_amqp:1 exchange="bucketevents" exchange_type="fanout" mandatory="false" no_wait="false" url="amqp://myuser:mypassword@localhost:5672" auto_deleted="false" delivery_mode="0" durable="false" internal="false" routing_key="bucketlogs"
```
MinIO supports all the exchanges available in [RabbitMQ](https://www.rabbitmq.com/). For this setup, we are using `fanout` exchange.
@ -144,7 +142,7 @@ MinIO also sends with the notifications two headers: `minio-bucket` and `minio-e
Note that, you can add as many AMQP server endpoint configurations as needed by providing an identifier (like "1" in the example above) for the AMQP instance and an object of per-server configuration parameters.
### Step 2: Enable bucket notification using MinIO client
### Step 2: Enable RabbitMQ bucket notification using MinIO client
We will enable bucket event notification to trigger whenever a JPEG image is uploaded or deleted `images` bucket on `myminio` server. Here ARN value is `arn:minio:sqs::1:amqp`. To understand more about ARN please follow [AWS ARN](http://docs.aws.amazon.com/general/latest/gr/aws-arns-and-namespaces.html) documentation.
@ -207,8 +205,6 @@ python rabbit.py
'{"Records":[{"eventVersion":"2.0","eventSource":"aws:s3","awsRegion":"","eventTime":"2016–09–08T22:34:38.226Z","eventName":"s3:ObjectCreated:Put","userIdentity":{"principalId":"minio"},"requestParameters":{"sourceIPAddress":"10.1.10.150:44576"},"responseElements":{},"s3":{"s3SchemaVersion":"1.0","configurationId":"Config","bucket":{"name":"images","ownerIdentity":{"principalId":"minio"},"arn":"arn:aws:s3:::images"},"object":{"key":"myphoto.jpg","size":200436,"sequencer":"147279EAF9F40933"}}}],"level":"info","msg":"","time":"2016–09–08T15:34:38–07:00"}'
```
<a name="MQTT"></a>
## Publish MinIO events MQTT
Install an MQTT Broker from [here](https://mosquitto.org/).
@ -266,14 +262,14 @@ notify_mqtt:1 broker="" password="" queue_dir="" queue_limit="0" reconnect_inter
Use `mc admin config set` command to update the configuration for the deployment. Restart the MinIO server to put the changes into effect. The server will print a line like `SQS ARNs: arn:minio:sqs::1:mqtt` at start-up if there were no errors.
```sh
$ mc admin config set myminio notify_mqtt:1 broker="tcp://localhost:1883" password="" queue_dir="" queue_limit="0" reconnect_interval="0s" keep_alive_interval="0s" qos="1" topic="minio" username=""
mc admin config set myminio notify_mqtt:1 broker="tcp://localhost:1883" password="" queue_dir="" queue_limit="0" reconnect_interval="0s" keep_alive_interval="0s" qos="1" topic="minio" username=""
```
MinIO supports any MQTT server that supports MQTT 3.1 or 3.1.1 and can connect to them over TCP, TLS, or a Websocket connection using `tcp://`, `tls://`, or `ws://` respectively as the scheme for the broker url. See the [Go Client](http://www.eclipse.org/paho/clients/golang/) documentation for more information.
Note that, you can add as many MQTT server endpoint configurations as needed by providing an identifier (like "1" in the example above) for the MQTT instance and an object of per-server configuration parameters.
### Step 2: Enable bucket notification using MinIO client
### Step 2: Enable MQTT bucket notification using MinIO client
We will enable bucket event notification to trigger whenever a JPEG image is uploaded or deleted `images` bucket on `myminio` server. Here ARN value is `arn:minio:sqs::1:mqtt`.
@ -332,8 +328,6 @@ python mqtt.py
{“Records”:[{“eventVersion”:”2.0",”eventSource”:”aws:s3",”awsRegion”:”",”eventTime”:”2016–09–08T22:34:38.226Z”,”eventName”:”s3:ObjectCreated:Put”,”userIdentity”:{“principalId”:”minio”},”requestParameters”:{“sourceIPAddress”:”10.1.10.150:44576"},”responseElements”:{},”s3":{“s3SchemaVersion”:”1.0",”configurationId”:”Config”,”bucket”:{“name”:”images”,”ownerIdentity”:{“principalId”:”minio”},”arn”:”arn:aws:s3:::images”},”object”:{“key”:”myphoto.jpg”,”size”:200436,”sequencer”:”147279EAF9F40933"}}}],”level”:”info”,”msg”:””,”time”:”2016–09–08T15:34:38–07:00"}
```
<a name="Elasticsearch"></a>
## Publish MinIO events via Elasticsearch
Install [Elasticsearch](https://www.elastic.co/downloads/elasticsearch) server.
@ -346,7 +340,7 @@ When the _access_ format is used, MinIO appends events as documents in an Elasti
The steps below show how to use this notification target in `namespace` format. The other format is very similar and is omitted for brevity.
### Step 1: Ensure minimum requirements are met
### Step 1: Ensure Elasticsearch minimum requirements are met
MinIO requires a 5.x series version of Elasticsearch. This is the latest major release series. Elasticsearch provides version upgrade migration guidelines [here](https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-upgrade.html).
@ -403,12 +397,12 @@ notify_elasticsearch:1 queue_limit="0" url="" format="namespace" index="" queue
Use `mc admin config set` command to update the configuration for the deployment. Restart the MinIO server to put the changes into effect. The server will print a line like `SQS ARNs: arn:minio:sqs::1:elasticsearch` at start-up if there were no errors.
```sh
$ mc admin config set myminio notify_elasticsearch:1 queue_limit="0" url="http://127.0.0.1:9200" format="namespace" index="minio_events" queue_dir="" username="" password=""
mc admin config set myminio notify_elasticsearch:1 queue_limit="0" url="http://127.0.0.1:9200" format="namespace" index="minio_events" queue_dir="" username="" password=""
```
Note that, you can add as many Elasticsearch server endpoint configurations as needed by providing an identifier (like "1" in the example above) for the Elasticsearch instance and an object of per-server configuration parameters.
### Step 3: Enable bucket notification using MinIO client
### Step 3: Enable Elastichsearch bucket notification using MinIO client
We will now enable bucket event notifications on a bucket named `images`. Whenever a JPEG image is created/overwritten, a new document is added or an existing document is updated in the Elasticsearch index configured above. When an existing object is deleted, the corresponding document is deleted from the index. Thus, the rows in the Elasticsearch index, reflect the `.jpg` objects in the `images` bucket.
@ -505,8 +499,6 @@ This output shows that a document has been created for the event in Elasticsearc
Here we see that the document ID is the bucket and object name. In case `access` format was used, the document ID would be automatically generated by Elasticsearch.
<a name="Redis"></a>
## Publish MinIO events via Redis
Install [Redis](http://redis.io/download) server. For illustrative purposes, we have set the database password as "yoursecret".
@ -565,12 +557,12 @@ notify_redis:1 address="" format="namespace" key="" password="" queue_dir="" que
Use `mc admin config set` command to update the configuration for the deployment.Restart the MinIO server to put the changes into effect. The server will print a line like `SQS ARNs: arn:minio:sqs::1:redis` at start-up if there were no errors.
```sh
$ mc admin config set myminio/ notify_redis:1 address="127.0.0.1:6379" format="namespace" key="bucketevents" password="yoursecret" queue_dir="" queue_limit="0"
mc admin config set myminio/ notify_redis:1 address="127.0.0.1:6379" format="namespace" key="bucketevents" password="yoursecret" queue_dir="" queue_limit="0"
```
Note that, you can add as many Redis server endpoint configurations as needed by providing an identifier (like "1" in the example above) for the Redis instance and an object of per-server configuration parameters.
### Step 2: Enable bucket notification using MinIO client
### Step 2: Enable Redis bucket notification using MinIO client
We will now enable bucket event notifications on a bucket named `images`. Whenever a JPEG image is created/overwritten, a new key is added or an existing key is updated in the Redis hash configured above. When an existing object is deleted, the corresponding key is deleted from the Redis hash. Thus, the rows in the Redis hash, reflect the `.jpg` objects in the `images` bucket.
@ -614,8 +606,6 @@ Here we see that MinIO performed `HSET` on `minio_events` key.
In case, `access` format was used, then `minio_events` would be a list, and the MinIO server would have performed an `RPUSH` to append to the list. A consumer of this list would ideally use `BLPOP` to remove list items from the left-end of the list.
<a name="NATS"></a>
## Publish MinIO events via NATS
Install NATS from [here](http://nats.io/).
@ -650,6 +640,7 @@ comment (sentence) optionally add a comment to this s
```
or environment variables
```
KEY:
notify_nats[:name] publish bucket notifications to NATS endpoints
@ -686,14 +677,14 @@ notify_nats:1 password="yoursecret" streaming_max_pub_acks_in_flight="10" subjec
Use `mc admin config set` command to update the configuration for the deployment.Restart MinIO server to reflect config changes. `bucketevents` is the subject used by NATS in this example.
```sh
$ mc admin config set myminio notify_nats:1 password="yoursecret" streaming_max_pub_acks_in_flight="10" subject="" address="0.0.0.0:4222" token="" username="yourusername" ping_interval="0" queue_limit="0" tls="off" streaming_async="on" queue_dir="" streaming_cluster_id="test-cluster" streaming_enable="on"
mc admin config set myminio notify_nats:1 password="yoursecret" streaming_max_pub_acks_in_flight="10" subject="" address="0.0.0.0:4222" token="" username="yourusername" ping_interval="0" queue_limit="0" tls="off" streaming_async="on" queue_dir="" streaming_cluster_id="test-cluster" streaming_enable="on"
```
MinIO server also supports [NATS Streaming mode](http://nats.io/documentation/streaming/nats-streaming-intro/) that offers additional functionality like `At-least-once-delivery`, and `Publisher rate limiting`. To configure MinIO server to send notifications to NATS Streaming server, update the MinIO server configuration file as follows:
Read more about sections `cluster_id`, `client_id` on [NATS documentation](https://github.com/nats-io/nats-streaming-server/blob/master/README.md). Section `maxPubAcksInflight` is explained [here](https://github.com/nats-io/stan.go#publisher-rate-limiting).
### Step 2: Enable bucket notification using MinIO client
### Step 2: Enable NATS bucket notification using MinIO client
We will enable bucket event notification to trigger whenever a JPEG image is uploaded or deleted from `images` bucket on `myminio` server. Here ARN value is `arn:minio:sqs::1:nats`. To understand more about ARN please follow [AWS ARN](http://docs.aws.amazon.com/general/latest/gr/aws-arns-and-namespaces.html) documentation.
@ -713,28 +704,28 @@ package main
// Import Go and NATS packages
import (
"log"
"runtime"
"log"
"runtime"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go"
)
func main() {
// Create server connection
natsConnection, _ := nats.Connect("nats://yourusername:yoursecret@localhost:4222")
log.Println("Connected")
// Create server connection
natsConnection, _ := nats.Connect("nats://yourusername:yoursecret@localhost:4222")
log.Println("Connected")
// Subscribe to subject
log.Printf("Subscribing to subject 'bucketevents'\n")
natsConnection.Subscribe("bucketevents", func(msg *nats.Msg) {
// Subscribe to subject
log.Printf("Subscribing to subject 'bucketevents'\n")
natsConnection.Subscribe("bucketevents", func(msg *nats.Msg) {
// Handle the message
log.Printf("Received message '%s\n", string(msg.Data)+"'")
})
// Handle the message
log.Printf("Received message '%s\n", string(msg.Data)+"'")
})
// Keep the connection alive
runtime.Goexit()
// Keep the connection alive
runtime.Goexit()
}
```
@ -766,52 +757,52 @@ package main
// Import Go and NATS packages
import (
"fmt"
"runtime"
"fmt"
"runtime"
"github.com/nats-io/stan.go"
"github.com/nats-io/stan.go"
)
func main() {
var stanConnection stan.Conn
var stanConnection stan.Conn
subscribe := func() {
fmt.Printf("Subscribing to subject 'bucketevents'\n")
stanConnection.Subscribe("bucketevents", func(m *stan.Msg) {
subscribe := func() {
fmt.Printf("Subscribing to subject 'bucketevents'\n")
stanConnection.Subscribe("bucketevents", func(m *stan.Msg) {
// Handle the message
fmt.Printf("Received a message: %s\n", string(m.Data))
})
}
// Handle the message
fmt.Printf("Received a message: %s\n", string(m.Data))
})
}
stanConnection, _ = stan.Connect("test-cluster", "test-client", stan.NatsURL("nats://yourusername:yoursecret@0.0.0.0:4222"), stan.SetConnectionLostHandler(func(c stan.Conn, _ error) {
go func() {
for {
// Reconnect if the connection is lost.
if stanConnection == nil || stanConnection.NatsConn() == nil || !stanConnection.NatsConn().IsConnected() {
stanConnection, _ = stan.Connect("test-cluster", "test-client", stan.NatsURL("nats://yourusername:yoursecret@0.0.0.0:4222"), stan.SetConnectionLostHandler(func(c stan.Conn, _ error) {
if c.NatsConn() != nil {
c.NatsConn().Close()
}
_ = c.Close()
}))
if stanConnection != nil {
subscribe()
}
stanConnection, _ = stan.Connect("test-cluster", "test-client", stan.NatsURL("nats://yourusername:yoursecret@0.0.0.0:4222"), stan.SetConnectionLostHandler(func(c stan.Conn, _ error) {
go func() {
for {
// Reconnect if the connection is lost.
if stanConnection == nil || stanConnection.NatsConn() == nil || !stanConnection.NatsConn().IsConnected() {
stanConnection, _ = stan.Connect("test-cluster", "test-client", stan.NatsURL("nats://yourusername:yoursecret@0.0.0.0:4222"), stan.SetConnectionLostHandler(func(c stan.Conn, _ error) {
if c.NatsConn() != nil {
c.NatsConn().Close()
}
_ = c.Close()
}))
if stanConnection != nil {
subscribe()
}
}
}
}
}
}()
}))
}()
}))
// Subscribe to subject
subscribe()
// Subscribe to subject
subscribe()
// Keep the connection alive
runtime.Goexit()
// Keep the connection alive
runtime.Goexit()
}
```
@ -834,8 +825,6 @@ The example `nats.go` program prints event notification to console.
Received a message: {"EventType":"s3:ObjectCreated:Put","Key":"images/myphoto.jpg","Records":[{"eventVersion":"2.0","eventSource":"minio:s3","awsRegion":"","eventTime":"2017-07-07T18:46:37Z","eventName":"s3:ObjectCreated:Put","userIdentity":{"principalId":"minio"},"requestParameters":{"sourceIPAddress":"192.168.1.80:55328"},"responseElements":{"x-amz-request-id":"14CF20BD1EFD5B93","x-minio-origin-endpoint":"http://127.0.0.1:9000"},"s3":{"s3SchemaVersion":"1.0","configurationId":"Config","bucket":{"name":"images","ownerIdentity":{"principalId":"minio"},"arn":"arn:aws:s3:::images"},"object":{"key":"myphoto.jpg","size":248682,"eTag":"f1671feacb8bbf7b0397c6e9364e8c92","contentType":"image/jpeg","userDefined":{"content-type":"image/jpeg"},"versionId":"1","sequencer":"14CF20BD1EFD5B93"}},"source":{"host":"192.168.1.80","port":"55328","userAgent":"MinIO (linux; amd64) minio-go/2.0.4 mc/DEVELOPMENT.GOGET"}}],"level":"info","msg":"","time":"2017-07-07T11:46:37-07:00"}
```
<a name="PostgreSQL"></a>
## Publish MinIO events via PostgreSQL
> NOTE: Until release RELEASE.2020-04-10T03-34-42Z PostgreSQL notification used to support following options:
@ -848,8 +837,8 @@ Received a message: {"EventType":"s3:ObjectCreated:Put","Key":"images/myphoto.jp
> database (string) database name (used only if `connection_string` is empty)
> ```
>
> These are now deprecated, if you plan to upgrade to any releases after *RELEASE.2020-04-10T03-34-42Z* make sure
> to migrate to only using *connection_string* option. To migrate, once you have upgraded all the servers use the
> These are now deprecated, if you plan to upgrade to any releases after _RELEASE.2020-04-10T03-34-42Z_ make sure
> to migrate to only using _connection_string_ option. To migrate, once you have upgraded all the servers use the
> following command to update the existing notification targets.
>
> ```
@ -858,7 +847,7 @@ Received a message: {"EventType":"s3:ObjectCreated:Put","Key":"images/myphoto.jp
>
> Please make sure this step is carried out, without this step PostgreSQL notification targets will not work,
> an error message will be shown on the console upon server upgrade/restart, make sure to follow the above
> instructions appropriately. For further questions please join our https://slack.min.io
> instructions appropriately. For further questions please join our <https://slack.min.io>
Install [PostgreSQL](https://www.postgresql.org/) database server. For illustrative purposes, we have set the "postgres" user password as `password` and created a database called `minio_events` to store the events.
@ -870,7 +859,7 @@ When the _access_ format is used, MinIO appends events to a table. It creates ro
The steps below show how to use this notification target in `namespace` format. The other format is very similar and is omitted for brevity.
### Step 1: Ensure minimum requirements are met
### Step 1: Ensure PostgresSQL minimum requirements are met
MinIO requires PostgreSQL version 9.5 or above. MinIO uses the [`INSERT ON CONFLICT`](https://www.postgresql.org/docs/9.5/static/sql-insert.html#SQL-ON-CONFLICT) (aka UPSERT) feature, introduced in version 9.5 and the [JSONB](https://www.postgresql.org/docs/9.4/static/datatype-json.html) data-type introduced in version 9.4.
@ -893,6 +882,7 @@ comment (sentence) optionally add a comment to this setti
```
or environment variables
```
KEY:
notify_postgres[:name] publish bucket notifications to Postgres databases
@ -924,12 +914,12 @@ notify_postgres:1 queue_dir="" connection_string="" queue_limit="0" table="" fo
Use `mc admin config set` command to update the configuration for the deployment. Restart the MinIO server to put the changes into effect. The server will print a line like `SQS ARNs: arn:minio:sqs::1:postgresql` at start-up if there were no errors.
```sh
$ mc admin config set myminio notify_postgres:1 connection_string="host=localhost port=5432 dbname=minio_events user=postgres password=password sslmode=disable" table="bucketevents" format="namespace"
mc admin config set myminio notify_postgres:1 connection_string="host=localhost port=5432 dbname=minio_events user=postgres password=password sslmode=disable" table="bucketevents" format="namespace"
```
Note that, you can add as many PostgreSQL server endpoint configurations as needed by providing an identifier (like "1" in the example above) for the PostgreSQL instance and an object of per-server configuration parameters.
### Step 3: Enable bucket notification using MinIO client
### Step 3: Enable PostgreSQL bucket notification using MinIO client
We will now enable bucket event notifications on a bucket named `images`. Whenever a JPEG image is created/overwritten, a new row is added or an existing row is updated in the PostgreSQL configured above. When an existing object is deleted, the corresponding row is deleted from the PostgreSQL table. Thus, the rows in the PostgreSQL table, reflect the `.jpg` objects in the `images` bucket.
@ -968,8 +958,6 @@ key | value
(1 row)
```
<a name="MySQL"></a>
## Publish MinIO events via MySQL
> NOTE: Until release RELEASE.2020-04-10T03-34-42Z MySQL notification used to support following options:
@ -982,8 +970,8 @@ key | value
> database (string) database name (used only if `dsn_string` is empty)
> ```
>
> These are now deprecated, if you plan to upgrade to any releases after *RELEASE.2020-04-10T03-34-42Z* make sure
> to migrate to only using *dsn_string* option. To migrate, once you have upgraded all the servers use the
> These are now deprecated, if you plan to upgrade to any releases after _RELEASE.2020-04-10T03-34-42Z_ make sure
> to migrate to only using _dsn_string_ option. To migrate, once you have upgraded all the servers use the
> following command to update the existing notification targets.
>
> ```
@ -992,7 +980,7 @@ key | value
>
> Please make sure this step is carried out, without this step MySQL notification targets will not work,
> an error message will be shown on the console upon server upgrade/restart, make sure to follow the above
> instructions appropriately. For further questions please join our https://slack.min.io
> instructions appropriately. For further questions please join our <https://slack.min.io>
Install MySQL from [here](https://dev.mysql.com/downloads/mysql/). For illustrative purposes, we have set the root password as `password` and created a database called `miniodb` to store the events.
@ -1004,7 +992,7 @@ When the _access_ format is used, MinIO appends events to a table. It creates ro
The steps below show how to use this notification target in `namespace` format. The other format is very similar and is omitted for brevity.
### Step 1: Ensure minimum requirements are met
### Step 1: Ensure MySQL minimum requirements are met
MinIO requires MySQL version 5.7.8 or above. MinIO uses the [JSON](https://dev.mysql.com/doc/refman/5.7/en/json.html) data-type introduced in version 5.7.8. We tested this setup on MySQL 5.7.17.
@ -1027,6 +1015,7 @@ comment (sentence) optionally add a comment to this setti
```
or environment variables
```
KEY:
notify_mysql[:name] publish bucket notifications to MySQL databases
@ -1059,14 +1048,14 @@ notify_mysql:myinstance enable=off format=namespace host= port= username= passwo
Use `mc admin config set` command to update MySQL notification configuration for the deployment with `dsn_string` parameter:
```sh
$ mc admin config set myminio notify_mysql:myinstance table="minio_images" dsn_string="root:xxxx@tcp(172.17.0.1:3306)/miniodb"
mc admin config set myminio notify_mysql:myinstance table="minio_images" dsn_string="root:xxxx@tcp(172.17.0.1:3306)/miniodb"
```
Note that, you can add as many MySQL server endpoint configurations as needed by providing an identifier (like "myinstance" in the example above) for each MySQL instance desired.
Restart the MinIO server to put the changes into effect. The server will print a line like `SQS ARNs: arn:minio:sqs::myinstance:mysql` at start-up, if there are no errors.
### Step 3: Enable bucket notification using MinIO client
### Step 3: Enable MySQL bucket notification using MinIO client
We will now setup bucket notifications on a bucket named `images`. Whenever a JPEG image object is created/overwritten, a new row is added or an existing row is updated in the MySQL table configured above. When an existing object is deleted, the corresponding row is deleted from the MySQL table. Thus, the rows in the MySQL table, reflect the `.jpg` objects in the `images` bucket.
@ -1106,8 +1095,6 @@ mysql> select * from minio_images;
```
<a name="apache-kafka"></a>
## Publish MinIO events via Kafka
Install Apache Kafka from [here](http://kafka.apache.org/).
@ -1143,6 +1130,7 @@ comment (sentence) optionally add a comment to this setting
```
or environment variables
```
KEY:
notify_kafka[:name] publish bucket notifications to Kafka endpoints
@ -1176,10 +1164,10 @@ notify_kafka:1 tls_skip_verify="off" queue_dir="" queue_limit="0" sasl="off" sa
Use `mc admin config set` command to update the configuration for the deployment. Restart the MinIO server to put the changes into effect. The server will print a line like `SQS ARNs: arn:minio:sqs::1:kafka` at start-up if there were no errors.`bucketevents` is the topic used by kafka in this example.
```sh
$ mc admin config set myminio notify_kafka:1 tls_skip_verify="off" queue_dir="" queue_limit="0" sasl="off" sasl_password="" sasl_username="" tls_client_auth="0" tls="off" client_tls_cert="" client_tls_key="" brokers="localhost:9092,localhost:9093" topic="bucketevents" version=""
mc admin config set myminio notify_kafka:1 tls_skip_verify="off" queue_dir="" queue_limit="0" sasl="off" sasl_password="" sasl_username="" tls_client_auth="0" tls="off" client_tls_cert="" client_tls_key="" brokers="localhost:9092,localhost:9093" topic="bucketevents" version=""
```
### Step 3: Enable bucket notification using MinIO client
### Step 3: Enable Kafka bucket notification using MinIO client
We will enable bucket event notification to trigger whenever a JPEG image is uploaded or deleted from `images` bucket on `myminio` server. Here ARN value is `arn:minio:sqs::1:kafka`. To understand more about ARN please follow [AWS ARN](http://docs.aws.amazon.com/general/latest/gr/aws-arns-and-namespaces.html) documentation.
@ -1263,8 +1251,6 @@ kafkacat -b localhost:9092 -t bucketevents
}
```
<a name="webhooks"></a>
## Publish MinIO events via Webhooks
[Webhooks](https://en.wikipedia.org/wiki/Webhook) are a way to receive information when it happens, rather than continually polling for that data.
@ -1288,6 +1274,7 @@ comment (sentence) optionally add a comment to this setting
```
or environment variables
```
KEY:
notify_webhook[:name] publish bucket notifications to webhook endpoints
@ -1311,10 +1298,10 @@ notify_webhook:1 endpoint="" auth_token="" queue_limit="0" queue_dir="" client_c
Use `mc admin config set` command to update the configuration for the deployment. Here the endpoint is the server listening for webhook notifications. Save the settings and restart the MinIO server for changes to take effect. Note that the endpoint needs to be live and reachable when you restart your MinIO server.
```sh
$ mc admin config set myminio notify_webhook:1 queue_limit="0" endpoint="http://localhost:3000" queue_dir=""
mc admin config set myminio notify_webhook:1 queue_limit="0" endpoint="http://localhost:3000" queue_dir=""
```
### Step 2: Enable bucket notification using MinIO client
### Step 2: Enable Webhook bucket notification using MinIO client
We will enable bucket event notification to trigger whenever a JPEG image is uploaded to `images` bucket on `myminio` server. Here ARN value is `arn:minio:sqs::1:webhook`. To learn more about ARN please follow [AWS ARN](http://docs.aws.amazon.com/general/latest/gr/aws-arns-and-namespaces.html) documentation.
@ -1365,8 +1352,6 @@ mc ls myminio/images-thumbnail
[2017-02-08 11:39:40 IST] 992B images-thumbnail.jpg
```
<a name="NSQ"></a>
## Publish MinIO events to NSQ
Install an NSQ Daemon from [here](https://nsq.io/). Or use the following Docker
@ -1397,6 +1382,7 @@ comment (sentence) optionally add a comment to this setting
```
or environment variables
```
KEY:
notify_nsq[:name] publish bucket notifications to NSQ endpoints
@ -1420,12 +1406,12 @@ notify_nsq:1 nsqd_address="" queue_dir="" queue_limit="0" tls="off" tls_skip_ve
Use `mc admin config set` command to update the configuration for the deployment. Restart the MinIO server to put the changes into effect. The server will print a line like `SQS ARNs: arn:minio:sqs::1:nsq` at start-up if there were no errors.
```sh
$ mc admin config set myminio notify_nsq:1 nsqd_address="127.0.0.1:4150" queue_dir="" queue_limit="0" tls="off" tls_skip_verify="on" topic="minio"
mc admin config set myminio notify_nsq:1 nsqd_address="127.0.0.1:4150" queue_dir="" queue_limit="0" tls="off" tls_skip_verify="on" topic="minio"
```
Note that, you can add as many NSQ daemon endpoint configurations as needed by providing an identifier (like "1" in the example above) for the NSQ instance and an object of per-server configuration parameters.
### Step 2: Enable bucket notification using MinIO client
### Step 2: Enable NSQ bucket notification using MinIO client
We will enable bucket event notification to trigger whenever a JPEG image is uploaded or deleted `images` bucket on `myminio` server. Here ARN value is `arn:minio:sqs::1:nsq`.

9
docs/bucket/quota/README.md
View File

@ -7,25 +7,26 @@ Buckets can be configured to have `Hard` quota - it disallows writes to the buck
> NOTE: Bucket quotas are not supported under gateway or standalone single disk deployments.
## Prerequisites
- Install MinIO - [MinIO Quickstart Guide](https://docs.min.io/docs/minio-quickstart-guide).
- [Use `mc` with MinIO Server](https://docs.min.io/docs/minio-client-quickstart-guide)
## Set bucket quota configuration
### Set a hard quota of 1GB for a bucket `mybucket` on MinIO object storage:
### Set a hard quota of 1GB for a bucket `mybucket` on MinIO object storage
```sh
$ mc admin bucket quota myminio/mybucket --hard 1gb
mc admin bucket quota myminio/mybucket --hard 1gb
```
### Verify the quota configured on `mybucket` on MinIO
```sh
$ mc admin bucket quota myminio/mybucket
mc admin bucket quota myminio/mybucket
```
### Clear bucket quota configuration for `mybucket` on MinIO
```sh
$ mc admin bucket quota myminio/mybucket --clear
mc admin bucket quota myminio/mybucket --clear
```

6
docs/bucket/replication/DESIGN.md
View File

@ -3,9 +3,11 @@
This document explains the design approach of server side bucket replication. If you're looking to get started with replication, we suggest you go through the [Bucket replication guide](https://github.com/minio/minio/blob/master/docs/bucket/replication/README.md) first.
## Overview
Replication relies on immutability provided by versioning to sync objects between the configured source and replication target. Replication results in the object data, metadata, last modification time and version ID all being identical between the source and target. Thus version ordering is automatically guaranteed on the source and target clusters.
### Replication of object version and metadata
If an object meets replication rules as set in the replication configuration, `X-Amz-Replication-Status` is first set to `PENDING` as the PUT operation completes and replication is queued (unless synchronous replication is in place). After replication is performed, the metadata on the source object version changes to `COMPLETED` or `FAILED` depending on whether replication succeeded. The object version on the target shows `X-Amz-Replication-Status` of `REPLICA`
All replication failures are picked up by the scanner which runs at a one minute frequency, each time scanning upto a sixteenth of the namespace. Object versions marked `PENDING` or `FAILED` are re-queued for replication.
@ -36,6 +38,7 @@ An additional header `X-Minio-Replication-Delete-Status` is returned which would
Note that synchronous replication, i.e. when remote target is configured with --sync mode in `mc admin bucket remote add` does not apply to `DELETE` operations. The version being deleted on the source cluster needs to maintain state and ensure that the operation is mirrored to the target cluster prior to completing on the source object version. Since this needs to account for the target cluster availability and the need to serialize concurrent DELETE operations on different versions of the same object during multi DELETE operations, the current implementation queues the `DELETE` operations in both sync and async modes.
### Existing object replication
Existing object replication works similar to regular replication. Objects qualifying for existing object replication are detected when scanner runs, and will be replicated if existing object replication is enabled and applicable replication rules are satisfied. Because replication depends on the immutability of versions, only pre-existing objects created while versioning was enabled can be replicated. Even if replication rules are disabled and re-enabled later, the objects created during the interim will be synced as the scanner queues them. For saving iops, objects qualifying for
existing object replication are not marked as `PENDING` prior to replication.
@ -46,6 +49,7 @@ If the remote site is fully lost and objects previously replicated need to be re
This is a slower operation that does not use replication queues and is designed to walk the namespace and replicate objects one at a time so as not to impede server load. Ideally, resync should not be initiated for multiple buckets simultaneously - progress of the syncing can be monitored by looking at `mc replicate resync status alias/bucket --remote-bucket <arn>`. In the event that resync operation failed to replicate some versions, they would be picked up by the healing mechanism in-built as part of the scanner. If the resync operation reports a failed status or in the event of a cluster restart while resync is in progress, a fresh `resync start` can be issued - this will replicate previously unsynced content at the cost of additional overhead in additional metadata updates.
### Multi destination replication
The replication design for multiple sites works in a similar manner as described above for two site scenario. However there are some
important exceptions.
@ -94,6 +98,7 @@ If 3 or more targets are participating in active-active replication, the replica
```
### Additional replication metadata for DeleteMarker
```
...
{
@ -150,5 +155,6 @@ If 3 or more targets are participating in active-active replication, the replica
```
## Explore Further
- [MinIO Bucket Versioning Implementation](https://docs.minio.io/docs/minio-bucket-versioning-guide.html)
- [MinIO Client Quickstart Guide](https://docs.minio.io/docs/minio-client-quickstart-guide.html)

29
docs/bucket/replication/README.md
View File

@ -7,6 +7,7 @@ The contents of this page have been migrated to the new [MinIO Baremetal Documen
To replicate objects in a bucket to a destination bucket on a target site either in the same cluster or a different cluster, start by enabling [versioning](https://docs.minio.io/docs/minio-bucket-versioning-guide.html) for both source and destination buckets. Finally, the target site and the destination bucket need to be configured on the source MinIO server.
## Highlights
- Supports source and destination buckets to have the same name unlike AWS S3, addresses variety of usecases such as *Splunk*, *Veeam* site to site DR.
- Supports object locking/retention across source and destination buckets natively out of the box, unlike AWS S3.
- Simpler implementation than [AWS S3 Bucket Replication Config](https://docs.aws.amazon.com/AmazonS3/latest/dev/replication-add-config.html) with requirements such as IAM Role, AccessControlTranslation, Metrics and SourceSelectionCriteria are not needed with MinIO.
@ -14,6 +15,7 @@ To replicate objects in a bucket to a destination bucket on a target site either
- Multi destination replication
## How to use?
Ensure that versioning is enabled on the source and target buckets with `mc version` command. If object locking is required, the buckets should have been created with `mc mb --with-lock`
Create a replication target on the source cluster as shown below:
@ -23,9 +25,10 @@ mc admin bucket remote add myminio/srcbucket https://accessKey:secretKey@replica
Remote ARN = 'arn:minio:replication:us-east-1:c5be6b16-769d-432a-9ef1-4567081f3566:destbucket'
```
> The user running the above command needs *s3:GetReplicationConfiguration* and *s3:GetBucketVersioning* permission on the source cluster. We do not recommend running root credentials/super admin with replication, instead create a dedicated user. The access credentials used at the destination requires *s3:ReplicateObject* permission.
> The user running the above command needs *s3:GetReplicationConfiguration* and *s3:GetBucketVersioning* permission on the source cluster. We do not recommend running root credentials/super admin with replication, instead create a dedicated user. The access credentials used at the destination requires *s3:ReplicateObject* permission.
The following minimal permission policy is needed by admin user setting up replication on the `source`:
```
{
"Version": "2012-10-17",
@ -57,6 +60,7 @@ The following minimal permission policy is needed by admin user setting up repli
```
The access key provided for the replication *target* cluster should have these minimal permissions:
```
{
"Version": "2012-10-17",
@ -96,9 +100,11 @@ The access key provided for the replication *target* cluster should have these m
]
}
```
Please note that the permissions required by the admin user on the target cluster can be more fine grained to exclude permissions like "s3:ReplicateDelete", "s3:GetBucketObjectLockConfiguration" etc depending on whether delete replication rules are set up or if object locking is disabled on `destbucket`. The above policies assume that replication of objects, tags and delete marker replication are all enabled on object lock enabled buckets. A sample script to setup replication is provided [here](https://github.com/minio/minio/blob/master/docs/bucket/replication/setup_replication.sh)
Once successfully created and authorized, the `mc admin bucket remote add` command generates a replication target ARN. This command lists all the currently authorized replication targets:
```
mc admin bucket remote ls myminio/srcbucket --service "replication"
Remote ARN = 'arn:minio:replication:us-east-1:c5be6b16-769d-432a-9ef1-4567081f3566:destbucket'
@ -157,19 +163,25 @@ To perform bi-directional replication, repeat the above process on the target si
![head](https://raw.githubusercontent.com/minio/minio/master/docs/bucket/replication/HEAD_bucket_replication.png)
## Replica Modification sync
If bi-directional replication is set up between two clusters, any metadata update on the REPLICA object is by default reflected back in the source object when `ReplicaModifications` status in the `SourceSelectionCriteria` is `Enabled`. In MinIO, this is enabled by default. If a metadata update is performed on the "REPLICA" object, its `X-Amz-Replication-Status` will change from `PENDING` to `COMPLETE` or `FAILED`, and the source object version will show `X-Amz-Replication-Status` of `REPLICA` once the replication operation is complete.
The replication configuration in use on a bucket can be viewed using the `mc replicate export alias/bucket` command.
To disable replica metadata modification syncing, use `mc replicate edit` with the --replicate flag.
```
$ mc replicate edit alias/bucket --id xyz.id --replicate "delete,delete-marker"
mc replicate edit alias/bucket --id xyz.id --replicate "delete,delete-marker"
```
To re-enable replica metadata modification syncing,
```
$ mc replicate edit alias/bucket --id xyz.id --replicate "delete,delete-marker,replica-metadata-sync"
mc replicate edit alias/bucket --id xyz.id --replicate "delete,delete-marker,replica-metadata-sync"
```
## MinIO Extension
### Replicating Deletes
Delete marker replication is allowed in [AWS V1 Configuration](https://aws.amazon.com/blogs/storage/managing-delete-marker-replication-in-amazon-s3/) but not in V2 configuration. The MinIO implementation above is based on V2 configuration, however it has been extended to allow both DeleteMarker replication and replication of versioned deletes with the `DeleteMarkerReplication` and `DeleteReplication` fields in the replication configuration above. By default, this is set to `Disabled` unless the user specifies it while adding a replication rule.
@ -183,16 +195,19 @@ Note that just like with [AWS](https://docs.aws.amazon.com/AmazonS3/latest/userg
To add a replication rule allowing both delete marker replication, versioned delete replication or both specify the --replicate flag with comma separated values as in the example below.
Additional permission of "s3:ReplicateDelete" action would need to be specified on the access key configured for the target cluster if Delete Marker replication or versioned delete replication is enabled.
```
mc replicate add myminio/srcbucket/Tax --priority 1 --remote-bucket "arn:minio:replication:us-east-1:c5be6b16-769d-432a-9ef1-4567081f3566:destbucket" --tags "Year=2019&Company=AcmeCorp" --storage-class "STANDARD" --replicate "delete,delete-marker"
Replication configuration applied successfully to myminio/srcbucket.
```
> NOTE: In mc versions RELEASE.2021-09-02T09-21-27Z and older, the remote target ARN needs to be passed in the --arn flag and actual remote bucket name in --remote-bucket flag of `mc replicate add`. For example, with the ARN above the replication configuration used to be added with
```
mc replicate add myminio/srcbucket/Tax --priority 1 --remote-bucket destbucket --arn "arn:minio:replication:us-east-1:c5be6b16-769d-432a-9ef1-4567081f3566:destbucket" --tags "Year=2019&Company=AcmeCorp" --storage-class "STANDARD" --replicate "delete,delete-marker"
Replication configuration applied successfully to myminio/srcbucket.
```
```
Also note that for `mc` version `RELEASE.2021-09-02T09-21-27Z` or older supports only a single remote target per bucket. To take advantage of multiple destination replication, use the latest version of `mc`
Status of delete marker replication can be viewed by doing a GET/HEAD on the object version - it will return a `X-Minio-Replication-DeleteMarker-Status` header and http response code of `405`. In the case of permanent deletes, if the delete replication is pending or failed to propagate to the target cluster, GET/HEAD will return additional `X-Minio-Replication-Delete-Status` header and a http response code of `405`.
@ -201,16 +216,19 @@ Status of delete marker replication can be viewed by doing a GET/HEAD on the obj
The status of replication can be monitored by configuring event notifications on the source and target buckets using `mc event add`.On the source side, the `s3:PutObject`, `s3:Replication:OperationCompletedReplication` and `s3:Replication:OperationFailedReplication` events show the status of replication in the `X-Amz-Replication-Status` metadata.
On the target bucket, `s3:PutObject` event shows `X-Amz-Replication-Status` status of `REPLICA` in the metadata. Additional metrics to monitor backlog state for the purpose of bandwidth management and resource allocation are exposed via Prometheus - see https://github.com/minio/minio/blob/master/docs/metrics/prometheus/list.md for more details.
On the target bucket, `s3:PutObject` event shows `X-Amz-Replication-Status` status of `REPLICA` in the metadata. Additional metrics to monitor backlog state for the purpose of bandwidth management and resource allocation are exposed via Prometheus - see <https://github.com/minio/minio/blob/master/docs/metrics/prometheus/list.md> for more details.
### Sync/Async Replication
By default, replication is completed asynchronously. If synchronous replication is desired, set the --sync flag while adding a
remote replication target using the `mc admin bucket remote add` command
```
mc admin bucket remote add myminio/srcbucket https://accessKey:secretKey@replica-endpoint:9000/destbucket --service replication --region us-east-1 --sync --healthcheck-seconds 100
```
### Existing object replication
Existing object replication as detailed [here](https://aws.amazon.com/blogs/storage/replicating-existing-objects-between-s3-buckets/) can be enabled by passing `existing-objects` as a value to `--replicate` flag while adding or editing a replication rule.
Once existing object replication is enabled, all objects or object prefixes that satisfy the replication rules and were created prior to adding replication configuration OR while replication rules were disabled will be synced to the target cluster. Depending on the number of previously existing objects, the existing objects that are now eligible to be replicated will eventually be synced to the target cluster as the scanner schedules them. This may be slower depending on the load on the cluster, latency and size of the namespace.
@ -226,6 +244,7 @@ Replication from a source bucket to multiple destination buckets is supported. F
Note that on the source side, the `X-Amz-Replication-Status` changes from `PENDING` to `COMPLETED` after replication succeeds to each of the targets. On the destination side, a `X-Amz-Replication-Status` status of `REPLICA` indicates that the object was replicated successfully. Any replication failures are automatically re-attempted during a periodic disk scanner cycle.
## Explore Further
- [MinIO Bucket Replication Design](https://github.com/minio/minio/blob/master/docs/bucket/replication/DESIGN.md)
- [MinIO Bucket Versioning Implementation](https://docs.minio.io/docs/minio-bucket-versioning-guide.html)
- [MinIO Client Quickstart Guide](https://docs.minio.io/docs/minio-client-quickstart-guide.html)

7
docs/bucket/retention/README.md
View File

@ -18,7 +18,7 @@ Independent of retention, an object can also be under legal hold. This effective
WORM on a bucket is enabled by setting object lock configuration. This configuration is applied to existing and new objects in the bucket. Below is an example sets `Governance` mode and one day retention time from object creation time of all objects in `mybucket`.
```sh
$ awscli s3api put-object-lock-configuration --bucket mybucket --object-lock-configuration 'ObjectLockEnabled=\"Enabled\",Rule={DefaultRetention={Mode=\"GOVERNANCE\",Days=1}}'
awscli s3api put-object-lock-configuration --bucket mybucket --object-lock-configuration 'ObjectLockEnabled=\"Enabled\",Rule={DefaultRetention={Mode=\"GOVERNANCE\",Days=1}}'
```
### Set object lock
@ -29,7 +29,7 @@ PutObject API allows setting per object retention mode and retention duration us
aws s3api put-object --bucket testbucket --key lockme --object-lock-mode GOVERNANCE --object-lock-retain-until-date "2019-11-20" --body /etc/issue
```
See https://docs.aws.amazon.com/AmazonS3/latest/dev/object-lock-overview.html for AWS S3 spec on object locking and permissions required for object retention and governance bypass overrides.
See <https://docs.aws.amazon.com/AmazonS3/latest/dev/object-lock-overview.html> for AWS S3 spec on object locking and permissions required for object retention and governance bypass overrides.
### Set legal hold on an object
@ -39,9 +39,10 @@ PutObject API allows setting legal hold using `x-amz-object-lock-legal-hold` hea
aws s3api put-object --bucket testbucket --key legalhold --object-lock-legal-hold-status ON --body /etc/issue
```
See https://docs.aws.amazon.com/AmazonS3/latest/dev/object-lock-overview.html for AWS S3 spec on object locking and permissions required for specifying legal hold.
See <https://docs.aws.amazon.com/AmazonS3/latest/dev/object-lock-overview.html> for AWS S3 spec on object locking and permissions required for specifying legal hold.
## Concepts
- If an object is under legal hold, it cannot be deleted unless the legal hold is explicitly removed for the respective version id. DeleteObjectVersion() would fail otherwise.
- In `Compliance` mode, objects cannot be deleted by anyone until retention period is expired for the respective version id. If user has requisite governance bypass permissions, an object's retention date can be extended in `Compliance` mode.
- Once object lock configuration is set to a bucket

23
docs/bucket/versioning/DESIGN.md
View File

@ -3,10 +3,10 @@
## Description of `xl.meta`
`xl.meta` is a new self describing backend format used by MinIO to support AWS S3 compatible versioning.
This file is the source of truth for each `version` at rest. `xl.meta` is a msgpack file serialized from a
This file is the source of truth for each `version` at rest. `xl.meta` is a msgpack file serialized from a
well defined data structure. To understand `xl.meta` here are the few things to start with
`xl.meta` carries first 8 bytes an XL header which describes the current format and the format version,
`xl.meta` carries first 8 bytes an XL header which describes the current format and the format version,
allowing the unmarshaller's to automatically use the right data structures to parse the subsequent content in the stream.
### v1.0
@ -43,6 +43,7 @@ Therefore, the metadata is wrapped as a binary array for easy skipping.
- DeleteMarker (a versionId to capture the DELETE sequences implemented primarily for AWS spec compatibility)
A sample msgpack-JSON `xl.meta`, you can debug the content inside `xl.meta` using [xl-meta.go](https://github.com/minio/minio/tree/master/docs/debugging#decoding-metadata) program.
```json
{
"Versions": [
@ -92,7 +93,7 @@ Version 1.3 introduces changes to help with [faster metadata reads and updates](
| versions | msgp int | Number of versions following
| header_1 | msgp bin array | Header of version 1
| metadata_1 | msgp bin array | Metadata of version 1
| ...header_n | msgp bin array | Header of last version
| ...header_n | msgp bin array | Header of last version
| ...metadata_n | msgp bin array | Metadata of last version
Each header contains a mspg array (tuple) encoded object:
@ -102,11 +103,11 @@ xlHeaderVersion version == 1:
```
//msgp:tuple xlMetaV2VersionHeader
type xlMetaV2VersionHeader struct {
VersionID [16]byte // Version UUID, raw.
ModTime int64 // Unix nanoseconds.
Signature [4]byte // Signature of metadata.
Type uint8 // Type if the version
Flags uint8
VersionID [16]byte // Version UUID, raw.
ModTime int64 // Unix nanoseconds.
Signature [4]byte // Signature of metadata.
Type uint8 // Type if the version
Flags uint8
}
```
@ -114,9 +115,9 @@ The following flags are defined:
```
const (
FreeVersion = 1 << 0
UsesDataDir = 1 << 1
InlineData = 1 << 2
FreeVersion = 1 << 0
UsesDataDir = 1 << 1
InlineData = 1 << 2
)
```

6
docs/bucket/versioning/README.md
View File

@ -31,6 +31,7 @@ To permanently delete an object you need to specify the version you want to dele
![delete_version_id](https://raw.githubusercontent.com/minio/minio/master/docs/bucket/versioning/versioning_DELETE_versionEnabled_id.png)
## Concepts
- All Buckets on MinIO are always in one of the following states: unversioned (the default) and all other existing deployments, versioning-enabled, or versioning-suspended.
- Versioning state applies to all of the objects in the versioning enabled bucket. The first time you enable a bucket for versioning, objects in the bucket are thereafter always versioned and given a unique version ID.
- Existing or newer buckets can be created with versioning enabled and eventually can be suspended as well. Existing versions of objects stay as is and can still be accessed using the version ID.
@ -38,13 +39,16 @@ To permanently delete an object you need to specify the version you want to dele
- **Versioning feature is only available in erasure coded and distributed erasure coded setups**.
## How to configure versioning on a bucket
Each bucket created has a versioning configuration associated with it. By default bucket is unversioned as shown below
```
<VersioningConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
</VersioningConfiguration>
```
To enable versioning, you send a request to MinIO with a versioning configuration with Status set to `Enabled`.
```
<VersioningConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<Status>Enabled</Status>
@ -52,6 +56,7 @@ To enable versioning, you send a request to MinIO with a versioning configuratio
```
Similarly to suspend versioning set the configuration with Status set to `Suspended`.
```
<VersioningConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<Status>Suspended</Status>
@ -160,6 +165,7 @@ public class IsVersioningEnabled {
```
## Explore Further
- [Use `minio-java` SDK with MinIO Server](https://docs.minio.io/docs/java-client-quickstart-guide.html)
- [Object Lock and Immutablity Guide](https://docs.minio.io/docs/minio-bucket-object-lock-guide.html)
- [MinIO Admin Complete Guide](https://docs.min.io/docs/minio-admin-complete-guide.html)

12
docs/chroot/README.md
View File

@ -3,10 +3,12 @@
Chroot allows user based namespace isolation on many standard Linux deployments.
## 1. Prerequisites
* Familiarity with [chroot](http://man7.org/linux/man-pages/man2/chroot.2.html)
* Chroot installed on your machine.
- Familiarity with [chroot](http://man7.org/linux/man-pages/man2/chroot.2.html)
- Chroot installed on your machine.
## 2. Install MinIO in Chroot
```sh
mkdir -p /mnt/export/${USER}/bin
wget https://dl.min.io/server/minio/release/linux-amd64/minio -O /mnt/export/${USER}/bin/minio
@ -14,12 +16,15 @@ chmod +x /mnt/export/${USER}/bin/minio
```
Bind your `proc` mount to the target chroot directory
```
sudo mount --bind /proc /mnt/export/${USER}/proc
```
## 3. Run Standalone MinIO in Chroot
### GNU/Linux
```sh
sudo chroot --userspec username:group /mnt/export/${USER} /bin/minio --config-dir=/.minio server /data
@ -30,9 +35,10 @@ SecretKey: X3RKxEeFOI8InuNWoPsbG+XEVoaJVCqbvxe+PTOa
...
```
Instance is now accessible on the host at port 9000, proceed to access the Web browser at http://127.0.0.1:9000/
Instance is now accessible on the host at port 9000, proceed to access the Web browser at <http://127.0.0.1:9000/>
## Explore Further
- [MinIO Erasure Code QuickStart Guide](https://docs.min.io/docs/minio-erasure-code-quickstart-guide)
- [Use `mc` with MinIO Server](https://docs.min.io/docs/minio-client-quickstart-guide)
- [Use `aws-cli` with MinIO Server](https://docs.min.io/docs/aws-cli-with-minio)

69
docs/compression/README.md
View File

@ -1,18 +1,18 @@
# Compression Guide [![Slack](https://slack.min.io/slack?type=svg)](https://slack.min.io)
MinIO server allows streaming compression to ensure efficient disk space usage.
Compression happens inflight, i.e objects are compressed before being written to disk(s).
MinIO uses [`klauspost/compress/s2`](https://github.com/klauspost/compress/tree/master/s2)
MinIO server allows streaming compression to ensure efficient disk space usage.
Compression happens inflight, i.e objects are compressed before being written to disk(s).
MinIO uses [`klauspost/compress/s2`](https://github.com/klauspost/compress/tree/master/s2)
streaming compression due to its stability and performance.
This algorithm is specifically optimized for machine generated content.
This algorithm is specifically optimized for machine generated content.
Write throughput is typically at least 500MB/s per CPU core,
and scales with the number of available CPU cores.
and scales with the number of available CPU cores.
Decompression speed is typically at least 1GB/s.
This means that in cases where raw IO is below these numbers
This means that in cases where raw IO is below these numbers
compression will not only reduce disk usage but also help increase system throughput.
Typically, enabling compression on spinning disk systems
Typically, enabling compression on spinning disk systems
will increase speed when the content can be compressed.
## Get Started
@ -23,7 +23,7 @@ Install MinIO - [MinIO Quickstart Guide](https://docs.min.io/docs/minio-quicksta
### 2. Run MinIO with compression
Compression can be enabled by updating the `compress` config settings for MinIO server config.
Compression can be enabled by updating the `compress` config settings for MinIO server config.
Config `compress` settings take extensions and mime-types to be compressed.
```bash
@ -38,18 +38,19 @@ Default config includes most common highly compressible content extensions and m
```
To show help on setting compression config values.
```bash
~ mc admin config set myminio compression
```
To enable compression for all content, no matter the extension and content type
To enable compression for all content, no matter the extension and content type
(except for the default excluded types) set BOTH extensions and mime types to empty.
```bash
~ mc admin config set myminio compression enable="on" extensions="" mime_types=""
```
The compression settings may also be set through environment variables.
The compression settings may also be set through environment variables.
When set, environment variables override the defined `compress` config settings in the server config.
```bash
@ -66,7 +67,7 @@ See [CRIME TLS](https://en.wikipedia.org/wiki/CRIME) as an example of this.
Therefore, compression is disabled when encrypting by default, and must be enabled separately.
Consult our security experts on [SUBNET](https://min.io/pricing) to help you evaluate if
Consult our security experts on [SUBNET](https://min.io/pricing) to help you evaluate if
your setup can use this feature combination safely.
To enable compression+encryption use:
@ -79,40 +80,40 @@ Or alternatively through the environment variable `MINIO_COMPRESS_ALLOW_ENCRYPTI
### 4. Excluded Types
- Already compressed objects are not fit for compression since they do not have compressible patterns.
- Already compressed objects are not fit for compression since they do not have compressible patterns.
Such objects do not produce efficient [`LZ compression`](https://en.wikipedia.org/wiki/LZ77_and_LZ78)
which is a fitness factor for a lossless data compression.
Pre-compressed input typically compresses in excess of 2GiB/s per core,
Pre-compressed input typically compresses in excess of 2GiB/s per core,
so performance impact should be minimal even if precompressed data is re-compressed.
Decompressing incompressible data has no significant performance impact.
Below is a list of common files and content-types which are typically not suitable for compression.
- Extensions
- Extensions
| `gz` | (GZIP)
| `bz2` | (BZIP2)
| `rar` | (WinRAR)
| `zip` | (ZIP)
| `7z` | (7-Zip)
| `xz` | (LZMA)
| `mp4` | (MP4)
| `mkv` | (MKV media)
| `mov` | (MOV)
| `gz` | (GZIP) |
| `bz2` | (BZIP2) |
| `rar` | (WinRAR) |
| `zip` | (ZIP) |
| `7z` | (7-Zip) |
| `xz` | (LZMA) |
| `mp4` | (MP4) |
| `mkv` | (MKV media) |
| `mov` | (MOV) |
- Content-Types
- Content-Types
| `video/*` |
| `audio/*` |
| `application/zip` |
| `application/x-gzip` |
| `application/zip` |
| `application/x-bz2` |
| `application/x-compress` |
| `application/x-xz` |
| `video/*` |
| `audio/*` |
| `application/zip` |
| `application/x-gzip` |
| `application/zip` |
| `application/x-bz2` |
| `application/x-compress` |
| `application/x-xz` |
All files with these extensions and mime types are excluded from compression,
All files with these extensions and mime types are excluded from compression,
even if compression is enabled for all types.
### 5. Notes
@ -121,7 +122,7 @@ even if compression is enabled for all types.
## To test the setup
To test this setup, practice put calls to the server using `mc` and use `mc ls` on
To test this setup, practice put calls to the server using `mc` and use `mc ls` on
the data directory to view the size of the object.
## Explore Further

19
docs/config/README.md
View File

@ -22,6 +22,7 @@ $ mc tree --files ~/.minio
You can provide a custom certs directory using `--certs-dir` command line option.
#### Credentials
On MinIO admin credentials or root credentials are only allowed to be changed using ENVs namely `MINIO_ROOT_USER` and `MINIO_ROOT_PASSWORD`.
```sh
@ -31,6 +32,7 @@ minio server /data
```
#### Site
```
KEY:
site label the server and its location
@ -42,6 +44,7 @@ comment (sentence) optionally add a comment to this setting
```
or environment variables
```
KEY:
site label the server and its location
@ -61,6 +64,7 @@ minio server /data
```
### Storage Class
By default, parity for objects with standard storage class is set to `N/2`, and parity for objects with reduced redundancy storage class objects is set to `2`. Read more about storage class support in MinIO server [here](https://github.com/minio/minio/blob/master/docs/erasure/storage-class/README.md).
```
@ -74,6 +78,7 @@ comment (sentence) optionally add a comment to this setting
```
or environment variables
```
KEY:
storage_class define object level redundancy
@ -85,6 +90,7 @@ MINIO_STORAGE_CLASS_COMMENT (sentence) optionally add a comment to this setti
```
### Cache
MinIO provides caching storage tier for primarily gateway deployments, allowing you to cache content for faster reads, cost savings on repeated downloads from the cloud.
```
@ -101,6 +107,7 @@ comment (sentence) optionally add a comment to this setting
```
or environment variables
```
KEY:
cache add caching storage tier
@ -115,6 +122,7 @@ MINIO_CACHE_COMMENT (sentence) optionally add a comment to this setting
```
#### Etcd
MinIO supports storing encrypted IAM assets in etcd, if KMS is configured. Please refer to how to encrypt your config and IAM credentials [here](https://github.com/minio/minio/blob/master/docs/kms/IAM.md)
> NOTE: if *path_prefix* is set then MinIO will not federate your buckets, namespaced IAM assets are assumed as isolated tenants, only buckets are considered globally unique but performing a lookup with a *bucket* which belongs to a different tenant will fail unlike federated setups where MinIO would port-forward and route the request to relevant cluster accordingly. This is a special feature, federated deployments should not need to set *path_prefix*.
@ -133,6 +141,7 @@ comment (sentence) optionally add a comment to this setting
```
or environment variables
```
KEY:
etcd federate multiple clusters for IAM and Bucket DNS
@ -147,6 +156,7 @@ MINIO_ETCD_COMMENT (sentence) optionally add a comment to this setting
```
### API
By default, there is no limitation on the number of concurrent requests that a server/cluster processes at the same time. However, it is possible to impose such limitation using the API subsystem. Read more about throttling limitation in MinIO server [here](https://github.com/minio/minio/blob/master/docs/throttle/README.md).
```
@ -170,6 +180,7 @@ MINIO_API_REMOTE_TRANSPORT_DEADLINE (duration) set the deadline for API reques
```
#### Notifications
Notification targets supported by MinIO are in the following list. To configure individual targets please refer to more detailed documentation [here](https://docs.min.io/docs/minio-bucket-notification-guide.html)
```
@ -186,14 +197,17 @@ notify_redis publish bucket notifications to Redis datastores
```
### Accessing configuration
All configuration changes can be made using [`mc admin config` get/set/reset/export/import commands](https://github.com/minio/mc/blob/master/docs/minio-admin-complete-guide.md).
#### List all config keys available
```
~ mc admin config set myminio/
```
#### Obtain help for each key
```
~ mc admin config set myminio/ <key>
```
@ -215,6 +229,7 @@ comment (sentence) optionally add a comment to this setting
```
To get ENV equivalent for each config args use `--env` flag
```
~ mc admin config set play/ etcd --env
KEY:
@ -311,7 +326,7 @@ minio server /data
### Domain
By default, MinIO supports path-style requests that are of the format http://mydomain.com/bucket/object. `MINIO_DOMAIN` environment variable is used to enable virtual-host-style requests. If the request `Host` header matches with `(.+).mydomain.com` then the matched pattern `$1` is used as bucket and the path is used as object. More information on path-style and virtual-host-style [here](http://docs.aws.amazon.com/AmazonS3/latest/dev/RESTAPI.html)
By default, MinIO supports path-style requests that are of the format <http://mydomain.com/bucket/object>. `MINIO_DOMAIN` environment variable is used to enable virtual-host-style requests. If the request `Host` header matches with `(.+).mydomain.com` then the matched pattern `$1` is used as bucket and the path is used as object. More information on path-style and virtual-host-style [here](http://docs.aws.amazon.com/AmazonS3/latest/dev/RESTAPI.html)
Example:
```sh
@ -320,11 +335,13 @@ minio server /data
```
For advanced use cases `MINIO_DOMAIN` environment variable supports multiple-domains with comma separated values.
```sh
export MINIO_DOMAIN=sub1.mydomain.com,sub2.mydomain.com
minio server /data
```
## Explore Further
* [MinIO Quickstart Guide](https://docs.min.io/docs/minio-quickstart-guide)
* [Configure MinIO Server with TLS](https://docs.min.io/docs/how-to-secure-access-to-minio-server-with-tls)

60
docs/debugging/README.md
View File

@ -1,42 +1,51 @@
# MinIO Server Debugging Guide [![Slack](https://slack.min.io/slack?type=svg)](https://slack.min.io) [![Docker Pulls](https://img.shields.io/docker/pulls/minio/minio.svg?maxAge=604800)](https://hub.docker.com/r/minio/minio/)
### HTTP Trace
## HTTP Trace
HTTP tracing can be enabled by using [`mc admin trace`](https://github.com/minio/mc/blob/master/docs/minio-admin-complete-guide.md#command-trace---display-minio-server-http-trace) command.
Example:
```sh
minio server /data
```
Default trace is succinct only to indicate the API operations being called and the HTTP response status.
```sh
mc admin trace myminio
```
To trace entire HTTP request
```sh
mc admin trace --verbose myminio
```
To trace entire HTTP request and also internode communication
```sh
mc admin trace --all --verbose myminio
```
### Subnet Health
## Subnet Health
Subnet Health diagnostics help ensure that the underlying infrastructure that runs MinIO is configured correctly, and is functioning properly. This test is one-shot long running one, that is recommended to be run as soon as the cluster is first provisioned, and each time a failure scenario is encountered. Note that the test incurs majority of the available resources on the system. Care must be taken when using this to debug failure scenario, so as to prevent larger outages. Health tests can be triggered using `mc admin subnet health` command.
Example:
```sh
minio server /data
```
The command takes no flags
```sh
mc admin subnet health myminio
```
The output printed will be of the form
```sh
● Admin Info ... ✔
● CPU ... ✔
@ -57,49 +66,29 @@ mc: Health data saved to dc-11-health_20200321053323.json.gz
The gzipped output contains debugging information for your system
### Decoding Metadata
## Decoding Metadata
Metadata is stored in `xl.meta` files for erasure coded objects.
Each disk in the set containing the object has this file.
The file format is a binary format and therefore requires tools to view values.
Metadata is stored in `xl.meta` files for erasure coded objects. Each disk in the set containing the object has this file. The file format is a binary format and therefore requires tools to view values.
#### Installing xl-meta
### Installing xl-meta
To install, [Go](https://golang.org/dl/) must be installed.
Once installed, execute this to install the binary:
To install, [Go](https://golang.org/dl/) must be installed. Once installed, execute this to install the binary:
```bash
go install github.com/minio/minio/docs/debugging/xl-meta@latest
```
#### Using xl-meta
### Using xl-meta
Executing `xl-meta` will look for an `xl.meta` in the current folder and decode it to JSON.
It is also possible to specify multiple files or wildcards, for example `xl-meta ./**/xl.meta` will output decoded metadata recursively.
It is possible to view what inline data is stored inline in the metadata using `--data` parameter `xl-meta -data xl.json` will display an id -> data size.
To export inline data to a file use the `--export` option.
Executing `xl-meta` will look for an `xl.meta` in the current folder and decode it to JSON. It is also possible to specify multiple files or wildcards, for example `xl-meta ./**/xl.meta` will output decoded metadata recursively. It is possible to view what inline data is stored inline in the metadata using `--data` parameter `xl-meta -data xl.json` will display an id -> data size. To export inline data to a file use the `--export` option.
### Remotely Inspecting backend data
`mc admin inspect` allows collecting files based on *path* from all backend drives.
`mc admin inspect` allows collecting files based on *path* from all backend drives. Matching files will be collected in a zip file with their respective host+drive+path. A MinIO host from October 2021 or later is required for full functionality. Syntax is `mc admin inspect ALIAS/path/to/files`. This can for example be used to collect `xl.meta` from objects that are misbehaving. To collect `xl.meta` from a specific object, for example placed at `ALIAS/bucket/path/to/file.txt` append `/xl.meta`, for instance `mc admin inspect ALIAS/bucket/path/to/file.txt/xl.meta`. All files can be collected, so this can also be used to retrieve `part.*` files, etc.
Matching files will be collected in a zip file with their respective host+drive+path.
Wildcards can be used, for example `mc admin inspect ALIAS/bucket/path/**/xl.meta` will collect all `xl.meta` recursively. `mc admin inspect ALIAS/bucket/path/to/file.txt/*/part.*` will collect parts for all versions for the object located at `bucket/path/to/file.txt`.
A MinIO host from October 2021 or later is required for full functionality.
Syntax is `mc admin inspect ALIAS/path/to/files`. This can for example be used to collect `xl.meta` from objects that are misbehaving.
To collect `xl.meta` from a specific object, for example placed at `ALIAS/bucket/path/to/file.txt` append `/xl.meta`, for instance `mc admin inspect ALIAS/bucket/path/to/file.txt/xl.meta`.
All files can be collected, so this can also be used to retrieve `part.*` files, etc.
Wildcards can be used, for example `mc admin inspect ALIAS/bucket/path/**/xl.meta` will collect all `xl.meta` recursively.
`mc admin inspect ALIAS/bucket/path/to/file.txt/*/part.*` will collect parts for all versions for the object located at `bucket/path/to/file.txt`.
`xl-meta` accepts zip files as input and will output all `xl.meta` files found within the archive.
For example:
`xl-meta` accepts zip files as input and will output all `xl.meta` files found within the archive. For example:
```
$ mc admin inspect play/test123/test*/xl.meta
@ -127,7 +116,7 @@ mc: Even with the decryption key, data stored with encryption cannot be accessed
This file can be decrypted using the decryption tool below:
#### Installing decryption tool
### Installing decryption tool
To install, [Go](https://golang.org/dl/) must be installed.
@ -136,7 +125,8 @@ Once installed, execute this to install the binary:
```bash
go install github.com/minio/minio/docs/debugging/inspect@latest
```
#### Usage
### Usage
To decrypt the file above:
@ -145,6 +135,4 @@ $ inspect -key=ad2b43d847fdb14e54c5836200177f7158b3f745433525f5d23c0e0208e50c994
Output decrypted to inspect.ad2b43d8.zip
```
If `--key` is not specified an interactive prompt will ask for it.
The file name will contain the beginning of the key. This can be used to verify that the key is for the encrypted file.
If `--key` is not specified an interactive prompt will ask for it. The file name will contain the beginning of the key. This can be used to verify that the key is for the encrypted file.

8
docs/disk-caching/DESIGN.md
View File

@ -40,7 +40,9 @@ minio gateway s3 https://s3.amazonaws.com
```
### Run MinIO gateway with cache on Docker Container
### Stable
Cache drives need to have `strictatime` or `relatime` enabled for disk caching feature. In this example, mount the xfs file system on /mnt/cache with `strictatime` or `relatime` enabled.
```sh
@ -48,21 +50,25 @@ truncate -s 4G /tmp/data
```
### Build xfs filesystem on /tmp/data
```
mkfs.xfs /tmp/data
```
### Create mount dir
```
sudo mkdir /mnt/cache #
```
### Mount xfs on /mnt/cache with atime.
### Mount xfs on /mnt/cache with atime
```
sudo mount -o relatime /tmp/data /mnt/cache
```
### Start using the cached drive with S3 gateway
```
podman run --net=host -e MINIO_ROOT_USER={s3-access-key} -e MINIO_ROOT_PASSWORD={s3-secret-key} \
-e MINIO_CACHE_DRIVES=/cache -e MINIO_CACHE_QUOTA=99 -e MINIO_CACHE_AFTER=0 \

5
docs/disk-caching/README.md
View File

@ -29,9 +29,8 @@ export MINIO_CACHE_WATERMARK_HIGH=90
minio gateway s3
```
The `CACHE_WATERMARK` numbers are percentages of `CACHE_QUOTA`.
In the example above this means that `MINIO_CACHE_WATERMARK_LOW` is effectively `0.8 * 0.7 * 100 = 56%` and the `MINIO_CACHE_WATERMARK_HIGH` is effectively `0.8 * 0.9 * 100 = 72%` of total disk space.
The `CACHE_WATERMARK` numbers are percentages of `CACHE_QUOTA`.
In the example above this means that `MINIO_CACHE_WATERMARK_LOW` is effectively `0.8 * 0.7 * 100 = 56%` and the `MINIO_CACHE_WATERMARK_HIGH` is effectively `0.8 * 0.9 * 100 = 72%` of total disk space.
### 3. Test your setup

34
docs/distributed/DECOMMISSION.md
View File

@ -1,21 +1,23 @@
## Decommissioning
# Decommissioning
Decommissiong is a mechanism in MinIO to drain older pools (usually with old hardware) and migrate the content from such pools to a newer pools (usually better hardware). Decommissioning spreads the data across all pools - for example if you decommission `pool1`, all the data from `pool1` shall be spread across `pool2` and `pool3` respectively.
### Features
## Features
- A pool in decommission still allows READ access to all its contents, newer WRITEs will be automatically scheduled to only new pools.
- All versioned buckets maintain the same order for "versions" for each objects after being decommissioned to the newer pools.
- A pool in decommission resumes from where it was left off (for example - in-case of cluster restarts or restarts attempted after a failed decommission attempt).
### How to decommission a pool?
## How to decommission a pool?
```
λ mc admin decommission start alias/ http://minio{1...2}/data{1...4}
```
### Status decommissioning a pool
## Status decommissioning a pool
### Decommissioning without args lists all pools
#### Decommissioning without args lists all pools
```
λ mc admin decommission status alias/
┌─────┬─────────────────────────────────┬──────────────────────────────────┬────────┐
@ -25,7 +27,8 @@ Decommissiong is a mechanism in MinIO to drain older pools (usually with old har
└─────┴─────────────────────────────────┴──────────────────────────────────┴────────┘
```
#### Decommissioning status
### Decommissioning status
```
λ mc admin decommission status alias/ http://minio{1...2}/data{1...4}
Decommissioning rate at 36 MiB/sec [4 TiB/50 TiB]
@ -39,13 +42,15 @@ Once it is **Complete**
Decommission of pool http://minio{1...2}/data{1...4} is complete, you may now remove it from server command line
```
#### A pool not under decommissioning will throw an error
### A pool not under decommissioning will throw an error
```
λ mc admin decommission status alias/ http://minio{1...2}/data{1...4}
ERROR: This pool is not scheduled for decommissioning currently.
```
### Canceling a decommission?
## Canceling a decommission?
Stop an on-going decommission in progress, mainly used in situations when the load may be too high and you may want to schedule the decommission at a later point in time.
`mc admin decommission cancel` without an argument, lists out any on-going decommission in progress.
@ -69,6 +74,7 @@ Stop an on-going decommission in progress, mainly used in situations when the lo
```
If for some reason decommission fails in between, the `status` will indicate decommission as failed instead.
```
λ mc admin decommission status alias/
┌─────┬─────────────────────────────────┬──────────────────────────────────┬──────────────────┐
@ -78,14 +84,16 @@ If for some reason decommission fails in between, the `status` will indicate dec
└─────┴─────────────────────────────────┴──────────────────────────────────┴──────────────────┘
```
### Restart a canceled or failed decommission?
## Restart a canceled or failed decommission?
```
λ mc admin decommission start alias/ http://minio{1...2}/data{1...4}
```
### When decommission is 'Complete'?
## When decommission is 'Complete'?
Once decommission is complete, it will be indicated with *Complete* status. *Complete* means that now you can now safely remove the first pool argument from the MinIO command line.
```
λ mc admin decommission status alias/
┌─────┬─────────────────────────────────┬──────────────────────────────────┬──────────┐
@ -103,10 +111,12 @@ Once decommission is complete, it will be indicated with *Complete* status. *Co
> Without a 'Complete' status any 'Active' or 'Draining' pool(s) are not allowed to be removed once configured.
### NOTE
## NOTE
- Empty delete marker's i.e objects with no other successor versions are not transitioned to the new pool, to avoid any empty metadata being recreated on the newer pool. We do not think this is needed, please open a GitHub issue if you think otherwise.
### TODO
## TODO
- Richer progress UI is not present at the moment, this will be addressed in subsequent releases. Currently however a RATE of data transfer and usage increase is displayed via `mc`.
- Transitioned Hot Tier's as pooled setups are not currently supported, attempting to decommission buckets with ILM Transition will be rejected by the server. This will be supported in future releases.
- Embedded Console UI does not support Decommissioning through the UI yet. This will be supported in future releases.

15
docs/distributed/DESIGN.md
View File

@ -1,7 +1,9 @@
# Distributed Server Design Guide [![Slack](https://slack.min.io/slack?type=svg)](https://slack.min.io)
This document explains the design, architecture and advanced use cases of the MinIO distributed server.
## Command-line
```
NAME:
minio server - start object storage server
@ -22,6 +24,7 @@ DIR:
## Common usage
Standalone erasure coded configuration with 4 sets with 16 disks each.
```
minio server dir{1...64}
```
@ -44,7 +47,7 @@ Expansion of ellipses and choice of erasure sets based on this expansion is an a
- Choice of erasure set size is automatic based on the number of disks available, let's say for example if there are 32 servers and 32 disks which is a total of 1024 disks. In this scenario 16 becomes the erasure set size. This is decided based on the greatest common divisor (GCD) of acceptable erasure set sizes ranging from *4 to 16*.
- *If total disks has many common divisors the algorithm chooses the minimum amounts of erasure sets possible for a erasure set size of any N*. In the example with 1024 disks - 4, 8, 16 are GCD factors. With 16 disks we get a total of 64 possible sets, with 8 disks we get a total of 128 possible sets, with 4 disks we get a total of 256 possible sets. So algorithm automatically chooses 64 sets, which is *16 * 64 = 1024* disks in total.
- *If total disks has many common divisors the algorithm chooses the minimum amounts of erasure sets possible for a erasure set size of any N*. In the example with 1024 disks - 4, 8, 16 are GCD factors. With 16 disks we get a total of 64 possible sets, with 8 disks we get a total of 128 possible sets, with 4 disks we get a total of 256 possible sets. So algorithm automatically chooses 64 sets, which is *16* 64 = 1024* disks in total.
- *If total number of nodes are of odd number then GCD algorithm provides affinity towards odd number erasure sets to provide for uniform distribution across nodes*. This is to ensure that same number of disks are pariticipating in any erasure set. For example if you have 2 nodes with 180 drives then GCD is 15 but this would lead to uneven distribution, one of the nodes would participate more drives. To avoid this the affinity is given towards nodes which leads to next best GCD factor of 12 which provides uniform distribution.
@ -55,6 +58,7 @@ minio server http://host{1...2}/export{1...8}
```
Expected expansion
```
> http://host1/export1
> http://host2/export1
@ -77,6 +81,7 @@ Expected expansion
*A noticeable trait of this expansion is that it chooses unique hosts such the setup provides maximum protection and availability.*
- Choosing an erasure set for the object is decided during `PutObject()`, object names are used to find the right erasure set using the following pseudo code.
```go
// hashes the key returning an integer.
func sipHashMod(key string, cardinality int, id [16]byte) int {
@ -88,6 +93,7 @@ func sipHashMod(key string, cardinality int, id [16]byte) int {
return int(sip.Sum64() % uint64(cardinality))
}
```
Input for the key is the object name specified in `PutObject()`, returns a unique index. This index is one of the erasure sets where the object will reside. This function is a consistent hash for a given object name i.e for a given object name the index returned is always the same.
- Write and Read quorum are required to be satisfied only across the erasure set for an object. Healing is also done per object within the erasure set which contains the object.
@ -96,7 +102,7 @@ Input for the key is the object name specified in `PutObject()`, returns a uniqu
- MinIO also supports expansion of existing clusters in server pools. Each pool is a self contained entity with same SLA's (read/write quorum) for each object as original cluster. By using the existing namespace for lookup validation MinIO ensures conflicting objects are not created. When no such object exists then MinIO simply uses the least used pool to place new objects.
__There are no limits on how many server pools can be combined__
### There are no limits on how many server pools can be combined
```
minio server http://host{1...32}/export{1...32} http://host{1...12}/export{1...12}
@ -112,6 +118,7 @@ In above example there are two server pools
Refer to the sizing guide with details on the default parity count chosen for different erasure stripe sizes [here](https://github.com/minio/minio/blob/master/docs/distributed/SIZING.md)
MinIO places new objects in server pools based on proportionate free space, per pool. Following pseudo code demonstrates this behavior.
```go
func getAvailablePoolIdx(ctx context.Context) int {
serverPools := z.getServerPoolsAvailableSpace(ctx)
@ -135,21 +142,25 @@ func getAvailablePoolIdx(ctx context.Context) int {
### Advanced use cases with multiple ellipses
Standalone erasure coded configuration with 4 sets with 16 disks each, which spawns disks across controllers.
```
minio server /mnt/controller{1...4}/data{1...16}
```
Standalone erasure coded configuration with 16 sets, 16 disks per set, across mounts and controllers.
```
minio server /mnt{1...4}/controller{1...4}/data{1...16}
```
Distributed erasure coded configuration with 2 sets, 16 disks per set across hosts.
```
minio server http://host{1...32}/disk1
```
Distributed erasure coded configuration with rack level redundancy 32 sets in total, 16 disks per set.
```
minio server http://rack{1...4}-host{1...8}.example.net/export{1...16}
```

29
docs/distributed/README.md
View File

@ -38,24 +38,24 @@ Install MinIO - [MinIO Quickstart Guide](https://docs.min.io/docs/minio-quicksta
To start a distributed MinIO instance, you just need to pass drive locations as parameters to the minio server command. Then, you’ll need to run the same command on all the participating nodes.
__NOTE:__
**NOTE:**
- All the nodes running distributed MinIO should share a common root credentials, for the nodes to connect and trust each other. To achieve this, it is __recommended__ to export root user and root password as environment variables, `MINIO_ROOT_USER` and `MINIO_ROOT_PASSWORD`, on all the nodes before executing MinIO server command. If not exported, default `minioadmin/minioadmin` credentials shall be used.
- __MinIO creates erasure-coding sets of *4* to *16* drives per set. The number of drives you provide in total must be a multiple of one of those numbers.__
- __MinIO chooses the largest EC set size which divides into the total number of drives or total number of nodes given - making sure to keep the uniform distribution i.e each node participates equal number of drives per set__.
- __Each object is written to a single EC set, and therefore is spread over no more than 16 drives.__
- __All the nodes running distributed MinIO setup are recommended to be homogeneous, i.e. same operating system, same number of disks and same network interconnects.__
- MinIO distributed mode requires __fresh directories__. If required, the drives can be shared with other applications. You can do this by using a sub-directory exclusive to MinIO. For example, if you have mounted your volume under `/export`, pass `/export/data` as arguments to MinIO server.
- All the nodes running distributed MinIO should share a common root credentials, for the nodes to connect and trust each other. To achieve this, it is **recommended** to export root user and root password as environment variables, `MINIO_ROOT_USER` and `MINIO_ROOT_PASSWORD`, on all the nodes before executing MinIO server command. If not exported, default `minioadmin/minioadmin` credentials shall be used.
- **MinIO creates erasure-coding sets of _4_ to _16_ drives per set. The number of drives you provide in total must be a multiple of one of those numbers.**
- **MinIO chooses the largest EC set size which divides into the total number of drives or total number of nodes given - making sure to keep the uniform distribution i.e each node participates equal number of drives per set**.
- **Each object is written to a single EC set, and therefore is spread over no more than 16 drives.**
- **All the nodes running distributed MinIO setup are recommended to be homogeneous, i.e. same operating system, same number of disks and same network interconnects.**
- MinIO distributed mode requires **fresh directories**. If required, the drives can be shared with other applications. You can do this by using a sub-directory exclusive to MinIO. For example, if you have mounted your volume under `/export`, pass `/export/data` as arguments to MinIO server.
- The IP addresses and drive paths below are for demonstration purposes only, you need to replace these with the actual IP addresses and drive paths/folders.
- Servers running distributed MinIO instances should be less than 15 minutes apart. You can enable [NTP](http://www.ntp.org/) service as a best practice to ensure same times across servers.
- `MINIO_DOMAIN` environment variable should be defined and exported for bucket DNS style support.
- Running Distributed MinIO on __Windows__ operating system is considered **experimental**. Please proceed with caution.
- Running Distributed MinIO on **Windows** operating system is considered **experimental**. Please proceed with caution.
Example 1: Start distributed MinIO instance on n nodes with m drives each mounted at `/export1` to `/exportm` (pictured below), by running this command on all the n nodes:
![Distributed MinIO, n nodes with m drives each](https://github.com/minio/minio/blob/master/docs/screenshots/Architecture-diagram_distributed_nm.png?raw=true)
#### GNU/Linux and macOS
### GNU/Linux and macOS
```sh
export MINIO_ROOT_USER=<ACCESS_KEY>
@ -63,11 +63,11 @@ export MINIO_ROOT_PASSWORD=<SECRET_KEY>
minio server http://host{1...n}/export{1...m}
```
> __NOTE:__ In above example `n` and `m` represent positive integers, *do not copy paste and expect it work make the changes according to local deployment and setup*.
> **NOTE:** In above example `n` and `m` represent positive integers, _do not copy paste and expect it work make the changes according to local deployment and setup_.
> **NOTE:** `{1...n}` shown have 3 dots! Using only 2 dots `{1..n}` will be interpreted by your shell and won't be passed to MinIO server, affecting the erasure coding order, which would impact performance and high availability. **Always use ellipses syntax `{1...n}` (3 dots!) for optimal erasure-code distribution**
> __NOTE:__ `{1...n}` shown have 3 dots! Using only 2 dots `{1..n}` will be interpreted by your shell and won't be passed to MinIO server, affecting the erasure coding order, which would impact performance and high availability. __Always use ellipses syntax `{1...n}` (3 dots!) for optimal erasure-code distribution__
### Expanding existing distributed setup
#### Expanding existing distributed setup
MinIO supports expanding distributed erasure coded clusters by specifying new set of clusters on the command-line as shown below:
```sh
@ -77,18 +77,21 @@ minio server http://host{1...n}/export{1...m} http://host{o...z}/export{1...m}
```
For example:
```
minio server http://host{1...4}/export{1...16} http://host{5...12}/export{1...16}
```
Now the server has expanded total storage by _(newly_added_servers\*m)_ more disks, taking the total count to _(existing_servers\*m)+(newly_added_servers\*m)_ disks. New object upload requests automatically start using the least used cluster. This expansion strategy works endlessly, so you can perpetually expand your clusters as needed. When you restart, it is immediate and non-disruptive to the applications. Each group of servers in the command-line is called a pool. There are 2 server pools in this example. New objects are placed in server pools in proportion to the amount of free space in each pool. Within each pool, the location of the erasure-set of drives is determined based on a deterministic hashing algorithm.
> __NOTE:__ __Each pool you add must have the same erasure coding parity configuration as the original pool, so the same data redundancy SLA is maintained.__
> **NOTE:** **Each pool you add must have the same erasure coding parity configuration as the original pool, so the same data redundancy SLA is maintained.**
## 3. Test your setup
To test this setup, access the MinIO server via browser or [`mc`](https://docs.min.io/docs/minio-client-quickstart-guide).
## Explore Further
- [MinIO Erasure Code QuickStart Guide](https://docs.min.io/docs/minio-erasure-code-quickstart-guide)
- [Use `mc` with MinIO Server](https://docs.min.io/docs/minio-client-quickstart-guide)
- [Use `aws-cli` with MinIO Server](https://docs.min.io/docs/aws-cli-with-minio)

8
docs/distributed/SIZING.md
View File

@ -1,6 +1,6 @@
## Erasure code sizing guide
# Erasure code sizing guide
### Toy Setups
## Toy Setups
Capacity constrained environments, MinIO will work but not recommended for production.
@ -13,7 +13,7 @@ Capacity constrained environments, MinIO will work but not recommended for produ
| 6 | 1 | 6 | 3 | 3 | 2 |
| 7 | 1 | 7 | 3 | 3 | 3 |
### Minimum System Configuration for Production
## Minimum System Configuration for Production
| servers | drives (per node) | stripe_size | parity chosen (default) | tolerance for reads (servers) | tolerance for writes (servers) |
|--------:|------------------:|------------:|------------------------:|------------------------------:|-------------------------------:|
@ -32,7 +32,7 @@ Capacity constrained environments, MinIO will work but not recommended for produ
| 15 | 2 | 15 | 4 | 4 | 4 |
| 16 | 2 | 16 | 4 | 4 | 4 |
If one or more disks are offline at the start of a PutObject or NewMultipartUpload operation the object will have additional data
If one or more disks are offline at the start of a PutObject or NewMultipartUpload operation the object will have additional data
protection bits added automatically to provide the regular safety for these objects up to 50% of the number of disks.
This will allow normal write operations to take place on systems that exceed the write tolerance.

30
docs/docker/README.md
View File

@ -3,9 +3,10 @@
See our web documentation on [Deploying MinIO in Standalone Mode](Deploy Standalone MinIO in a Container) for a more structured tutorial on deploying MinIO in a container.
## Prerequisites
Docker installed on your machine. Download the relevant installer from [here](https://www.docker.com/community-edition#/download).
## Run Standalone MinIO on Docker.
## Run Standalone MinIO on Docker
*Note*: Standalone MinIO is intended for early development and evaluation. For production clusters, deploy a [Distributed](https://docs.min.io/minio/baremetal/installation/deployment-and-management.html) MinIO deployment.
@ -22,7 +23,8 @@ docker run \
To create a MinIO container with persistent storage, you need to map local persistent directories from the host OS to virtual config. To do this, run the below commands
#### GNU/Linux and macOS
### GNU/Linux and macOS
```sh
mkdir -p ~/minio/data
@ -38,7 +40,8 @@ docker run \
The command creates a new local directory `~/minio/data` in your user home directory. It then starts the MinIO container with the `-v` argument to map the local path (`~/minio/data`) to the specified virtual container directory (`/data`). When MinIO writes data to `/data`, that data is actually written to the local path `~/minio/data` where it can persist between container restarts.
#### Windows
### Windows
```sh
docker run \
-p 9000:9000 \
@ -51,14 +54,17 @@ docker run \
```
## Run Distributed MinIO on Docker
Distributed MinIO can be deployed via [Docker Compose](https://docs.min.io/docs/deploy-minio-on-docker-compose). This means Docker Compose lets you quickly get started with Distributed MinIO on your computer - ideal for development, testing, staging environments. We recommend kubernetes based deployment for production level deployment https://github.com/minio/operator.
Distributed MinIO can be deployed via [Docker Compose](https://docs.min.io/docs/deploy-minio-on-docker-compose). This means Docker Compose lets you quickly get started with Distributed MinIO on your computer - ideal for development, testing, staging environments. We recommend kubernetes based deployment for production level deployment <https://github.com/minio/operator>.
## MinIO Docker Tips
### MinIO Custom Access and Secret Keys
To override MinIO's auto-generated keys, you may pass secret and access keys explicitly as environment variables. MinIO server also allows regular strings as access and secret keys.
#### GNU/Linux and macOS
```sh
docker run \
-p 9000:9000 \
@ -71,6 +77,7 @@ docker run \
```
#### Windows
```powershell
docker run \
-p 9000:9000 \
@ -83,12 +90,15 @@ docker run \
```
### Run MinIO Docker as a regular user
Docker provides standardized mechanisms to run docker containers as non-root users.
#### GNU/Linux and macOS
On Linux and macOS you can use `--user` to run the container as regular user.
> NOTE: make sure --user has write permission to *${HOME}/data* prior to using `--user`.
```sh
mkdir -p ${HOME}/data
docker run \
@ -103,6 +113,7 @@ docker run \
```
#### Windows
On windows you would need to use [Docker integrated windows authentication](https://success.docker.com/article/modernizing-traditional-dot-net-applications#integratedwindowsauthentication) and [Create a container with Active Directory Support](https://blogs.msdn.microsoft.com/containerstuff/2017/01/30/create-a-container-with-active-directory-support/)
> NOTE: make sure your AD/Windows user has write permissions to *D:\data* prior to using `credentialspec=`.
@ -120,6 +131,7 @@ docker run \
```
### MinIO Custom Access and Secret Keys using Docker secrets
To override MinIO's auto-generated keys, you may pass secret and access keys explicitly by creating access and secret keys as [Docker secrets](https://docs.docker.com/engine/swarm/secrets/). MinIO server also allows regular strings as access and secret keys.
```
@ -128,6 +140,7 @@ echo "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY" | docker secret create secret_ke
```
Create a MinIO service using `docker service` to read from Docker secrets.
```
docker service create --name="minio-service" --secret="access_key" --secret="secret_key" quay.io/minio/minio server /data
```
@ -135,7 +148,9 @@ docker service create --name="minio-service" --secret="access_key" --secret="sec
Read more about `docker service` [here](https://docs.docker.com/engine/swarm/how-swarm-mode-works/services/)
#### MinIO Custom Access and Secret Key files
To use other secret names follow the instructions above and replace `access_key` and `secret_key` with your custom names (e.g. `my_secret_key`,`my_custom_key`). Run your service with
```
docker service create --name="minio-service" \
--secret="my_access_key" \
@ -144,13 +159,16 @@ docker service create --name="minio-service" \
--env="MINIO_ROOT_PASSWORD_FILE=my_secret_key" \
quay.io/minio/minio server /data
```
`MINIO_ROOT_USER_FILE` and `MINIO_ROOT_PASSWORD_FILE` also support custom absolute paths, in case Docker secrets are mounted to custom locations or other tools are used to mount secrets into the container. For example, HashiCorp Vault injects secrets to `/vault/secrets`. With the custom names above, set the environment variables to
```
MINIO_ROOT_USER_FILE=/vault/secrets/my_access_key
MINIO_ROOT_PASSWORD_FILE=/vault/secrets/my_secret_key
```
### Retrieving Container ID
To use Docker commands on a specific container, you need to know the `Container ID` for that container. To get the `Container ID`, run
```sh
@ -160,6 +178,7 @@ docker ps -a
`-a` flag makes sure you get all the containers (Created, Running, Exited). Then identify the `Container ID` from the output.
### Starting and Stopping Containers
To start a stopped container, you can use the [`docker start`](https://docs.docker.com/engine/reference/commandline/start/) command.
```sh
@ -167,11 +186,13 @@ docker start <container_id>
```
To stop a running container, you can use the [`docker stop`](https://docs.docker.com/engine/reference/commandline/stop/) command.
```sh
docker stop <container_id>
```
### MinIO container logs
To access MinIO logs, you can use the [`docker logs`](https://docs.docker.com/engine/reference/commandline/logs/) command.
```sh
@ -179,6 +200,7 @@ docker logs <container_id>
```
### Monitor MinIO Docker Container
To monitor the resources used by MinIO container, you can use the [`docker stats`](https://docs.docker.com/engine/reference/commandline/stats/) command.
```sh

12
docs/erasure/storage-class/README.md
View File

@ -2,7 +2,7 @@
MinIO server supports storage class in erasure coding mode. This allows configurable data and parity disks per object.
This page is intended as a summary of MinIO Erasure Coding. For a more complete explanation, see https://docs.min.io/minio/baremetal/concepts/erasure-coding.html.
This page is intended as a summary of MinIO Erasure Coding. For a more complete explanation, see <https://docs.min.io/minio/baremetal/concepts/erasure-coding.html>.
## Overview
@ -85,7 +85,7 @@ export MINIO_STORAGE_CLASS_RRS=EC:2
Storage class can also be set via `mc admin config` get/set commands to update the configuration. Refer [storage class](https://github.com/minio/minio/tree/master/docs/config#storage-class) for
more details.
*Note*
#### Note
- If `STANDARD` storage class is set via environment variables or `mc admin config` get/set commands, and `x-amz-storage-class` is not present in request metadata, MinIO server will
apply `STANDARD` storage class to the object. This means the data and parity disks will be used as set in `STANDARD` storage class.
@ -100,22 +100,22 @@ In below example `minio-go` is used to set the storage class to `REDUCED_REDUNDA
```go
s3Client, err := minio.New("localhost:9000", "YOUR-ACCESSKEYID", "YOUR-SECRETACCESSKEY", true)
if err != nil {
log.Fatalln(err)
log.Fatalln(err)
}
object, err := os.Open("my-testfile")
if err != nil {
log.Fatalln(err)
log.Fatalln(err)
}
defer object.Close()
objectStat, err := object.Stat()
if err != nil {
log.Fatalln(err)
log.Fatalln(err)
}
n, err := s3Client.PutObject("my-bucketname", "my-objectname", object, objectStat.Size(), minio.PutObjectOptions{ContentType: "application/octet-stream", StorageClass: "REDUCED_REDUNDANCY"})
if err != nil {
log.Fatalln(err)
log.Fatalln(err)
}
log.Println("Uploaded", "my-objectname", " of size: ", n, "Successfully.")
```

18
docs/extensions/s3zip/README.md
View File

@ -1,40 +1,40 @@
# Perform S3 operations in a ZIP content[![Slack](https://slack.min.io/slack?type=svg)](https://slack.min.io) [![Docker Pulls](https://img.shields.io/docker/pulls/minio/minio.svg?maxAge=604800)](https://hub.docker.com/r/minio/minio/)
### Overview
## Overview
MinIO implements an S3 extension to list, stat and download files inside a ZIP file stored in any bucket. A perfect use case scenario is when you have a lot of small files archived in multiple ZIP files. Uploading them is faster than uploading small files individually. Besides, your S3 applications will be able to access to the data with little performance overhead.
The main limitation is that to update or delete content of a file inside a ZIP file the entire ZIP file must be replaced.
### How to enable S3 ZIP behavior ?
## How to enable S3 ZIP behavior ?
Ensure to set the following header `x-minio-extract` to `true` in your S3 requests.
### How to access to files inside a ZIP archive
## How to access to files inside a ZIP archive
Accessing to contents inside an archive can be done using regular S3 API with a modified request path. You just need to append the path of the content inside the archive to the path of the archive itself.
e.g.:
To download `2021/taxes.csv` archived in `financial.zip` and stored under a bucket named `company-data`, you can issue a GET request using the following path 'company-data/financial.zip/2021/taxes.csv`
### Contents properties
## Contents properties
All properties except the file size are tied to the zip file. This means that modification date, headers, tags, etc. can only be set for the zip file as a whole. In similar fashion, replication will replicate the zip file as a whole and not individual files.
### Code Examples
## Code Examples
[Using minio-go library](https://github.com/minio/minio/blob/master/docs/extensions/s3zip/examples/minio-go/main.go)
[Using AWS JS SDK v2](https://github.com/minio/minio/blob/master/docs/extensions/s3zip/examples/aws-js/main.js)
[Using boto3](https://github.com/minio/minio/blob/master/docs/extensions/s3zip/examples/boto3/main.py)
## Requirements and limits
### Requirements and limits
- ListObjectsV2 can only list the most recent ZIP archive version of your object, applicable only for versioned buckets.
- ListObjectsV2 API calls must be used to list zip file content.
- Names inside ZIP files are kept unmodified, but some may lead to invalid paths. See [Object key naming guidelines](https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-keys.html) on safe names.
- This API behavior is limited for following **read** operations on files inside a zip archive:
- `HeadObject`
- `GetObject`
- `ListObjectsV2`
- `HeadObject`
- `GetObject`
- `ListObjectsV2`
- A maximum of 100,000 files inside a single ZIP archive is recommended for best performance and memory usage trade-off.
- If the ZIP file directory isn't located within the last 100MB the file will not be parsed.

7
docs/federation/lookup/README.md
View File

@ -1,14 +1,15 @@
*Federation feature is deprecated and should be avoided for future deployments*
# Federation Quickstart Guide [![Slack](https://slack.min.io/slack?type=svg)](https://slack.min.io) *Federation feature is deprecated and should be avoided for future deployments*
# Federation Quickstart Guide [![Slack](https://slack.min.io/slack?type=svg)](https://slack.min.io)
This document explains how to configure MinIO with `Bucket lookup from DNS` style federation.
## Get started
### 1. Prerequisites
Install MinIO - [MinIO Quickstart Guide](https://docs.min.io/docs/minio-quickstart-guide).
### 2. Run MinIO in federated mode
Bucket lookup from DNS federation requires two dependencies
- etcd (for bucket DNS service records)
@ -39,8 +40,6 @@ This is comma separated list of IP addresses to which buckets created on this Mi
a bucket `bucket1` created on current MinIO instance will be accessible as `bucket1.domain.com`, and the DNS entry for
`bucket1.domain.com` will point to IP address set in `MINIO_PUBLIC_IPS`.
*Note*
- This field is mandatory for standalone and erasure code MinIO server deployments, to enable federated mode.
- This field is optional for distributed deployments. If you don't set this field in a federated setup, we use the IP addresses of
hosts passed to the MinIO server startup and use them for DNS entries.

4
docs/gateway/README.md
View File

@ -1,9 +1,11 @@
# MinIO Gateway [![Slack](https://slack.min.io/slack?type=svg)](https://slack.min.io)
## Support
Gateway implementations are frozen and are not accepting any new features. Please reports any bugs at https://github.com/minio/minio/issues . If you are an existing customer please login to https://subnet.min.io for production support.
Gateway implementations are frozen and are not accepting any new features. Please reports any bugs at <https://github.com/minio/minio/issues> . If you are an existing customer please login to <https://subnet.min.io> for production support.
## Implementations
MinIO Gateway adds Amazon S3 compatibility layer to third party NAS and Cloud Storage vendors. MinIO Gateway is implemented to facilitate migration of existing from your existing legacy or cloud vendors to MinIO distributed server deployments.
- [NAS](https://github.com/minio/minio/blob/master/docs/gateway/nas.md)

17
docs/gateway/azure.md
View File

@ -1,8 +1,15 @@
# MinIO Azure Gateway [![Slack](https://slack.min.io/slack?type=svg)](https://slack.min.io)
MinIO Gateway adds Amazon S3 compatibility to Microsoft Azure Blob Storage.
## Support
Gateway implementations are frozen and are not accepting any new features. Please reports any bugs at <https://github.com/minio/minio/issues> . If you are an existing customer please login to <https://subnet.min.io> for production support.
## Run MinIO Gateway for Microsoft Azure Blob Storage
### Using Docker
```
podman run \
-p 9000:9000 \
@ -14,27 +21,33 @@ podman run \
```
### Using Binary
```
export MINIO_ROOT_USER=azureaccountname
export MINIO_ROOT_PASSWORD=azureaccountkey
minio gateway azure
```
## Test using MinIO Console
MinIO Gateway comes with an embedded web based object browser. Point your web browser to http://127.0.0.1:9000 to ensure that your server has started successfully.
MinIO Gateway comes with an embedded web based object browser. Point your web browser to <http://127.0.0.1:9000> to ensure that your server has started successfully.
| Dashboard | Creating a bucket |
| ------------- | ------------- |
| ![Dashboard](https://github.com/minio/minio/blob/master/docs/screenshots/pic1.png?raw=true) | ![Dashboard](https://github.com/minio/minio/blob/master/docs/screenshots/pic2.png?raw=true) |
## Test using MinIO Client `mc`
`mc` provides a modern alternative to UNIX commands such as ls, cat, cp, mirror, diff etc. It supports filesystems and Amazon S3 compatible cloud storage services.
### Configure `mc`
```
mc alias set myazure http://gateway-ip:9000 azureaccountname azureaccountkey
```
### List containers on Microsoft Azure
```
mc ls myazure
[2017-02-22 01:50:43 PST] 0B ferenginar/
@ -47,6 +60,7 @@ mc ls myazure
If you do not want to share the credentials of the Azure blob storage with your users/applications, you can set the original credentials in the shell environment using `AZURE_STORAGE_ACCOUNT` and `AZURE_STORAGE_KEY` variables and assign different access/secret keys to `MINIO_ROOT_USER` and `MINIO_ROOT_PASSWORD`.
### Known limitations
Gateway inherits the following Azure limitations:
- Only read-only bucket policy supported at bucket level, all other variations will return API Notimplemented error.
@ -59,6 +73,7 @@ Other limitations:
- Bucket notification APIs are not supported.
## Explore Further
- [`mc` command-line interface](https://docs.min.io/docs/minio-client-quickstart-guide)
- [`aws` command-line interface](https://docs.min.io/docs/aws-cli-with-minio)
- [`minio-go` Go SDK](https://docs.min.io/docs/golang-client-quickstart-guide)

18
docs/gateway/gcs.md
View File

@ -2,13 +2,14 @@
MinIO GCS Gateway allows you to access Google Cloud Storage (GCS) with Amazon S3-compatible APIs
- [Run MinIO Gateway for GCS](#run-minio-gateway-for-gcs)
- [Test Using MinIO Console](#test-using-minio-browser)
- [Test Using MinIO Client](#test-using-minio-client)
## Support
Gateway implementations are frozen and are not accepting any new features. Please reports any bugs at <https://github.com/minio/minio/issues> . If you are an existing customer please login to <https://subnet.min.io> for production support.
## 1. Run MinIO Gateway for GCS
### 1.1 Create a Service Account key for GCS and get the Credentials File
1. Navigate to the [API Console Credentials page](https://console.developers.google.com/project/_/apis/credentials).
2. Select a project or create a new project. Note the project ID.
3. Select the **Create credentials** dropdown on the **Credentials** page, and click **Service account key**.
@ -20,6 +21,7 @@ MinIO GCS Gateway allows you to access Google Cloud Storage (GCS) with Amazon S3
**Note:** For alternate ways to set up *Application Default Credentials*, see [Setting Up Authentication for Server to Server Production Applications](https://developers.google.com/identity/protocols/application-default-credentials).
### 1.2 Run MinIO GCS Gateway Using Docker
```sh
podman run \
-p 9000:9000 \
@ -43,7 +45,7 @@ minio gateway gcs yourprojectid
## 2. Test Using MinIO Console
MinIO Gateway comes with an embedded web-based object browser that outputs content to http://127.0.0.1:9000. To test that MinIO Gateway is running, open a web browser, navigate to http://127.0.0.1:9000, and ensure that the object browser is displayed.
MinIO Gateway comes with an embedded web-based object browser that outputs content to <http://127.0.0.1:9000>. To test that MinIO Gateway is running, open a web browser, navigate to <http://127.0.0.1:9000>, and ensure that the object browser is displayed.
| Dashboard | Creating a bucket |
| ------------- | ------------- |
@ -78,6 +80,7 @@ A response similar to this one should be displayed:
```
### 3.3 Known limitations
MinIO Gateway has the following limitations when used with GCS:
* It only supports read-only and write-only bucket policies at the bucket level; all other variations will return `API Not implemented`.
@ -88,6 +91,7 @@ Other limitations:
* Bucket notification APIs are not supported.
## 4. Explore Further
- [`mc` command-line interface](https://docs.min.io/docs/minio-client-quickstart-guide)
- [`aws` command-line interface](https://docs.min.io/docs/aws-cli-with-minio)
- [`minio-go` Go SDK](https://docs.min.io/docs/golang-client-quickstart-guide)
* [`mc` command-line interface](https://docs.min.io/docs/minio-client-quickstart-guide)
* [`aws` command-line interface](https://docs.min.io/docs/aws-cli-with-minio)
* [`minio-go` Go SDK](https://docs.min.io/docs/golang-client-quickstart-guide)

18
docs/gateway/hdfs.md
View File

@ -1,12 +1,19 @@
# MinIO HDFS Gateway [![Slack](https://slack.minio.io/slack?type=svg)](https://slack.minio.io)
MinIO HDFS gateway adds Amazon S3 API support to Hadoop HDFS filesystem. Applications can use both the S3 and file APIs concurrently without requiring any data migration. Since the gateway is stateless and shared-nothing, you may elastically provision as many MinIO instances as needed to distribute the load.
> NOTE: Intention of this gateway implementation it to make it easy to migrate your existing data on HDFS clusters to MinIO clusters using standard tools like `mc` or `aws-cli`, if the goal is to use HDFS perpetually we recommend that HDFS should be used directly for all write operations.
## Support
Gateway implementations are frozen and are not accepting any new features. Please reports any bugs at <https://github.com/minio/minio/issues> . If you are an existing customer please login to <https://subnet.min.io> for production support.
## Run MinIO Gateway for HDFS Storage
### Using Binary
Namenode information is obtained by reading `core-site.xml` automatically from your hadoop environment variables *$HADOOP_HOME*
```
export MINIO_ROOT_USER=minio
export MINIO_ROOT_PASSWORD=minio123
@ -14,6 +21,7 @@ minio gateway hdfs
```
You can also override the namenode endpoint as shown below.
```
export MINIO_ROOT_USER=minio
export MINIO_ROOT_PASSWORD=minio123
@ -21,7 +29,9 @@ minio gateway hdfs hdfs://namenode:8200
```
### Using Docker
Using docker is experimental, most Hadoop environments are not dockerized and may require additional steps in getting this to work properly. You are better off just using the binary in this situation.
```
podman run \
-p 9000:9000 \
@ -46,12 +56,14 @@ To enable kerberos authentication, you need to set `hadoop.security.authenticati
```
MinIO will load `krb5.conf` from environment variable `KRB5_CONFIG` or default location `/etc/krb5.conf`.
```sh
export KRB5_CONFIG=/path/to/krb5.conf
```
If you want MinIO to use ccache for authentication, set environment variable `KRB5CCNAME` to the credential cache file path,
or MinIO will use the default location `/tmp/krb5cc_%{uid}`.
```sh
export KRB5CCNAME=/path/to/krb5cc
```
@ -71,7 +83,8 @@ export KRB5REALM=REALM.COM
```
## Test using MinIO Console
*MinIO gateway* comes with an embedded web based object browser. Point your web browser to http://127.0.0.1:9000 to ensure that your server has started successfully.
*MinIO gateway* comes with an embedded web based object browser. Point your web browser to <http://127.0.0.1:9000> to ensure that your server has started successfully.
| Dashboard | Creating a bucket |
| ------------- | ------------- |
@ -97,7 +110,9 @@ mc ls myhdfs
```
### Known limitations
Gateway inherits the following limitations of HDFS storage layer:
- No bucket policy support (HDFS has no such concept)
- No bucket notification APIs are not supported (HDFS has no support for fsnotify)
- No server side encryption support (Intentionally not implemented)
@ -105,6 +120,7 @@ Gateway inherits the following limitations of HDFS storage layer:
- Concurrent multipart operations are not supported (HDFS lacks safe locking support, or poorly implemented)
## Explore Further
- [`mc` command-line interface](https://docs.minio.io/docs/minio-client-quickstart-guide)
- [`aws` command-line interface](https://docs.minio.io/docs/aws-cli-with-minio)
- [`minio-go` Go SDK](https://docs.minio.io/docs/golang-client-quickstart-guide)

8
docs/gateway/nas.md
View File

@ -2,6 +2,10 @@
MinIO Gateway adds Amazon S3 compatibility to NAS storage. You may run multiple minio instances on the same shared NAS volume as a distributed object gateway.
## Support
Gateway implementations are frozen and are not accepting any new features. Please reports any bugs at <https://github.com/minio/minio/issues> . If you are an existing customer please login to <https://subnet.min.io> for production support.
## Run MinIO Gateway for NAS Storage
### Using Docker
@ -29,7 +33,7 @@ minio gateway nas /shared/nasvol
## Test using MinIO Console
MinIO Gateway comes with an embedded web based object browser. Point your web browser to http://127.0.0.1:9000 to ensure that your server has started successfully.
MinIO Gateway comes with an embedded web based object browser. Point your web browser to <http://127.0.0.1:9000> to ensure that your server has started successfully.
| Dashboard | Creating a bucket |
| ------------- | ------------- |
@ -96,12 +100,14 @@ NAS gateway implementation allows symlinks on regular files,
- Delete of actual file automatically makes symlink'ed file invisible, dangling symlinks won't be visible.
#### Caveats
- Disallows follow of directory symlinks to avoid security issues, and leaving them as is on namespace makes them very inconsistent.
- Dangling symlinks are ignored automatically.
*Directory symlinks is not and will not be supported as there are no safe ways to handle them.*
## Explore Further
- [`mc` command-line interface](https://docs.min.io/docs/minio-client-quickstart-guide)
- [`aws` command-line interface](https://docs.min.io/docs/aws-cli-with-minio)
- [`minio-go` Go SDK](https://docs.min.io/docs/golang-client-quickstart-guide)

18
docs/gateway/s3.md
View File

@ -2,10 +2,16 @@
MinIO S3 Gateway adds MinIO features like MinIO Console and disk caching to AWS S3 or any other AWS S3 compatible service.
## Support
Gateway implementations are frozen and are not accepting any new features. Please reports any bugs at <https://github.com/minio/minio/issues> . If you are an existing customer please login to <https://subnet.min.io> for production support.
## Run MinIO Gateway for AWS S3
As a prerequisite to run MinIO S3 gateway, you need valid AWS S3 access key and secret key by default. Optionally you can also set custom access/secret key, when you have rotating AWS IAM credentials or AWS credentials through environment variables (i.e. AWS_ACCESS_KEY_ID)
### Using Docker
```
podman run \
-p 9000:9000 \
@ -17,6 +23,7 @@ podman run \
```
### Using Binary
```
export MINIO_ROOT_USER=aws_s3_access_key
export MINIO_ROOT_PASSWORD=aws_s3_secret_key
@ -24,9 +31,11 @@ minio gateway s3
```
### Using Binary in EC2
Using IAM rotating credentials for AWS S3
If you are using an S3 enabled IAM role on an EC2 instance for S3 access, MinIO will still require env vars MINIO_ROOT_USER and MINIO_ROOT_PASSWORD to be set for its internal use. These may be set to any value which meets the length requirements. Access key length should be at least 3, and secret key length at least 8 characters.
```
export MINIO_ROOT_USER=custom_access_key
export MINIO_ROOT_PASSWORD=custom_secret_key
@ -73,9 +82,11 @@ Minimum permissions required if you wish to provide restricted access with your
```
## Run MinIO Gateway for AWS S3 compatible services
As a prerequisite to run MinIO S3 gateway on an AWS S3 compatible service, you need valid access key, secret key and service endpoint.
## Run MinIO Gateway with double-encryption
MinIO gateway to S3 supports encryption of data at rest. Three types of encryption modes are supported
- encryption can be set to ``pass-through`` to backend only for SSE-S3, SSE-C is not allowed passthrough.
@ -87,7 +98,6 @@ This can be specified by setting MINIO_GATEWAY_SSE environment variable. If MINI
To specify ``double encryption``, MINIO_GATEWAY_SSE environment variable needs to be set to "s3" for sse-s3
and "c" for sse-c encryption. More than one encryption option can be set, delimited by ";". Objects are encrypted at the gateway and the gateway also does a pass-through to backend. Note that in the case of SSE-C encryption, gateway derives a unique SSE-C key for pass through from the SSE-C client key using a key derivation function (KDF).
```sh
curl -sSL --tlsv1.2 \
-O 'https://raw.githubusercontent.com/minio/kes/master/root.key' \
@ -104,6 +114,7 @@ minio gateway s3
```
### Using Docker
```
podman run -p 9000:9000 --name minio-s3 \
-e "MINIO_ROOT_USER=access_key" \
@ -112,6 +123,7 @@ podman run -p 9000:9000 --name minio-s3 \
```
### Using Binary
```
export MINIO_ROOT_USER=access_key
export MINIO_ROOT_PASSWORD=secret_key
@ -119,6 +131,7 @@ minio gateway s3 https://s3_compatible_service_endpoint:port
```
## MinIO Caching
MinIO edge caching allows storing content closer to the applications. Frequently accessed objects are stored in a local disk based cache. Edge caching with MinIO gateway feature allows
- Dramatic improvements for time to first byte for any object.
@ -127,7 +140,8 @@ MinIO edge caching allows storing content closer to the applications. Frequently
Refer [this document](https://docs.min.io/docs/minio-disk-cache-guide.html) to get started with MinIO Caching.
## MinIO Console
MinIO Gateway comes with an embedded web based object browser. Point your web browser to http://127.0.0.1:9000 to ensure that your server has started successfully.
MinIO Gateway comes with an embedded web based object browser. Point your web browser to <http://127.0.0.1:9000> to ensure that your server has started successfully.
| Dashboard | Creating a bucket |
| ------------- | ------------- |

17
docs/hotfixes.md
View File

@ -1,4 +1,5 @@
# Introduction
This document outlines how to make hotfix binaries and containers for MinIO?. The main focus in this article is about how to backport patches to a specific branch and finally building binaries/containers.
## Pre-pre requisite
@ -7,6 +8,7 @@ This document outlines how to make hotfix binaries and containers for MinIO?. Th
- A working knowledge of AWS S3 API behaviors and corner cases.
## Pre-requisite for backporting any fixes
Fixes that are allowed a backport must satisfy any of the following criteria's:
- A fix must not be a feature, for example.
@ -50,16 +52,20 @@ Date: Sat Jun 15 11:27:17 2019 -0700
- There is always a possibility of a fix that is new, it is advised that the developer must make sure that the fix is sent upstream, reviewed and merged to the master branch.
## Creating a hotfix branch
Customers in MinIO are allowed LTS on any release they choose to standardize. Production setups seldom change and require maintenance. Hotfix branches are such maintenance branches that allow customers to operate a production cluster without drastic changes to their deployment.
## Backporting a fix
Developer is advised to clone the MinIO source and checkout the MinIO release tag customer is currently on.
```
λ git checkout RELEASE.2021-04-22T15-44-28Z
```
Create a branch and proceed to push the branch **upstream**
> (upstream here points to git@github.com:minio/minio.git)
```
λ git branch -m RELEASE.2021-04-22T15-44-28Z.hotfix
λ git push -u upstream RELEASE.2021-04-22T15-44-28Z.hotfix
@ -87,16 +93,19 @@ Date: Mon Nov 8 08:41:27 2021 -0800
Once the **patch** is successfully applied, developer must run tests to alidate the fix that was backported by running following tests, locally.
Unit tests
```
λ make test
```
Verify different type of MinIO deployments work
```
λ make verify
```
Verify if healing and replacing a drive works
```
λ make verify-healing
```
@ -104,17 +113,19 @@ Verify if healing and replacing a drive works
At this point in time the backport is ready to be submitted as a pull request to the relevant branch. A pull request is recommended to ensure [mint](http://github.com/minio/mint) tests are validated. Pull request also ensures code-reviews for the backports incase of any unforeseen regressions.
### Building a hotfix binary and container
To add a hotfix tag to the binary version and embed the relevant
`commit-id` following build helpers are available
To add a hotfix tag to the binary version and embed the relevant `commit-id` following build helpers are available
#### Builds the hotfix binary
```
λ CRED_DIR=/media/builder/minio make hotfix
```
#### Builds the hotfix container
```
λ CRED_DIR=/media/builder/minio make docker-hotfix
```
Once this has been provided to the customer relevant binary will be uploaded from our *release server* securely, directly to https://dl.minio.io/server/minio/hotfixes/archive/
Once this has been provided to the customer relevant binary will be uploaded from our *release server* securely, directly to <https://dl.minio.io/server/minio/hotfixes/archive/>

19
docs/integrations/veeam/README.md
View File

@ -2,15 +2,18 @@
When using Veeam Backup and Replication, you can use S3 compatible object storage such as MinIO as a capacity tier for backups. This disaggregates storage for the Veeam infrastructure and allows you to retain control of your data. With the ease of use of setup and administration of MinIO, it allows a Veeam backup admin to easily deploy their own object store for capacity tiering.
__Prerequisites__
## Prerequisites
- One or both of Veeam Backup and Replication with support for S3 compatible object store (e.g. 9.5.4) and Veeam Backup for Office365 (VBO)
- MinIO object storage set up per https://docs.min.io/docs/minio-quickstart-guide.html
- Veeam requires TLS connections to the object storage. This can be configured per https://docs.minio.io/docs/how-to-secure-access-to-minio-server-with-tls.html
- MinIO object storage set up per <https://docs.min.io/docs/minio-quickstart-guide.html>
- Veeam requires TLS connections to the object storage. This can be configured per <https://docs.minio.io/docs/how-to-secure-access-to-minio-server-with-tls.html>
- The S3 bucket, Access Key and Secret Key have to be created before and outside of Veeam.
- Configure the minio client for the Veeam MinIO endpoint - https://docs.min.io/docs/minio-client-quickstart-guide.html
- Configure the minio client for the Veeam MinIO endpoint - <https://docs.min.io/docs/minio-client-quickstart-guide.html>
## Setting up an S3 compatible object store for Veeam Backup and Replication
### Create a bucket for Veeam backups
Create a bucket for Veeam Backup, e.g.,
```
@ -23,10 +26,11 @@ mc mb myminio/veeambackup
mc mb -l myminio/veeambackup
```
> Object locking requires erasure coding enabled on the minio server. For more information see https://docs.minio.io/docs/minio-erasure-code-quickstart-guide.html.
> Object locking requires erasure coding enabled on the minio server. For more information see <https://docs.minio.io/docs/minio-erasure-code-quickstart-guide.html>.
### Add MinIO as an object store for Veeam
Follow the steps from the Veeam documentation for adding MinIO as an object store - https://helpcenter.veeam.com/docs/backup/vsphere/adding_s3c_object_storage.html?ver=100
Follow the steps from the Veeam documentation for adding MinIO as an object store - <https://helpcenter.veeam.com/docs/backup/vsphere/adding_s3c_object_storage.html?ver=100>
For Veeam Backup with Immutability, choose the amount of days you want to make backups immutable for
@ -41,6 +45,7 @@ For Veeam Backup with Immutability, choose the amount of days you want to make b
- On the Capacity Tier screen, check the box to Extend scale-out backup repository capacity with object storage checkbox and select the object storage. If you want to be able to test backup data immediately after a job is run, under the object storage selection, check the "Copy" box and uncheck the "Move" box.
### Create a backup job
#### Backup Virtual Machines with Veeam Backup and Replication
- Under Home > Jobs > Backup in Navigation Pane, click on Backup Job button in the ribbon and choose Virtual Machine. Follow the on screen wizard.
@ -56,6 +61,7 @@ For Veeam Backup with Immutability, choose the amount of days you want to make b
![Choose Immutability Options for Backups](https://raw.githubusercontent.com/minio/minio/master/docs/integrations/veeam/screenshots/backup_job_immutable_days.png)
#### Backup Office 365 with VBO
- Create a new bucket for VBO backups
```
@ -77,6 +83,7 @@ mc mb -l myminio/vbo
- When you create your backup job, choose the backup repository you created above.
## Test the setup
The next time the backup job runs, you can use the `mc admin trace myminio` command and verify traffic is flowing to the MinIO nodes. For Veeam Backup and Replication you will need to wait for the backup to complete to the performance tier before it migrates data to the capacity tier (i.e., MinIO).
```

21
docs/kms/IAM.md
View File

@ -2,7 +2,7 @@
MinIO supports encrypting config, IAM assets with KMS provided keys. If the KMS is not enabled, MinIO will store the config, IAM data as plain text erasure coded in its backend.
### MinIO KMS Quick Start
## MinIO KMS Quick Start
MinIO supports two ways of encrypting IAM and configuration data.
You can either use KES - together with an external KMS - or, much simpler,
@ -12,36 +12,42 @@ to set it up refer to our [KMS Guide](https://github.com/minio/minio/blob/master
Instead of configuring an external KMS you can start with a single key by
setting the env. variable `MINIO_KMS_SECRET_KEY`. It expects the following
format:
```sh
MINIO_KMS_SECRET_KEY=<key-name>:<base64-value>
```
First generate a 256 bit random key via:
```sh
$ cat /dev/urandom | head -c 32 | base64 -
OSMM+vkKUTCvQs9YL/CVMIMt43HFhkUpqJxTmGl6rYw=
```
Now, you can set `MINIO_KMS_SECRET_KEY` like this:
```sh
export MINIO_KMS_SECRET_KEY=my-minio-key:OSMM+vkKUTCvQs9YL/CVMIMt43HFhkUpqJxTmGl6rYw=
```
> You can choose an arbitrary name for the key - instead of `my-minio-key`.
> Please note that loosing the `MINIO_KMS_SECRET_KEY` will cause data loss
> since you will not be able to decrypt the IAM/configuration data anymore.
For distributed MinIO deployments, specify the *same* `MINIO_KMS_SECRET_KEY` for each MinIO server process.
For distributed MinIO deployments, specify the *same* `MINIO_KMS_SECRET_KEY` for each MinIO server process.
At any point in time you can switch from `MINIO_KMS_SECRET_KEY` to a full KMS
deployment. You just need to import the generated key into KES - for example via
the KES CLI once you have successfully setup KES:
```sh
kes key create my-minio-key OSMM+vkKUTCvQs9YL/CVMIMt43HFhkUpqJxTmGl6rYw=
```
- For instructions on setting up KES, see the [KES Getting Started guide](https://github.com/minio/kes/wiki/Getting-Started)
- For instructions on setting up KES, see the [KES Getting Started guide](https://github.com/minio/kes/wiki/Getting-Started)
- For instructions on using KES for encrypting the MinIO backend, follow the [KMS Quick Start](https://github.com/minio/minio/tree/master/docs/kms). The SSE-S3 configuration setup also supports MinIO KMS backend encryption.
### FAQ
## FAQ
> Why is this change needed?
@ -51,13 +57,14 @@ Now, MinIO encrypts IAM / configuration and S3 objects with a KMS, if present. T
change unified the key-management aspect within MinIO.
The unified KMS-based approach has several advantages:
- Key management is now centralized. There is one way to change or rotate encryption keys.
- Key management is now centralized. There is one way to change or rotate encryption keys.
There used to be two different mechanisms - one for regular S3 objects and one for IAM data.
- Reduced server startup time. For IAM encryption with the root credentials, MinIO had
- Reduced server startup time. For IAM encryption with the root credentials, MinIO had
to use a memory-hard function (Argon2) that (on purpose) consumes a lot of memory and CPU.
The new KMS-based approach can use a key derivation function that is orders of magnitudes
cheaper w.r.t. memory and CPU.
- Root credentials can now be changed easily. Before, a two-step process was required to
- Root credentials can now be changed easily. Before, a two-step process was required to
change the cluster root credentials since they were used to en/decrypt the IAM data.
So, both - the old and new credentials - had to be present at the same time during a rotation
and the old credentials had to be removed once the rotation completed. This process is now gone.

16
docs/kms/README.md
View File

@ -6,7 +6,8 @@ MinIO uses a key-management-system (KMS) to support SSE-S3. If a client requests
MinIO supports multiple KMS implementations via our [KES](https://github.com/minio/kes#kes) project. We run a KES instance at `https://play.min.io:7373` for you to experiment and quickly get started. To run MinIO with a KMS just fetch the root identity, set the following environment variables and then start your MinIO server. If you havn't installed MinIO, yet, then follow the MinIO [install instructions](https://docs.min.io/docs/minio-quickstart-guide) first.
#### 1. Fetch the root identity
### 1. Fetch the root identity
As the initial step, fetch the private key and certificate of the root identity:
```sh
@ -15,7 +16,7 @@ curl -sSL --tlsv1.2 \
-O 'https://raw.githubusercontent.com/minio/kes/master/root.cert'
```
#### 2. Set the MinIO-KES configuration
### 2. Set the MinIO-KES configuration
```sh
export MINIO_KMS_KES_ENDPOINT=https://play.min.io:7373
@ -24,7 +25,7 @@ export MINIO_KMS_KES_CERT_FILE=root.cert
export MINIO_KMS_KES_KEY_NAME=my-minio-key
```
#### 3. Start the MinIO Server
### 3. Start the MinIO Server
```sh
export MINIO_ROOT_USER=minio
@ -39,6 +40,7 @@ minio server ~/export
## Configuration Guides
A typical MinIO deployment that uses a KMS for SSE-S3 looks like this:
```
┌────────────┐
│ ┌──────────┴─┬─────╮ ┌────────────┐
@ -61,7 +63,6 @@ The main difference between various MinIO-KMS deployments is the KMS implementat
| [Google Cloud Platform SecretManager](https://github.com/minio/kes/wiki/GCP-SecretManager) | Cloud KMS. MinIO in combination with a managed KMS installation |
| [FS](https://github.com/minio/kes/wiki/Filesystem-Keystore) | Local testing or development (**Not recommended for production**) |
The MinIO-KES configuration is always the same - regardless of the underlying KMS implementation. Checkout the MinIO-KES [configuration example](https://github.com/minio/kes/wiki/MinIO-Object-Storage).
### Further references
@ -72,27 +73,34 @@ The MinIO-KES configuration is always the same - regardless of the underlying KM
- [Understand the KES server concepts](https://github.com/minio/kes/wiki/Concepts)
## Auto Encryption
Auto-Encryption is useful when MinIO administrator wants to ensure that all data stored on MinIO is encrypted at rest.
### Using `mc encrypt` (recommended)
MinIO automatically encrypts all objects on buckets if KMS is successfully configured and bucket encryption configuration is enabled for each bucket as shown below:
```
mc encrypt set sse-s3 myminio/bucket/
```
Verify if MinIO has `sse-s3` enabled
```
mc encrypt info myminio/bucket/
Auto encryption 'sse-s3' is enabled
```
### Using environment (not-recommended)
MinIO automatically encrypts all objects on buckets if KMS is successfully configured and following ENV is enabled:
```
export MINIO_KMS_AUTO_ENCRYPTION=on
```
### Verify auto-encryption
> Note that auto-encryption only affects requests without S3 encryption headers. So, if a S3 client sends
> e.g. SSE-C headers, MinIO will encrypt the object with the key sent by the client and won't reach out to
> the configured KMS.

37
docs/logging/README.md
View File

@ -1,19 +1,24 @@
# MinIO Logging Quickstart Guide [![Slack](https://slack.min.io/slack?type=svg)](https://slack.min.io)
This document explains how to configure MinIO server to log to different logging targets.
## Log Targets
MinIO supports currently two target types
- console
- http
### Console Target
### Logging Console Target
Console target is on always and cannot be disabled.
### HTTP Target
### Logging HTTP Target
HTTP target logs to a generic HTTP endpoint in JSON format and is not enabled by default. To enable HTTP target logging you would have to update your MinIO server configuration using `mc admin config set` command.
Assuming `mc` is already [configured](https://docs.min.io/docs/minio-client-quickstart-guide.html)
```
mc admin config get myminio/ logger_webhook
logger_webhook:name1 auth_token="" endpoint=""
@ -27,6 +32,7 @@ mc admin service restart myminio
NOTE: `http://endpoint:port/path` is a placeholder value to indicate the URL format, please change this accordingly as per your configuration.
MinIO also honors environment variable for HTTP target logging as shown below, this setting will override the endpoint settings in the MinIO server config.
```
export MINIO_LOGGER_WEBHOOK_ENABLE_target1="on"
export MINIO_LOGGER_WEBHOOK_AUTH_TOKEN_target1="token"
@ -35,9 +41,11 @@ minio server /mnt/data
```
## Audit Targets
Assuming `mc` is already [configured](https://docs.min.io/docs/minio-client-quickstart-guide.html)
### HTTP Target
### Audit HTTP Target
```
mc admin config get myminio/ audit_webhook
audit_webhook:name1 enable=off endpoint= auth_token= client_cert= client_key=
@ -51,6 +59,7 @@ mc admin service restart myminio
NOTE: `http://endpoint:port/path` is a placeholder value to indicate the URL format, please change this accordingly as per your configuration.
MinIO also honors environment variable for HTTP target Audit logging as shown below, this setting will override the endpoint settings in the MinIO server config.
```
export MINIO_AUDIT_WEBHOOK_ENABLE_target1="on"
export MINIO_AUDIT_WEBHOOK_AUTH_TOKEN_target1="token"
@ -63,11 +72,12 @@ minio server /mnt/data
Setting this environment variable automatically enables audit logging to the HTTP target. The audit logging is in JSON format as described below.
NOTE:
- `timeToFirstByte` and `timeToResponse` will be expressed in Nanoseconds.
- Additionally in the case of the erasure coded setup `tags.objectErasureMap` provides per object details about
- Pool number the object operation was performed on.
- Set number the object operation was performed on.
- The list of disks participating in this operation belong to the set.
- Pool number the object operation was performed on.
- Set number the object operation was performed on.
- The list of disks participating in this operation belong to the set.
```json
{
@ -131,7 +141,9 @@ NOTE:
```
### Kafka Target
Assuming that you already have Apache Kafka configured and running.
```
mc admin config set myminio/ audit_kafka
KEY:
@ -154,6 +166,7 @@ comment (sentence) optionally add a comment to this setting
```
Configure MinIO to send audit logs to locally running Kafka brokers
```
mc admin config set myminio/ audit_kafka:target1 brokers=localhost:29092 topic=auditlog
mc admin service restart myminio/
@ -201,12 +214,14 @@ minio server /mnt/data
Setting this environment variable automatically enables audit logging to the Kafka target. The audit logging is in JSON format as described below.
NOTE:
- `timeToFirstByte` and `timeToResponse` will be expressed in Nanoseconds.
- Additionally in the case of the erasure coded setup `tags.objectErasureMap` provides per object details about
- Pool number the object operation was performed on.
- Set number the object operation was performed on.
- The list of disks participating in this operation belong to the set.
- Pool number the object operation was performed on.
- Set number the object operation was performed on.
- The list of disks participating in this operation belong to the set.
## Explore Further
* [MinIO Quickstart Guide](https://docs.min.io/docs/minio-quickstart-guide)
* [Configure MinIO Server with TLS](https://docs.min.io/docs/how-to-secure-access-to-minio-server-with-tls)
- [MinIO Quickstart Guide](https://docs.min.io/docs/minio-quickstart-guide)
- [Configure MinIO Server with TLS](https://docs.min.io/docs/how-to-secure-access-to-minio-server-with-tls)

9
docs/metrics/README.md
View File

@ -1,8 +1,8 @@
## MinIO Monitoring Guide
# MinIO Monitoring Guide
MinIO server exposes monitoring data over endpoints. Monitoring tools can pick the data from these endpoints. This document lists the monitoring endpoints and relevant documentation.
### Healthcheck Probe
## Healthcheck Probe
MinIO server has two healthcheck related un-authenticated endpoints, a liveness probe to indicate if server is responding, cluster probe to check if server can be taken down for maintenance.
@ -11,7 +11,7 @@ MinIO server has two healthcheck related un-authenticated endpoints, a liveness
Read more on how to use these endpoints in [MinIO healthcheck guide](https://github.com/minio/minio/blob/master/docs/metrics/healthcheck/README.md).
### Prometheus Probe
## Prometheus Probe
MinIO allows reading metrics for the entire cluster from any single node. This allows for metrics collection for a MinIO instance across all servers. Thus, metrics collection for instances behind a load balancer can be done without any knowledge of the individual node addresses. The cluster wide metrics can be read at
`<Address for MinIO Service>/minio/v2/metrics/cluster`.
@ -21,7 +21,6 @@ The additional node specific metrics which include additional go metrics or proc
To use this endpoint, setup Prometheus to scrape data from this endpoint. Read more on how to configure and use Prometheus to monitor MinIO server in [How to monitor MinIO server with Prometheus](https://github.com/minio/minio/blob/master/docs/metrics/prometheus/README.md).
**Deprecated metrics monitoring**
### **Deprecated metrics monitoring**
- Prometheus' data available at `/minio/prometheus/metrics` is deprecated

18
docs/metrics/healthcheck/README.md
View File

@ -1,8 +1,8 @@
## MinIO Healthcheck
# MinIO Healthcheck
MinIO server exposes three un-authenticated, healthcheck endpoints liveness probe and a cluster probe at `/minio/health/live` and `/minio/health/cluster` respectively.
### Liveness probe
## Liveness probe
This probe always responds with '200 OK'. Only fails if 'etcd' is configured and unreachable. This behavior is specific to gateway. When liveness probe fails, Kubernetes like platforms restart the container.
@ -19,7 +19,7 @@ livenessProbe:
failureThreshold: 3
```
### Readiness probe
## Readiness probe
This probe always responds with '200 OK'. Only fails if 'etcd' is configured and unreachable. This behavior is specific to gateway. When readiness probe fails, Kubernetes like platforms turn-off routing to the container.
@ -36,8 +36,10 @@ readinessProbe:
failureThreshold: 3
```
### Cluster probe
#### Cluster-writeable probe
## Cluster probe
### Cluster-writeable probe
This probe is not useful in almost all cases, this is meant for administrators to see if write quorum is available in any given cluster. The reply is '200 OK' if cluster has write quorum if not it returns '503 Service Unavailable'.
```
@ -55,7 +57,8 @@ X-Xss-Protection: 1; mode=block
Date: Tue, 21 Jul 2020 00:36:14 GMT
```
#### Cluster-readable probe
### Cluster-readable probe
This probe is not useful in almost all cases, this is meant for administrators to see if read quorum is available in any given cluster. The reply is '200 OK' if cluster has read quorum if not it returns '503 Service Unavailable'.
```
@ -73,7 +76,8 @@ X-Xss-Protection: 1; mode=block
Date: Tue, 21 Jul 2020 00:36:14 GMT
```
#### Checking cluster health for maintenance
### Checking cluster health for maintenance
You may query the cluster probe endpoint to check if the node which received the request can be taken down for maintenance, if the server replies back '412 Precondition Failed' this means you will lose HA. '200 OK' means you are okay to proceed.
```

17
docs/metrics/prometheus/README.md
View File

@ -6,20 +6,8 @@ Prometheus offers a multi-dimensional data model with time series data identifie
MinIO exports Prometheus compatible data by default as an authorized endpoint at `/minio/v2/metrics/cluster`. Users looking to monitor their MinIO instances can point Prometheus configuration to scrape data from this endpoint. This document explains how to setup Prometheus and configure it to scrape data from MinIO servers.
**Table of Contents**
- [Prerequisites](#prerequisites)
- [1. Download Prometheus](#1-download-prometheus)
- [2. Configure authentication type for Prometheus metrics](#2-configure-authentication-type-for-prometheus-metrics)
- [3. Configuring Prometheus](#3-configuring-prometheus)
- [3.1 Authenticated Prometheus config](#31-authenticated-prometheus-config)
- [3.2 Public Prometheus config](#32-public-prometheus-config)
- [4. Update `scrape_configs` section in prometheus.yml](#4-update-scrapeconfigs-section-in-prometheusyml)
- [5. Start Prometheus](#5-start-prometheus)
- [6. Configure Grafana](#6-configure-grafana)
- [List of metrics exposed by MinIO](#list-of-metrics-exposed-by-minio)
## Prerequisites
To get started with MinIO, refer [MinIO QuickStart Document](https://docs.min.io/docs/minio-quickstart-guide).
Follow below steps to get started with MinIO monitoring using Prometheus.
@ -80,6 +68,7 @@ If Prometheus endpoint authentication type is set to `public`. Following prometh
This can be collected from any server once per collection.
##### Cluster
```yaml
scrape_configs:
- job_name: minio-job
@ -90,7 +79,9 @@ scrape_configs:
```
##### Node (optional)
Optionally you can also collect per node metrics. This needs to be done on a per server instance.
```yaml
scrape_configs:
- job_name: minio-job

5
docs/metrics/prometheus/grafana/README.md
View File

@ -15,9 +15,6 @@ Refer to the dashboard [json file here](https://raw.githubusercontent.com/minio/
![Grafana](https://raw.githubusercontent.com/minio/minio/master/docs/metrics/prometheus/grafana/grafana-minio.png)
Replication metrics can also be viewed in the Grafana dashboard using [json file here](https://raw.githubusercontent.com/minio/minio/master/docs/metrics/prometheus/grafana/minio-replication.json).
![Grafana](https://raw.githubusercontent.com/minio/minio/master/docs/metrics/prometheus/grafana/grafana-replication.png)
![Grafana](https://raw.githubusercontent.com/minio/minio/master/docs/metrics/prometheus/grafana/grafana-replication.png)

16
docs/minio-limits.md
View File

@ -1,7 +1,8 @@
## MinIO Server Limits Per Tenant
# MinIO Server Limits Per Tenant
For best deployment experience MinIO recommends operating systems RHEL/CentOS 8.x or later, Ubuntu 18.04 LTS or later. These operating systems package the latest 'xfsprogs' that support large scale deployments.
### Erasure Code (Multiple Drives / Servers)
## Erasure Code (Multiple Drives / Servers)
| Item | Specification |
|:----------------------------------------------------------------|:--------------|
@ -15,7 +16,7 @@ For best deployment experience MinIO recommends operating systems RHEL/CentOS 8.
| Read quorum | N/2 |
| Write quorum | N/2+1 |
### Limits of S3 API
## Limits of S3 API
| Item | Specification |
|:--------------------------------------------------------------------------------|:----------------------------------------------|
@ -33,10 +34,11 @@ For best deployment experience MinIO recommends operating systems RHEL/CentOS 8.
| Maximum length for object names | 1024 |
| Maximum length for '/' separated object name segment | 255 |
### List of Amazon S3 API's not supported on MinIO
## List of Amazon S3 API's not supported on MinIO
We found the following APIs to be redundant or less useful outside of AWS S3. If you have a different view on any of the APIs we missed, please open a [GitHub issue](https://github.com/minio/minio/issues).
#### List of Amazon S3 Bucket API's not supported on MinIO
### List of Amazon S3 Bucket API's not supported on MinIO
- BucketACL (Use [bucket policies](https://docs.min.io/docs/minio-client-complete-guide#policy) instead)
- BucketCORS (CORS enabled by default on all buckets for all HTTP verbs)
@ -44,11 +46,11 @@ We found the following APIs to be redundant or less useful outside of AWS S3. If
- BucketAnalytics, BucketMetrics, BucketLogging (Use [bucket notification](https://docs.min.io/docs/minio-client-complete-guide#events) APIs)
- BucketRequestPayment
#### List of Amazon S3 Object API's not supported on MinIO
### List of Amazon S3 Object API's not supported on MinIO
- ObjectACL (Use [bucket policies](https://docs.min.io/docs/minio-client-complete-guide#policy) instead)
### Object name restrictions on MinIO
## Object name restrictions on MinIO
- Object names that contain characters `^*|\/&";` are unsupported on Windows platform or any other file systems that do not support filenames with special charaters. **This list is non exhaustive, it depends on the operating system and filesystem under use - please consult your operating system vendor**. MinIO recommends using Linux based deployments for production workloads.
- Objects should not have conflicting objects as parents, applications using this behavior should change their behavior and use proper unique keys, for example situations such as following conflicting key patterns are not supported.

6
docs/multi-tenancy/README.md
View File

@ -6,7 +6,7 @@ This topic provides commands to set up different configurations of hosts, nodes,
2. [Distributed Deployment](#distributed-deployment)
3. [Cloud Scale Deployment](#cloud-scale-deployment)
## <a name="standalone-deployment"></a>1. Standalone Deployment
## 1. Standalone Deployment
To host multiple tenants on a single machine, run one MinIO Server per tenant with a dedicated HTTPS port, configuration, and data directory.
@ -34,7 +34,7 @@ minio server --address :9003 /disk{1...4}/data/tenant3
![Example-2](https://github.com/minio/minio/blob/master/docs/screenshots/Example-2.jpg?raw=true)
## <a name="distributed-deployment"></a>2. Distributed Deployment
## 2. Distributed Deployment
To host multiple tenants in a distributed environment, run several distributed MinIO Server instances concurrently.
@ -62,6 +62,6 @@ minio server --address :9003 http://192.168.10.1{1...4}/data/tenant3
**Note**: On distributed systems, root credentials are recommend to be defined by exporting the `MINIO_ROOT_USER` and `MINIO_ROOT_PASSWORD` environment variables. If no value is set MinIO setup will assume `minioadmin/minioadmin` as default credentials. If a domain is required, it must be specified by defining and exporting the `MINIO_DOMAIN` environment variable.
## <a name="cloud-scale-deployment"></a>Cloud Scale Deployment
## Cloud Scale Deployment
A container orchestration platform (e.g. Kubernetes) is recommended for large-scale, multi-tenant MinIO deployments. See the [MinIO Deployment Quickstart Guide](https://docs.min.io/docs/minio-deployment-quickstart-guide) to get started with MinIO on orchestration platforms.

30
docs/multi-user/README.md
View File

@ -1,18 +1,23 @@
# MinIO Multi-user Quickstart Guide [![Slack](https://slack.min.io/slack?type=svg)](https://slack.min.io)
MinIO supports multiple long term users in addition to default user created during server startup. New users can be added after server starts up, and server can be configured to deny or allow access to buckets and resources to each of these users. This document explains how to add/remove users and modify their access rights.
## Get started
In this document we will explain in detail on how to configure multiple users.
### 1. Prerequisites
- Install mc - [MinIO Client Quickstart Guide](https://docs.min.io/docs/minio-client-quickstart-guide.html)
- Install MinIO - [MinIO Quickstart Guide](https://docs.min.io/docs/minio-quickstart-guide)
- Configure etcd (optional needed only in gateway or federation mode) - [Etcd V3 Quickstart Guide](https://github.com/minio/minio/blob/master/docs/sts/etcd.md)
### 2. Create a new user with canned policy
Use [`mc admin policy`](https://docs.min.io/docs/minio-admin-complete-guide.html#policies) to create canned policies. Server provides a default set of canned policies namely `writeonly`, `readonly` and `readwrite` *(these policies apply to all resources on the server)*. These can be overridden by custom policies using `mc admin policy` command.
Create new canned policy file `getonly.json`. This policy enables users to download all objects under `my-bucketname`.
```json
cat > getonly.json << EOF
{
@ -34,86 +39,106 @@ EOF
```
Create new canned policy by name `getonly` using `getonly.json` policy file.
```
mc admin policy add myminio getonly getonly.json
```
Create a new user `newuser` on MinIO use `mc admin user`.
```
mc admin user add myminio newuser newuser123
```
Once the user is successfully created you can now apply the `getonly` policy for this user.
```
mc admin policy set myminio getonly user=newuser
```
### 3. Create a new group
```
mc admin group add myminio newgroup newuser
```
Once the group is successfully created you can now apply the `getonly` policy for this group.
```
mc admin policy set myminio getonly group=newgroup
```
### 4. Disable user
Disable user `newuser`.
```
mc admin user disable myminio newuser
```
Disable group `newgroup`.
```
mc admin group disable myminio newgroup
```
### 5. Remove user
Remove the user `newuser`.
```
mc admin user remove myminio newuser
```
Remove the user `newuser` from a group.
```
mc admin group remove myminio newgroup newuser
```
Remove the group `newgroup`.
```
mc admin group remove myminio newgroup
```
### 6. Change user or group policy
Change the policy for user `newuser` to `putonly` canned policy.
```
mc admin policy set myminio putonly user=newuser
```
Change the policy for group `newgroup` to `putonly` canned policy.
```
mc admin policy set myminio putonly group=newgroup
```
### 7. List all users or groups
List all enabled and disabled users.
```
mc admin user list myminio
```
List all enabled or disabled groups.
```
mc admin group list myminio
```
### 8. Configure `mc`
```
mc alias set myminio-newuser http://localhost:9000 newuser newuser123 --api s3v4
mc cat myminio-newuser/my-bucketname/my-objectname
```
### Policy Variables
You can use policy variables in the *Resource* element and in string comparisons in the *Condition* element.
You can use a policy variable in the Resource element, but only in the resource portion of the ARN. This portion of the ARN appears after the 5th colon (:). You can't use a variable to replace parts of the ARN before the 5th colon, such as the service or account. The following policy might be attached to a group. It gives each of the users in the group full programmatic access to a user-specific object (their own "home directory") in MinIO.
@ -143,6 +168,7 @@ You can use a policy variable in the Resource element, but only in the resource
If the user is authenticating using an STS credential which was authorized from OpenID connect we allow all `jwt:*` variables specified in the JWT specification, custom `jwt:*` or extensions are not supported.
List of policy variables for OpenID based STS.
```
"jwt:sub"
"jwt:iss"
@ -169,6 +195,7 @@ List of policy variables for OpenID based STS.
```
Following example shows OpenID users with full programmatic access to a OpenID user-specific directory (their own "home directory") in MinIO.
```
{
"Version": "2012-10-17",
@ -192,6 +219,7 @@ Following example shows OpenID users with full programmatic access to a OpenID u
```
If the user is authenticating using an STS credential which was authorized from AD/LDAP we allow `ldap:*` variables, currently only supports `ldap:username`. Following example shows LDAP users full programmatic access to a LDAP user-specific directory (their own "home directory") in MinIO.
```
{
"Version": "2012-10-17",
@ -237,8 +265,8 @@ If the user is authenticating using an STS credential which was authorized from
- *aws:UserAgent* - This value is a string that contains information about the requester's client application. This string is generated by the client and can be unreliable. You can only use this context key from `mc` or other MinIO SDKs which standardize the User-Agent string.
- *aws:username* - This is a string containing the friendly name of the current user, this value would point to STS temporary credential in `AssumeRole`ed requests, instead use `jwt:preferred_username` in case of OpenID connect and `ldap:username` in case of AD/LDAP connect. *aws:userid* is an alias to *aws:username* in MinIO.
## Explore Further
- [MinIO Client Complete Guide](https://docs.min.io/docs/minio-client-complete-guide)
- [MinIO STS Quickstart Guide](https://docs.min.io/docs/minio-sts-quickstart-guide)
- [MinIO Admin Complete Guide](https://docs.min.io/docs/minio-admin-complete-guide.html)

22
docs/multi-user/admin/README.md
View File

@ -1,18 +1,23 @@
# MinIO Admin Multi-user Quickstart Guide [![Slack](https://slack.min.io/slack?type=svg)](https://slack.min.io)
MinIO supports multiple admin users in addition to default operator credential created during server startup. New admins can be added after server starts up, and server can be configured to deny or allow access to different admin operations for these users. This document explains how to add/remove admin users and modify their access rights.
## Get started
In this document we will explain in detail on how to configure admin users.
### 1. Prerequisites
- Install mc - [MinIO Client Quickstart Guide](https://docs.min.io/docs/minio-client-quickstart-guide.html)
- Install MinIO - [MinIO Quickstart Guide](https://docs.min.io/docs/minio-quickstart-guide)
### 2. Create a new admin user with CreateUser, DeleteUser and ConfigUpdate permissions
Use [`mc admin policy`](https://docs.min.io/docs/minio-admin-complete-guide.html#policies) to create custom admin policies.
Create new canned policy file `adminManageUser.json`. This policy enables admin user to
manage other users.
```json
cat > adminManageUser.json << EOF
{
@ -43,11 +48,13 @@ EOF
```
Create new canned policy by name `userManager` using `userManager.json` policy file.
```
mc admin policy add myminio userManager adminManageUser.json
```
Create a new admin user `admin1` on MinIO use `mc admin user`.
```
mc admin user add myminio admin1 admin123
```
@ -61,6 +68,7 @@ mc admin policy set myminio userManager user=admin1
This admin user will then be allowed to perform create/delete user operations via `mc admin user`
### 3. Configure `mc` and create another user user1 with attached policy user1policy
```
mc alias set myminio-admin1 http://localhost:9000 admin1 admin123 --api s3v4
@ -70,10 +78,13 @@ mc admin policy set myminio-admin1 user1policy user=user1
```
### 4. List of permissions defined for admin operations
#### Config management permissions
- admin:ConfigUpdate
#### User management permissions
- admin:CreateUser
- admin:DeleteUser
- admin:ListUsers
@ -82,6 +93,7 @@ mc admin policy set myminio-admin1 user1policy user=user1
- admin:GetUser
#### Service management permissions
- admin:ServerInfo
- admin:ServerUpdate
- admin:StorageInfo
@ -101,6 +113,7 @@ mc admin policy set myminio-admin1 user1policy user=user1
- admin:BandwidthMonitor
#### User/Group management permissions
- admin:AddUserToGroup
- admin:RemoveUserFromGroup
- admin:GetGroup
@ -109,6 +122,7 @@ mc admin policy set myminio-admin1 user1policy user=user1
- admin:DisableGroup
#### Policy management permissions
- admin:CreatePolicy
- admin:DeletePolicy
- admin:GetPolicy
@ -116,34 +130,42 @@ mc admin policy set myminio-admin1 user1policy user=user1
- admin:ListUserPolicies
#### Heal management permissions
- admin:Heal
#### Service account management permissions
- admin:CreateServiceAccount
- admin:UpdateServiceAccount
- admin:RemoveServiceAccount
- admin:ListServiceAccounts
#### Bucket quota management permissions
- admin:SetBucketQuota
- admin:GetBucketQuota
#### Bucket target management permissions
- admin:SetBucketTarget
- admin:GetBucketTarget
#### Remote tier management permissions
- admin:SetTier
- admin:ListTier
#### Give full admin permissions
- admin:*
### 5. Using an external IDP for admin users
Admin users can also be externally managed by an IDP by configuring admin policy with
special permissions listed above. Follow [MinIO STS Quickstart Guide](https://docs.min.io/docs/minio-sts-quickstart-guide) to manage users with an IDP.
## Explore Further
- [MinIO Client Complete Guide](https://docs.min.io/docs/minio-client-complete-guide)
- [MinIO STS Quickstart Guide](https://docs.min.io/docs/minio-sts-quickstart-guide)
- [MinIO Admin Complete Guide](https://docs.min.io/docs/minio-admin-complete-guide.html)

2
docs/orchestration/README.md
View File

@ -7,8 +7,8 @@ MinIO is a cloud-native application designed to scale in a sustainable manner in
| [`Kubernetes`](https://docs.min.io/docs/deploy-minio-on-kubernetes) |
| [`Docker Compose`](https://docs.min.io/docs/deploy-minio-on-docker-compose) (only for test setups) |
## Why is MinIO cloud-native?
The term cloud-native revolves around the idea of applications deployed as micro services, that scale well. It is not about just retrofitting monolithic applications onto modern container based compute environment. A cloud-native application is portable and resilient by design, and can scale horizontally by simply replicating. Modern orchestration platforms like Kubernetes, DC/OS make replicating and managing containers in huge clusters easier than ever.
While containers provide isolated application execution environment, orchestration platforms allow seamless scaling by helping replicate and manage containers. MinIO extends this by adding isolated storage environment for each tenant.

9
docs/orchestration/docker-compose/README.md
View File

@ -39,7 +39,7 @@ or
docker stack deploy --compose-file docker-compose.yaml minio
```
Distributed instances are now accessible on the host at ports 9000, proceed to access the Web browser at http://127.0.0.1:9000/. Here 4 MinIO server instances are reverse proxied through Nginx load balancing.
Distributed instances are now accessible on the host at ports 9000, proceed to access the Web browser at <http://127.0.0.1:9000/>. Here 4 MinIO server instances are reverse proxied through Nginx load balancing.
### Notes
@ -53,6 +53,7 @@ Distributed instances are now accessible on the host at ports 9000, proceed to a
Read more about distributed MinIO [here](https://docs.min.io/docs/distributed-minio-quickstart-guide).
### Explore Further
- [Overview of Docker Compose](https://docs.docker.com/compose/overview/)
- [MinIO Docker Quickstart Guide](https://docs.min.io/docs/minio-docker-quickstart-guide)
- [MinIO Erasure Code QuickStart Guide](https://docs.min.io/docs/minio-erasure-code-quickstart-guide)
* [Overview of Docker Compose](https://docs.docker.com/compose/overview/)
* [MinIO Docker Quickstart Guide](https://docs.min.io/docs/minio-docker-quickstart-guide)
* [MinIO Erasure Code QuickStart Guide](https://docs.min.io/docs/minio-erasure-code-quickstart-guide)

39
docs/security/README.md
View File

@ -3,6 +3,7 @@
## Server-Side Encryption
MinIO supports two different types of server-side encryption ([SSE](#sse)):
- **SSE-C**: The MinIO server en/decrypts an object with a secret key provided by the S3 client as part of the HTTP request headers. Therefore, [SSE-C](#ssec) requires TLS/HTTPS.
- **SSE-S3**: The MinIO server en/decrypts an object with a secret key managed by a KMS. Therefore, MinIO requires a valid KMS configuration for [SSE-S3](#sses3).
@ -29,6 +30,8 @@ To summarize for any encrypted object there exists (at least) three different ke
The MinIO server uses an authenticated encryption scheme ([AEAD](#aead)) to en/decrypt and authenticate the object content. The AEAD is combined with some state to build a *Secure Channel*. A *Secure Channel* is a cryptographic construction that ensures confidentiality and integrity of the processed data. In particular the *Secure Channel* splits the plaintext content into fixed size chunks and en/decrypts each chunk separately using an unique key-nonce combination.
##### Figure 1 - Secure Channel construction
```
plaintext := chunk_0 || chunk_1 || chunk_2 || ...
| | |
@ -38,7 +41,6 @@ plaintext := chunk_0 || chunk_1 || chunk_2
| | |
ciphertext := sealed_chunk_0 || sealed_chunk_1 || sealed_chunk_2 || ...
```
<center>Figure 1 - Secure Channel construction</center>
In case of a S3 multi-part operation each part is en/decrypted with the scheme shown in Figure 1. However, for each part an unique secret key is derived from the OEK and the part number using a PRF. So in case of multi-part not the OEK but the output of `PRF(OEK, part_id)` is used as secret key.
@ -46,8 +48,8 @@ In case of a S3 multi-part operation each part is en/decrypted with the scheme s
The SSE schemes described in [Secret Keys](#Secret-Keys) and [Content Encryption](#Content-Encryption) are generic over the cryptographic primitives. However, the MinIO server uses the following cryptographic primitive implementations:
- [PRF](#prf): HMAC-SHA-256
- [AEAD](#aead): AES-256-GCM if the CPU supports AES-NI, ChaCha20-Poly1305 otherwise. More specifically AES-256-GCM is only selected for X86-64 CPUs with AES-NI extension.
- [PRF](#prf): HMAC-SHA-256
- [AEAD](#aead): AES-256-GCM if the CPU supports AES-NI, ChaCha20-Poly1305 otherwise. More specifically AES-256-GCM is only selected for X86-64 CPUs with AES-NI extension.
Further any secret key (apart from the KMS-generated ones) is 256 bits long. The KMS-generated keys may be 256 bits but this depends on the KMS capabilities and configuration.
@ -67,8 +69,8 @@ MinIO does not assume or require that the client-provided key is unique. It may
S3 clients can change the client-provided key of an existing object. Therefore an S3 client must perform a S3 COPY operation where the copy source and destination are equal. Further the COPY request headers must contain the current and the new client key:
- `X-Amz-Server-Side-Encryption-Customer-Key`: Base64 encoded new key.
- `X-Amz-Copy-Source-Server-Side-Encryption-Customer-Key`: Base64 encoded current key.
- `X-Amz-Server-Side-Encryption-Customer-Key`: Base64 encoded new key.
- `X-Amz-Copy-Source-Server-Side-Encryption-Customer-Key`: Base64 encoded current key.
Such a special COPY request is also known as S3 SSE-C key rotation.
@ -85,6 +87,8 @@ More details about supported KMS implementations and configuration can be found
The MinIO server requests a new data key from the KMS for each uploaded object and uses that data key as EK. Additionally it stores the encrypted form of the data key and the master key ID as part of the object metadata. The plain data only resides in RAM during the en/decryption process. The MinIO server does not store any SSE-related key at the KMS. Instead the KMS is treated as trusted component that performs key sealing/unsealing operations to build a key hierarchy:
#### Figure 2 - KMS key hierarchy
```
CMK (master key)
|
@ -110,13 +114,13 @@ The MinIO server requests a new data key from the KMS for each uploaded object a
| object_metadata_1 | | object_metadata_2 |
+-------------------+ +-------------------+
```
<center>Figure 2 - KMS key hierarchy</center>
#### Key rotation - Basic Operation
The MinIO server supports key rotation for SSE-S3 encrypted objects. The minio server decrypts the OEK using the current encrypted data key and the master key ID of the object metadata. If this succeeds, the server requests a new data key from the KMS using the master key ID of the **current MinIO KMS configuration** and re-wraps the *OEK* with a new *KEK* derived from the new data key / EK:
##### Figure 3 - KMS data key rotation
```
object metadata KMS
| |
@ -153,7 +157,6 @@ The MinIO server supports key rotation for SSE-S3 encrypted objects. The minio s
4) Derive a new KEK from the new data key and re-encrypt the OEK with it.
5) Store the encrypted OEK encrypted data key and master key ID in object metadata.
```
<center>Figure 3 - KMS data key rotation</center>
Only the root/admin user can perform an SSE-S3 key rotation using the Admin-API via [mc](https://github.com/minio/mc). For more details about how to perform key management operations using the CLI refer to [mc admin guide](https://github.com/minio/mc/blob/master/docs/minio-admin-complete-guide.md) or run `mc admin kms key`.
@ -167,13 +170,13 @@ The MinIO server requires an available KMS to en/decrypt SSE-S3 encrypted object
## Acronyms
- <a name="aead"></a>**AEAD**: Authenticated Encryption with Associated Data
- <a name="csprng"></a>**CSPRNG**: Cryptographically Secure Pseudo Random Number Generator
- <a name="ek"></a>**EK**: External Key
- <a name="iv"></a>**IV**: Initialization Vector
- <a name="kek"></a>**KEK**: Key Encryption Key
- <a name="oek"></a>**OEK**: Object Encryption Key
- <a name="prf"></a>**PRF**: Pseudo Random Function
- <a name="sse"></a>**SSE**: Server-Side Encryption
- <a name="ssec"></a>**SSE-C**: Server-Side Encryption with client-provided Keys
- <a name="sses3"></a>**SSE-S3**: Server-Side Encryption with a KMS
- **AEAD**: Authenticated Encryption with Associated Data
- **CSPRNG**: Cryptographically Secure Pseudo Random Number Generator
- **EK**: External Key
- **IV**: Initialization Vector
- **KEK**: Key Encryption Key
- **OEK**: Object Encryption Key
- **PRF**: Pseudo Random Function
- **SSE**: Server-Side Encryption
- **SSE-C**: Server-Side Encryption with client-provided Keys
- **SSE-S3**: Server-Side Encryption with a KMS

25
docs/select/README.md
View File

@ -1,17 +1,18 @@
# Select API Quickstart Guide [![Slack](https://slack.min.io/slack?type=svg)](https://slack.min.io)
Traditional retrieval of objects is always as whole entities, i.e GetObject for a 5 GiB object, will always return 5 GiB of data. S3 Select API allows us to retrieve a subset of data by using simple SQL expressions. By using Select API to retrieve only the data needed by the application, drastic performance improvements can be achieved.
You can use the Select API to query objects with following features:
- Objects must be in CSV, JSON, or Parquet(*) format.
- Objects must be in CSV, JSON, or Parquet(*) format.
- UTF-8 is the only encoding type the Select API supports.
- GZIP or BZIP2 - CSV and JSON files can be compressed using GZIP, BZIP2, [ZSTD](https://facebook.github.io/zstd/), and streaming formats of [LZ4](https://lz4.github.io/lz4/), [S2](https://github.com/klauspost/compress/tree/master/s2#s2-compression) and [SNAPPY](http://google.github.io/snappy/).
- GZIP or BZIP2 - CSV and JSON files can be compressed using GZIP, BZIP2, [ZSTD](https://facebook.github.io/zstd/), and streaming formats of [LZ4](https://lz4.github.io/lz4/), [S2](https://github.com/klauspost/compress/tree/master/s2#s2-compression) and [SNAPPY](http://google.github.io/snappy/).
- Parquet API supports columnar compression for using GZIP, Snappy, LZ4. Whole object compression is not supported for Parquet objects.
- Server-side encryption - The Select API supports querying objects that are protected with server-side encryption.
Type inference and automatic conversion of values is performed based on the context when the value is un-typed (such as when reading CSV data). If present, the CAST function overrides automatic conversion.
The [mc sql](https://docs.min.io/docs/minio-client-complete-guide.html#sql) command can be used for executing queries using the command line.
The [mc sql](https://docs.min.io/docs/minio-client-complete-guide.html#sql) command can be used for executing queries using the command line.
(*) Parquet is disabled on the MinIO server by default. See below how to enable it.
@ -22,17 +23,20 @@ Parquet is DISABLED by default since hostile crafted input can easily crash the
If you are in a controlled environment where it is safe to assume no hostile content can be uploaded to your cluster you can safely enable Parquet.
To enable Parquet set the environment variable `MINIO_API_SELECT_PARQUET=on`.
# Example using Python API
# Example using Python API
## 1. Prerequisites
- Install MinIO Server from [here](https://docs.min.io/docs/minio-quickstart-guide).
- Familiarity with AWS S3 API.
- Familiarity with Python and installing dependencies.
## 2. Install boto3
Install `aws-sdk-python` from AWS SDK for Python official docs [here](https://aws.amazon.com/sdk-for-python/)
## 3. Example
As an example, let us take a gzip compressed CSV file. Without S3 Select, we would need to download, decompress and process the entire CSV to get the data you needed. With Select API, can use a simple SQL expression to return only the data from the CSV you’re interested in, instead of retrieving the entire object. Following Python example shows how to retrieve the first column `Location` from an object containing data in CSV format.
Please replace ``endpoint_url``,``aws_access_key_id``, ``aws_secret_access_key``, ``Bucket`` and ``Key`` with your local setup in this ``select.py`` file.
@ -74,15 +78,18 @@ for event in r['Payload']:
```
## 4. Run the Program
Upload a sample dataset to MinIO using the following commands.
```sh
$ curl "https://population.un.org/wpp/Download/Files/1_Indicators%20(Standard)/CSV_FILES/WPP2019_TotalPopulationBySex.csv" > TotalPopulation.csv
$ mc mb myminio/mycsvbucket
$ gzip TotalPopulation.csv
$ mc cp TotalPopulation.csv.gz myminio/mycsvbucket/sampledata/
curl "https://population.un.org/wpp/Download/Files/1_Indicators%20(Standard)/CSV_FILES/WPP2019_TotalPopulationBySex.csv" > TotalPopulation.csv
mc mb myminio/mycsvbucket
gzip TotalPopulation.csv
mc cp TotalPopulation.csv.gz myminio/mycsvbucket/sampledata/
```
Now let us proceed to run our select example to query for `Location` which matches `United States`.
```sh
$ python3 select.py
840,United States of America,2,Medium,1950,1950.5,79233.218,79571.179,158804.395
@ -105,6 +112,7 @@ Stats details bytesProcessed:
For a more detailed SELECT SQL reference, please see [here](https://docs.aws.amazon.com/AmazonS3/latest/dev/s3-glacier-select-sql-reference-select.html)
## 5. Explore Further
- [Use `mc` with MinIO Server](https://docs.min.io/docs/minio-client-quickstart-guide)
- [Use `mc sql` with MinIO Server](https://docs.min.io/docs/minio-client-complete-guide.html#sql)
- [Use `minio-go` SDK with MinIO Server](https://docs.min.io/docs/golang-client-quickstart-guide)
@ -113,6 +121,7 @@ For a more detailed SELECT SQL reference, please see [here](https://docs.aws.ama
- [The MinIO documentation website](https://docs.min.io)
## 6. Implementation Status
- Full AWS S3 [SELECT SQL](https://docs.aws.amazon.com/AmazonS3/latest/dev/s3-glacier-select-sql-reference-select.html) syntax is supported.
- All [operators](https://docs.aws.amazon.com/AmazonS3/latest/dev/s3-glacier-select-sql-reference-operators.html) are supported.
- All aggregation, conditional, type-conversion and string functions are supported.

62
docs/shared-backend/DESIGN.md
View File

@ -1,15 +1,12 @@
Introduction [![Slack](https://slack.min.io/slack?type=svg)](https://slack.min.io)
------------
# Introduction [![Slack](https://slack.min.io/slack?type=svg)](https://slack.min.io)
This feature allows MinIO to serve a shared NAS drive across multiple MinIO instances. There are no special configuration changes required to enable this feature. Access to files stored on NAS volume are locked and synchronized by default.
Motivation
----------
## Motivation
Since MinIO instances serve the purpose of a single tenant there is an increasing requirement where users want to run multiple MinIO instances on a same backend which is managed by an existing NAS (NFS, GlusterFS, Other distributed filesystems) rather than a local disk. This feature is implemented also with minimal disruption in mind for the user and overall UI.
Restrictions
------------
## Restrictions
* A PutObject() is blocked and waits if another GetObject() is in progress.
* A CompleteMultipartUpload() is blocked and waits if another PutObject() or GetObject() is in progress.
@ -24,11 +21,13 @@ Running MinIO instances on shared backend is no different than running on a stan
Example 1: Start MinIO instance on a shared backend mounted and available at `/path/to/nfs-volume`.
On linux server1
```shell
minio gateway nas /path/to/nfs-volume
```
On linux server2
```shell
minio gateway nas /path/to/nfs-volume
```
@ -38,11 +37,13 @@ minio gateway nas /path/to/nfs-volume
Example 1: Start MinIO instance on a shared backend mounted and available at `\\remote-server\cifs`.
On windows server1
```cmd
minio.exe gateway nas \\remote-server\cifs\data
```
On windows server2
```cmd
minio.exe gateway nas \\remote-server\cifs\data
```
@ -50,47 +51,48 @@ minio.exe gateway nas \\remote-server\cifs\data
Alternatively if `\\remote-server\cifs` is mounted as `D:\` drive.
On windows server1
```cmd
minio.exe gateway nas D:\data
```
On windows server2
```cmd
minio.exe gateway nas D:\data
```
Architecture
------------------
## Architecture
## POSIX/Win32 Locks
### POSIX/Win32 Locks
### Lock process
#### Lock process
With in the same MinIO instance locking is handled by existing in-memory namespace locks (**sync.RWMutex** et. al). To synchronize locks between many MinIO instances we leverage POSIX `fcntl()` locks on Unixes and on Windows `LockFileEx()` Win32 API. Requesting write lock block if there are any read locks held by neighboring MinIO instance on the same path. So does the read lock if there are any active write locks in-progress.
### Unlock process
#### Unlock process
Unlocking happens on filesystems locks by just closing the file descriptor (fd) which was initially requested for lock operation. Closing the fd tells the kernel to relinquish all the locks held on the path by the current process. This gets trickier when there are many readers on the same path by the same process, it would mean that closing an fd relinquishes locks for all concurrent readers as well. To properly manage this situation a simple fd reference count is implemented, the same fd is shared between many readers. When readers start closing on the fd we start reducing the reference count, once reference count has reached zero we can be sure that there are no more readers active. So we proceed and close the underlying file descriptor which would relinquish the read lock held on the path.
This doesn't apply for the writes because there is always one writer and many readers for any unique object.
## Handling Concurrency.
### Handling Concurrency
An example here shows how the contention is handled with GetObject().
GetObject() holds a read lock on `fs.json`.
```go
fsMetaPath := pathJoin(fs.fsPath, minioMetaBucket, bucketMetaPrefix, bucket, object, fsMetaJSONFile)
rlk, err := fs.rwPool.Open(fsMetaPath)
if err != nil {
return toObjectErr(err, bucket, object)
}
defer rlk.Close()
fsMetaPath := pathJoin(fs.fsPath, minioMetaBucket, bucketMetaPrefix, bucket, object, fsMetaJSONFile)
rlk, err := fs.rwPool.Open(fsMetaPath)
if err != nil {
return toObjectErr(err, bucket, object)
}
defer rlk.Close()
... you can perform other operations here ...
_, err = io.Copy(writer, reader)
_, err = io.Copy(writer, reader)
... after successful copy operation unlocks the read lock ...
```
@ -98,19 +100,19 @@ GetObject() holds a read lock on `fs.json`.
A concurrent PutObject is requested on the same object, PutObject() attempts a write lock on `fs.json`.
```go
fsMetaPath := pathJoin(fs.fsPath, minioMetaBucket, bucketMetaPrefix, bucket, object, fsMetaJSONFile)
wlk, err := fs.rwPool.Create(fsMetaPath)
if err != nil {
return ObjectInfo{}, toObjectErr(err, bucket, object)
}
// This close will allow for locks to be synchronized on `fs.json`.
defer wlk.Close()
fsMetaPath := pathJoin(fs.fsPath, minioMetaBucket, bucketMetaPrefix, bucket, object, fsMetaJSONFile)
wlk, err := fs.rwPool.Create(fsMetaPath)
if err != nil {
return ObjectInfo{}, toObjectErr(err, bucket, object)
}
// This close will allow for locks to be synchronized on `fs.json`.
defer wlk.Close()
```
Now from the above snippet the following code one can notice that until the GetObject() returns writing to the client. Following portion of the code will block.
```go
wlk, err := fs.rwPool.Create(fsMetaPath)
wlk, err := fs.rwPool.Create(fsMetaPath)
```
This restriction is needed so that corrupted data is not returned to the client in between I/O. The logic works vice-versa as well an on-going PutObject(), GetObject() would wait for the PutObject() to complete.
@ -121,15 +123,15 @@ Consider for example 3 servers sharing the same backend
On minio1
- DeleteObject(object1) --> lock acquired on `fs.json` while object1 is being deleted.
* DeleteObject(object1) --> lock acquired on `fs.json` while object1 is being deleted.
On minio2
- PutObject(object1) --> lock waiting until DeleteObject finishes.
* PutObject(object1) --> lock waiting until DeleteObject finishes.
On minio3
- PutObject(object1) --> (concurrent request during PutObject minio2 checking if `fs.json` exists)
* PutObject(object1) --> (concurrent request during PutObject minio2 checking if `fs.json` exists)
Once lock is acquired the minio2 validates if the file really exists to avoid obtaining lock on an fd which is already deleted. But this situation calls for a race with a third server which is also attempting to write the same file before the minio2 can validate if the file exists. It might be potentially possible `fs.json` is created so the lock acquired by minio2 might be invalid and can lead to a potential inconsistency.

21
docs/shared-backend/README.md
View File

@ -1,20 +1,20 @@
# Shared Backend MinIO Quickstart Guide [![Slack](https://slack.min.io/slack?type=svg)](https://slack.min.io) [![Docker Pulls](https://img.shields.io/docker/pulls/minio/minio.svg?maxAge=604800)](https://hub.docker.com/r/minio/minio/)
MinIO shared mode lets you use single [NAS](https://en.wikipedia.org/wiki/Network-attached_storage) (like NFS, GlusterFS, and other
distributed filesystems) as the storage backend for multiple MinIO servers. Synchronization among MinIO servers is taken care by design.
MinIO shared mode lets you use single [NAS](https://en.wikipedia.org/wiki/Network-attached_storage) (like NFS, GlusterFS, and other
distributed filesystems) as the storage backend for multiple MinIO servers. Synchronization among MinIO servers is taken care by design.
Read more about the MinIO shared mode design [here](https://github.com/minio/minio/blob/master/docs/shared-backend/DESIGN.md).
MinIO shared mode is developed to solve several real world use cases, without any special configuration changes. Some of these are
- You have already invested in NAS and would like to use MinIO to add S3 compatibility to your storage tier.
- You need to use NAS with an S3 interface due to your application architecture requirements.
- You expect huge traffic and need a load balanced S3 compatible server, serving files from a single NAS backend.
- You expect huge traffic and need a load balanced S3 compatible server, serving files from a single NAS backend.
With a proxy running in front of multiple, shared mode MinIO servers, it is very easy to create a Highly Available, load balanced, AWS S3 compatible storage system.
With a proxy running in front of multiple, shared mode MinIO servers, it is very easy to create a Highly Available, load balanced, AWS S3 compatible storage system.
# Get started
If you're aware of stand-alone MinIO set up, the installation and running remains the same.
If you're aware of stand-alone MinIO set up, the installation and running remains the same.
## 1. Prerequisites
@ -24,12 +24,10 @@ Install MinIO - [MinIO Quickstart Guide](https://docs.min.io/docs/minio-quicksta
To run MinIO shared backend instances, you need to start multiple MinIO servers pointing to the same backend storage. We'll see examples on how to do this in the following sections.
*Note*
- All the nodes running shared MinIO need to have same access key and secret key. To achieve this, we export access key and secret key as environment variables on all the nodes before executing MinIO server command.
- The drive paths below are for demonstration purposes only, you need to replace these with the actual drive paths/folders.
#### MinIO shared mode on Ubuntu 16.04 LTS.
### MinIO shared mode on Ubuntu 16.04 LTS
You'll need the path to the shared volume, e.g. `/path/to/nfs-volume`. Then run the following commands on all the nodes you'd like to launch MinIO.
@ -39,7 +37,7 @@ export MINIO_ROOT_PASSWORD=<SECRET_KEY>
minio gateway nas /path/to/nfs-volume
```
#### MinIO shared mode on Windows 2012 Server
### MinIO shared mode on Windows 2012 Server
You'll need the path to the shared volume, e.g. `\\remote-server\smb`. Then run the following commands on all the nodes you'd like to launch MinIO.
@ -49,9 +47,9 @@ set MINIO_ROOT_PASSWORD=my-password
minio.exe gateway nas \\remote-server\smb\export
```
*Windows Tip*
### Windows Tip
If a remote volume, e.g. `\\remote-server\smb` is mounted as a drive, e.g. `M:\`. You can use [`net use`](https://technet.microsoft.com/en-us/library/bb490717.aspx) command to map the drive to a folder.
If a remote volume, e.g. `\\remote-server\smb` is mounted as a drive, e.g. `M:\`. You can use [`net use`](https://technet.microsoft.com/en-us/library/bb490717.aspx) command to map the drive to a folder.
```cmd
set MINIO_ROOT_USER=my-username
@ -65,6 +63,7 @@ minio.exe gateway nas M:\export
To test this setup, access the MinIO server via browser or [`mc`](https://docs.min.io/docs/minio-client-quickstart-guide). You’ll see the uploaded files are accessible from the all the MinIO shared backend endpoints.
## Explore Further
- [Use `mc` with MinIO Server](https://docs.min.io/docs/minio-client-quickstart-guide)
- [Use `aws-cli` with MinIO Server](https://docs.min.io/docs/aws-cli-with-minio)
- [Use `s3cmd` with MinIO Server](https://docs.min.io/docs/s3cmd-with-minio)

17
docs/site-replication/README.md
View File

@ -1,4 +1,5 @@
# Automatic Site Replication
This feature allows multiple independent MinIO sites (or clusters) that are using the same external IDentity Provider (IDP) to be configured as replicas. In this situation the set of replica sites are referred to as peer sites or just sites. When site-replication is enabled on a set of sites, the following changes are replicated to all other sites:
- Creation and deletion of buckets and objects
@ -31,27 +32,27 @@ The following Bucket features will **not be replicated**, is designed to differ
- Configure an alias in `mc` for each of the sites. For example if you have three MinIO sites, you may run:
```sh
$ mc alias set minio1 https://minio1.example.com:9000 adminuser adminpassword
$ mc alias set minio2 https://minio2.example.com:9000 adminuser adminpassword
$ mc alias set minio3 https://minio3.example.com:9000 adminuser adminpassword
mc alias set minio1 https://minio1.example.com:9000 adminuser adminpassword
mc alias set minio2 https://minio2.example.com:9000 adminuser adminpassword
mc alias set minio3 https://minio3.example.com:9000 adminuser adminpassword
```
or
```sh
$ export MC_HOST_minio1=https://adminuser:adminpassword@minio1.example.com
$ export MC_HOST_minio2=https://adminuser:adminpassword@minio2.example.com
$ export MC_HOST_minio3=https://adminuser:adminpassword@minio3.example.com
export MC_HOST_minio1=https://adminuser:adminpassword@minio1.example.com
export MC_HOST_minio2=https://adminuser:adminpassword@minio2.example.com
export MC_HOST_minio3=https://adminuser:adminpassword@minio3.example.com
```
- Add site replication configuration with:
```sh
$ mc admin replicate add minio1 minio2 minio3
mc admin replicate add minio1 minio2 minio3
```
- Once the above command returns success, you may query site replication configuration with:
```sh
$ mc admin replicate info minio1
mc admin replicate info minio1
```

14
docs/sts/README.md
View File

@ -1,4 +1,5 @@
# MinIO STS Quickstart Guide [![Slack](https://slack.min.io/slack?type=svg)](https://slack.min.io)
The MinIO Security Token Service (STS) is an endpoint service that enables clients to request temporary credentials for MinIO resources. Temporary credentials work almost identically to default admin credentials, with some differences:
- Temporary credentials are short-term, as the name implies. They can be configured to last for anywhere from a few minutes to several hours. After the credentials expire, MinIO no longer recognizes them or allows any kind of access from API requests made with them.
@ -11,6 +12,7 @@ Following are advantages for using temporary credentials:
- Temporary credentials have a limited lifetime, there is no need to rotate them or explicitly revoke them. Expired temporary credentials cannot be reused.
## Identity Federation
| AuthN | Description |
| :---------------------- | ------------------------------------------ |
| [**WebIdentity**](https://github.com/minio/minio/blob/master/docs/sts/web-identity.md) | Let users request temporary credentials using any OpenID(OIDC) compatible web identity providers such as KeyCloak, Dex, Facebook, Google etc. |
@ -18,8 +20,10 @@ Following are advantages for using temporary credentials:
| [**AssumeRole**](https://github.com/minio/minio/blob/master/docs/sts/assume-role.md) | Let MinIO users request temporary credentials using user access and secret keys. |
### Understanding JWT Claims
> NOTE: JWT claims are only meant for WebIdentity and ClientGrants.
> AssumeRole or LDAP users can skip the entire portion and directly visit one of the links below.
>
> - [**AssumeRole**](https://github.com/minio/minio/blob/master/docs/sts/assume-role.md)
> - [**AD/LDAP**](https://github.com/minio/minio/blob/master/docs/sts/ldap.md)
@ -30,18 +34,22 @@ The id_token received is a signed JSON Web Token (JWT). Use a JWT decoder to dec
| policy | _string_ or _[]string_ or _comma_separated_value_ | Canned policy name to be applied for STS credentials. (Mandatory) - This can be configured to any desired value such as `roles` or `groups` by setting the environment variable `MINIO_IDENTITY_OPENID_CLAIM_NAME` |
## Get started
In this document we will explain in detail on how to configure all the prerequisites.
> NOTE: If you are interested in AssumeRole API only, skip to [here](https://github.com/minio/minio/blob/master/docs/sts/assume-role.md)
### Prerequisites
- [Configuring keycloak](https://github.com/minio/minio/blob/master/docs/sts/keycloak.md) or [Configuring Casdoor](https://github.com/minio/minio/blob/master/docs/sts/casdoor.md)
- [Configuring etcd (optional needed only in gateway or federation mode)](https://github.com/minio/minio/blob/master/docs/sts/etcd.md)
### Setup MinIO with Identity Provider
Make sure we have followed the previous step and configured each software independently, once done we can now proceed to use MinIO STS API and MinIO server to use these credentials to perform object API operations.
#### KeyCloak
```
export MINIO_ROOT_USER=minio
export MINIO_ROOT_PASSWORD=minio123
@ -51,6 +59,7 @@ minio server /mnt/data
```
#### Casdoor
```
export MINIO_ROOT_USER=minio
export MINIO_ROOT_PASSWORD=minio123
@ -60,6 +69,7 @@ minio server /mnt/data
```
### Setup MinIO Gateway with Keycloak and Etcd
Make sure we have followed the previous step and configured each software independently, once done we can now proceed to use MinIO STS API and MinIO gateway to use these credentials to perform object API operations.
> NOTE: MinIO gateway requires etcd to be configured to use STS API.
@ -74,6 +84,7 @@ minio gateway s3
```
### Using WebIdentiy API
On another terminal run `web-identity.go` a sample client application which obtains JWT id_tokens from an identity provider, in our case its Keycloak. Uses the returned id_token response to get new temporary credentials from the MinIO server using the STS API call `AssumeRoleWithWebIdentity`.
```
@ -103,11 +114,12 @@ These credentials can now be used to perform MinIO API operations.
### Using MinIO Console
- Open MinIO URL on the browser, lets say http://localhost:9000/
- Open MinIO URL on the browser, lets say <http://localhost:9000/>
- Click on `Login with SSO`
- User will be redirected to the Keycloak user login page, upon successful login the user will be redirected to MinIO page and logged in automatically,
the user should see now the buckets and objects they have access to.
## Explore Further
- [MinIO Admin Complete Guide](https://docs.min.io/docs/minio-admin-complete-guide.html)
- [The MinIO documentation website](https://docs.min.io)

19
docs/sts/assume-role.md
View File

@ -10,7 +10,9 @@ Returns a set of temporary security credentials that you can use to access MinIO
The temporary security credentials returned by this API consists of an access key, a secret key, and a security token. Applications can use these temporary security credentials to sign calls to MinIO API operations. The policy applied to these temporary credentials is inherited from the MinIO user credentials. By default, the temporary security credentials created by AssumeRole last for one hour. However, use the optional DurationSeconds parameter to specify the duration of the credentials. This value varies from 900 seconds (15 minutes) up to the maximum session duration of 365 days.
## API Request Parameters
### Version
Indicates STS API version information, the only supported value is '2011-06-15'. This value is borrowed from AWS STS API documentation for compatibility reasons.
| Params | Value |
@ -19,9 +21,11 @@ Indicates STS API version information, the only supported value is '2011-06-15'.
| *Required* | *Yes* |
### AUTHPARAMS
Indicates STS API Authorization information. If you are familiar with AWS Signature V4 Authorization header, this STS API supports signature V4 authorization as mentioned [here](https://docs.aws.amazon.com/general/latest/gr/signature-version-4.html)
### DurationSeconds
The duration, in seconds. The value can range from 900 seconds (15 minutes) up to 365 days. If value is higher than this setting, then operation fails. By default, the value is set to 3600 seconds.
| Params | Value |
@ -31,6 +35,7 @@ The duration, in seconds. The value can range from 900 seconds (15 minutes) up t
| *Required* | *No* |
### Policy
An IAM policy in JSON format that you want to use as an inline session policy. This parameter is optional. Passing policies to this operation returns new temporary credentials. The resulting session's permissions are the intersection of the canned policy name and the policy set here. You cannot use this policy to grant more permissions than those allowed by the canned policy name being assumed.
| Params | Value |
@ -40,17 +45,21 @@ An IAM policy in JSON format that you want to use as an inline session policy. T
| *Required* | *No* |
### Response Elements
XML response for this API is similar to [AWS STS AssumeRole](https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html#API_AssumeRole_ResponseElements)
### Errors
XML error response for this API is similar to [AWS STS AssumeRole](https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html#API_AssumeRole_Errors)
## Sample `POST` Request
```
http://minio:9000/?Action=AssumeRole&DurationSeconds=3600&Version=2011-06-15&Policy={"Version":"2012-10-17","Statement":[{"Sid":"Stmt1","Effect":"Allow","Action":"s3:*","Resource":"arn:aws:s3:::*"}]}&AUTHPARAMS
```
## Sample Response
```
<?xml version="1.0" encoding="UTF-8"?>
<AssumeRoleResponse xmlns="https://sts.amazonaws.com/doc/2011-06-15/">
@ -73,15 +82,17 @@ http://minio:9000/?Action=AssumeRole&DurationSeconds=3600&Version=2011-06-15&Pol
```
## Using AssumeRole API
```
$ export MINIO_ROOT_USER=minio
$ export MINIO_ROOT_PASSWORD=minio123
$ minio server ~/test
export MINIO_ROOT_USER=minio
export MINIO_ROOT_PASSWORD=minio123
minio server ~/test
```
Create new users following the multi-user guide [here](https://docs.min.io/docs/minio-multi-user-quickstart-guide.html)
### Testing an example with awscli tool
> Use the same username and password created in the previous steps.
```
@ -112,7 +123,6 @@ $ aws --profile foobar --endpoint-url http://localhost:9000 sts assume-role --po
The included program in this directory can also be used for testing:
``` shell
$ go run assume-role.go -u foobar -p foo12345 -d
Only displaying credentials:
@ -123,5 +133,6 @@ SessionToken: eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJhY2Nlc3NLZXkiOiIyN1lEUllFT
```
## Explore Further
- [MinIO Admin Complete Guide](https://docs.min.io/docs/minio-admin-complete-guide.html)
- [The MinIO documentation website](https://docs.min.io)

18
docs/sts/casdoor.md
View File

@ -5,9 +5,10 @@ Casdoor is a UI-first centralized authentication / Single-Sign-On (SSO) platform
## Prerequisites
Configure and install casdoor server by following [Casdoor Server Installation](https://casdoor.org/docs/basic/server-installation).
For a quick installation, docker-compose reference configs are also available on the [Casdoor Try with Docker](https://casdoor.org/docs/basic/try-with-docker).
For a quick installation, docker-compose reference configs are also available on the [Casdoor Try with Docker](https://casdoor.org/docs/basic/try-with-docker).
### Configure Casdoor
- Go to Applications
- Create or use an existing Casdoor application
- Edit the application
@ -23,13 +24,15 @@ For a quick installation, docker-compose reference configs are also available on
- Open your favorite browser and visit: **http://`CASDOOR_ENDPOINT`/.well-known/openid-configuration**, you will see the OIDC configure of Casdoor.
### Configure MinIO
```
$ export MINIO_ROOT_USER=minio
$ export MINIO_ROOT_PASSWORD=minio123
$ minio server /mnt/export
export MINIO_ROOT_USER=minio
export MINIO_ROOT_PASSWORD=minio123
minio server /mnt/export
```
Here are all the available options to configure OpenID connect
```
mc admin config set myminio/ identity_openid
@ -46,6 +49,7 @@ comment (sentence) optionally add a comment to this setting
```
and ENV based options
```
mc admin config set myminio/ identity_openid --env
@ -62,17 +66,21 @@ MINIO_IDENTITY_OPENID_COMMENT (sentence) optionally add a comment to this
```
Set `identity_openid` config with `config_url`, `client_id` and restart MinIO
```
~ mc admin config set myminio identity_openid config_url="http://CASDOOR_ENDPOINT/.well-known/openid-configuration" client_id=<client id> client_secret=<client secret> claim_name="tag"
```
> NOTE: As MinIO needs to use a claim attribute in JWT for its policy, you should configure it in casdoor as well. Currently, casdoor uses `tag` as a workaround for configuring MinIO's policy.
Once successfully set restart the MinIO instance.
```
mc admin service restart myminio
```
### Using WebIdentiy API
On another terminal run `web-identity.go` a sample client application which obtains JWT id_tokens from an identity provider, in our case its Keycloak. Uses the returned id_token response to get new temporary credentials from the MinIO server using the STS API call `AssumeRoleWithWebIdentity`.
```
@ -96,7 +104,7 @@ This will open the login page of Casdoor, upon successful login, STS credentials
### Using MinIO Console
- Open MinIO URL on the browser, lets say http://localhost:9000/
- Open MinIO URL on the browser, lets say <http://localhost:9000/>
- Click on `Login with SSO`
- User will be redirected to the Casdoor user login page, upon successful login the user will be redirected to MinIO page and logged in automatically,
the user should see now the buckets and objects they have access to.

19
docs/sts/client-grants.md
View File

@ -9,7 +9,9 @@ Calling AssumeRoleWithClientGrants does not require the use of MinIO default cre
By default, the temporary security credentials created by AssumeRoleWithClientGrants last for one hour. However, use the optional DurationSeconds parameter to specify the duration of the credentials. This value varies from 900 seconds (15 minutes) up to the maximum session duration of 365 days.
## API Request Parameters
### Token
The OAuth 2.0 access token that is provided by the identity provider. Application must get this token by authenticating the application using client credential grants before the application makes an AssumeRoleWithClientGrants call.
| Params | Value |
@ -19,6 +21,7 @@ The OAuth 2.0 access token that is provided by the identity provider. Applicatio
| *Required* | *Yes* |
### Version
Indicates STS API version information, the only supported value is '2011-06-15'. This value is borrowed from AWS STS API documentation for compatibility reasons.
| Params | Value |
@ -27,6 +30,7 @@ Indicates STS API version information, the only supported value is '2011-06-15'.
| *Required* | *Yes* |
### DurationSeconds
The duration, in seconds. The value can range from 900 seconds (15 minutes) up to 365 days. If value is higher than this setting, then operation fails. By default, the value is set to 3600 seconds. If no *DurationSeconds* is specified expiry seconds is obtained from *Token*.
| Params | Value |
@ -36,6 +40,7 @@ The duration, in seconds. The value can range from 900 seconds (15 minutes) up t
| *Required* | *No* |
### Policy
An IAM policy in JSON format that you want to use as an inline session policy. This parameter is optional. Passing policies to this operation returns new temporary credentials. The resulting session's permissions are the intersection of the canned policy name and the policy set here. You cannot use this policy to grant more permissions than those allowed by the canned policy name being assumed.
| Params | Value |
@ -45,17 +50,21 @@ An IAM policy in JSON format that you want to use as an inline session policy. T
| *Required* | *No* |
### Response Elements
XML response for this API is similar to [AWS STS AssumeRoleWithWebIdentity](https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRoleWithWebIdentity.html#API_AssumeRoleWithWebIdentity_ResponseElements)
### Errors
XML error response for this API is similar to [AWS STS AssumeRoleWithWebIdentity](https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRoleWithWebIdentity.html#API_AssumeRoleWithWebIdentity_Errors)
## Sample `POST` Request
```
http://minio.cluster:9000?Action=AssumeRoleWithClientGrants&DurationSeconds=3600&Token=eyJ4NXQiOiJOVEF4Wm1NeE5ETXlaRGczTVRVMVpHTTBNekV6T0RKaFpXSTRORE5sWkRVMU9HRmtOakZpTVEiLCJraWQiOiJOVEF4Wm1NeE5ETXlaRGczTVRVMVpHTTBNekV6T0RKaFpXSTRORE5sWkRVMU9HRmtOakZpTVEiLCJhbGciOiJSUzI1NiJ9.eyJhdWQiOiJQb0VnWFA2dVZPNDVJc0VOUm5nRFhqNUF1NVlhIiwiYXpwIjoiUG9FZ1hQNnVWTzQ1SXNFTlJuZ0RYajVBdTVZYSIsImlzcyI6Imh0dHBzOlwvXC9sb2NhbGhvc3Q6OTQ0M1wvb2F1dGgyXC90b2tlbiIsImV4cCI6MTU0MTgwOTU4MiwiaWF0IjoxNTQxODA1OTgyLCJqdGkiOiI2Y2YyMGIwZS1lNGZmLTQzZmQtYTdiYS1kYTc3YTE3YzM2MzYifQ.Jm29jPliRvrK6Os34nSK3rhzIYLFjE__zdVGNng3uGKXGKzP3We_i6NPnhA0szJXMOKglXzUF1UgSz8MctbaxFS8XDusQPVe4LkB_45hwBm6TmBxzui911nt-1RbBLN_jZIlvl2lPrbTUH5hSn9kEkph6seWanTNQpz9tNEoVa6R_OX3kpJqxe8tLQUWw453A1JTwFNhdHa6-f1K8_Q_eEZ_4gOYINQ9t_fhTibdbkXZkJQFLop-Jwoybi9s4nwQU_dATocgcufq5eCeNItQeleT-23lGxIz0X7CiJrJynYLdd-ER0F77SumqEb5iCxhxuf4H7dovwd1kAmyKzLxpw&Version=2011-06-15
```
## Sample Response
```
<?xml version="1.0" encoding="UTF-8"?>
<AssumeRoleWithClientGrantsResponse xmlns="https://sts.amazonaws.com/doc/2011-06-15/">
@ -76,6 +85,7 @@ http://minio.cluster:9000?Action=AssumeRoleWithClientGrants&DurationSeconds=3600
```
## Using ClientGrants API
```
export MINIO_ROOT_USER=minio
export MINIO_ROOT_PASSWORD=minio123
@ -92,13 +102,14 @@ $ go run client-grants.go -cid PoEgXP6uVO45IsENRngDXj5Au5Ya -csec eKsw6z8CtOJVBt
##### Credentials
{
"accessKey": "NUIBORZYTV2HG2BMRSXR",
"secretKey": "qQlP5O7CFPc5m5IXf1vYhuVTFj7BRVJqh0FqZ86S",
"expiration": "2018-08-21T17:10:29-07:00",
"sessionToken": "eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJhY2Nlc3NLZXkiOiJOVUlCT1JaWVRWMkhHMkJNUlNYUiIsImF1ZCI6IlBvRWdYUDZ1Vk80NUlzRU5SbmdEWGo1QXU1WWEiLCJhenAiOiJQb0VnWFA2dVZPNDVJc0VOUm5nRFhqNUF1NVlhIiwiZXhwIjoxNTM0ODk2NjI5LCJpYXQiOjE1MzQ4OTMwMjksImlzcyI6Imh0dHBzOi8vbG9jYWxob3N0Ojk0NDMvb2F1dGgyL3Rva2VuIiwianRpIjoiNjY2OTZjZTctN2U1Ny00ZjU5LWI0MWQtM2E1YTMzZGZiNjA4In0.eJONnVaSVHypiXKEARSMnSKgr-2mlC2Sr4fEGJitLcJF_at3LeNdTHv0_oHsv6ZZA3zueVGgFlVXMlREgr9LXA"
"accessKey": "NUIBORZYTV2HG2BMRSXR",
"secretKey": "qQlP5O7CFPc5m5IXf1vYhuVTFj7BRVJqh0FqZ86S",
"expiration": "2018-08-21T17:10:29-07:00",
"sessionToken": "eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJhY2Nlc3NLZXkiOiJOVUlCT1JaWVRWMkhHMkJNUlNYUiIsImF1ZCI6IlBvRWdYUDZ1Vk80NUlzRU5SbmdEWGo1QXU1WWEiLCJhenAiOiJQb0VnWFA2dVZPNDVJc0VOUm5nRFhqNUF1NVlhIiwiZXhwIjoxNTM0ODk2NjI5LCJpYXQiOjE1MzQ4OTMwMjksImlzcyI6Imh0dHBzOi8vbG9jYWxob3N0Ojk0NDMvb2F1dGgyL3Rva2VuIiwianRpIjoiNjY2OTZjZTctN2U1Ny00ZjU5LWI0MWQtM2E1YTMzZGZiNjA4In0.eJONnVaSVHypiXKEARSMnSKgr-2mlC2Sr4fEGJitLcJF_at3LeNdTHv0_oHsv6ZZA3zueVGgFlVXMlREgr9LXA"
}
```
## Explore Further
- [MinIO Admin Complete Guide](https://docs.min.io/docs/minio-admin-complete-guide.html)
- [The MinIO documentation website](https://docs.min.io)

29
docs/sts/dex.md
View File

@ -23,6 +23,7 @@ time="2020-07-12T20:45:50Z" level=info msg="listening (http) on 0.0.0.0:5556"
```
### Configure MinIO server with Dex
```
~ export MINIO_IDENTITY_OPENID_CLAIM_NAME=name
~ export MINIO_IDENTITY_OPENID_CONFIG_URL=http://127.0.0.1:5556/dex/.well-known/openid-configuration
@ -42,6 +43,7 @@ time="2020-07-12T20:45:50Z" level=info msg="listening (http) on 0.0.0.0:5556"
```
Contents of `allaccess.json`
```json
{
"Version": "2012-10-17",
@ -59,7 +61,8 @@ Contents of `allaccess.json`
}
```
### Visit http://localhost:8080
### Visit <http://localhost:8080>
You will be redirected to dex login screen - click "Login with email", enter username password
> username: admin@example.com
> password: password
@ -67,25 +70,27 @@ You will be redirected to dex login screen - click "Login with email", enter use
and then click "Grant access"
On the browser now you shall see the list of buckets output, along with your temporary credentials obtained from MinIO.
```
{
"buckets": [
"dl.minio.equipment",
"dl.minio.service-fulfillment",
"testbucket"
],
"credentials": {
"AccessKeyID": "Q31CVS1PSCJ4OTK2YVEM",
"SecretAccessKey": "rmDEOKARqKYmEyjWGhmhLpzcncyu7Jf8aZ9bjDic",
"SessionToken": "eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJhY2Nlc3NLZXkiOiJRMzFDVlMxUFNDSjRPVEsyWVZFTSIsImF0X2hhc2giOiI4amItZFE2OXRtZEVueUZaMUttNWhnIiwiYXVkIjoiZXhhbXBsZS1hcHAiLCJlbWFpbCI6ImFkbWluQGV4YW1wbGUuY29tIiwiZW1haWxfdmVyaWZpZWQiOnRydWUsImV4cCI6IjE1OTQ2MDAxODIiLCJpYXQiOjE1OTQ1ODkzODQsImlzcyI6Imh0dHA6Ly8xMjcuMC4wLjE6NTU1Ni9kZXgiLCJuYW1lIjoiYWRtaW4iLCJzdWIiOiJDaVF3T0dFNE5qZzBZaTFrWWpnNExUUmlOek10T1RCaE9TMHpZMlF4TmpZeFpqVTBOallTQld4dlkyRnMifQ.nrbzIJz99Om7TvJ04jnSTmhvlM7aR9hMM1Aqjp2ONJ1UKYCvegBLrTu6cYR968_OpmnAGJ8vkd7sIjUjtR4zbw",
"SignerType": 1
}
"buckets": [
"dl.minio.equipment",
"dl.minio.service-fulfillment",
"testbucket"
],
"credentials": {
"AccessKeyID": "Q31CVS1PSCJ4OTK2YVEM",
"SecretAccessKey": "rmDEOKARqKYmEyjWGhmhLpzcncyu7Jf8aZ9bjDic",
"SessionToken": "eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJhY2Nlc3NLZXkiOiJRMzFDVlMxUFNDSjRPVEsyWVZFTSIsImF0X2hhc2giOiI4amItZFE2OXRtZEVueUZaMUttNWhnIiwiYXVkIjoiZXhhbXBsZS1hcHAiLCJlbWFpbCI6ImFkbWluQGV4YW1wbGUuY29tIiwiZW1haWxfdmVyaWZpZWQiOnRydWUsImV4cCI6IjE1OTQ2MDAxODIiLCJpYXQiOjE1OTQ1ODkzODQsImlzcyI6Imh0dHA6Ly8xMjcuMC4wLjE6NTU1Ni9kZXgiLCJuYW1lIjoiYWRtaW4iLCJzdWIiOiJDaVF3T0dFNE5qZzBZaTFrWWpnNExUUmlOek10T1RCaE9TMHpZMlF4TmpZeFpqVTBOallTQld4dlkyRnMifQ.nrbzIJz99Om7TvJ04jnSTmhvlM7aR9hMM1Aqjp2ONJ1UKYCvegBLrTu6cYR968_OpmnAGJ8vkd7sIjUjtR4zbw",
"SignerType": 1
}
}
```
Now you have successfully configured Dex IdP with MinIO.
> NOTE: Dex supports groups with external connectors so you can use `groups` as policy claim instead of `name`.
```
export MINIO_IDENTITY_OPENID_CLAIM_NAME=groups
```

16
docs/sts/etcd.md
View File

@ -1,11 +1,15 @@
# etcd V3 Quickstart Guide [![Slack](https://slack.min.io/slack?type=svg)](https://slack.min.io)
etcd is a distributed key value store that provides a reliable way to store data across a cluster of machines.
## Get started
### 1. Prerequisites
- Docker 18.03 or above, refer here for [installation](https://docs.docker.com/install/).
### 2. Start etcd
etcd uses [gcr.io/etcd-development/etcd](https://console.cloud.google.com/gcr/images/etcd-development/GLOBAL/etcd) as a primary container registry.
```
@ -32,7 +36,9 @@ rm -rf /tmp/etcd-data.tmp && mkdir -p /tmp/etcd-data.tmp && \
You may also setup etcd with TLS following this documentation [here](https://coreos.com/etcd/docs/latest/op-guide/security.html)
### 3. Setup MinIO with etcd
MinIO server expects environment variable for etcd as `MINIO_ETCD_ENDPOINTS`, this environment variable takes many comma separated entries.
```
export MINIO_ETCD_ENDPOINTS=http://localhost:2379
minio server /data
@ -41,6 +47,7 @@ minio server /data
NOTE: If `etcd` is configured with `Client-to-server authentication with HTTPS client certificates` then you need to use additional envs such as `MINIO_ETCD_CLIENT_CERT` pointing to path to `etcd-client.crt` and `MINIO_ETCD_CLIENT_CERT_KEY` path to `etcd-client.key` .
### 4. Test with MinIO STS API
Once etcd is configured, **any STS configuration** will work including Client Grants, Web Identity or AD/LDAP.
For example, you can configure STS with Client Grants (KeyCloak) using the guides at [MinIO STS Quickstart Guide](https://docs.min.io/docs/minio-sts-quickstart-guide) and [KeyCloak Configuration Guide](https://github.com/minio/minio/blob/master/docs/sts/keycloak.md). Once this is done, STS credentials can be generated:
@ -50,15 +57,16 @@ go run client-grants.go -cid PoEgXP6uVO45IsENRngDXj5Au5Ya -csec eKsw6z8CtOJVBtrO
##### Credentials
{
"accessKey": "IRBLVDGN5QGMDCMO1X8V",
"secretKey": "KzS3UZKE7xqNdtRbKyfcWgxBS6P1G4kwZn4DXKuY",
"expiration": "2018-08-21T15:49:38-07:00",
"sessionToken": "eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJhY2Nlc3NLZXkiOiJJUkJMVkRHTjVRR01EQ01PMVg4ViIsImF1ZCI6IlBvRWdYUDZ1Vk80NUlzRU5SbmdEWGo1QXU1WWEiLCJhenAiOiJQb0VnWFA2dVZPNDVJc0VOUm5nRFhqNUF1NVlhIiwiZXhwIjoxNTM0ODkxNzc4LCJpYXQiOjE1MzQ4ODgxNzgsImlzcyI6Imh0dHBzOi8vbG9jYWxob3N0Ojk0NDMvb2F1dGgyL3Rva2VuIiwianRpIjoiMTg0NDMyOWMtZDY1YS00OGEzLTgyMjgtOWRmNzNmZTgzZDU2In0.4rKsZ8VkZnIS_ALzfTJ9UbEKPFlQVvIyuHw6AWTJcDFDVgQA2ooQHmH9wUDnhXBi1M7o8yWJ47DXP-TLPhwCgQ"
"accessKey": "IRBLVDGN5QGMDCMO1X8V",
"secretKey": "KzS3UZKE7xqNdtRbKyfcWgxBS6P1G4kwZn4DXKuY",
"expiration": "2018-08-21T15:49:38-07:00",
"sessionToken": "eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJhY2Nlc3NLZXkiOiJJUkJMVkRHTjVRR01EQ01PMVg4ViIsImF1ZCI6IlBvRWdYUDZ1Vk80NUlzRU5SbmdEWGo1QXU1WWEiLCJhenAiOiJQb0VnWFA2dVZPNDVJc0VOUm5nRFhqNUF1NVlhIiwiZXhwIjoxNTM0ODkxNzc4LCJpYXQiOjE1MzQ4ODgxNzgsImlzcyI6Imh0dHBzOi8vbG9jYWxob3N0Ojk0NDMvb2F1dGgyL3Rva2VuIiwianRpIjoiMTg0NDMyOWMtZDY1YS00OGEzLTgyMjgtOWRmNzNmZTgzZDU2In0.4rKsZ8VkZnIS_ALzfTJ9UbEKPFlQVvIyuHw6AWTJcDFDVgQA2ooQHmH9wUDnhXBi1M7o8yWJ47DXP-TLPhwCgQ"
}
```
These credentials can now be used to perform MinIO API operations, these credentials automatically expire in 1hr. To understand more about credential expiry duration and client grants STS API read further [here](https://github.com/minio/minio/blob/master/docs/sts/client-grants.md).
## Explore Further
- [MinIO STS Quickstart Guide](https://docs.min.io/docs/minio-sts-quickstart-guide)
- [The MinIO documentation website](https://docs.min.io)

21
docs/sts/keycloak.md
View File

@ -5,9 +5,10 @@ Keycloak is an open source Identity and Access Management solution aimed at mode
## Prerequisites
Configure and install keycloak server by following [Keycloak Installation Guide](https://www.keycloak.org/docs/latest/server_installation/#installing-the-software).
For a quick installation, docker-compose reference configs are also available on the [Keycloak GitHub](https://github.com/keycloak/keycloak-containers/tree/main/docker-compose-examples).
For a quick installation, docker-compose reference configs are also available on the [Keycloak GitHub](https://github.com/keycloak/keycloak-containers/tree/main/docker-compose-examples).
### Configure Keycloak Realm
- Go to Clients
- Click on account
- Settings
@ -38,9 +39,10 @@ For a quick installation, docker-compose reference configs are also available on
- `Claim JSON Type` is `string`
- Save
- Open http://localhost:8080/auth/realms/{your-realm-name}/.well-known/openid-configuration to verify OpenID discovery document, verify it has `authorization_endpoint` and `jwks_uri`
- Open <http://localhost:8080/auth/realms/{your-realm-name}/.well-known/openid-configuration> to verify OpenID discovery document, verify it has `authorization_endpoint` and `jwks_uri`
### Enable Keycloak Admin REST API support
Before being able to authenticate against the Admin REST API using a client_id and a client_secret you need to make sure the client is configured as it follows:
- `account` client_id is a confidential client that belongs to the realm `{realm}`
@ -77,13 +79,15 @@ curl \
```
### Configure MinIO
```
$ export MINIO_ROOT_USER=minio
$ export MINIO_ROOT_PASSWORD=minio123
$ minio server /mnt/export
export MINIO_ROOT_USER=minio
export MINIO_ROOT_PASSWORD=minio123
minio server /mnt/export
```
Here are all the available options to configure OpenID connect
```
mc admin config set myminio/ identity_openid
@ -100,6 +104,7 @@ comment (sentence) optionally add a comment to this setting
```
and ENV based options
```
mc admin config set myminio/ identity_openid --env
@ -116,17 +121,21 @@ MINIO_IDENTITY_OPENID_COMMENT (sentence) optionally add a comment to this
```
Set `identity_openid` config with `config_url`, `client_id` and restart MinIO
```
~ mc admin config set myminio identity_openid config_url="http://localhost:8080/auth/realms/{your-realm-name}/.well-known/openid-configuration" client_id="account"
```
> NOTE: You can configure the `scopes` parameter to restrict the OpenID scopes requested by minio to the IdP, for example, `"openid,policy_role_attribute"`, being `policy_role_attribute` a client_scope / client_mapper that maps a role attribute called policy to a `policy` claim returned by Keycloak
Once successfully set restart the MinIO instance.
```
mc admin service restart myminio
```
### Using WebIdentiy API
Client ID can be found by clicking any of the clients listed [here](http://localhost:8080/auth/admin/master/console/#/realms/minio/clients). If you have followed the above steps docs, the default Client ID will be `account`.
```
@ -156,7 +165,7 @@ These credentials can now be used to perform MinIO API operations.
### Using MinIO Console
- Open MinIO URL on the browser, lets say http://localhost:9000/
- Open MinIO URL on the browser, lets say <http://localhost:9000/>
- Click on `Login with SSO`
- User will be redirected to the Keycloak user login page, upon successful login the user will be redirected to MinIO page and logged in automatically,
the user should see now the buckets and objects they have access to.

62
docs/sts/ldap.md
View File

@ -1,6 +1,7 @@
# AssumeRoleWithLDAPIdentity [![Slack](https://slack.min.io/slack?type=svg)](https://slack.min.io)
## Introduction
MinIO provides a custom STS API that allows integration with LDAP based corporate environments including Microsoft Active Directory. The MinIO server uses a separate LDAP service account to lookup user information. The login flow for a user is as follows:
- User provides their AD/LDAP username and password to the STS API.
@ -21,7 +22,6 @@ To ensure that changes in the LDAP directory are reflected in object storage acc
**Please note that when AD/LDAP is configured, MinIO will not support long term users defined internally.** Only AD/LDAP users (and the root user) are allowed. In addition to this, the server will not support operations on users or groups using `mc admin user` or `mc admin group` commands except `mc admin user info` and `mc admin group info` to list set policies for users and groups. This is because users and groups are defined externally in AD/LDAP.
## Configuring AD/LDAP on MinIO
LDAP STS configuration can be performed via MinIO's standard configuration API (i.e. using `mc admin config set/get` commands) or equivalently via environment variables. For brevity we refer to environment variables here.
@ -116,7 +116,7 @@ export MINIO_IDENTITY_LDAP_TLS_SKIP_VERIFY=on
### Variable substitution in configuration strings
In the configuration variables, `%s` is substituted with the *username* from the STS request and `%d` is substituted with the *distinguished username (user DN)* of the LDAP user. Please see the following table for which configuration variables support these substitution variables:
In the configuration variables, `%s` is substituted with the _username_ from the STS request and `%d` is substituted with the _distinguished username (user DN)_ of the LDAP user. Please see the following table for which configuration variables support these substitution variables:
| Variable | Supported substitutions |
|---------------------------------------------|-------------------------|
@ -146,62 +146,70 @@ mc admin policy set myminio mypolicy group='cn=projectx,ou=groups,ou=hwengg,dc=m
## API Request Parameters
### LDAPUsername
Is AD/LDAP username to login. Application must ask user for this value to successfully obtain rotating access credentials from AssumeRoleWithLDAPIdentity.
| Params | Value |
| :-- | :-- |
| *Type* | *String* |
| *Length Constraints* | *Minimum length of 2. Maximum length of 2048.* |
| *Required* | *Yes* |
| _Type_ | _String_ |
| _Length Constraints_ | _Minimum length of 2. Maximum length of 2048._ |
| _Required_ | _Yes_ |
### LDAPPassword
Is AD/LDAP username password to login. Application must ask user for this value to successfully obtain rotating access credentials from AssumeRoleWithLDAPIdentity.
| Params | Value |
| :-- | :-- |
| *Type* | *String* |
| *Length Constraints* | *Minimum length of 4. Maximum length of 2048.* |
| *Required* | *Yes* |
| _Type_ | _String_ |
| _Length Constraints_ | _Minimum length of 4. Maximum length of 2048._ |
| _Required_ | _Yes_ |
### Version
Indicates STS API version information, the only supported value is '2011-06-15'. This value is borrowed from AWS STS API documentation for compatibility reasons.
| Params | Value |
| :-- | :-- |
| *Type* | *String* |
| *Required* | *Yes* |
| _Type_ | _String_ |
| _Required_ | _Yes_ |
### DurationSeconds
The duration, in seconds. The value can range from 900 seconds (15 minutes) up to 365 days. If value is higher than this setting, then operation fails. By default, the value is set to 3600 seconds.
| Params | Value |
| :-- | :-- |
| *Type* | *Integer* |
| *Valid Range* | *Minimum value of 900. Maximum value of 31536000.* |
| *Required* | *No* |
| _Type_ | _Integer_ |
| _Valid Range_ | _Minimum value of 900. Maximum value of 31536000._ |
| _Required_ | _No_ |
### Policy
An IAM policy in JSON format that you want to use as an inline session policy. This parameter is optional. Passing policies to this operation returns new temporary credentials. The resulting session's permissions are the intersection of the canned policy name and the policy set here. You cannot use this policy to grant more permissions than those allowed by the canned policy name being assumed.
| Params | Value |
| :-- | :-- |
| *Type* | *String* |
| *Valid Range* | *Minimum length of 1. Maximum length of 2048.* |
| *Required* | *No* |
| _Type_ | _String_ |
| _Valid Range_ | _Minimum length of 1. Maximum length of 2048._ |
| _Required_ | _No_ |
### Response Elements
XML response for this API is similar to [AWS STS AssumeRoleWithWebIdentity](https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRoleWithWebIdentity.html#API_AssumeRoleWithWebIdentity_ResponseElements)
### Errors
XML error response for this API is similar to [AWS STS AssumeRoleWithWebIdentity](https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRoleWithWebIdentity.html#API_AssumeRoleWithWebIdentity_Errors)
## Sample `POST` Request
```
http://minio.cluster:9000?Action=AssumeRoleWithLDAPIdentity&LDAPUsername=foouser&LDAPPassword=foouserpassword&Version=2011-06-15&DurationSeconds=7200
```
## Sample Response
```
<?xml version="1.0" encoding="UTF-8"?>
<AssumeRoleWithLDAPIdentityResponse xmlns="https://sts.amazonaws.com/doc/2011-06-15/">
@ -224,17 +232,20 @@ http://minio.cluster:9000?Action=AssumeRoleWithLDAPIdentity&LDAPUsername=foouser
## Using LDAP STS API
With multiple OU hierarchies for users, and multiple group search base DN's.
```
$ export MINIO_ROOT_USER=minio
$ export MINIO_ROOT_PASSWORD=minio123
$ export MINIO_IDENTITY_LDAP_SERVER_ADDR='my.ldap-active-dir-server.com:636'
$ export MINIO_IDENTITY_LDAP_LOOKUP_BIND_DN='cn=admin,dc=min,dc=io'
$ export MINIO_IDENTITY_LDAP_LOOKUP_BIND_PASSWORD=admin
$ export MINIO_IDENTITY_LDAP_GROUP_SEARCH_BASE_DN='dc=minioad,dc=local;dc=somedomain,dc=com'
$ export MINIO_IDENTITY_LDAP_GROUP_SEARCH_FILTER='(&(objectclass=groupOfNames)(member=%d))'
$ minio server ~/test
export MINIO_ROOT_USER=minio
export MINIO_ROOT_PASSWORD=minio123
export MINIO_IDENTITY_LDAP_SERVER_ADDR='my.ldap-active-dir-server.com:636'
export MINIO_IDENTITY_LDAP_LOOKUP_BIND_DN='cn=admin,dc=min,dc=io'
export MINIO_IDENTITY_LDAP_LOOKUP_BIND_PASSWORD=admin
export MINIO_IDENTITY_LDAP_GROUP_SEARCH_BASE_DN='dc=minioad,dc=local;dc=somedomain,dc=com'
export MINIO_IDENTITY_LDAP_GROUP_SEARCH_FILTER='(&(objectclass=groupOfNames)(member=%d))'
minio server ~/test
```
You can make sure it works appropriately using our [example program](https://raw.githubusercontent.com/minio/minio/master/docs/sts/ldap.go):
```
$ go run ldap.go -u foouser -p foopassword
@ -248,5 +259,6 @@ $ go run ldap.go -u foouser -p foopassword
```
## Explore Further
- [MinIO Admin Complete Guide](https://docs.min.io/docs/minio-admin-complete-guide.html)
- [The MinIO documentation website](https://docs.min.io)

7
docs/sts/tls.md
View File

@ -1,6 +1,7 @@
# AssumeRoleWithCertificate [![Slack](https://slack.min.io/slack?type=svg)](https://slack.min.io)
## Introduction
MinIO provides a custom STS API that allows authentication with client X.509 / TLS certificates.
A major advantage of certificate-based authentication compared to other STS authentication methods, like OpenID Connect or LDAP/AD, is that client authentication works without any additional/external component that must be constantly available. Therefore, certificate-based authentication may provide better availability / lower operational complexity.
@ -17,11 +18,13 @@ MINIO_IDENTITY_TLS_SKIP_VERIFY (on|off) trust client certificates without ve
```
The MinIO TLS STS API is disabled by default. However, it can be *enabled* by setting environment variable:
```
export MINIO_IDENTITY_TLS_ENABLE=on
```
## Example
MinIO exposes a custom S3 STS API endpoint as `Action=AssumeRoleWithCertificate`. A client has to send an HTTP `POST` request to `https://<host>:<port>?Action=AssumeRoleWithCertificate&Version=2011-06-15`. Since the authentication and authorization happens via X.509 certificates the client has to send the request over **TLS** and has to provide
a client certificate.
@ -55,6 +58,7 @@ A client can request temp. S3 credentials via the STS API. It can authenticate v
In case of certificate-based authentication, MinIO has to map the client-provided certificate to an S3 policy. MinIO does this via the subject common name field of the X.509 certificate. So, MinIO will associate a certificate with a subject `CN = foobar` to a S3 policy named `foobar`.
The following self-signed certificate is issued for `consoleAdmin`. So, MinIO would associate it with the pre-defined `consoleAdmin` policy.
```
Certificate:
Data:
@ -87,6 +91,7 @@ Certificate:
7d:ab:b8:e9:75:ec:b4:39:fb:c8:cf:53:16:5b:1f:15:b6:7f:
5a:d1:35:2d:fc:31:3a:10:e7:0c
```
> Observe the `Subject: CN = consoleAdmin` field.
Also, note that the certificate has to contain the `Extended Key Usage: TLS Web Client Authentication`. Otherwise, MinIO would not accept the certificate as client certificate.
@ -106,7 +111,7 @@ Further, the temp. S3 credentials will never out-live the client certificate. Fo
*Applications that use direct S3 API will work fine, however interactive users uploading content using (when POSTing to the presigned URL an app generates) a popup becomes visible on browser to provide client certs, you would have to manually cancel and continue. This may be annoying to use but there is no workaround for now.*
## Explore Further
- [MinIO Admin Complete Guide](https://docs.min.io/docs/minio-admin-complete-guide.html)
- [The MinIO documentation website](https://docs.min.io)

20
docs/sts/web-identity.md
View File

@ -8,14 +8,16 @@ By default, the temporary security credentials created by AssumeRoleWithWebIdent
## Access Control Policies
MinIO's AssumeRoleWithWebIdentity supports specifying access control policies in two ways:
MinIO's AssumeRoleWithWebIdentity supports specifying access control policies in two ways:
1. Role Policy (Recommended): When specified, all users authenticating via this API are authorized to (only) use the specified role policy. The policy to associate with such users is specified when configuring OpenID provider in the server, via the `role_policy` configuration parameter or the `MINIO_IDENTITY_OPENID_ROLE_POLICY` environment variable. The value is a comma-separated list of IAM access policy names already defined in the server. In this situation, the server prints a role ARN at startup that must be specified as a `RoleARN` API request parameter in the STS AssumeRoleWithWebIdentity API call.
2. `id_token` claims: When the role policy is not configured, MinIO looks for a specific claim in the `id_token` (JWT) returned by the OpenID provider. The default claim is `policy` and can be overridden by the `claim_name` configuration parameter or the `MINIO_IDENTITY_OPENID_CLAIM_NAME` environment variable. The claim value can be a string (comma-separated list) or an array of IAM access policy names defined in the server. A `RoleARN` API request parameter *must not* be specified in the STS AssumeRoleWithWebIdentity API call.
## API Request Parameters
### WebIdentityToken
The OAuth 2.0 id_token that is provided by the web identity provider. Application must get this token by authenticating the user who is using your application with a web identity provider before the application makes an AssumeRoleWithWebIdentity call.
| Params | Value |
@ -25,6 +27,7 @@ The OAuth 2.0 id_token that is provided by the web identity provider. Applicatio
| *Required* | *Yes* |
### WebIdentityAccessToken (MinIO Extension)
There are situations when identity provider does not provide user claims in `id_token` instead it needs to be retrieved from UserInfo endpoint, this extension is only useful in this scenario. This is rare so use it accordingly depending on your Identity provider implementation. `access_token` is available as part of the OIDC authentication flow similar to `id_token`.
| Params | Value |
@ -33,6 +36,7 @@ There are situations when identity provider does not provide user claims in `id_
| *Required* | *No* |
### RoleARN
The role ARN to use. This must be specified if and only if the web identity provider is configured with a role policy.
| Params | Value |
@ -41,6 +45,7 @@ The role ARN to use. This must be specified if and only if the web identity prov
| *Required* | *No* |
### Version
Indicates STS API version information, the only supported value is '2011-06-15'. This value is borrowed from AWS STS API documentation for compatibility reasons.
| Params | Value |
@ -49,6 +54,7 @@ Indicates STS API version information, the only supported value is '2011-06-15'.
| *Required* | *Yes* |
### DurationSeconds
The duration, in seconds. The value can range from 900 seconds (15 minutes) up to 365 days. If value is higher than this setting, then operation fails. By default, the value is set to 3600 seconds. If no *DurationSeconds* is specified expiry seconds is obtained from *WebIdentityToken*.
| Params | Value |
@ -58,6 +64,7 @@ The duration, in seconds. The value can range from 900 seconds (15 minutes) up t
| *Required* | *No* |
### Policy
An IAM policy in JSON format that you want to use as an inline session policy. This parameter is optional. Passing policies to this operation returns new temporary credentials. The resulting session's permissions are the intersection of the canned policy name and the policy set here. You cannot use this policy to grant more permissions than those allowed by the canned policy name being assumed.
| Params | Value |
@ -67,17 +74,21 @@ An IAM policy in JSON format that you want to use as an inline session policy. T
| *Required* | *No* |
### Response Elements
XML response for this API is similar to [AWS STS AssumeRoleWithWebIdentity](https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRoleWithWebIdentity.html#API_AssumeRoleWithWebIdentity_ResponseElements)
### Errors
XML error response for this API is similar to [AWS STS AssumeRoleWithWebIdentity](https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRoleWithWebIdentity.html#API_AssumeRoleWithWebIdentity_Errors)
## Sample `POST` Request
```
http://minio.cluster:9000?Action=AssumeRoleWithWebIdentity&DurationSeconds=3600&WebIdentityToken=eyJ4NXQiOiJOVEF4Wm1NeE5ETXlaRGczTVRVMVpHTTBNekV6T0RKaFpXSTRORE5sWkRVMU9HRmtOakZpTVEiLCJraWQiOiJOVEF4Wm1NeE5ETXlaRGczTVRVMVpHTTBNekV6T0RKaFpXSTRORE5sWkRVMU9HRmtOakZpTVEiLCJhbGciOiJSUzI1NiJ9.eyJhdWQiOiJQb0VnWFA2dVZPNDVJc0VOUm5nRFhqNUF1NVlhIiwiYXpwIjoiUG9FZ1hQNnVWTzQ1SXNFTlJuZ0RYajVBdTVZYSIsImlzcyI6Imh0dHBzOlwvXC9sb2NhbGhvc3Q6OTQ0M1wvb2F1dGgyXC90b2tlbiIsImV4cCI6MTU0MTgwOTU4MiwiaWF0IjoxNTQxODA1OTgyLCJqdGkiOiI2Y2YyMGIwZS1lNGZmLTQzZmQtYTdiYS1kYTc3YTE3YzM2MzYifQ.Jm29jPliRvrK6Os34nSK3rhzIYLFjE__zdVGNng3uGKXGKzP3We_i6NPnhA0szJXMOKglXzUF1UgSz8MctbaxFS8XDusQPVe4LkB_45hwBm6TmBxzui911nt-1RbBLN_jZIlvl2lPrbTUH5hSn9kEkph6seWanTNQpz9tNEoVa6R_OX3kpJqxe8tLQUWw453A1JTwFNhdHa6-f1K8_Q_eEZ_4gOYINQ9t_fhTibdbkXZkJQFLop-Jwoybi9s4nwQU_dATocgcufq5eCeNItQeleT-23lGxIz0X7CiJrJynYLdd-ER0F77SumqEb5iCxhxuf4H7dovwd1kAmyKzLxpw&Version=2011-06-15
```
## Sample Response
```
<?xml version="1.0" encoding="UTF-8"?>
<AssumeRoleWithWebIdentityResponse xmlns="https://sts.amazonaws.com/doc/2011-06-15/">
@ -98,6 +109,7 @@ http://minio.cluster:9000?Action=AssumeRoleWithWebIdentity&DurationSeconds=3600&
```
## Using WebIdentity API
```
export MINIO_ROOT_USER=minio
export MINIO_ROOT_PASSWORD=minio123
@ -109,6 +121,7 @@ minio server /mnt/export
```
or using `mc`
```
mc admin config get myminio identity_openid
identity_openid config_url=https://accounts.google.com/.well-known/openid-configuration client_id=843351d4-1080-11ea-aa20-271ecba3924a
@ -123,6 +136,7 @@ $ go run web-identity.go -cid 204367807228-ok7601k6gj1pgge7m09h7d79co8p35xx.apps
```
> NOTE: for a reasonable test outcome, make sure the assumed user has at least permission/policy to list all buckets. That policy would look like below:
```
{
"version": "2012-10-17",
@ -142,13 +156,14 @@ $ go run web-identity.go -cid 204367807228-ok7601k6gj1pgge7m09h7d79co8p35xx.apps
## Authorization Flow
- Visit http://localhost:8080, login will direct the user to the Google OAuth2 Auth URL to obtain a permission grant.
- Visit <http://localhost:8080>, login will direct the user to the Google OAuth2 Auth URL to obtain a permission grant.
- The redirection URI (callback handler) receives the OAuth2 callback, verifies the state parameter, and obtains a Token.
- Using the id_token the callback handler further talks to Google OAuth2 Token URL to obtain an JWT id_token.
- Once obtained the JWT id_token is further sent to STS endpoint i.e MinIO to retrieve temporary credentials.
- Temporary credentials are displayed on the browser upon successful retrieval.
## Using MinIO Console
To support WebIdentity based login for MinIO Console, set openid configuration and restart MinIO
```
@ -170,5 +185,6 @@ JWT token returned by the Identity Provider should include a custom claim for th
- Upon successful login on Identity Provider page the user will be automatically logged into MinIO Console.
## Explore Further
- [MinIO Admin Complete Guide](https://docs.min.io/docs/minio-admin-complete-guide.html)
- [The MinIO documentation website](https://docs.min.io)

23
docs/sts/wso2.md
View File

@ -1,12 +1,16 @@
# WSO2 Quickstart Guide [![Slack](https://slack.min.io/slack?type=svg)](https://slack.min.io)
WSO2 is an Identity Server open source and is released under Apache Software License Version 2.0, this document covers configuring WSO2 to be used as an identity provider for MinIO server STS API.
## Get started
### 1. Prerequisites
- JAVA 1.8 and above installed already and JAVA_HOME points to JAVA 1.8 installation.
- Download WSO2 follow their [installation guide](https://docs.wso2.com/display/IS540/Installation+Guide).
### 2. Configure WSO2
Once WSO2 is up and running, configure WSO2 to generate Self contained id_tokens. In OAuth 2.0 specification there are primarily two ways to provide id_tokens
1. The id_token is an identifier that is hard to guess. For example, a randomly generated string of sufficient length, that the server handling the protected resource can use to lookup the associated authorization information.
@ -15,12 +19,15 @@ Once WSO2 is up and running, configure WSO2 to generate Self contained id_tokens
WSO2 generates tokens in first style by default, but if to be used with MinIO we should configure WSO2 to provide JWT tokens instead.
### 3. Generate Self-contained Access Tokens
By default, a UUID is issued as an id_token in WSO2 Identity Server, which is of the first type above. But, it also can be configured to issue a self-contained id_token (JWT), which is of the second type above.
- Open the `<IS_HOME>/repository/conf/identity/identity.xml` file and uncomment the following entry under `<OAuth>` element.
```
<IdentityOAuthTokenGenerator>org.wso2.carbon.identity.oauth2.token.JWTTokenIssuer</IdentityOAuthTokenGenerator>
```
- Restart the server.
- Configure an [OAuth service provider](https://docs.wso2.com/display/IS540/Adding+and+Configuring+a+Service+Provider).
- Initiate an id_token request to the WSO2 Identity Server, over a known [grant type](https://docs.wso2.com/display/IS540/OAuth+2.0+Grant+Types). For example, the following cURL command illustrates the syntax of an id_token request that can be initiated over the [Client Credentials Grant](https://docs.wso2.com/display/IS540/Client+Credentials+Grant) grant type.
@ -31,16 +38,19 @@ By default, a UUID is issued as an id_token in WSO2 Identity Server, which is of
- By default, `<IS_HTTPS_PORT>` has been set to 9443. However, if the port offset has been incremented by n, the default port value needs to be incremented by n.
Request
```
curl -u <CLIENT_ID>:<CLIENT_SECRET> -k -d "grant_type=client_credentials" -H "Content-Type:application/x-www-form-urlencoded" https://<IS_HOST>:<IS_HTTPS_PORT>/oauth2/token
```
Example:
```
curl -u PoEgXP6uVO45IsENRngDXj5Au5Ya:eKsw6z8CtOJVBtrOWvhRWL4TUCga -k -d "grant_type=client_credentials" -H "Content-Type:application/x-www-form-urlencoded" https://localhost:9443/oauth2/token
```
In response, the self-contained JWT id_token will be returned as shown below.
```
{
"id_token": "eyJ4NXQiOiJOVEF4Wm1NeE5ETXlaRGczTVRVMVpHTTBNekV6T0RKaFpXSTRORE5sWkRVMU9HRmtOakZpTVEiLCJraWQiOiJOVEF4Wm1NeE5ETXlaRGczTVRVMVpHTTBNekV6T0RKaFpXSTRORE5sWkRVMU9HRmtOakZpTVEiLCJhbGciOiJSUzI1NiJ9.eyJhdWQiOiJQb0VnWFA2dVZPNDVJc0VOUm5nRFhqNUF1NVlhIiwiYXpwIjoiUG9FZ1hQNnVWTzQ1SXNFTlJuZ0RYajVBdTVZYSIsImlzcyI6Imh0dHBzOlwvXC9sb2NhbGhvc3Q6OTQ0M1wvb2F1dGgyXC90b2tlbiIsImV4cCI6MTUzNDg5MTc3OCwiaWF0IjoxNTM0ODg4MTc4LCJqdGkiOiIxODQ0MzI5Yy1kNjVhLTQ4YTMtODIyOC05ZGY3M2ZlODNkNTYifQ.ELZ8ujk2Xp9xTGgMqnCa5ehuimaAPXWlSCW5QeBbTJIT4M5OB_2XEVIV6p89kftjUdKu50oiYe4SbfrxmLm6NGSGd2qxkjzJK3SRKqsrmVWEn19juj8fz1neKtUdXVHuSZu6ws_bMDy4f_9hN2Jv9dFnkoyeNT54r4jSTJ4A2FzN2rkiURheVVsc8qlm8O7g64Az-5h4UGryyXU4zsnjDCBKYk9jdbEpcUskrFMYhuUlj1RWSASiGhHHHDU5dTRqHkVLIItfG48k_fb-ehU60T7EFWH1JBdNjOxM9oN_yb0hGwOjLUyCUJO_Y7xcd5F4dZzrBg8LffFmvJ09wzHNtQ",
@ -50,6 +60,7 @@ In response, the self-contained JWT id_token will be returned as shown below.
```
### 4. JWT Claims
The id_token received is a signed JSON Web Token (JWT). Use a JWT decoder to decode the id_token to access the payload of the token that includes following JWT claims:
| Claim Name | Type | Claim Value |
@ -67,7 +78,9 @@ Using the above `id_token` we can perform an STS request to MinIO to get tempora
**We recommend setting `policy` as a custom claim for the JWT service provider follow [here](https://docs.wso2.com/display/IS550/Configuring+Claims+for+a+Service+Provider) and [here](https://docs.wso2.com/display/IS550/Handling+Custom+Claims+with+the+JWT+Bearer+Grant+Type) for relevant docs on how to configure claims for a service provider.**
### 5. Setup MinIO with OpenID configuration URL
MinIO server expects environment variable for OpenID configuration url as `MINIO_IDENTITY_OPENID_CONFIG_URL`, this environment variable takes a single entry.
```
export MINIO_IDENTITY_OPENID_CONFIG_URL=https://localhost:9443/oauth2/oidcdiscovery/.well-known/openid-configuration
export MINIO_IDENTITY_OPENID_CLIENT_ID="843351d4-1080-11ea-aa20-271ecba3924a"
@ -75,20 +88,22 @@ minio server /mnt/data
```
Assuming that MinIO server is configured to support STS API by following the doc [MinIO STS Quickstart Guide](https://docs.min.io/docs/minio-sts-quickstart-guide), execute the following command to temporary credentials from MinIO server.
```
go run client-grants.go -cid PoEgXP6uVO45IsENRngDXj5Au5Ya -csec eKsw6z8CtOJVBtrOWvhRWL4TUCga
##### Credentials
{
"accessKey": "IRBLVDGN5QGMDCMO1X8V",
"secretKey": "KzS3UZKE7xqNdtRbKyfcWgxBS6P1G4kwZn4DXKuY",
"expiration": "2018-08-21T15:49:38-07:00",
"sessionToken": "eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJhY2Nlc3NLZXkiOiJJUkJMVkRHTjVRR01EQ01PMVg4ViIsImF1ZCI6IlBvRWdYUDZ1Vk80NUlzRU5SbmdEWGo1QXU1WWEiLCJhenAiOiJQb0VnWFA2dVZPNDVJc0VOUm5nRFhqNUF1NVlhIiwiZXhwIjoxNTM0ODkxNzc4LCJpYXQiOjE1MzQ4ODgxNzgsImlzcyI6Imh0dHBzOi8vbG9jYWxob3N0Ojk0NDMvb2F1dGgyL3Rva2VuIiwianRpIjoiMTg0NDMyOWMtZDY1YS00OGEzLTgyMjgtOWRmNzNmZTgzZDU2In0.4rKsZ8VkZnIS_ALzfTJ9UbEKPFlQVvIyuHw6AWTJcDFDVgQA2ooQHmH9wUDnhXBi1M7o8yWJ47DXP-TLPhwCgQ"
"accessKey": "IRBLVDGN5QGMDCMO1X8V",
"secretKey": "KzS3UZKE7xqNdtRbKyfcWgxBS6P1G4kwZn4DXKuY",
"expiration": "2018-08-21T15:49:38-07:00",
"sessionToken": "eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJhY2Nlc3NLZXkiOiJJUkJMVkRHTjVRR01EQ01PMVg4ViIsImF1ZCI6IlBvRWdYUDZ1Vk80NUlzRU5SbmdEWGo1QXU1WWEiLCJhenAiOiJQb0VnWFA2dVZPNDVJc0VOUm5nRFhqNUF1NVlhIiwiZXhwIjoxNTM0ODkxNzc4LCJpYXQiOjE1MzQ4ODgxNzgsImlzcyI6Imh0dHBzOi8vbG9jYWxob3N0Ojk0NDMvb2F1dGgyL3Rva2VuIiwianRpIjoiMTg0NDMyOWMtZDY1YS00OGEzLTgyMjgtOWRmNzNmZTgzZDU2In0.4rKsZ8VkZnIS_ALzfTJ9UbEKPFlQVvIyuHw6AWTJcDFDVgQA2ooQHmH9wUDnhXBi1M7o8yWJ47DXP-TLPhwCgQ"
}
```
These credentials can now be used to perform MinIO API operations, these credentials automatically expire in 1hr. To understand more about credential expiry duration and client grants STS API read further [here](https://github.com/minio/minio/blob/master/docs/sts/client-grants.md).
## Explore Further
- [MinIO STS Quickstart Guide](https://docs.min.io/docs/minio-sts-quickstart-guide)
- [The MinIO documentation website](https://docs.min.io)

4
docs/throttle/README.md
View File

@ -8,7 +8,9 @@ MinIO server allows to throttle incoming requests:
These values are enabled using server's configuration or environment variables.
## Examples
### Configuring connection limit
If you have traditional spinning (hdd) drives, some applications with high concurrency might require MinIO cluster to be tuned such that to avoid random I/O on the drives. The way to convert high concurrent I/O into a sequential I/O is by reducing the number of concurrent operations allowed per cluster. This allows MinIO cluster to be operationally resilient to such workloads, while also making sure the drives are at optimal efficiency and responsive.
Example: Limit a MinIO cluster to accept at max 1600 simultaneous S3 API requests across all nodes of the cluster.
@ -30,6 +32,7 @@ mc admin service restart myminio/
> NOTE: A zero value of `requests_max` means MinIO will automatically calculate requests based on available RAM size and that is the default behavior.
### Configuring connection (wait) deadline
This value works in conjunction with max connection setting, setting this value allows for long waiting requests to quickly time out when there is no slot available to perform the request.
This will reduce the pileup of waiting requests when clients are not configured with timeouts. Default wait time is *10 seconds* if *MINIO_API_REQUESTS_MAX* is enabled. This may need to be tuned to your application needs.
@ -50,4 +53,3 @@ or
mc admin config set myminio/ api requests_max=1600 requests_deadline=2m
mc admin service restart myminio/
```

34
docs/tls/README.md
View File

@ -7,24 +7,26 @@ This guide explains how to configure MinIO Server with TLS certificates on Linux
3. [Generate and use Self-signed Keys and Certificates with MinIO](#generate-use-self-signed-keys-certificates)
4. [Install Certificates from Third-party CAs](#install-certificates-from-third-party-cas)
## <a name="install-minio-server"></a>1. Install MinIO Server
## 1. Install MinIO Server
Install MinIO Server using the instructions in the [MinIO Quickstart Guide](https://docs.min.io/docs/minio-quickstart-guide).
## <a name="use-an-existing-key-and-certificate-with-minio"></a>2. Use an Existing Key and Certificate with MinIO
## 2. Use an Existing Key and Certificate with MinIO
This section describes how to use a private key and public certificate that have been obtained from a certificate authority (CA). If these files have not been obtained, skip to [3. Generate Self-signed Certificates](#generate-use-self-signed-keys-certificates) or generate them with [Let's Encrypt](https://letsencrypt.org) using these instructions: [Generate Let's Encrypt certificate using Certbot for MinIO](https://docs.min.io/docs/generate-let-s-encypt-certificate-using-concert-for-minio.html).
Copy the existing private key and public certificate to the `certs` directory. The default certs directory is:
* **Linux:** `${HOME}/.minio/certs`
* **Windows:** `%%USERPROFILE%%\.minio\certs`
**Note:**
* Location of custom certs directory can be specified using `--certs-dir` command line option.
* Inside the `certs` directory, the private key must by named `private.key` and the public key must be named `public.crt`.
* A certificate signed by a CA contains information about the issued identity (e.g. name, expiry, public key) and any intermediate certificates. The root CA is not included.
## <a name="generate-use-self-signed-keys-certificates"></a>3. Generate and use Self-signed Keys and Certificates with MinIO
## 3. Generate and use Self-signed Keys and Certificates with MinIO
This section describes how to generate a self-signed certificate using various tools:
@ -34,10 +36,11 @@ This section describes how to generate a self-signed certificate using various t
* 3.4 [Use GnuTLS (for Windows) to Generate a Certificate](#using-gnu-tls)
**Note:**
* MinIO only supports keys and certificates in PEM format on Linux and Windows.
* MinIO doesn't currently support PFX certificates.
### <a name="using-go"></a>3.1 Use `certgen` to Generate a Certificate
### 3.1 Use `certgen` to Generate a Certificate
Download [`certgen`](https://github.com/minio/certgen/releases/latest) for your specific operating system and platform.
@ -54,7 +57,7 @@ A response similar to this one should be displayed:
2018/11/21 10:16:18 wrote private.key
```
### <a name="using-open-ssl"></a>3.2 Use OpenSSL to Generate a Certificate
### 3.2 Use OpenSSL to Generate a Certificate
Use one of the following methods to generate a certificate using `openssl`:
@ -62,7 +65,7 @@ Use one of the following methods to generate a certificate using `openssl`:
* 3.2.2 [Generate a private key with RSA](#generate-private-key-with-rsa)
* 3.2.3 [Generate a self-signed certificate](#generate-a-self-signed-certificate)
#### 3.2.1 <a name="generate-private-key-with-ecdsa"></a>Generate a private key with ECDSA.
#### 3.2.1 Generate a private key with ECDSA
Use the following command to generate a private key with ECDSA:
@ -85,13 +88,14 @@ openssl ecparam -genkey -name prime256v1 | openssl ec -aes256 -out private.key -
**Note:** NIST curves P-384 and P-521 are not currently supported.
#### 3.2.2 <a name="generate-private-key-with-rsa"></a>Generate a private key with RSA.
#### 3.2.2 Generate a private key with RSA
Use the following command to generate a private key with RSA:
```sh
openssl genrsa -out private.key 2048
```
A response similar to this one should be displayed:
```
@ -119,7 +123,7 @@ The default OpenSSL format for private encrypted keys is PKCS-8, but MinIO only
openssl rsa -in private-pkcs8-key.key -aes256 -passout pass:PASSWORD -out private.key
```
#### <a name="generate-a-self-signed-certificate"></a>3.2.3 Generate a self-signed certificate.
#### 3.2.3 Generate a self-signed certificate
Create a file named `openssl.conf` with the content below. Set `IP.1` and/or `DNS.1` to point to the correct IP/DNS addresses:
@ -151,11 +155,12 @@ Run `openssl` by specifying the configuration file and enter a passphrase if pro
openssl req -new -x509 -nodes -days 730 -keyout private.key -out public.crt -config openssl.conf
```
### <a name="using-gnu-tls"></a>3.3 Use GnuTLS (for Windows) to Generate a Certificate
### 3.3 Use GnuTLS (for Windows) to Generate a Certificate
This section describes how to use GnuTLS on Windows to generate a certificate.
#### 3.3.1 Install and configure GnuTLS.
#### 3.3.1 Install and configure GnuTLS
Download and decompress the Windows version of GnuTLS from [here](http://www.gnutls.org/download.html).
Use PowerShell to add the path of the extracted GnuTLS binary to the system path:
@ -166,7 +171,8 @@ setx path "%path%;C:\Users\MyUser\Downloads\gnutls-3.4.9-w64\bin"
**Note:** PowerShell may need to be restarted for this change to take effect.
#### 3.3.2 Generate a private key:
#### 3.3.2 Generate a private key
Run the following command to generate a private `.key` file:
```
@ -179,7 +185,7 @@ A response similar to this one should be displayed:
Generating a 3072 bit RSA private key...
```
#### 3.3.3 Generate a public certificate:
#### 3.3.3 Generate a public certificate
Create a file called `cert.cnf` with the content below. This file contains all of the information necessary to generate a certificate using `certtool.exe`:
@ -224,13 +230,15 @@ Run `certtool.exe` and specify the configuration file to generate a certificate:
certtool.exe --generate-self-signed --load-privkey private.key --template cert.cnf --outfile public.crt
```
## <a name="install-certificates-from-third-party-cas"></a>4. Install Certificates from Third-party CAs
## 4. Install Certificates from Third-party CAs
MinIO can connect to other servers, including MinIO nodes or other server types such as NATs and Redis. If these servers use certificates that were not registered with a known CA, add trust for these certificates to MinIO Server by placing these certificates under one of the following MinIO configuration paths:
* **Linux:** `~/.minio/certs/CAs/`
* **Windows**: `C:\Users\<Username>\.minio\certs\CAs`
# Explore Further
* [TLS Configuration for MinIO server on Kubernetes](https://github.com/minio/minio/tree/master/docs/tls/kubernetes)
* [MinIO Client Complete Guide](https://docs.min.io/docs/minio-client-complete-guide)
* [Generate Let's Encrypt Certificate](https://docs.min.io/docs/generate-let-s-encypt-certificate-using-concert-for-minio)

8
docs/tls/kubernetes/README.md
View File

@ -16,8 +16,8 @@ For testing purposes, here is [how to create self-signed certificates](https://g
## 2. Create Kubernetes secret
[Kubernetes secrets](https://kubernetes.io/docs/concepts/configuration/secret) are intended to hold sensitive information.
We'll use secrets to hold the TLS certificate and key. To create a secret, update the paths to `private.key` and `public.crt`
[Kubernetes secrets](https://kubernetes.io/docs/concepts/configuration/secret) are intended to hold sensitive information.
We'll use secrets to hold the TLS certificate and key. To create a secret, update the paths to `private.key` and `public.crt`
below.
Then type
@ -26,7 +26,7 @@ Then type
kubectl create secret generic tls-ssl-minio --from-file=path/to/private.key --from-file=path/to/public.crt
```
Cross check if the secret is created successfully using
Cross check if the secret is created successfully using
```sh
kubectl get secrets
@ -67,5 +67,5 @@ Here the name of `volumeMount` should match the name of `volume` created previou
the MinIO server's config sub-directory that is used to store certificates. By default, the location is
`/<user-running-minio>/.minio/certs`.
*Tip*: In a standard Kubernetes configuration, this will be `/root/.minio/certs`. Kubernetes will mount the secrets volume read-only,
*Tip*: In a standard Kubernetes configuration, this will be `/root/.minio/certs`. Kubernetes will mount the secrets volume read-only,
so avoid setting `mountPath` to a path that MinIO server expects to write to.

55
helm/minio/README.md
View File

@ -1,30 +1,28 @@
# MinIO Helm Chart
[![Slack](https://slack.min.io/slack?type=svg)](https://slack.min.io) [![license](https://img.shields.io/badge/license-AGPL%20V3-blue)](https://github.com/minio/minio/blob/master/LICENSE)
MinIO is a High Performance Object Storage released under GNU Affero General Public License v3.0. It is API compatible with Amazon S3 cloud storage service. Use MinIO to build high performance infrastructure for machine learning, analytics and application data workloads.
For more detailed documentation please visit [here](https://docs.minio.io/)
Introduction
------------
## Introduction
This chart bootstraps MinIO Cluster on [Kubernetes](http://kubernetes.io) using the [Helm](https://helm.sh) package manager.
Prerequisites
-------------
## Prerequisites
- Helm cli with Kubernetes cluster configured.
- PV provisioner support in the underlying infrastructure. (We recommend using https://github.com/minio/direct-csi)
- PV provisioner support in the underlying infrastructure. (We recommend using <https://github.com/minio/direct-csi>)
- Use Kubernetes version v1.19 and later for best experience.
Configure MinIO Helm repo
--------------------
## Configure MinIO Helm repo
```bash
helm repo add minio https://charts.min.io/
```
Installing the Chart
--------------------
### Installing the Chart
Install this chart using:
@ -34,8 +32,7 @@ helm install --namespace minio --set rootUser=rootuser,rootPassword=rootpass123
The command deploys MinIO on the Kubernetes cluster in the default configuration. The [configuration](#configuration) section lists the parameters that can be configured during installation.
Upgrading the Chart
-------------------
### Upgrading the Chart
You can use Helm to update MinIO version in a live release. Assuming your release is named as `my-release`, get the values using the command:
@ -51,8 +48,7 @@ helm upgrade -f old_values.yaml my-release minio/minio
Default upgrade strategies are specified in the `values.yaml` file. Update these fields if you'd like to use a different strategy.
Configuration
-------------
### Configuration
Refer the [Values file](./values.yaml) for all the possible config fields.
@ -70,8 +66,7 @@ Alternately, you can provide a YAML file that specifies parameter values while i
helm install --name my-release -f values.yaml minio/minio
```
Persistence
-----------
### Persistence
This chart provisions a PersistentVolumeClaim and mounts corresponding persistent volume to default location `/export`. You'll need physical storage available in the Kubernetes cluster for this to work. If you'd rather use `emptyDir`, disable PersistentVolumeClaim by:
@ -81,8 +76,7 @@ helm install --set persistence.enabled=false minio/minio
> *"An emptyDir volume is first created when a Pod is assigned to a Node, and exists as long as that Pod is running on that node. When a Pod is removed from a node for any reason, the data in the emptyDir is deleted forever."*
Existing PersistentVolumeClaim
------------------------------
### Existing PersistentVolumeClaim
If a Persistent Volume Claim already exists, specify it during installation.
@ -94,8 +88,7 @@ If a Persistent Volume Claim already exists, specify it during installation.
helm install --set persistence.existingClaim=PVC_NAME minio/minio
```
NetworkPolicy
-------------
### NetworkPolicy
To enable network policy for MinIO,
install [a networking plugin that implements the Kubernetes
@ -103,7 +96,7 @@ NetworkPolicy spec](https://kubernetes.io/docs/tasks/administer-cluster/declare-
and set `networkPolicy.enabled` to `true`.
For Kubernetes v1.5 & v1.6, you must also turn on NetworkPolicy by setting
the DefaultDeny namespace annotation. Note: this will enforce policy for _all_ pods in the namespace:
the DefaultDeny namespace annotation. Note: this will enforce policy for *all* pods in the namespace:
```
kubectl annotate namespace default "net.beta.kubernetes.io/network-policy={\"ingress\":{\"isolation\":\"DefaultDeny\"}}"
@ -115,33 +108,33 @@ For more precise policy, set `networkPolicy.allowExternal=true`. This will
only allow pods with the generated client label to connect to MinIO.
This label will be displayed in the output of a successful install.
Existing secret
---------------
### Existing secret
Instead of having this chart create the secret for you, you can supply a preexisting secret, much
like an existing PersistentVolumeClaim.
First, create the secret:
```bash
kubectl create secret generic my-minio-secret --from-literal=rootUser=foobarbaz --from-literal=rootPassword=foobarbazqux
```
Then install the chart, specifying that you want to use an existing secret:
```bash
helm install --set existingSecret=my-minio-secret minio/minio
```
The following fields are expected in the secret:
| .data.<key> in Secret | Corresponding variable | Description | Required |
|:----------------------|:-----------------------|:---------------|:---------|
| `rootUser` | `rootUser` | Root user. | yes |
| `rootPassword` | `rootPassword` | Root password. | yes |
| .data.\<key\> in Secret | Corresponding variable | Description | Required |
|:------------------------|:-----------------------|:---------------|:---------|
| `rootUser` | `rootUser` | Root user. | yes |
| `rootPassword` | `rootPassword` | Root password. | yes |
All corresponding variables will be ignored in values file.
Configure TLS
-------------
### Configure TLS
To enable TLS for MinIO containers, acquire TLS certificates from a CA or create self-signed certificates. While creating / acquiring certificates ensure the corresponding domain names are set as per the standard [DNS naming conventions](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#pod-identity) in a Kubernetes StatefulSet (for a distributed MinIO setup). Then create a secret using
@ -181,8 +174,7 @@ or
--set trustedCertsSecret=minio-trusted-certs
```
Create buckets after install
---------------------------
### Create buckets after install
Install the chart, specifying the buckets you want to create after install:
@ -196,8 +188,7 @@ Description of the configuration parameters used above -
- `buckets[].policy` - can be one of none|download|upload|public
- `buckets[].purge` - purge if bucket exists already
Uninstalling the Chart
----------------------
## Uninstalling the Chart
Assuming your release is named as `my-release`, delete it using the command: