With this change, MinIO's ILM supports transitioning objects to a remote tier.
This change includes support for Azure Blob Storage, AWS S3 compatible object
storage incl. MinIO and Google Cloud Storage as remote tier storage backends.
Some new additions include:
- Admin APIs remote tier configuration management
- Simple journal to track remote objects to be 'collected'
This is used by object API handlers which 'mutate' object versions by
overwriting/replacing content (Put/CopyObject) or removing the version
itself (e.g DeleteObjectVersion).
- Rework of previous ILM transition to fit the new model
In the new model, a storage class (a.k.a remote tier) is defined by the
'remote' object storage type (one of s3, azure, GCS), bucket name and a
prefix.
* Fixed bugs, review comments, and more unit-tests
- Leverage inline small object feature
- Migrate legacy objects to the latest object format before transitioning
- Fix restore to particular version if specified
- Extend SharedDataDirCount to handle transitioned and restored objects
- Restore-object should accept version-id for version-suspended bucket (#12091)
- Check if remote tier creds have sufficient permissions
- Bonus minor fixes to existing error messages
Co-authored-by: Poorna Krishnamoorthy <poorna@minio.io>
Co-authored-by: Krishna Srinivas <krishna@minio.io>
Signed-off-by: Harshavardhana <harsha@minio.io>
This commit changes the config/IAM encryption
process. Instead of encrypting config data
(users, policies etc.) with the root credentials
MinIO now encrypts this data with a KMS - if configured.
Therefore, this PR moves the MinIO-KMS configuration (via
env. variables) to a "top-level" configuration.
The KMS configuration cannot be stored in the config file
since it is used to decrypt the config file in the first
place.
As a consequence, this commit also removes support for
Hashicorp Vault - which has been deprecated anyway.
Signed-off-by: Andreas Auernhammer <aead@mail.de>
- adding oauth support to MinIO browser (#8400) by @kanagaraj
- supports multi-line get/set/del for all config fields
- add support for comments, allow toggle
- add extensive validation of config before saving
- support MinIO browser to support proper claims, using STS tokens
- env support for all config parameters, legacy envs are also
supported with all documentation now pointing to latest ENVs
- preserve accessKey/secretKey from FS mode setups
- add history support implements three APIs
- ClearHistory
- RestoreHistory
- ListHistory
- add help command support for each config parameters
- all the bug fixes after migration to KV, and other bug
fixes encountered during testing.
There are multiple possibilities for running MinIO within
a container e.g. configurable address, non-root user etc.
This makes it difficult to identify actual IP / Port to
use to check healthcheck status from within a container.
It is simpler to use external healthcheck mechanisms
like healthcheck command in docker-compose to check
for MinIO health status. This is similar to how checks
work in Kubernetes as well.
This PR removes the healthcheck script used inside
Docker container and ad documentation on how to
use docker-compose based healthcheck mechanism.
There is no reliable way to handle fallbacks for
MinIO deployments, due to various command line
options and multiple locations which require
access inside container.
Parsing command line options is tricky to figure
out which is the backend disk etc, we did try
to fix this in implementations of check-user.go
but it wasn't complete and introduced more bugs.
This PR simplifies the entire approach to rather
than running Docker container as non-root by default
always, it allows users to opt-in. Such that they
are aware that that is what they are planning to do.
In-fact there are other ways docker containers can
be run as regular users, without modifying our
internal behavior and adding more complexities.
This allows MinIO containers to run properly without
expecting higher privileges in situations where following
restrictions on containers are used
- docker run --user uid:gid
- docker-compose up (with docker-compose.yml with user)
```yml
...
user: "1001:1001"
command: minio server /data
...
```
- All openshift containers
Fixes#7773
As a part of #7302, MinIO server's (configured with https) response when it
encounters http request has changed from 403 to 400 and the custom message
"SSL Required" is removed.
Accordingly healthcheck script is updated to check for status 400 before
trying https request.
Fixes#7517
Go script makes it easy to read/maintain. Also updated the timeout
in Dockerfiles from 5s to default 30s and test interval to 1m
Higher timeout makes sense as server may sometimes respond slowly
if under high load as reported in #6974Fixes#6974
This PR adds readiness and liveness endpoints to probe Minio server
instance health. Endpoints can only be accessed without authentication
and the paths are /minio/health/live and /minio/health/ready for
liveness and readiness respectively.
The new healthcheck liveness endpoint is used for Docker healthcheck
now.
Fixes#5357Fixes#5514
Harshavardhana
Krishnan Parthasarathi
Andreas Auernhammer
Nitish Tiwari
unlimitedbits
Bitworks LLC
kannappanr
Praveen raj Mani
Philipp Dallig
Lucas
tparker00
Paul Trunk
Arnaud de Mouhy