MyFavMirrors
/
minio
mirror of https://github.com/minio/minio.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
minio/docs/site-replication
History
Harshavardhana b7c5e45fff
heal: isObjectDangling should return false when it cannot decide (#14053) 
In a multi-pool setup when disks are coming up, or in a single pool
setup let's say with 100's of erasure sets with a slow network.

It's possible when healing is attempted on `.minio.sys/config`
folder, it can lead to healing unexpectedly deleting some policy
files as dangling due to a mistake in understanding when `isObjectDangling`
is considered to be 'true'.

This issue happened in commit 30135eed86
when we assumed the validMeta with empty ErasureInfo is considered
to be fully dangling. This implementation issue gets exposed when
the server is starting up.

This is most easily seen with multiple-pool setups because of the
disconnected fashion pools that come up. The decision to purge the
object as dangling is taken incorrectly prior to the correct state
being achieved on each pool, when the corresponding drive let's say
returns 'errDiskNotFound', a 'delete' is triggered. At this point,
the 'drive' comes online because this is part of the startup sequence
as drives can come online lazily.

This kind of situation exists because we allow (totalDisks/2) number
of drives to be online when the server is being restarted.

Implementation made an incorrect assumption here leading to policies
getting deleted.

Added tests to capture the implementation requirements.
 		2022-01-07 19:11:54 -08:00
..
README.md Update site replication docs (#14044) 2022-01-06 17:53:23 -08:00
gen-oidc-sts-cred.go fix: progagation of service accounts for site replication (#14054) 2022-01-07 17:41:43 -08:00
ldap.yaml fix: do not list delete-marked objects (#13864) 2021-12-08 17:34:52 -08:00
run-multi-site-ldap.sh heal: isObjectDangling should return false when it cannot decide (#14053) 2022-01-07 19:11:54 -08:00
run-multi-site-minio-idp.sh heal: isObjectDangling should return false when it cannot decide (#14053) 2022-01-07 19:11:54 -08:00
run-multi-site-oidc.sh fix: progagation of service accounts for site replication (#14054) 2022-01-07 17:41:43 -08:00
rw.json fix: site replication issues and add tests (#13861) 2021-12-08 11:50:15 -08:00

README.md

Automatic Site Replication

This feature allows multiple independent MinIO sites (or clusters) that are using the same external IDentity Provider (IDP) to be configured as replicas. In this situation the set of replica sites are referred to as peer sites or just sites. When site-replication is enabled on a set of sites, the following changes are replicated to all other sites:

  • Creation and deletion of buckets and objects
  • Creation and deletion of all IAM users, groups, policies and their mappings to users or groups
  • Creation of STS credentials
  • Creation and deletion of service accounts (except those owned by the root user)
  • Changes to Bucket features such as:
    • Bucket Policies
    • Bucket Tags
    • Bucket Object-Lock configurations (including retention and legal hold configuration)
    • Bucket Encryption configuration

NOTE: Bucket versioning is automatically enabled for all new and existing buckets on all replicated sites.

The following Bucket features will not be replicated, is designed to differ between sites:

  • Bucket notification configuration
  • Bucket lifecycle (ILM) configuration

Pre-requisites

  • Initially, only one of the sites added for replication may have data. After site-replication is successfully configured, this data is replicated to the other (initially empty) sites. Subsequently, objects may be written to any of the sites, and they will be replicated to all other sites.
  • All sites must have the same deployment credentials (i.e. MINIO_ROOT_USER, MINIO_ROOT_PASSWORD).
  • Removing a site is not allowed from a set of replicated sites once configured.
  • All sites must be using the same external IDP(s) if any.
  • For SSE-S3 or SSE-KMS encryption via KMS, all sites must have access to a central KMS deployment. This can be achieved via a central KES server or multiple KES servers (say one per site) connected via a central KMS (Vault) server.

Configuring Site Replication

  • Configure an alias in mc for each of the sites. For example if you have three MinIO sites, you may run:
$ mc alias set minio1 https://minio1.example.com:9000 adminuser adminpassword
$ mc alias set minio2 https://minio2.example.com:9000 adminuser adminpassword
$ mc alias set minio3 https://minio3.example.com:9000 adminuser adminpassword

or

$ export MC_HOST_minio1=https://adminuser:adminpassword@minio1.example.com
$ export MC_HOST_minio2=https://adminuser:adminpassword@minio2.example.com
$ export MC_HOST_minio3=https://adminuser:adminpassword@minio3.example.com
  • Add site replication configuration with:
$ mc admin replicate add minio1 minio2 minio3
  • Once the above command returns success, you may query site replication configuration with:
$ mc admin replicate info minio1