MyFavMirrors/minio
1
0
Fork 0
mirror of https://github.com/minio/minio.git synced 2025-11-22 18:47:43 -05:00
Code Issues Packages Projects Releases Wiki Activity

Allow caching only in gateway mode. (#8232)

Browse Source 
This PR changes cache on PUT behavior to background fill the cache
after PutObject completes. This will avoid concurrency issues as in #8219.

Added cleanup of partially filled cache to prevent cache corruption
- Fixes #8208
This commit is contained in:
poornas
2019-09-16 14:24:04 -07:00
committed by kannappanr
parent 208efb843b
commit 76df027264
5 changed files with 42 additions and 112 deletions
Download Patch File Download Diff File Expand all files Collapse all files

13
docs/disk-caching/DESIGN.md
View File

@@ -5,7 +5,7 @@ This document explains some basic assumptions and design approach, limits of the
## Command-line
```
minio server -h
minio gateway <name> -h
...
...
CACHE:
@@ -16,17 +16,18 @@ minio server -h
...
...
7. Start minio server with edge caching enabled on '/mnt/drive1', '/mnt/drive2' and '/mnt/export1 ... /mnt/export24',
7. Start MinIO gateway to s3 with edge caching enabled on '/mnt/drive1', '/mnt/drive2' and '/mnt/export1 ... /mnt/export24',
exclude all objects under 'mybucket', exclude all objects with '.pdf' as extension
with expiry up to 40 days.
$ export MINIO_CACHE_DRIVES="/mnt/drive1;/mnt/drive2;/mnt/export{1..24}"
$ export MINIO_CACHE_EXCLUDE="mybucket/*;*.pdf"
$ export MINIO_CACHE_EXPIRY=40
$ export MINIO_CACHE_MAXUSE=80
$ minio server /home/shared
$ minio gateway s3
```
## Assumptions
- Disk cache size defaults to 80% of your drive capacity.
- The cache drives are required to be a filesystem mount point with [`atime`](http://kerolasa.github.io/filetimes.html) support to be enabled on the drive. Alternatively writable directories with atime support can be specified in MINIO_CACHE_DRIVES
- Expiration of each cached entry takes user provided expiry as a hint, and defaults to 90 days if not provided.
@@ -34,6 +35,7 @@ minio server -h
- An object is only cached when drive has sufficient disk space.
## Behavior
Disk caching caches objects for **downloaded** objects i.e
- Caches new objects for entries not found in cache while downloading. Otherwise serves from the cache.
@@ -50,9 +52,10 @@ master key to automatically encrypt all cached content.
> NOTE: Expiration happens automatically based on the configured interval as explained above, frequently accessed objects stay alive in cache for a significantly longer time.
### Crash Recovery
Upon restart of minio server after a running minio process is killed or crashes, disk caching resumes automatically. The garbage collection cycle resumes and any previously cached entries are served from cache.
Upon restart of minio gateway after a running minio process is killed or crashes, disk caching resumes automatically. The garbage collection cycle resumes and any previously cached entries are served from cache.
## Limits
- Bucket policies are not cached, so anonymous operations are not supported when backend is offline.
- Objects are distributed using deterministic hashing among the list of configured cache drives. If one or more drives go offline, or cache drive configuration is altered in any way, performance may degrade to a linear lookup time depending on the number of disks in cache.