High Performance Object Storage for AI
Go to file
Harshavardhana 12f67d47f1 Fix a possible race during PutObject() (#5376)
Under any concurrent removeObjects in progress
might have removed the parents of the same prefix
for which there is an ongoing putObject request.
An inconsistent situation may arise as explained
below even under sufficient locking.

PutObject is almost successful at the last stage when
a temporary file is renamed to its actual namespace
at `a/b/c/object1`. Concurrently a RemoveObject is
also in progress at the same prefix for an `a/b/c/object2`.

To create the object1 at location `a/b/c` PutObject has
to create all the parents recursively.

```
a/b/c - os.MkdirAll loops through has now created
        'a/' and 'b/' about to create 'c/'
a/b/c/object2 - at this point 'c/' and 'object2'
        are deleted about to delete b/
```

Now for os.MkdirAll loop the expected situation is
that top level parent 'a/b/' exists which it created
, such that it can create 'c/' - since removeObject
and putObject do not compete for lock due to holding
locks at different resources. removeObject proceeds
to delete parent 'b/' since 'c/' is not yet present,
once deleted 'os.MkdirAll' would receive an error as
syscall.ENOENT which would fail the putObject request.

This PR tries to address this issue by implementing
a safer/guarded approach where we would retry an operation
such as `os.MkdirAll` and `os.Rename` if both operations
observe syscall.ENOENT.

Fixes #5254
2018-01-13 22:43:02 +05:30
.github Update GitHub PR template to include mint tests (#4970) 2017-10-28 07:14:19 -07:00
browser Correct color code of Excel file icon (#5361) 2018-01-05 16:19:41 +05:30
buildscripts Convert gateways into respective packages (#5200) 2017-12-05 17:58:09 -08:00
cmd Fix a possible race during PutObject() (#5376) 2018-01-13 22:43:02 +05:30
dockerscripts Add custom secret names for Docker (#5355) 2018-01-09 10:46:25 +05:30
docs Add chroot environment doc for minio (#5366) 2018-01-12 07:55:40 +05:30
pkg restrict TLS cipher suites of the server (#5245) 2018-01-13 09:12:11 +05:30
snap Update go version in snap file (#5256) 2017-12-04 15:49:31 -08:00
vendor gateway/manta: Add support for RBAC (#5332) 2018-01-05 13:30:29 +05:30
.gitignore Implement Sia Gateway (#5114) 2017-11-22 12:12:10 -08:00
.jshintrc browser: Listing should append instead of replacing previous listing (#4188) 2017-04-28 09:30:26 -07:00
.mailmap Update AB's duplicate email address 2015-08-20 16:10:01 -07:00
.mention-bot Add mention-bot config 2016-05-20 13:53:15 -07:00
.travis.yml Move to go1.9.1 as default environment. (#5041) 2017-10-09 22:23:59 -07:00
appveyor.yml Convert gateways into respective packages (#5200) 2017-12-05 17:58:09 -08:00
CONTRIBUTING.md Refactor CONTRIBUTING.md to make it more reader friendly (#4918) 2017-09-18 11:02:45 -07:00
Dockerfile Add custom secret names for Docker (#5355) 2018-01-09 10:46:25 +05:30
Dockerfile.dev Add custom secret names for Docker (#5355) 2018-01-09 10:46:25 +05:30
Dockerfile.release Add custom secret names for Docker (#5355) 2018-01-09 10:46:25 +05:30
hound.yml browser: Listing should append instead of replacing previous listing (#4188) 2017-04-28 09:30:26 -07:00
LICENSE Initial commit 2014-10-30 21:51:52 -07:00
main_test.go Move to go1.9.1 as default environment. (#5041) 2017-10-09 22:23:59 -07:00
main.go Convert gateways into respective packages (#5200) 2017-12-05 17:58:09 -08:00
MAINTAINERS.md Fix the server startup messages and help text. (#3211) 2016-11-09 23:37:12 -08:00
Makefile Convert gateways into respective packages (#5200) 2017-12-05 17:58:09 -08:00
minio.spec pkg: Update the rpm spec with latest release. (#4187) 2017-04-28 12:35:02 -07:00
NOTICE Fix typo 2016-03-09 13:55:58 +01:00
README_zh_CN.md Add chinese translation docs. (#5224) 2017-11-22 15:15:40 -08:00
README.md Fix snap docs with instructions to run (#5266) 2017-12-07 08:38:58 +05:30

Minio Quickstart Guide

Slack Go Report Card Docker Pulls codecov Snap Status

Minio is an object storage server released under Apache License v2.0. It is compatible with Amazon S3 cloud storage service. It is best suited for storing unstructured data such as photos, videos, log files, backups and container / VM images. Size of an object can range from a few KBs to a maximum of 5TB.

Minio server is light enough to be bundled with the application stack, similar to NodeJS, Redis and MySQL.

Docker Container

Stable

docker pull minio/minio
docker run -p 9000:9000 minio/minio server /data

Edge

docker pull minio/minio:edge
docker run -p 9000:9000 minio/minio:edge server /data

Please visit Minio Docker quickstart guide for more here

macOS

Homebrew

Install minio packages using Homebrew

brew install minio/stable/minio
minio server /data

NOTE: If you previously installed minio using brew install minio then it is recommended that you reinstall minio from minio/stable/minio official repo instead.

brew uninstall minio
brew install minio/stable/minio

Binary Download

Platform Architecture URL
Apple macOS 64-bit Intel https://dl.minio.io/server/minio/release/darwin-amd64/minio
chmod 755 minio
./minio server /data

GNU/Linux

Binary Download

Platform Architecture URL
GNU/Linux 64-bit Intel https://dl.minio.io/server/minio/release/linux-amd64/minio
chmod +x minio
./minio server /data

Snap

Install minio using Snap

sudo snap install minio --edge

Start minio using snap run command

sudo snap connect minio:mount-observe
sudo snap run minio server /data

Microsoft Windows

Binary Download

Platform Architecture URL
Microsoft Windows 64-bit https://dl.minio.io/server/minio/release/windows-amd64/minio.exe
minio.exe server D:\Photos

FreeBSD

Port

Install minio packages using pkg

pkg install minio
sysrc minio_enable=yes
sysrc minio_disks=/home/user/Photos
service minio start

Install from Source

Source installation is only intended for developers and advanced users. If you do not have a working Golang environment, please follow How to install Golang.

go get -u github.com/minio/minio

Test using Minio Browser

Minio Server comes with an embedded web based object browser. Point your web browser to http://127.0.0.1:9000 ensure your server has started successfully.

Screenshot

Test using Minio Client mc

mc provides a modern alternative to UNIX commands like ls, cat, cp, mirror, diff etc. It supports filesystems and Amazon S3 compatible cloud storage services. Follow the Minio Client Quickstart Guide for further instructions.

Explore Further

Contribute to Minio Project

Please follow Minio Contributor's Guide