High Performance Object Storage for AI
Go to file
Krishna Srinivas e5b411caf4 startup: do not fail for non-EADDRINUSE errors. Fixes #1234 2016-03-17 01:36:08 +05:30
buildscripts Merge pull request #1167 from harshavardhana/fix-release-tag 2016-02-23 17:16:26 -08:00
pkg bucketpolicy: Improve bucket policy validation, avoid nested rules. 2016-03-15 17:50:23 -07:00
vendor vendor: Update ui-assets with new changes and release. 2016-03-10 17:36:32 -08:00
.gitignore build/vet: Fix all the shadowing reports with go1.6 2016-02-23 14:34:39 -08:00
.mailmap Update AB's duplicate email address 2015-08-20 16:10:01 -07:00
.travis.yml build/vet: Fix all the shadowing reports with go1.6 2016-02-23 14:34:39 -08:00
AWS-SDK-GO.md doc: Cleanup aws-sdk-go.md 2015-11-23 13:53:55 -08:00
Browser.md jwt: Deprecate RSA usage, use HMAC instead. 2016-01-27 03:38:33 -08:00
Bucket-Policy.md bucketpolicy: Improve bucket policy validation, avoid nested rules. 2016-03-15 17:50:23 -07:00
CONTRIBUTING.md Migrate this project to minio micro services code 2015-10-16 11:26:08 -07:00
Caddy.md Update Caddy.md 2016-03-14 20:51:16 -07:00
Docker.md Update Docker.md 2016-03-15 10:01:58 -07:00
Dockerfile Alpine & CA-Certs for SSL support 2016-02-20 11:15:19 +01:00
INSTALLGO.md build/vet: Fix all the shadowing reports with go1.6 2016-02-23 14:34:39 -08:00
LICENSE Initial commit 2014-10-30 21:51:52 -07:00
MAINTAINERS.md build: Add release builds, now generated with 'make release' 2016-02-23 15:14:02 -08:00
Makefile build: Fix release tag. 2016-02-23 16:56:41 -08:00
NOTICE Fix typo 2016-03-09 13:55:58 +01:00
README.md Update README.md 2016-03-02 11:07:23 -08:00
accesslog-handler.go xl: Moved to minio/minio - fixes #1112 2016-02-11 15:43:36 -08:00
api-auth-utils.go use environment variables to set and override access and secret keys at server startup 2016-02-22 10:49:02 -08:00
api-datatypes.go api: Implement multiple objects Delete api - fixes #956 2016-03-06 18:31:50 -08:00
api-errors.go error: Add proper prefixes for s3Error codes. 2016-03-10 18:38:46 -08:00
api-headers.go cleanup: Rename ObjectMetadata as ObjectInfo. 2016-03-11 16:58:08 -08:00
api-headers_test.go handlers: Cleanup time handlers helpers. 2016-03-07 10:47:45 -08:00
api-resources.go listObjects: Channel based changes. 2016-01-25 20:39:38 -08:00
api-response.go cleanup: Rename ObjectMetadata as ObjectInfo. 2016-03-11 16:58:08 -08:00
appveyor.yml build/vet: Fix all the shadowing reports with go1.6 2016-02-23 14:34:39 -08:00
auth-handler.go error: Add proper prefixes for s3Error codes. 2016-03-10 18:38:46 -08:00
bucket-handlers.go bucketpolicy: Improve bucket policy validation, avoid nested rules. 2016-03-15 17:50:23 -07:00
bucket-policy-handlers.go bucketpolicy: Improve bucket policy validation, avoid nested rules. 2016-03-15 17:50:23 -07:00
bucket-policy-parser.go bucketpolicy: Improve bucket policy validation, avoid nested rules. 2016-03-15 17:50:23 -07:00
bucket-policy.go bucketpolicy: Improve bucket policy validation, avoid nested rules. 2016-03-15 17:50:23 -07:00
build-constants.go build: Add release builds, now generated with 'make release' 2016-02-23 15:14:02 -08:00
commands.go Add trie to verify wrong inputs, and provide meaningful messages 2015-09-17 16:49:08 -07:00
config-logger-main.go console: Fix console color printing on windows. 2016-03-04 10:07:19 -08:00
config-main.go Add logger command - also migrate from old config to newer config 2015-10-21 00:02:16 -07:00
config-version-main.go console: Fix console color printing on windows. 2016-03-04 10:07:19 -08:00
damerau-levenshtein.go xl: Moved to minio/minio - fixes #1112 2016-02-11 15:43:36 -08:00
flags.go flags: Remove anonymous, ratelimit, json and web-address flags. 2016-02-02 18:37:09 -08:00
generic-handlers.go error: Add proper prefixes for s3Error codes. 2016-03-10 18:38:46 -08:00
globals.go build/vet: Fix all the shadowing reports with go1.6 2016-02-23 14:34:39 -08:00
httprange.go xl: Moved to minio/minio - fixes #1112 2016-02-11 15:43:36 -08:00
jwt.go presigned: Fix a bug in presigned request verification. 2016-02-18 02:23:12 -08:00
logger-file-hook.go xl: Moved to minio/minio - fixes #1112 2016-02-11 15:43:36 -08:00
logger-mongo-hook.go xl: Moved to minio/minio - fixes #1112 2016-02-11 15:43:36 -08:00
logger-syslog-hook_nix.go xl: Moved to minio/minio - fixes #1112 2016-02-11 15:43:36 -08:00
logger-syslog-hook_windows.go xl: Moved to minio/minio - fixes #1112 2016-02-11 15:43:36 -08:00
logger.go xl: Moved to minio/minio - fixes #1112 2016-02-11 15:43:36 -08:00
logger_test.go xl: Moved to minio/minio - fixes #1112 2016-02-11 15:43:36 -08:00
main.go console: Fix console color printing on windows. 2016-03-04 10:07:19 -08:00
main_test.go Migrate this project to minio micro services code 2015-10-16 11:26:08 -07:00
notifier.go xl: Moved to minio/minio - fixes #1112 2016-02-11 15:43:36 -08:00
object-handlers.go api/object: Add CopyObject to support match/modified copy headers 2016-03-12 10:54:23 -08:00
routers.go vendor: Add minio-go vendor updates. 2016-03-10 14:33:15 -08:00
runtime-checks.go console: Fix console color printing on windows. 2016-03-04 10:07:19 -08:00
server-config.go accessPolicy: Implement Put, Get, Delete access policy. 2016-03-08 17:44:50 -08:00
server-main.go startup: do not fail for non-EADDRINUSE errors. Fixes #1234 2016-03-17 01:36:08 +05:30
server_fs_test.go bucketpolicy: Improve bucket policy validation, avoid nested rules. 2016-03-15 17:50:23 -07:00
signature_utils_test.go xl: Moved to minio/minio - fixes #1112 2016-02-11 15:43:36 -08:00
trie.go Add trie to verify wrong inputs, and provide meaningful messages 2015-09-17 16:49:08 -07:00
typed-errors.go jsonrpc: WrapError() makes jsonrpc return unnecessary details in the error message. 2016-02-10 21:22:37 +05:30
update-main.go console: Fix console color printing on windows. 2016-03-04 10:07:19 -08:00
utils.go fix: handle Transfer-Encoding for make bucket 2016-02-05 19:01:39 +05:30
version-main.go console: Fix console color printing on windows. 2016-03-04 10:07:19 -08:00
version_test.go Move all server and controller packages into top-level 2015-09-19 01:07:42 -07:00
web-handlers.go caching: disable caching of index.html and enable caching for other UI asset files. 2016-02-27 12:22:26 +05:30

README.md

Minio Gitter

Minio is an object storage server compatible with Amazon S3 and licensed under Apache license 2.0.

Description

Minio is an open source object storage server released under Apache License V2. It is compatible with Amazon S3 cloud storage service. Minio follows a minimalist design philosophy. Minio is light enough to be bundled with the application stack. It sits on the side of NodeJS, Redis, MySQL and the likes. Unlike databases, Minio stores objects such as photos, videos, log files, backups, container / VM images and so on. Minio is best suited for storing blobs of information ranging from KBs to TBs each. In a simplistic sense, it is like a FTP server with a simple get / put API over HTTP.

Minio currently implements two backends

  • Filesystem (FS) - is available and ready for general purpose use. This version of the Minio binary is built using Filesystem storage backend for magnetic and solid state disks.
  • ErasureCoded (XL) - is work in progress and not ready for general purpose use.

Minio Client

Minio Client (mc) provides a modern alternative to Unix commands like ls, cat, cp, sync, and diff. It supports POSIX compatible filesystems and Amazon S3 compatible cloud storage systems. It is entirely written in Golang.

Amazon S3 Compatible Client Libraries

Install Build StatusBuild status

GNU/Linux

Download minio for:

$ chmod +x minio
$ ./minio --help

OS X

Download minio from https://dl.minio.io/server/minio/release/darwin-amd64/minio

$ chmod 755 minio
$ ./minio --help

Microsoft Windows

Download minio for:

C:\Users\Username\Downloads> minio.exe --help

Docker container

Download minio for docker.

$ docker pull minio/minio

Read more here on How to configure data volume containers for Minio?

Source

NOTE: Source installation is intended for only developers and advanced users. For general use, please download official releases from https://minio.io/download.

If you do not have a working Golang environment, please follow Install Golang.

$ go get -d github.com/minio/minio
$ cd $GOPATH/src/github.com/minio/minio
$ make

How to use Minio?

NAME:
  minio server - Start Minio cloud storage server.

USAGE:
  minio server [OPTION VALUE] PATH

  OPTION = expiry        VALUE = NN[h|m|s] [DEFAULT=Unlimited]
  OPTION = min-free-disk VALUE = NN% [DEFAULT: 10%]

EXAMPLES:
  1. Start minio server on Linux.
        $ minio server /home/shared

  2. Start minio server on Windows.
        $ minio server C:\MyShare

  3. Start minio server bound to a specific IP:PORT, when you have multiple network interfaces.
        $ minio --address 192.168.1.101:9000 server /home/shared

  4. Start minio server with minimum free disk threshold to 5%
        $ minio server min-free-disk 5% /home/shared/Pictures

  5. Start minio server with minimum free disk threshold to 15% with auto expiration set to 1h
        $ minio server min-free-disk 15% expiry 1h /home/shared/Documents

Start Minio server.

$ minio server ~/Photos
AccessKey: WLGDGYAQYIGI833EV05A  SecretKey: BYvgJM101sHngl2uzjXS/OBF/aMxAN06JrJ3qJlF  Region: us-east-1

Minio Object Storage:
    http://127.0.0.1:9000
    http://10.0.0.3:9000

Minio Browser:
    http://127.0.0.1:9000
    http://10.0.0.3:9000

To configure Minio Client:
    $ wget https://dl.minio.io/client/mc/release/darwin-amd64/mc
    $ chmod 755 mc
    $ ./mc config host add myminio http://localhost:9000 WLGDGYAQYIGI833EV05A BYvgJM101sHngl2uzjXS/OBF/aMxAN06JrJ3qJlF

How to use AWS CLI with Minio?

This section assumes that you have already installed aws-cli, if not please visit https://aws.amazon.com/cli/

To configure aws-cli, type aws configure and follow below steps.

$ aws configure
AWS Access Key ID [None]: YOUR_ACCESS_KEY_HERE
AWS Secret Access Key [None]: YOUR_SECRET_KEY_HERE
Default region name [None]: us-east-1
Default output format [None]: ENTER

Additionally enable aws-cli to use AWS Signature Version '4' for Minio server.

$ aws configure set default.s3.signature_version s3v4

To list your buckets.

$ aws --endpoint-url http://localhost:9000 s3 ls
2016-01-07 16:38:23 testbucket

To list contents inside bucket.

$ aws --endpoint-url http://localhost:9000 s3 ls s3://testbucket
                           PRE test/
2015-12-17 08:46:41   12232928 vim
2016-01-07 16:38:23   32232928 emacs
2015-12-09 08:05:24     138504 s3cmd

How to use AWS SDK with Minio?

Please follow the documentation here - Using aws-sdk-go with Minio server

How to use s3cmd with Minio?

This section assumes that you have already installed s3cmd, if not please visit http://s3tools.org/s3cmd

Edit the following fields in your s3cmd configuration file ~/.s3cfg .

host_base = localhost:9000
host_bucket = localhost:9000
access_key = YOUR_ACCESS_KEY_HERE
secret_key = YOUR_SECRET_KEY_HERE

To list your buckets.

$ s3cmd ls s3://
2015-12-09 16:12  s3://testbbucket

To list contents inside buckets.

$ s3cmd ls s3://testbucket/
                       DIR   s3://testbucket/test/
2015-12-09 16:05    138504   s3://testbucket/newfile

Contribute to Minio Project

Please follow Minio Contributor's Guide

Jobs

If you think in Lisp or Haskell and hack in go, you would blend right in. Send your github link to callhome@minio.io.