Minio GCS Gateway

Minio GCS Gateway adds Amazon S3 compatibility to Google Cloud Storage.

Run Minio Gateway for GCS

Create service account key for GCS and get the credentials file

Go to the API Console Credentials page.
Select your project or create a new project. Note down your project ID.
On the Credentials page, select the Create credentials drop-down, then select Service account key.
From the Service account drop-down, select New service account
Fill up Service account name and Service account ID
For the Role, click the selec dropdown to choose Storage -> Storage Admin (Full control of GCS resources)
Click the Create button. This will download a credentials file to your desktop. Let's call this credentials.json

Note: Alternate ways to setup Application Default Credentials is explained here

Using Docker

docker run -p 9000:9000 --name gcs-s3 \
 -v /path/to/credentials.json:/credentials.json \
 -e "GOOGLE_APPLICATION_CREDENTIALS=/credentials.json" \
 -e "MINIO_ACCESS_KEY=minioaccountname" \
 -e "MINIO_SECRET_KEY=minioaccountkey" \
 minio/minio gateway gcs yourprojectid

Using Binary

export GOOGLE_APPLICATION_CREDENTIALS=/path/to/credentials.json
export MINIO_ACCESS_KEY=minioaccesskey
export MINIO_SECRET_KEY=miniosecretkey
minio gateway gcs yourprojectid

Test using Minio Browser

Minio Gateway comes with an embedded web based object browser. Point your web browser to http://127.0.0.1:9000 ensure your server has started successfully.

Test using Minio Client `mc`

mc provides a modern alternative to UNIX commands such as ls, cat, cp, mirror, diff etc. It supports filesystems and Amazon S3 compatible cloud storage services.

Configure `mc`

mc config host add mygcs http://gateway-ip:9000 minioaccesskey miniosecretkey

List containers on GCS

mc ls mygcs
[2017-02-22 01:50:43 PST]     0B ferenginar/
[2017-02-26 21:43:51 PST]     0B my-container/
[2017-02-26 22:10:11 PST]     0B test-container1/

Known limitations

Gateway inherits the following GCS limitations:

Maximum number of multipart parts per upload is 1024.
Only read-only or write-only bucket policy supported at bucket level, all other variations will return API Notimplemented error.
List Multipart Uploads and List Object parts always returns empty list. i.e Client will need to remember all the parts that it has uploaded and use it for Complete Multipart Upload

Other limitations:

Bucket notification APIs are not supported.

3.2 KiB Raw Blame History