MyFavMirrors/minio
1
0
Fork 0
mirror of https://github.com/minio/minio.git synced 2024-12-26 15:15:55 -05:00
Code Issues Packages Projects Releases Wiki Activity
2,742 Commits 4 Branches 506 Tags 189 MiB
ba7a55c321
Commit Graph

2742 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Harshavardhana
8986a6802a Fix ListMultipartUploads 'mc ls -I' now works properly. 2016-04-05 16:39:02 -07:00
Harshavardhana
3fcc60de91 Move the files and rename some functions. 
- Rename dir.go as 'fs-multipart-dir.go'
- Move the push/pop to fs-multipart.go and rename them as save/lookup.
- Rename objectInfo instances in fs-multipart as multipartObjInfo.
 2016-04-05 12:26:19 -07:00
Harshavardhana
9632c94e7a Fix list objects test and remove all the old unnecessary files. 
- Fix tests for new changes.
- Change Golang err as 'e' for the time being, before we bring in
  probe removal change.
- Remove old structs and temporary files.
 2016-04-05 12:07:19 -07:00
Bala.FA
083e4e9479 backend/fs: Refactor multipart upload 
This patch modifies multipart upload related functions as below

* New multipart upload call creates file
  EXPORT_DIR/.minio/BUCKET/PATH/TO/OBJECT/UPLOAD_ID.uploadid

* Put object part call creates file
  EXPORT_DIR/.minio/BUCKET/PATH/TO/OBJECT/UPLOAD_ID.PART_NUMBER.MD5SUM_STRING

* Abort multipart call removes all files matching
  EXPORT_DIR/.minio/BUCKET/PATH/TO/OBJECT/UPLOAD_ID.*

* Complete multipart call does
  1. creates a staging file
  EXPORT_DIR/.minio/BUCKET/PATH/TO/OBJECT/UPLOAD_ID.complete.TEMP_NAME
  then renames to
  EXPORT_DIR/.minio/BUCKET/PATH/TO/OBJECT/UPLOAD_ID.complete
  2. rename staging file
  EXPORT_DIR/.minio/BUCKET/PATH/TO/OBJECT/UPLOAD_ID.complete
  to EXPORT_DIR/BUCKET/PATH/TO/OBJECT
 2016-04-05 22:22:29 +05:30
Harshavardhana
c69fdf0cf2 listObjects: Cleanup and naming conventions. 
- Marker should be escaped outside in handlers.

- Delimiter should be handled outside in handlers.

- Add missing comments and change the function names.

- Handle case of 'maxKeys' when its set to '0', its a valid

  case and should be treated as such.
 2016-04-04 19:55:07 -07:00
Krishna Srinivas
85ab1df5a8 listObjects: do not do stat during readdir() 
* listObjects: improve response time by not doing stat during readDir() operation.

* listObjects: Add windows support.

* listObjects: Readdir() in batches to conserve memory. Add solaris build.

* listObjects: cleanup code.
 2016-04-04 17:27:55 -07:00
Harshavardhana
7623e0f8e8 docker: Fix bug in start.sh arguments. 2016-04-04 11:30:18 -07:00
Anand Babu (AB) Periasamy
9843aa1f7a Merge pull request #1284 from harshavardhana/list 
objectAPI: Fix object API interface, remove unnecessary structs.
 2016-04-03 21:13:48 -07:00
Harshavardhana
0479d4976b objectAPI: Fix object API interface, remove unnecessary structs. 
ObjectAPI changes.
```
ListObjects(bucket, prefix, marker, delimiter string, maxKeys int) (ListObjectsInfo, *probe.Error)
ListMultipartUploads(bucket, objectPrefix, keyMarker, uploadIDMarker, delimiter string, maxUploads int) (ListMultipartsInfo, *probe.Error)
ListObjectParts(bucket, object, uploadID string, partNumberMarker, maxParts int) (ListPartsInfo, *probe.Error)
CompleteMultipartUpload(bucket string, object string, uploadID string, parts []completePart) (ObjectInfo, *probe.Error)
```
 2016-04-03 15:25:01 -07:00
Anand Babu (AB) Periasamy
12515eabe2 Merge pull request #1280 from harshavardhana/region 
signature: No need to validate region for getBucketLocation and listBuckets
 2016-04-03 01:01:05 -07:00
Harshavardhana
a6a4e7e297 signature: No need to validate region for getBucketLocation and listBuckets. 
This type of check is added for making sure that we can support
custom regions.

ListBuckets and GetBucketLocation are always "us-east-1" rest
should look for the configured region.

Fixes #1278
 2016-04-02 18:42:32 -07:00
Anand Babu (AB) Periasamy
2c793a2ea7 Merge pull request #1282 from harshavardhana/remove-old-code 
cleanup: Remove old donut/xl code and erasure implementation.
 2016-04-02 18:14:21 -07:00
Anand Babu (AB) Periasamy
2bb262cc56 Merge pull request #1279 from harshavardhana/backend 
config: Migrate to the new version. Remove backend details.
 2016-04-02 17:59:21 -07:00
Harshavardhana
379e0abf03 cleanup: Remove old donut/xl code and erasure implementation. 
This is a change to bring in 'klauspost/reedsolomon' library
in #1270 patch.
 2016-04-02 17:30:35 -07:00
Harshavardhana
484ba91b08 config: Migrate to the new version. Remove backend details. 
Migrate to new config format v4.
```
{
	"version": "4",
	"credential": {
		"accessKey": "WLGDGYAQYIGI833EV05A",
		"secretKey": "BYvgJM101sHngl2uzjXS/OBF/aMxAN06JrJ3qJlF"
	},
	"region": "us-east-1",
	"logger": {
		"console": {
			"enable": true,
			"level": "fatal"
		},
		"file": {
			"enable": false,
			"fileName": "",
			"level": "error"
		},
		"syslog": {
			"enable": false,
			"address": "",
			"level": "debug"
		}
	}
}
```

This patch also updates [minio cli spec](./minio.md)
 2016-04-02 17:29:31 -07:00
Harshavardhana
6037fe66e9 minio: Simplify for gosimple tool complaints. 2016-04-02 17:28:54 -07:00
Harshavardhana
ba3a5805c1 vendorize: Add updated ui-assets.go. 2016-04-02 17:27:36 -07:00
Anand Babu (AB) Periasamy
33830bfcae Merge pull request #1273 from harshavardhana/fs-linux 
fs: Break fs package to top-level and introduce ObjectAPI interface.
 2016-04-01 17:03:23 -07:00
Harshavardhana
efc80343e3 fs: Break fs package to top-level and introduce ObjectAPI interface. 
ObjectAPI interface brings in changes needed for XL ObjectAPI layer.

The new interface for any ObjectAPI layer is as below

```
// ObjectAPI interface.
type ObjectAPI interface {
        // Bucket resource API.
        DeleteBucket(bucket string) *probe.Error
        ListBuckets() ([]BucketInfo, *probe.Error)
        MakeBucket(bucket string) *probe.Error
        GetBucketInfo(bucket string) (BucketInfo, *probe.Error)

        // Bucket query API.
        ListObjects(bucket, prefix, marker, delimiter string, maxKeys int) (ListObjectsResult, *probe.Error)
        ListMultipartUploads(bucket string, resources BucketMultipartResourcesMetadata) (BucketMultipartResourcesMetadata, *probe.Error)

        // Object resource API.
        GetObject(bucket, object string, startOffset int64) (io.ReadCloser, *probe.Error)
        GetObjectInfo(bucket, object string) (ObjectInfo, *probe.Error)
        PutObject(bucket string, object string, size int64, data io.Reader, metadata map[string]string) (ObjectInfo, *probe.Error)
        DeleteObject(bucket, object string) *probe.Error

        // Object query API.
        NewMultipartUpload(bucket, object string) (string, *probe.Error)
        PutObjectPart(bucket, object, uploadID string, partID int, size int64, data io.Reader, md5Hex string) (string, *probe.Error)
        ListObjectParts(bucket, object string, resources ObjectResourcesMetadata) (ObjectResourcesMetadata, *probe.Error)
        CompleteMultipartUpload(bucket string, object string, uploadID string, parts []CompletePart) (ObjectInfo, *probe.Error)
        AbortMultipartUpload(bucket, object, uploadID string) *probe.Error
}
```
 2016-04-01 15:58:39 -07:00
Harshavardhana
272c5165aa Merge pull request #1272 from krishnasrinivas/get-auth 
GetAuth implementation. min/max check for accessKey and secretKey.
 2016-04-01 09:38:58 -07:00
Krishna Srinivas
e318925f62 credentials: min/max length check for credentials. 2016-04-01 21:52:39 +05:30
Harshavardhana
2395c42fb5 Merge pull request #1277 from krishnasrinivas/remove-minio-go2 
UI-handler: remove minio-go dependancy.
 2016-04-01 08:56:16 -07:00
Harshavardhana
9333dc3294 Merge pull request #1204 from hackintoshrao/test-bucket 
Test: Changes to TestPutBucket to catch the race
 2016-04-01 08:53:34 -07:00
Karthic Rao
30fc970eab Changes to TestPutBucket to catch the race 2016-04-01 15:21:16 +05:30
Krishna Srinivas
331890c4c8 UI-handler: remove minio-go dependancy. 2016-04-01 13:56:32 +05:30
Anand Babu (AB) Periasamy
ae5c65d3c6 Merge pull request #1275 from harshavardhana/signature 
error: Signature errors should be returned with APIErrorCode.
 2016-03-31 23:37:38 -07:00
Harshavardhana
02ad48466d error: Signature errors should be returned with APIErrorCode. 
The reasoning is that we can reply back with wide range of
S3 error responses, which would provide more richer context
to S3 client.

Fixes #1267
 2016-03-31 23:28:40 -07:00
Harshavardhana
a84c466a40 Merge pull request #1251 from harshavardhana/release-fixes 
release: gz doesn't preserve permissions use tar.gz
 2016-03-30 14:34:19 -07:00
Harshavardhana
956142be37 Merge pull request #1271 from krishnasrinivas/set-auth2 
UI: implement SetAuth/GenerateAuth handlers for changing credentials.
 2016-03-29 09:38:18 -07:00
Krishna Srinivas
5201905ad0 UI: implement SetAuth/GenerateAuth handlers for changing credentials. 2016-03-29 21:08:36 +05:30
Anand Babu (AB) Periasamy
186998ad99 Merge pull request #1266 from harshavardhana/cleanup 
routers: Move API and Web routers into their own files.
 2016-03-27 14:25:02 -07:00
Harshavardhana
aa8c9bad54 routers: Move API and Web routers into their own files. 
This is done to ensure we have a clean way to add new routers such as

  - diskRouter
  - configRouter
  - lockRouter
 2016-03-27 13:28:36 -07:00
Harshavardhana
59ee5a547c release: gz doesn't preserve permissions use tar.gz 
And fix various other issues with release script.
 2016-03-26 23:44:32 -07:00
Harshavardhana
1502e2f29f Merge pull request #1265 from vadmeste/add_fbsd_support 
Add simple FreeBSD support, make the minio project compilable
 2016-03-26 22:28:07 -07:00
Anand Babu (AB) Periasamy
90a46faf31 Merge pull request #1228 from harshavardhana/signature-cleanup 
signature: Move signature outside, use a layered approach for signature verification
 2016-03-26 15:46:52 -07:00
Harshavardhana
9dca46e156 signature: Use a layered approach for signature verification. 
Signature calculation has now moved out from being a package to
top-level as a layered mechanism.

In case of payload calculation with body, go-routines are initiated
to simultaneously write and calculate shasum. Errors are sent
over the writer so that the lower layer removes the temporary files
properly.
 2016-03-26 15:21:05 -07:00
Anis Elleuch
663f24064b Add simple FreeBSD support, make the minio project compilable 2016-03-26 22:39:34 +01:00
Harshavardhana
1b0bc814c4 docker: Fix docker Makefile. 2016-03-24 22:53:13 -07:00
Anand Babu (AB) Periasamy
cd5992c6db Merge pull request #1262 from harshavardhana/docker-file 
docker: Fix docker command entry.
 2016-03-24 22:44:09 -07:00
Harshavardhana
1ef5ab3c28 docker: Fix docker command entry. 2016-03-24 20:38:36 -07:00
Anand Babu (AB) Periasamy
5bd47861d6 Merge pull request #1261 from harshavardhana/update-message 
minio: Server upon start displays a message if update is available.
 2016-03-24 20:25:36 -07:00
Harshavardhana
3538c9f598 minio: Server upon start displays a message if update is available. 
This code also handles to turn itself off when network is not
available and if request fails. Also prints only when the update
is available.
 2016-03-24 20:03:51 -07:00
Anand Babu (AB) Periasamy
24ae5467c8 Merge pull request #1260 from harshavardhana/minio 
server: Print a message if not backends are configured.
 2016-03-24 17:23:03 -07:00
Harshavardhana
36267eb6e2 server: Print a message if not backends are configured. 2016-03-24 10:47:54 -07:00
Harshavardhana
8255590b3c config/main: set the missing value. 2016-03-24 10:41:42 -07:00
Anand Babu (AB) Periasamy
4f6cf5a6b2 Merge pull request #1123 from harshavardhana/rewrite-v1 
config/main: Re-write config files - add to new config v3
 2016-03-24 08:52:35 -07:00
Harshavardhana
aaf97ea02c config/main: Re-write config files - add to new config v3 
- New config format.

```
{
	"version": "3",
	"address": ":9000",
    "backend": {
          "type": "fs",
          "disk": "/path"
    },
	"credential": {
		"accessKey": "WLGDGYAQYIGI833EV05A",
		"secretKey": "BYvgJM101sHngl2uzjXS/OBF/aMxAN06JrJ3qJlF"
	},
	"region": "us-east-1",
	"logger": {
		"file": {
			"enable": false,
			"fileName": "",
			"level": "error"
		},
		"syslog": {
			"enable": false,
			"address": "",
			"level": "debug"
		},
		"console": {
			"enable": true,
			"level": "fatal"
		}
	}
}
```

New command lines in lieu of supporting XL.

Minio initialize filesystem backend.
~~~
$ minio init fs <path>
~~~

Minio initialize XL backend.
~~~
$ minio init xl <url1>...<url16>
~~~

For 'fs' backend it starts the server.
~~~
$ minio server
~~~

For 'xl' backend it waits for servers to join.
~~~
$ minio server
... [PROGRESS BAR] of servers connecting
~~~

Now on other servers execute 'join' and they connect.
~~~
....
minio join <url1> -- from <url2> && minio server
minio join <url1> -- from <url3> && minio server
...
...
minio join <url1> -- from <url16> && minio server
~~~
 2016-03-23 19:16:09 -07:00
Harshavardhana
85e50f2bb9 Merge pull request #1258 from hackintoshrao/list-object-prefix-response 
api: GetObjectInfo fix
 2016-03-23 11:09:22 -07:00
Karthic Rao
c8570edaab Issue of 'mc ls' when prefix is a directory fixed, tests added for GetObjectInfo 2016-03-23 19:42:04 +05:30
Anand Babu (AB) Periasamy
f1161d830f Merge pull request #1257 from harshavardhana/content-md5 
routers: Fix order of PostPolicyHandlers and headers.
 2016-03-22 20:49:21 -07:00