Commit Graph

3241 Commits

Author SHA1 Message Date
Harshavardhana
a0635dcdd9 XL: Do not rely on getLoadBalancedQuorumDisks for NS consistency. (#2243)
The reason is any function relying on `getLoadBalancedQuorumDisks`
cannot possibly have an idempotent behavior.

The problem comes from given a set of N disks returning just a
shuffled N/2 disks.  In case of a scenario where we have N/2
number of failed disks, the returned value of `getLoadBalancedQuorumDisks`
is not equal to the same failed disks so essentially calls using such
disks might succeed or fail randomly at different intervals in time.

This proposal change is we move to `getLoadBalancedDisks()`
and use the shuffled N disks as a whole. Since most of the time we might
hit a good disk since we are not reducing our solution space. This
also provides consistent behavior for all the functions which rely
on shuffled disks.

Fixes #2242
2016-07-21 00:27:08 -07:00
Dee Koder
41f4f2806d screenshots: update with the latest optimized image. (#2249) 2016-07-20 16:15:26 -07:00
Dee Koder
2a972ef1fd images: Move screenshot for docs inside docs/screenshots directory. (#2248)
* images: Move screenshot for docs inside docs/screenshots directory. Use optimized images.

* images: This fix optimizes the images for the Erasure Code Quick Start Guide
2016-07-20 13:52:30 -07:00
Harshavardhana
c1e953b368 api: Set content-encoding properly if set. (#2245)
Additionally don't set content-type if not present, golang http
server automaticaly handles this and sets it automatically.
2016-07-20 12:40:20 -07:00
Krishna Srinivas
18728a0b59 XL/erasure-read: refactor erasure read and add tests (#2232) 2016-07-20 01:30:30 -07:00
Harshavardhana
cef26fd6ea XL: Refactor usage of reduceErrs and consistent behavior. (#2240)
This refactor is also needed in lieu of our quorum
requirement change for the newly understood logic behind
klauspost/reedsolom implementation.
2016-07-19 19:24:32 -07:00
Dee Koder
f67c930731 doc: Fixed spacing with respect to code blocks. (#2241) 2016-07-19 19:08:43 -07:00
GarimaKapoor
3589a58179 Update Minio-erasure-code-quickStart-guide.md 2016-07-19 17:59:05 -07:00
Dee Koder
e8155abc18 screenshot: Use the full path to the screenshot when embedding images (#2239) 2016-07-19 17:48:18 -07:00
Dee Koder
2e8360120d headings: We need to add a consistent heading for all docs. Adding Minio FreeBSD QuickStart Guide in the title. (#2233) 2016-07-19 14:56:34 -07:00
Dee Koder
02b191222c headings: Added standardized heading for this document. (#2234) 2016-07-19 14:56:20 -07:00
Dee Koder
b699795901 docs: Remove additional headings. Added standard heading. Include numbering. (#2235) 2016-07-19 14:30:32 -07:00
Harshavardhana
86d31e99d5 api: use checkAuth now at PutBucket, DeleteBucket handlers. (#2225)
Additionally add a unit test for isReqAuthenticated function.
2016-07-18 23:56:27 -07:00
Krishna Srinivas
897d78d113 erasureReadFile and erasureCreateFile testcases. (#2229)
* unit-tests: Unit tests for erasureCreateFile and erasureReadFile.

* appendFile() should return errXLWriteQuorum.

* TestErasureReadFileOffsetLength() tests erasureReadFile() for different offset and lengths.

* Fix for the failure seen in the erasure read unit test case. Issue #2227

* Move common erasure setup code to newErasureTestSetup()

* Review fixes. Add few more test cases for erasureReadFile.
2016-07-18 23:56:16 -07:00
Harshavardhana
1f706e067d api: xmlDecoder should honor contentLength. (#2226)
This is needed so that we avoid reading large amounts
of data from compromised clients.
2016-07-18 21:20:17 -07:00
Krishnan Parthasarathi
5cc9e4e214 fs/XL: Return IncompleteBody{} error for short writes (#2228) 2016-07-18 19:06:48 -07:00
Krishna Srinivas
27a5b61f40 tree-walk: optimize tree walk such that leaf detection of entries is delayed till the entry is sent on the treeWalkResult channel. (#2220) 2016-07-17 15:16:52 -07:00
Harshavardhana
aeac902747 API: ListBuckets doesn't have a body, we should never read the body. (#2218)
ListBuckets was incorrectly reading the body of the request, fix it.
2016-07-17 13:23:15 -07:00
Harshavardhana
aaf7803831 api: Requests should be differentiated if possible based on http router. (#2219)
In current master ListObjectsV2 was merged into ListObjectsHandler
which also implements V1 API as well.

Move the detection of ListObject types to its rightful place
in http router.
2016-07-17 12:32:05 -07:00
Krishna Srinivas
8cc163e51a Refactor xl.GetObject and erasureReadFile. (#2211)
* XL: Refactor xl.GetObject and erasureReadFile. erasureReadFile() responsible for just erasure coding, it takes ordered disks and checkSum slice.

* move getOrderedPartsMetadata and getOrderedDisks to xl-v1-utils.go

* Review fixes.
2016-07-16 08:35:30 -07:00
Harshavardhana
2d38046a5a utils: BucketNames with double periods and ip address should be rejected. (#2213)
Fixes #2212
2016-07-15 17:30:37 -07:00
Harshavardhana
cbb6b48b94 doc: update README.md 2016-07-15 16:12:54 -07:00
Harshavardhana
d0636d633d doc: Move FreeBSD.md to docs. 2016-07-15 16:09:01 -07:00
Harshavardhana
41187fc2ef docker: Fix docker edge build 2016-07-15 15:10:38 -07:00
koolhead17
204ec2c6c0 doc:README.md/Updated to sync with docs.minio.io (#2210)
* doc:README.md/Updated to sync with docs.minio.io

* doc:README.me/Modified the minio server output terminal to reflect new release changes.

* docs:README.md/Modified and changed location of other markdown files.
2016-07-15 15:03:59 -07:00
Krishnan Parthasarathi
3bce5db6d1 tests: Add tests to treeWalk for sortedness, recursive listing and isEnd behaviour (#2209) 2016-07-14 18:37:43 -07:00
Harshavardhana
35d438e0ff vendorize: update all vendorized packages. (#2206)
Bring in new changes from upstream for all the packages.

Important ones include
   - gorilla/mux
   - logrus
   - jwt
2016-07-14 14:59:20 -07:00
Krishna Srinivas
b090c7112e Refactor of xl.PutObjectPart and erasureCreateFile. (#2193)
* XL: Refactor of xl.PutObjectPart and erasureCreateFile.

* GetCheckSum and AddCheckSum methods for xlMetaV1

* Simple unit test case for erasureCreateFile()
2016-07-14 14:59:01 -07:00
Harshavardhana
af6109f89a update: Remove extraneous '/' in update message. (#2207) 2016-07-14 14:08:16 -07:00
Anis Elleuch
3f27734c22 Use normal color instead of forced white for users who have bright terminal background (#2200) 2016-07-13 14:27:36 -07:00
Harshavardhana
cdf1373f8e XL: Ignore and continue for cases when bucket does not exist. (#2205)
Fixes #2201
Fixes #2204
2016-07-13 13:44:33 -07:00
Krishnan Parthasarathi
45240f158d xl: Make namespace locking granular for PutObject (#2199) 2016-07-13 11:56:25 -07:00
Harshavardhana
0bd6b67ca5 server: Sort ips based on their last octet value. (#2198) 2016-07-13 06:34:59 -07:00
Harshavardhana
8c84df5e74 server: Change color codes for headings and sub-headings. (#2197)
This patch changes the color coding used for headings, sub-headings
and values as finalized.
2016-07-13 00:56:00 -07:00
Harshavardhana
dc3bafb194 XL: isQuorum rename as isDiskQuorum, word it properly. (#2196) 2016-07-13 00:29:48 -07:00
Harshavardhana
3b69b4ada4 server: Change server startup message. (#2195)
This change brings in the new agreed startup message
for the server.

Adds additional links point to Minio SDKs as well.
2016-07-12 23:21:18 -07:00
Krishnan Parthasarathi
0610527868 XL: PutObjectPart update checksum, re-read from xl.json for the part being written. (#2191) 2016-07-12 18:23:40 -07:00
Harshavardhana
0fcfb5df3b XL/fs: Change minioMetaBucket different than '.minio' config dir. (#2190)
This fixes corruption of config directory seen when minio server
exports 'home' directory.

```
minio server ~
```
2016-07-12 15:21:29 -07:00
Harshavardhana
623e0f9243 XL: listOnlineDisks should use modTime instead of version. (#2166)
This change is needed to make reading from objects future proof
in-terms of handling online disks. Our current counter is not
based on affirmative knowledge and relies on arithmetic sequence
which can lead to bugs.

Using modTime simplifies the understanding of `xl.json` and future
tooling / debugging of the format.
2016-07-12 15:20:31 -07:00
utsl42
e5cd35aad0 XL: GetObjectInfo() store and retrieve user-defined object metadata. (#2189) 2016-07-12 12:45:17 -07:00
Anis Elleuch
5cd629adca XL/fs: DeleteVol should not return error cleaning multipart dir for errVolumeNotFound (#2188) 2016-07-12 10:07:32 -07:00
Anis Elleuch
0fddf3fe17 Avoid creating tmp directories under .minio/tmp/ to facilitate cleaning (#2187) 2016-07-12 09:38:45 -07:00
karthic rao
ac6ff67546 Tool for running benchmark comparison of 2 commits (#2161) 2016-07-12 02:08:38 -07:00
Harshavardhana
126865e8df XL/bucket: Remove bucket should cleanup incomplete uploads as well. (#2173)
This behavior is in accordance with S3.

Fixes #2170
2016-07-12 01:01:47 -07:00
Krishnan Parthasarathi
1c82b81408 Rename parts/objects only on onlineDisks (#2185) 2016-07-11 22:53:54 -07:00
Bala FA
749a94f6c9 tests: Add tests for signature-jwt code (#2169)
Fixes #1989
2016-07-11 21:57:40 -07:00
Harshavardhana
e9647b5f12 API/CopyObject: Refactor the code and handle if-modified-since as well. (#2183)
This completes the S3 spec behavior for CopyObject API as reported
by `s3verify`.
2016-07-11 19:24:34 -07:00
Krishnan Parthasarathi
bef72f26db xl: Make locking more granular for PutObjectPart requests (#2168) 2016-07-11 17:24:49 -07:00
Harshavardhana
ede4dd0f9c server: update command should check for 3s from 1ms. (#2175)
Programmer error :-)
2016-07-11 16:22:10 -07:00
Bala FA
bfc59b7d50 tests: improve unit tests for xl-v1-metadata. (#2172)
Fixes #2124
2016-07-11 11:42:01 -07:00