MyFavMirrors/minio
1
0
Fork 0
mirror of https://github.com/minio/minio.git synced 2024-12-26 07:05:55 -05:00
Code Issues Packages Projects Releases Wiki Activity
3,176 Commits 4 Branches 506 Tags 189 MiB
ad19bf0ec1
Commit Graph

44 Commits

Author SHA1 Message Date
Harshavardhana
9212e11b90 XL/GetObject: When disk is not available, checksum should be empty. (#2276) 2016-07-24 22:49:27 -07:00
Harshavardhana
79bab6b561 XL: Operations on uploads.json should cater for disk being unavailable. (#2277) 2016-07-24 18:08:15 -07:00
Harshavardhana
6c2fb19ed7 docs: Removed and purged uneeded docs. (#2273) 2016-07-24 03:32:45 -07:00
Harshavardhana
0add96f655 fs: Save metadata for objects in minioMetaBucket directory. (#2251) 2016-07-21 17:31:14 -07:00
Krishna Srinivas
303f216150 tests: xl-v1-metadata.go, xl-v1-multipart-common.go - remove unused methods, add enhance tests to improve code coverage. (#2260) 2016-07-21 15:00:11 -07:00
Harshavardhana
a0635dcdd9 XL: Do not rely on getLoadBalancedQuorumDisks for NS consistency. (#2243) 
The reason is any function relying on `getLoadBalancedQuorumDisks`
cannot possibly have an idempotent behavior.

The problem comes from given a set of N disks returning just a
shuffled N/2 disks.  In case of a scenario where we have N/2
number of failed disks, the returned value of `getLoadBalancedQuorumDisks`
is not equal to the same failed disks so essentially calls using such
disks might succeed or fail randomly at different intervals in time.

This proposal change is we move to `getLoadBalancedDisks()`
and use the shuffled N disks as a whole. Since most of the time we might
hit a good disk since we are not reducing our solution space. This
also provides consistent behavior for all the functions which rely
on shuffled disks.

Fixes #2242
 2016-07-21 00:27:08 -07:00
Harshavardhana
cef26fd6ea XL: Refactor usage of reduceErrs and consistent behavior. (#2240) 
This refactor is also needed in lieu of our quorum
requirement change for the newly understood logic behind
klauspost/reedsolom implementation.
 2016-07-19 19:24:32 -07:00
Krishna Srinivas
b090c7112e Refactor of xl.PutObjectPart and erasureCreateFile. (#2193) 
* XL: Refactor of xl.PutObjectPart and erasureCreateFile.

* GetCheckSum and AddCheckSum methods for xlMetaV1

* Simple unit test case for erasureCreateFile()
 2016-07-14 14:59:01 -07:00
Harshavardhana
cdf1373f8e XL: Ignore and continue for cases when bucket does not exist. (#2205) 
Fixes #2201
Fixes #2204
 2016-07-13 13:44:33 -07:00
Harshavardhana
dc3bafb194 XL: isQuorum rename as isDiskQuorum, word it properly. (#2196) 2016-07-13 00:29:48 -07:00
Harshavardhana
623e0f9243 XL: listOnlineDisks should use modTime instead of version. (#2166) 
This change is needed to make reading from objects future proof
in-terms of handling online disks. Our current counter is not
based on affirmative knowledge and relies on arithmetic sequence
which can lead to bugs.

Using modTime simplifies the understanding of `xl.json` and future
tooling / debugging of the format.
 2016-07-12 15:20:31 -07:00
Krishnan Parthasarathi
bef72f26db xl: Make locking more granular for PutObjectPart requests (#2168) 2016-07-11 17:24:49 -07:00
Harshavardhana
de468f92ec posix: ReadAll should handle the case when parent is not a dir. (#2163) 
It can happen so that a read request can come for a file which
already has a parent i.e a file.

This fix handles this scenario - fixes #2047
 2016-07-11 00:15:37 -07:00
Krishna Srinivas
aa7079fc7b XL/GetObject: If quorum not available during GetObject appropriate error should be returned. (#2135) 2016-07-10 17:12:22 -07:00
Harshavardhana
6266328a85 XL/metadata: use new hashOrder algorithm for newXLMeta. (#2147) 2016-07-08 15:39:21 -07:00
frankw
63b3f1dcfd Use new algorithm to get fixed random order of disks (#2147) 2016-07-08 15:38:47 -07:00
Harshavardhana
ca1b1921c4 XL: Implement ignore errors. (#2136) 
Each metadata ops have a list of errors which can be
ignored, this is essentially needed when

  - disks are not found
  - disks are found but cannot be accessed (permission denied)
  - disks are there but fresh disks were added

This is needed since we don't have healing code in place where
it would have healed the fresh disks added.

Fixes #2072
 2016-07-07 22:10:27 -07:00
Harshavardhana
ae936a0147 XL: Relax write quorum further to N/2 + 1. (#2018) 
This changes behavior in some parts of the code
as well address it.

Fixes #2016
 2016-06-29 02:10:40 -07:00
Harshavardhana
e8990e42c2 XL: Make allocations simpler avoid redundant allocs. (#1961) 
- Reduce 10MiB buffers for loopy calls to use 128KiB.
- start using 128KiB buffer where needed.
 2016-06-24 02:06:23 -07:00
Harshavardhana
50d25ca94a XL: Change AppendFile() to return only error (#1932) 
AppendFile ensures that it appends the entire buffer. Returns
an error otherwise, this patch removes the necessity for the
caller to look for 'n' return on short writes.

Ref #1893
 2016-06-19 15:31:13 -07:00
Harshavardhana
e1aad066c6 XL: CompleteMultipart should ignore last part is 0bytes. (#1931) 
Fixes #1917
 2016-06-19 14:51:20 -07:00
Harshavardhana
8c0942bf0d XL: Remove usage of reduceErr and make it isQuorum verification. (#1909) 
Fixes #1908
 2016-06-18 00:27:51 +05:30
Bala FA
61598ed02f posix: return errFaultyDisk on I/O errors. (#1885) 
When I/O error is occured more than allowed limit, posix returns
errFaultyDisk.

Fixes #1884
 2016-06-08 22:02:10 -07:00
Harshavardhana
82fd907933 XL/PutObject: Handle all pending cases of DiskNotFound. 2016-06-03 11:40:44 -07:00
Harshavardhana
fb95c1fad3 XL: Bring in some modularity into format verification and healing. (#1832) 2016-06-02 16:34:15 -07:00
Harshavardhana
ae311aa53b XL: Cleanup, comments and all the updated functions. (#1830) 2016-06-01 16:43:31 -07:00
Harshavardhana
c493ab5d0d XL: Bring in sha512 checksum support. (#1797) 2016-05-31 20:23:31 -07:00
Harshavardhana
445dc22118 XL: Cleanup and add more comments. (#1807) 2016-05-30 16:51:59 -07:00
Harshavardhana
a4a0ea605b XL: Fix GetObject erasure decode issues. (#1793) 2016-05-29 15:38:14 -07:00
Harshavardhana
5e8de786b3 XL: Truly use unique id's in temp directory. (#1790) 
This also helps in avoiding cleaning up directories after.

Additionally this patch also fixes the problem of Range offsets.
 2016-05-29 00:42:09 -07:00
Harshavardhana
feb337098d XL: bring in new storage API. (#1780) 
Fixes #1771
 2016-05-28 16:12:51 -07:00
Harshavardhana
d65101a8c8 XL: Implement strided erasure distribution. (#1772) 
Strided erasure distribution uses a new randomized
block distribution for each Put operation. This
information is captured inside `xl.json` for subsequent
Get operations.
 2016-05-28 15:15:53 -07:00
Harshavardhana
b2293c2bf4 XL: Rename, cleanup and add more comments. (#1769) 
- xl-v1-bucket.go - removes a whole bunch of code.
- {xl-v1,fs-v1}-metadata.go - add a lot of comments and rename functions
   appropriately.
 2016-05-28 15:15:53 -07:00
Harshavardhana
553fdb9211 XL: Bring in support for object versions written during writeQuorum. (#1762) 
Erasure is initialized as needed depending on the quorum and onlineDisks.
This way we can manage the quorum at the object layer.
 2016-05-28 15:15:53 -07:00
Harshavardhana
a9e778f460 XL/fs: initObjectLayer should cleanup tmpMetaPrefix in parallel. (#1752) 
Fixes #1747
 2016-05-28 15:15:53 -07:00
Harshavardhana
ee6645f421 XL: Add additional PartNumber variable as part of xl.json (#1750) 
This is needed for verification of incoming parts and to
support variadic part uploads. Which should be sorted
properly.

Fixes #1740
 2016-05-28 15:15:53 -07:00
Harshavardhana
1e393c6c5b XL: Add new metadata for checksum. (#1743) 2016-05-28 15:15:53 -07:00
Harshavardhana
a00a5c6e7e XL: Multipart update uploads.json properly. (#1741) 2016-05-28 15:15:53 -07:00
Harshavardhana
293d246f95 XL/FS: Rewrite in new format. 2016-05-28 15:15:53 -07:00
Harshavardhana
4e34e03dd4 xl/fs: Split object layer into interface. (#1415) 2016-04-29 14:24:10 -07:00
Harshavardhana
a1a667ae5d xl: Change fileMetadata to xlMetadata. (#1404) 
Finalized backend format

```
{
    "version": "1.0.0",
    "stat": {
        "size": 24256,
        "modTime": "2016-04-28T00:11:37.843Z"
    },
    "erasure": {
        "data": 5,
        "parity": 5,
        "blockSize": 4194304
    ],
    "minio": {
        "release": "RELEASE.2016-04-28T00-09-47Z"
    }
}
```
 2016-04-28 19:27:02 -07:00
Krishna Srinivas
8c85815106 xl: refactor functions to xl-v1-common.go xl-v1-utils.go. (#1357) 2016-04-25 12:47:31 -07:00
Krishna Srinivas
becc814531 Xl layer selfheal quorum2 
* xl/selfheal: selfheal based on read quorum on GET

* xl: getReadableDisks() also returns whether self-heal is needed so that this info can be used by ReadFile/SelfHeal/StatFile.

* xl: trigger selfheal from StatFile.
 2016-04-25 12:47:31 -07:00
Harshavardhana
9bd9441107 xl: Simplify reading metadata and add a new fileMetadata type. (#1346) 2016-04-25 12:47:31 -07:00