MyFavMirrors/minio
1
0
Fork 0
mirror of https://github.com/minio/minio.git synced 2024-12-25 14:45:54 -05:00
Code Issues Packages Projects Releases Wiki Activity
3,220 Commits 4 Branches 506 Tags 189 MiB
402af93da2
Commit Graph

61 Commits

Author SHA1 Message Date
Harshavardhana
f503ac3db8 XL/Erasure: Make bit-rot verification based on xl.json algo. (#2299) 
Currently `xl.json` saves algorithm information for bit-rot
verification. Since the bit-rot algo's can change in the
future make sure the erasureReadFile doesn't default to
a particular algo. Instead use the checkSumInfo.
 2016-07-28 02:20:34 -07:00
Harshavardhana
2f7358a8a6 XL: erasure Index should have its corresponding distribution order. (#2300) 2016-07-27 11:57:08 -07:00
Krishna Srinivas
043ddbd834 optimize memory allocation during erasure-read by using temporary buffer pool. (#2259) 
* XL/erasure-read: optimize memory allocation during erasure-read by using temporary buffer pool.

With the change the buffer needed during GetObject by erasureReadFile is allocated only once.
 2016-07-25 14:17:01 -07:00
Harshavardhana
9212e11b90 XL/GetObject: When disk is not available, checksum should be empty. (#2276) 2016-07-24 22:49:27 -07:00
Harshavardhana
cef26fd6ea XL: Refactor usage of reduceErrs and consistent behavior. (#2240) 
This refactor is also needed in lieu of our quorum
requirement change for the newly understood logic behind
klauspost/reedsolom implementation.
 2016-07-19 19:24:32 -07:00
Krishnan Parthasarathi
5cc9e4e214 fs/XL: Return IncompleteBody{} error for short writes (#2228) 2016-07-18 19:06:48 -07:00
Krishna Srinivas
8cc163e51a Refactor xl.GetObject and erasureReadFile. (#2211) 
* XL: Refactor xl.GetObject and erasureReadFile. erasureReadFile() responsible for just erasure coding, it takes ordered disks and checkSum slice.

* move getOrderedPartsMetadata and getOrderedDisks to xl-v1-utils.go

* Review fixes.
 2016-07-16 08:35:30 -07:00
Krishna Srinivas
b090c7112e Refactor of xl.PutObjectPart and erasureCreateFile. (#2193) 
* XL: Refactor of xl.PutObjectPart and erasureCreateFile.

* GetCheckSum and AddCheckSum methods for xlMetaV1

* Simple unit test case for erasureCreateFile()
 2016-07-14 14:59:01 -07:00
Harshavardhana
cdf1373f8e XL: Ignore and continue for cases when bucket does not exist. (#2205) 
Fixes #2201
Fixes #2204
 2016-07-13 13:44:33 -07:00
Krishnan Parthasarathi
45240f158d xl: Make namespace locking granular for PutObject (#2199) 2016-07-13 11:56:25 -07:00
Harshavardhana
dc3bafb194 XL: isQuorum rename as isDiskQuorum, word it properly. (#2196) 2016-07-13 00:29:48 -07:00
Harshavardhana
623e0f9243 XL: listOnlineDisks should use modTime instead of version. (#2166) 
This change is needed to make reading from objects future proof
in-terms of handling online disks. Our current counter is not
based on affirmative knowledge and relies on arithmetic sequence
which can lead to bugs.

Using modTime simplifies the understanding of `xl.json` and future
tooling / debugging of the format.
 2016-07-12 15:20:31 -07:00
utsl42
e5cd35aad0 XL: GetObjectInfo() store and retrieve user-defined object metadata. (#2189) 2016-07-12 12:45:17 -07:00
Krishnan Parthasarathi
1c82b81408 Rename parts/objects only on onlineDisks (#2185) 2016-07-11 22:53:54 -07:00
Krishnan Parthasarathi
bef72f26db xl: Make locking more granular for PutObjectPart requests (#2168) 2016-07-11 17:24:49 -07:00
Krishna Srinivas
ae80f8ca35 ObjectLayer/GetObject: Should return the right error value. Fix done in FS and XL. (#2133) 
fixes #2117
 2016-07-09 13:01:32 -07:00
Harshavardhana
c0c8a8430e XL/PutObject: Add single putObject and multipart caching. (#2115) 
- Additionally adds test cases as well for object cache.
- Adds auto-expiry with expiration and cleanup time interval.

Fixes #2080
Fixes #2091
 2016-07-08 20:34:27 -07:00
Harshavardhana
6266328a85 XL/metadata: use new hashOrder algorithm for newXLMeta. (#2147) 2016-07-08 15:39:21 -07:00
Harshavardhana
ec35330ebb XL/fs: GetObject should validate all its inputs. (#2142) 
Fixes #2141
Fixes #2139
 2016-07-08 07:46:49 -07:00
Harshavardhana
ddf3245677 xl/fs: offset and length cannot be negative. (#2121) 
Fixes #2119
 2016-07-07 01:30:34 -07:00
Krishna Srinivas
26b7d79a85 XL/ObjectCache: DeleteObject() should delete the object from the object cache. (#2113) 
fixes #2103
 2016-07-06 10:25:42 -07:00
Krishna Srinivas
01cbacd803 object-cache: use golang bytes.Buffer and bytes.NewReader instead of custom implementation. (#2108) 2016-07-06 01:29:49 -07:00
karthic rao
a35341448f XL: Cache: Purging partially cached content upon erasureReadFile failure (#2104) 2016-07-05 12:48:54 -07:00
Harshavardhana
8a028a9efb handler/PUT: Handle signature verification through a custom reader. (#2066) 
Change brings in a new signVerifyReader which provides a io.Reader
compatible reader, additionally implements Verify() function.

Verify() function validates the signature present in the incoming
request. This approach is choosen to avoid complexities involved
in using io.Pipe().

Thanks to Krishna for his inputs on this.

Fixes #2058
Fixes #2054
Fixes #2087
 2016-07-05 01:04:50 -07:00
Harshavardhana
d2557bb538 XL: GetObject caching implemented for XL. (#2017) 
The object cache implementation is XL cache, which defaults
to 8GB worth of read cache. Currently GetObject() transparently
writes to this cache upon first client read and then subsequently
serves reads from the same cache.

Currently expiration is not implemented.
 2016-07-03 17:15:38 -07:00
Harshavardhana
734e779b19 XL/erasureCreate: Create a limit reader if size is specified. (#2059) 
This is needed so that we only write data which was requested
for, using a limit reader avoids spurious reads on the incoming
client data. Additionally using limit reader provides server
safety from rogue clients sending copious amounts of data (for
example a denial of service attack).

This patch also caters for size == -1 when content encoding from
a client is set as chunked, we happily read till io.EOF
 2016-07-01 14:33:28 -07:00
Bala FA
4c1a11aae6 XL: allow meta bucket name appended with tmp meta prefix. (#2007) 2016-06-29 02:28:46 -07:00
Harshavardhana
4db2b03312 XL: Rename objectN to part.N (#2019) 
Fixes #2015
 2016-06-27 21:42:33 -07:00
Harshavardhana
2e1f66c37d XL: Handle quorum situations properly for write operations. (#1986) 
Adds two test cases one for

 - PutObject when write quorum is not available.
 - PutObjectPart when write quorum is not available.

Fixes #1951
 2016-06-27 10:01:09 -07:00
Krishna Srinivas
d4bea5fbf8 XL/erasure-read: Add Comments and enable bitrot detection. 2016-06-23 00:34:47 +05:30
Krishna Srinivas
17efaaa902 XL/erasure-read: Support parallel reads from disks. 2016-06-23 00:34:47 +05:30
Krishnan Parthasarathi
393c504de0 Renaming a part from tmp namespace needs to be handled different from… (#1944) 
* Renaming a part from tmp namespace needs to be handled different from renaming of an object

* Made argument passing in xl.rename and xl.undoRename explicit
 2016-06-20 19:11:55 -07:00
Aakash Muttineni
4ee2136b28 Unit tests for PUT object when object already exists (#1904) 
* fs/xl tests for multiple put object requests
* xl fix for put object on directory
* Unit tests fix windows test issue.
 2016-06-20 06:18:47 -07:00
Bala FA
1ea1dba528 erasure-readfile: write to given Writer than returning buffer. (#1910) 
Fixes #1889
 2016-06-19 13:35:26 -07:00
Harshavardhana
8c0942bf0d XL: Remove usage of reduceErr and make it isQuorum verification. (#1909) 
Fixes #1908
 2016-06-18 00:27:51 +05:30
Krishna Srinivas
d31b38aac8 XL/GetObject: pick the xl.json with highest version for metadata information. (#1914) 
fixes #1913
 2016-06-17 10:56:18 +05:30
Krishna Srinivas
365f80efa3 XL/DeleteObject: delete call on a prefix should not delete the entire tree structure. (#1916) 
fixes #1915
 2016-06-17 10:48:43 +05:30
Anand Babu (AB) Periasamy
f51d34cedd Do not guess content-type for objects with no extension (#1918) 2016-06-17 10:12:02 +05:30
Krishna Srinivas
de1c7d33eb XL: appendFile should return error if quorum is not met. (#1898) 
Fixes #1890
 2016-06-15 00:24:49 +05:30
Bala FA
d13e6e7156 XL: return error if DeleteObject() fails. (#1878) 
Previously DeleteObject() does not return any error if write quorum is
not available.  This patch fixes the issue by returning errors.
 2016-06-07 11:35:03 -07:00
Harshavardhana
82fd907933 XL/PutObject: Handle all pending cases of DiskNotFound. 2016-06-03 11:40:44 -07:00
Krishna Srinivas
b00ac40c35 XL/PutObject: Calculate size if not provided by the client and update xl.json with the correct size. (#1844) 2016-06-02 17:09:47 -07:00
Harshavardhana
fb95c1fad3 XL: Bring in some modularity into format verification and healing. (#1832) 2016-06-02 16:34:15 -07:00
Harshavardhana
ae311aa53b XL: Cleanup, comments and all the updated functions. (#1830) 2016-06-01 16:43:31 -07:00
Harshavardhana
c493ab5d0d XL: Bring in sha512 checksum support. (#1797) 2016-05-31 20:23:31 -07:00
Krishnan Parthasarathi
967c2b2940 Handled possible short writes to httpResponseWriter (#1804) 
* XL: Handled possible short writes to httpResponseWriter

* Added tests for Range Header combinations
 2016-05-30 11:27:15 -07:00
Harshavardhana
a4a0ea605b XL: Fix GetObject erasure decode issues. (#1793) 2016-05-29 15:38:14 -07:00
Harshavardhana
5e8de786b3 XL: Truly use unique id's in temp directory. (#1790) 
This also helps in avoiding cleaning up directories after.

Additionally this patch also fixes the problem of Range offsets.
 2016-05-29 00:42:09 -07:00
Harshavardhana
feb337098d XL: bring in new storage API. (#1780) 
Fixes #1771
 2016-05-28 16:12:51 -07:00
Krishnan Parthasarathi
5f679d9d1e Rename back multipart objects if read/write Quorum was unavailable (#1773) 2016-05-28 15:15:53 -07:00