6266328a85
XL/metadata: use new hashOrder algorithm for newXLMeta. ( #2147 )
2016-07-08 15:39:21 -07:00
ca1b1921c4
XL: Implement ignore errors. ( #2136 )
...
Each metadata ops have a list of errors which can be
ignored, this is essentially needed when
- disks are not found
- disks are found but cannot be accessed (permission denied)
- disks are there but fresh disks were added
This is needed since we don't have healing code in place where
it would have healed the fresh disks added.
Fixes #2072
2016-07-07 22:10:27 -07:00
f55093cdd6
multipart: During multipart list the listing go-routine should be saved to the List-pool. ( #2130 )
2016-07-07 09:06:35 -07:00
8a028a9efb
handler/PUT: Handle signature verification through a custom reader. ( #2066 )
...
Change brings in a new signVerifyReader which provides a io.Reader
compatible reader, additionally implements Verify() function.
Verify() function validates the signature present in the incoming
request. This approach is choosen to avoid complexities involved
in using io.Pipe().
Thanks to Krishna for his inputs on this.
Fixes #2058
Fixes #2054
Fixes #2087
2016-07-05 01:04:50 -07:00
7a8b8cd0a1
tree-walk: unify FS and XL tree-walk with functional approach. ( #2027 )
2016-07-04 01:49:27 -07:00
734e779b19
XL/erasureCreate: Create a limit reader if size is specified. ( #2059 )
...
This is needed so that we only write data which was requested
for, using a limit reader avoids spurious reads on the incoming
client data. Additionally using limit reader provides server
safety from rogue clients sending copious amounts of data (for
example a denial of service attack).
This patch also caters for size == -1 when content encoding from
a client is set as chunked, we happily read till io.EOF
2016-07-01 14:33:28 -07:00
64899e5197
xl: Used unique tmp file to update xl.json in putObjectPart ( #2043 )
...
An in-place update to xl.json amidst concurrent PutObjectPart operations
lead to racy updates to xl.json making it un-parseable. To avoid this,
we create a unique tmp file where updates to xl.json are staged before
renaming it to the final location.
2016-06-30 16:28:01 -07:00
ae936a0147
XL: Relax write quorum further to N/2 + 1. ( #2018 )
...
This changes behavior in some parts of the code
as well address it.
Fixes #2016
2016-06-29 02:10:40 -07:00
748dc80047
API: add writePartTooSmallErrorResponse to extend standard error responses. ( #2005 )
...
This function is added to extend the standard error responses.
Which is needed in some cases for example CompleteMultipartUpload
should respond with ErrPartTooSmall error when parts uploaded are
lesser than 5MB (i.e minimum allowed size per part).
Fixes #1536
2016-06-28 14:51:49 -07:00
4db2b03312
XL: Rename objectN to part.N ( #2019 )
...
Fixes #2015
2016-06-27 21:42:33 -07:00
2e1f66c37d
XL: Handle quorum situations properly for write operations. ( #1986 )
...
Adds two test cases one for
- PutObject when write quorum is not available.
- PutObjectPart when write quorum is not available.
Fixes #1951
2016-06-27 10:01:09 -07:00
393c504de0
Renaming a part from tmp namespace needs to be handled different from… ( #1944 )
...
* Renaming a part from tmp namespace needs to be handled different from renaming of an object
* Made argument passing in xl.rename and xl.undoRename explicit
2016-06-20 19:11:55 -07:00
4ee2136b28
Unit tests for PUT object when object already exists ( #1904 )
...
* fs/xl tests for multiple put object requests
* xl fix for put object on directory
* Unit tests fix windows test issue.
2016-06-20 06:18:47 -07:00
e1aad066c6
XL: CompleteMultipart should ignore last part is 0bytes. ( #1931 )
...
Fixes #1917
2016-06-19 14:51:20 -07:00
8c0942bf0d
XL: Remove usage of reduceErr and make it isQuorum verification. ( #1909 )
...
Fixes #1908
2016-06-18 00:27:51 +05:30
de1c7d33eb
XL: appendFile should return error if quorum is not met. ( #1898 )
...
Fixes #1890
2016-06-15 00:24:49 +05:30
276282957e
Test for Complete Multipart Upload. ( #1888 )
2016-06-10 18:43:16 +05:30
61598ed02f
posix: return errFaultyDisk on I/O errors. ( #1885 )
...
When I/O error is occured more than allowed limit, posix returns
errFaultyDisk.
Fixes #1884
2016-06-08 22:02:10 -07:00
51f3d4e0ca
XL/multipart: statPart should ignore errDiskNotFound. ( #1862 )
...
startPart should also take uploadId and partName as arguments.
2016-06-07 18:15:04 -07:00
1cf1532ca3
XL: Implement ListObjects channel and pool management.
2016-06-03 12:17:54 -07:00
82fd907933
XL/PutObject: Handle all pending cases of DiskNotFound.
2016-06-03 11:40:44 -07:00
b00ac40c35
XL/PutObject: Calculate size if not provided by the client and update xl.json with the correct size. ( #1844 )
2016-06-02 17:09:47 -07:00
fb95c1fad3
XL: Bring in some modularity into format verification and healing. ( #1832 )
2016-06-02 16:34:15 -07:00
aa1d769b1e
FS/Multipart: remove uploads.json on complete-multipart if no more uploadIDs are present for the object. ( #1843 )
...
Fixes #1835
2016-06-02 15:54:00 -07:00
ae311aa53b
XL: Cleanup, comments and all the updated functions. ( #1830 )
2016-06-01 16:43:31 -07:00
c493ab5d0d
XL: Bring in sha512 checksum support. ( #1797 )
2016-05-31 20:23:31 -07:00
445dc22118
XL: Cleanup and add more comments. ( #1807 )
2016-05-30 16:51:59 -07:00
b466f27705
Nslock fixes ( #1803 )
...
* XL/Multipart: Support parallel upload of parts by doing NS locking appropriately.
* XL/Multipart: hold lock on the multipart upload while aborting.
2016-05-30 11:26:10 -07:00
a4a0ea605b
XL: Fix GetObject erasure decode issues. ( #1793 )
2016-05-29 15:38:14 -07:00
5e8de786b3
XL: Truly use unique id's in temp directory. ( #1790 )
...
This also helps in avoiding cleaning up directories after.
Additionally this patch also fixes the problem of Range offsets.
2016-05-29 00:42:09 -07:00
feb337098d
XL: bring in new storage API. ( #1780 )
...
Fixes #1771
2016-05-28 16:12:51 -07:00
c87f259820
Remove parts that are missing in CompleteMultipartUpload ( #1786 )
...
* Remove parts that are missing in CompleteMultipartUpload
* Moved isUploadIDExists under proper namespace locks
* Moved code that deletes part files to a function
2016-05-28 15:15:53 -07:00
41a5b3908b
XL/ListParts: take the size from xl.json instead of backend file size as it will be different. ( #1781 )
...
Fixes #1779
2016-05-28 15:15:53 -07:00
3fb0b5e455
XL/Multipart: check existance upload uploadID after lock. ( #1778 )
...
Fixes #1767
2016-05-28 15:15:53 -07:00
302ec27fa2
Fixed race during parallel PutObjectPart requests ( #1775 )
...
The race is between two parallel PutObjectPart requests updating partsInfo in xl.json.
Previously, it was being updated under a RLock().
2016-05-28 15:15:53 -07:00
d65101a8c8
XL: Implement strided erasure distribution. ( #1772 )
...
Strided erasure distribution uses a new randomized
block distribution for each Put operation. This
information is captured inside `xl.json` for subsequent
Get operations.
2016-05-28 15:15:53 -07:00
6dc8323684
FS/ListMultipart: Fix FS list-multipart to work for unit test cases.
2016-05-28 15:15:53 -07:00
b2293c2bf4
XL: Rename, cleanup and add more comments. ( #1769 )
...
- xl-v1-bucket.go - removes a whole bunch of code.
- {xl-v1,fs-v1}-metadata.go - add a lot of comments and rename functions
appropriately.
2016-05-28 15:15:53 -07:00
553fdb9211
XL: Bring in support for object versions written during writeQuorum. ( #1762 )
...
Erasure is initialized as needed depending on the quorum and onlineDisks.
This way we can manage the quorum at the object layer.
2016-05-28 15:15:53 -07:00
cae4782973
XL: explicit deleteObject is not needed after rename failure. ( #1760 )
...
Reason is renameObject() does deleteObject() upon writeQuorum
failure if not keeps the successfully renamed parts if we have
reached readQuorum.
2016-05-28 15:15:53 -07:00
a4771265cf
XL: Abortmultipart should update `uploads.json` properly. ( #1757 )
2016-05-28 15:15:53 -07:00
a9e778f460
XL/fs: initObjectLayer should cleanup tmpMetaPrefix in parallel. ( #1752 )
...
Fixes #1747
2016-05-28 15:15:53 -07:00
ee6645f421
XL: Add additional PartNumber variable as part of `xl.json` ( #1750 )
...
This is needed for verification of incoming parts and to
support variadic part uploads. Which should be sorted
properly.
Fixes #1740
2016-05-28 15:15:53 -07:00
6d84e84b3c
XL/mutltipart: fix partnumber to partname association. ( #1739 )
...
Fixes #1738
2016-05-28 15:15:53 -07:00
a00a5c6e7e
XL: Multipart update uploads.json properly. ( #1741 )
2016-05-28 15:15:53 -07:00
293d246f95
XL/FS: Rewrite in new format.
2016-05-28 15:15:53 -07:00
Harshavardhana
Krishna Srinivas
Krishnan Parthasarathi
Aakash Muttineni
karthic rao
Bala FA
Harshavardhana
Krishna Srinivas