Commit Graph

2882 Commits

Author SHA1 Message Date
Harshavardhana
a9e778f460 XL/fs: initObjectLayer should cleanup tmpMetaPrefix in parallel. (#1752)
Fixes #1747
2016-05-28 15:15:53 -07:00
Harshavardhana
ee6645f421 XL: Add additional PartNumber variable as part of xl.json (#1750)
This is needed for verification of incoming parts and to
support variadic part uploads. Which should be sorted
properly.

Fixes #1740
2016-05-28 15:15:53 -07:00
Harshavardhana
a97230dd56 XL/erasure: Reset dataBlocks to reduce the memory usage. (#1749)
Fixes #1748
2016-05-28 15:15:53 -07:00
Harshavardhana
1e393c6c5b XL: Add new metadata for checksum. (#1743) 2016-05-28 15:15:53 -07:00
Krishna Srinivas
b38b9fea79 XL/erasure: fix for skipping 0 padding. (#1737)
Fixes #1736
2016-05-28 15:15:53 -07:00
Krishna Srinivas
6d84e84b3c XL/mutltipart: fix partnumber to partname association. (#1739)
Fixes #1738
2016-05-28 15:15:53 -07:00
Harshavardhana
a00a5c6e7e XL: Multipart update uploads.json properly. (#1741) 2016-05-28 15:15:53 -07:00
Harshavardhana
ed43d5e02b No need to delete file inside erasure code (#1732) 2016-05-28 15:15:53 -07:00
Harshavardhana
293d246f95 XL/FS: Rewrite in new format. 2016-05-28 15:15:53 -07:00
Anand Babu (AB) Periasamy
63c65b4635 filter GOPATH from stack trace (#1755) 2016-05-25 02:32:35 -07:00
Harshavardhana
64b0976e1b Remove probe and tasker. (#1733)
Fixes #1717
2016-05-24 18:43:33 -07:00
Aakash Muttineni
b48b2e7f7c Part ID check (#1730)
* Added check in PutObjectPartHandler to make sure part ID does not exceed 10000. ErrInvalidMaxParts written to response if part ID exceeds the maximum value.
2016-05-24 01:52:47 -07:00
Krishnan Parthasarathi
584813e214 Used MINIO_PROFILE_DIR for saving profile information of a minio server (#1722)
To specify the directory where profiling information should be saved
  ```
    export MINIO_PROFILE_DIR=/path/to/profile/dir
  ```
By default, profiling information would be saved in a directory created
using ioutil.TempDir, which would be displayed in stdout on starting the
minio server.
2016-05-22 22:11:39 -07:00
Krishnan Parthasarathi
3a980eac1a Fix shadowing of err variable (#1718) 2016-05-21 00:43:47 -07:00
Harshavardhana
5a4b074ca0 XL: PutObject incorrectly returned after deleting multipart object. (#1715)
Fixes #1714
2016-05-20 17:27:48 -07:00
Harshavardhana
f76d975304 xl: StatVol and ListVols should handle cases when disks are missing. (#1703)
Remove unnecessary code, handle cases of missing disk.
2016-05-20 16:45:53 -07:00
Harshavardhana
6015a7a3cd Add mention-bot config 2016-05-20 13:53:15 -07:00
Krishna Srinivas
b83b87a7f6 XL/Incompleteuploads: list should save the tree-walk go routine to the map if eof is not reached. (#1695) 2016-05-20 12:09:21 -07:00
Krishna Srinivas
5b95f097d4 multipart: listing does not skip uploadIDmarker. (#1708)
Fixes #1706
2016-05-20 11:48:28 -07:00
Harshavardhana
e4240aa58f XL/objects: Initialize format.json outside of erasure. (#1640)
Fixes #1636

New format now generates a UUID and includes it along with
the order of disks. So that UUID is the real order of disks
and on command line user is able to specify disks in any order.

This pre-dominantly solves our dilemma.
```
{
   "format" : "xl",
   "xl" : {
      "version" : "1",
      "disk": "00e4cf06-5bf5-4bb5-b885-4b2fff4a7959",
      "jbod" : [
         "00e4cf06-5bf5-4bb5-b885-4b2fff4a7959",
         ....
         "c47d2608-5067-4ed7-b1e4-fb81bdbb549f",
         "a543293e-99f1-4310-b540-1e450878e844",
         "18f97cbe-529a-456a-b6d4-0feacf64534d"
      ]
   },
   "version" : "1"
}
```
2016-05-20 02:22:22 -07:00
Andreas Linz
f5dfa895a5 Exit with error code if minio server fails to start (#1704)
This commit replaces the call to `errorIf` with `fatalIf`, so that the
minio server exits with an non-zero exit status if something fails, e.g.
the port was already openend by another process.
2016-05-20 02:05:52 -07:00
Harshavardhana
50c328ff19 XL: RenameFile should rename and cleanup on writeQuorum. (#1702)
Fixes #1683
2016-05-20 01:56:46 -07:00
Anand Babu (AB) Periasamy
b8405ca172 simplify profiler cleanup 2016-05-19 19:19:32 -07:00
Harshavardhana
f6d9e73548 posix: Do not lowercase names, return as is. Object layer will filter them out. (#1699) 2016-05-19 18:52:55 -07:00
Harshavardhana
7ae5470395 XL: simplify isMultipartObject not need to handle unknown errors. (#1686)
Unknown errors are just logged with errorIf.
2016-05-19 17:10:33 -07:00
Harshavardhana
9fdb69563d handler: CopyObject should save metadata. (#1698)
- Content-Type
- Content-Encoding
- ETag

Fixes #1682
2016-05-19 17:10:08 -07:00
Harshavardhana
00d0558131 XL: Enable tests for content-type for Head and Get
Fixes #1674
2016-05-19 15:34:18 -07:00
Bala.FA
13e4618309 XL/fs: Return saved content-type during GetObject
Fixes #1674
2016-05-19 15:34:18 -07:00
Krishnan Parthasarathi
2f05aacbf2 Stop profiling on exit of main goroutine (#1670)
* Stop profiling on exit of main goroutine

Previously, profiling was stopped since Stop() method was called on exit of cli.BeforeFunc.
This lead to profiling to be stopped prematurely.

* Moved profiling switch statement to a separate func
2016-05-19 14:50:54 -07:00
Krishna Srinivas
dc36594ef4 XL/heal: Should skip healing if CreateFile() failed on the part which needed healing. (#1693)
Fixes #1684
2016-05-19 11:32:47 -07:00
Krishna Srinivas
537568f9f9 XL/ListVols: Fix panic. Skip if slice is nil. (#1694)
Fixes #1692
2016-05-19 11:32:19 -07:00
Harshavardhana
62b4fd6964 XL: Close the reader properly. 2016-05-18 20:16:19 -07:00
Harshavardhana
7d6ed50fc2 objects: Save all the incoming metadata properly. (#1688)
For both multipart and single put operation
2016-05-18 19:54:25 -07:00
Frank
af85acf388 Added ETag as an exposed header (required for multi part uploads) (#1681)
* Added ETag as an exposed header (required for multi part uploads)

* Fix formatting on adding ETag for exposed headers
2016-05-18 19:17:32 -07:00
Frank
a4fef436c8 Fix formatting for adding ETag for exposed headers (#1687) 2016-05-18 19:17:17 -07:00
Harshavardhana
404364ba73 XL/fs: ListMultipartUploads should list only requested entries. (#1668)
Fixes #1665
2016-05-18 15:06:29 -07:00
Krishna Srinivas
3c1ef3fee2 Locking: move locking code from xl-erasure to xl-objects. (#1673)
Fixes #1639 #1649 #1650 #1624
2016-05-18 15:05:23 -07:00
Bala FA
a0865122a7 XL/objectLayer: Save additional meta data during PutObject (#1672)
Fixes  #1602
2016-05-18 13:56:11 -07:00
Krishna Srinivas
824c8a39f1 XL/Multipart: If the part is already uploaded ignore the newly uploaded part. (#1677)
Fixes #1644
2016-05-18 13:37:28 -07:00
Krishna Srinivas
71b9341fc7 XL/Multipart: Cleanup uploads.json after abort-multipart-upload. (#1678)
Fixes #1663
2016-05-18 13:30:58 -07:00
Harshavardhana
b69a97aed4 server: Set rLimit properly to the max. (#1676)
4000 is too small to handle 500 go-routines.

Fixes #1666
2016-05-18 11:34:24 -07:00
karthic rao
2da34e4668 vendor changes to pkg/profile (#1671) 2016-05-18 09:22:06 -07:00
Harshavardhana
4bc923e63b XL/fs: Optimize calling isBucketExist() (#1656)
* posix: Avoid using getAllVolumeInfo() in getVolumeDir()

This is necessary compromise to avoid significant slowness this
causes under load. The compromise is also substantial in a way
so that to avoid penalizing common cases v/s special cases.

For buckets with Caps on Unixes, we filter buckets based on the
latest anyways, so this is completely acceptable.

* XL/fs: Change the usage of verification of existance of buckets.

Optimize calling isBucketExists, it is not needed for all call
paths. isBucketExist should be called only for calls which use
temporary volume location for operations, for the rest rely on
the errors returned on their original call path.

Remove usage of filtering as well across all volume names.
2016-05-17 21:22:27 -07:00
Harshavardhana
4214da65af XL/fs: MakeVol replies should be consistent. (#1667)
Fixes #1658
Fixes #1633
2016-05-17 18:41:17 -07:00
Krishnan Parthasarathi
596fe65e84 Write pprof output files under config dir supplied (#1660)
Since config dir, supplied as command line argument, is parsed after pprof
output directory is determined, pprof output files are  written in ~/.minio/profile
directory instead of <configDir>/profile/. This change fixes this behaviour.
2016-05-17 11:44:40 -07:00
Krishna Srinivas
39865c0d2e XL/Multipart: Fix list multipart output. (#1661)
Fixes #1541
2016-05-17 11:44:32 -07:00
Harshavardhana
1760687c83 XL: Make sure to create proper temporary files for renames to succeed. (#1654)
Renames work in a special manner, temporary location files should
be created properly.

Fixes #1653
Fixes #1651
2016-05-16 15:40:57 -07:00
Harshavardhana
9472299308 logging: Log only for unhandled errors, remove all the debug logging. (#1652)
This patch brings in the removal of debug logging altogether, instead
we bring in the functionality of being able to trace the errors properly
pointing back to the origination of the problem.

To enable tracing you need to enable "MINIO_TRACE" set to "1" or "true"
environment variable which would print back traces whenever there is an
error which is unhandled or at the handler layer.

By default this tracing is turned off and only user level logging is
provided.
2016-05-16 14:31:28 -07:00
Harshavardhana
8828fd1e5c vendor: Remove unused packages. 2016-05-15 00:04:56 -07:00
Harshavardhana
7de206cb85 XL: ListVols should provide consistent view. (#1648)
Additionally get list of all volumes in parallel for aggregation
and quorum verification.

Fixes #1647
2016-05-14 23:57:57 -07:00