Commit Graph

2958 Commits

Author SHA1 Message Date
Harshavardhana
7ae5470395 XL: simplify isMultipartObject not need to handle unknown errors. (#1686)
Unknown errors are just logged with errorIf.
2016-05-19 17:10:33 -07:00
Harshavardhana
9fdb69563d handler: CopyObject should save metadata. (#1698)
- Content-Type
- Content-Encoding
- ETag

Fixes #1682
2016-05-19 17:10:08 -07:00
Harshavardhana
00d0558131 XL: Enable tests for content-type for Head and Get
Fixes #1674
2016-05-19 15:34:18 -07:00
Bala.FA
13e4618309 XL/fs: Return saved content-type during GetObject
Fixes #1674
2016-05-19 15:34:18 -07:00
Krishnan Parthasarathi
2f05aacbf2 Stop profiling on exit of main goroutine (#1670)
* Stop profiling on exit of main goroutine

Previously, profiling was stopped since Stop() method was called on exit of cli.BeforeFunc.
This lead to profiling to be stopped prematurely.

* Moved profiling switch statement to a separate func
2016-05-19 14:50:54 -07:00
Krishna Srinivas
dc36594ef4 XL/heal: Should skip healing if CreateFile() failed on the part which needed healing. (#1693)
Fixes #1684
2016-05-19 11:32:47 -07:00
Krishna Srinivas
537568f9f9 XL/ListVols: Fix panic. Skip if slice is nil. (#1694)
Fixes #1692
2016-05-19 11:32:19 -07:00
Harshavardhana
62b4fd6964 XL: Close the reader properly. 2016-05-18 20:16:19 -07:00
Harshavardhana
7d6ed50fc2 objects: Save all the incoming metadata properly. (#1688)
For both multipart and single put operation
2016-05-18 19:54:25 -07:00
Frank
af85acf388 Added ETag as an exposed header (required for multi part uploads) (#1681)
* Added ETag as an exposed header (required for multi part uploads)

* Fix formatting on adding ETag for exposed headers
2016-05-18 19:17:32 -07:00
Frank
a4fef436c8 Fix formatting for adding ETag for exposed headers (#1687) 2016-05-18 19:17:17 -07:00
Harshavardhana
404364ba73 XL/fs: ListMultipartUploads should list only requested entries. (#1668)
Fixes #1665
2016-05-18 15:06:29 -07:00
Krishna Srinivas
3c1ef3fee2 Locking: move locking code from xl-erasure to xl-objects. (#1673)
Fixes #1639 #1649 #1650 #1624
2016-05-18 15:05:23 -07:00
Bala FA
a0865122a7 XL/objectLayer: Save additional meta data during PutObject (#1672)
Fixes  #1602
2016-05-18 13:56:11 -07:00
Krishna Srinivas
824c8a39f1 XL/Multipart: If the part is already uploaded ignore the newly uploaded part. (#1677)
Fixes #1644
2016-05-18 13:37:28 -07:00
Krishna Srinivas
71b9341fc7 XL/Multipart: Cleanup uploads.json after abort-multipart-upload. (#1678)
Fixes #1663
2016-05-18 13:30:58 -07:00
Harshavardhana
b69a97aed4 server: Set rLimit properly to the max. (#1676)
4000 is too small to handle 500 go-routines.

Fixes #1666
2016-05-18 11:34:24 -07:00
karthic rao
2da34e4668 vendor changes to pkg/profile (#1671) 2016-05-18 09:22:06 -07:00
Harshavardhana
4bc923e63b XL/fs: Optimize calling isBucketExist() (#1656)
* posix: Avoid using getAllVolumeInfo() in getVolumeDir()

This is necessary compromise to avoid significant slowness this
causes under load. The compromise is also substantial in a way
so that to avoid penalizing common cases v/s special cases.

For buckets with Caps on Unixes, we filter buckets based on the
latest anyways, so this is completely acceptable.

* XL/fs: Change the usage of verification of existance of buckets.

Optimize calling isBucketExists, it is not needed for all call
paths. isBucketExist should be called only for calls which use
temporary volume location for operations, for the rest rely on
the errors returned on their original call path.

Remove usage of filtering as well across all volume names.
2016-05-17 21:22:27 -07:00
Harshavardhana
4214da65af XL/fs: MakeVol replies should be consistent. (#1667)
Fixes #1658
Fixes #1633
2016-05-17 18:41:17 -07:00
Krishnan Parthasarathi
596fe65e84 Write pprof output files under config dir supplied (#1660)
Since config dir, supplied as command line argument, is parsed after pprof
output directory is determined, pprof output files are  written in ~/.minio/profile
directory instead of <configDir>/profile/. This change fixes this behaviour.
2016-05-17 11:44:40 -07:00
Krishna Srinivas
39865c0d2e XL/Multipart: Fix list multipart output. (#1661)
Fixes #1541
2016-05-17 11:44:32 -07:00
Harshavardhana
1760687c83 XL: Make sure to create proper temporary files for renames to succeed. (#1654)
Renames work in a special manner, temporary location files should
be created properly.

Fixes #1653
Fixes #1651
2016-05-16 15:40:57 -07:00
Harshavardhana
9472299308 logging: Log only for unhandled errors, remove all the debug logging. (#1652)
This patch brings in the removal of debug logging altogether, instead
we bring in the functionality of being able to trace the errors properly
pointing back to the origination of the problem.

To enable tracing you need to enable "MINIO_TRACE" set to "1" or "true"
environment variable which would print back traces whenever there is an
error which is unhandled or at the handler layer.

By default this tracing is turned off and only user level logging is
provided.
2016-05-16 14:31:28 -07:00
Harshavardhana
8828fd1e5c vendor: Remove unused packages. 2016-05-15 00:04:56 -07:00
Harshavardhana
7de206cb85 XL: ListVols should provide consistent view. (#1648)
Additionally get list of all volumes in parallel for aggregation
and quorum verification.

Fixes #1647
2016-05-14 23:57:57 -07:00
Harshavardhana
498ce1e9bb handler: Add a waitgroup to avoid expect100Continue crash. (#1623)
This waitgroup allows for safe blocking operation where we can cleanly
control the flow of the writes and the underlying pipe altogether.

Fixes #1553
2016-05-14 17:18:00 -07:00
Harshavardhana
5b29cefd40 api: DeleteObject should always return 204. (#1645)
Fixes #1643
2016-05-14 15:47:19 -07:00
Hori Ryota
e03ebfd13b Add default cmd (#1625) 2016-05-14 02:58:50 -07:00
Harshavardhana
74c23a3544 docs: Move developer docs from top-level to its own directory. (#1642) 2016-05-14 02:47:16 -07:00
Harshavardhana
025054fb36 XL: CreateFile/ReadFile should write and read from all disks in parallel. (#1612)
* XL: CreateFile should write to all disks in parallel.

* XL: ReadFile should read from all disks in parallel.
2016-05-14 01:57:04 -07:00
Bala FA
7264cd2ab3 Fix error message when wrong set of disks are passed (#1634)
Previously when wrong set of disks are given with last minio server
run, it throws unclear error message.  This is fixed by returning
appropriate errors.

Fixes #1591
2016-05-13 23:04:10 -07:00
Harshavardhana
0e4e9c4bc1 XL: ListDir should return each List from a random disk in the set. (#1613)
Fixes #1609
2016-05-13 18:12:26 -07:00
Krishna Srinivas
8099396ff0 xl/putObject: Should take care of the situation if an object already exists at the location. (#1606)
Fixes  #1598 #1594 #1595
2016-05-13 11:52:36 -07:00
Krishna Srinivas
d267696110 Validation: Reject object names with trailing "/". (#1619)
Fixes #1616
2016-05-13 11:43:06 -07:00
Bala FA
43539a0c86 posix: parseDirents() should follow symlink and get values. (#1631)
Previously parseDirents() ignores symbolic links.  This patch fixes
the issue by following the symlink using os.Stat().

Fixes #1545
2016-05-13 11:39:48 -07:00
Krishnan Parthasarathi
9e45d138cc Closed readCloser for each multipart in xl.GetObject (#1629)
* Closed readCloser for each multipart in xl.GetObject
2016-05-13 04:50:13 -07:00
karthic rao
ee8605e333 Make bucket failure fix with high concurrent load (#1630) 2016-05-13 04:03:38 -07:00
karthic rao
e4958f9757 Removing regexp check and adding string based check, regexp check was unnecessary here (#1627) 2016-05-13 03:33:53 -07:00
Harshavardhana
b62774d32f storage/xl: Return errVolumeAccessDenied if disks cannot be accessed. (#1621)
Fixes #1614
2016-05-12 21:01:11 -07:00
koolhead17
d6e0f3ab33 added awscli commands & minor typo fix. (#1578) 2016-05-12 16:35:11 -07:00
Bala FA
3ff0a56e62 XL: Ignore errDiskNotFound in certain situations (#1610)
When a disk is removed while an operation is going on
(eg. single/multipart put object, list/multipart list objects etc),
its required to ignore errDiskNotFound error and continue the
operation.

Fixes #1552
2016-05-11 23:42:14 -07:00
Harshavardhana
50431e91a6 erasure: Handle failed disks so that we initialize properly if they are missing. (#1607)
Fixes #1592
Fixes #1579
2016-05-11 18:58:32 -07:00
Harshavardhana
d4745c7d6a object: PutObjectHandler should set the md5Sum properly. (#1604)
Additionally add a test case as well for validating for us
to reply BadDigest properly.

Fixes #1603
2016-05-11 16:13:37 -07:00
Bala FA
adbcafefad xl/CreateFile: handle errFileNameTooLong error properly (#1523)
When errFileNameTooLong error is returned from posix, xl.CreateFile()
treats the error specially by returning the same error immediately.

Fixes #1501
2016-05-11 12:55:02 -07:00
Harshavardhana
86e5d71519 erasure: MakeVol, DeleteVol and StatVol should hold locks. (#1597)
Since there is a good amount of overlap, each code has to lock
properly for the operation they are going to perform.

- MakeVol create vols in a routine on all disks, hold locks.
- DeleteVol delete vols in a routine on all disks, hold locks.
- StatVol stat vols in a routine on all disks, hold locks.

Fixes #1588
2016-05-11 12:54:21 -07:00
Harshavardhana
72748d2073 erasure: healVolume err should be different from shadowed version. (#1590)
Multiple go-routines updating the same shadowed variable can
cause a data race, avoid it by using its own err variable.

Fixes #1589
2016-05-11 01:36:09 -07:00
Harshavardhana
49141eb3e4 http: Remove minhttp package and use standard Golang http. (#1587)
The functionality provided by minhttp will be implemented
cleanly through our own APIs. Since we are not going to
send SIGUSR2 and manage configuration in that manner, it
doesn't make sense to use minhttp.

Fixes #1586
2016-05-10 18:03:00 -07:00
Harshavardhana
d1fa1d9352 Remove binary files from previous commit. 2016-05-10 15:49:17 -07:00
karthic rao
26e2c4bf4d Replacing fastsha256 with crypto/sha256 package from golang standard package (#1584) 2016-05-10 14:20:11 -07:00