Commit Graph

3695 Commits

Author SHA1 Message Date
karthic rao
2e0742e309 bucket policy: Support for '?' wildcard. (#2353)
- Support for '?' wildcard for resource matching.

- Wildcard package is added with Match functions.

- Wildcard.Match supports '*' and wild.MatchExtended supports both '*'
  and '?' wildcards in the pattern string.

- Tests for the same for the wide range of cases.
2016-08-04 00:41:32 -07:00
Anis Elleuch
cc0d5b6fe0 webapi: ServerInfo returns the list variables/values in the environnment of the Minio server (#2351) 2016-08-03 13:47:03 -07:00
Harshavardhana
2db51e9d61 server/config: config should be updated only when we edit the credentials. (#2345) 2016-08-02 16:48:21 -07:00
Krishna Srinivas
69fd196471 Object-cache: enforce cache size to be less than RAM. (#2338) 2016-08-02 10:04:35 -07:00
karthic rao
1494af485e tests: signature-utils test (#2342) 2016-08-01 20:54:11 -07:00
Harshavardhana
c1d70f1f9e server/config: Create 'certs' directory in initServerConfig(). (#2331)
certs directory was created only if config was not present, our
expectancy is we need 'certs' directory to be present all the
time making it easier to be documented.
2016-07-30 14:55:20 -07:00
karthic rao
9baf599c91 tests: Unit tests and fixes for copyBuffer. (#2333)
- Unit tests for copyBuffer.
- Shadowing fix for copyBuffer.
2016-07-30 13:36:43 -07:00
Harshavardhana
8d090a20ce server: set globalCacheSize honoring system limits for max memory. (#2332)
On unix systems it is possible to set max memory used by
running processes using 'ulimit -m' or 'syscall.RLIMIT_AS'.

A process whence exceeds this limit, kernel would pro-actively
kill such a server with OOM. To avoid this problem of defaulting
our cache size to 8GB we should look for if the current system
limits are lower and set the cache size appropriately.
2016-07-30 08:50:49 -07:00
karthic rao
5b86dd7659 Tests: Cleanup/Enhancement: Add few more cases to posix.ReadFile tests and use a cleaner posixTestSetup for posix tests (#2330) 2016-07-30 01:26:19 -07:00
Jesse Lucas
4b05b6a6c1 Refactoring checkPortAvailability to check each tcp network (tcp, tcp4, tcp6) if a port is taken. (#2325) 2016-07-29 18:11:00 -07:00
Jesse Lucas
851d05161a Adding return error value to checkPortAvailability to enable testing of function. Adding checkport_test.go to test checkPortAvailability. Updated server-main.go to use error value from checkPortAvailability and calls fatalIf if an error is returned. (#2322) 2016-07-29 14:05:31 -07:00
Harshavardhana
cf9ba7b88f tests: Add missing unit test for posix.ReadFile. (#2319) 2016-07-28 21:57:11 -07:00
Krishnan Parthasarathi
50dae0ab04 bucket-policy: Migrate bucket policy to minioMetaBuket/buckets (#2321) 2016-07-28 20:49:08 -07:00
Anis Elleuch
14cefd352c Heal corrupted formats of disks already containing objects (#2297) 2016-07-28 16:49:58 -07:00
Frank
f239fcac67 Switched to faster minio/sha256-simd implementation (#2320) 2016-07-28 14:44:37 -07:00
Anis Elleuch
dcc3463e48 Limit POST form fields and file size + Generic Request Size limiter (#2317)
* Use less memory when receiving a file via multipart
* Add generic http request maximum size limiter to secure against malicious clients
2016-07-28 12:02:22 -07:00
Krishna Srinivas
7850d17f48 web-browser: disable minio browser when environmental variable MINIO_BROWSER=off (#2315)
fixes #2314
2016-07-28 04:00:33 -07:00
Harshavardhana
f503ac3db8 XL/Erasure: Make bit-rot verification based on xl.json algo. (#2299)
Currently `xl.json` saves algorithm information for bit-rot
verification. Since the bit-rot algo's can change in the
future make sure the erasureReadFile doesn't default to
a particular algo. Instead use the checkSumInfo.
2016-07-28 02:20:34 -07:00
Harshavardhana
65f71ce0c5 browser: Object upload should save metadata and notify. (#2309)
Object upload from browser should save additional
incoming metadata. Additionally should also notify
through bucket notifications once they are set.

Fixes #2292
2016-07-27 21:11:15 -07:00
Harshavardhana
ad19bf0ec1 server: Add update referral for update notification URL. (#2308) 2016-07-27 19:59:19 -07:00
Harshavardhana
f0067babe0 handlers: Add 'crossdomain.xml' handler. (#2305)
Fixes #2301
2016-07-27 19:53:55 -07:00
karthic rao
6b5fce826b placing defer file.Close() right after opening it (#2306) 2016-07-27 19:22:32 -07:00
Anis Elleuch
8b3cb3a0de POST Policy, multiple fixes: AccessDenied with unmet conditions, ${filename} in Key, missing filename in multipart (#2304)
* Unsatisfied conditions will return AccessDenied instead of MissingFields

* Require form-field `file` in POST policy and make `filename` an optional attribute

* S3 feature: Replace  in Key by filename attribute passed in multipart
2016-07-27 17:51:55 -07:00
Harshavardhana
2f7358a8a6 XL: erasure Index should have its corresponding distribution order. (#2300) 2016-07-27 11:57:08 -07:00
Harshavardhana
77248bd6e8 api: Notify events only if bucket notifications are set. (#2293)
While the existing code worked, it went to an entire cycle
of constructing event structure and end up not sending it.

Avoid this in the first place, but returning quickly if
notifications are not set on the bucket.
2016-07-26 19:10:02 -07:00
Harshavardhana
3054b74260 docs: Fix startup message for server as well. 2016-07-26 15:54:11 -07:00
koolhead17
7d42d09da8 Doc: Replaced README & FreeBSD docs with updated minio server splash (#2298)
screen.
2016-07-26 15:46:41 -07:00
Anis Elleuch
95ddf061ab Rate limit is working and supports limited waiting clients (#2295) 2016-07-26 14:17:11 -07:00
karthic rao
5fe72cf205 Removing readAllMeta from xl-v1-healing.go and placing it in xl-v1-utils.go (#2296) 2016-07-26 11:34:48 -07:00
Harshavardhana
1e3d80552f XL: format.json healing should cater for mismatching order. (#2285)
Fresh disks can be provided in any order, we need to make sure
to preserve existing disk order and populate the fresh disks
in new positions.

Thanks for Anis Elleuch <vadmeste@gmail.com> for finding this issue.
2016-07-26 03:18:47 -07:00
Harshavardhana
f253dfc922 docs: Fix erasure code image embedding issue. 2016-07-26 00:09:57 -07:00
Harshavardhana
1f9e38e3cd api: Add bucket notification util tests. (#2289) 2016-07-26 00:01:35 -07:00
karthic rao
530ed67b59 Adding leak test framework (#2254) 2016-07-25 20:39:14 -07:00
Harshavardhana
a2b6f0524d XL/erasure: Remove deprecated copyN function. (#2288) 2016-07-25 20:36:56 -07:00
karthic rao
091d80666a Enhancement for Erasure encode test. (#2287) 2016-07-25 20:36:41 -07:00
Harshavardhana
efbf7dbc0f api: Bucket notification add filter rules check and validate. (#2272)
These filtering techniques are used to validate
object names for their prefix and suffix.
2016-07-25 17:53:55 -07:00
Krishna Srinivas
043ddbd834 optimize memory allocation during erasure-read by using temporary buffer pool. (#2259)
* XL/erasure-read: optimize memory allocation during erasure-read by using temporary buffer pool.

With the change the buffer needed during GetObject by erasureReadFile is allocated only once.
2016-07-25 14:17:01 -07:00
Dee Koder
04f90bd463 doc: Broken links fixed in the Explore further section. (#2281) 2016-07-24 22:53:35 -07:00
Harshavardhana
9212e11b90 XL/GetObject: When disk is not available, checksum should be empty. (#2276) 2016-07-24 22:49:27 -07:00
Harshavardhana
79bab6b561 XL: Operations on uploads.json should cater for disk being unavailable. (#2277) 2016-07-24 18:08:15 -07:00
Krishnan Parthasarathi
7e5a78985d tests: Using listObjects clean up remaining tree walk go routines. (#2278)
* fs: Set nextMarker independent of it having a slash or not.
* tests: Using listObjects clean up remaining tree walk go routines.
* tests: Use slices to hold data instead of enumerating test cases by hand

... also fixed numbering of test cases.
2016-07-24 15:52:12 -07:00
Anis Elleuch
b0b919a1d6 Server http and https on the same port using a customized server (#2247) 2016-07-24 12:30:57 -07:00
Harshavardhana
6c2fb19ed7 docs: Removed and purged uneeded docs. (#2273) 2016-07-24 03:32:45 -07:00
Harshavardhana
f248089523 api: Implement bucket notification. (#2271)
* Implement basic S3 notifications through queues

Supports multiple queues and three basic queue types:

1. NilQueue -- messages don't get sent anywhere
2. LogQueue -- messages get logged
3. AmqpQueue -- messages are sent to an AMQP queue

* api: Implement bucket notification.

Supports two different queue types

- AMQP
- ElasticSearch.

* Add support for redis
2016-07-23 22:51:12 -07:00
Harshavardhana
f85d94288d api: extract http headers with some supported header list. (#2268) 2016-07-22 20:31:45 -07:00
Harshavardhana
55cb55675c api/multipart: Send S3 compatible error message, missing second sentence. (#2270) 2016-07-22 17:05:40 -07:00
koolhead17
7e076577de Update Minio-erasure-code-quickStart-guide.md (#2269)
Minor update to link the URL.
2016-07-22 13:35:27 -07:00
Harshavardhana
5d118141cd XL: Remove deadcode unionChecksumInfo. (#2261) 2016-07-21 19:07:00 -07:00
karthic rao
646ff2c64d Get Object disk not found test (#2264)
Test: GetObject disk not found test
2016-07-21 19:06:50 -07:00
Harshavardhana
0add96f655 fs: Save metadata for objects in minioMetaBucket directory. (#2251) 2016-07-21 17:31:14 -07:00