Commit Graph

38 Commits

Author SHA1 Message Date
kannappanr 76ddf4d32f Log x-amz-request-id as log and XML error response (#6173)
Currently, requestid field in logEntry is not populated, as the
requestid field gets set at the very end.
It is now set before regular handler functions. This is also
useful in setting it as part of the XML error response.

Travis build for ppc64le has been quite inconsistent and stays queued
for most of the time. Removing this build as part of Travis.yml for
the time being.
2018-07-20 18:46:32 -07:00
Krishna Srinivas e40a5e05e1 Do notification in background to not block S3 client REST calls (#6005) 2018-07-03 11:09:36 -07:00
Harshavardhana 4886bfbc72 fix: Avoid more crashes due to concurrent map usage (#5912)
This PR fixes another situation where a crash occurs
thanks to @krishnasrinivas for reproducing this

Fixes #5897
2018-05-09 15:11:51 -07:00
Harshavardhana 98f81ced86 fix: Avoid concurrent map writes in go-routines (#5898)
Fixes #5897
2018-05-09 11:25:38 -07:00
Harshavardhana b6ca39ea48 Support migrating inconsistent bucket policies (#5855)
Previously we used allow bucket policies without
`Version` field to be set to any given value, but
this behavior is inconsistent with AWS S3.

PR #5790 addressed this by making bucket policies
stricter and cleaner, but this causes a breaking
change causing any existing policies perhaps without
`Version` field or the field to be empty to fail upon
server startup.

This PR brings a code to migrate under these scenarios
as a one time operation.
2018-04-27 15:02:54 -07:00
Bala FA 0d52126023 Enhance policy handling to support SSE and WORM (#5790)
- remove old bucket policy handling
- add new policy handling
- add new policy handling unit tests

This patch brings support to bucket policy to have more control not
limiting to anonymous.  Bucket owner controls to allow/deny any rest
API.

For example server side encryption can be controlled by allowing
PUT/GET objects with encryptions including bucket owner.
2018-04-24 15:53:30 -07:00
Bala FA 76cc65531c use package name correctly (#5827) 2018-04-21 19:23:54 -07:00
kannappanr cef992a395
Remove error package and cause functions (#5784) 2018-04-10 09:36:37 -07:00
kannappanr f8a3fd0c2a
Create logger package and rename errorIf to LogIf (#5678)
Removing message from error logging
Replace errors.Trace with LogIf
2018-04-05 15:04:40 -07:00
Krishna Srinivas e452377b24 Add context to the object-interface methods.
Make necessary changes to xl fs azure sia
2018-03-15 16:28:25 -07:00
Bala FA 0e4431725c make notification as separate package (#5294)
* Remove old notification files

* Add net package

* Add event package

* Modify minio to take new notification system
2018-03-15 13:03:41 -07:00
poornas 4f73fd9487 Unify gateway and object layer. (#5487)
* Unify gateway and object layer. Bring bucket policies into
object layer.
2018-02-09 15:19:30 -08:00
kannappanr 20584dc08f
Remove unnecessary errors printed on the console (#5386)
Some of the errors printed on server console can be
removed as those error message is unnecessary.

Fixes #5385
2018-01-11 11:42:05 -08:00
Krishna Srinivas 14e6c5ec08 Simplify the steps to make changes to config.json (#5186)
This change introduces following simplified steps to follow 
during config migration.

```
 // Steps to move from version N to version N+1
 // 1. Add new struct serverConfigVN+1 in config-versions.go
 // 2. Set configCurrentVersion to "N+1"
 // 3. Set serverConfigCurrent to serverConfigVN+1
 // 4. Add new migration function (ex. func migrateVNToVN+1()) in config-migrate.go
 // 5. Call migrateVNToVN+1() from migrateConfig() in config-migrate.go
 // 6. Make changes in config-current_test.go for any test change
```
2017-11-29 13:12:47 -08:00
Harshavardhana 203ac8edaa Bucket policies should use minio-go/pkg/policy instead. (#5090) 2017-10-27 16:14:06 -07:00
Aaron Walker 5db533c024 bucket-policy: Add IPAddress/NotIPAddress conditions support (#4736) 2017-08-05 01:00:05 -07:00
Harshavardhana 533338bdeb all/windows: Be case in-sensitive about pattern matching. (#3682)
Resource strings and paths are case insensitive on windows
deployments but if user happens to use upper case instead of
lower case for certain configuration params like bucket
policies and bucket notification config. We might not honor
them which leads to a wrong behavior on windows.

This is windows only behavior, for all other platforms case
is still kept sensitive.
2017-02-03 23:27:50 -08:00
Krishnan Parthasarathi 2665aba555 Fail PutBucketPolicy if conditions are incompatible with actions. (#3659) 2017-01-30 09:20:16 -08:00
Krishnan Parthasarathi 9b6bcb30d9 Fix handling of StringNotEquals condition operator (#3660) 2017-01-30 09:18:10 -08:00
Krishnan Parthasarathi b408d0e87d Add aws:Referer condition key support. (#3641)
This change implements bucket policy enhancements required to restrict access based on HTTP referer.
See https://docs.aws.amazon.com/AmazonS3/latest/dev/example-bucket-policies.html#example-bucket-policies-use-case-4 for more information.

Fixes #3540
2017-01-29 19:45:11 -08:00
Harshavardhana 51fa4f7fe3 Make PutObject a nop for an object which ends with "/" and size is '0' (#3603)
This helps majority of S3 compatible applications while not returning
an error upon directory create request.

Fixes #2965
2017-01-20 16:33:01 -08:00
Harshavardhana 926c75d0b5 api: Set appropriate content-type for success/error responses. (#3537)
Golang HTTP client automatically detects content-type but
for S3 clients this content-type might be incorrect or
might misbehave.

For example:

```
Content-Type: text/xml; charset=utf-8
```

Should be
```
Content-Type: application/xml
```

Allow this to be set properly.
2017-01-06 00:37:00 -08:00
Harshavardhana 4daa0d2cee lock: Moving locking to handler layer. (#3381)
This is implemented so that the issues like in the
following flow don't affect the behavior of operation.

```
GetObjectInfo()
.... --> Time window for mutation (no lock held)
.... --> Time window for mutation (no lock held)
GetObject()
```

This happens when two simultaneous uploads are made
to the same object the object has returned wrong
info to the client.

Another classic example is "CopyObject" API itself
which reads from a source object and copies to
destination object.

Fixes #3370
Fixes #2912
2016-12-10 16:15:12 -08:00
Bala FA 825000bc34 Use humanize constants for KiB, MiB and GiB units. (#3322) 2016-11-22 18:18:22 -08:00
Bala FA bef0a50bc1 Cleanup and fixes (#3273)
* newRequestID() (previously generateUploadID()) returns string than byte array.
* Remove unclear comments and added appropriate comments.
* SHA-256, MD5 Hash functions return Hex/Base64 encoded string than byte array.
* Remove duplicate MD5 hasher functions.
* Rename listObjectsValidateArgs() into validateListObjectsArgs()
* Remove repeated auth check code in all bucket request handlers.
* Remove abbreviated names in bucket-metadata
* Avoid nested if in bucketPolicyMatchStatement()
* Use ioutil.ReadFile() instead of os.Open() and ioutil.ReadAll()
* Set crossDomainXML as constant.
2016-11-21 13:51:05 -08:00
Harshavardhana 398421b9f5 xl/bootup: Server bootup shouldn't return for missing buckets. (#3255)
Ref #3196
2016-11-14 15:45:00 -08:00
Aditya Manthramurthy dd0698d14c Improve namespace lock API: (#3203)
- abstract out instrumentation information.
- use separate lockInstance type that encapsulates the nsMutex, volume,
  path and opsID as the frontend or top-level lock object.
2016-11-09 10:58:41 -08:00
Aditya Manthramurthy 3977d6b7bd Lock bucket while modifying its metadata (Fixes #2979) (#3019)
- When modifying notification configuration
- When modifying listener configuration
- When modifying policy configuration

With this change we also stop early checking if the bucket exists, since
that uses a Read-lock and causes a deadlock due to the outer Write-lock.
2016-10-24 19:52:24 -07:00
Aditya Manthramurthy 0aabc1d8d9 Use Peer RPC to propagate bucket policy changes (#2891) 2016-10-13 09:19:04 -07:00
Harshavardhana 5885ffc8ae signature: Add legacy signature v2 support transparently. (#2811)
Add new tests as well.
2016-09-30 14:32:13 -07:00
Harshavardhana 64083b9227 signature: Region changes should be handled just like AWS. (#2805)
- PutBucket happens with 'us-east-1'.
- ListBuckets happens with any region.
- GetBucketLocation happens with 'us-east-1' and location is returned.
2016-09-29 15:51:00 -07:00
Harshavardhana 6aa2fc95c0 Revert "bucket: refactor policies and fix bugs related to enforcing policies. (#2766)"
This reverts commit ca5ca8332b.
2016-09-26 19:32:33 -07:00
Harshavardhana ca5ca8332b bucket: refactor policies and fix bugs related to enforcing policies. (#2766)
This patch also addresses the problem of double caching at
object layer once at XL and another at handler layer.
2016-09-22 23:47:48 -07:00
Krishna Srinivas 9358ee011b logging: Print stack trace in case of errors.
fixes #1827
2016-09-13 21:18:30 -07:00
Harshavardhana 4917038f55 Move the ObjectAPI() resource to be beginning of each handlers.
We should return back proper errors so that the clients can
retry until server has been initialized.
2016-09-13 21:18:30 -07:00
Harshavardhana 7270ca4157 pkg/wildcard: Simplify the wildcard logic further. (#2555) 2016-08-27 00:27:17 -07:00
Harshavardhana 975eb31973 api/bucketPolicy: Use minio-go/pkg/set and fix bucket policy regression. (#2506)
Current master has a regression 'mc policy <policy-type> alias/bucket/prefix'
does not work anymore, due to the way new minio-go changes do json marshalling.
This led to a regression on server side when a ``prefix`` is provided
policy is rejected as malformed from th server which is not the case with
AWS S3.

This patch uses the new ``minio-go/pkg/set`` package to address the
unmarshalling problems.

Fixes #2503
2016-08-20 03:16:38 -07:00
Harshavardhana bccf549463 server: Move all the top level files into cmd folder. (#2490)
This change brings a change which was done for the 'mc'
package to allow for clean repo and have a cleaner
github drop in experience.
2016-08-18 16:23:42 -07:00