Commit Graph

4886 Commits

Author SHA1 Message Date
Andreas Auernhammer
f60765ac93 add some security HTTP headers (#5814)
This change adds some security headers like Content-Security-Policy.
It does not set the HSTS header because Content-Security-Policy prevents
mixed HTTP and HTTPS content and the server does not use cookies.
However it is a header which could be added later on.

It also moves some header added by #5805 from a vendored file
to a generic handler.

Fixes ##5813
2018-04-12 15:57:41 -07:00
Harshavardhana
1f07545e2a
Improve init messages for distributed setup (#5786)
Fixes #5531
2018-04-12 15:43:38 -07:00
rwagner_inf
e39de65367 Add security HTTP Headers (#5805)
Some HTTP security headers in Minio.
To avoid problems with XSS and Clickjacking attacks.

X-Frame-Options
X-Frame-Options response header improve the protection
of web applications against Clickjacking. It declares a
policy communicated from a host to the client browser
on whether the browser must not display the transmitted
content in frames of other web pages.

X-XSS-Protection
This header enables the Cross-site scripting (XSS) filter in your browser.
2018-04-12 14:30:06 -07:00
Harshavardhana
bd8d6e3c4e Return bucketNameInvalid properly in B2 gateway (#5787)
Fixes #5758
2018-04-12 11:20:21 +05:30
Harshavardhana
4a874dfbc1
Ignore prefix renames when dest directory is not empty (#5798)
Also make sure to not modify the underlying errors from
layers, we should return the error as is and one object
layer should translate the errors.

Fixes #5797
2018-04-11 17:15:42 -07:00
Bala FA
650c6ee8fb event/target/webhook: treat all 2xx as success. (#5792)
Fixes #5769
2018-04-10 17:45:54 -07:00
kannappanr
57a3d9c16c
Modify fatalIf, startup and update message logging code (#5780)
Use a common logging framework to log fatalIf, startup, Info and Update
messages.
2018-04-10 09:37:14 -07:00
kannappanr
cef992a395
Remove error package and cause functions (#5784) 2018-04-10 09:36:37 -07:00
Harshavardhana
217fb470a7 Add a check to check if disk is writable (#5662)
This check is a pre-emptive check to return
error early before we attempt to use the disk
for any other operations later.

refer #5645
2018-04-10 09:26:09 +05:30
Harshavardhana
eb0deabd73 Fail when TLS is configured and HTTP endpoints are provided (#5771) 2018-04-10 07:40:23 +05:30
dingjs
a706c21f70 Tranlsate disk-caching/README.md,nas.md,large-bucket/README.md to Chinese. (#5775) 2018-04-09 17:41:16 -07:00
Harshavardhana
41c31d10ca
Enable s3,gcs,manta,oss,b2 gateways as production ready (#5795) 2018-04-09 16:33:48 -07:00
Harshavardhana
1d31ad499f Make sure to re-load reference format after HealFormat (#5772)
This PR introduces ReloadFormat API call at objectlayer
to facilitate this. Previously we repurposed HealFormat
but we never ended up updating our reference format on
peers.

Fixes #5700
2018-04-09 22:55:41 +05:30
Krishna Srinivas
ae8e863ff4 disk.String() represents just path and not URL when disk is a local disk (#5785) 2018-04-06 16:59:31 -07:00
Andreas Auernhammer
da9f0e324e return AWS S3 compatible error for invalid but equal keys during key rotation (#5783)
This change let the server return the S3 error for a key rotation
if the source key is not valid but equal to the destination key.

This change also fixes the SSE-C error messages since AWS returns error messages
ending with a '.'.

Fixes #5625
2018-04-06 14:15:23 -07:00
Harshavardhana
73f7a98590 Enable go1.10.1 version (#5666) 2018-04-06 11:07:39 -07:00
Andreas Auernhammer
2f20d90465 set storage class in FS mode (#5779)
This change sets the storage class of the object-info if a storage
class was specified during PUT. The server now replies with the
storage class which was set during uploading the object in FS mode.

Fixes #5777
2018-04-06 20:50:02 +05:30
Dee Koder
c30f75de28 Update screenshots. Use png instead of jpgs. (#5770) 2018-04-06 13:51:25 +05:30
Harshavardhana
861550446a
Allow upto 15 minutes of timeskew for RPC calls as well (#5774)
Default installations of cloned VMs on VMware like env
might experience serious problems with time skewing,
allow for a higher value instead of 3 seconds we are
moving to 15 minutes just like API level skew.

Access to internet and configuring ntp might not be possible,
in such situations providing atleast a 15 minute skew could
cater for majority of situations.
2018-04-05 23:55:31 -07:00
kannappanr
f8a3fd0c2a
Create logger package and rename errorIf to LogIf (#5678)
Removing message from error logging
Replace errors.Trace with LogIf
2018-04-05 15:04:40 -07:00
kannappanr
91fd8ffeb7
Move http package inside cmd (#5776) 2018-04-05 14:19:27 -07:00
Nitish Tiwari
9eb94fe8c8 Fix StorageClass field in ListObject/ListObjectV2 response (#5766)
Fixes: #5754
2018-04-05 10:56:28 -07:00
Harshavardhana
30ab71c9d9 Add missing healthcheck router for gateway (#5764) 2018-04-05 07:37:54 +05:30
Kaan Kabalak
38ce19badd Rename minio-limitations.md to minio-limits.md in Chinese translation (#5753)
Applies the change in #5541 for Chinese docs
2018-04-04 19:00:19 -07:00
Minio Trusted
c4a70fdee9 Update yaml files to latest version RELEASE.2018-04-04T05-20-54Z 2018-04-04 05:25:19 +00:00
Harshavardhana
85a57d2021 Make sure to close the disk connections (#5752)
Since we do not re-use storageDisks after moving
the connections to object layer we should close them
appropriately otherwise we have a lot of connection
leaks and these can compound as the time goes by.

This PR also refactors the initialization code to
re-use storageDisks for given set of endpoints until
we have confirmed a valid reference format.
2018-04-04 10:28:48 +05:30
Nitish Tiwari
d67e423a32 Add firewall-cmd entry for firewall configuration (#5755) 2018-04-03 17:15:24 -07:00
Edward Vielmetti
e76fe435c4 "account_id" for "accound_id" (#5757) 2018-04-03 17:06:59 -07:00
Vladimir Cores
1a71004c91 Multi files selection and upload from Browser (#5711) 2018-04-03 16:44:37 -07:00
Edward Vielmetti
1e50951a4a typo fix (#5756) 2018-04-03 21:00:35 +05:30
Harshavardhana
2c5f2e9669 Stop deleting 'format.json' upon unsuccessful save (#5747)
An issue was reproduced when there a no more inodes
available on an existing setup of 4 disks, now we
took one of the disks and reformatted it to relinquish
inodes. Now we attempt to bring the fresh disk back
into setup and perform a heal - at this point creating
new `format.json` fails on existing disks since they
do not have more inodes available.

At this point due to quorum failure, we end up deleting
existing `format.json` as well, this PR removes the code
which deletes existing `format.json` as there is no need
to delete them.
2018-04-03 10:48:06 +05:30
Harshavardhana
b052bfa79e Add necessary safe headers under CORS (#5750)
Fixes #5748
2018-04-02 17:51:32 -07:00
Harshavardhana
8eb838bf91 Extend quorum based listing for prefixes (#5749)
Previous PR 2afd196c83 fixed
the issue of quorum based listing for regular objects, this
PR continues on this idea by extending this support to
object directory prefixes as well.

Fixes #5733
2018-04-02 17:26:34 -07:00
poornas
98315b8e60 Update disk cache documentation (#5743) 2018-04-02 10:25:08 +05:30
kannappanr
b87cc3d643 Get proper GOPATH in trim function (#5744)
Set GOPATH string to empty in build-constants.go
Check for both compile time GOPATH and default GOPATH
while trimming the file path in the stack trace.
Fixes #5741
2018-03-30 19:13:25 -07:00
Harshavardhana
4d02f9bccf fix gateway docs comprehensively (#5740) 2018-03-30 14:16:06 +05:30
Minio Trusted
cf8d12d169 Update yaml files to latest version RELEASE.2018-03-30T00-38-44Z 2018-03-30 00:41:38 +00:00
Krishna Srinivas
804a4f9c15 Fix backend format for disk-cache - not to use FS format.json (#5732) 2018-03-29 14:38:26 -07:00
Nitish Tiwari
328076f773
Fix help message to be uniform across env vars (#5735) 2018-03-30 01:59:21 +05:30
Nitish Tiwari
22e3ace36e Apply storageClass data while config migration (#5737)
Fixes #5736
2018-03-29 13:21:58 -07:00
Harshavardhana
ef61b36c5a Fix PUT bucket notification deadlocks (#5734)
This PR fixes two different variant of deadlocks in
notification.

- holding write lock on the bucket competing with read lock
- holding competing locks on read/save notification config
2018-03-29 12:00:20 -07:00
Harshavardhana
35b3913d22 Fix block id length upto 8bytes only for azure gateway. (#5731)
This PR also reverts commit 2f9354b17e
to bring back 8 byte block id requirement for azure gateway.
2018-03-29 09:54:47 -07:00
Harshavardhana
228c8f05f4 Improve some wording in disk caching docs (#5730) 2018-03-29 10:12:14 +05:30
poornas
af024a9c69 Remove deadcode related to multipart cleanup for fs (#5716)
The cleanup code is no longer needed as we moved to lockfree 
multipart backend for fs
2018-03-29 08:26:52 +05:30
Minio Trusted
5a7ab7b06e Update yaml files to latest version RELEASE.2018-03-28T23-45-53Z 2018-03-28 23:48:23 +00:00
poornas
5006e28367 Add disk-cache and worm to user-agent (#5729) 2018-03-28 16:26:27 -07:00
poornas
a3e806ed61 Add disk based edge caching support. (#5182)
This PR adds disk based edge caching support for minio server.

Cache settings can be configured in config.json to take list of disk drives,
cache expiry in days and file patterns to exclude from cache or via environment
variables MINIO_CACHE_DRIVES, MINIO_CACHE_EXCLUDE and MINIO_CACHE_EXPIRY

Design assumes that Atime support is enabled and the list of cache drives is
fixed.
 - Objects are cached on both GET and PUT/POST operations.
 - Expiry is used as hint to evict older entries from cache, or if 80% of cache
   capacity is filled.
 - When object storage backend is down, GET, LIST and HEAD operations fetch
   object seamlessly from cache.

Current Limitations
 - Bucket policies are not cached, so anonymous operations are not supported in
   offline mode.
 - Objects are distributed using deterministic hashing among list of cache
   drives specified.If one or more drives go offline, or cache drive
   configuration is altered - performance could degrade to linear lookup.

Fixes #4026
2018-03-28 14:14:06 -07:00
Harshavardhana
2f9354b17e Use uuid.New() for uploadID in azure gateway (#5721) 2018-03-28 11:17:25 -07:00
Harshavardhana
6e9c853312 After healing re-load disks with the new format (#5718)
This PR also fixes correct calculation of drive states
before and after healing of objects.

Fixes #5700
Fixes #5708
2018-03-28 06:41:39 +05:30
poornas
76d1e8bbcd change fs.json format to include checksum fields (#5685) 2018-03-27 17:23:10 -07:00