Commit Graph

17 Commits

Author SHA1 Message Date
Aditya Manthramurthy
3977d6b7bd Lock bucket while modifying its metadata (Fixes #2979) (#3019)
- When modifying notification configuration
- When modifying listener configuration
- When modifying policy configuration

With this change we also stop early checking if the bucket exists, since
that uses a Read-lock and causes a deadlock due to the outer Write-lock.
2016-10-24 19:52:24 -07:00
Harshavardhana
4b5b363c6c tests: Fix race between SetBucketListenerConfig and eventNotifyForBucketNotifications (#3041) 2016-10-22 02:35:33 -07:00
Harshavardhana
83b364891d tests: Fix a potential race in ListenBucketNotificationHandler. (#3040) 2016-10-21 22:56:27 -07:00
Harshavardhana
ece559afe2 api: Do not use sqs for ListenBucketNotification. (#3023)
Add more tests. Fixes #3024
2016-10-21 01:25:17 -07:00
Aditya Manthramurthy
8876e0a80a Delete bucket listener config file from disk (#3016) 2016-10-20 16:09:19 -07:00
Aditya Manthramurthy
d02cb963d5 Fix listen-bucket (Fixes #2942) (#2949)
Don't close socket while re-initializing notify-listeners, as the rpc
client object is shared between notify-listeners and peer clients.

Also, improves SendRPC() readability by using GetPeerClient().
2016-10-16 20:52:10 -07:00
Aditya Manthramurthy
6199aa0707 Peer RPCs for bucket notifications (#2877)
* Implements a Peer RPC router that sends info to all Minio servers in the cluster.
* Bucket notifications are propagated to all nodes via this RPC router.
* Bucket listener configuration is persisted to separate object layer
  file (`listener.json`) and peer RPCs are used to communicate changes
  throughout the cluster.
* When events are generated, RPC calls to send them to other servers
  where bucket listeners may be connected is implemented.
* Some bucket notification tests are now disabled as they cannot work in
  the new design.
* Minor fix in `funcFromPC` to use `path.Join`
2016-10-12 01:03:50 -07:00
Aditya Manthramurthy
315e66858c Add PostgreSQL notifier (#2739) (#2824)
* The user is required to specify a table name and database connection
  information in the configuration file.

* INSERTs and DELETEs are done via prepared statements for speed.

* Assumes a table structure, and requires PostgreSQL 9.5 or above due to
  the use of UPSERT.

* Creates the table if it does not exist with the given table name using
  a query like:

    CREATE TABLE myminio (
        key varchar PRIMARY KEY,
        value JSONB
    );

* Vendors some required libraries.
2016-10-03 17:29:55 -07:00
Anis Elleuch
1e6afac3bd Add NATS notifier (#2795) 2016-09-29 23:42:10 -07:00
Krishnan Parthasarathi
5fdd768903 Make addition of TopicConfig to globalEventNotifier go-routine safe (#2806) 2016-09-28 22:46:19 -07:00
Harshavardhana
018c90dae7 events: ElasticSearch doesnt support objects with '/' in them. (#2747)
Fix this by using a unique sha256 generated for each unique key.
2016-09-20 16:36:18 -07:00
Harshavardhana
0a3448c8b6 events: Change event notifiers to delete and update keys. (#2742)
ElasticSearch and Redis are both treated like a database.
Each indexs are based on the object names uniquely indentifying
the event. Upon each delete event of the named object deletes
the index on elasticsearch and redis respectively.
2016-09-20 02:11:17 -07:00
Harshavardhana
ba2ba328da server: Fixes for various conditions
- Fix distributed branch to be able to run FS version.
- Fix distributed branch to be able to run XL local disks.
- Ignore initialization failures of notification and bucket
  policies, the codepath should load whatever is possible.
2016-09-13 21:18:30 -07:00
Krishna Srinivas
9358ee011b logging: Print stack trace in case of errors.
fixes #1827
2016-09-13 21:18:30 -07:00
Harshavardhana
fa6e9540a8 server: We shouldn't exit the server in lazy init. (#2548)
Avoid fatalIf instead these are non-critical errors,
continue running the server.

 - initializing bucket notifications
 - initializing bucket policies.
 - migrating bucket policies failure.

Fixes #2547
2016-09-13 21:18:30 -07:00
Harshavardhana
040af08473 server: Startup message now prints configured ARNs. (#2653)
Fixes #2628
2016-09-10 02:23:28 -07:00
Harshavardhana
bccf549463 server: Move all the top level files into cmd folder. (#2490)
This change brings a change which was done for the 'mc'
package to allow for clean repo and have a cleaner
github drop in experience.
2016-08-18 16:23:42 -07:00