MyFavMirrors/minio
1
0
Fork 0
mirror of https://github.com/minio/minio.git synced 2025-01-26 14:13:16 -05:00
Code Issues Packages Projects Releases Wiki Activity

notification queue limit has no maxLimit (#9380)

Browse Source 
New value defaults to 100K events by default,
but users can tune this value upto any value
they seem necessary.

* increase the limit to maxint64 while validating
This commit is contained in:
Praveen raj Mani 2020-04-18 13:50:56 +05:30 committed by GitHub
parent 75107d7698
commit c79358c67e
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
13 changed files with 34 additions and 64 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
cmd/config/notify/help.go
View File

@ -24,7 +24,7 @@ import (
const (
formatComment = `'namespace' reflects current bucket/object list and 'access' reflects a journal of object operations, defaults to 'namespace'`
queueDirComment = `staging dir for undelivered messages e.g. '/home/events'`
queueLimitComment = `maximum limit for undelivered messages, defaults to '10000'`
queueLimitComment = `maximum limit for undelivered messages, defaults to '100000'`
)
// Help template inputs for all notification targets

60
docs/bucket/notifications/README.md
View File

@ -69,7 +69,7 @@ internal (on|off) set to 'on' for exchange to be not used directly by p
auto_deleted (on|off) auto delete queue when set to 'on', when there are no consumers
delivery_mode (number) set to '1' for non-persistent or '2' for persistent queue
queue_dir (path) staging dir for undelivered messages e.g. '/home/events'
queue_limit (number) maximum limit for undelivered messages, defaults to '10000'
queue_limit (number) maximum limit for undelivered messages, defaults to '100000'
comment (sentence) optionally add a comment to this setting
```
@ -92,11 +92,11 @@ MINIO_NOTIFY_AMQP_INTERNAL (on|off) set to 'on' for exchange to be not
MINIO_NOTIFY_AMQP_AUTO_DELETED (on|off) auto delete queue when set to 'on', when there are no consumers
MINIO_NOTIFY_AMQP_DELIVERY_MODE (number) set to '1' for non-persistent or '2' for persistent queue
MINIO_NOTIFY_AMQP_QUEUE_DIR (path) staging dir for undelivered messages e.g. '/home/events'
MINIO_NOTIFY_AMQP_QUEUE_LIMIT (number) maximum limit for undelivered messages, defaults to '10000'
MINIO_NOTIFY_AMQP_QUEUE_LIMIT (number) maximum limit for undelivered messages, defaults to '100000'
MINIO_NOTIFY_AMQP_COMMENT (sentence) optionally add a comment to this setting
```
MinIO supports persistent event store. The persistent store will backup events when the AMQP broker goes offline and replays it when the broker comes back online. The event store can be configured by setting the directory path in `queue_dir` field and the maximum limit of events in the queue_dir in `queue_limit` field. For eg, the `queue_dir` can be `/home/events` and `queue_limit` can be `1000`. By default, the `queue_limit` is set to 10000.
MinIO supports persistent event store. The persistent store will backup events when the AMQP broker goes offline and replays it when the broker comes back online. The event store can be configured by setting the directory path in `queue_dir` field and the maximum limit of events in the queue_dir in `queue_limit` field. For eg, the `queue_dir` can be `/home/events` and `queue_limit` can be `1000`. By default, the `queue_limit` is set to 100000.
To update the configuration, use `mc admin config get notify_amqp` command to get the current configuration for `notify_amqp`.
@ -203,7 +203,7 @@ qos (number) set the quality of service priority, defaults t
keep_alive_interval (duration) keep-alive interval for MQTT connections in s,m,h,d
reconnect_interval (duration) reconnect interval for MQTT connections in s,m,h,d
queue_dir (path) staging dir for undelivered messages e.g. '/home/events'
queue_limit (number) maximum limit for undelivered messages, defaults to '10000'
queue_limit (number) maximum limit for undelivered messages, defaults to '100000'
comment (sentence) optionally add a comment to this setting
```
@ -223,11 +223,11 @@ MINIO_NOTIFY_MQTT_QOS (number) set the quality of service pr
MINIO_NOTIFY_MQTT_KEEP_ALIVE_INTERVAL (duration) keep-alive interval for MQTT connections in s,m,h,d
MINIO_NOTIFY_MQTT_RECONNECT_INTERVAL (duration) reconnect interval for MQTT connections in s,m,h,d
MINIO_NOTIFY_MQTT_QUEUE_DIR (path) staging dir for undelivered messages e.g. '/home/events'
MINIO_NOTIFY_MQTT_QUEUE_LIMIT (number) maximum limit for undelivered messages, defaults to '10000'
MINIO_NOTIFY_MQTT_QUEUE_LIMIT (number) maximum limit for undelivered messages, defaults to '100000'
MINIO_NOTIFY_MQTT_COMMENT (sentence) optionally add a comment to this setting
```
MinIO supports persistent event store. The persistent store will backup events when the MQTT broker goes offline and replays it when the broker comes back online. The event store can be configured by setting the directory path in `queue_dir` field and the maximum limit of events in the queue_dir in `queue_limit` field. For eg, the `queue_dir` can be `/home/events` and `queue_limit` can be `1000`. By default, the `queue_limit` is set to 10000.
MinIO supports persistent event store. The persistent store will backup events when the MQTT broker goes offline and replays it when the broker comes back online. The event store can be configured by setting the directory path in `queue_dir` field and the maximum limit of events in the queue_dir in `queue_limit` field. For eg, the `queue_dir` can be `/home/events` and `queue_limit` can be `1000`. By default, the `queue_limit` is set to 100000.
To update the configuration, use `mc admin config get` command to get the current configuration.
@ -336,7 +336,7 @@ url* (url) Elasticsearch server's address, with optional
index* (string) Elasticsearch index to store/update events, index is auto-created
format* (namespace*|access) 'namespace' reflects current bucket/object list and 'access' reflects a journal of object operations, defaults to 'namespace'
queue_dir (path) staging dir for undelivered messages e.g. '/home/events'
queue_limit (number) maximum limit for undelivered messages, defaults to '10000'
queue_limit (number) maximum limit for undelivered messages, defaults to '100000'
comment (sentence) optionally add a comment to this setting
```
@ -352,13 +352,13 @@ MINIO_NOTIFY_ELASTICSEARCH_URL* (url) Elasticsearch serve
MINIO_NOTIFY_ELASTICSEARCH_INDEX* (string) Elasticsearch index to store/update events, index is auto-created
MINIO_NOTIFY_ELASTICSEARCH_FORMAT* (namespace*|access) 'namespace' reflects current bucket/object list and 'access' reflects a journal of object operations, defaults to 'namespace'
MINIO_NOTIFY_ELASTICSEARCH_QUEUE_DIR (path) staging dir for undelivered messages e.g. '/home/events'
MINIO_NOTIFY_ELASTICSEARCH_QUEUE_LIMIT (number) maximum limit for undelivered messages, defaults to '10000'
MINIO_NOTIFY_ELASTICSEARCH_QUEUE_LIMIT (number) maximum limit for undelivered messages, defaults to '100000'
MINIO_NOTIFY_ELASTICSEARCH_COMMENT (sentence) optionally add a comment to this setting
```
For example: `http://localhost:9200` or with authentication info `http://elastic:MagicWord@127.0.0.1:9200`.
MinIO supports persistent event store. The persistent store will backup events when the Elasticsearch broker goes offline and replays it when the broker comes back online. The event store can be configured by setting the directory path in `queue_dir` field and the maximum limit of events in the queue_dir in `queue_limit` field. For eg, the `queue_dir` can be `/home/events` and `queue_limit` can be `1000`. By default, the `queue_limit` is set to 10000.
MinIO supports persistent event store. The persistent store will backup events when the Elasticsearch broker goes offline and replays it when the broker comes back online. The event store can be configured by setting the directory path in `queue_dir` field and the maximum limit of events in the queue_dir in `queue_limit` field. For eg, the `queue_dir` can be `/home/events` and `queue_limit` can be `1000`. By default, the `queue_limit` is set to 100000.
If Elasticsearch has authentication enabled, the credentials can be supplied to MinIO via the `url` parameter formatted as `PROTO://USERNAME:PASSWORD@ELASTICSEARCH_HOST:PORT`.
@ -502,7 +502,7 @@ key* (string) Redis key to store/update events, key is auto-
format* (namespace*|access) 'namespace' reflects current bucket/object list and 'access' reflects a journal of object operations, defaults to 'namespace'
password (string) Redis server password
queue_dir (path) staging dir for undelivered messages e.g. '/home/events'
queue_limit (number) maximum limit for undelivered messages, defaults to '10000'
queue_limit (number) maximum limit for undelivered messages, defaults to '100000'
comment (sentence) optionally add a comment to this setting
```
@ -518,11 +518,11 @@ MINIO_NOTIFY_REDIS_KEY* (string) Redis key to store/update e
MINIO_NOTIFY_REDIS_FORMAT* (namespace*|access) 'namespace' reflects current bucket/object list and 'access' reflects a journal of object operations, defaults to 'namespace'
MINIO_NOTIFY_REDIS_PASSWORD (string) Redis server password
MINIO_NOTIFY_REDIS_QUEUE_DIR (path) staging dir for undelivered messages e.g. '/home/events'
MINIO_NOTIFY_REDIS_QUEUE_LIMIT (number) maximum limit for undelivered messages, defaults to '10000'
MINIO_NOTIFY_REDIS_QUEUE_LIMIT (number) maximum limit for undelivered messages, defaults to '100000'
MINIO_NOTIFY_REDIS_COMMENT (sentence) optionally add a comment to this setting
```
MinIO supports persistent event store. The persistent store will backup events when the Redis broker goes offline and replays it when the broker comes back online. The event store can be configured by setting the directory path in `queue_dir` field and the maximum limit of events in the queue_dir in `queue_limit` field. For eg, the `queue_dir` can be `/home/events` and `queue_limit` can be `1000`. By default, the `queue_limit` is set to 10000.
MinIO supports persistent event store. The persistent store will backup events when the Redis broker goes offline and replays it when the broker comes back online. The event store can be configured by setting the directory path in `queue_dir` field and the maximum limit of events in the queue_dir in `queue_limit` field. For eg, the `queue_dir` can be `/home/events` and `queue_limit` can be `1000`. By default, the `queue_limit` is set to 100000.
To update the configuration, use `mc admin config get` command to get the current configuration.
@ -591,7 +591,7 @@ Install NATS from [here](http://nats.io/).
### Step 1: Add NATS endpoint to MinIO
MinIO supports persistent event store. The persistent store will backup events when the NATS broker goes offline and replays it when the broker comes back online. The event store can be configured by setting the directory path in `queue_dir` field and the maximum limit of events in the queue_dir in `queue_limit` field. For eg, the `queue_dir` can be `/home/events` and `queue_limit` can be `1000`. By default, the `queue_limit` is set to 10000.
MinIO supports persistent event store. The persistent store will backup events when the NATS broker goes offline and replays it when the broker comes back online. The event store can be configured by setting the directory path in `queue_dir` field and the maximum limit of events in the queue_dir in `queue_limit` field. For eg, the `queue_dir` can be `/home/events` and `queue_limit` can be `1000`. By default, the `queue_limit` is set to 100000.
```
KEY:
@ -614,7 +614,7 @@ cert_authority (string) path to certificate chain of the t
client_cert (string) client cert for NATS mTLS auth
client_key (string) client cert key for NATS mTLS auth
queue_dir (path) staging dir for undelivered messages e.g. '/home/events'
queue_limit (number) maximum limit for undelivered messages, defaults to '10000'
queue_limit (number) maximum limit for undelivered messages, defaults to '100000'
comment (sentence) optionally add a comment to this setting
```
@ -641,7 +641,7 @@ MINIO_NOTIFY_NATS_CERT_AUTHORITY (string) path to certific
MINIO_NOTIFY_NATS_CLIENT_CERT (string) client cert for NATS mTLS auth
MINIO_NOTIFY_NATS_CLIENT_KEY (string) client cert key for NATS mTLS auth
MINIO_NOTIFY_NATS_QUEUE_DIR (path) staging dir for undelivered messages e.g. '/home/events'
MINIO_NOTIFY_NATS_QUEUE_LIMIT (number) maximum limit for undelivered messages, defaults to '10000'
MINIO_NOTIFY_NATS_QUEUE_LIMIT (number) maximum limit for undelivered messages, defaults to '100000'
MINIO_NOTIFY_NATS_COMMENT (sentence) optionally add a comment to this setting
```
@ -856,7 +856,7 @@ connection_string* (string) Postgres server connection-string e.g.
table* (string) DB table name to store/update events, table is auto-created
format* (namespace*|access) 'namespace' reflects current bucket/object list and 'access' reflects a journal of object operations, defaults to 'namespace'
queue_dir (path) staging dir for undelivered messages e.g. '/home/events'
queue_limit (number) maximum limit for undelivered messages, defaults to '10000'
queue_limit (number) maximum limit for undelivered messages, defaults to '100000'
comment (sentence) optionally add a comment to this setting
```
@ -871,11 +871,11 @@ MINIO_NOTIFY_POSTGRES_CONNECTION_STRING* (string) Postgres server c
MINIO_NOTIFY_POSTGRES_TABLE* (string) DB table name to store/update events, table is auto-created
MINIO_NOTIFY_POSTGRES_FORMAT* (namespace*|access) 'namespace' reflects current bucket/object list and 'access' reflects a journal of object operations, defaults to 'namespace'
MINIO_NOTIFY_POSTGRES_QUEUE_DIR (path) staging dir for undelivered messages e.g. '/home/events'
MINIO_NOTIFY_POSTGRES_QUEUE_LIMIT (number) maximum limit for undelivered messages, defaults to '10000'
MINIO_NOTIFY_POSTGRES_QUEUE_LIMIT (number) maximum limit for undelivered messages, defaults to '100000'
MINIO_NOTIFY_POSTGRES_COMMENT (sentence) optionally add a comment to this setting
```
MinIO supports persistent event store. The persistent store will backup events when the PostgreSQL connection goes offline and replays it when the broker comes back online. The event store can be configured by setting the directory path in `queue_dir` field and the maximum limit of events in the queue_dir in `queue_limit` field. For eg, the `queue_dir` can be `/home/events` and `queue_limit` can be `1000`. By default, the `queue_limit` is set to 10000.
MinIO supports persistent event store. The persistent store will backup events when the PostgreSQL connection goes offline and replays it when the broker comes back online. The event store can be configured by setting the directory path in `queue_dir` field and the maximum limit of events in the queue_dir in `queue_limit` field. For eg, the `queue_dir` can be `/home/events` and `queue_limit` can be `1000`. By default, the `queue_limit` is set to 100000.
Note that for illustration here, we have disabled SSL. In the interest of security, for production this is not recommended.
To update the configuration, use `mc admin config get` command to get the current configuration.
@ -985,7 +985,7 @@ dsn_string* (string) MySQL data-source-name connection string e.g.
table* (string) DB table name to store/update events, table is auto-created
format* (namespace*|access) 'namespace' reflects current bucket/object list and 'access' reflects a journal of object operations, defaults to 'namespace'
queue_dir (path) staging dir for undelivered messages e.g. '/home/events'
queue_limit (number) maximum limit for undelivered messages, defaults to '10000'
queue_limit (number) maximum limit for undelivered messages, defaults to '100000'
comment (sentence) optionally add a comment to this setting
```
@ -1000,13 +1000,13 @@ MINIO_NOTIFY_MYSQL_DSN_STRING* (string) MySQL data-source-name conn
MINIO_NOTIFY_MYSQL_TABLE* (string) DB table name to store/update events, table is auto-created
MINIO_NOTIFY_MYSQL_FORMAT* (namespace*|access) 'namespace' reflects current bucket/object list and 'access' reflects a journal of object operations, defaults to 'namespace'
MINIO_NOTIFY_MYSQL_QUEUE_DIR (path) staging dir for undelivered messages e.g. '/home/events'
MINIO_NOTIFY_MYSQL_QUEUE_LIMIT (number) maximum limit for undelivered messages, defaults to '10000'
MINIO_NOTIFY_MYSQL_QUEUE_LIMIT (number) maximum limit for undelivered messages, defaults to '100000'
MINIO_NOTIFY_MYSQL_COMMENT (sentence) optionally add a comment to this setting
```
`dsn_string` is required and is of form `"<user>:<password>@tcp(<host>:<port>)/<database>"`
MinIO supports persistent event store. The persistent store will backup events if MySQL connection goes offline and then replays the stored events when the broken connection comes back up. The event store can be configured by setting a directory path in `queue_dir` field, and the maximum number of events, which can be stored in a `queue_dir`, in `queue_limit` field. For example, `queue_dir` can be set to `/home/events` and `queue_limit` can be set to `1000`. By default, the `queue_limit` is set to `10000`.
MinIO supports persistent event store. The persistent store will backup events if MySQL connection goes offline and then replays the stored events when the broken connection comes back up. The event store can be configured by setting a directory path in `queue_dir` field, and the maximum number of events, which can be stored in a `queue_dir`, in `queue_limit` field. For example, `queue_dir` can be set to `/home/events` and `queue_limit` can be set to `1000`. By default, the `queue_limit` is set to `100000`.
Before updating the configuration, let's start with `mc admin config get` command to get the current configuration.
@ -1077,7 +1077,7 @@ MinIO requires Kafka version 0.10 or 0.9. Internally MinIO uses the [Shopify/sar
### Step 2: Add Kafka endpoint to MinIO
MinIO supports persistent event store. The persistent store will backup events when the kafka broker goes offline and replays it when the broker comes back online. The event store can be configured by setting the directory path in `queue_dir` field and the maximum limit of events in the queue_dir in `queue_limit` field. For eg, the `queue_dir` can be `/home/events` and `queue_limit` can be `1000`. By default, the `queue_limit` is set to 10000.
MinIO supports persistent event store. The persistent store will backup events when the kafka broker goes offline and replays it when the broker comes back online. The event store can be configured by setting the directory path in `queue_dir` field and the maximum limit of events in the queue_dir in `queue_limit` field. For eg, the `queue_dir` can be `/home/events` and `queue_limit` can be `1000`. By default, the `queue_limit` is set to 100000.
```
KEY:
@ -1096,7 +1096,7 @@ tls_skip_verify (on|off) trust server TLS without verification, defaults to
client_tls_cert (path) path to client certificate for mTLS auth
client_tls_key (path) path to client key for mTLS auth
queue_dir (path) staging dir for undelivered messages e.g. '/home/events'
queue_limit (number) maximum limit for undelivered messages, defaults to '10000'
queue_limit (number) maximum limit for undelivered messages, defaults to '100000'
version (string) specify the version of the Kafka cluster e.g '2.2.0'
comment (sentence) optionally add a comment to this setting
```
@ -1120,7 +1120,7 @@ MINIO_NOTIFY_KAFKA_TLS_SKIP_VERIFY (on|off) trust server TLS wit
MINIO_NOTIFY_KAFKA_CLIENT_TLS_CERT (path) path to client certificate for mTLS auth
MINIO_NOTIFY_KAFKA_CLIENT_TLS_KEY (path) path to client key for mTLS auth
MINIO_NOTIFY_KAFKA_QUEUE_DIR (path) staging dir for undelivered messages e.g. '/home/events'
MINIO_NOTIFY_KAFKA_QUEUE_LIMIT (number) maximum limit for undelivered messages, defaults to '10000'
MINIO_NOTIFY_KAFKA_QUEUE_LIMIT (number) maximum limit for undelivered messages, defaults to '100000'
MINIO_NOTIFY_KAFKA_COMMENT (sentence) optionally add a comment to this setting
MINIO_NOTIFY_KAFKA_VERSION (string) specify the version of the Kafka cluster e.g. '2.2.0'
```
@ -1230,7 +1230,7 @@ kafkacat -b localhost:9092 -t bucketevents
### Step 1: Add Webhook endpoint to MinIO
MinIO supports persistent event store. The persistent store will backup events when the webhook goes offline and replays it when the broker comes back online. The event store can be configured by setting the directory path in `queue_dir` field and the maximum limit of events in the queue_dir in `queue_limit` field. For eg, the `queue_dir` can be `/home/events` and `queue_limit` can be `1000`. By default, the `queue_limit` is set to 10000.
MinIO supports persistent event store. The persistent store will backup events when the webhook goes offline and replays it when the broker comes back online. The event store can be configured by setting the directory path in `queue_dir` field and the maximum limit of events in the queue_dir in `queue_limit` field. For eg, the `queue_dir` can be `/home/events` and `queue_limit` can be `1000`. By default, the `queue_limit` is set to 100000.
```
KEY:
@ -1240,7 +1240,7 @@ ARGS:
endpoint* (url) webhook server endpoint e.g. http://localhost:8080/minio/events
auth_token (string) opaque string or JWT authorization token
queue_dir (path) staging dir for undelivered messages e.g. '/home/events'
queue_limit (number) maximum limit for undelivered messages, defaults to '10000'
queue_limit (number) maximum limit for undelivered messages, defaults to '100000'
comment (sentence) optionally add a comment to this setting
```
@ -1254,7 +1254,7 @@ MINIO_NOTIFY_WEBHOOK_ENABLE* (on|off) enable notify_webhook target, defa
MINIO_NOTIFY_WEBHOOK_ENDPOINT* (url) webhook server endpoint e.g. http://localhost:8080/minio/events
MINIO_NOTIFY_WEBHOOK_AUTH_TOKEN (string) opaque string or JWT authorization token
MINIO_NOTIFY_WEBHOOK_QUEUE_DIR (path) staging dir for undelivered messages e.g. '/home/events'
MINIO_NOTIFY_WEBHOOK_QUEUE_LIMIT (number) maximum limit for undelivered messages, defaults to '10000'
MINIO_NOTIFY_WEBHOOK_QUEUE_LIMIT (number) maximum limit for undelivered messages, defaults to '100000'
MINIO_NOTIFY_WEBHOOK_COMMENT (sentence) optionally add a comment to this setting
```
@ -1333,7 +1333,7 @@ docker run --rm -p 4150-4151:4150-4151 nsqio/nsq /nsqd
### Step 1: Add NSQ endpoint to MinIO
MinIO supports persistent event store. The persistent store will backup events when the NSQ broker goes offline and replays it when the broker comes back online. The event store can be configured by setting the directory path in `queue_dir` field and the maximum limit of events in the queue_dir in `queue_limit` field. For eg, the `queue_dir` can be `/home/events` and `queue_limit` can be `1000`. By default, the `queue_limit` is set to 10000.
MinIO supports persistent event store. The persistent store will backup events when the NSQ broker goes offline and replays it when the broker comes back online. The event store can be configured by setting the directory path in `queue_dir` field and the maximum limit of events in the queue_dir in `queue_limit` field. For eg, the `queue_dir` can be `/home/events` and `queue_limit` can be `1000`. By default, the `queue_limit` is set to 100000.
To update the configuration, use `mc admin config get` command to get the current configuration for `notify_nsq`.
@ -1347,7 +1347,7 @@ topic* (string) NSQ topic
tls (on|off) set to 'on' to enable TLS
tls_skip_verify (on|off) trust server TLS without verification, defaults to "on" (verify)
queue_dir (path) staging dir for undelivered messages e.g. '/home/events'
queue_limit (number) maximum limit for undelivered messages, defaults to '10000'
queue_limit (number) maximum limit for undelivered messages, defaults to '100000'
comment (sentence) optionally add a comment to this setting
```
@ -1363,7 +1363,7 @@ MINIO_NOTIFY_NSQ_TOPIC* (string) NSQ topic
MINIO_NOTIFY_NSQ_TLS (on|off) set to 'on' to enable TLS
MINIO_NOTIFY_NSQ_TLS_SKIP_VERIFY (on|off) trust server TLS without verification, defaults to "on" (verify)
MINIO_NOTIFY_NSQ_QUEUE_DIR (path) staging dir for undelivered messages e.g. '/home/events'
MINIO_NOTIFY_NSQ_QUEUE_LIMIT (number) maximum limit for undelivered messages, defaults to '10000'
MINIO_NOTIFY_NSQ_QUEUE_LIMIT (number) maximum limit for undelivered messages, defaults to '100000'
MINIO_NOTIFY_NSQ_COMMENT (sentence) optionally add a comment to this setting
```

3
pkg/event/target/amqp.go
View File

@ -101,9 +101,6 @@ func (a *AMQPArgs) Validate() error {
return errors.New("queueDir path should be absolute")
}
}
if a.QueueLimit > 10000 {
return errors.New("queueLimit should not exceed 10000")
}
return nil
}

3
pkg/event/target/elasticsearch.go
View File

@ -74,9 +74,6 @@ func (a ElasticsearchArgs) Validate() error {
if a.Index == "" {
return errors.New("empty index value")
}
if a.QueueLimit > 10000 {
return errors.New("queueLimit should not exceed 10000")
}
return nil
}

3
pkg/event/target/kafka.go
View File

@ -110,9 +110,6 @@ func (k KafkaArgs) Validate() error {
return errors.New("queueDir path should be absolute")
}
}
if k.QueueLimit > 10000 {
return errors.New("queueLimit should not exceed 10000")
}
if k.Version != "" {
if _, err := sarama.ParseKafkaVersion(k.Version); err != nil {
return err

3
pkg/event/target/mqtt.go
View File

@ -99,9 +99,6 @@ func (m MQTTArgs) Validate() error {
return errors.New("qos should be set to 1 or 2 if queueDir is set")
}
}
if m.QueueLimit > 10000 {
return errors.New("queueLimit should not exceed 10000")
}
return nil
}

3
pkg/event/target/mysql.go
View File

@ -161,9 +161,6 @@ func (m MySQLArgs) Validate() error {
return errors.New("queueDir path should be absolute")
}
}
if m.QueueLimit > 10000 {
return errors.New("queueLimit should not exceed 10000")
}
return nil
}

3
pkg/event/target/nats.go
View File

@ -136,9 +136,6 @@ func (n NATSArgs) Validate() error {
return errors.New("queueDir path should be absolute")
}
}
if n.QueueLimit > 10000 {
return errors.New("queueLimit should not exceed 10000")
}
return nil
}

3
pkg/event/target/nsq.go
View File

@ -80,9 +80,6 @@ func (n NSQArgs) Validate() error {
return errors.New("queueDir path should be absolute")
}
}
if n.QueueLimit > 10000 {
return errors.New("queueLimit should not exceed 10000")
}
return nil
}

3
pkg/event/target/postgresql.go
View File

@ -159,9 +159,6 @@ func (p PostgreSQLArgs) Validate() error {
return errors.New("queueDir path should be absolute")
}
}
if p.QueueLimit > 10000 {
return errors.New("queueLimit should not exceed 10000")
}
return nil
}

6
pkg/event/target/queuestore.go
View File

@ -30,8 +30,8 @@ import (
)
const (
maxLimit = 10000 // Max store limit.
eventExt = ".event"
defaultLimit = 100000 // Default store limit.
eventExt = ".event"
)
// QueueStore - Filestore for persisting events.
@ -45,7 +45,7 @@ type QueueStore struct {
// NewQueueStore - Creates an instance for QueueStore.
func NewQueueStore(directory string, limit uint64) Store {
if limit == 0 {
limit = maxLimit
limit = defaultLimit
_, maxRLimit, err := sys.GetMaxOpenFileLimit()
if err == nil {
// Limit the maximum number of entries

3
pkg/event/target/redis.go
View File

@ -89,9 +89,6 @@ func (r RedisArgs) Validate() error {
return errors.New("queueDir path should be absolute")
}
}
if r.QueueLimit > 10000 {
return errors.New("queueLimit should not exceed 10000")
}
return nil
}

3
pkg/event/target/webhook.go
View File

@ -70,9 +70,6 @@ func (w WebhookArgs) Validate() error {
return errors.New("queueDir path should be absolute")
}
}
if w.QueueLimit > maxLimit {
return errors.New("queueLimit should not exceed 10000")
}
return nil
}