Update queue workers for v1.15 (#15999)

* Update queue workers for v1.15

* update app.example.ini

* update re queue path

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Andrew Thornton <art27@cantab.net>

diff --git a/custom/conf/app.example.ini b/custom/conf/app.example.ini
index f37157c3b10e1ce9c908531a75bcb5070d5fd088..2f3fc1f1d850460931920a4896bd2a0b4910ca99 100644 (file)
--- a/custom/conf/app.example.ini
+++ b/custom/conf/app.example.ini
@@ -1145,8 +1145,8 @@ PATH =
 ;;
 ;; When ISSUE_INDEXER_QUEUE_TYPE is levelqueue, this will be the path where the queue will be saved.
 ;; This can be overridden by `ISSUE_INDEXER_QUEUE_CONN_STR`.
-;; default is indexers/issues.queue
-;ISSUE_INDEXER_QUEUE_DIR = indexers/issues.queue
+;; default is queues/common\r
+;ISSUE_INDEXER_QUEUE_DIR = queues/common\r
 ;;
 ;; When `ISSUE_INDEXER_QUEUE_TYPE` is `redis`, this will store the redis connection string.
 ;; When `ISSUE_INDEXER_QUEUE_TYPE` is `levelqueue`, this is a directory or additional options of
@@ -1201,7 +1201,7 @@ PATH =
 ;; default to persistable-channel
 ;TYPE = persistable-channel
 ;;
-;; data-dir for storing persistable queues and level queues, individual queues will be named by their type
+;; data-dir for storing persistable queues and level queues, individual queues will default to `queues/common` meaning the queue is shared.\r
 ;DATADIR = queues/
 ;;
 ;; Default queue length before a channel queue will block
@@ -1231,7 +1231,7 @@ PATH =
 ;TIMEOUT = 15m30s
 ;;
 ;; Create a pool with this many workers
-;WORKERS = 1
+;WORKERS = 0
 ;;
 ;; Dynamically scale the worker pool to at this many workers
 ;MAX_WORKERS = 10
@@ -1243,7 +1243,7 @@ PATH =
 ;BOOST_TIMEOUT = 5m
 ;;
 ;; During a boost add BOOST_WORKERS
-;BOOST_WORKERS = 5
+;BOOST_WORKERS = 1
 
 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

diff --git a/docs/content/doc/advanced/config-cheat-sheet.en-us.md b/docs/content/doc/advanced/config-cheat-sheet.en-us.md
index ee701b657248264466aaa30bacbeff63869a5994..28ab922c439ebb419144ecc2206d0474a2684888 100644 (file)
--- a/docs/content/doc/advanced/config-cheat-sheet.en-us.md
+++ b/docs/content/doc/advanced/config-cheat-sheet.en-us.md
@@ -350,7 +350,7 @@ relation to port exhaustion.
 - `ISSUE_INDEXER_PATH`: **indexers/issues.bleve**: Index file used for issue search; available when ISSUE_INDEXER_TYPE is bleve and elasticsearch.
 - The next 4 configuration values are deprecated and should be set in `queue.issue_indexer` however are kept for backwards compatibility:
 - `ISSUE_INDEXER_QUEUE_TYPE`: **levelqueue**: Issue indexer queue, currently supports:`channel`, `levelqueue`, `redis`.
-- `ISSUE_INDEXER_QUEUE_DIR`: **indexers/issues.queue**: When `ISSUE_INDEXER_QUEUE_TYPE` is `levelqueue`, this will be the path where the queue will be saved.
+- `ISSUE_INDEXER_QUEUE_DIR`: **queues/common**: When `ISSUE_INDEXER_QUEUE_TYPE` is `levelqueue`, this will be the path where the queue will be saved. (Previously this was `indexers/issues.queue`.)\r
 - `ISSUE_INDEXER_QUEUE_CONN_STR`: **addrs=127.0.0.1:6379 db=0**: When `ISSUE_INDEXER_QUEUE_TYPE` is `redis`, this will store the redis connection string. When `ISSUE_INDEXER_QUEUE_TYPE` is `levelqueue`, this is a directory or additional options of the form `leveldb://path/to/db?option=value&....`, and overrides `ISSUE_INDEXER_QUEUE_DIR`.
 - `ISSUE_INDEXER_QUEUE_BATCH_NUMBER`: **20**: Batch queue number.
 
@@ -370,7 +370,7 @@ relation to port exhaustion.
 ## Queue (`queue` and `queue.*`)
 
 - `TYPE`: **persistable-channel**: General queue type, currently support: `persistable-channel` (uses a LevelDB internally), `channel`, `level`, `redis`, `dummy`
-- `DATADIR`: **queues/**: Base DataDir for storing persistent and level queues. `DATADIR` for individual queues can be set in `queue.name` sections but will default to `DATADIR/`**`name`**.
+- `DATADIR`: **queues/**: Base DataDir for storing persistent and level queues. `DATADIR` for individual queues can be set in `queue.name` sections but will default to `DATADIR/`**`common`**. (Previously each queue would default to `DATADIR/`**`name`**.)\r
 - `LENGTH`: **20**: Maximal queue size before channel queues block
 - `BATCH_LENGTH`: **20**: Batch data before passing to the handler
 - `CONN_STR`: **redis://127.0.0.1:6379/0**: Connection string for the redis queue type. Options can be set using query params. Similarly LevelDB options can also be set using: **leveldb://relative/path?option=value** or **leveldb:///absolute/path?option=value**, and will override `DATADIR`
@@ -381,11 +381,11 @@ relation to port exhaustion.
 - `MAX_ATTEMPTS`: **10**: Maximum number of attempts to create the wrapped queue
 - `TIMEOUT`: **GRACEFUL_HAMMER_TIME + 30s**: Timeout the creation of the wrapped queue if it takes longer than this to create.
 - Queues by default come with a dynamically scaling worker pool. The following settings configure this:
-- `WORKERS`: **1**: Number of initial workers for the queue.
+- `WORKERS`: **0** (v1.14 and before: **1**): Number of initial workers for the queue. 
 - `MAX_WORKERS`: **10**: Maximum number of worker go-routines for the queue.
 - `BLOCK_TIMEOUT`: **1s**: If the queue blocks for this time, boost the number of workers - the `BLOCK_TIMEOUT` will then be doubled before boosting again whilst the boost is ongoing.
 - `BOOST_TIMEOUT`: **5m**: Boost workers will timeout after this long.
-- `BOOST_WORKERS`: **5**: This many workers will be added to the worker pool if there is a boost.
+- `BOOST_WORKERS`: **1** (v1.14 and before: **5**): This many workers will be added to the worker pool if there is a boost.
 
 ## Admin (`admin`)