aboutsummaryrefslogtreecommitdiffstats
path: root/modules/setting/queue.go
Commit message (Collapse)AuthorAgeFilesLines
* Add generic set type (#21408)KN4CK3R2022-10-121-13/+14
| | | | | This PR adds a generic set type to get rid of maps used as sets. Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
* Enable deprecation error for v1.17.0 (#18341)Gusted2022-01-201-18/+38
| | | Co-authored-by: Andrew Thornton <art27@cantab.net>
* A better go code formatter, and now `make fmt` can run in Windows (#17684)wxiaoguang2021-11-171-0/+1
| | | | * go build / format tools * re-format imports
* Make the Mirror Queue a queue (#17326)zeripath2021-10-171-25/+39
| | | | | | | | | Convert the old mirror syncing queue to the more modern queue format. Fix a bug in the from the repo-archive queue PR - the assumption was made that uniqueness could be enforced with by checking equality in a map in channel unique queues - however this only works for primitive types - which was the initial intention but is an imperfect. This is fixed by marshalling the data and placing the martialled data in the unique map instead. The documentation is also updated to add information about the deprecated configuration values. Signed-off-by: Andrew Thornton <art27@cantab.net>
* Remove ParseQueueConnStr as it is unused (#16878)zeripath2021-08-311-33/+0
| | | | | Remove ParseQueueConnStr as `modules/nosql` has taken over all of its functions. Signed-off-by: Andrew Thornton <art27@cantab.net>
* Clean-up the settings hierarchy for issue_indexer queue (#16001)zeripath2021-06-161-6/+9
| | | | | | | | | There are a couple of settings in `[indexer]` relating to the `issue_indexer` queue which override settings in unpredictable ways. This PR adjusts this hierarchy and makes explicit that these settings are deprecated. Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Use filepath.ToSlash and Join in indexer defaults and queues (#15971)zeripath2021-05-251-3/+3
| | | | | | | | | As revealed by #15964 there is inconsistent use of filepath Join and path Join for these directories. The best thing to do is to use filepath.Join but then ToSlash them for consistency. Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: John Olheiser <john.olheiser@gmail.com>
* Change default queue settings to be low go-routines (#15964)zeripath2021-05-241-3/+3
| | | | | | | | This PR suggests a change to the default configuration for queues: * Use a common DATADIR for the queues * Set starting workers to 0 and make boost a single worker Signed-off-by: Andrew Thornton <art27@cantab.net>
* Slightly simplify the queue settings code to help reduce the risk of ↵zeripath2020-10-151-3/+7
| | | | | | | problems (#12976) Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Avoid setitng the CONN_STR in queues unless it is meant to be set (#13025)zeripath2020-10-041-2/+1
| | | | | | | | | | | Since the move to common leveldb and common redis the disk queue code will check the connection string before defaulting to the DATADIR. Therefore we should ensure that the connection string is kept empty unless it is actually set. Fix #13023 Signed-off-by: Andrew Thornton <art27@cantab.net>
* Add Unique Queue infrastructure and move TestPullRequests to this (#9856)zeripath2020-02-021-0/+18
| | | | | | | | | | | | | | | | | | | | | * Upgrade levelqueue to version 0.2.0 This adds functionality for Unique Queues * Add UniqueQueue interface and functions to create them * Add UniqueQueue implementations * Move TestPullRequests over to use UniqueQueue * Reduce code duplication * Add bytefifos * Ensure invalid types are logged * Fix close race in PersistableChannelQueue Shutdown
* Queue: Make WorkerPools and Queues flushable (#10001)zeripath2020-01-281-10/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Make WorkerPools and Queues flushable Adds Flush methods to Queues and the WorkerPool Further abstracts the WorkerPool Adds a final step to Flush the queues in the defer from PrintCurrentTest Fixes an issue with Settings inheritance in queues Signed-off-by: Andrew Thornton <art27@cantab.net> * Change to for loop * Add IsEmpty and begin just making the queues composed WorkerPools * subsume workerpool into the queues and create a flushable interface * Add manager command * Move flushall to queue.Manager and add to testlogger * As per @guillep2k * as per @guillep2k * Just make queues all implement flushable and clean up the wrapped queue flushes * cope with no timeout Co-authored-by: Lauris BH <lauris@nix.lv>
* Move mailer to use a queue (#9789)zeripath2020-01-161-7/+17
| | | | | | | | * Move mailer to use a queue * Make sectionMap map[string]bool * Ensure that Message is json encodable
* fix #9648 | use filepath.IsAbs instead of path.IsAbs (#9651)65432020-01-081-5/+6
| | | | | | | | * fix #9648 * found next Co-authored-by: Antoine GIRARD <sapk@users.noreply.github.com>
* Graceful Queues: Issue Indexing and Tasks (#9363)zeripath2020-01-071-0/+159
* Queue: Add generic graceful queues with settings * Queue & Setting: Add worker pool implementation * Queue: Add worker settings * Queue: Make resizing worker pools * Queue: Add name variable to queues * Queue: Add monitoring * Queue: Improve logging * Issues: Gracefulise the issues indexer Remove the old now unused specific queues * Task: Move to generic queue and gracefulise * Issues: Standardise the issues indexer queue settings * Fix test * Queue: Allow Redis to connect to unix * Prevent deadlock during early shutdown of issue indexer * Add MaxWorker settings to queues * Merge branch 'master' into graceful-queues * Update modules/indexer/issues/indexer.go Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com> * Update modules/indexer/issues/indexer.go Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com> * Update modules/queue/queue_channel.go Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com> * Update modules/queue/queue_disk.go * Update modules/queue/queue_disk_channel.go Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com> * Rename queue.Description to queue.ManagedQueue as per @guillep2k * Cancel pool workers when removed * Remove dependency on queue from setting * Update modules/queue/queue_redis.go Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com> * As per @guillep2k add mutex locks on shutdown/terminate * move unlocking out of setInternal * Add warning if number of workers < 0 * Small changes as per @guillep2k * No redis host specified not found * Clean up documentation for queues * Update docs/content/doc/advanced/config-cheat-sheet.en-us.md * Update modules/indexer/issues/indexer_test.go * Ensure that persistable channel queue is added to manager * Rename QUEUE_NAME REDIS_QUEUE_NAME * Revert "Rename QUEUE_NAME REDIS_QUEUE_NAME" This reverts commit 1f83b4fc9b9dabda186257b38c265fe7012f90df. Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com> Co-authored-by: Lauris BH <lauris@nix.lv> Co-authored-by: techknowlogick <matti@mdranta.net> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
generated by cgit v1.2.3 (git 2.39.1) at 2025-01-16 02:54:36 +0000