aboutsummaryrefslogtreecommitdiffstats
path: root/modules/queue
Commit message (Collapse)AuthorAgeFilesLines
* Multiple small admin dashboard fixes (#12153)zeripath2020-07-051-0/+1
| | | | | | | | | * Remove spurious spacing between Maintenance Operations and its table on dashboard * Prevent (EXTRA string) comments in Task headers * Redirect tasks started from monitor page back to monitor * Fix #12107 - redirects from process cancel should use AppSubUrl * When wrapping queues set the name correctly Signed-off-by: Andrew Thornton <art27@cantab.net>
* Prevent timer leaks in Workerpool and others (#11333)zeripath2020-05-082-19/+6
| | | | | | | | | | | | | | | | There is a potential memory leak in `Workerpool` due to the intricacies of `time.Timer` stopping. Whenever a `time.Timer` is `Stop`ped its channel must be cleared using a `select` if the result of the `Stop()` is `false`. Unfortunately in `Workerpool` these were checked the wrong way round. However, there were a few other places that were not being checked. Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: techknowlogick <techknowlogick@gitea.io> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Improve config logging when WrappedQueue times out (#11174)Lunny Xiao2020-04-221-1/+5
| | | | | | | | | | | | Before ```sh Unable to set the internal queue for -wrapper Error: Timedout creating queue redis with cfg []byte{0x7b, 0x22, 0x41, 0x64, 0x64, 0x72, 0x65, 0x73, 0x73, 0x65, 0x73, 0x22, 0x3a, 0x22, 0x31, 0x32, 0x37, 0x2e, 0x30, 0x2e, 0x30, 0x2e, 0x31, 0x3a, 0x36, 0x33, 0x37, 0x39, 0x22, 0x2c, 0x22, 0x42, 0x61, 0x74, 0x63, 0x68, 0x4c, 0x65, 0x6e, 0x67, 0x74, 0x68, 0x22, 0x3a, 0x32, 0x30, 0x2c, 0x22, 0x42, 0x6c, 0x6f, 0x63, 0x6b, 0x54, 0x69, 0x6d, 0x65, 0x6f, 0x75, 0x74, 0x22, 0x3a, 0x31, 0x30, 0x30, 0x30, 0x30, 0x30, 0x30, 0x30, 0x30, 0x30, 0x2c, 0x22, 0x42, 0x6f, 0x6f, 0x73, 0x74, 0x54, 0x69, 0x6d, 0x65, 0x6f, 0x75, 0x74, 0x22, 0x3a, 0x33, 0x30, 0x30, 0x30, 0x30, 0x30 ...... ``` After ```sh Unable to set the internal queue for -wrapper Error: Timedout creating queue redis with cfg "{\"Addresses\":\"127.0.0.1:6379\",\"BatchLength\":20,\"BlockTimeout\":1000000000,\"BoostTimeout\":300000000000,\"BoostWorkers\":5,\"DBIndex\":0,\"DataDir\":\".../data/queues/mail\",\"MaxWorkers\":10,\"Name\":\"mail\",\"Network\":\"\",\"Password\":\"\",\"QueueLength\":20,\"QueueName\":\"mail_queue\",\"SetName\":\"\",\"Workers\":1}" in ```
* Avoid logging []byte in queue failures - convert to string first (#10865)zeripath2020-03-291-1/+8
| | | | | Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
* Fix queue log param (#10733)Lunny Xiao2020-03-161-3/+3
|
* Fix Workerpool deadlock (#10283)zeripath2020-02-152-3/+6
| | | | | | * Prevent deadlock on boost * Force a boost in testchannelqueue
* Fix queue pop error and stat empty repository error (#10248)Lunny Xiao2020-02-122-2/+4
| | | | | | * Fix queue pop error and stat empty repository error * Fix error
* Add Unique Queue infrastructure and move TestPullRequests to this (#9856)zeripath2020-02-0215-403/+1371
| | | | | | | | | | | | | | | | | | | | | * Upgrade levelqueue to version 0.2.0 This adds functionality for Unique Queues * Add UniqueQueue interface and functions to create them * Add UniqueQueue implementations * Move TestPullRequests over to use UniqueQueue * Reduce code duplication * Add bytefifos * Ensure invalid types are logged * Fix close race in PersistableChannelQueue Shutdown
* Fix broken FlushAll (#10101)zeripath2020-02-011-4/+4
| | | | | | | | | | * go function contexting is not what you expect * Apply suggestions from code review Co-Authored-By: Lauris BH <lauris@nix.lv> Co-authored-by: Lauris BH <lauris@nix.lv>
* Fix leveldb test race (#10054)Lunny Xiao2020-01-301-0/+18
| | | | | Co-authored-by: Lauris BH <lauris@nix.lv> Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Queue: Make WorkerPools and Queues flushable (#10001)zeripath2020-01-2812-292/+628
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Make WorkerPools and Queues flushable Adds Flush methods to Queues and the WorkerPool Further abstracts the WorkerPool Adds a final step to Flush the queues in the defer from PrintCurrentTest Fixes an issue with Settings inheritance in queues Signed-off-by: Andrew Thornton <art27@cantab.net> * Change to for loop * Add IsEmpty and begin just making the queues composed WorkerPools * subsume workerpool into the queues and create a flushable interface * Add manager command * Move flushall to queue.Manager and add to testlogger * As per @guillep2k * as per @guillep2k * Just make queues all implement flushable and clean up the wrapped queue flushes * cope with no timeout Co-authored-by: Lauris BH <lauris@nix.lv>
* Update queue_redis.go (#9910)zeripath2020-01-211-5/+6
| | | | Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Remove unused lock (#9709)zeripath2020-01-112-1/+3
|
* Missed q.lock.Unlock() will cause panic (#9705)zeripath2020-01-111-1/+0
|
* Graceful Queues: Issue Indexing and Tasks (#9363)zeripath2020-01-0713-0/+2132
* Queue: Add generic graceful queues with settings * Queue & Setting: Add worker pool implementation * Queue: Add worker settings * Queue: Make resizing worker pools * Queue: Add name variable to queues * Queue: Add monitoring * Queue: Improve logging * Issues: Gracefulise the issues indexer Remove the old now unused specific queues * Task: Move to generic queue and gracefulise * Issues: Standardise the issues indexer queue settings * Fix test * Queue: Allow Redis to connect to unix * Prevent deadlock during early shutdown of issue indexer * Add MaxWorker settings to queues * Merge branch 'master' into graceful-queues * Update modules/indexer/issues/indexer.go Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com> * Update modules/indexer/issues/indexer.go Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com> * Update modules/queue/queue_channel.go Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com> * Update modules/queue/queue_disk.go * Update modules/queue/queue_disk_channel.go Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com> * Rename queue.Description to queue.ManagedQueue as per @guillep2k * Cancel pool workers when removed * Remove dependency on queue from setting * Update modules/queue/queue_redis.go Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com> * As per @guillep2k add mutex locks on shutdown/terminate * move unlocking out of setInternal * Add warning if number of workers < 0 * Small changes as per @guillep2k * No redis host specified not found * Clean up documentation for queues * Update docs/content/doc/advanced/config-cheat-sheet.en-us.md * Update modules/indexer/issues/indexer_test.go * Ensure that persistable channel queue is added to manager * Rename QUEUE_NAME REDIS_QUEUE_NAME * Revert "Rename QUEUE_NAME REDIS_QUEUE_NAME" This reverts commit 1f83b4fc9b9dabda186257b38c265fe7012f90df. Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com> Co-authored-by: Lauris BH <lauris@nix.lv> Co-authored-by: techknowlogick <matti@mdranta.net> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
generated by cgit v1.2.3 (git 2.39.1) at 2025-01-15 23:47:04 +0000