summaryrefslogtreecommitdiffstats
path: root/modules/indexer/issues
Commit message (Collapse)AuthorAgeFilesLines
* Replace `interface{}` with `any` (#25686) (#25687)silverwind2023-07-043-6/+6
| | | | Same perl replacement as https://github.com/go-gitea/gitea/pull/25686 but for 1.20 to ease future backporting.
* Improve queue and logger context (#24924)wxiaoguang2023-05-261-39/+32
| | | | | | | | | | | | | | | | | | | | | | | | Before there was a "graceful function": RunWithShutdownFns, it's mainly for some modules which doesn't support context. The old queue system doesn't work well with context, so the old queues need it. After the queue refactoring, the new queue works with context well, so, use Golang context as much as possible, the `RunWithShutdownFns` could be removed (replaced by RunWithCancel for context cancel mechanism), the related code could be simplified. This PR also fixes some legacy queue-init problems, eg: * typo : archiver: "unable to create codes indexer queue" => "unable to create repo-archive queue" * no nil check for failed queues, which causes unfriendly panic After this PR, many goroutines could have better display name: ![image](https://github.com/go-gitea/gitea/assets/2114189/701b2a9b-8065-4137-aeaa-0bda2b34604a) ![image](https://github.com/go-gitea/gitea/assets/2114189/f1d5f50f-0534-40f0-b0be-f2c9daa5fe92)
* Rewrite logger system (#24726)wxiaoguang2023-05-211-17/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ## ⚠️ Breaking The `log.<mode>.<logger>` style config has been dropped. If you used it, please check the new config manual & app.example.ini to make your instance output logs as expected. Although many legacy options still work, it's encouraged to upgrade to the new options. The SMTP logger is deleted because SMTP is not suitable to collect logs. If you have manually configured Gitea log options, please confirm the logger system works as expected after upgrading. ## Description Close #12082 and maybe more log-related issues, resolve some related FIXMEs in old code (which seems unfixable before) Just like rewriting queue #24505 : make code maintainable, clear legacy bugs, and add the ability to support more writers (eg: JSON, structured log) There is a new document (with examples): `logging-config.en-us.md` This PR is safer than the queue rewriting, because it's just for logging, it won't break other logic. ## The old problems The logging system is quite old and difficult to maintain: * Unclear concepts: Logger, NamedLogger, MultiChannelledLogger, SubLogger, EventLogger, WriterLogger etc * Some code is diffuclt to konw whether it is right: `log.DelNamedLogger("console")` vs `log.DelNamedLogger(log.DEFAULT)` vs `log.DelLogger("console")` * The old system heavily depends on ini config system, it's difficult to create new logger for different purpose, and it's very fragile. * The "color" trick is difficult to use and read, many colors are unnecessary, and in the future structured log could help * It's difficult to add other log formats, eg: JSON format * The log outputer doesn't have full control of its goroutine, it's difficult to make outputer have advanced behaviors * The logs could be lost in some cases: eg: no Fatal error when using CLI. * Config options are passed by JSON, which is quite fragile. * INI package makes the KEY in `[log]` section visible in `[log.sub1]` and `[log.sub1.subA]`, this behavior is quite fragile and would cause more unclear problems, and there is no strong requirement to support `log.<mode>.<logger>` syntax. ## The new design See `logger.go` for documents. ## Screenshot <details> ![image](https://github.com/go-gitea/gitea/assets/2114189/4462d713-ba39-41f5-bb08-de912e67e1ff) ![image](https://github.com/go-gitea/gitea/assets/2114189/b188035e-f691-428b-8b2d-ff7b2199b2f9) ![image](https://github.com/go-gitea/gitea/assets/2114189/132e9745-1c3b-4e00-9e0d-15eaea495dee) </details> ## TODO * [x] add some new tests * [x] fix some tests * [x] test some sub-commands (manually ....) --------- Co-authored-by: Jason Song <i@wolfogre.com> Co-authored-by: delvh <dev.lh@web.de> Co-authored-by: Giteabot <teabot@gitea.io>
* Some refactors for issues stats (#24793)Lunny Xiao2023-05-191-1/+1
| | | | | | | | This PR - [x] Move some functions from `issues.go` to `issue_stats.go` and `issue_label.go` - [x] Remove duplicated issue options `UserIssueStatsOption` to keep only one `IssuesOptions`
* Remove unnecessary code (#24610)Lunny Xiao2023-05-101-1/+1
| | | As title, remove unnecessary code.
* Rewrite queue (#24505)wxiaoguang2023-05-085-95/+30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | # ⚠️ Breaking Many deprecated queue config options are removed (actually, they should have been removed in 1.18/1.19). If you see the fatal message when starting Gitea: "Please update your app.ini to remove deprecated config options", please follow the error messages to remove these options from your app.ini. Example: ``` 2023/05/06 19:39:22 [E] Removed queue option: `[indexer].ISSUE_INDEXER_QUEUE_TYPE`. Use new options in `[queue.issue_indexer]` 2023/05/06 19:39:22 [E] Removed queue option: `[indexer].UPDATE_BUFFER_LEN`. Use new options in `[queue.issue_indexer]` 2023/05/06 19:39:22 [F] Please update your app.ini to remove deprecated config options ``` Many options in `[queue]` are are dropped, including: `WRAP_IF_NECESSARY`, `MAX_ATTEMPTS`, `TIMEOUT`, `WORKERS`, `BLOCK_TIMEOUT`, `BOOST_TIMEOUT`, `BOOST_WORKERS`, they can be removed from app.ini. # The problem The old queue package has some legacy problems: * complexity: I doubt few people could tell how it works. * maintainability: Too many channels and mutex/cond are mixed together, too many different structs/interfaces depends each other. * stability: due to the complexity & maintainability, sometimes there are strange bugs and difficult to debug, and some code doesn't have test (indeed some code is difficult to test because a lot of things are mixed together). * general applicability: although it is called "queue", its behavior is not a well-known queue. * scalability: it doesn't seem easy to make it work with a cluster without breaking its behaviors. It came from some very old code to "avoid breaking", however, its technical debt is too heavy now. It's a good time to introduce a better "queue" package. # The new queue package It keeps using old config and concept as much as possible. * It only contains two major kinds of concepts: * The "base queue": channel, levelqueue, redis * They have the same abstraction, the same interface, and they are tested by the same testing code. * The "WokerPoolQueue", it uses the "base queue" to provide "worker pool" function, calls the "handler" to process the data in the base queue. * The new code doesn't do "PushBack" * Think about a queue with many workers, the "PushBack" can't guarantee the order for re-queued unhandled items, so in new code it just does "normal push" * The new code doesn't do "pause/resume" * The "pause/resume" was designed to handle some handler's failure: eg: document indexer (elasticsearch) is down * If a queue is paused for long time, either the producers blocks or the new items are dropped. * The new code doesn't do such "pause/resume" trick, it's not a common queue's behavior and it doesn't help much. * If there are unhandled items, the "push" function just blocks for a few seconds and then re-queue them and retry. * The new code doesn't do "worker booster" * Gitea's queue's handlers are light functions, the cost is only the go-routine, so it doesn't make sense to "boost" them. * The new code only use "max worker number" to limit the concurrent workers. * The new "Push" never blocks forever * Instead of creating more and more blocking goroutines, return an error is more friendly to the server and to the end user. There are more details in code comments: eg: the "Flush" problem, the strange "code.index" hanging problem, the "immediate" queue problem. Almost ready for review. TODO: * [x] add some necessary comments during review * [x] add some more tests if necessary * [x] update documents and config options * [x] test max worker / active worker * [x] re-run the CI tasks to see whether any test is flaky * [x] improve the `handleOldLengthConfiguration` to provide more friendly messages * [x] fine tune default config values (eg: length?) ## Code coverage: ![image](https://user-images.githubusercontent.com/2114189/236620635-55576955-f95d-4810-b12f-879026a3afdf.png)
* Refactor config provider (#24245)Lunny Xiao2023-04-251-2/+1
| | | | | | | This PR introduces more abstract about `ConfigProvider` and hides more `ini` references. --------- Co-authored-by: delvh <dev.lh@web.de>
* Fix meilisearch not working when searching across multiple repositories (#24109)Brecht Van Lommel2023-04-141-2/+4
| | | | | | This would happen in the issue and pull request dashboards, while the per repository lists worked fine. Use OR instead of AND for repo IDs.
* Add meilisearch support (#23136)techknowlogick2023-03-282-1/+199
| | | | | Add meilisearch support Fixes #20665
* Refactor the setting to make unit test easier (#22405)Lunny Xiao2023-02-201-3/+3
| | | | | | | | | | | | | | | | | | | | | | Some bugs caused by less unit tests in fundamental packages. This PR refactor `setting` package so that create a unit test will be easier than before. - All `LoadFromXXX` files has been splited as two functions, one is `InitProviderFromXXX` and `LoadCommonSettings`. The first functions will only include the code to create or new a ini file. The second function will load common settings. - It also renames all functions in setting from `newXXXService` to `loadXXXSetting` or `loadXXXFrom` to make the function name less confusing. - Move `XORMLog` to `SQLLog` because it's a better name for that. Maybe we should finally move these `loadXXXSetting` into the `XXXInit` function? Any idea? --------- Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: delvh <dev.lh@web.de>
* Make issue and code search support camel case (#22829)sillyguodong2023-02-121-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | Fixes #22714 ### Changes: 1. Add a token filter which named "camelCase" between custom unicode token filter and "to_lower" token filter when add custom analyzer. ### Notice: If users want this feature to work, they should delete folder under {giteaPath}/data/indexers and restart application. Then application will create a new IndexMapping. ### Screenshots: ![image](https://user-images.githubusercontent.com/33891828/217715692-c18c41f2-57a1-4727-861c-470935c8e0c8.png) ### Others: I originally attempted to give users the ability to configure the "token_filters" in the "app.ini" file. But I found that if users does not strictly follow a right order to register "token_filters", they won't get the expected results. I think it is difficult to ask users to do this. So I finally give up this idea. --------- Co-authored-by: Jason Song <i@wolfogre.com> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Implement FSFE REUSE for golang files (#21840)flynnnnnnnnnn2022-11-276-12/+6
| | | | | | | | | Change all license headers to comply with REUSE specification. Fix #16132 Co-authored-by: flynnnnnnnnnn <flynnnnnnnnnn@github> Co-authored-by: John Olheiser <john.olheiser@gmail.com>
* Add `context.Context` to more methods (#21546)KN4CK3R2022-11-191-9/+9
| | | | | | | This PR adds a context parameter to a bunch of methods. Some helper `xxxCtx()` methods got replaced with the normal name now. Co-authored-by: delvh <dev.lh@web.de> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Replace all instances of fmt.Errorf(%v) with fmt.Errorf(%w) (#21551)delvh2022-10-241-1/+1
| | | | | | | | | Found using `find . -type f -name '*.go' -print -exec vim {} -c ':%s/fmt\.Errorf(\(.*\)%v\(.*\)err/fmt.Errorf(\1%w\2err/g' -c ':wq' \;` Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: Andrew Thornton <art27@cantab.net> Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
* test: use `T.TempDir` to create temporary test directory (#21043)Eng Zer Jun2022-09-042-19/+3
| | | | | | | | | | | | | | | | | | | | | | | | A testing cleanup. This pull request replaces `os.MkdirTemp` with `t.TempDir`. We can use the `T.TempDir` function from the `testing` package to create temporary directory. The directory created by `T.TempDir` is automatically removed when the test and all its subtests complete. This saves us at least 2 lines (error check, and cleanup) on every instance, or in some cases adds cleanup that we forgot. Reference: https://pkg.go.dev/testing#T.TempDir ```go func TestFoo(t *testing.T) { // before tmpDir, err := os.MkdirTemp("", "") require.NoError(t, err) defer os.RemoveAll(tmpDir) // now tmpDir := t.TempDir() } ``` Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
* Move issues related files into models/issues (#19931)Lunny Xiao2022-06-132-8/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | * Move access and repo permission to models/perm/access * fix test * fix git test * Move functions sequence * Some improvements per @KN4CK3R and @delvh * Move issues related code to models/issues * Move some issues related sub package * Merge * Fix test * Fix test * Fix test * Fix test * Rename some files
* fix: some typos (#19956)yutotnh2022-06-131-1/+1
|
* Move some repository related code into sub package (#19711)Lunny Xiao2022-06-061-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Move some repository related code into sub package * Move more repository functions out of models * Fix lint * Some performance optimization for webhooks and others * some refactors * Fix lint * Fix * Update modules/repository/delete.go Co-authored-by: delvh <dev.lh@web.de> * Fix test * Merge * Fix test * Fix test * Fix test * Fix test Co-authored-by: delvh <dev.lh@web.de>
* Move almost all functions' parameter db.Engine to context.Context (#19748)Lunny Xiao2022-05-201-1/+1
| | | | * Move almost all functions' parameter db.Engine to context.Context * remove some unnecessary wrap functions
* Prevent dangling archiver goroutine (#19516)zeripath2022-04-261-1/+1
| | | | | | | | | | | | | | | | | | | | | | Within doArchive there is a service goroutine that performs the archiving function. This goroutine reports its error using a `chan error` called `done`. Prior to this PR this channel had 0 capacity meaning that the goroutine would block until the `done` channel was cleared - however there are a couple of ways in which this channel might not be read. The simplest solution is to add a single space of capacity to the goroutine which will mean that the goroutine will always complete and even if the `done` channel is not read it will be simply garbage collected away. (The PR also contains two other places when setting up the indexers which do not leak but where the blocking of the sending goroutine is also unnecessary and so we should just add a small amount of capacity and let the sending goroutine complete as soon as it can.) Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: 6543 <6543@obermui.de>
* User specific repoID or xorm builder conditions for issue search (#19475)65432022-04-251-1/+1
| | | | | | | | | * extend models.IssuesOptions to have more specific repo filter options * use new options * unrelated refactor * rm RepoIDs
* Use a struct as test options (#19393)Lunny Xiao2022-04-141-1/+3
| | | | | | | * Use a struct as test options * Fix name * Fix test
* Add Goroutine stack inspector to admin/monitor (#19207)zeripath2022-03-311-0/+12
| | | | | | | | | | | | | | | | | | | Continues on from #19202. Following the addition of pprof labels we can now more easily understand the relationship between a goroutine and the requests that spawn them. This PR takes advantage of the labels and adds a few others, then provides a mechanism for the monitoring page to query the pprof goroutine profile. The binary profile that results from this profile is immediately piped in to the google library for parsing this and then stack traces are formed for the goroutines. If the goroutine is within a context or has been created from a goroutine within a process context it will acquire the process description labels for that process. The goroutines are mapped with there associate pids and any that do not have an associated pid are placed in a group at the bottom as unbound. In this way we should be able to more easily examine goroutines that have been stuck. A manager command `gitea manager processes` is also provided that can export the processes (with or without stacktraces) to the command line. Signed-off-by: Andrew Thornton <art27@cantab.net>
* Automatically pause queue if index service is unavailable (#15066)Lauris BH2022-01-276-41/+199
| | | | | | * Handle keyword search error when issue indexer service is not available * Implement automatic disabling and resume of code indexer queue
* Pause queues (#15928)zeripath2022-01-221-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Start adding mechanism to return unhandled data Signed-off-by: Andrew Thornton <art27@cantab.net> * Create pushback interface Signed-off-by: Andrew Thornton <art27@cantab.net> * Add Pausable interface to WorkerPool and Manager Signed-off-by: Andrew Thornton <art27@cantab.net> * Implement Pausable and PushBack for the bytefifos Signed-off-by: Andrew Thornton <art27@cantab.net> * Implement Pausable and Pushback for ChannelQueues and ChannelUniqueQueues Signed-off-by: Andrew Thornton <art27@cantab.net> * Wire in UI for pausing Signed-off-by: Andrew Thornton <art27@cantab.net> * add testcases and fix a few issues Signed-off-by: Andrew Thornton <art27@cantab.net> * fix build Signed-off-by: Andrew Thornton <art27@cantab.net> * prevent "race" in the test Signed-off-by: Andrew Thornton <art27@cantab.net> * fix jsoniter mismerge Signed-off-by: Andrew Thornton <art27@cantab.net> * fix conflicts Signed-off-by: Andrew Thornton <art27@cantab.net> * fix format Signed-off-by: Andrew Thornton <art27@cantab.net> * Add warnings for no worker configurations and prevent data-loss with redis/levelqueue Signed-off-by: Andrew Thornton <art27@cantab.net> * Use StopTimer Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: Lauris BH <lauris@nix.lv> Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: techknowlogick <techknowlogick@gitea.io> Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
* format with gofumpt (#18184)65432022-01-205-45/+37
| | | | | | | | | | | * gofumpt -w -l . * gofumpt -w -l -extra . * Add linter * manual fix * change make fmt
* Enable deprecation error for v1.17.0 (#18341)Gusted2022-01-202-4/+4
| | | Co-authored-by: Andrew Thornton <art27@cantab.net>
* Add missing `X-Total-Count` and fix some related bugs (#17968)qwerty2872021-12-151-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Add missing `X-Total-Count` and fix some related bugs Adds `X-Total-Count` header to APIs that return a list but doesn't have it yet. Fixed bugs: * not returned after reporting error (https://github.com/qwerty287/gitea/blob/39eb82446c6fe5da3d79124e1f701f3795625b69/routers/api/v1/user/star.go#L70) * crash with index out of bounds, API issue/issueSubscriptions I also found various endpoints that return lists but do not apply/support pagination yet: ``` /repos/{owner}/{repo}/issues/{index}/labels /repos/{owner}/{repo}/issues/comments/{id}/reactions /repos/{owner}/{repo}/branch_protections /repos/{owner}/{repo}/contents /repos/{owner}/{repo}/hooks/git /repos/{owner}/{repo}/issue_templates /repos/{owner}/{repo}/releases/{id}/assets /repos/{owner}/{repo}/reviewers /repos/{owner}/{repo}/teams /user/emails /users/{username}/heatmap ``` If this is not expected, an new issue should be opened. Closes #13043 * fmt * Update routers/api/v1/repo/issue_subscription.go Co-authored-by: KN4CK3R <admin@oldschoolhack.me> * Use FindAndCount Co-authored-by: KN4CK3R <admin@oldschoolhack.me> Co-authored-by: 6543 <6543@obermui.de>
* Move repository model into models/repo (#17933)Lunny Xiao2021-12-102-2/+4
| | | | | | | | | | | | | | | * Some refactors related repository model * Move more methods out of repository * Move repository into models/repo * Fix test * Fix test * some improvements * Remove unnecessary function
* Move user related model into models/user (#17781)Lunny Xiao2021-11-241-1/+1
| | | | | | | | | | | | | * Move user related model into models/user * Fix lint for windows * Fix windows lint * Fix windows lint * Move some tests in models * Merge
* A better go code formatter, and now `make fmt` can run in Windows (#17684)wxiaoguang2021-11-172-2/+2
| | | | * go build / format tools * re-format imports
* Decouple unit test code from business code (#17623)wxiaoguang2021-11-121-4/+4
|
* Move login related structs and functions to models/login (#17093)Lunny Xiao2021-09-241-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Move login related structs and functions to models/login * Fix test * Fix lint * Fix lint * Fix lint of windows * Fix lint * Fix test * Fix test * Only load necessary fixtures when preparing unit tests envs * Fix lint * Fix test * Fix test * Fix error log * Fix error log * Fix error log * remove unnecessary change * fix error log * merge main branch
* refactor: move from io/ioutil to io and os package (#17109)Eng Zer Jun2021-09-222-4/+4
| | | | | | | | | The io/ioutil package has been deprecated as of Go 1.16, see https://golang.org/doc/go1.16#ioutil. This commit replaces the existing io/ioutil functions with their new definitions in io and os packages. Signed-off-by: Eng Zer Jun <engzerjun@gmail.com> Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Move db related basic functions to models/db (#17075)Lunny Xiao2021-09-191-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | * Move db related basic functions to models/db * Fix lint * Fix lint * Fix test * Fix lint * Fix lint * revert unnecessary change * Fix test * Fix wrong replace string * Use *Context * Correct committer spelling and fix wrong replaced words Co-authored-by: zeripath <art27@cantab.net>
* Fix data race in bleve indexer (#16474)Lunny Xiao2021-07-181-2/+4
| | | * Fix data race in bleve indexer
* Multiple Queue improvements: LevelDB Wait on empty, shutdown empty shadow ↵zeripath2021-05-151-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | level queue, reduce goroutines etc (#15693) * move shutdownfns, terminatefns and hammerfns out of separate goroutines Coalesce the shutdownfns etc into a list of functions that get run at shutdown rather then have them run at goroutines blocked on selects. This may help reduce the background select/poll load in certain configurations. * The LevelDB queues can actually wait on empty instead of polling Slight refactor to cause leveldb queues to wait on empty instead of polling. * Shutdown the shadow level queue once it is empty * Remove bytefifo additional goroutine for readToChan as it can just be run in run * Remove additional removeWorkers goroutine for workers * Simplify the AtShutdown and AtTerminate functions and add Channel Flusher * Add shutdown flusher to CUQ * move persistable channel shutdown stuff to Shutdown Fn * Ensure that UPCQ has the correct config * handle shutdown during the flushing * reduce risk of race between zeroBoost and addWorkers * prevent double shutdown Signed-off-by: Andrew Thornton <art27@cantab.net>
* Upgrade blevesearch dependency to v2.0.1 (#14346)Lauris BH2021-01-172-9/+9
| | | | | | | * Upgrade blevesearch dependency to v2.0.1 * Update rupture to v1.0.0 * Fix test
* Sort issue search results by revelance (#14353)Lauris BH2021-01-163-3/+4
|
* Refactor Logger (#13294)zeripath2020-10-311-2/+2
| | | | | | Refactor Logger to make a logger interface and make it possible to wrap loggers for specific purposes. Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Remove hardcoded ES indexername (#12521)Wim2020-08-181-1/+1
|
* Re-attempt to delete temporary upload if the file is locked by another ↵zeripath2020-08-113-6/+7
| | | | | | | | | | | process (#12447) Replace all calls to os.Remove/os.RemoveAll by retrying util.Remove/util.RemoveAll and remove circular dependencies from util. Fix #12339 Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: silverwind <me@silverwind.io>
* Log the indexer path on failure (#11172)zeripath2020-04-221-3/+4
| | | | | | Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com> Co-authored-by: Lauris BH <lauris@nix.lv>
* Handle panic in indexer initialisation better (#10534)zeripath2020-02-281-0/+8
| | | | | * Handle panic in indexer initialisation better * as per @guillep2k
* Issue search support elasticsearch (#9428)Lunny Xiao2020-02-133-10/+254
| | | | | | | | | | | | * Issue search support elasticsearch * Fix lint * Add indexer name on app.ini * add a warnning on SearchIssuesByKeyword * improve code
* Stop hanging issue indexer initialisation from preventing shutdown (#10243)zeripath2020-02-121-17/+15
| | | | | Co-authored-by: Lauris BH <lauris@nix.lv> Co-authored-by: Antoine GIRARD <sapk@users.noreply.github.com>
* API add/generalize pagination (#9452)SpaWn2KiLl2020-01-241-2/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * paginate results * fixed deadlock * prevented breaking change * updated swagger * go fmt * fixed find topic * go mod tidy * go mod vendor with go1.13.5 * fixed repo find topics * fixed unit test * added Limit method to Engine struct; use engine variable when provided; fixed gitignore * use ItemsPerPage for default pagesize; fix GetWatchers, getOrgUsersByOrgID and GetStargazers; fix GetAllCommits headers; reverted some changed behaviors * set Page value on Home route * improved memory allocations * fixed response headers * removed logfiles * fixed import order * import order * improved swagger * added function to get models.ListOptions from context * removed pagesize diff on unit test * fixed imports * removed unnecessary struct field * fixed go fmt * scoped PR * code improvements * code improvements * go mod tidy * fixed import order * fixed commit statuses session * fixed files headers * fixed headers; added pagination for notifications * go mod tidy * go fmt * removed Private from user search options; added setting.UI.IssuePagingNum as default valeu on repo's issues list * Apply suggestions from code review Co-Authored-By: 6543 <6543@obermui.de> Co-Authored-By: zeripath <art27@cantab.net> * fixed build error * CI.restart() * fixed merge conflicts resolve * fixed conflicts resolve * improved FindTrackedTimesOptions.ToOptions() method * added backwards compatibility on ListReleases request; fixed issue tracked time ToSession * fixed build error; fixed swagger template * fixed swagger template * fixed ListReleases backwards compatibility * added page to user search route Co-authored-by: techknowlogick <matti@mdranta.net> Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: zeripath <art27@cantab.net>
* Add "Update Branch" button to Pull Requests (#9784)65432020-01-171-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * add Divergence * add Update Button * first working version * re-use code * split raw merge commands and db-change functions (notify, cache, ...) * use rawMerge (remove redundant code) * own function to get Diverging of PRs * use FlashError * correct Error Msg * hook is triggerd ... so remove comment * add "branch2" to "user2/repo1" because it unit-test "TestPullView_ReviewerMissed" use it but dont exist jet :/ * move GetPerm to IsUserAllowedToUpdate * add Flash Success MSG * imprufe code - remove useless js chage * fix-lint * TEST: add PullRequest ID:5 Repo: user2/repo1 Base: branch1 Head: pr-to-update * correct comments * make PR5 outdated * fix Tests * WIP: add pull update test * update revs * update locales * working TEST * update UI * misspell * change style * add 1s delay so rev exist * move row up (before merge row) * fix lint nit * UI remove divider * Update style * nits * do it right * introduce IsSameRepo * remove useless check Co-authored-by: Lauris BH <lauris@nix.lv>
* Graceful Queues: Issue Indexing and Tasks (#9363)zeripath2020-01-077-411/+133
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Queue: Add generic graceful queues with settings * Queue & Setting: Add worker pool implementation * Queue: Add worker settings * Queue: Make resizing worker pools * Queue: Add name variable to queues * Queue: Add monitoring * Queue: Improve logging * Issues: Gracefulise the issues indexer Remove the old now unused specific queues * Task: Move to generic queue and gracefulise * Issues: Standardise the issues indexer queue settings * Fix test * Queue: Allow Redis to connect to unix * Prevent deadlock during early shutdown of issue indexer * Add MaxWorker settings to queues * Merge branch 'master' into graceful-queues * Update modules/indexer/issues/indexer.go Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com> * Update modules/indexer/issues/indexer.go Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com> * Update modules/queue/queue_channel.go Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com> * Update modules/queue/queue_disk.go * Update modules/queue/queue_disk_channel.go Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com> * Rename queue.Description to queue.ManagedQueue as per @guillep2k * Cancel pool workers when removed * Remove dependency on queue from setting * Update modules/queue/queue_redis.go Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com> * As per @guillep2k add mutex locks on shutdown/terminate * move unlocking out of setInternal * Add warning if number of workers < 0 * Small changes as per @guillep2k * No redis host specified not found * Clean up documentation for queues * Update docs/content/doc/advanced/config-cheat-sheet.en-us.md * Update modules/indexer/issues/indexer_test.go * Ensure that persistable channel queue is added to manager * Rename QUEUE_NAME REDIS_QUEUE_NAME * Revert "Rename QUEUE_NAME REDIS_QUEUE_NAME" This reverts commit 1f83b4fc9b9dabda186257b38c265fe7012f90df. Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com> Co-authored-by: Lauris BH <lauris@nix.lv> Co-authored-by: techknowlogick <matti@mdranta.net> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Stop various tests from adding to the source tree (#9515)zeripath2019-12-283-6/+47
| | | | | | | Instead of just adding test generated files to .gitignore prevent them from being produced in the first place. Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
generated by cgit v1.2.3 (git 2.39.1) at 2024-12-27 06:30:29 +0000