summaryrefslogtreecommitdiffstats
path: root/modules/indexer
Commit message (Collapse)AuthorAgeFilesLines
* Improve queue and logger context (#24924)wxiaoguang2023-05-263-47/+38
| | | | | | | | | | | | | | | | | | | | | | | | Before there was a "graceful function": RunWithShutdownFns, it's mainly for some modules which doesn't support context. The old queue system doesn't work well with context, so the old queues need it. After the queue refactoring, the new queue works with context well, so, use Golang context as much as possible, the `RunWithShutdownFns` could be removed (replaced by RunWithCancel for context cancel mechanism), the related code could be simplified. This PR also fixes some legacy queue-init problems, eg: * typo : archiver: "unable to create codes indexer queue" => "unable to create repo-archive queue" * no nil check for failed queues, which causes unfriendly panic After this PR, many goroutines could have better display name: ![image](https://github.com/go-gitea/gitea/assets/2114189/701b2a9b-8065-4137-aeaa-0bda2b34604a) ![image](https://github.com/go-gitea/gitea/assets/2114189/f1d5f50f-0534-40f0-b0be-f2c9daa5fe92)
* Allow skipping forks and mirrors from being indexed (#23187)techknowlogick2023-05-251-0/+27
| | | | | | This PR adds two new options to disable repo/code search indexing of both forks and mirrors. Related: #22842
* Rewrite logger system (#24726)wxiaoguang2023-05-212-33/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ## ⚠️ Breaking The `log.<mode>.<logger>` style config has been dropped. If you used it, please check the new config manual & app.example.ini to make your instance output logs as expected. Although many legacy options still work, it's encouraged to upgrade to the new options. The SMTP logger is deleted because SMTP is not suitable to collect logs. If you have manually configured Gitea log options, please confirm the logger system works as expected after upgrading. ## Description Close #12082 and maybe more log-related issues, resolve some related FIXMEs in old code (which seems unfixable before) Just like rewriting queue #24505 : make code maintainable, clear legacy bugs, and add the ability to support more writers (eg: JSON, structured log) There is a new document (with examples): `logging-config.en-us.md` This PR is safer than the queue rewriting, because it's just for logging, it won't break other logic. ## The old problems The logging system is quite old and difficult to maintain: * Unclear concepts: Logger, NamedLogger, MultiChannelledLogger, SubLogger, EventLogger, WriterLogger etc * Some code is diffuclt to konw whether it is right: `log.DelNamedLogger("console")` vs `log.DelNamedLogger(log.DEFAULT)` vs `log.DelLogger("console")` * The old system heavily depends on ini config system, it's difficult to create new logger for different purpose, and it's very fragile. * The "color" trick is difficult to use and read, many colors are unnecessary, and in the future structured log could help * It's difficult to add other log formats, eg: JSON format * The log outputer doesn't have full control of its goroutine, it's difficult to make outputer have advanced behaviors * The logs could be lost in some cases: eg: no Fatal error when using CLI. * Config options are passed by JSON, which is quite fragile. * INI package makes the KEY in `[log]` section visible in `[log.sub1]` and `[log.sub1.subA]`, this behavior is quite fragile and would cause more unclear problems, and there is no strong requirement to support `log.<mode>.<logger>` syntax. ## The new design See `logger.go` for documents. ## Screenshot <details> ![image](https://github.com/go-gitea/gitea/assets/2114189/4462d713-ba39-41f5-bb08-de912e67e1ff) ![image](https://github.com/go-gitea/gitea/assets/2114189/b188035e-f691-428b-8b2d-ff7b2199b2f9) ![image](https://github.com/go-gitea/gitea/assets/2114189/132e9745-1c3b-4e00-9e0d-15eaea495dee) </details> ## TODO * [x] add some new tests * [x] fix some tests * [x] test some sub-commands (manually ....) --------- Co-authored-by: Jason Song <i@wolfogre.com> Co-authored-by: delvh <dev.lh@web.de> Co-authored-by: Giteabot <teabot@gitea.io>
* Some refactors for issues stats (#24793)Lunny Xiao2023-05-191-1/+1
| | | | | | | | This PR - [x] Move some functions from `issues.go` to `issue_stats.go` and `issue_label.go` - [x] Remove duplicated issue options `UserIssueStatsOption` to keep only one `IssuesOptions`
* Remove unnecessary code (#24610)Lunny Xiao2023-05-102-2/+2
| | | As title, remove unnecessary code.
* Rewrite queue (#24505)wxiaoguang2023-05-0811-162/+65
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | # ⚠️ Breaking Many deprecated queue config options are removed (actually, they should have been removed in 1.18/1.19). If you see the fatal message when starting Gitea: "Please update your app.ini to remove deprecated config options", please follow the error messages to remove these options from your app.ini. Example: ``` 2023/05/06 19:39:22 [E] Removed queue option: `[indexer].ISSUE_INDEXER_QUEUE_TYPE`. Use new options in `[queue.issue_indexer]` 2023/05/06 19:39:22 [E] Removed queue option: `[indexer].UPDATE_BUFFER_LEN`. Use new options in `[queue.issue_indexer]` 2023/05/06 19:39:22 [F] Please update your app.ini to remove deprecated config options ``` Many options in `[queue]` are are dropped, including: `WRAP_IF_NECESSARY`, `MAX_ATTEMPTS`, `TIMEOUT`, `WORKERS`, `BLOCK_TIMEOUT`, `BOOST_TIMEOUT`, `BOOST_WORKERS`, they can be removed from app.ini. # The problem The old queue package has some legacy problems: * complexity: I doubt few people could tell how it works. * maintainability: Too many channels and mutex/cond are mixed together, too many different structs/interfaces depends each other. * stability: due to the complexity & maintainability, sometimes there are strange bugs and difficult to debug, and some code doesn't have test (indeed some code is difficult to test because a lot of things are mixed together). * general applicability: although it is called "queue", its behavior is not a well-known queue. * scalability: it doesn't seem easy to make it work with a cluster without breaking its behaviors. It came from some very old code to "avoid breaking", however, its technical debt is too heavy now. It's a good time to introduce a better "queue" package. # The new queue package It keeps using old config and concept as much as possible. * It only contains two major kinds of concepts: * The "base queue": channel, levelqueue, redis * They have the same abstraction, the same interface, and they are tested by the same testing code. * The "WokerPoolQueue", it uses the "base queue" to provide "worker pool" function, calls the "handler" to process the data in the base queue. * The new code doesn't do "PushBack" * Think about a queue with many workers, the "PushBack" can't guarantee the order for re-queued unhandled items, so in new code it just does "normal push" * The new code doesn't do "pause/resume" * The "pause/resume" was designed to handle some handler's failure: eg: document indexer (elasticsearch) is down * If a queue is paused for long time, either the producers blocks or the new items are dropped. * The new code doesn't do such "pause/resume" trick, it's not a common queue's behavior and it doesn't help much. * If there are unhandled items, the "push" function just blocks for a few seconds and then re-queue them and retry. * The new code doesn't do "worker booster" * Gitea's queue's handlers are light functions, the cost is only the go-routine, so it doesn't make sense to "boost" them. * The new code only use "max worker number" to limit the concurrent workers. * The new "Push" never blocks forever * Instead of creating more and more blocking goroutines, return an error is more friendly to the server and to the end user. There are more details in code comments: eg: the "Flush" problem, the strange "code.index" hanging problem, the "immediate" queue problem. Almost ready for review. TODO: * [x] add some necessary comments during review * [x] add some more tests if necessary * [x] update documents and config options * [x] test max worker / active worker * [x] re-run the CI tasks to see whether any test is flaky * [x] improve the `handleOldLengthConfiguration` to provide more friendly messages * [x] fine tune default config values (eg: length?) ## Code coverage: ![image](https://user-images.githubusercontent.com/2114189/236620635-55576955-f95d-4810-b12f-879026a3afdf.png)
* Refactor config provider (#24245)Lunny Xiao2023-04-252-4/+2
| | | | | | | This PR introduces more abstract about `ConfigProvider` and hides more `ini` references. --------- Co-authored-by: delvh <dev.lh@web.de>
* Use more specific test methods (#24265)KN4CK3R2023-04-221-1/+1
| | | | Co-authored-by: silverwind <me@silverwind.io> Co-authored-by: Giteabot <teabot@gitea.io>
* Allow adding new files to an empty repo (#24164)wxiaoguang2023-04-193-4/+12
| | | ![image](https://user-images.githubusercontent.com/2114189/232561612-2bfcfd0a-fc04-47ba-965f-5d0bcea46c54.png)
* Fix meilisearch not working when searching across multiple repositories (#24109)Brecht Van Lommel2023-04-141-2/+4
| | | | | | This would happen in the issue and pull request dashboards, while the per repository lists worked fine. Use OR instead of AND for repo IDs.
* Add meilisearch support (#23136)techknowlogick2023-03-282-1/+199
| | | | | Add meilisearch support Fixes #20665
* Refactor the setting to make unit test easier (#22405)Lunny Xiao2023-02-202-5/+5
| | | | | | | | | | | | | | | | | | | | | | Some bugs caused by less unit tests in fundamental packages. This PR refactor `setting` package so that create a unit test will be easier than before. - All `LoadFromXXX` files has been splited as two functions, one is `InitProviderFromXXX` and `LoadCommonSettings`. The first functions will only include the code to create or new a ini file. The second function will load common settings. - It also renames all functions in setting from `newXXXService` to `loadXXXSetting` or `loadXXXFrom` to make the function name less confusing. - Move `XORMLog` to `SQLLog` because it's a better name for that. Maybe we should finally move these `loadXXXSetting` into the `XXXInit` function? Any idea? --------- Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: delvh <dev.lh@web.de>
* Make issue and code search support camel case (#22829)sillyguodong2023-02-122-4/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | Fixes #22714 ### Changes: 1. Add a token filter which named "camelCase" between custom unicode token filter and "to_lower" token filter when add custom analyzer. ### Notice: If users want this feature to work, they should delete folder under {giteaPath}/data/indexers and restart application. Then application will create a new IndexMapping. ### Screenshots: ![image](https://user-images.githubusercontent.com/33891828/217715692-c18c41f2-57a1-4727-861c-470935c8e0c8.png) ### Others: I originally attempted to give users the ability to configure the "token_filters" in the "app.ini" file. But I found that if users does not strictly follow a right order to register "token_filters", they won't get the expected results. I think it is difficult to ask users to do this. So I finally give up this idea. --------- Co-authored-by: Jason Song <i@wolfogre.com> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* refactor some functions to support ctx as first parameter (#21878)Lunny Xiao2022-12-033-3/+3
| | | | Co-authored-by: KN4CK3R <admin@oldschoolhack.me> Co-authored-by: Lauris BH <lauris@nix.lv>
* Implement FSFE REUSE for golang files (#21840)flynnnnnnnnnn2022-11-2720-40/+20
| | | | | | | | | Change all license headers to comply with REUSE specification. Fix #16132 Co-authored-by: flynnnnnnnnnn <flynnnnnnnnnn@github> Co-authored-by: John Olheiser <john.olheiser@gmail.com>
* Show syntax lexer name in file view/blame (#21814)silverwind2022-11-191-1/+4
| | | | | | | | | | | | | | | | | | | | | | Show which Chroma Lexer is used to highlight the file in the file header. It's useful for development to see what was detected, and I think it's not bad info to have for the user: <img width="233" alt="Screenshot 2022-11-14 at 22 31 16" src="https://user-images.githubusercontent.com/115237/201770854-44933dfc-70a4-487c-8457-1bb3cc43ea62.png"> <img width="226" alt="Screenshot 2022-11-14 at 22 36 06" src="https://user-images.githubusercontent.com/115237/201770856-9260ce6f-6c0f-442c-92b5-201e5b113188.png"> <img width="194" alt="Screenshot 2022-11-14 at 22 36 26" src="https://user-images.githubusercontent.com/115237/201770857-6f56591b-80ea-42cc-8ea5-21b9156c018b.png"> Also, I improved the way this header overflows on small screens: <img width="354" alt="Screenshot 2022-11-14 at 22 44 36" src="https://user-images.githubusercontent.com/115237/201774828-2ddbcde1-da15-403f-bf7a-6248449fa2c5.png"> Co-authored-by: delvh <dev.lh@web.de> Co-authored-by: Lauris BH <lauris@nix.lv> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com> Co-authored-by: John Olheiser <john.olheiser@gmail.com>
* Add `context.Context` to more methods (#21546)KN4CK3R2022-11-191-9/+9
| | | | | | | This PR adds a context parameter to a bunch of methods. Some helper `xxxCtx()` methods got replaced with the normal name now. Co-authored-by: delvh <dev.lh@web.de> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Replace all instances of fmt.Errorf(%v) with fmt.Errorf(%w) (#21551)delvh2022-10-243-3/+3
| | | | | | | | | Found using `find . -type f -name '*.go' -print -exec vim {} -c ':%s/fmt\.Errorf(\(.*\)%v\(.*\)err/fmt.Errorf(\1%w\2err/g' -c ':wq' \;` Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: Andrew Thornton <art27@cantab.net> Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
* Refactor git command arguments and make all arguments to be safe to be used ↵wxiaoguang2022-10-233-7/+7
| | | | | | | (#21535) Follow #21464 Make all git command arguments strictly safe. Most changes are one-to-one replacing, keep all existing logic.
* test: use `T.TempDir` to create temporary test directory (#21043)Eng Zer Jun2022-09-043-28/+4
| | | | | | | | | | | | | | | | | | | | | | | | A testing cleanup. This pull request replaces `os.MkdirTemp` with `t.TempDir`. We can use the `T.TempDir` function from the `testing` package to create temporary directory. The directory created by `T.TempDir` is automatically removed when the test and all its subtests complete. This saves us at least 2 lines (error check, and cleanup) on every instance, or in some cases adds cleanup that we forgot. Reference: https://pkg.go.dev/testing#T.TempDir ```go func TestFoo(t *testing.T) { // before tmpDir, err := os.MkdirTemp("", "") require.NoError(t, err) defer os.RemoveAll(tmpDir) // now tmpDir := t.TempDir() } ``` Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
* Fix `dump-repo` git init, fix wrong error type for NullDownloader (#20182)wxiaoguang2022-07-011-1/+1
| | | | | * Fix `dump-repo` git init * fix wrong error type for NullDownloader
* Add more linters to improve code readability (#19989)Wim2022-06-202-2/+2
| | | | | | | | | | Add nakedret, unconvert, wastedassign, stylecheck and nolintlint linters to improve code readability - nakedret - https://github.com/alexkohler/nakedret - nakedret is a Go static analysis tool to find naked returns in functions greater than a specified function length. - unconvert - https://github.com/mdempsky/unconvert - Remove unnecessary type conversions - wastedassign - https://github.com/sanposhiho/wastedassign - wastedassign finds wasted assignment statements. - notlintlint - Reports ill-formed or insufficient nolint directives - stylecheck - https://staticcheck.io/docs/checks/#ST - keep style consistent - excluded: [ST1003 - Poorly chosen identifier](https://staticcheck.io/docs/checks/#ST1003) and [ST1005 - Incorrectly formatted error string](https://staticcheck.io/docs/checks/#ST1005)
* Move issues related files into models/issues (#19931)Lunny Xiao2022-06-132-8/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | * Move access and repo permission to models/perm/access * fix test * fix git test * Move functions sequence * Some improvements per @KN4CK3R and @delvh * Move issues related code to models/issues * Move some issues related sub package * Merge * Fix test * Fix test * Fix test * Fix test * Rename some files
* fix: some typos (#19956)yutotnh2022-06-131-1/+1
|
* Fix data-race problems in git module (quick patch) (#19934)wxiaoguang2022-06-111-5/+0
| | | | | | | * Fix data-race problems in git module * use HomeDir instead of setting.RepoRootPath Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Refactor git module, make Gitea use internal git config (#19732)wxiaoguang2022-06-101-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Refactor git module, make Gitea use internal git config, add safe.directory config * introduce git.InitSimple and git.InitWithConfigSync, make serv cmd use gitconfig * use HOME instead of GIT_CONFIG_GLOBAL, because git always needs a correct HOME * fix cmd env in cmd/serv.go * fine tune error message * Fix a incorrect test case * fix configAddNonExist * fix configAddNonExist logic, add `--fixed-value` flag, add tests * add configSetNonExist function in case it's needed. * use configSetNonExist for `user.name` and `user.email` * add some comments * Update cmd/serv.go Co-authored-by: zeripath <art27@cantab.net> * Update cmd/serv.go Co-authored-by: zeripath <art27@cantab.net> * Update modules/git/git.go Co-authored-by: zeripath <art27@cantab.net> * Update modules/setting/setting.go Co-authored-by: zeripath <art27@cantab.net> * Update modules/git/repo_attribute.go Co-authored-by: zeripath <art27@cantab.net> * fix spaces in messages * use `configSet("core.protectNTFS", ...)` instead of `globalCommandArgs` * remove GIT_CONFIG_NOSYSTEM, continue to use system's git config * Update cmd/serv.go Co-authored-by: zeripath <art27@cantab.net> * fix merge * remove code for safe.directory * separate git.CommonEnvs to CommonGitCmdEnvs and CommonCmdServEnvs * avoid Golang's data race error Co-authored-by: zeripath <art27@cantab.net> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Move some repository related code into sub package (#19711)Lunny Xiao2022-06-061-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Move some repository related code into sub package * Move more repository functions out of models * Fix lint * Some performance optimization for webhooks and others * some refactors * Fix lint * Fix * Update modules/repository/delete.go Co-authored-by: delvh <dev.lh@web.de> * Fix test * Merge * Fix test * Fix test * Fix test * Fix test Co-authored-by: delvh <dev.lh@web.de>
* Move almost all functions' parameter db.Engine to context.Context (#19748)Lunny Xiao2022-05-205-5/+6
| | | | * Move almost all functions' parameter db.Engine to context.Context * remove some unnecessary wrap functions
* Prevent dangling archiver goroutine (#19516)zeripath2022-04-262-2/+2
| | | | | | | | | | | | | | | | | | | | | | Within doArchive there is a service goroutine that performs the archiving function. This goroutine reports its error using a `chan error` called `done`. Prior to this PR this channel had 0 capacity meaning that the goroutine would block until the `done` channel was cleared - however there are a couple of ways in which this channel might not be read. The simplest solution is to add a single space of capacity to the goroutine which will mean that the goroutine will always complete and even if the `done` channel is not read it will be simply garbage collected away. (The PR also contains two other places when setting up the indexers which do not leak but where the blocking of the sending goroutine is also unnecessary and so we should just add a small amount of capacity and let the sending goroutine complete as soon as it can.) Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: 6543 <6543@obermui.de>
* User specific repoID or xorm builder conditions for issue search (#19475)65432022-04-251-1/+1
| | | | | | | | | * extend models.IssuesOptions to have more specific repo filter options * use new options * unrelated refactor * rm RepoIDs
* Use a struct as test options (#19393)Lunny Xiao2022-04-143-3/+9
| | | | | | | * Use a struct as test options * Fix name * Fix test
* Remove `git.Command.Run` and `git.Command.RunInDir*` (#19280)wxiaoguang2022-04-013-19/+22
| | | | | | Follows #19266, #8553, Close #18553, now there are only three `Run..(&RunOpts{})` functions. * before: `stdout, err := RunInDir(path)` * now: `stdout, _, err := RunStdString(&git.RunOpts{Dir:path})`
* Add Goroutine stack inspector to admin/monitor (#19207)zeripath2022-03-312-1/+18
| | | | | | | | | | | | | | | | | | | Continues on from #19202. Following the addition of pprof labels we can now more easily understand the relationship between a goroutine and the requests that spawn them. This PR takes advantage of the labels and adds a few others, then provides a mechanism for the monitoring page to query the pprof goroutine profile. The binary profile that results from this profile is immediately piped in to the google library for parsing this and then stack traces are formed for the goroutines. If the goroutine is within a context or has been created from a goroutine within a process context it will acquire the process description labels for that process. The goroutines are mapped with there associate pids and any that do not have an associated pid are placed in a group at the bottom as unbound. In this way we should be able to more easily examine goroutines that have been stuck. A manager command `gitea manager processes` is also provided that can export the processes (with or without stacktraces) to the command line. Signed-off-by: Andrew Thornton <art27@cantab.net>
* Make git.OpenRepository accept Context (#19260)65432022-03-301-1/+1
| | | | | * OpenRepositoryCtx -> OpenRepository * OpenRepository -> openRepositoryWithDefaultContext, only for internal usage
* Prevent intermittent failures in RepoIndexerTest (2) (#19229)zeripath2022-03-271-0/+5
| | | | | | | | | | | So whilst #19225 fixes one issue it caused another. We need to initialise the Git module first. Related #19225 Fix #19162 Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: 6543 <6543@obermui.de>
* Prevent intermittent failures in RepoIndexerTest (#19225)zeripath2022-03-271-2/+8
| | | | | | | | | | | The RepoIndexerTest is failing with considerable frequency due to a race inherrent in its design. This PR adjust this test to avoid the reliance on waiting for the populate repo indexer to run and forcibly adds the repo to the queue. It then flushes the queue. It may be worth separating out the tests somewhat by testing the Index function directly away from the queue however, this forceful method should solve the current problem. Signed-off-by: Andrew Thornton <art27@cantab.net>
* Prevent Stats Indexer reporting error if repo dir missing (#18870)zeripath2022-02-241-0/+3
| | | | | | | | | | Repositories missing their directory should not report an error from the stats indexer. Close #18847 Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Lock gofumpt to v0.3.0 and run it (#18866)silverwind2022-02-231-1/+2
| | | | | We can't depend on `latest` version of gofumpt because the output will not be stable across versions. Lock it down to the latest version released yesterday and run it again.
* Delete old git.NewCommand() and use it as git.NewCommandContext() (#18552)65432022-02-063-6/+6
|
* Automatically pause queue if index service is unavailable (#15066)Lauris BH2022-01-2712-74/+379
| | | | | | * Handle keyword search error when issue indexer service is not available * Implement automatic disabling and resume of code indexer queue
* Pause queues (#15928)zeripath2022-01-223-5/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Start adding mechanism to return unhandled data Signed-off-by: Andrew Thornton <art27@cantab.net> * Create pushback interface Signed-off-by: Andrew Thornton <art27@cantab.net> * Add Pausable interface to WorkerPool and Manager Signed-off-by: Andrew Thornton <art27@cantab.net> * Implement Pausable and PushBack for the bytefifos Signed-off-by: Andrew Thornton <art27@cantab.net> * Implement Pausable and Pushback for ChannelQueues and ChannelUniqueQueues Signed-off-by: Andrew Thornton <art27@cantab.net> * Wire in UI for pausing Signed-off-by: Andrew Thornton <art27@cantab.net> * add testcases and fix a few issues Signed-off-by: Andrew Thornton <art27@cantab.net> * fix build Signed-off-by: Andrew Thornton <art27@cantab.net> * prevent "race" in the test Signed-off-by: Andrew Thornton <art27@cantab.net> * fix jsoniter mismerge Signed-off-by: Andrew Thornton <art27@cantab.net> * fix conflicts Signed-off-by: Andrew Thornton <art27@cantab.net> * fix format Signed-off-by: Andrew Thornton <art27@cantab.net> * Add warnings for no worker configurations and prevent data-loss with redis/levelqueue Signed-off-by: Andrew Thornton <art27@cantab.net> * Use StopTimer Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: Lauris BH <lauris@nix.lv> Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: techknowlogick <techknowlogick@gitea.io> Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
* format with gofumpt (#18184)65432022-01-2012-107/+89
| | | | | | | | | | | * gofumpt -w -l . * gofumpt -w -l -extra . * Add linter * manual fix * change make fmt
* Enable deprecation error for v1.17.0 (#18341)Gusted2022-01-202-4/+4
| | | Co-authored-by: Andrew Thornton <art27@cantab.net>
* Propagate context and ensure git commands run in request context (#17868)zeripath2022-01-196-31/+35
| | | | | | | | | This PR continues the work in #17125 by progressively ensuring that git commands run within the request context. This now means that the if there is a git repo already open in the context it will be used instead of reopening it. Signed-off-by: Andrew Thornton <art27@cantab.net>
* Upgrade bleve from v2.0.6 to v2.3.0 (#18132)Lunny Xiao2022-01-011-1/+1
|
* Prevent hang in git cat-file if repository is not a valid repository and ↵zeripath2021-12-162-0/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | other fixes (#17991) This PR contains multiple fixes. The most important of which is: * Prevent hang in git cat-file if the repository is not a valid repository Unfortunately it appears that if git cat-file is run in an invalid repository it will hang until stdin is closed. This will result in deadlocked /pulls pages and dangling git cat-file calls if a broken repository is tried to be reviewed or pulls exists for a broken repository. Fix #14734 Fix #9271 Fix #16113 Otherwise there are a few small other fixes included which this PR was initially intending to fix: * Fix panic on partial compares due to missing PullRequestWorkInProgressPrefixes * Fix links on pulls pages due to regression from #17551 - by making most /issues routes match /pulls too - Fix #17983 * Fix links on feeds pages due to another regression from #17551 but also fix issue with syncing tags - Fix #17943 * Add missing locale entries for oauth group claims * Prevent NPEs if ColorFormat is called on nil users, repos or teams.
* Add missing `X-Total-Count` and fix some related bugs (#17968)qwerty2872021-12-153-3/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Add missing `X-Total-Count` and fix some related bugs Adds `X-Total-Count` header to APIs that return a list but doesn't have it yet. Fixed bugs: * not returned after reporting error (https://github.com/qwerty287/gitea/blob/39eb82446c6fe5da3d79124e1f701f3795625b69/routers/api/v1/user/star.go#L70) * crash with index out of bounds, API issue/issueSubscriptions I also found various endpoints that return lists but do not apply/support pagination yet: ``` /repos/{owner}/{repo}/issues/{index}/labels /repos/{owner}/{repo}/issues/comments/{id}/reactions /repos/{owner}/{repo}/branch_protections /repos/{owner}/{repo}/contents /repos/{owner}/{repo}/hooks/git /repos/{owner}/{repo}/issue_templates /repos/{owner}/{repo}/releases/{id}/assets /repos/{owner}/{repo}/reviewers /repos/{owner}/{repo}/teams /user/emails /users/{username}/heatmap ``` If this is not expected, an new issue should be opened. Closes #13043 * fmt * Update routers/api/v1/repo/issue_subscription.go Co-authored-by: KN4CK3R <admin@oldschoolhack.me> * Use FindAndCount Co-authored-by: KN4CK3R <admin@oldschoolhack.me> Co-authored-by: 6543 <6543@obermui.de>
* Move repository model into models/repo (#17933)Lunny Xiao2021-12-1012-37/+41
| | | | | | | | | | | | | | | * Some refactors related repository model * Move more methods out of repository * Move repository into models/repo * Fix test * Fix test * some improvements * Remove unnecessary function
* Make Requests Processes and create process hierarchy. Associate ↵zeripath2021-11-303-3/+10
| | | | | | | | | OpenRepository with context. (#17125) This PR registers requests with the process manager and manages hierarchy within the processes. Git repos are then associated with a context, (usually the request's context) - with sub commands using this context as their base context. Signed-off-by: Andrew Thornton <art27@cantab.net>
* Move user related model into models/user (#17781)Lunny Xiao2021-11-241-1/+1
| | | | | | | | | | | | | * Move user related model into models/user * Fix lint for windows * Fix windows lint * Fix windows lint * Move some tests in models * Merge
generated by cgit v1.2.3 (git 2.39.1) at 2025-03-11 22:41:08 +0000