summaryrefslogtreecommitdiffstats
path: root/services/repository
Commit message (Collapse)AuthorAgeFilesLines
* Fix loadOneBranch panic (#29938) (#29939)wxiaoguang2024-03-201-4/+6
| | | | | | | Backport #29938 Try to fix #29936 Far from ideal, but still better than panic.
* Add cache for dashbaord commit status (#29932)Lunny Xiao2024-03-202-44/+138
| | | backport #29444
* Do not use lower tag names to find releases/tags (#29261) (#29262)Jason Song2024-02-191-6/+6
| | | | | | | | | | | | | | | | Backport #29261. Fix #26090, see https://github.com/go-gitea/gitea/issues/26090#issuecomment-1952013206 Since `TagName` stores the original tag name and `LowerTagName` stores the lower tag name, it doesn't make sense to use lowercase tags as `TagNames` in `FindReleasesOptions`. https://github.com/go-gitea/gitea/blob/5e72526da4e915791f03af056890e16821bde052/services/repository/push.go#L396-L397 While the only other usage looks correct: https://github.com/go-gitea/gitea/blob/5e72526da4e915791f03af056890e16821bde052/routers/web/repo/repo.go#L416
* Fix an actions schedule bug (#28942) (#28999)Giteabot2024-01-311-0/+10
| | | | | | | | | | | | | | | Backport #28942 by @Zettat123 In #28691, schedule plans will be deleted when a repo's actions unit is disabled. But when the unit is enabled, the schedule plans won't be created again. This PR fixes the bug. The schedule plans will be created again when the actions unit is re-enabled --------- Co-authored-by: Zettat123 <zettat123@gmail.com> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Fix `DeleteCollaboration` transaction behaviour (#28886) (#28889)Giteabot2024-01-221-2/+5
| | | | | | | | | | | | | Backport #28886 by @KN4CK3R The method can't be called with an outer transaction because if the user is not a collaborator the outer transaction will be rolled back even if the inner transaction uses the no-error path. `has == 0` leads to `return nil` which cancels the transaction. A standalone call of this method does nothing but if used with an outer transaction, that will be canceled. Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
* Fix reverting a merge commit failing (#28794) (#28825)Mihir Joshi2024-01-216-7/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Backport https://github.com/go-gitea/gitea/pull/28794 Fixes #22236 --- Error occurring currently while trying to revert commit using read-tree -m approach: > 2022/12/26 16:04:43 ...rvices/pull/patch.go:240:AttemptThreeWayMerge() [E] [63a9c61a] Unable to run read-tree -m! Error: exit status 128 - fatal: this operation must be run in a work tree > - fatal: this operation must be run in a work tree We need to clone a non-bare repository for `git read-tree -m` to work. https://github.com/go-gitea/gitea/commit/bb371aee6ecf5e570cdf7b5f7f0d6f47a607a325 adds support to create a non-bare cloned temporary upload repository. After cloning a non-bare temporary upload repository, we [set default index](https://github.com/go-gitea/gitea/blob/main/services/repository/files/cherry_pick.go#L37) (`git read-tree HEAD`). This operation ends up resetting the git index file (see investigation details below), due to which, we need to call `git update-index --refresh` afterward. Here's the diff of the index file before and after we execute SetDefaultIndex: https://www.diffchecker.com/hyOP3eJy/ Notice the **ctime**, **mtime** are set to 0 after SetDefaultIndex. You can reproduce the same behavior using these steps: ```bash $ git clone https://try.gitea.io/me-heer/test.git -s -b main $ cd test $ git read-tree HEAD $ git read-tree -m 1f085d7ed8 1f085d7ed8 9933caed00 error: Entry '1' not uptodate. Cannot merge. ``` After which, we can fix like this: ```bash $ git update-index --refresh $ git read-tree -m 1f085d7ed8 1f085d7ed8 9933caed00 ```
* Fix schedule tasks bugs (#28691) (#28780)Lunny Xiao2024-01-142-1/+112
| | | | | | | | | | | | | | Fix #28157 Backport #28691 This PR fix the possible bugs about actions schedule. - Move `UpdateRepositoryUnit` and `SetRepoDefaultBranch` from models to service layer - Remove schedules plan from database and cancel waiting & running schedules tasks in this repository when actions unit has been disabled or global disabled. - Remove schedules plan from database and cancel waiting & running schedules tasks in this repository when default branch changed.
* Remove unnecessary syncbranchToDB with tests (#28624) (#28629)Lunny Xiao2023-12-291-20/+9
| | | | | | | | | | Replace #28625 Backport #28624 by lunny #28361 introduced `syncBranchToDB` in `CreateNewBranchFromCommit`. This PR will revert the change because it's unnecessary. Every push will already be checked by `syncBranchToDB`. This PR also created a test to ensure it's right.
* Also sync DB branches on push if necessary (#28361) (#28403)Lunny Xiao2023-12-112-32/+72
| | | | | | | | | | | | | | | | | | | | | | | | | | | Fix #28056 Backport #28361 This PR will check whether the repo has zero branch when pushing a branch. If that, it means this repository hasn't been synced. The reason caused that is after user upgrade from v1.20 -> v1.21, he just push branches without visit the repository user interface. Because all repositories routers will check whether a branches sync is necessary but push has not such check. For every repository, it has two states, synced or not synced. If there is zero branch for a repository, then it will be assumed as non-sync state. Otherwise, it's synced state. So if we think it's synced, we just need to update branch/insert new branch. Otherwise do a full sync. So that, for every push, there will be almost no extra load added. It's high performance than yours. For the implementation, we in fact will try to update the branch first, if updated success with affect records > 0, then all are done. Because that means the branch has been in the database. If no record is affected, that means the branch does not exist in database. So there are two possibilities. One is this is a new branch, then we just need to insert the record. Another is the branches haven't been synced, then we need to sync all the branches into database.
* Delete repos of org when purge delete user (#27273) (#27728)65432023-11-014-7/+45
| | | | | | | | | Fixes https://codeberg.org/forgejo/forgejo/issues/1514 Backports #27273 --------- Co-authored-by: JakobDev <jakobdev@gmx.de>
* Add doctor dbconsistency fix to delete repos with no owner (#27290) (#27693)65432023-10-221-6/+9
| | | Backport #27290
* Another round of `db.DefaultContext` refactor (#27103) (#27262)Giteabot2023-09-253-3/+3
| | | | | | | | Backport #27103 by @JakobDev Part of #27065 Co-authored-by: JakobDev <jakobdev@gmx.de> Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
* Search branches (#27055)Lunny Xiao2023-09-171-1/+2
| | | | | | | | | Resolve #25233 <img width="1315" alt="图片" src="https://github.com/go-gitea/gitea/assets/81045/3ba59b58-471a-4e1b-985c-87edac2268c0"> <img width="1297" alt="图片" src="https://github.com/go-gitea/gitea/assets/81045/b6caa12f-323b-4f70-9c44-ef91cb71a26c">
* Next round of `db.DefaultContext` refactor (#27089)JakobDev2023-09-166-12/+14
| | | Part of #27065
* More refactoring of `db.DefaultContext` (#27083)JakobDev2023-09-154-19/+18
| | | Next step of #27065
* Reduce usage of `db.DefaultContext` (#27073)JakobDev2023-09-141-2/+2
| | | | | | | | | | | | | | Part of #27065 This reduces the usage of `db.DefaultContext`. I think I've got enough files for the first PR. When this is merged, I will continue working on this. Considering how many files this PR affect, I hope it won't take to long to merge, so I don't end up in the merge conflict hell. --------- Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
* Replace 'userxx' with 'orgxx' in all test files when the user type is org ↵Nanguan Lin2023-09-141-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | (#27052) Currently 'userxx' and 'orgxx' are both used as username in test files when the user type is org, which is confusing. This PR replaces all 'userxx' with 'orgxx' when the user type is org(`user.type==1`). Some non-trivial changes 1. Rename `user3` dir to `org3` in `tests/git-repositories-meta` 2. Change `end` in `issue reference` because 'org3' is one char shorter than 'user3' ![ksnip_20230913-112819](https://github.com/go-gitea/gitea/assets/70063547/442988c5-4cf4-49b8-aa01-4dd6bf0ca954) 3. Change the search result number of `user/repo2` because `user3/repo21` can't be searched now ![ksnip_20230913-112931](https://github.com/go-gitea/gitea/assets/70063547/d9ebeba4-479f-4110-9a85-825efbc981fd) 4. Change the first org name getting from API because the result is ordered by alphabet asc and now `org 17` is before `org25` ![JW8U7NIO(J$H _YCRB36H)T](https://github.com/go-gitea/gitea/assets/70063547/f55a685c-cf24-40e5-a87f-3a2327319548) ![)KFD411O4I8RB5ZOH7E0 Z3](https://github.com/go-gitea/gitea/assets/70063547/a0dc3299-249c-46f6-91cb-d15d4ee88dd5) Other modifications are just find all and replace all. Unit tests with SQLite are all passed. --------- Co-authored-by: caicandong <1290147055@qq.com>
* Move some functions to service layer (#26969)Lunny Xiao2023-09-082-0/+75
|
* move repository deletion to service layer (#26948)Lunny Xiao2023-09-088-14/+484
| | | Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
* Move createrepository from module to service layer (#26927)Lunny Xiao2023-09-064-4/+467
| | | | Repository creation depends on many models, so moving it to service layer is better.
* Move notification interface to services layer (#26915)Lunny Xiao2023-09-058-26/+26
| | | Extract from #22266
* Move feed notification service layer (#26908)Lunny Xiao2023-09-051-2/+2
| | | Extract from #22266
* Move web/api context related testing function into a separate package (#26859)wxiaoguang2023-09-015-62/+62
| | | | | | | | | Just like `models/unittest`, the testing helper functions should be in a separate package: `contexttest` And complete the TODO: > // TODO: move this function to other packages, because it depends on "models" package
* Sync tags when adopting repos (#26816)Zettat1232023-08-311-0/+4
| | | | | | | | | | Fixes #26138 Sync the tags into database when adopting repos --------- Co-authored-by: silverwind <me@silverwind.io> Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
* Fix some slice append usages (#26778)Chongyi Zheng2023-08-291-3/+3
| | | Co-authored-by: delvh <dev.lh@web.de>
* Add link to job details and tooltip to commit status in repo list in ↵yp053272023-08-211-2/+3
| | | | | | | | | | | dashboard (#26326) Tooltip: ![image](https://github.com/go-gitea/gitea/assets/18380374/237cb545-7844-424b-b995-1008eaaaedec) Link to the target job: ![image](https://github.com/go-gitea/gitea/assets/18380374/0c11a97f-6517-47f2-8773-f381488c084e)
* Sync repo's IsEmpty status correctly (#26517)wxiaoguang2023-08-171-1/+3
| | | Close #26509
* Show branches and tags that contain a commit (#25180)delvh2023-07-271-0/+55
| | | | | | | | | | | | | | | | | | | | | | | Now, you can see for a commit which existing branches and tags contain it. You first have to click on the `load branches and tags` button, they are not preloaded by default. All branches and tags are ordered descending by creation date. You can even see without much hassle if the given commit is already part of the default branch. Closes #25152 ## Screenshots ### Initial ![image](https://github.com/go-gitea/gitea/assets/51889757/84db2c0b-aaef-4f69-ab92-0b812793d2ad) ### Loaded ![image](https://github.com/go-gitea/gitea/assets/51889757/a9b84e66-8e44-4c55-b017-c37f4a45f41b) --------- Co-authored-by: silverwind <me@silverwind.io> Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
* Fix bugs in LFS meta garbage collection (#26122)Zettat1232023-07-262-2/+67
| | | | | | | | This PR - Fix #26093. Replace `time.Time` with `timeutil.TimeStamp` - Fix #26135. Add missing `xorm:"extends"` to `CountLFSMetaObject` for LFS meta object query - Add a unit test for LFS meta object garbage collection
* Add context parameter to some database functions (#26055)Lunny Xiao2023-07-221-3/+3
| | | | | To avoid deadlock problem, almost database related functions should be have ctx as the first parameter. This PR do a refactor for some of these functions.
* Support copy protected branch from template repository (#25889)Lunny Xiao2023-07-211-0/+29
| | | Fix #14303
* Fix bug when pushing to a pull request which enabled dismiss approval ↵Lunny Xiao2023-07-201-2/+2
| | | | | | | | | | | | | | | automatically (#25882) Fix #25858 The option `dissmiss stale approvals` was listed on protected branch but never implemented. This PR fixes that. <img width="1006" alt="图片" src="https://github.com/go-gitea/gitea/assets/81045/60bfa968-4db7-4c24-b8be-2e5978f91bb9"> <img width="1021" alt="图片" src="https://github.com/go-gitea/gitea/assets/81045/8dabc14d-2dfe-40c2-94ed-24fcbf6e0e8f">
* Refactor "Content" for file uploading (#25851)wxiaoguang2023-07-181-11/+12
| | | | | | | | | | | | | | | Before: the concept "Content string" is used everywhere. It has some problems: 1. Sometimes it means "base64 encoded content", sometimes it means "raw binary content" 2. It doesn't work with large files, eg: uploading a 1G LFS file would make Gitea process OOM This PR does the refactoring: use "ContentReader" / "ContentBase64" instead of "Content" This PR is not breaking because the key in API JSON is still "content": `` ContentBase64 string `json:"content"` ``
* Do not "guess" the file encoding/BOM when using API to upload files (#25828)wxiaoguang2023-07-121-102/+3
| | | | | | | | Related issue: #18368 It doesn't seem right to "guess" the file encoding/BOM when using API to upload files. The API should save the uploaded content as-is.
* Simplify the LFS GC logger usage (#25717)wxiaoguang2023-07-061-21/+17
| | | | | | | | | Remove unnecessary `if opts.Logger != nil` checks. * For "CLI doctor" mode, output to the console's "logger.Info". * For "Web Task" mode, output to the default "logger.Debug", to avoid flooding the server's log in a busy production instance. Co-authored-by: Giteabot <teabot@gitea.io>
* Replace `interface{}` with `any` (#25686)silverwind2023-07-041-2/+2
| | | | | Result of running `perl -p -i -e 's#interface\{\}#any#g' **/*` and `make fmt`. Basically the same [as golang did](https://github.com/golang/go/commit/2580d0e08d5e9f979b943758d3c49877fb2324cb).
* Fix branch commit message too long problem (#25588)Lunny Xiao2023-06-301-1/+1
| | | | | | When branch's commit CommitMessage is too long, the column maybe too short.(TEXT 16K for mysql). This PR will fix it to only store the summary because these message will only show on branch list or possible future search?
* Sync branches into databases (#22743)Lunny Xiao2023-06-297-22/+258
| | | | | | | | | | | | | | | | | | | | | | | | | | Related #14180 Related #25233 Related #22639 Close #19786 Related #12763 This PR will change all the branches retrieve method from reading git data to read database to reduce git read operations. - [x] Sync git branches information into database when push git data - [x] Create a new table `Branch`, merge some columns of `DeletedBranch` into `Branch` table and drop the table `DeletedBranch`. - [x] Read `Branch` table when visit `code` -> `branch` page - [x] Read `Branch` table when list branch names in `code` page dropdown - [x] Read `Branch` table when list git ref compare page - [x] Provide a button in admin page to manually sync all branches. - [x] Sync branches if repository is not empty but database branches are empty when visiting pages with branches list - [x] Use `commit_time desc` as the default FindBranch order by to keep consistent as before and deleted branches will be always at the end. --------- Co-authored-by: Jason Song <i@wolfogre.com>
* Add Adopt repository event and handler (#25497)Lunny Xiao2023-06-261-11/+13
| | | | | | | Fix #14304 --------- Co-authored-by: delvh <dev.lh@web.de>
* Refactor web package and context package (#25298)wxiaoguang2023-06-185-12/+12
| | | | | | | | | | | | | 1. The "web" package shouldn't depends on "modules/context" package, instead, let each "web context" register themselves to the "web" package. 2. The old Init/Free doesn't make sense, so simplify it * The ctx in "Init(ctx)" is never used, and shouldn't be used that way * The "Free" is never called and shouldn't be called because the SSPI instance is shared --------- Co-authored-by: Giteabot <teabot@gitea.io>
* Fix profile render when the README.md size is larger than 1024 bytes (#25131)yp053272023-06-131-1/+1
| | | | | | | | | | | | | | | | | Fixes https://github.com/go-gitea/gitea/issues/25094 `GetBlobContent` will only get the first 1024 bytes, if the README.md size is larger than 1024 bytes, We can not render the rest of them. After this fix, we should provide the limited size to read when call `GetBlobContent`. After: ![image](https://github.com/go-gitea/gitea/assets/18380374/22a42936-4cf8-40b4-a5c7-e384082beb0d) --------- Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
* Remove duplicated functions when deleting a branch (#25128)Lunny Xiao2023-06-082-9/+3
| | | | | | | | | | Extract from #22743 `DeleteBranch` will trigger a push update event, so that `pull_service.CloseBranchPulls` has been invoked twice and `AddDeletedBranch` is better to be moved to push update then even user delete a branch via git command, it will also be triggered. Co-authored-by: Giteabot <teabot@gitea.io>
* Use RepositoryList instead of []*Repository (#25074)Lunny Xiao2023-06-051-1/+1
|
* Update repo's default branch when adding new files in an empty one (#25017)wxiaoguang2023-05-311-8/+8
| | | | | | Fix #25014 Only API needs this fix. On the Web UI, users could only add new file on the default branch.
* API endpoint for changing/creating/deleting multiple files (#24887)Denys Konovalov2023-05-293-368/+279
| | | | | | | | | | | This PR creates an API endpoint for creating/updating/deleting multiple files in one API call similar to the solution provided by [GitLab](https://docs.gitlab.com/ee/api/commits.html#create-a-commit-with-multiple-files-and-actions). To archive this, the CreateOrUpdateRepoFile and DeleteRepoFIle functions in files service are unified into one function supporting multiple files and actions. Resolves #14619
* Improve queue and logger context (#24924)wxiaoguang2023-05-262-9/+7
| | | | | | | | | | | | | | | | | | | | | | | | Before there was a "graceful function": RunWithShutdownFns, it's mainly for some modules which doesn't support context. The old queue system doesn't work well with context, so the old queues need it. After the queue refactoring, the new queue works with context well, so, use Golang context as much as possible, the `RunWithShutdownFns` could be removed (replaced by RunWithCancel for context cancel mechanism), the related code could be simplified. This PR also fixes some legacy queue-init problems, eg: * typo : archiver: "unable to create codes indexer queue" => "unable to create repo-archive queue" * no nil check for failed queues, which causes unfriendly panic After this PR, many goroutines could have better display name: ![image](https://github.com/go-gitea/gitea/assets/2114189/701b2a9b-8065-4137-aeaa-0bda2b34604a) ![image](https://github.com/go-gitea/gitea/assets/2114189/f1d5f50f-0534-40f0-b0be-f2c9daa5fe92)
* Use the type RefName for all the needed places and fix pull mirror sync bugs ↵Lunny Xiao2023-05-263-27/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | (#24634) This PR replaces all string refName as a type `git.RefName` to make the code more maintainable. Fix #15367 Replaces #23070 It also fixed a bug that tags are not sync because `git remote --prune origin` will not remove local tags if remote removed. We in fact should use `git fetch --prune --tags origin` but not `git remote update origin` to do the sync. Some answer from ChatGPT as ref. > If the git fetch --prune --tags command is not working as expected, there could be a few reasons why. Here are a few things to check: > >Make sure that you have the latest version of Git installed on your system. You can check the version by running git --version in your terminal. If you have an outdated version, try updating Git and see if that resolves the issue. > >Check that your Git repository is properly configured to track the remote repository's tags. You can check this by running git config --get-all remote.origin.fetch and verifying that it includes +refs/tags/*:refs/tags/*. If it does not, you can add it by running git config --add remote.origin.fetch "+refs/tags/*:refs/tags/*". > >Verify that the tags you are trying to prune actually exist on the remote repository. You can do this by running git ls-remote --tags origin to list all the tags on the remote repository. > >Check if any local tags have been created that match the names of tags on the remote repository. If so, these local tags may be preventing the git fetch --prune --tags command from working properly. You can delete local tags using the git tag -d command. --------- Co-authored-by: delvh <dev.lh@web.de>
* Add status indicator on main home screen for each repo (#24638)Yarden Shoham2023-05-132-0/+18
| | | | | | | | | | | | | | | | | | | | It will show the calculated commit status state of the latest commit on the default branch for each repository in the dashboard repo list - Closes #15620 # Before ![image](https://github.com/go-gitea/gitea/assets/20454870/aa1326c7-43c0-458a-a798-3102c766bcf9) # After ![image](https://github.com/go-gitea/gitea/assets/20454870/8658cc03-2224-442a-b1c8-bf64126e4575) --------- Signed-off-by: Yarden Shoham <git@yardenshoham.com> Co-authored-by: delvh <dev.lh@web.de> Co-authored-by: Giteabot <teabot@gitea.io>
* Improve avatar uploading / resizing / compressing, remove Fomantic card ↵wxiaoguang2023-05-131-5/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | module (#24653) Fixes: #8972 Fixes: #24263 And I think it also (partially) fix #24263 (no need to convert) , because users could upload any supported image format if it isn't larger than AVATAR_MAX_ORIGIN_SIZE The main idea: * if the uploaded file size is not larger than AVATAR_MAX_ORIGIN_SIZE, use the origin * if the resized size is larger than the origin, use the origin Screenshots: JPG: <details> ![image](https://github.com/go-gitea/gitea/assets/2114189/70e98bb0-ecb9-4c4e-a89f-4a37d4e37f8e) </details> APNG: <details> ![image](https://github.com/go-gitea/gitea/assets/2114189/9055135b-5e2d-4152-bd72-596fcb7c6671) ![image](https://github.com/go-gitea/gitea/assets/2114189/50364caf-f7f6-4241-a289-e485fe4cd582) </details> WebP (animated) <details> ![image](https://github.com/go-gitea/gitea/assets/2114189/f642eb85-498a-49a5-86bf-0a7b04089ae0) </details> The only exception: if a WebP image is larger than MaxOriginSize and it is animated, then current `webp` package can't decode it, so only in this case it isn't supported. IMO no need to support such case: why a user would upload a 1MB animated webp as avatar? crazy ..... --------- Co-authored-by: silverwind <me@silverwind.io>
* Rewrite queue (#24505)wxiaoguang2023-05-082-15/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | # ⚠️ Breaking Many deprecated queue config options are removed (actually, they should have been removed in 1.18/1.19). If you see the fatal message when starting Gitea: "Please update your app.ini to remove deprecated config options", please follow the error messages to remove these options from your app.ini. Example: ``` 2023/05/06 19:39:22 [E] Removed queue option: `[indexer].ISSUE_INDEXER_QUEUE_TYPE`. Use new options in `[queue.issue_indexer]` 2023/05/06 19:39:22 [E] Removed queue option: `[indexer].UPDATE_BUFFER_LEN`. Use new options in `[queue.issue_indexer]` 2023/05/06 19:39:22 [F] Please update your app.ini to remove deprecated config options ``` Many options in `[queue]` are are dropped, including: `WRAP_IF_NECESSARY`, `MAX_ATTEMPTS`, `TIMEOUT`, `WORKERS`, `BLOCK_TIMEOUT`, `BOOST_TIMEOUT`, `BOOST_WORKERS`, they can be removed from app.ini. # The problem The old queue package has some legacy problems: * complexity: I doubt few people could tell how it works. * maintainability: Too many channels and mutex/cond are mixed together, too many different structs/interfaces depends each other. * stability: due to the complexity & maintainability, sometimes there are strange bugs and difficult to debug, and some code doesn't have test (indeed some code is difficult to test because a lot of things are mixed together). * general applicability: although it is called "queue", its behavior is not a well-known queue. * scalability: it doesn't seem easy to make it work with a cluster without breaking its behaviors. It came from some very old code to "avoid breaking", however, its technical debt is too heavy now. It's a good time to introduce a better "queue" package. # The new queue package It keeps using old config and concept as much as possible. * It only contains two major kinds of concepts: * The "base queue": channel, levelqueue, redis * They have the same abstraction, the same interface, and they are tested by the same testing code. * The "WokerPoolQueue", it uses the "base queue" to provide "worker pool" function, calls the "handler" to process the data in the base queue. * The new code doesn't do "PushBack" * Think about a queue with many workers, the "PushBack" can't guarantee the order for re-queued unhandled items, so in new code it just does "normal push" * The new code doesn't do "pause/resume" * The "pause/resume" was designed to handle some handler's failure: eg: document indexer (elasticsearch) is down * If a queue is paused for long time, either the producers blocks or the new items are dropped. * The new code doesn't do such "pause/resume" trick, it's not a common queue's behavior and it doesn't help much. * If there are unhandled items, the "push" function just blocks for a few seconds and then re-queue them and retry. * The new code doesn't do "worker booster" * Gitea's queue's handlers are light functions, the cost is only the go-routine, so it doesn't make sense to "boost" them. * The new code only use "max worker number" to limit the concurrent workers. * The new "Push" never blocks forever * Instead of creating more and more blocking goroutines, return an error is more friendly to the server and to the end user. There are more details in code comments: eg: the "Flush" problem, the strange "code.index" hanging problem, the "immediate" queue problem. Almost ready for review. TODO: * [x] add some necessary comments during review * [x] add some more tests if necessary * [x] update documents and config options * [x] test max worker / active worker * [x] re-run the CI tasks to see whether any test is flaky * [x] improve the `handleOldLengthConfiguration` to provide more friendly messages * [x] fine tune default config values (eg: length?) ## Code coverage: ![image](https://user-images.githubusercontent.com/2114189/236620635-55576955-f95d-4810-b12f-879026a3afdf.png)
generated by cgit v1.2.3 (git 2.39.1) at 2025-04-18 23:23:33 +0000