summaryrefslogtreecommitdiffstats
path: root/modules/repository
Commit message (Collapse)AuthorAgeFilesLines
* Move database operations of merging a pull request to post receive hook and ↵Giteabot2024-05-081-0/+8
| | | | | | | | | | | | | | | | | | | | | add a transaction (#30805) (#30888) Backport #30805 by @lunny Merging PR may fail because of various problems. The pull request may have a dirty state because there is no transaction when merging a pull request. ref https://github.com/go-gitea/gitea/pull/25741#issuecomment-2074126393 This PR moves all database update operations to post-receive handler for merging a pull request and having a database transaction. That means if database operations fail, then the git merging will fail, the git client will get a fail result. There are already many tests for pull request merging, so we don't need to add a new one. Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com> Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
* Make "sync branch" also sync object format and add tests (#30878) (#30880)Giteabot2024-05-072-0/+41
| | | | | Backport #30878 by wxiaoguang Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
* Enable more `revive` linter rules (#30608)silverwind2024-04-221-1/+0
| | | | | | | | | | | Noteable additions: - `redefines-builtin-id` forbid variable names that shadow go builtins - `empty-lines` remove unnecessary empty lines that `gofumpt` does not remove for some reason - `superfluous-else` eliminate more superfluous `else` branches Rules are also sorted alphabetically and I cleaned up various parts of `.golangci.yml`.
* Add user blocking (#29028)KN4CK3R2024-03-042-1/+9
| | | | | | | | | | | | | | | | | | | | | | Fixes #17453 This PR adds the abbility to block a user from a personal account or organization to restrict how the blocked user can interact with the blocker. The docs explain what's the consequence of blocking a user. Screenshots: ![grafik](https://github.com/go-gitea/gitea/assets/1666336/4ed884f3-e06a-4862-afd3-3b8aa2488dc6) ![grafik](https://github.com/go-gitea/gitea/assets/1666336/ae6d4981-f252-4f50-a429-04f0f9f1cdf1) ![grafik](https://github.com/go-gitea/gitea/assets/1666336/ca153599-5b0f-4b4a-90fe-18bdfd6f0b6b) --------- Co-authored-by: Lauris BH <lauris@nix.lv>
* Enable/disable owner and repo projects independently (#28805)Denys Konovalov2024-03-041-0/+6
| | | | | | | | | | | | | | | Part of #23318 Add menu in repo settings to allow for repo admin to decide not just if projects are enabled or disabled per repo, but also which kind of projects (repo-level/owner-level) are enabled. If repo projects disabled, don't show the projects tab. ![grafik](https://github.com/go-gitea/gitea/assets/47871822/b9b43fb4-824b-47f9-b8e2-12004313647c) --------- Co-authored-by: delvh <dev.lh@web.de>
* Move migration functions to services layer (#29497)Lunny Xiao2024-03-011-243/+0
|
* Move generate from module to service (#29465)Lunny Xiao2024-02-283-522/+0
|
* Ignore the linux anchor point to avoid linux migrate failure (#29295)Lunny Xiao2024-02-221-1/+3
| | | | | | | | | | Fix #28843 This PR will bypass the pushUpdateTag to database failure when syncAllTags. An error log will be recorded. --------- Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
* Always write proc-receive hook for all git versions (#29287)wxiaoguang2024-02-211-7/+5
|
* bump to use go 1.22 (#29119)techknowlogick2024-02-141-1/+1
|
* Add merge style `fast-forward-only` (#28954)Chris Copeland2024-02-121-1/+5
| | | | | | | | | With this option, it is possible to require a linear commit history with the following benefits over the next best option `Rebase+fast-forward`: The original commits continue existing, with the original signatures continuing to stay valid instead of being rewritten, there is no merge commit, and reverting commits becomes easier. Closes #24906
* Fix bug for generated repository object format (#28969)Lunny Xiao2024-01-281-16/+15
| | | | A repository generated from a template repository should have the same git ObjectFormat.
* Simplify how git repositories are opened (#28937)Lunny Xiao2024-01-273-4/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ## Purpose This is a refactor toward building an abstraction over managing git repositories. Afterwards, it does not matter anymore if they are stored on the local disk or somewhere remote. ## What this PR changes We used `git.OpenRepository` everywhere previously. Now, we should split them into two distinct functions: Firstly, there are temporary repositories which do not change: ```go git.OpenRepository(ctx, diskPath) ``` Gitea managed repositories having a record in the database in the `repository` table are moved into the new package `gitrepo`: ```go gitrepo.OpenRepository(ctx, repo_model.Repo) ``` Why is `repo_model.Repository` the second parameter instead of file path? Because then we can easily adapt our repository storage strategy. The repositories can be stored locally, however, they could just as well be stored on a remote server. ## Further changes in other PRs - A Git Command wrapper on package `gitrepo` could be created. i.e. `NewCommand(ctx, repo_model.Repository, commands...)`. `git.RunOpts{Dir: repo.RepoPath()}`, the directory should be empty before invoking this method and it can be filled in the function only. #28940 - Remove the `RepoPath()`/`WikiPath()` functions to reduce the possibility of mistakes. --------- Co-authored-by: delvh <dev.lh@web.de>
* Don't remove all mirror repository's releases when mirroring (#28817)Lunny Xiao2024-01-262-6/+146
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fix #22066 # Purpose This PR fix the releases will be deleted when mirror repository sync the tags. # The problem In the previous implementation of #19125. All releases record in databases of one mirror repository will be deleted before sync. Ref: https://github.com/go-gitea/gitea/pull/19125/files#diff-2aa04998a791c30e5a02b49a97c07fcd93d50e8b31640ce2ddb1afeebf605d02R481 # The Pros This PR introduced a new method which will load all releases from databases and all tags on git data into memory. And detect which tags needs to be inserted, which tags need to be updated or deleted. Only tags releases(IsTag=true) which are not included in git data will be deleted, only tags which sha1 changed will be updated. So it will not delete any real releases include drafts. # The Cons The drawback is the memory usage will be higher than before if there are many tags on this repository. This PR defined a special release struct to reduce columns loaded from database to memory.
* Allow to sync tags from admin dashboard (#28045)JakobDev2024-01-241-0/+16
| | | | | | | | | Inspired by #28043 This PR adds a option to the Admin Dashboard to sync all tags to the database. ![grafik](https://github.com/go-gitea/gitea/assets/15185051/26ac51ef-82a4-4fd9-a6a6-5aefec612ff6)
* Move more functions to db.Find (#28419)Lunny Xiao2024-01-151-1/+2
| | | | | | | | | Following #28220 This PR move more functions to use `db.Find`. --------- Co-authored-by: delvh <dev.lh@web.de>
* Make offline mode as default to no connect external avatar service by ↵Lunny Xiao2023-12-211-1/+2
| | | | | | | default (#28548) To keep user's privacy, make offline mode as true by default. Users can still change it from installation ui and app.ini
* Improve ObjectFormat interface (#28496)Lunny Xiao2023-12-192-6/+5
| | | | | | | | | | | | | | | | | | | | | | | The 4 functions are duplicated, especially as interface methods. I think we just need to keep `MustID` the only one and remove other 3. ``` MustID(b []byte) ObjectID MustIDFromString(s string) ObjectID NewID(b []byte) (ObjectID, error) NewIDFromString(s string) (ObjectID, error) ``` Introduced the new interfrace method `ComputeHash` which will replace the interface `HasherInterface`. Now we don't need to keep two interfaces. Reintroduced `git.NewIDFromString` and `git.MustIDFromString`. The new function will detect the hash length to decide which objectformat of it. If it's 40, then it's SHA1. If it's 64, then it's SHA256. This will be right if the commitID is a full one. So the parameter should be always a full commit id. @AdamMajer Please review.
* Adjust object format interface (#28469)Lunny Xiao2023-12-173-5/+5
| | | | | | | - Remove `ObjectFormatID` - Remove function `ObjectFormatFromID`. - Use `Sha1ObjectFormat` directly but not a pointer because it's an empty struct. - Store `ObjectFormatName` in `repository` struct
* Abstract hash function usage (#28138)Adam Majer2023-12-134-9/+14
| | | | | | Refactor Hash interfaces and centralize hash function. This will allow easier introduction of different hash function later on. This forms the "no-op" part of the SHA256 enablement patch.
* Second part of refactor `db.Find` (#28194)Lunny Xiao2023-12-111-5/+3
| | | Continue of #27798 and move more functions to `db.Find` and `db.Count`.
* Remove GetByBean method because sometimes it's danger when query condition ↵Lunny Xiao2023-12-072-10/+13
| | | | | | | | | | | | | | | | | | | | parameter is zero and also introduce new generic methods (#28220) The function `GetByBean` has an obvious defect that when the fields are empty values, it will be ignored. Then users will get a wrong result which is possibly used to make a security problem. To avoid the possibility, this PR removed function `GetByBean` and all references. And some new generic functions have been introduced to be used. The recommand usage like below. ```go // if query an object according id obj, err := db.GetByID[Object](ctx, id) // query with other conditions obj, err := db.Get[Object](ctx, builder.Eq{"a": a, "b":b}) ```
* Ignore "non-existing" errors when getDirectorySize calculates the size (#28276)wxiaoguang2023-11-291-15/+12
| | | | | | | | | The git command may operate the git directory (add/remove) files in any time. So when the code iterates the directory, some files may disappear during the "walk". All "IsNotExist" errors should be ignored. Fix #26765
* Ignore temporary files for directory size (#28265)Earl Warren2023-11-281-1/+5
| | | | Co-authored-by: Gusted <postmaster@gusted.xyz>
* Refactor system setting (#27000)wxiaoguang2023-10-051-8/+1
| | | | | | | | | This PR reduces the complexity of the system setting system. It only needs one line to introduce a new option, and the option can be used anywhere out-of-box. It is still high-performant (and more performant) because the config values are cached in the config system.
* Even more `db.DefaultContext` refactor (#27352)JakobDev2023-10-032-2/+2
| | | | | | | | Part of #27065 --------- Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com> Co-authored-by: delvh <dev.lh@web.de>
* make writing main test easier (#27270)Lunny Xiao2023-09-281-4/+1
| | | | | | | | | This PR removed `unittest.MainTest` the second parameter `TestOptions.GiteaRoot`. Now it detects the root directory by current working directory. --------- Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
* Another round of `db.DefaultContext` refactor (#27103)JakobDev2023-09-252-11/+12
| | | | | | | Part of #27065 --------- Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
* Add `RemoteAddress` to mirrors (#26952)KN4CK3R2023-09-161-0/+5
| | | | | This PR adds a new field `RemoteAddress` to both mirror types which contains the sanitized remote address for easier (database) access to that information. Will be used in the audit PR if merged.
* Reduce usage of `db.DefaultContext` (#27073)JakobDev2023-09-141-4/+6
| | | | | | | | | | | | | | Part of #27065 This reduces the usage of `db.DefaultContext`. I think I've got enough files for the first PR. When this is merged, I will continue working on this. Considering how many files this PR affect, I hope it won't take to long to merge, so I don't end up in the merge conflict hell. --------- Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
* Fix context cache bug & enable context cache for dashabord commits' authors ↵Lunny Xiao2023-09-112-45/+21
| | | | | | | | | | | (#26991) Unfortunately, when a system setting hasn't been stored in the database, it cannot be cached. Meanwhile, this PR also uses context cache for push email avatar display which should avoid to read user table via email address again and again. According to my local test, this should reduce dashboard elapsed time from 150ms -> 80ms .
* move repository deletion to service layer (#26948)Lunny Xiao2023-09-081-0/+2
| | | Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
* Move createrepository from module to service layer (#26927)Lunny Xiao2023-09-068-454/+23
| | | | Repository creation depends on many models, so moving it to service layer is better.
* Fix bug for ctx usage (#26762)Lunny Xiao2023-08-281-4/+4
| | | | | Regression from #26158 Fix #26684
* Refactor and enhance issue indexer to support both searching, filtering and ↵Jason Song2023-07-311-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | paging (#26012) Fix #24662. Replace #24822 and #25708 (although it has been merged) ## Background In the past, Gitea supported issue searching with a keyword and conditions in a less efficient way. It worked by searching for issues with the keyword and obtaining limited IDs (as it is heavy to get all) on the indexer (bleve/elasticsearch/meilisearch), and then querying with conditions on the database to find a subset of the found IDs. This is why the results could be incomplete. To solve this issue, we need to store all fields that could be used as conditions in the indexer and support both keyword and additional conditions when searching with the indexer. ## Major changes - Redefine `IndexerData` to include all fields that could be used as filter conditions. - Refactor `Search(ctx context.Context, kw string, repoIDs []int64, limit, start int, state string)` to `Search(ctx context.Context, options *SearchOptions)`, so it supports more conditions now. - Change the data type stored in `issueIndexerQueue`. Use `IndexerMetadata` instead of `IndexerData` in case the data has been updated while it is in the queue. This also reduces the storage size of the queue. - Enhance searching with Bleve/Elasticsearch/Meilisearch, make them fully support `SearchOptions`. Also, update the data versions. - Keep most logic of database indexer, but remove `issues.SearchIssueIDsByKeyword` in `models` to avoid confusion where is the entry point to search issues. - Start a Meilisearch instance to test it in unit tests. - Add unit tests with almost full coverage to test Bleve/Elasticsearch/Meilisearch indexer. --------- Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Support copy protected branch from template repository (#25889)Lunny Xiao2023-07-211-11/+13
| | | Fix #14303
* Fix branch commit message too long problem (#25588)Lunny Xiao2023-06-301-4/+4
| | | | | | When branch's commit CommitMessage is too long, the column maybe too short.(TEXT 16K for mysql). This PR will fix it to only store the summary because these message will only show on branch list or possible future search?
* Sync branches into databases (#22743)Lunny Xiao2023-06-293-1/+146
| | | | | | | | | | | | | | | | | | | | | | | | | | Related #14180 Related #25233 Related #22639 Close #19786 Related #12763 This PR will change all the branches retrieve method from reading git data to read database to reduce git read operations. - [x] Sync git branches information into database when push git data - [x] Create a new table `Branch`, merge some columns of `DeletedBranch` into `Branch` table and drop the table `DeletedBranch`. - [x] Read `Branch` table when visit `code` -> `branch` page - [x] Read `Branch` table when list branch names in `code` page dropdown - [x] Read `Branch` table when list git ref compare page - [x] Provide a button in admin page to manually sync all branches. - [x] Sync branches if repository is not empty but database branches are empty when visiting pages with branches list - [x] Use `commit_time desc` as the default FindBranch order by to keep consistent as before and deleted branches will be always at the end. --------- Co-authored-by: Jason Song <i@wolfogre.com>
* Split lfs size from repository size (#22900)a10121127962023-06-281-1/+1
| | | | | | | | | | | | | | | | | | | | releated to #21820 - Split `Size` in repository table as two new colunms, one is `GitSize` for git size, the other is `LFSSize` for lfs data. still store full size in `Size` colunm. - Show full size on ui, but show each of them by a `title`; example: ![image](https://user-images.githubusercontent.com/25342410/218636251-e200f085-d7e7-4a25-9ff1-b586a63e07a9.png) - Return full size in api response. --------- Signed-off-by: a1012112796 <1012112796@qq.com> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com> Co-authored-by: silverwind <me@silverwind.io> Co-authored-by: DmitryFrolovTri <23313323+DmitryFrolovTri@users.noreply.github.com> Co-authored-by: Giteabot <teabot@gitea.io>
* Move some regexp out of functions (#25430)John Olheiser2023-06-211-3/+3
| | | | | | | | | | /cc @KN4CK3R https://github.com/go-gitea/gitea/pull/25294#discussion_r1237425343 I also searched the codebase and found a few more. --------- Signed-off-by: jolheiser <john.olheiser@gmail.com>
* Substitute variables in path names of template repos too (#25294)Kyle D2023-06-202-2/+41
| | | | | | | | | | | | | | | | | | | | | | | | | | ### Summary Extend the template variable substitution to replace file paths. This can be helpful for setting up log files & directories that should match the repository name. ### PR Changes - Move files matching glob pattern when setting up repos from template - For security, added ~escaping~ sanitization for cross-platform support and to prevent directory traversal (thanks @silverwind for the reference) - Added unit testing for escaping function - Fixed the integration tests for repo template generation by passing the repo_template_id - Updated the integration testfiles to add some variable substitution & assert the outputs I had to fix the existing repo template integration test and extend it to add a check for variable substitutions. Example: ![image](https://github.com/go-gitea/gitea/assets/12700993/621feb09-0ef3-460e-afa8-da74cd84fa4e)
* Remove stars when repo goes private (#19904)Wim2023-06-051-0/+4
| | | Fixes #18600
* Use git command instead of the ini package to remove the `origin` remote ↵Lunny Xiao2023-06-051-12/+9
| | | | (#25066)
* Refactor INI package (first step) (#25024)wxiaoguang2023-06-021-2/+2
| | | | | | | | | The INI package has many bugs and quirks, and in fact it is unmaintained. This PR is the first step for the INI package refactoring: * Use Gitea's "config_provider" to provide INI access * Deprecate the INI package by golangci.yml rule
* Use the type RefName for all the needed places and fix pull mirror sync bugs ↵Lunny Xiao2023-05-262-35/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | (#24634) This PR replaces all string refName as a type `git.RefName` to make the code more maintainable. Fix #15367 Replaces #23070 It also fixed a bug that tags are not sync because `git remote --prune origin` will not remove local tags if remote removed. We in fact should use `git fetch --prune --tags origin` but not `git remote update origin` to do the sync. Some answer from ChatGPT as ref. > If the git fetch --prune --tags command is not working as expected, there could be a few reasons why. Here are a few things to check: > >Make sure that you have the latest version of Git installed on your system. You can check the version by running git --version in your terminal. If you have an outdated version, try updating Git and see if that resolves the issue. > >Check that your Git repository is properly configured to track the remote repository's tags. You can check this by running git config --get-all remote.origin.fetch and verifying that it includes +refs/tags/*:refs/tags/*. If it does not, you can add it by running git config --add remote.origin.fetch "+refs/tags/*:refs/tags/*". > >Verify that the tags you are trying to prune actually exist on the remote repository. You can do this by running git ls-remote --tags origin to list all the tags on the remote repository. > >Check if any local tags have been created that match the names of tags on the remote repository. If so, these local tags may be preventing the git fetch --prune --tags command from working properly. You can delete local tags using the git tag -d command. --------- Co-authored-by: delvh <dev.lh@web.de>
* Improve avatar uploading / resizing / compressing, remove Fomantic card ↵wxiaoguang2023-05-131-4/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | module (#24653) Fixes: #8972 Fixes: #24263 And I think it also (partially) fix #24263 (no need to convert) , because users could upload any supported image format if it isn't larger than AVATAR_MAX_ORIGIN_SIZE The main idea: * if the uploaded file size is not larger than AVATAR_MAX_ORIGIN_SIZE, use the origin * if the resized size is larger than the origin, use the origin Screenshots: JPG: <details> ![image](https://github.com/go-gitea/gitea/assets/2114189/70e98bb0-ecb9-4c4e-a89f-4a37d4e37f8e) </details> APNG: <details> ![image](https://github.com/go-gitea/gitea/assets/2114189/9055135b-5e2d-4152-bd72-596fcb7c6671) ![image](https://github.com/go-gitea/gitea/assets/2114189/50364caf-f7f6-4241-a289-e485fe4cd582) </details> WebP (animated) <details> ![image](https://github.com/go-gitea/gitea/assets/2114189/f642eb85-498a-49a5-86bf-0a7b04089ae0) </details> The only exception: if a WebP image is larger than MaxOriginSize and it is animated, then current `webp` package can't decode it, so only in this case it isn't supported. IMO no need to support such case: why a user would upload a 1MB animated webp as avatar? crazy ..... --------- Co-authored-by: silverwind <me@silverwind.io>
* Replace placeholders in licenses (#24354)Jason Song2023-05-053-2/+301
| | | | | | | | | | | | | | | | | | | | | | | | | Replace #22117. Implement it in a more maintainable way. Some licenses have placeholders e.g. the BSD licenses start with this line: ``` Copyright (c) <year> <owner>. ``` This PR replaces the placeholders with the correct value when initialize a new repo. ### FAQ - Why not use a regex? It will be a pretty complicated regex which could be hard to maintain. - There're still missing placeholders. There are over 500 licenses, it's impossible for anyone to inspect all of them alone. Please help to add them if you find any, and it is also OK to leave them for the future. --------- Co-authored-by: Giteabot <teabot@gitea.io>
* Fix unclear `IsRepositoryExist` logic (#24374)wxiaoguang2023-04-281-1/+1
| | | | | | | | | | | | | | | There was only one `IsRepositoryExist` function, it did: `has && isDir` However it's not right, and it would cause 500 error when creating a new repository if the dir exists. Then, it was changed to `has || isDir`, it is still incorrect, it affects the "adopt repo" logic. To make the logic clear: * IsRepositoryModelOrDirExist * IsRepositoryModelExist
* Use a general approach to access custom/static/builtin assets (#24022)wxiaoguang2023-04-121-1/+1
| | | | | | | | | | | | | | | | | | | The idea is to use a Layered Asset File-system (modules/assetfs/layered.go) For example: when there are 2 layers: "custom", "builtin", when access to asset "my/page.tmpl", the Layered Asset File-system will first try to use "custom" assets, if not found, then use "builtin" assets. This approach will hugely simplify a lot of code, make them testable. Other changes: * Simplify the AssetsHandlerFunc code * Simplify the `gitea embedded` sub-command code --------- Co-authored-by: Jason Song <i@wolfogre.com> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Make label templates have consistent behavior and priority (#23749)wxiaoguang2023-04-103-50/+100
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fix https://github.com/go-gitea/gitea/issues/23715 Other related PRs: * #23717 * #23716 * #23719 This PR is different from others, it tries to resolve the problem fundamentally (and brings more benefits) Although it looks like some more lines are added, actually many new lines are for tests. ---- Before, the code was just "guessing" the file type and try to parse them. <details> ![image](https://user-images.githubusercontent.com/2114189/228002245-57d58e27-1078-4da9-bf42-5bc0b264c6ce.png) </details> This PR: * Always remember the original option file names, and always use correct parser for them. * Another benefit is that we can sort the Label Templates now (before there was a map, its key order is undefined) ![image](https://user-images.githubusercontent.com/2114189/228002432-931b9f18-3908-484b-a36b-04760c9ad132.png)
generated by cgit v1.2.3 (git 2.39.1) at 2025-04-08 08:25:36 +0000