aboutsummaryrefslogtreecommitdiffstats
path: root/services/repository/migrate.go
Commit message (Collapse)AuthorAgeFilesLines
* Replace update repository function in some places (#34566)Lunny Xiao2025-06-171-1/+5
| | | | | | | | | | | | `UpdateAllCols` is dangerous, the columns should be updated when necessary. This PR replaces some `updateRepository` invokes to reduce possible problems and wrongly updated time. Some parts have been fixed in #34388, but some are hidden in the function `updateRepository`. Alternatively, using `UpdateRepositoryColsNoAutoTime` to update the changed columns. Some `UpdateRepoSize` invokes are duplicated, so they will be removed when extracting from `updateRepository`.
* Performance optimization for tags synchronization (#34355)Lunny Xiao2025-05-221-2/+2
| | | | | | | | | The tags synchronization is very slow for a non-mirror repository with many tags especially forking. This PR make all repositories' tags synchronization use the same function and remove the low performance synchronization function. The commit count of tag now will not be stored into database when syncing. Since the commits count will always be read from cache or git data, the `NumCommits` in the release table will be updated for the first read from git data.
* Rework create/fork/adopt/generate repository to make sure resources will be ↵Lunny Xiao2025-04-071-8/+2
| | | | | | | | | | | | | | | | | | | cleanup once failed (#31035) Fix #28144 To make the resources will be cleanup once failed. All repository operations now follow a consistent pattern: - 1. Create a database record for the repository with the status being_migrated. - 2. Register a deferred cleanup function to delete the repository and its related data if the operation fails. - 3. Perform the actual Git and database operations step by step. - 4. Upon successful completion, update the repository’s status to ready. The adopt operation is a special case — if it fails, the repository on disk should not be deleted.
* Refactor Branch struct in package modules/git (#33980)Lunny Xiao2025-04-021-3/+3
| | | | The `Branch` struct in `modules/git` package is unnecessary. We can just use a `string` to represent a branch
* Fix some migration and repo name problems (#33986)wxiaoguang2025-03-241-0/+14
| | | | | | 1. Ignore empty inputs in `UnmarshalHandleDoubleEncode` 2. Ignore non-existing `stateEvent.User` in gitlab migration 3. Enable `release` and `wiki` units when they are selected in migration 4. Sanitize repo name for migration and new repo
* Don't create duplicated functions for code repositories and wiki ↵Lunny Xiao2025-03-191-2/+2
| | | | | | | | | | repositories (#33924) Fix https://github.com/go-gitea/gitea/pull/33910#pullrequestreview-2688913865 This PR changed the Repositroy interface in `gitrepo` package which makes it only focus the relative path in the disk and abstract whether it's a wiki repository or not.
* Move hooks function to gitrepo and reduce expose repopath (#33890)Lunny Xiao2025-03-161-4/+4
| | | | Extract from #28966 Follow #33874
* Remove context from git struct (#33793)TheFox0x72025-03-041-7/+7
| | | | Argument is moved from struct init in command run, which lets us remove context from struct.
* Support the new exit code for `git remote` subcommands for git version ↵yp053272025-01-071-4/+3
| | | | | | | | | >=2.30.0 (#33129) Fix #32889 --------- Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
* Refactor pprof labels and process desc (#32909)wxiaoguang2024-12-201-1/+0
| | | | | | * Deprecate "gopid" in log, it is not useful and requires very hacky approach * Remove "git.Command.SetDescription" because it is not useful and only makes the logs too flexible
* Support repo license (#24872)yp053272024-10-011-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Close #278 Close #24076 ## Solutions: - Use [google/licenseclassifier](https://github.com/google/licenseclassifier/) Test result between [google/licensecheck](https://github.com/google/licensecheck) and [go-license-detector](https://github.com/go-enry/go-license-detector): https://github.com/go-gitea/gitea/pull/24872#issuecomment-1560361167 Test result between [google/licensecheck](https://github.com/google/licensecheck) and [google/licenseclassifier](https://github.com/google/licenseclassifier/): https://github.com/go-gitea/gitea/pull/24872#issuecomment-1576092178 - Generate License Convert Name List to avoid import license templates with same contents Gitea automatically get latest license data from[ spdx/license-list-data](https://github.com/spdx/license-list-data). But unfortunately, some license templates have same contents. #20915 [click here to see the list](https://github.com/go-gitea/gitea/pull/24872#issuecomment-1584141684) So we will generate a list of these license templates with same contents and create a new file to save the result when using `make generate-license`. (Need to decide the save path) - Save License info into a new table `repo_license` Can easily support searching repo by license in the future. ## Screen shot Single License: ![image](https://github.com/go-gitea/gitea/assets/18380374/41260bd7-0b4c-4038-8592-508706cffa9f) Multiple Licenses: ![image](https://github.com/go-gitea/gitea/assets/18380374/34ce2f73-7e18-446b-9b96-ecc4fb61bd70) Triggers: - [x] Push commit to default branch - [x] Create repo - [x] Mirror repo - [x] When Default Branch is changed, licenses should be updated Todo: - [x] Save Licenses info in to DB when there's a change to license file in the commit - [x] DB Migration - [x] A nominal test? - [x] Select which library to use(https://github.com/go-gitea/gitea/pull/24872#issuecomment-1560361167) - [x] API Support - [x] Add repo license table - ~Select license in settings if there are several licenses(Not recommended)~ - License board(later, not in this PR) ![image](https://github.com/go-gitea/gitea/assets/18380374/2c3c3bf8-bcc2-4c6d-8ce0-81d1a9733878) --------- Co-authored-by: silverwind <me@silverwind.io> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com> Co-authored-by: Denys Konovalov <kontakt@denyskon.de> Co-authored-by: delvh <dev.lh@web.de> Co-authored-by: KN4CK3R <admin@oldschoolhack.me> Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: 6543 <m.huber@kithara.com> Co-authored-by: a1012112796 <1012112796@qq.com> Co-authored-by: techknowlogick <techknowlogick@gitea.com>
* Distinguish LFS object errors to ignore missing objects during migration ↵Jason Song2024-07-311-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | (#31702) Fix #31137. Replace #31623 #31697. When migrating LFS objects, if there's any object that failed (like some objects are losted, which is not really critical), Gitea will stop migrating LFS immediately but treat the migration as successful. This PR checks the error according to the [LFS api doc](https://github.com/git-lfs/git-lfs/blob/main/docs/api/batch.md#successful-responses). > LFS object error codes should match HTTP status codes where possible: > > - 404 - The object does not exist on the server. > - 409 - The specified hash algorithm disagrees with the server's acceptable options. > - 410 - The object was removed by the owner. > - 422 - Validation error. If the error is `404`, it's safe to ignore it and continue migration. Otherwise, stop the migration and mark it as failed to ensure data integrity of LFS objects. And maybe we should also ignore others errors (maybe `410`? I'm not sure what's the difference between "does not exist" and "removed by the owner".), we can add it later when some users report that they have failed to migrate LFS because of an error which should be ignored.
* Move get/set default branch from git package to gitrepo package to hide ↵Lunny Xiao2024-03-081-8/+1
| | | | repopath (#29126)
* Make wiki default branch name changable (#29603)wxiaoguang2024-03-061-30/+58
| | | | | | | | | Fix #29000 Fix #28685 Fix #18568 Related: #27497 And by the way fix #24036, add a Cancel button there (one line)
* Move migration functions to services layer (#29497)Lunny Xiao2024-03-011-0/+264
generated by cgit v1.2.3 (git 2.39.1) at 2025-07-14 19:51:35 +0000