aboutsummaryrefslogtreecommitdiffstats
path: root/modules/migrations
Commit message (Collapse)AuthorAgeFilesLines
* Make TestOneDevDownloadRepo work again & more resistant (#16987)65432021-09-081-12/+7
| | | | | * Update TestOneDevDownloadRepo since OneDev api changed * Use test functions for migrations
* Resolve TODO: Enable pagination on GiteaDownloader.GetComments() & update ↵65432021-09-062-41/+37
| | | | | | | another TODO (#16963) * Update TODO in migrations * Resolve TODO: enable pagination on GiteaDownloader.GetComments()
* Ignore review comment when ref commit is missed (#16905)Lunny Xiao2021-09-011-1/+2
|
* Gitlab Migrator: dont ignore reactions of last request (#16903)65432021-09-011-6/+12
| | | Fix bug related to early breaking when migrating reactions.
* Fix dump and restore respository (#16698)Lunny Xiao2021-08-302-2/+9
| | | | | | | | | | | * Fix dump and restore * return different error message for get commit * Fix missing delete release attachment when deleting repository * Fix ci and add some comments Co-authored-by: zeripath <art27@cantab.net>
* Add migrate from OneDev (#16356)KN4CK3R2021-08-2218-92/+926
| | | | | | | | | | | | | | * Use context to simplify logic. * Added migration from OneDev. This PR adds [OneDev](https://code.onedev.io/) as migration source. Supported: - [x] Milestones - [x] Issues - [x] Pull Requests - [x] Comments - [x] Reviews - [x] Labels
* Add proxy settings and support for migration and webhook (#16704)Lunny Xiao2021-08-184-7/+55
| | | | | | | | | | | | | | | | | | | | | | | | | | * Add proxy settings and support for migration and webhook * Fix default value * Add newline for example ini * Add lfs proxy support * Fix lint * Follow @zeripath's review * Fix git clone * Fix test * missgin http requests for proxy * use empty Co-authored-by: zeripath <art27@cantab.net> Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: zeripath <art27@cantab.net>
* Refactored and fixed migration tests. (#16714)KN4CK3R2021-08-1710-485/+744
| | | Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Update issue_index to finish migration (#16685)65432021-08-131-0/+5
| | | | | * update issue_index to finish migration * One Func to RecalculateIssueIndexForRepo
* [API] generalize list header (#16551)65432021-08-121-2/+2
| | | | | | | | | | | | | * Add info about list endpoints to CONTRIBUTING.md * Let all list endpoints return X-Total-Count header * Add TODOs for GetCombinedCommitStatusByRef * Fix models/issue_stopwatch.go * Rrefactor models.ListDeployKeys * Introduce helper func and use them for SetLinkHeader related func
* Upgrade github.com/google/go-github v32.1.0 -> v37.0.0 (#16661)65432021-08-102-157/+73
| | | | | * Upgrade github.com/google/go-github vv32.1.0 -> v37.0.0 * refactor: use GetX() func to reduce code
* Fix various documentation, user-facing, and source comment typos (#16367)luzpaz2021-07-086-6/+6
| | | | | * Fix various doc, user-facing, and source comment typos Found via `codespell -q 3 -S ./options/locale,./vendor -L ba,pullrequest,pullrequests,readby`
* Make the github migration less rate limit waiting to get comment per page ↵Lunny Xiao2021-06-3013-53/+191
| | | | | | | | | | | | | | | from repository but not per issue (#16070) * Make the github migration less rate limit waiting to get comment per page from repository but not per issue * Fix lint * adjust Downloader interface * Fix missed reviews * Fix test * Remove unused struct
* Add Status Updates whilst Gitea migrations are occurring (#15076)zeripath2021-06-164-6/+28
| | | | | | | | | | | | | | | | | | | | | | | | | | | * Add migrating message Signed-off-by: Andrew Thornton <art27@cantab.net> * simplify messenger Signed-off-by: Andrew Thornton <art27@cantab.net> * make messenger an interface Signed-off-by: Andrew Thornton <art27@cantab.net> * rename Signed-off-by: Andrew Thornton <art27@cantab.net> * prepare for merge Signed-off-by: Andrew Thornton <art27@cantab.net> * as per tech Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: 6543 <6543@obermui.de>
* small refactor for retry downloader (#16137)a10121127962021-06-131-134/+83
| | | Signed-off-by: a1012112796 <1012112796@qq.com>
* Fixed assert statements. (#16089)KN4CK3R2021-06-075-18/+18
|
* Copy git data from disk when restore repository (#16066)Lunny Xiao2021-06-043-6/+11
|
* Encrypt migration credentials at rest (#15895)Norwin2021-05-311-4/+7
| | | | | | | | | | | | | | | | | | | | | | | | * encrypt migration credentials in task persistence Not sure this is the best approach, we could encrypt the entire `PayloadContent` instead. Also instead of clearing individual fields in payload content, we could just delete the task once it has (successfully) finished..? * remove credentials of past migrations * only run DB migration for completed tasks * fix binding * add omitempty * never serialize unencrypted credentials * fix import order Co-authored-by: techknowlogick <techknowlogick@gitea.io> Co-authored-by: zeripath <art27@cantab.net> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Fix get tag when migration (#16014)Lunny Xiao2021-05-291-1/+1
| | | Co-authored-by: zeripath <art27@cantab.net>
* GitHub: migrate draft releases too (#15884)65432021-05-152-34/+31
| | | | | * GitHub: migrate draft releases too * refactor
* Close the gitrepo when deleting the repository (#15876)zeripath2021-05-141-0/+1
| | | | Signed-off-by: Andrew Thornton <art27@cantab.net>
* Move restore repo to internal router and invoke from command to avoid open ↵Lunny Xiao2021-05-101-11/+41
| | | | | | | | | | | the same db file or queues files (#15790) * Move restore repo to internal router and invoke from command to avoid open the same db file or queues files * Follow @zeripath's review * set no timeout for resotre repo private request * make restore repo cancelable
* migration: github: if rate limit is not enabled, ignore it (#15490)65432021-04-151-0/+5
|
* Add LFS Migration and Mirror (#14726)KN4CK3R2021-04-083-0/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Implemented LFS client. * Implemented scanning for pointer files. * Implemented downloading of lfs files. * Moved model-dependent code into services. * Removed models dependency. Added TryReadPointerFromBuffer. * Migrated code from service to module. * Centralised storage creation. * Removed dependency from models. * Moved ContentStore into modules. * Share structs between server and client. * Moved method to services. * Implemented lfs download on clone. * Implemented LFS sync on clone and mirror update. * Added form fields. * Updated templates. * Fixed condition. * Use alternate endpoint. * Added missing methods. * Fixed typo and make linter happy. * Detached pointer parser from gogit dependency. * Fixed TestGetLFSRange test. * Added context to support cancellation. * Use ReadFull to probably read more data. * Removed duplicated code from models. * Moved scan implementation into pointer_scanner_nogogit. * Changed method name. * Added comments. * Added more/specific log/error messages. * Embedded lfs.Pointer into models.LFSMetaObject. * Moved code from models to module. * Moved code from models to module. * Moved code from models to module. * Reduced pointer usage. * Embedded type. * Use promoted fields. * Fixed unexpected eof. * Added unit tests. * Implemented migration of local file paths. * Show an error on invalid LFS endpoints. * Hide settings if not used. * Added LFS info to mirror struct. * Fixed comment. * Check LFS endpoint. * Manage LFS settings from mirror page. * Fixed selector. * Adjusted selector. * Added more tests. * Added local filesystem migration test. * Fixed typo. * Reset settings. * Added special windows path handling. * Added unit test for HTTPClient. * Added unit test for BasicTransferAdapter. * Moved into util package. * Test if LFS endpoint is allowed. * Added support for git:// * Just use a static placeholder as the displayed url may be invalid. * Reverted to original code. * Added "Advanced Settings". * Updated wording. * Added discovery info link. * Implemented suggestion. * Fixed missing format parameter. * Added Pointer.IsValid(). * Always remove model on error. * Added suggestions. * Use channel instead of array. * Update routers/repo/migrate.go * fmt Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: zeripath <art27@cantab.net>
* Add size to Save function (#15264)zeripath2021-04-031-1/+1
| | | | | Fix #15253 Signed-off-by: Andrew Thornton <art27@cantab.net>
* [Vendor] update gitea-sdk v0.14.0 (#15103)65432021-03-221-22/+0
| | | | | * upgraded code.gitea.io/sdk/gitea v0.13.2 => v0.14.0 * rm workaround
* Do not convert file path to lowercase (#15023)KN4CK3R2021-03-182-3/+7
| | | | | | | | * Do not convert file path to lowercase. * lint * Check against lowercase hostname.
* Ensure validation occurs on clone addresses too (#14994)zeripath2021-03-152-24/+71
| | | | | | | | | | | | | | | | | | | | | * Ensure validation occurs on clone addresses too Fix #14984 Signed-off-by: Andrew Thornton <art27@cantab.net> * fix lint Signed-off-by: Andrew Thornton <art27@cantab.net> * fix test Signed-off-by: Andrew Thornton <art27@cantab.net> * Fix api tests Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Move Workaround for #12675 into it's own function (#14922)65432021-03-081-7/+22
| | | | | * Move Workatround for #12675 into it's own function * use more reliable solution (as tea do)
* Re-enable import local paths after reversion from #13610 (#14925)zeripath2021-03-082-0/+19
| | | | | | | | | PR #13610 unfortunately disabled importing repositories from local paths. This PR restores this functionality. Fix #14700 Signed-off-by: Andrew Thornton <art27@cantab.net>
* Fix a couple of CommentAsPatch issues. (#14804)zeripath2021-02-271-8/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | * CutDiffAroundLine makes the incorrect assumption that `---` and `+++` always represent part of the header of a diff. This PR adds a flag to its parsing to prevent this problem and adds a streaming parsing technique to CutDiffAroundLine using an io.pipe instead of just sending data to an unbounded buffer. Fix #14711 Signed-off-by: Andrew Thornton <art27@cantab.net> * Handle unquoted comment patch files When making comment patches unfortunately the patch does not always quote the filename This makes the diff --git header ambiguous again. This PR finally adds handling for ambiguity in to parse patch Fix #14812 Signed-off-by: Andrew Thornton <art27@cantab.net> * Add in testing for no error There is no way currently for CutDiffAroundLine in this test to cause an error however, it should still be tested. Signed-off-by: Andrew Thornton <art27@cantab.net>
* Fix repo-restore bug with poster not replaced (#14759)Lunny Xiao2021-02-222-11/+28
| | | | | | | | | * Fix restore bug * Fix restore bug Co-authored-by: Lauris BH <lauris@nix.lv> Co-authored-by: John Olheiser <john.olheiser@gmail.com> Co-authored-by: 6543 <6543@obermui.de>
* Fix github download on migration (#14703)Lunny Xiao2021-02-181-4/+19
| | | | | | | * Fix github download on migration * Use Context for Client Co-authored-by: zeripath <art27@cantab.net>
* Fix rate limit bug when downloading assets on migrating from github (#14564)Lunny Xiao2021-02-071-0/+5
|
* Add support to migrate from gogs (#14342)65432021-01-2115-298/+860
| | | | | | | | | | | | | Add support to migrate gogs: * issues * comments * labels * milestones * wiki Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com> Co-authored-by: Andrew Thornton <art27@cantab.net>
* Add MirrorInterval to the API (#14163)Paul Barton2021-01-022-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Added MirrorInterval to the API * Remove MirrorInterval from CreateRepository * Removed Duplicate UpdateMirror Function * Updated Error Logging * Update Log Message for is not Mirror Co-authored-by: 6543 <6543@obermui.de> * Delete Debug Statement that snuck in Co-authored-by: zeripath <art27@cantab.net> * Add Check for If Interval is too small * Output to API Call * Add Error Object when time is Less than Min Interval * Frequency Error Message Co-authored-by: zeripath <art27@cantab.net> * Allow Zero Mirror Interval Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: zeripath <art27@cantab.net>
* Dump github/gitlab/gitea repository data to a local directory and restore to ↵Lunny Xiao2020-12-2720-216/+1101
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | gitea (#12244) * Dump github/gitlab repository data to a local directory * Fix lint * Adjust directory structure * Allow migration special units * Allow migration ignore release assets * Fix lint * Add restore repository * stage the changes * Merge * Fix lint * Update the interface * Add some restore methods * Finish restore * Add comments * Fix restore * Add a token flag * Fix bug * Fix test * Fix test * Fix bug * Fix bug * Fix lint * Fix restore * refactor downloader * fmt * Fix bug isEnd detection on getIssues * Refactor maxPerPage * Remove unused codes * Remove unused codes * Fix bug * Fix restore * Fix dump * Uploader should not depend downloader * use release attachment name but not id * Fix restore bug * Fix lint * Fix restore bug * Add a method of DownloadFunc for base.Release to make uploader not depend on downloader * fix Release yml marshal * Fix trace information * Fix bug when dump & restore * Save relative path on yml file * Fix bug * Use relative path * Update docs * Use git service string but not int * Recognize clone addr to service type
* Migration: drop to long repo topics (#14152)65432020-12-271-0/+9
|
* [API] Add Ref to Issue (#13946)zeripath2020-12-132-0/+2
| | | | | | | | | | * Add Ref to api.Issue This PR adds Ref to api.Issue and adds the backend parts to the migrator. Fix #13918 Signed-off-by: Andrew Thornton <art27@cantab.net>
* Migrations: Use Process Manager to create own Context (#13792)65432020-12-021-1/+1
|
* Add Allow-/Block-List for Migrate & Mirrors (#13610)65432020-11-282-1/+108
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * add black list and white list support for migrating repositories * fix fmt * fix lint * fix vendor * fix modules.txt * clean diff * specify log message * use blocklist/allowlist * allways use lowercase to match url * Apply allow/block * Settings: use existing "migrations" section * convert domains lower case * dont store unused value * Block private addresses for migration by default * fix lint * use proposed-upstream func to detect private IP addr * a nit * add own error for blocked migration, add tests, imprufe api * fix test * fix-if-localhost-is-ipv4 * rename error & error message * rename setting options * Apply suggestions from code review Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com> Co-authored-by: zeripath <art27@cantab.net> Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* finaly fix gitlab migration with subdir 2.0 (#13646)65432020-11-201-2/+6
| | | | | | | | | * final fix 2.0? * ignore Approvals for pulls if not found * CI.restart() Co-authored-by: Lauris BH <lauris@nix.lv>
* finaly fix gitlab migration with subdir (#13629)65432020-11-191-2/+9
| | | | | * finaly fix #13535 * add logging
* Migration: Gitlab: Support Subdirectory (#13563)65432020-11-161-0/+18
| | | Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Migrations should not fail for comment reactions (#13352)zeripath2020-10-291-1/+5
| | | | | An extension to #13444 - where we now ensure that comment reaction failures do not cause migrations failure Signed-off-by: Andrew Thornton <art27@cantab.net>
* Migration failure during reaction migration from gitea (#13344)zeripath2020-10-282-3/+12
| | | | | | | | | | | | | | * Migrating reactions is just not that important A failure during migrating reactions should not cause failure of migration. Signed-off-by: Andrew Thornton <art27@cantab.net> * When checking issue reactions check the correct permission Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Add migrated pulls to pull request task queue (#13331)zeripath2020-10-272-8/+10
| | | | | | | | | | | | | * Add migrated pulls to pull request task queue Fix #13321 Signed-off-by: Andrew Thornton <art27@cantab.net> * Improve error reports Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Fix bug isEnd detection on getIssues/getPullRequests (#13299)Lunny Xiao2020-10-252-32/+50
|
* Migrations: Gitea should not fail just because of no apiConfig return (#13229)65432020-10-221-6/+7
| | | | | | | | | | | | * close #13227 * log it :+1: Co-authored-by: zeripath <art27@cantab.net> Co-authored-by: zeripath <art27@cantab.net> Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Gitea 2 Gitea migration (#12657)65432020-10-1416-32/+1082
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * first draft * update gitea sdk to 9e280adb4da * adapt feat of updated sdk * releases now works * break the Reactions loop * use convertGiteaLabel * fix endless loop because paggination is not supported there !!! * rename gitea local uploader files * pagination can bite you in the ass * Version Checks * lint * docs * rename gitea sdk import to miss future conficts * go-swagger: dont scan the sdk structs * make sure gitea can shutdown gracefully * make GetPullRequests and GetIssues similar * rm useles * Add Test: started ... * ... add tests ... * Add tests and Fixing things * Workaround missing SHA * Adapt: Ensure that all migration requests are cancellable (714ab71ddc4260937b1480519d453d2dc4e77dd6) * LINT: fix misspells in test set * adapt ListMergeRequestAwardEmoji * update sdk * Return error when creating giteadownloader failed * update sdk * adapt new sdk * adopt new features * check version before err * adapt: 'migrate service type switch page' * optimize * Fix DefaultBranch * impruve * handle subPath * fix test * Fix ReviewCommentPosition * test GetReviews * add DefaultBranch int test set * rm unused * Update SDK to v0.13.0 * addopt sdk changes * found better link * format template * Update Docs * Update Gitea SDK (v0.13.1)
generated by cgit v1.2.3 (git 2.39.1) at 2025-01-21 11:24:51 +0000