aboutsummaryrefslogtreecommitdiffstats
path: root/cmd/migrate_storage.go
Commit message (Collapse)AuthorAgeFilesLines
* Fix migrate storage bug (#28830)Lunny Xiao2024-01-201-0/+6
|
* fix: typo (#27009)MomentQYC2023-09-111-1/+1
|
* Refactor to use urfave/cli/v2 (#25959)wxiaoguang2023-07-211-23/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Replace #10912 And there are many new tests to cover the CLI behavior There were some concerns about the "option order in hook scripts" (https://github.com/go-gitea/gitea/pull/10912#issuecomment-1137543314), it's not a problem now. Because the hook script uses `/gitea hook --config=/app.ini pre-receive` format. The "config" is a global option, it can appear anywhere. ---- ## ⚠️ BREAKING ⚠️ This PR does it best to avoid breaking anything. The major changes are: * `gitea` itself won't accept web's options: `--install-port` / `--pid` / `--port` / `--quiet` / `--verbose` .... They are `web` sub-command's options. * Use `./gitea web --pid ....` instead * `./gitea` can still run the `web` sub-command as shorthand, with default options * The sub-command's options must follow the sub-command * Before: `./gitea --sub-opt subcmd` might equal to `./gitea subcmd --sub-opt` (well, might not ...) * After: only `./gitea subcmd --sub-opt` could be used * The global options like `--config` are not affected
* Fix all possible setting error related storages and added some tests (#23911)Lunny Xiao2023-06-141-13/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | Follow up #22405 Fix #20703 This PR rewrites storage configuration read sequences with some breaks and tests. It becomes more strict than before and also fixed some inherit problems. - Move storage's MinioConfig struct into setting, so after the configuration loading, the values will be stored into the struct but not still on some section. - All storages configurations should be stored on one section, configuration items cannot be overrided by multiple sections. The prioioty of configuration is `[attachment]` > `[storage.attachments]` | `[storage.customized]` > `[storage]` > `default` - For extra override configuration items, currently are `SERVE_DIRECT`, `MINIO_BASE_PATH`, `MINIO_BUCKET`, which could be configured in another section. The prioioty of the override configuration is `[attachment]` > `[storage.attachments]` > `default`. - Add more tests for storages configurations. - Update the storage documentations. --------- Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
* Support migrating storage for actions log via command line (#24679)Jason Song2023-05-121-1/+19
| | | | | | | Close #24677 --------- Co-authored-by: Giteabot <teabot@gitea.io>
* Make minio package support legacy MD5 checksum (#23768)wxiaoguang2023-03-281-8/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | A feedback from discord: https://discord.com/channels/322538954119184384/561007778139734027/1090185427115319386 Some storages like: * https://developers.cloudflare.com/r2/api/s3/api/ * https://www.backblaze.com/b2/docs/s3_compatible_api.html They do not support "x-amz-checksum-algorithm" header But minio recently uses that header with CRC32C by default. So we have to tell minio to use legacy MD5 checksum. I guess this needs to be backported because IIRC we 1.19 and 1.20 are using similar minio package. The minio package code for SendContentMD5 looks like this: <details> <img width="755" alt="image" src="https://user-images.githubusercontent.com/2114189/228186768-4f2f6f67-62b9-4aee-9251-5af714ad9674.png"> </details>
* Refactor the setting to make unit test easier (#22405)Lunny Xiao2023-02-201-1/+1
| | | | | | | | | | | | | | | | | | | | | | Some bugs caused by less unit tests in fundamental packages. This PR refactor `setting` package so that create a unit test will be easier than before. - All `LoadFromXXX` files has been splited as two functions, one is `InitProviderFromXXX` and `LoadCommonSettings`. The first functions will only include the code to create or new a ini file. The second function will load common settings. - It also renames all functions in setting from `newXXXService` to `loadXXXSetting` or `loadXXXFrom` to make the function name less confusing. - Move `XORMLog` to `SQLLog` because it's a better name for that. Maybe we should finally move these `loadXXXSetting` into the `XXXInit` function? Any idea? --------- Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: delvh <dev.lh@web.de>
* Implement FSFE REUSE for golang files (#21840)flynnnnnnnnnn2022-11-271-2/+1
| | | | | | | | | Change all license headers to comply with REUSE specification. Fix #16132 Co-authored-by: flynnnnnnnnnn <flynnnnnnnnnn@github> Co-authored-by: John Olheiser <john.olheiser@gmail.com>
* Merge db.Iterate and IterateObjects (#21641)Lunny Xiao2022-10-311-6/+6
| | | These two functions are similiar, merge them.
* fix hard-coded timeout and error panic in API archive download endpoint (#20925)Peter Gardfjäll2022-08-291-5/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | * fix hard-coded timeout and error panic in API archive download endpoint This commit updates the `GET /api/v1/repos/{owner}/{repo}/archive/{archive}` endpoint which prior to this PR had a couple of issues. 1. The endpoint had a hard-coded 20s timeout for the archiver to complete after which a 500 (Internal Server Error) was returned to client. For a scripted API client there was no clear way of telling that the operation timed out and that it should retry. 2. Whenever the timeout _did occur_, the code used to panic. This was caused by the API endpoint "delegating" to the same call path as the web, which uses a slightly different way of reporting errors (HTML rather than JSON for example). More specifically, `api/v1/repo/file.go#GetArchive` just called through to `web/repo/repo.go#Download`, which expects the `Context` to have a `Render` field set, but which is `nil` for API calls. Hence, a `nil` pointer error. The code addresses (1) by dropping the hard-coded timeout. Instead, any timeout/cancelation on the incoming `Context` is used. The code addresses (2) by updating the API endpoint to use a separate call path for the API-triggered archive download. This avoids producing HTML-errors on errors (it now produces JSON errors). Signed-off-by: Peter Gardfjäll <peter.gardfjall.work@gmail.com>
* Add migrate repo archiver and packages storage support on command line (#20757)Lunny Xiao2022-08-161-35/+48
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Add migrate repo archiver and packages storage support on command line * Fix typo * Use stdCtx * Use packageblob and fix command description * Add migrate packages unit tests * Fix comment year * Fix the migrate storage command line description * Update cmd/migrate_storage.go Co-authored-by: zeripath <art27@cantab.net> * Update cmd/migrate_storage.go Co-authored-by: zeripath <art27@cantab.net> * Update cmd/migrate_storage.go Co-authored-by: zeripath <art27@cantab.net> * Fix test Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: zeripath <art27@cantab.net>
* Move some code into models/git (#19879)Lunny Xiao2022-06-121-2/+2
| | | | | | | | | | | | | | | | | | | * Move access and repo permission to models/perm/access * fix test * Move some git related files into sub package models/git * Fix build * fix git test * move lfs to sub package * move more git related functions to models/git * Move functions sequence * Some improvements per @KN4CK3R and @delvh
* Some repository refactors (#17950)Lunny Xiao2021-12-121-1/+1
| | | | | | | | | * some repository refactors * remove unnecessary code * Fix test * Remove unnecessary banner
* Move repository model into models/repo (#17933)Lunny Xiao2021-12-101-1/+1
| | | | | | | | | | | | | | | * Some refactors related repository model * Move more methods out of repository * Move repository into models/repo * Fix test * Fix test * some improvements * Remove unnecessary function
* Improve install code to avoid low-level mistakes. (#17779)wxiaoguang2021-12-011-1/+0
| | | | | | | | | | | | | | * Improve install code to avoid low-level mistakes. If a user tries to do a re-install in a Gitea database, they gets a warning and double check. When Gitea runs, it never create empty app.ini automatically. Also some small (related) refactoring: * Refactor db.InitEngine related logic make it more clean (especially for the install code) * Move some i18n strings out from setting.go to make the setting.go can be easily maintained. * Show errors in CLI code if an incorrect app.ini is used. * APP_DATA_PATH is created when installing, and checked when starting (no empty directory is created any more).
* Move user related model into models/user (#17781)Lunny Xiao2021-11-241-1/+2
| | | | | | | | | | | | | * Move user related model into models/user * Fix lint for windows * Fix windows lint * Fix windows lint * Move some tests in models * Merge
* Move attachment into models/repo/ (#17650)Lunny Xiao2021-11-191-1/+2
| | | | | | | * Move attachment into models/repo/ * Fix test * Fix bug
* Fix bug on admin subcommand (#17533)Lunny Xiao2021-11-071-1/+4
| | | | | | | * Fix bug on admin subcommand * Add signals for all initDB Co-authored-by: Lauris BH <lauris@nix.lv>
* Rename db Engines related functions (#17481)Lunny Xiao2021-10-301-1/+1
| | | | | * Rename db Engines related functions Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
* Move db related basic functions to models/db (#17075)Lunny Xiao2021-09-191-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | * Move db related basic functions to models/db * Fix lint * Fix lint * Fix test * Fix lint * Fix lint * revert unnecessary change * Fix test * Fix wrong replace string * Use *Context * Correct committer spelling and fix wrong replaced words Co-authored-by: zeripath <art27@cantab.net>
* Always emit the configuration path (#17036)zeripath2021-09-131-0/+1
| | | | | | Often when handling problems it is not clear which configuration file Gitea is using. This PR simply ensures that the configuration file is emitted. Signed-off-by: Andrew Thornton <art27@cantab.net>
* Fix various documentation, user-facing, and source comment typos (#16367)luzpaz2021-07-081-1/+1
| | | | | * Fix various doc, user-facing, and source comment typos Found via `codespell -q 3 -S ./options/locale,./vendor -L ba,pullrequest,pullrequests,readby`
* Add --quiet and --verbose to gitea web to control initial logging (#16260)zeripath2021-06-261-4/+4
| | | | | | | | | | | | | | | | | | | | | | One of the repeatedly reported issues has been that gitea produces too much console logging during set up even if the console logger is turned off. Fundamentally this is due to some otherwise very helpful logging that has to occur before logging is set up. This has come to a head with the merging of #16243 where otherwise potentially helpful Trace logging in the git module now appears on the console. This PR proposes three things: 1. Change the initial default logger to Info not Trace. 2. Change the logging for the AppPath things to Info in recompense. 3. Add two new command line options to gitea web: --quiet and --verbose `gitea web -q` or `gitea web --quiet` will only log Fatal level initially. `gitea web -verbose` will log at Trace. Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Avatars and Repo avatars support storing in minio (#12516)Lunny Xiao2020-10-141-2/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Avatar support minio * Support repo avatar minio storage * Add missing migration * Fix bug * Fix test * Add test for minio store type on avatars and repo avatars; Add documents * Fix bug * Fix bug * Add back missed avatar link method * refactor codes * Simplify the codes * Code improvements * Fix lint * Fix test mysql * Fix test mysql * Fix test mysql * Fix settings * Fix test * fix test * Fix bug
* Provide self-registering storage system (#12978)zeripath2020-10-121-14/+23
| | | | | | | | | | | | | | | | | | | * Provide self-registering storage system Signed-off-by: Andrew Thornton <art27@cantab.net> * More simplification Signed-off-by: Andrew Thornton <art27@cantab.net> * Remove old strings from setting Signed-off-by: Andrew Thornton <art27@cantab.net> * oops attachments not attachment Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Add default storage configurations (#12813)Lunny Xiao2020-09-291-8/+9
| | | | Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: zeripath <art27@cantab.net>
* LFS support to be stored on minio (#12518)Lunny Xiao2020-09-081-31/+43
| | | | | | | | | | | | | | | | | * LFS support to be stored on minio * Fix test * Fix lint * Fix lint * Fix check * Fix test * Update documents and add migration for LFS * Fix some bugs
* Add a storage layer for attachments (#11387)Lunny Xiao2020-08-181-0/+147
* Add a storage layer for attachments * Fix some bug * fix test * Fix copyright head and lint * Fix bug * Add setting for minio and flags for migrate-storage * Add documents * fix lint * Add test for minio store type on attachments * fix test * fix test * Apply suggestions from code review Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com> * Add warning when storage migrated successfully * Fix drone * fix test * rebase * Fix test * display the error on console * Move minio test to amd64 since minio docker don't support arm64 * refactor the codes * add trace * Fix test * remove log on xorm * Fi download bug * Add a storage layer for attachments * Add setting for minio and flags for migrate-storage * fix lint * Add test for minio store type on attachments * Apply suggestions from code review Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com> * Fix drone * fix test * Fix test * display the error on console * Move minio test to amd64 since minio docker don't support arm64 * refactor the codes * add trace * Fix test * Add URL function to serve attachments directly from S3/Minio * Add ability to enable/disable redirection in attachment configuration * Fix typo * Add a storage layer for attachments * Add setting for minio and flags for migrate-storage * fix lint * Add test for minio store type on attachments * Apply suggestions from code review Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com> * Fix drone * fix test * Fix test * display the error on console * Move minio test to amd64 since minio docker don't support arm64 * don't change unrelated files * Fix lint * Fix build * update go.mod and go.sum * Use github.com/minio/minio-go/v6 * Remove unused function * Upgrade minio to v7 and some other improvements * fix lint * Fix go mod Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com> Co-authored-by: Tyler <tystuyfzand@gmail.com>
generated by cgit v1.2.3 (git 2.39.1) at 2025-01-04 08:29:16 +0000