summaryrefslogtreecommitdiffstats
path: root/models/migrations
Commit message (Collapse)AuthorAgeFilesLines
* Fix migration test (#28659)Lunny Xiao2023-12-301-9/+9
|
* Add combined index for issue_user.uid and issue_id (#28080)sebastian-sauer2023-12-145-0/+98
| | | | | | | | fixes #27877 --------- Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* handle repository.size column being NULL in migration v263 (#28336)Nate Levesque2023-12-051-1/+6
| | | | | | | | | | | | | | | | | | This resolves a problem I encountered while updating gitea from 1.20.4 to 1.21. For some reason (correct or otherwise) there are some values in `repository.size` that are NULL in my gitea database which cause this migration to fail due to the NOT NULL constraints. Log snippet (excuse the escape characters) ``` ESC[36mgitea |ESC[0m 2023-12-04T03:52:28.573122395Z 2023/12/04 03:52:28 ...ations/migrations.go:641:Migrate() [I] Migration[263]: Add git_size and lfs_size columns to repository table ESC[36mgitea |ESC[0m 2023-12-04T03:52:28.608705544Z 2023/12/04 03:52:28 routers/common/db.go:36:InitDBEngine() [E] ORM engine initialization attempt #3/10 failed. Error: migrate: migration[263]: Add git_size and lfs_size columns to repository table failed: NOT NULL constraint failed: repository.git_size ``` I assume this should be reasonably safe since `repository.git_size` has a default value of 0 but I don't know if that value being 0 in the odd situation where `repository.size == NULL` has any problematic consequences.
* Add Index to pull_auto_merge.doer_id (#27811)JakobDev2023-10-302-0/+18
| | | | | | | | Reported at https://codeberg.org/forgejo/forgejo/issues/1677 This column is used by [deleteUser](https://github.com/go-gitea/gitea/blob/f089be91da44ee3e85fbe326a204c7c8a5df3d31/services/user/delete.go#L90). Note: This PR contains a Migration, so we can't backport it to 1.21.
* Enhanced auth token / remember me (#27606)KN4CK3R2023-10-142-0/+23
| | | | | | | | | | | | | | | | | Closes #27455 > The mechanism responsible for long-term authentication (the 'remember me' cookie) uses a weak construction technique. It will hash the user's hashed password and the rands value; it will then call the secure cookie code, which will encrypt the user's name with the computed hash. If one were able to dump the database, they could extract those two values to rebuild that cookie and impersonate a user. That vulnerability exists from the date the dump was obtained until a user changed their password. > > To fix this security issue, the cookie could be created and verified using a different technique such as the one explained at https://paragonie.com/blog/2015/04/secure-authentication-php-with-long-term-persistence#secure-remember-me-cookies. The PR removes the now obsolete setting `COOKIE_USERNAME`.
* Restore warning commit status (#27504)silverwind2023-10-081-3/+0
| | | | | | | | | | | | | Partial revert of https://github.com/go-gitea/gitea/pull/25839. This commit status is used by a number of external integrations, so I think we should not remove it (See https://github.com/go-gitea/gitea/pull/25839#issuecomment-1729002077). This is a rare case where an existing migration needed to be alterted to avoid data loss. --------- Co-authored-by: delvh <dev.lh@web.de> Co-authored-by: Giteabot <teabot@gitea.io>
* Rename the default themes to gitea-light, gitea-dark, gitea-auto (#27419)silverwind2023-10-062-0/+35
| | | | | | | | | | | | | | | | | | | | | | | | | Part of https://github.com/go-gitea/gitea/issues/27097: - `gitea` theme is renamed to `gitea-light` - `arc-green` theme is renamed to `gitea-dark` - `auto` theme is renamed to `gitea-auto` I put both themes in separate CSS files, removing all colors from the base CSS. Existing users will be migrated to the new theme names. The dark theme recolor will follow in a separate PR. ## :warning: BREAKING :warning: 1. If there are existing custom themes with the names `gitea-light` or `gitea-dark`, rename them before this upgrade and update the `theme` column in the `user` table for each affected user. 2. The theme in `<html>` has moved from `class="theme-name"` to `data-theme="name"`, existing customizations that depend on should be updated. --------- Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com> Co-authored-by: Giteabot <teabot@gitea.io>
* Refactor system setting (#27000)wxiaoguang2023-10-051-41/+1
| | | | | | | | | This PR reduces the complexity of the system setting system. It only needs one line to introduce a new option, and the option can be used anywhere out-of-box. It is still high-performant (and more performant) because the config values are cached in the config system.
* Add Index to `action.user_id` (#27403)JakobDev2023-10-032-0/+18
| | | | | | Another Column that needs a Index. Found at https://codeberg.org/forgejo/discussions/issues/61#issuecomment-1258744. Co-authored-by: Giteabot <teabot@gitea.io>
* Add Index to `comment.dependent_issue_id` (#27325)JakobDev2023-09-282-0/+18
| | | | | | This Column is missing index. It is used by [issue_service.deleteIssue](https://github.com/go-gitea/gitea/blob/7ea2a910cebaf51cfd13c0941029c404e408ae54/services/issue/issue.go#L300). Co-authored-by: Giteabot <teabot@gitea.io>
* Fix more yaml lint errors (#27284)Lunny Xiao2023-09-264-4/+1
| | | Fix #27268
* Add index to `issue_user.issue_id` (#27154)JakobDev2023-09-212-0/+18
| | | | | | | | | | | | This fixes a performance bottleneck. It was discovered by Codeberg. Every where query on that table (which has grown big over time) uses this column, but there is no index on it. See this part of the log which was posted on Matrix: ``` 2023/09/10 00:52:01 ...rs/web/repo/issue.go:1446:ViewIssue() [W] [Slow SQL Query] UPDATE `issue_user` SET is_read=? WHERE uid=? AND issue_id=? [true x y] - 51.395434887s 2023/09/10 00:52:01 ...rs/web/repo/issue.go:1447:ViewIssue() [E] ReadBy: Error 1205 (HY000): Lock wait timeout exceeded; try restarting transaction 2023/09/10 00:52:01 ...eb/routing/logger.go:102:func1() [I] router: completed GET /Codeberg/Community/issues/1201 for [::ffff:xxx]:0, 500 Internal Server Error in 52384.2ms @ repo/issue.go:1256(repo.ViewIssue) ```
* Fix wrong xorm get usage on migration (#27111)Lunny Xiao2023-09-181-19/+14
| | | | | | | | | | | | | | | | | | Fix the bug on try.gitea.io ```log 2023/09/18 01:48:41 ...ations/migrations.go:635:Migrate() [I] Migration[276]: Add RemoteAddress to mirrors 2023/09/18 01:48:41 routers/common/db.go:34:InitDBEngine() [E] ORM engine initialization attempt #7/10 failed. Error: migrate: migration[276]: Add RemoteAddress to mirrors failed: exit status 128 - fatal: not a git repository (or any parent up to mount point /) Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set). - fatal: not a git repository (or any parent up to mount point /) Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set). ``` Caused by #26952 --------- Co-authored-by: Jason Song <i@wolfogre.com>
* Fix wrong migration for email address (#27106)Lunny Xiao2023-09-171-18/+70
| | | | | On Iterate, `sess` should not be used in the closure function body. Caused by #26952
* Add `RemoteAddress` to mirrors (#26952)KN4CK3R2023-09-162-0/+134
| | | | | This PR adds a new field `RemoteAddress` to both mirror types which contains the sanitized remote address for easier (database) access to that information. Will be used in the audit PR if merged.
* Add a new column schedule_id for action_run to track (#26975)Lunny Xiao2023-09-082-0/+17
| | | | | | | | Fix #26971 And the UI now will display it's scheduled but not triggered by a push. <img width="954" alt="图片" src="https://github.com/go-gitea/gitea/assets/81045/d211845c-457e-4c3e-af1f-a0d654d3f365">
* Artifacts retention and auto clean up (#26131)FuXiaoHei2023-09-062-0/+38
| | | | | | | | | | | | | | Currently, Artifact does not have an expiration and automatic cleanup mechanism, and this feature needs to be added. It contains the following key points: - [x] add global artifact retention days option in config file. Default value is 90 days. - [x] add cron task to clean up expired artifacts. It should run once a day. - [x] support custom retention period from `retention-days: 5` in `upload-artifact@v3`. - [x] artifacts link in actions view should be non-clickable text when expired.
* Replace deprecated `elliptic.Marshal` (#26800)Chongyi Zheng2023-08-291-2/+5
| | | | | | | | | | | | | In PR #26786, the Go version for golangci-lint is bumped to 1.21. This causes the following error: ``` models/migrations/v1_16/v210.go:132:23: SA1019: elliptic.Marshal has been deprecated since Go 1.21: for ECDH, use the crypto/ecdh package. This function returns an encoding equivalent to that of PublicKey.Bytes in crypto/ecdh. (staticcheck) PublicKey: elliptic.Marshal(elliptic.P256(), parsed.PubKey.X, parsed.PubKey.Y), ``` The change now uses [func (*PublicKey) ECDH](https://pkg.go.dev/crypto/ecdsa#PublicKey.ECDH), which is added in Go 1.20.
* chore(actions): support cron schedule task (#26655)Lunny Xiao2023-08-242-0/+47
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Replace #22751 1. only support the default branch in the repository setting. 2. autoload schedule data from the schedule table after starting the service. 3. support specific syntax like `@yearly`, `@monthly`, `@weekly`, `@daily`, `@hourly` ## How to use See the [GitHub Actions document](https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#schedule) for getting more detailed information. ```yaml on: schedule: - cron: '30 5 * * 1,3' - cron: '30 5 * * 2,4' jobs: test_schedule: runs-on: ubuntu-latest steps: - name: Not on Monday or Wednesday if: github.event.schedule != '30 5 * * 1,3' run: echo "This step will be skipped on Monday and Wednesday" - name: Every time run: echo "This step will always run" ``` Signed-off-by: Bo-Yi.Wu <appleboy.tw@gmail.com> --------- Co-authored-by: Jason Song <i@wolfogre.com> Co-authored-by: techknowlogick <techknowlogick@gitea.io> Co-authored-by: wxiaoguang <wxiaoguang@gmail.com> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Add optimistic lock to ActionRun table (#26563)Jason Song2023-08-212-0/+16
| | | | | | | | | Should fix #26559. How xorm works: https://xorm.io/docs/chapter-06/1.lock/ --------- Co-authored-by: Giteabot <teabot@gitea.io>
* Allow to archive labels (#26478)puni98692023-08-142-0/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ## Archived labels This adds the structure to allow for archived labels. Archived labels are, just like closed milestones or projects, a medium to hide information without deleting it. It is especially useful if there are outdated labels that should no longer be used without deleting the label entirely. ## Changes 1. UI and API have been equipped with the support to mark a label as archived 2. The time when a label has been archived will be stored in the DB ## Outsourced for the future There's no special handling for archived labels at the moment. This will be done in the future. ## Screenshots ![image](https://github.com/go-gitea/gitea/assets/80308335/208f95cd-42e4-4ed7-9a1f-cd2050a645d4) ![image](https://github.com/go-gitea/gitea/assets/80308335/746428e0-40bb-45b3-b992-85602feb371d) Part of https://github.com/go-gitea/gitea/issues/25237 --------- Co-authored-by: delvh <dev.lh@web.de> Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
* Rename `Sync2` -> `Sync` (#26479)delvh2023-08-13115-176/+176
| | | | | | | | | The xorm `Sync2` has already been deprecated in favor of `Sync`, so let's do the same inside the Gitea codebase. Command used to replace everything: ```sh for i in $(ag Sync2 --files-with-matches); do vim $i -c ':%sno/Sync2/Sync/g' -c ':wq'; done ```
* Fix stderr usages (#26477)wxiaoguang2023-08-131-3/+1
|
* Fix typo in metadata (#26207)KN4CK3R2023-07-302-0/+28
| | | | | https://github.com/go-gitea/gitea/pull/26194#discussion_r1277094525 There is no need to backport because these names are just used internal.
* Prevent primary key update on migration (#26192)KN4CK3R2023-07-281-1/+1
| | | | | | | | | | | | | | | Fixes #25918 The migration fails on MSSQL because xorm tries to update the primary key column. xorm prevents this if the column is marked as auto increment: https://gitea.com/xorm/xorm/src/commit/c622cdaf893fbfe3f40a6b79f6bc17ee10f53865/internal/statements/update.go#L38-L40 I think it would be better if xorm would check for primary key columns here because updating such columns is bad practice. It looks like if that auto increment check should do the same. fyi @lunny
* Drop the correct deleted branch table (#26028)yp053272023-07-263-1/+15
| | | | There's a typo in #22743 The correct table name is `deleted_branch` not `deleted_branches`
* Implement auto-cancellation of concurrent jobs if the event is push (#25716)Bo-Yi Wu2023-07-252-0/+18
| | | | | | | | | | | | | | | | | | - cancel running jobs if the event is push - Add a new function `CancelRunningJobs` to cancel all running jobs of a run - Update `FindRunOptions` struct to include `Ref` field and update its condition in `toConds` function - Implement auto cancellation of running jobs in the same workflow in `notify` function related task: https://github.com/go-gitea/gitea/pull/22751/ --------- Signed-off-by: Bo-Yi Wu <appleboy.tw@gmail.com> Signed-off-by: appleboy <appleboy.tw@gmail.com> Co-authored-by: Jason Song <i@wolfogre.com> Co-authored-by: delvh <dev.lh@web.de>
* Reduce unnecessary DB queries for Actions tasks (#25199)sillyguodong2023-07-242-0/+25
| | | | | | | | | | | | | | | | | | | | | | | Close #24544 Changes: - Create `action_tasks_version` table to store the latest version of each scope (global, org and repo). - When a job with the status of `waiting` is created, the tasks version of the scopes it belongs to will increase. - When the status of a job already in the database is updated to `waiting`, the tasks version of the scopes it belongs to will increase. - On Gitea side, in `FeatchTask()`, will try to query the `action_tasks_version` record of the scope of the runner that call `FetchTask()`. If the record does not exist, will insert a row. Then, Gitea will compare the version passed from runner to Gitea with the version in database, if inconsistent, try pick task. Gitea always returns the latest version from database to the runner. Related: - Protocol: https://gitea.com/gitea/actions-proto-def/pulls/10 - Runner: https://gitea.com/gitea/act_runner/pulls/219
* Remove commit status running and warning to align GitHub (#25839)caicandong2023-07-212-0/+28
| | | | | | | | | | | | | | | | | | | | | | Fix #25776. Close #25826. In the discussion of #25776, @wolfogre's suggestion was to remove the commit status of `running` and `warning` to keep it consistent with github. references: - https://docs.github.com/en/rest/commits/statuses?apiVersion=2022-11-28#about-commit-statuses ## :warning: BREAKING :warning: So the commit status of Gitea will be consistent with GitHub, only `pending`, `success`, `error` and `failure`, while `warning` and `running` are not supported anymore. --------- Co-authored-by: Jason Song <i@wolfogre.com>
* Actions Artifacts support uploading multiple files and directories (#24874)FuXiaoHei2023-07-212-0/+21
| | | | | | | | | | | | | | current actions artifacts implementation only support single file artifact. To support multiple files uploading, it needs: - save each file to each db record with same run-id, same artifact-name and proper artifact-path - need change artifact uploading url without artifact-id, multiple files creates multiple artifact-ids - support `path` in download-artifact action. artifact should download to `{path}/{artifact-path}`. - in repo action view, it provides zip download link in artifacts list in summary page, no matter this artifact contains single or multiple files.
* Update tool dependencies, lock govulncheck and actionlint (#25655)silverwind2023-07-091-3/+3
| | | | | | | | | | - Update all tool dependencies - Lock `govulncheck` and `actionlint` to their latest tags --------- Co-authored-by: 6543 <m.huber@kithara.com> Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
* Less naked returns (#25713)65432023-07-074-37/+36
| | | | | just a step towards #25655 and some related refactoring
* Replace `interface{}` with `any` (#25686)silverwind2023-07-048-25/+25
| | | | | Result of running `perl -p -i -e 's#interface\{\}#any#g' **/*` and `make fmt`. Basically the same [as golang did](https://github.com/golang/go/commit/2580d0e08d5e9f979b943758d3c49877fb2324cb).
* Sync branches into databases (#22743)Lunny Xiao2023-06-292-0/+95
| | | | | | | | | | | | | | | | | | | | | | | | | | Related #14180 Related #25233 Related #22639 Close #19786 Related #12763 This PR will change all the branches retrieve method from reading git data to read database to reduce git read operations. - [x] Sync git branches information into database when push git data - [x] Create a new table `Branch`, merge some columns of `DeletedBranch` into `Branch` table and drop the table `DeletedBranch`. - [x] Read `Branch` table when visit `code` -> `branch` page - [x] Read `Branch` table when list branch names in `code` page dropdown - [x] Read `Branch` table when list git ref compare page - [x] Provide a button in admin page to manually sync all branches. - [x] Sync branches if repository is not empty but database branches are empty when visiting pages with branches list - [x] Use `commit_time desc` as the default FindBranch order by to keep consistent as before and deleted branches will be always at the end. --------- Co-authored-by: Jason Song <i@wolfogre.com>
* Split lfs size from repository size (#22900)a10121127962023-06-282-0/+43
| | | | | | | | | | | | | | | | | | | | releated to #21820 - Split `Size` in repository table as two new colunms, one is `GitSize` for git size, the other is `LFSSize` for lfs data. still store full size in `Size` colunm. - Show full size on ui, but show each of them by a `title`; example: ![image](https://user-images.githubusercontent.com/25342410/218636251-e200f085-d7e7-4a25-9ff1-b586a63e07a9.png) - Return full size in api response. --------- Signed-off-by: a1012112796 <1012112796@qq.com> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com> Co-authored-by: silverwind <me@silverwind.io> Co-authored-by: DmitryFrolovTri <23313323+DmitryFrolovTri@users.noreply.github.com> Co-authored-by: Giteabot <teabot@gitea.io>
* Support `pull_request_target` event (#25229)Zettat1232023-06-262-1/+18
| | | | | | | | | | | Fix #25088 This PR adds the support for [`pull_request_target`](https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#pull_request_target) workflow trigger. `pull_request_target` is similar to `pull_request`, but the workflow triggered by the `pull_request_target` event runs in the context of the base branch of the pull request rather than the head branch. Since the workflow from the base is considered trusted, it can access the secrets and doesn't need approvals to run.
* Refactor path & config system (#25330)wxiaoguang2023-06-211-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | # The problem There were many "path tricks": * By default, Gitea uses its program directory as its work path * Gitea tries to use the "work path" to guess its "custom path" and "custom conf (app.ini)" * Users might want to use other directories as work path * The non-default work path should be passed to Gitea by GITEA_WORK_DIR or "--work-path" * But some Gitea processes are started without these values * The "serv" process started by OpenSSH server * The CLI sub-commands started by site admin * The paths are guessed by SetCustomPathAndConf again and again * The default values of "work path / custom path / custom conf" can be changed when compiling # The solution * Use `InitWorkPathAndCommonConfig` to handle these path tricks, and use test code to cover its behaviors. * When Gitea's web server runs, write the WORK_PATH to "app.ini", this value must be the most correct one, because if this value is not right, users would find that the web UI doesn't work and then they should be able to fix it. * Then all other sub-commands can use the WORK_PATH in app.ini to initialize their paths. * By the way, when Gitea starts for git protocol, it shouldn't output any log, otherwise the git protocol gets broken and client blocks forever. The "work path" priority is: WORK_PATH in app.ini > cmd arg --work-path > env var GITEA_WORK_DIR > builtin default The "app.ini" searching order is: cmd arg --config > cmd arg "work path / custom path" > env var "work path / custom path" > builtin default ## ⚠️ BREAKING If your instance's "work path / custom path / custom conf" doesn't meet the requirements (eg: work path must be absolute), Gitea will report a fatal error and exit. You need to set these values according to the error log. ---- Close #24818 Close #24222 Close #21606 Close #21498 Close #25107 Close #24981 Maybe close #24503 Replace #23301 Replace #22754 And maybe more
* Support configuration variables on Gitea Actions (#24724)sillyguodong2023-06-202-0/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Co-Author: @silverwind @wxiaoguang Replace: #24404 See: - [defining configuration variables for multiple workflows](https://docs.github.com/en/actions/learn-github-actions/variables#defining-configuration-variables-for-multiple-workflows) - [vars context](https://docs.github.com/en/actions/learn-github-actions/contexts#vars-context) Related to: - [x] protocol: https://gitea.com/gitea/actions-proto-def/pulls/7 - [x] act_runner: https://gitea.com/gitea/act_runner/pulls/157 - [x] act: https://gitea.com/gitea/act/pulls/43 #### Screenshoot Create Variable: ![image](https://user-images.githubusercontent.com/33891828/236758288-032b7f64-44e7-48ea-b07d-de8b8b0e3729.png) ![image](https://user-images.githubusercontent.com/33891828/236758174-5203f64c-1d0e-4737-a5b0-62061dee86f8.png) Workflow: ```yaml test_vars: runs-on: ubuntu-latest steps: - name: Print Custom Variables run: echo "${{ vars.test_key }}" - name: Try to print a non-exist var run: echo "${{ vars.NON_EXIST_VAR }}" ``` Actions Log: ![image](https://user-images.githubusercontent.com/33891828/236759075-af0c5950-368d-4758-a8ac-47a96e43b6e2.png) --- This PR just implement the org / user (depends on the owner of the current repository) and repo level variables, The Environment level variables have not been implemented. Because [Environment](https://docs.github.com/en/actions/deployment/targeting-different-environments/using-environments-for-deployment#about-environments) is a module separate from `Actions`. Maybe it would be better to create a new PR to do it. --------- Co-authored-by: silverwind <me@silverwind.io> Co-authored-by: wxiaoguang <wxiaoguang@gmail.com> Co-authored-by: Giteabot <teabot@gitea.io>
* Add missing `v` in migrations.go (#25252)yp053272023-06-141-2/+2
|
* Fix description of drop custom_labels migration (#25243)sillyguodong2023-06-142-3/+3
| | | Follow: #24806
* Fix all possible setting error related storages and added some tests (#23911)Lunny Xiao2023-06-143-6/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | Follow up #22405 Fix #20703 This PR rewrites storage configuration read sequences with some breaks and tests. It becomes more strict than before and also fixed some inherit problems. - Move storage's MinioConfig struct into setting, so after the configuration loading, the values will be stored into the struct but not still on some section. - All storages configurations should be stored on one section, configuration items cannot be overrided by multiple sections. The prioioty of configuration is `[attachment]` > `[storage.attachments]` | `[storage.customized]` > `[storage]` > `default` - For extra override configuration items, currently are `SERVE_DIRECT`, `MINIO_BASE_PATH`, `MINIO_BUCKET`, which could be configured in another section. The prioioty of the override configuration is `[attachment]` > `[storage.attachments]` > `default`. - Add more tests for storages configurations. - Update the storage documentations. --------- Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
* Support changing labels of Actions runner without re-registration (#24806)sillyguodong2023-06-133-0/+46
| | | | | | | | | | | | | | | | | | | close #24540 related: - Protocol: https://gitea.com/gitea/actions-proto-def/pulls/9 - Runner side: https://gitea.com/gitea/act_runner/pulls/201 changes: - Add column of `labels` to table `action_runner`, and combine the value of `agent_labels` and `custom_labels` column to `labels` column. - Store `labels` when registering `act_runner`. - Update `labels` when `act_runner` starting and calling `Declare`. - Users cannot modify the `custom labels` in edit page any more. other changes: - Store `version` when registering `act_runner`. - If runner is latest version, parse version from `Declare`. But older version runner still parse version from request header.
* Redesign Scoped Access Tokens (#24767)Jack Hay2023-06-044-0/+486
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ## Changes - Adds the following high level access scopes, each with `read` and `write` levels: - `activitypub` - `admin` (hidden if user is not a site admin) - `misc` - `notification` - `organization` - `package` - `issue` - `repository` - `user` - Adds new middleware function `tokenRequiresScopes()` in addition to `reqToken()` - `tokenRequiresScopes()` is used for each high-level api section - _if_ a scoped token is present, checks that the required scope is included based on the section and HTTP method - `reqToken()` is used for individual routes - checks that required authentication is present (but does not check scope levels as this will already have been handled by `tokenRequiresScopes()` - Adds migration to convert old scoped access tokens to the new set of scopes - Updates the user interface for scope selection ### User interface example <img width="903" alt="Screen Shot 2023-05-31 at 1 56 55 PM" src="https://github.com/go-gitea/gitea/assets/23248839/654766ec-2143-4f59-9037-3b51600e32f3"> <img width="917" alt="Screen Shot 2023-05-31 at 1 56 43 PM" src="https://github.com/go-gitea/gitea/assets/23248839/1ad64081-012c-4a73-b393-66b30352654c"> ## tokenRequiresScopes Design Decision - `tokenRequiresScopes()` was added to more reliably cover api routes. For an incoming request, this function uses the given scope category (say `AccessTokenScopeCategoryOrganization`) and the HTTP method (say `DELETE`) and verifies that any scoped tokens in use include `delete:organization`. - `reqToken()` is used to enforce auth for individual routes that require it. If a scoped token is not present for a request, `tokenRequiresScopes()` will not return an error ## TODO - [x] Alphabetize scope categories - [x] Change 'public repos only' to a radio button (private vs public). Also expand this to organizations - [X] Disable token creation if no scopes selected. Alternatively, show warning - [x] `reqToken()` is missing from many `POST/DELETE` routes in the api. `tokenRequiresScopes()` only checks that a given token has the correct scope, `reqToken()` must be used to check that a token (or some other auth) is present. - _This should be addressed in this PR_ - [x] The migration should be reviewed very carefully in order to minimize access changes to existing user tokens. - _This should be addressed in this PR_ - [x] Link to api to swagger documentation, clarify what read/write/delete levels correspond to - [x] Review cases where more than one scope is needed as this directly deviates from the api definition. - _This should be addressed in this PR_ - For example: ```go m.Group("/users/{username}/orgs", func() { m.Get("", reqToken(), org.ListUserOrgs) m.Get("/{org}/permissions", reqToken(), org.GetUserOrgsPermissions) }, tokenRequiresScopes(auth_model.AccessTokenScopeCategoryUser, auth_model.AccessTokenScopeCategoryOrganization), context_service.UserAssignmentAPI()) ``` ## Future improvements - [ ] Add required scopes to swagger documentation - [ ] Redesign `reqToken()` to be opt-out rather than opt-in - [ ] Subdivide scopes like `repository` - [ ] Once a token is created, if it has no scopes, we should display text instead of an empty bullet point - [ ] If the 'public repos only' option is selected, should read categories be selected by default Closes #24501 Closes #24799 Co-authored-by: Jonathan Tran <jon@allspice.io> Co-authored-by: Kyle D <kdumontnu@gmail.com> Co-authored-by: silverwind <me@silverwind.io>
* Add the ability to pin Issues (#24406)JakobDev2023-05-252-0/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This adds the ability to pin important Issues and Pull Requests. You can also move pinned Issues around to change their Position. Resolves #2175. ## Screenshots ![grafik](https://user-images.githubusercontent.com/15185051/235123207-0aa39869-bb48-45c3-abe2-ba1e836046ec.png) ![grafik](https://user-images.githubusercontent.com/15185051/235123297-152a16ea-a857-451d-9a42-61f2cd54dd75.png) ![grafik](https://user-images.githubusercontent.com/15185051/235640782-cbfe25ec-6254-479a-a3de-133e585d7a2d.png) The Design was mostly copied from the Projects Board. ## Implementation This uses a new `pin_order` Column in the `issue` table. If the value is set to 0, the Issue is not pinned. If it's set to a bigger value, the value is the Position. 1 means it's the first pinned Issue, 2 means it's the second one etc. This is dived into Issues and Pull requests for each Repo. ## TODO - [x] You can currently pin as many Issues as you want. Maybe we should add a Limit, which is configurable. GitHub uses 3, but I prefer 6, as this is better for bigger Projects, but I'm open for suggestions. - [x] Pin and Unpin events need to be added to the Issue history. - [x] Tests - [x] Migration **The feature itself is currently fully working, so tester who may find weird edge cases are very welcome!** --------- Co-authored-by: silverwind <me@silverwind.io> Co-authored-by: Giteabot <teabot@gitea.io>
* Rewrite logger system (#24726)wxiaoguang2023-05-211-2/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ## ⚠️ Breaking The `log.<mode>.<logger>` style config has been dropped. If you used it, please check the new config manual & app.example.ini to make your instance output logs as expected. Although many legacy options still work, it's encouraged to upgrade to the new options. The SMTP logger is deleted because SMTP is not suitable to collect logs. If you have manually configured Gitea log options, please confirm the logger system works as expected after upgrading. ## Description Close #12082 and maybe more log-related issues, resolve some related FIXMEs in old code (which seems unfixable before) Just like rewriting queue #24505 : make code maintainable, clear legacy bugs, and add the ability to support more writers (eg: JSON, structured log) There is a new document (with examples): `logging-config.en-us.md` This PR is safer than the queue rewriting, because it's just for logging, it won't break other logic. ## The old problems The logging system is quite old and difficult to maintain: * Unclear concepts: Logger, NamedLogger, MultiChannelledLogger, SubLogger, EventLogger, WriterLogger etc * Some code is diffuclt to konw whether it is right: `log.DelNamedLogger("console")` vs `log.DelNamedLogger(log.DEFAULT)` vs `log.DelLogger("console")` * The old system heavily depends on ini config system, it's difficult to create new logger for different purpose, and it's very fragile. * The "color" trick is difficult to use and read, many colors are unnecessary, and in the future structured log could help * It's difficult to add other log formats, eg: JSON format * The log outputer doesn't have full control of its goroutine, it's difficult to make outputer have advanced behaviors * The logs could be lost in some cases: eg: no Fatal error when using CLI. * Config options are passed by JSON, which is quite fragile. * INI package makes the KEY in `[log]` section visible in `[log.sub1]` and `[log.sub1.subA]`, this behavior is quite fragile and would cause more unclear problems, and there is no strong requirement to support `log.<mode>.<logger>` syntax. ## The new design See `logger.go` for documents. ## Screenshot <details> ![image](https://github.com/go-gitea/gitea/assets/2114189/4462d713-ba39-41f5-bb08-de912e67e1ff) ![image](https://github.com/go-gitea/gitea/assets/2114189/b188035e-f691-428b-8b2d-ff7b2199b2f9) ![image](https://github.com/go-gitea/gitea/assets/2114189/132e9745-1c3b-4e00-9e0d-15eaea495dee) </details> ## TODO * [x] add some new tests * [x] fix some tests * [x] test some sub-commands (manually ....) --------- Co-authored-by: Jason Song <i@wolfogre.com> Co-authored-by: delvh <dev.lh@web.de> Co-authored-by: Giteabot <teabot@gitea.io>
* Implement actions artifacts (#22738)FuXiaoHei2023-05-192-0/+35
| | | | | | | | | | | | | | | | | | | | | | | | | | Implement action artifacts server api. This change is used for supporting https://github.com/actions/upload-artifact and https://github.com/actions/download-artifact in gitea actions. It can run sample workflow from doc https://docs.github.com/en/actions/using-workflows/storing-workflow-data-as-artifacts. The api design is inspired by https://github.com/nektos/act/blob/master/pkg/artifacts/server.go and includes some changes from gitea internal structs and methods. Actions artifacts contains two parts: - Gitea server api and storage (this pr implement basic design without some complex cases supports) - Runner communicate with gitea server api (in comming) Old pr https://github.com/go-gitea/gitea/pull/22345 is outdated after actions merged. I create new pr from main branch. ![897f7694-3e0f-4f7c-bb4b-9936624ead45](https://user-images.githubusercontent.com/2142787/219382371-eb3cf810-e4e0-456b-a8ff-aecc2b1a1032.jpeg) Add artifacts list in actions workflow page.
* Rewrite queue (#24505)wxiaoguang2023-05-082-185/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | # ⚠️ Breaking Many deprecated queue config options are removed (actually, they should have been removed in 1.18/1.19). If you see the fatal message when starting Gitea: "Please update your app.ini to remove deprecated config options", please follow the error messages to remove these options from your app.ini. Example: ``` 2023/05/06 19:39:22 [E] Removed queue option: `[indexer].ISSUE_INDEXER_QUEUE_TYPE`. Use new options in `[queue.issue_indexer]` 2023/05/06 19:39:22 [E] Removed queue option: `[indexer].UPDATE_BUFFER_LEN`. Use new options in `[queue.issue_indexer]` 2023/05/06 19:39:22 [F] Please update your app.ini to remove deprecated config options ``` Many options in `[queue]` are are dropped, including: `WRAP_IF_NECESSARY`, `MAX_ATTEMPTS`, `TIMEOUT`, `WORKERS`, `BLOCK_TIMEOUT`, `BOOST_TIMEOUT`, `BOOST_WORKERS`, they can be removed from app.ini. # The problem The old queue package has some legacy problems: * complexity: I doubt few people could tell how it works. * maintainability: Too many channels and mutex/cond are mixed together, too many different structs/interfaces depends each other. * stability: due to the complexity & maintainability, sometimes there are strange bugs and difficult to debug, and some code doesn't have test (indeed some code is difficult to test because a lot of things are mixed together). * general applicability: although it is called "queue", its behavior is not a well-known queue. * scalability: it doesn't seem easy to make it work with a cluster without breaking its behaviors. It came from some very old code to "avoid breaking", however, its technical debt is too heavy now. It's a good time to introduce a better "queue" package. # The new queue package It keeps using old config and concept as much as possible. * It only contains two major kinds of concepts: * The "base queue": channel, levelqueue, redis * They have the same abstraction, the same interface, and they are tested by the same testing code. * The "WokerPoolQueue", it uses the "base queue" to provide "worker pool" function, calls the "handler" to process the data in the base queue. * The new code doesn't do "PushBack" * Think about a queue with many workers, the "PushBack" can't guarantee the order for re-queued unhandled items, so in new code it just does "normal push" * The new code doesn't do "pause/resume" * The "pause/resume" was designed to handle some handler's failure: eg: document indexer (elasticsearch) is down * If a queue is paused for long time, either the producers blocks or the new items are dropped. * The new code doesn't do such "pause/resume" trick, it's not a common queue's behavior and it doesn't help much. * If there are unhandled items, the "push" function just blocks for a few seconds and then re-queue them and retry. * The new code doesn't do "worker booster" * Gitea's queue's handlers are light functions, the cost is only the go-routine, so it doesn't make sense to "boost" them. * The new code only use "max worker number" to limit the concurrent workers. * The new "Push" never blocks forever * Instead of creating more and more blocking goroutines, return an error is more friendly to the server and to the end user. There are more details in code comments: eg: the "Flush" problem, the strange "code.index" hanging problem, the "immediate" queue problem. Almost ready for review. TODO: * [x] add some necessary comments during review * [x] add some more tests if necessary * [x] update documents and config options * [x] test max worker / active worker * [x] re-run the CI tasks to see whether any test is flaky * [x] improve the `handleOldLengthConfiguration` to provide more friendly messages * [x] fine tune default config values (eg: length?) ## Code coverage: ![image](https://user-images.githubusercontent.com/2114189/236620635-55576955-f95d-4810-b12f-879026a3afdf.png)
* Merge setting.InitXXX into one function with options (#24389)Lunny Xiao2023-05-041-1/+1
| | | | This PR will merge 3 Init functions on setting packages as 1 and introduce an options struct.
* Add Debian package registry (#24426)KN4CK3R2023-05-022-0/+25
| | | | | | | | | | | | | | | | | | | | | | | | Co-authored-by: @awkwardbunny This PR adds a Debian package registry. You can follow [this tutorial](https://www.baeldung.com/linux/create-debian-package) to build a *.deb package for testing. Source packages are not supported at the moment and I did not find documentation of the architecture "all" and how these packages should be treated. ![grafik](https://user-images.githubusercontent.com/1666336/218126879-eb80a866-775c-4c8e-8529-5797203a64e6.png) Part of #20751. Revised copy of #22854. --------- Co-authored-by: Brian Hong <brian@hongs.me> Co-authored-by: techknowlogick <techknowlogick@gitea.io> Co-authored-by: Giteabot <teabot@gitea.io>
* Revert "Add Debian package registry" (#24412)Yarden Shoham2023-04-282-25/+0
| | | Reverts go-gitea/gitea#22854