Rowan Bohde [Thu, 21 Nov 2024 04:30:48 +0000 (22:30 -0600)]
disable gravatar in test (#32529)
When running e2e tests on flaky networks, gravatar can cause a timeout
and test failures. Turn off, and populate avatars on e2e test suite run
to make them reliable.
Rowan Bohde [Wed, 20 Nov 2024 15:24:09 +0000 (09:24 -0600)]
allow the actions user to login via the jwt token (#32527)
We have some actions that leverage the Gitea API that began receiving
401 errors, with a message that the user was not found. These actions
use the `ACTIONS_RUNTIME_TOKEN` env var in the actions job to
authenticate with the Gitea API. The format of this env var in actions
jobs changed with go-gitea/gitea/pull/28885 to be a JWT (with a
corresponding update to `act_runner`) Since it was a JWT, the OAuth
parsing logic attempted to parse it as an OAuth token, and would return
user not found, instead of falling back to look up the running task and
assigning it to the actions user.
Make ACTIONS_RUNTIME_TOKEN in action runners could be used,
attempting to parse Oauth JWTs. The code to parse potential old
`ACTION_RUNTIME_TOKEN` was kept in case someone is running an older
version of act_runner that doesn't support the Actions JWT.
Marcell Mars [Wed, 20 Nov 2024 14:22:48 +0000 (15:22 +0100)]
Support HTTP POST requests to `/userinfo`, aligning to OpenID Core specification (#32578)
This PR adds support for the HTTP POST requests to `/userinfo` endpoint.
While the OpenID Core specification says both are supported and
recommends using HTTP GET.
Kerwin Bryant [Wed, 20 Nov 2024 00:39:57 +0000 (08:39 +0800)]
Optimize installation-page experience (#32558)
![3000-gogitea-gitea-kiagpwhqbx1 ws-us116 gitpod io_
(1)](https://github.com/user-attachments/assets/7f9ff835-7122-420e-83a9-218a1b9c7030)
Highlight the path of the configuration file with a label-style emphasis
and provide a quick copy button.
Kerwin Bryant [Tue, 19 Nov 2024 06:57:55 +0000 (14:57 +0800)]
Fix a compilation error in the Gitpod environment (#32559)
When opening the latest code in **Gitpod** and running `make
lint-backend`, the following error occurs:
```bash
gitpod /workspace/gitea (main) $ make lint-backend
go run github.com/golangci/golangci-lint/cmd/golangci-lint@v1.60.3 run
# internal/profilerecord
compile: version "go1.23.1" does not match go tool version "go1.22.9"
# internal/goarch
compile: version "go1.23.1" does not match go tool version "go1.22.9"
# unicode/utf8
compile: version "go1.23.1" does not match go tool version "go1.22.9"
# internal/coverage/rtcov
compile: version "go1.23.1" does not match go tool version "go1.22.9"
# internal/byteorder
compile: version "go1.23.1" does not match go tool version "go1.22.9"
# cmp
compile: version "go1.23.1" does not match go tool version "go1.22.9"
# internal/itoa
compile: version "go1.23.1" does not match go tool version "go1.22.9"
# internal/race
compile: version "go1.23.1" does not match go tool version "go1.22.9"
# internal/goos
compile: version "go1.23.1" does not match go tool version "go1.22.9"
# internal/unsafeheader
compile: version "go1.23.1" does not match go tool version "go1.22.9"
# unicode
compile: version "go1.23.1" does not match go tool version "go1.22.9"
# internal/godebugs
compile: version "go1.23.1" does not match go tool version "go1.22.9"
# internal/asan
compile: version "go1.23.1" does not match go tool version "go1.22.9"
# math/bits
compile: version "go1.23.1" does not match go tool version "go1.22.9"
# internal/goexperiment
compile: version "go1.23.1" does not match go tool version "go1.22.9"
# internal/msan
compile: version "go1.23.1" does not match go tool version "go1.22.9"
# internal/runtime/atomic
compile: version "go1.23.1" does not match go tool version "go1.22.9"
# sync/atomic
compile: version "go1.23.1" does not match go tool version "go1.22.9"
# internal/runtime/syscall
compile: version "go1.23.1" does not match go tool version "go1.22.9"
# crypto/internal/alias
compile: version "go1.23.1" does not match go tool version "go1.22.9"
# encoding
compile: version "go1.23.1" does not match go tool version "go1.22.9"
# log/internal
compile: version "go1.23.1" does not match go tool version "go1.22.9"
# vendor/golang.org/x/crypto/cryptobyte/asn1
compile: version "go1.23.1" does not match go tool version "go1.22.9"
# github.com/golangci/golangci-lint/pkg/exitcodes
compile: version "go1.23.1" does not match go tool version "go1.22.9"
# internal/cpu
compile: version "go1.23.1" does not match go tool version "go1.22.9"
# unicode/utf16
compile: version "go1.23.1" does not match go tool version "go1.22.9"
# container/list
compile: version "go1.23.1" does not match go tool version "go1.22.9"
# crypto/subtle
compile: version "go1.23.1" does not match go tool version "go1.22.9"
# internal/goversion
compile: version "go1.23.1" does not match go tool version "go1.22.9"
# golang.org/x/exp/maps
compile: version "go1.23.1" does not match go tool version "go1.22.9"
# github.com/ccojocar/zxcvbn-go/match
compile: version "go1.23.1" does not match go tool version "go1.22.9"
# golang.org/x/exp/constraints
compile: version "go1.23.1" does not match go tool version "go1.22.9"
# golang.org/x/tools/internal/packagesinternal
compile: version "go1.23.1" does not match go tool version "go1.22.9"
# github.com/quasilyte/go-ruleguard/dsl/types
compile: version "go1.23.1" does not match go tool version "go1.22.9"
# vendor/golang.org/x/crypto/internal/alias
compile: version "go1.23.1" does not match go tool version "go1.22.9"
# internal/nettrace
compile: version "go1.23.1" does not match go tool version "go1.22.9"
# github.com/google/go-cmp/cmp/internal/flags
compile: version "go1.23.1" does not match go tool version "go1.22.9"
# github.com/gobwas/glob/util/runes
compile: version "go1.23.1" does not match go tool version "go1.22.9"
# internal/platform
compile: version "go1.23.1" does not match go tool version "go1.22.9"
# crypto/internal/boring/sig
compile: version "go1.23.1" does not match go tool version "go1.22.9"
# github.com/quasilyte/gogrep/internal/stdinfo
compile: version "go1.23.1" does not match go tool version "go1.22.9"
# github.com/daixiang0/gci/pkg/utils
compile: version "go1.23.1" does not match go tool version "go1.22.9"
# github.com/quasilyte/stdinfo
compile: version "go1.23.1" does not match go tool version "go1.22.9"
# github.com/Antonboom/testifylint/internal/testify
compile: version "go1.23.1" does not match go tool version "go1.22.9"
# hash/maphash
compile: version "go1.23.1" does not match go tool version "go1.22.9"
# github.com/nunnatsa/ginkgolinter/version
compile: version "go1.23.1" does not match go tool version "go1.22.9"
# google.golang.org/protobuf/internal/flags
compile: version "go1.23.1" does not match go tool version "go1.22.9"
make: *** [Makefile:413: lint-go] Error 1
```
wxiaoguang [Mon, 18 Nov 2024 05:25:42 +0000 (13:25 +0800)]
Refactor markup render system (#32533)
Remove unmaintainable sanitizer rules. No need to add special "class"
regexp rules anymore, use RenderInternal.SafeAttr instead, more details
(and examples) are in the tests
Lunny Xiao [Mon, 18 Nov 2024 03:06:25 +0000 (19:06 -0800)]
Refactor find forks and fix possible bugs that weak permissions check (#32528)
- Move models/GetForks to services/FindForks
- Add doer as a parameter of FindForks to check permissions
- Slight performance optimization for get forks API with batch loading
of repository units
- Add tests for forking repository to organizations
wxiaoguang [Fri, 15 Nov 2024 15:45:07 +0000 (23:45 +0800)]
Improve testing and try to fix MySQL hanging (#32515)
By some CI fine tunes (`run tests`), SQLite & MSSQL could complete
in about 12~13 minutes (before > 14), MySQL could complete in 18 minutes
(before: about 23 or even > 30)
Major changes:
1. use tmpfs for MySQL storage
1. run `make test-mysql` instead of `make integration-test-coverage`
because the code coverage is not really used at the moment.
1. refactor testlogger to make it more reliable and be able to report
stuck stacktrace
1. do not requeue failed items when a queue is being flushed (failed
items would keep failing and make flush uncompleted)
1. reduce the file sizes for testing
1. use math ChaCha20 random data instead of crypot/rand (for testing
purpose only)
1. no need to `DeleteRepository` in `TestLinguist`
1. other related refactoring to make code easier to maintain
Lunny Xiao [Fri, 15 Nov 2024 04:04:20 +0000 (20:04 -0800)]
Remove transaction for archive download (#32186)
Since there is a status column in the database, the transaction is
unnecessary when downloading an archive. The transaction is blocking
database operations, especially with SQLite.
Rowan Bohde [Thu, 14 Nov 2024 19:28:46 +0000 (13:28 -0600)]
Reduce integration test overhead (#32475)
In profiling integration tests, I found a couple places where per-test
overhead could be reduced:
* Avoiding disk IO by synchronizing instead of deleting & copying test
Git repository data. This saves ~100ms per test on my machine
* When flushing queues in `PrintCurrentTest`, invoke `FlushWithContext`
in a parallel.
wxiaoguang [Thu, 14 Nov 2024 18:48:41 +0000 (02:48 +0800)]
Remove jQuery import from some files (#32512)
Many files do not directly depend on jQuery now.
To clarify the usage: use `fomanticQuery` to operate Fomantic
components.
Then developers could focus on removing the remaining jQuery usages by
searching `import $` globally.
21 files now:
```
./components/RepoBranchTagSelector.vue:3:import $ from 'jquery';
./features/admin/common.ts:1:import $ from 'jquery';
./features/admin/emails.ts:1:import $ from 'jquery';
./features/common-button.ts:1:import $ from 'jquery';
./features/comp/ComboMarkdownEditor.ts:3:import $ from 'jquery'; (I am working on it, there will be a new PR)
./features/comp/LabelEdit.ts:1:import $ from 'jquery';
./features/notification.ts:1:import $ from 'jquery';
./features/org-team.ts:1:import $ from 'jquery';
./features/repo-code.ts:1:import $ from 'jquery';
./features/repo-common.ts:1:import $ from 'jquery';
./features/repo-diff.ts:1:import $ from 'jquery';
./features/repo-editor.ts:1:import $ from 'jquery';
./features/repo-issue-content.ts:1:import $ from 'jquery';
./features/repo-issue-list.ts:1:import $ from 'jquery';
./features/repo-issue-sidebar.ts:1:import $ from 'jquery';
./features/repo-issue.ts:1:import $ from 'jquery';
./features/repo-legacy.ts:1:import $ from 'jquery';
./features/repo-new.ts:1:import $ from 'jquery';
./features/repo-projects.ts:1:import $ from 'jquery';
./features/repo-settings.ts:1:import $ from 'jquery';
./features/repo-template.ts:1:import $ from 'jquery';
```
wxiaoguang [Thu, 14 Nov 2024 05:02:11 +0000 (13:02 +0800)]
Refactor render system (#32492)
There were too many patches to the Render system, it's really difficult
to make further improvements.
This PR clears the legacy problems and fix TODOs.
1. Rename `RenderContext.Type` to `RenderContext.MarkupType` to clarify
its usage.
2. Use `ContentMode` to replace `meta["mode"]` and `IsWiki`, to clarify
the rendering behaviors.
3. Use "wiki" mode instead of "mode=gfm + wiki=true"
4. Merge `renderByType` and `renderByFile`
5. Add more comments
----
The problem of "mode=document": in many cases it is not set, so many
non-comment places use comment's hard line break incorrectly
1. move "internal-lfs" route mock to "common-lfs"
2. fine tune tests
3. fix "realm" strings, according to RFC:
https://datatracker.ietf.org/doc/html/rfc2617:
* realm = "realm" "=" realm-value
* realm-value = quoted-string
4. clarify some names of the middlewares, rename `ignXxx` to `optXxx` to
match `reqXxx`, and rename ambiguous `requireSignIn` to `reqGitSignIn`
6543 [Tue, 12 Nov 2024 03:44:24 +0000 (04:44 +0100)]
Limit org member view of restricted users (#32211)
currently restricted users can only see the repos of teams in orgs they
are part at.
they also should only see the users that are also part at the same team.
rust allows renaming dependencies such as when depending on multiple
versions of the same package. This is not supported by gitea as
discovered in #31500 . This PR tries to address that.
wxiaoguang [Tue, 12 Nov 2024 02:38:22 +0000 (10:38 +0800)]
Refactor LFS SSH and internal routers (#32473)
Gitea instance keeps reporting a lot of errors like "LFS SSH transfer connection denied, pure SSH protocol is disabled". When starting debugging the problem, there are more problems found. Try to address most of them:
* avoid unnecessary server side error logs (change `fail()` to not log them)
* figure out the broken tests/user2/lfs.git (added comments)
* avoid `migratePushMirrors` failure when a repository doesn't exist (ignore them)
* avoid "Authorization" (internal&lfs) header conflicts, remove the tricky "swapAuth" and use "X-Gitea-Internal-Auth"
* make internal token comparing constant time (it wasn't a serous problem because in a real world it's nearly impossible to timing-attack the token, but good to fix and backport)
* avoid duplicate routers (introduce AddOwnerRepoGitLFSRoutes)
* avoid "internal (private)" routes using session/web context (they should use private context)
* fix incorrect "path" usages (use "filepath")
* fix incorrect mocked route point handling (need to check func nil correctly)
* split some tests from "git general tests" to "git misc tests" (to keep "git_general_test.go" simple)
Still no correct result for Git LFS SSH tests. So the code is kept there
(`tests/integration/git_lfs_ssh_test.go`) and a FIXME explains the details.
Zettat123 [Mon, 11 Nov 2024 00:11:00 +0000 (08:11 +0800)]
Add `DEFAULT_MIRROR_REPO_UNITS` and `DEFAULT_TEMPLATE_REPO_UNITS` options (#32416)
Resolve #30350
The action unit of mirrors and templates should be disabled by default.
This PR adds `DEFAULT_MIRROR_REPO_UNITS` and
`DEFAULT_TEMPLATE_REPO_UNITS` options to allow users to specify default
units for mirrors and templates.
Thanks to @lng2020 for the
[idea](https://github.com/go-gitea/gitea/issues/30350#issuecomment-2053942243)
silverwind [Sat, 9 Nov 2024 18:03:55 +0000 (19:03 +0100)]
Fix mermaid diagram height when initially hidden (#32457)
In a hidden iframe, `document.body.clientHeight` is not reliable. Use
`IntersectionObserver` to detect the visibility change and update the
height there.
wxiaoguang [Fri, 8 Nov 2024 17:55:32 +0000 (01:55 +0800)]
Fix issue sidebar (#32455)
Fix #32453
Major changes:
* revert the `<div class="divider"></div>` in
`templates/repo/issue/branch_selector_field.tmpl` (it was removed by
mistake in ##32444)
* remove incorrect `<div class="inline field">` in
`templates/repo/issue/sidebar/allow_maintainer_edit.tmpl`
* use `gt-ellipsis` to replace the "title" class in the dependency list,
then `.repository .issue-content-right .ui.list .title` could be removed
* remove the "relaxed" from dependency list, then there is no padding,
then `.repository .issue-content-right .ui.list .dependency` could be
removed (`white-space` doesn't have effect either because there is
`gt-ellipsis`)
* remove dead code `.repository .issue-content-right #deadlineForm input
`
Kemal Zebari [Thu, 7 Nov 2024 19:56:53 +0000 (11:56 -0800)]
Only provide the commit summary for Discord webhook push events (#32432)
Resolves #32371.
#31970 should have just showed the commit summary, but
`strings.SplitN()` was misused such that we did not perform any
splitting at all and just used the message. This was not caught in the
unit test made in that PR since the test commit summary was > 50 (which
truncated away the commit description).
This snapshot resolves this and adds another unit test to ensure that we
only show the commit summary.
Lunny Xiao [Thu, 7 Nov 2024 06:41:49 +0000 (22:41 -0800)]
Add new event commit status creation and webhook implementation (#27151)
This PR introduces a new event which is similar as Github's. When a new
commit status submitted, the event will be trigged. That means, now we
can receive all feedback from CI/CD system in webhooks or other notify
systems.
Lunny Xiao [Thu, 7 Nov 2024 03:28:11 +0000 (19:28 -0800)]
Move AddCollabrator and CreateRepositoryByExample to service layer (#32419)
- [x] Move `CreateRepositoryByExample` to service layer
- [x] Move `AddCollabrator` to service layer
- [x] Add a new parameter for `AddCollabrator` so that changing mode
immediately after that will become unnecessary.
Kemal Zebari [Wed, 6 Nov 2024 21:34:32 +0000 (13:34 -0800)]
Include file extension checks in attachment API (#32151)
From testing, I found that issue posters and users with repository write
access are able to edit attachment names in a way that circumvents the
instance-level file extension restrictions using the edit attachment
APIs. This snapshot adds checks for these endpoints.
Bruno Sofiato [Wed, 6 Nov 2024 20:51:20 +0000 (17:51 -0300)]
Updated tokenizer to better matching when search for code snippets (#32261)
This PR improves the accuracy of Gitea's code search.
Currently, Gitea does not consider statements such as
`onsole.log("hello")` as hits when the user searches for `log`. The
culprit is how both ES and Bleve are tokenizing the file contents (in
both cases, `console.log` is a whole token).
In ES' case, we changed the tokenizer to
[simple_pattern_split](https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-simplepatternsplit-tokenizer.html#:~:text=The%20simple_pattern_split%20tokenizer%20uses%20a,the%20tokenization%20is%20generally%20faster.).
In such a case, tokens are words formed by digits and letters. In
Bleve's case, it employs a
[letter](https://blevesearch.com/docs/Tokenizers/) tokenizer.
Resolves #32220
---------
Signed-off-by: Bruno Sofiato <bruno.sofiato@gmail.com>
wxiaoguang [Wed, 6 Nov 2024 20:21:53 +0000 (04:21 +0800)]
Correctly query the primary button in a form (#32438)
The "primary button" is used at many places, but sometimes they might
conflict (due to button switch, hidden panel, dropdown menu, etc).
Sometimes we could add a special CSS class for the buttons, but
sometimes not (see the comment of QuickSubmit)
This PR introduces `querySingleVisibleElem` to help to get the correct
primary button (the only visible one), and prevent from querying the
wrong buttons.
wxiaoguang [Tue, 5 Nov 2024 06:35:54 +0000 (14:35 +0800)]
Refactor RepoRefByType (#32413)
1. clarify the "filepath" could(should) contain "{ref}"
2. remove unclear RepoRefLegacy and RepoRefAny, use RepoRefUnknown to guess
3. by the way, avoid using AppURL
wxiaoguang [Mon, 4 Nov 2024 10:59:50 +0000 (18:59 +0800)]
Refactor markup package (#32399)
To make the markup package easier to maintain:
1. Split some go files into small files
2. Use a shared util.NopCloser, remove duplicate code
3. Remove unused functions
Rowan Bohde [Sat, 2 Nov 2024 03:29:37 +0000 (22:29 -0500)]
improve performance of diffs (#32393)
This has two major changes that significantly reduce the amount of work
done for large diffs:
* Kill a running git process when reaching the maximum number of files
in a diff, preventing it from processing the entire diff.
* When loading a diff with the URL param `file-only=true`, skip loading
stats. This speeds up loading both hidden files of a diff and sections
of a diff when clicking the "Show More" button.
A couple of minor things from profiling are also included:
* Reuse existing repo in `PrepareViewPullInfo` if head and base are the
same.
The performance impact is going to depend heavily on the individual diff
and the hardware it runs on, but when testing locally on a diff changing
100k+ lines over hundreds of files, I'm seeing a roughly 75% reduction
in time to load the result of "Show More"
- Adds a DB fixture for actions artifacts
- Adds artifacts test files
- Clears artifacts test files between each run
- Note: I initially initialized the artifacts only for artifacts tests,
but because the files are small it only takes ~8ms, so I changed it to
always run in test setup for simplicity
- Fix some otherwise flaky tests by making them not depend on previous
tests
Zettat123 [Thu, 31 Oct 2024 15:28:25 +0000 (23:28 +0800)]
Fix `missing signature key` error when pulling Docker images with `SERVE_DIRECT` enabled (#32365)
Fix #28121
I did some tests and found that the `missing signature key` error is
caused by an incorrect `Content-Type` header. Gitea correctly sets the
`Content-Type` header when serving files.
https://github.com/go-gitea/gitea/blob/348d1d0f322ca57c459acd902f54821d687ca804/routers/api/packages/container/container.go#L712-L717
However, when `SERVE_DIRECT` is enabled, the `Content-Type` header may
be set to an incorrect value by the storage service. To fix this issue,
we can use query parameters to override response header values.
In this PR, I introduced a new parameter to the `URL` method to support
additional parameters.
```
URL(path, name string, reqParams url.Values) (*url.URL, error)
```
---
Most S3-like services support specifying the content type when storing
objects. However, Gitea always use `application/octet-stream`.
Therefore, I believe we also need to improve the `Save` method to
support storing objects with the correct content type.
silverwind [Thu, 31 Oct 2024 14:57:40 +0000 (15:57 +0100)]
Fix a number of typescript issues (#32308)
- Prefer
[window.location.assign](https://developer.mozilla.org/en-US/docs/Web/API/Location/assign)
over assigning to
[window.location](https://developer.mozilla.org/en-US/docs/Web/API/Window/location)
which typescript does not like. This works in all browsers including
PaleMoon.
- Fix all typescript issues in `web_src/js/webcomponents`, no behaviour
changes.
- ~~Workaround bug in `@typescript-eslint/no-unnecessary-type-assertion`
rule.~~
- Omit vendored file from type checks.
- `tsc` error count is reduce by 53 with these changes.
Royce Remer [Wed, 30 Oct 2024 05:41:55 +0000 (22:41 -0700)]
Add new [lfs_client].BATCH_SIZE and [server].LFS_MAX_BATCH_SIZE config settings. (#32307)
This contains two backwards-compatible changes:
* in the lfs http_client, the number of lfs oids requested per batch is
loaded from lfs_client#BATCH_SIZE and defaulted to the previous value of
20
* in the lfs server/service, the max number of lfs oids allowed in a
batch api request is loaded from server#LFS_MAX_BATCH_SIZE and defaults
to 'nil' which equates to the previous behavior of 'infinite'
Rowan Bohde [Wed, 30 Oct 2024 05:12:48 +0000 (00:12 -0500)]
remove unused call to $.HeadRepo in view_title template (#32317)
This is only populated in
[`ParseCompareInfo`](https://github.com/search?q=repo%3Ago-gitea%2Fgitea%20%20.Data%5B%22HeadRepo%22%5D&type=code)
which is called in two handlers:
*
[`CompareAndPullRequestPost`](https://github.com/go-gitea/gitea/blob/9206fbb55fd28f21720072fce6a36cc22277934c/routers/web/repo/pull.go#L1246)
- a JSON post handler that doesn't render templates
*
[`CompareDiff`](https://github.com/go-gitea/gitea/blob/9206fbb55fd28f21720072fce6a36cc22277934c/routers/web/repo/compare.go#L706)
- which can render `diff/box.tmpl` and `diff/compare.tmpl`
6543 [Tue, 29 Oct 2024 14:43:47 +0000 (15:43 +0100)]
Optimize branch protection rule loading (#32280)
before if it was nonglob each load would try to glob it and the check
that is not glob ... now we only do that once and no future loading will
trigger it