wxiaoguang [Fri, 15 Nov 2024 15:45:07 +0000 (23:45 +0800)]
Improve testing and try to fix MySQL hanging (#32515)
By some CI fine tunes (`run tests`), SQLite & MSSQL could complete
in about 12~13 minutes (before > 14), MySQL could complete in 18 minutes
(before: about 23 or even > 30)
Major changes:
1. use tmpfs for MySQL storage
1. run `make test-mysql` instead of `make integration-test-coverage`
because the code coverage is not really used at the moment.
1. refactor testlogger to make it more reliable and be able to report
stuck stacktrace
1. do not requeue failed items when a queue is being flushed (failed
items would keep failing and make flush uncompleted)
1. reduce the file sizes for testing
1. use math ChaCha20 random data instead of crypot/rand (for testing
purpose only)
1. no need to `DeleteRepository` in `TestLinguist`
1. other related refactoring to make code easier to maintain
Lunny Xiao [Fri, 15 Nov 2024 04:04:20 +0000 (20:04 -0800)]
Remove transaction for archive download (#32186)
Since there is a status column in the database, the transaction is
unnecessary when downloading an archive. The transaction is blocking
database operations, especially with SQLite.
Rowan Bohde [Thu, 14 Nov 2024 19:28:46 +0000 (13:28 -0600)]
Reduce integration test overhead (#32475)
In profiling integration tests, I found a couple places where per-test
overhead could be reduced:
* Avoiding disk IO by synchronizing instead of deleting & copying test
Git repository data. This saves ~100ms per test on my machine
* When flushing queues in `PrintCurrentTest`, invoke `FlushWithContext`
in a parallel.
wxiaoguang [Thu, 14 Nov 2024 18:48:41 +0000 (02:48 +0800)]
Remove jQuery import from some files (#32512)
Many files do not directly depend on jQuery now.
To clarify the usage: use `fomanticQuery` to operate Fomantic
components.
Then developers could focus on removing the remaining jQuery usages by
searching `import $` globally.
21 files now:
```
./components/RepoBranchTagSelector.vue:3:import $ from 'jquery';
./features/admin/common.ts:1:import $ from 'jquery';
./features/admin/emails.ts:1:import $ from 'jquery';
./features/common-button.ts:1:import $ from 'jquery';
./features/comp/ComboMarkdownEditor.ts:3:import $ from 'jquery'; (I am working on it, there will be a new PR)
./features/comp/LabelEdit.ts:1:import $ from 'jquery';
./features/notification.ts:1:import $ from 'jquery';
./features/org-team.ts:1:import $ from 'jquery';
./features/repo-code.ts:1:import $ from 'jquery';
./features/repo-common.ts:1:import $ from 'jquery';
./features/repo-diff.ts:1:import $ from 'jquery';
./features/repo-editor.ts:1:import $ from 'jquery';
./features/repo-issue-content.ts:1:import $ from 'jquery';
./features/repo-issue-list.ts:1:import $ from 'jquery';
./features/repo-issue-sidebar.ts:1:import $ from 'jquery';
./features/repo-issue.ts:1:import $ from 'jquery';
./features/repo-legacy.ts:1:import $ from 'jquery';
./features/repo-new.ts:1:import $ from 'jquery';
./features/repo-projects.ts:1:import $ from 'jquery';
./features/repo-settings.ts:1:import $ from 'jquery';
./features/repo-template.ts:1:import $ from 'jquery';
```
wxiaoguang [Thu, 14 Nov 2024 05:02:11 +0000 (13:02 +0800)]
Refactor render system (#32492)
There were too many patches to the Render system, it's really difficult
to make further improvements.
This PR clears the legacy problems and fix TODOs.
1. Rename `RenderContext.Type` to `RenderContext.MarkupType` to clarify
its usage.
2. Use `ContentMode` to replace `meta["mode"]` and `IsWiki`, to clarify
the rendering behaviors.
3. Use "wiki" mode instead of "mode=gfm + wiki=true"
4. Merge `renderByType` and `renderByFile`
5. Add more comments
----
The problem of "mode=document": in many cases it is not set, so many
non-comment places use comment's hard line break incorrectly
1. move "internal-lfs" route mock to "common-lfs"
2. fine tune tests
3. fix "realm" strings, according to RFC:
https://datatracker.ietf.org/doc/html/rfc2617:
* realm = "realm" "=" realm-value
* realm-value = quoted-string
4. clarify some names of the middlewares, rename `ignXxx` to `optXxx` to
match `reqXxx`, and rename ambiguous `requireSignIn` to `reqGitSignIn`
6543 [Tue, 12 Nov 2024 03:44:24 +0000 (04:44 +0100)]
Limit org member view of restricted users (#32211)
currently restricted users can only see the repos of teams in orgs they
are part at.
they also should only see the users that are also part at the same team.
rust allows renaming dependencies such as when depending on multiple
versions of the same package. This is not supported by gitea as
discovered in #31500 . This PR tries to address that.
wxiaoguang [Tue, 12 Nov 2024 02:38:22 +0000 (10:38 +0800)]
Refactor LFS SSH and internal routers (#32473)
Gitea instance keeps reporting a lot of errors like "LFS SSH transfer connection denied, pure SSH protocol is disabled". When starting debugging the problem, there are more problems found. Try to address most of them:
* avoid unnecessary server side error logs (change `fail()` to not log them)
* figure out the broken tests/user2/lfs.git (added comments)
* avoid `migratePushMirrors` failure when a repository doesn't exist (ignore them)
* avoid "Authorization" (internal&lfs) header conflicts, remove the tricky "swapAuth" and use "X-Gitea-Internal-Auth"
* make internal token comparing constant time (it wasn't a serous problem because in a real world it's nearly impossible to timing-attack the token, but good to fix and backport)
* avoid duplicate routers (introduce AddOwnerRepoGitLFSRoutes)
* avoid "internal (private)" routes using session/web context (they should use private context)
* fix incorrect "path" usages (use "filepath")
* fix incorrect mocked route point handling (need to check func nil correctly)
* split some tests from "git general tests" to "git misc tests" (to keep "git_general_test.go" simple)
Still no correct result for Git LFS SSH tests. So the code is kept there
(`tests/integration/git_lfs_ssh_test.go`) and a FIXME explains the details.
Zettat123 [Mon, 11 Nov 2024 00:11:00 +0000 (08:11 +0800)]
Add `DEFAULT_MIRROR_REPO_UNITS` and `DEFAULT_TEMPLATE_REPO_UNITS` options (#32416)
Resolve #30350
The action unit of mirrors and templates should be disabled by default.
This PR adds `DEFAULT_MIRROR_REPO_UNITS` and
`DEFAULT_TEMPLATE_REPO_UNITS` options to allow users to specify default
units for mirrors and templates.
Thanks to @lng2020 for the
[idea](https://github.com/go-gitea/gitea/issues/30350#issuecomment-2053942243)
silverwind [Sat, 9 Nov 2024 18:03:55 +0000 (19:03 +0100)]
Fix mermaid diagram height when initially hidden (#32457)
In a hidden iframe, `document.body.clientHeight` is not reliable. Use
`IntersectionObserver` to detect the visibility change and update the
height there.
wxiaoguang [Fri, 8 Nov 2024 17:55:32 +0000 (01:55 +0800)]
Fix issue sidebar (#32455)
Fix #32453
Major changes:
* revert the `<div class="divider"></div>` in
`templates/repo/issue/branch_selector_field.tmpl` (it was removed by
mistake in ##32444)
* remove incorrect `<div class="inline field">` in
`templates/repo/issue/sidebar/allow_maintainer_edit.tmpl`
* use `gt-ellipsis` to replace the "title" class in the dependency list,
then `.repository .issue-content-right .ui.list .title` could be removed
* remove the "relaxed" from dependency list, then there is no padding,
then `.repository .issue-content-right .ui.list .dependency` could be
removed (`white-space` doesn't have effect either because there is
`gt-ellipsis`)
* remove dead code `.repository .issue-content-right #deadlineForm input
`
Kemal Zebari [Thu, 7 Nov 2024 19:56:53 +0000 (11:56 -0800)]
Only provide the commit summary for Discord webhook push events (#32432)
Resolves #32371.
#31970 should have just showed the commit summary, but
`strings.SplitN()` was misused such that we did not perform any
splitting at all and just used the message. This was not caught in the
unit test made in that PR since the test commit summary was > 50 (which
truncated away the commit description).
This snapshot resolves this and adds another unit test to ensure that we
only show the commit summary.
Lunny Xiao [Thu, 7 Nov 2024 06:41:49 +0000 (22:41 -0800)]
Add new event commit status creation and webhook implementation (#27151)
This PR introduces a new event which is similar as Github's. When a new
commit status submitted, the event will be trigged. That means, now we
can receive all feedback from CI/CD system in webhooks or other notify
systems.
Lunny Xiao [Thu, 7 Nov 2024 03:28:11 +0000 (19:28 -0800)]
Move AddCollabrator and CreateRepositoryByExample to service layer (#32419)
- [x] Move `CreateRepositoryByExample` to service layer
- [x] Move `AddCollabrator` to service layer
- [x] Add a new parameter for `AddCollabrator` so that changing mode
immediately after that will become unnecessary.
Kemal Zebari [Wed, 6 Nov 2024 21:34:32 +0000 (13:34 -0800)]
Include file extension checks in attachment API (#32151)
From testing, I found that issue posters and users with repository write
access are able to edit attachment names in a way that circumvents the
instance-level file extension restrictions using the edit attachment
APIs. This snapshot adds checks for these endpoints.
Bruno Sofiato [Wed, 6 Nov 2024 20:51:20 +0000 (17:51 -0300)]
Updated tokenizer to better matching when search for code snippets (#32261)
This PR improves the accuracy of Gitea's code search.
Currently, Gitea does not consider statements such as
`onsole.log("hello")` as hits when the user searches for `log`. The
culprit is how both ES and Bleve are tokenizing the file contents (in
both cases, `console.log` is a whole token).
In ES' case, we changed the tokenizer to
[simple_pattern_split](https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-simplepatternsplit-tokenizer.html#:~:text=The%20simple_pattern_split%20tokenizer%20uses%20a,the%20tokenization%20is%20generally%20faster.).
In such a case, tokens are words formed by digits and letters. In
Bleve's case, it employs a
[letter](https://blevesearch.com/docs/Tokenizers/) tokenizer.
Resolves #32220
---------
Signed-off-by: Bruno Sofiato <bruno.sofiato@gmail.com>
wxiaoguang [Wed, 6 Nov 2024 20:21:53 +0000 (04:21 +0800)]
Correctly query the primary button in a form (#32438)
The "primary button" is used at many places, but sometimes they might
conflict (due to button switch, hidden panel, dropdown menu, etc).
Sometimes we could add a special CSS class for the buttons, but
sometimes not (see the comment of QuickSubmit)
This PR introduces `querySingleVisibleElem` to help to get the correct
primary button (the only visible one), and prevent from querying the
wrong buttons.
wxiaoguang [Tue, 5 Nov 2024 06:35:54 +0000 (14:35 +0800)]
Refactor RepoRefByType (#32413)
1. clarify the "filepath" could(should) contain "{ref}"
2. remove unclear RepoRefLegacy and RepoRefAny, use RepoRefUnknown to guess
3. by the way, avoid using AppURL
wxiaoguang [Mon, 4 Nov 2024 10:59:50 +0000 (18:59 +0800)]
Refactor markup package (#32399)
To make the markup package easier to maintain:
1. Split some go files into small files
2. Use a shared util.NopCloser, remove duplicate code
3. Remove unused functions
Rowan Bohde [Sat, 2 Nov 2024 03:29:37 +0000 (22:29 -0500)]
improve performance of diffs (#32393)
This has two major changes that significantly reduce the amount of work
done for large diffs:
* Kill a running git process when reaching the maximum number of files
in a diff, preventing it from processing the entire diff.
* When loading a diff with the URL param `file-only=true`, skip loading
stats. This speeds up loading both hidden files of a diff and sections
of a diff when clicking the "Show More" button.
A couple of minor things from profiling are also included:
* Reuse existing repo in `PrepareViewPullInfo` if head and base are the
same.
The performance impact is going to depend heavily on the individual diff
and the hardware it runs on, but when testing locally on a diff changing
100k+ lines over hundreds of files, I'm seeing a roughly 75% reduction
in time to load the result of "Show More"
- Adds a DB fixture for actions artifacts
- Adds artifacts test files
- Clears artifacts test files between each run
- Note: I initially initialized the artifacts only for artifacts tests,
but because the files are small it only takes ~8ms, so I changed it to
always run in test setup for simplicity
- Fix some otherwise flaky tests by making them not depend on previous
tests
Zettat123 [Thu, 31 Oct 2024 15:28:25 +0000 (23:28 +0800)]
Fix `missing signature key` error when pulling Docker images with `SERVE_DIRECT` enabled (#32365)
Fix #28121
I did some tests and found that the `missing signature key` error is
caused by an incorrect `Content-Type` header. Gitea correctly sets the
`Content-Type` header when serving files.
https://github.com/go-gitea/gitea/blob/348d1d0f322ca57c459acd902f54821d687ca804/routers/api/packages/container/container.go#L712-L717
However, when `SERVE_DIRECT` is enabled, the `Content-Type` header may
be set to an incorrect value by the storage service. To fix this issue,
we can use query parameters to override response header values.
In this PR, I introduced a new parameter to the `URL` method to support
additional parameters.
```
URL(path, name string, reqParams url.Values) (*url.URL, error)
```
---
Most S3-like services support specifying the content type when storing
objects. However, Gitea always use `application/octet-stream`.
Therefore, I believe we also need to improve the `Save` method to
support storing objects with the correct content type.
silverwind [Thu, 31 Oct 2024 14:57:40 +0000 (15:57 +0100)]
Fix a number of typescript issues (#32308)
- Prefer
[window.location.assign](https://developer.mozilla.org/en-US/docs/Web/API/Location/assign)
over assigning to
[window.location](https://developer.mozilla.org/en-US/docs/Web/API/Window/location)
which typescript does not like. This works in all browsers including
PaleMoon.
- Fix all typescript issues in `web_src/js/webcomponents`, no behaviour
changes.
- ~~Workaround bug in `@typescript-eslint/no-unnecessary-type-assertion`
rule.~~
- Omit vendored file from type checks.
- `tsc` error count is reduce by 53 with these changes.
Royce Remer [Wed, 30 Oct 2024 05:41:55 +0000 (22:41 -0700)]
Add new [lfs_client].BATCH_SIZE and [server].LFS_MAX_BATCH_SIZE config settings. (#32307)
This contains two backwards-compatible changes:
* in the lfs http_client, the number of lfs oids requested per batch is
loaded from lfs_client#BATCH_SIZE and defaulted to the previous value of
20
* in the lfs server/service, the max number of lfs oids allowed in a
batch api request is loaded from server#LFS_MAX_BATCH_SIZE and defaults
to 'nil' which equates to the previous behavior of 'infinite'
Rowan Bohde [Wed, 30 Oct 2024 05:12:48 +0000 (00:12 -0500)]
remove unused call to $.HeadRepo in view_title template (#32317)
This is only populated in
[`ParseCompareInfo`](https://github.com/search?q=repo%3Ago-gitea%2Fgitea%20%20.Data%5B%22HeadRepo%22%5D&type=code)
which is called in two handlers:
*
[`CompareAndPullRequestPost`](https://github.com/go-gitea/gitea/blob/9206fbb55fd28f21720072fce6a36cc22277934c/routers/web/repo/pull.go#L1246)
- a JSON post handler that doesn't render templates
*
[`CompareDiff`](https://github.com/go-gitea/gitea/blob/9206fbb55fd28f21720072fce6a36cc22277934c/routers/web/repo/compare.go#L706)
- which can render `diff/box.tmpl` and `diff/compare.tmpl`
6543 [Tue, 29 Oct 2024 14:43:47 +0000 (15:43 +0100)]
Optimize branch protection rule loading (#32280)
before if it was nonglob each load would try to glob it and the check
that is not glob ... now we only do that once and no future loading will
trigger it
Anbraten [Mon, 28 Oct 2024 20:15:05 +0000 (21:15 +0100)]
Migrate vue components to setup (#32329)
Migrated a handful Vue components to the `setup` syntax using
composition api as it has better Typescript support and is becoming the
new default in the Vue ecosystem.
Left some larger components untouched for now to not go to crazy in this
single PR:
- [ ] DiffCommitSelector.vue
- [ ] RepoActionView.vue
- [ ] RepoContributors.vue
- [ ] DashboardRepoList.vue
- [ ] RepoBranchTagSelector.vue
Tim [Wed, 23 Oct 2024 04:39:43 +0000 (06:39 +0200)]
Make admins adhere to branch protection rules (#32248)
This introduces a new flag `BlockAdminMergeOverride` on the branch
protection rules that prevents admins/repo owners from bypassing branch
protection rules and merging without approvals or failing status checks.
cloudchamb3r [Wed, 16 Oct 2024 12:39:47 +0000 (21:39 +0900)]
Fix null errors on conversation holder (#32258) (#32266)
fix #32258
Errors in the issue was due to unhandled null check. so i fixed it.
### Detailed description for Issue & Fix
To reproduce that issue, the comment must be deleted on Conversation
tab.
#### Before Delete
<img width="1032" alt="image"
src="https://github.com/user-attachments/assets/72df61ba-7db6-44c9-bebc-ca1178dd27f1">
#### After Delete (AS-IS)
<img width="1010" alt="image"
src="https://github.com/user-attachments/assets/36fa537e-4f8e-4535-8d02-e538c50f0dd8">
gitea already have remove logic for `timeline-item-group`, but because
of null ref exception the later logic that removes `timeline-item-group`
could be not be called correctly.
谈笑风生间 [Wed, 16 Oct 2024 09:10:05 +0000 (17:10 +0800)]
Support requested_reviewers data in comment webhook events (#26178)
close #25833
Currently, the information for "requested_reviewers" is only included in
the webhook event for reviews. I would like to suggest adding this
information to the webhook event for "PullRequest comment" as well, as
they both pertain to the "PullRequest" event.
Also, The reviewer information for the Pull Request is not displayed
when it is approved or rejected.
in https://github.com/go-gitea/gitea/pull/27775 , it do some changes to
only show diff file tree when more than one file changed. But looks it
also break the `diff-file-list` logic, which looks not expected change.
so try fix it.