- When pasting a URL over another URL, replace the URL instead of
creating a useless `[url](url)`. This is the 1-line change
[here](https://github.com/go-gitea/gitea/pull/31948/files#diff-be8e94d7e3da33b187381f53d28095107bd0cf29ae9a9e997e4f422f4a54479cR122).
- Always run `initTextareaEvents`, previously it was not run when
`dropzoneEl` was not present like when attachements are disabled on the
server. Refactored the function to gracefully handle absent `dropzoneEl`
and rename the function to a better name.
silverwind [Thu, 29 Aug 2024 14:51:51 +0000 (16:51 +0200)]
Update JS and PY dependencies (#31940)
- `stylelint` is excluded because of deprecation warnings added in
16.8.2, depending on
https://github.com/AndyOGo/stylelint-declaration-strict-value/issues/379
- `eslint-plugin-sonarjs@2` adds a lot of new rules, but I have not
gotten around to reviewing them yet, so rule config is unchanged.
- Fixes https://github.com/go-gitea/gitea/security/dependabot/70
- Tested code editor and contributor graph
Jason Song [Thu, 29 Aug 2024 03:48:21 +0000 (11:48 +0800)]
Refactor globallock (#31933)
Follow #31908. The main refactor is that it has removed the returned
context of `Lock`.
The returned context of `Lock` in old code is to provide a way to let
callers know that they have lost the lock. But in most cases, callers
shouldn't cancel what they are doing even it has lost the lock. And the
design would confuse developers and make them use it incorrectly.
See the discussion history:
https://github.com/go-gitea/gitea/pull/31813#discussion_r1732041513 and
https://github.com/go-gitea/gitea/pull/31813#discussion_r1734078998
It's a breaking change, but since the new module hasn't been used yet, I
think it's OK to not add the `pr/breaking` label.
## Design principles
It's almost copied from #31908, but with some changes.
### Use spinlock even in memory implementation (unchanged)
In actual use cases, users may cancel requests. `sync.Mutex` will block
the goroutine until the lock is acquired even if the request is
canceled. And the spinlock is more suitable for this scenario since it's
possible to give up the lock acquisition.
Although the spinlock consumes more CPU resources, I think it's
acceptable in most cases.
### Do not expose the mutex to callers (unchanged)
If we expose the mutex to callers, it's possible for callers to reuse
the mutex, which causes more complexity.
For example:
```go
lock := GetLocker(key)
lock.Lock()
// ...
// even if the lock is unlocked, we cannot GC the lock,
// since the caller may still use it again.
lock.Unlock()
lock.Lock()
// ...
lock.Unlock()
// callers have to GC the lock manually.
RemoveLocker(key)
```
In this PR, we only expose `ReleaseFunc` to callers. So callers just
need to call `ReleaseFunc` to release the lock, and do not need to care
about the lock's lifecycle.
```go
release, err := locker.Lock(ctx, key)
if err != nil {
return err
}
// ...
release()
// if callers want to lock again, they have to re-acquire the lock.
release, err := locker.Lock(ctx, key)
// ...
```
In this way, it's also much easier for redis implementation to extend
the mutex automatically, so that callers do not need to care about the
lock's lifecycle. See also
https://github.com/go-gitea/gitea/pull/31813#discussion_r1722659743
### Use "release" instead of "unlock" (unchanged)
For "unlock", it has the meaning of "unlock an acquired lock". So it's
not acceptable to call "unlock" when failed to acquire the lock, or call
"unlock" multiple times. It causes more complexity for callers to decide
whether to call "unlock" or not.
So we use "release" instead of "unlock" to make it clear. Whether the
lock is acquired or not, callers can always call "release", and it's
also safe to call "release" multiple times.
But the code DO NOT expect callers to not call "release" after acquiring
the lock. If callers forget to call "release", it will cause resource
leak. That's why it's always safe to call "release" without extra
checks: to avoid callers to forget to call it.
### Acquired locks could be lost, but the callers shouldn't stop
Unlike `sync.Mutex` which will be locked forever once acquired until
calling `Unlock`, for distributed lock, the acquired lock could be lost.
For example, the caller has acquired the lock, and it holds the lock for
a long time since auto-extending is working for redis. However, it lost
the connection to the redis server, and it's impossible to extend the
lock anymore.
In #31908, it will cancel the context to make the operation stop, but
it's not safe. Many operations are not revert-able. If they have been
interrupted, then the instance goes corrupted. So `Lock` won't return
`ctx` anymore in this PR.
Jason Song [Mon, 26 Aug 2024 14:27:57 +0000 (22:27 +0800)]
Introduce globallock as distributed locks (#31908)
To help #31813, but do not replace it, since this PR just introduces the
new module but misses some work:
- New option in settings. `#31813` has done it.
- Use the locks in business logic. `#31813` has done it.
So I think the most efficient way is to merge this PR first (if it's
acceptable) and then finish #31813.
## Design principles
### Use spinlock even in memory implementation
In actual use cases, users may cancel requests. `sync.Mutex` will block
the goroutine until the lock is acquired even if the request is
canceled. And the spinlock is more suitable for this scenario since it's
possible to give up the lock acquisition.
Although the spinlock consumes more CPU resources, I think it's
acceptable in most cases.
### Do not expose the mutex to callers
If we expose the mutex to callers, it's possible for callers to reuse
the mutex, which causes more complexity.
For example:
```go
lock := GetLocker(key)
lock.Lock()
// ...
// even if the lock is unlocked, we cannot GC the lock,
// since the caller may still use it again.
lock.Unlock()
lock.Lock()
// ...
lock.Unlock()
// callers have to GC the lock manually.
RemoveLocker(key)
```
In this PR, we only expose `ReleaseFunc` to callers. So callers just
need to call `ReleaseFunc` to release the lock, and do not need to care
about the lock's lifecycle.
```go
_, release, err := locker.Lock(ctx, key)
if err != nil {
return err
}
// ...
release()
// if callers want to lock again, they have to re-acquire the lock.
_, release, err := locker.Lock(ctx, key)
// ...
```
In this way, it's also much easier for redis implementation to extend
the mutex automatically, so that callers do not need to care about the
lock's lifecycle. See also
https://github.com/go-gitea/gitea/pull/31813#discussion_r1722659743
### Use "release" instead of "unlock"
For "unlock", it has the meaning of "unlock an acquired lock". So it's
not acceptable to call "unlock" when failed to acquire the lock, or call
"unlock" multiple times. It causes more complexity for callers to decide
whether to call "unlock" or not.
So we use "release" instead of "unlock" to make it clear. Whether the
lock is acquired or not, callers can always call "release", and it's
also safe to call "release" multiple times.
But the code DO NOT expect callers to not call "release" after acquiring
the lock. If callers forget to call "release", it will cause resource
leak. That's why it's always safe to call "release" without extra
checks: to avoid callers to forget to call it.
### Acquired locks could be lost
Unlike `sync.Mutex` which will be locked forever once acquired until
calling `Unlock`, in the new module, the acquired lock could be lost.
For example, the caller has acquired the lock, and it holds the lock for
a long time since auto-extending is working for redis. However, it lost
the connection to the redis server, and it's impossible to extend the
lock anymore.
If the caller don't stop what it's doing, another instance which can
connect to the redis server could acquire the lock, and do the same
thing, which could cause data inconsistency.
So the caller should know what happened, the solution is to return a new
context which will be canceled if the lock is lost or released:
silverwind [Sun, 25 Aug 2024 17:23:13 +0000 (19:23 +0200)]
Update mermaid to v11 (#31913)
Update mermaid to
[v11](https://github.com/mermaid-js/mermaid/releases/tag/v11.0.0) and
enable the new [`suppressErrorRendering`
option](https://github.com/mermaid-js/mermaid/pull/4359) to ensure
mermaid never renders error elements into the DOM (we have per-chart
error rendering, so don't need it). Tested various chart types.
BTW, I was unable to reproduce that error rendering from mermaid with
`suppressErrorRendering: false` and I thought we had some CSS to hide
the error element, but I could not find it, not even in git history.
william-allspice [Wed, 21 Aug 2024 05:40:18 +0000 (00:40 -0500)]
Move lock icon position and add additional tooltips to branch list page (#31839)
This Pull Request adds missing tool tips for the protected, copy, and rss icons on the branch list page. It also moved protected icon position after the branch name.
Lunny Xiao [Tue, 20 Aug 2024 17:04:57 +0000 (01:04 +0800)]
Refactor the usage of batch catfile (#31754)
When opening a repository, it will call `ensureValidRepository` and also
`CatFileBatch`. But sometimes these will not be used until repository
closed. So it's a waste of CPU to invoke 3 times git command for every
open repository.
This PR removed all of these from `OpenRepository` but only kept
checking whether the folder exists. When a batch is necessary, the
necessary functions will be invoked.
胖梁 [Mon, 19 Aug 2024 02:38:40 +0000 (10:38 +0800)]
Actions support workflow dispatch event (#28163)
fix #23668
My plan:
* In the `actions.list` method, if workflow is selected and IsAdmin,
check whether the on event contains `workflow_dispatch`. If so, display
a `Run workflow` button to allow the user to manually trigger the run.
* Providing a form that allows users to select target brach or tag, and
these parameters can be configured in yaml
* Simple form validation, `required` input cannot be empty
* Add a route `/actions/run`, and an `actions.Run` method to handle
* Add `WorkflowDispatchPayload` struct to pass the Webhook event payload
to the runner when triggered, this payload carries the `inputs` values
and other fields, doc: [workflow_dispatch
payload](https://docs.github.com/en/webhooks/webhook-events-and-payloads#workflow_dispatch)
Other PRs
* the `Workflow.WorkflowDispatchConfig()` method still return non-nil
when workflow_dispatch is not defined. I submitted a PR
https://gitea.com/gitea/act/pulls/85 to fix it. Still waiting for them
to process.
Behavior should be same with github, but may cause confusion. Here's a
quick reminder.
*
[Doc](https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#workflow_dispatch)
Said: This event will `only` trigger a workflow run if the workflow file
is `on the default branch`.
* If the workflow yaml file only exists in a non-default branch, it
cannot be triggered. (It will not even show up in the workflow list)
* If the same workflow yaml file exists in each branch at the same time,
the version of the default branch is used. Even if `Use workflow from`
selects another branch
![image](https://github.com/go-gitea/gitea/assets/3114995/4bf596f3-426b-48e8-9b8f-0f6d18defd79)
```yaml
name: Docker Image CI
Adrian Hirt [Sat, 17 Aug 2024 03:34:27 +0000 (05:34 +0200)]
Fix overflowing content in action run log (#31842)
When a long line with characters such as dots is returned by a step in
an action (e.g. by the output of the Ruby on Rails test runner), it
overflows the log container, causing the page to scroll sideways.
This PR adds the CSS `overflow-wrap: anywhere;` to the
`.job-step-section .job-step-logs .job-log-line .log-msg` selector,
which causes such lines to wrap as well
Jason Song [Fri, 16 Aug 2024 17:04:54 +0000 (01:04 +0800)]
Avoid returning without written ctx when posting PR (#31843)
Fix #31625.
If `pull_service.NewPullRequest` return an error which misses each `if`
check, `CompareAndPullRequestPost` will return immediately, since it
doesn't write the HTTP response, a 200 response with empty body will be
sent to clients.
Not sure what kind of error can cause it to happen, so this PR just
expose it. And we can fix it when users report that creating PRs failed
with error responses.
It's all my guess since I cannot reproduce the problem, but even if it's
not related, the code here needs to be improved.
Zettat123 [Fri, 16 Aug 2024 12:40:51 +0000 (20:40 +0800)]
Fix raw wiki links (#31825)
Fix #31395
This regression is introduced by #30273. To find out how GitHub handles
this case, I did [some
tests](https://github.com/go-gitea/gitea/issues/31395#issuecomment-2278929115).
I use redirect in this PR instead of checking if the corresponding `.md`
file exists when rendering the link because GitHub also uses redirect.
With this PR, there is no need to resolve the raw wiki link when
rendering a wiki page. If a wiki link points to a raw file, access will
be redirected to the raw link.
yp05327 [Thu, 15 Aug 2024 16:34:24 +0000 (01:34 +0900)]
Add missing repository type filter parameters to pager (#31832)
Fix #31807
ps: the newly added params's value will be changed.
When the first time you selected the filter, the values of params will
be `0` or `1`
But in pager it will be `true` or `false`.
So do we have `boolToInt` function?
Rowan Bohde [Wed, 14 Aug 2024 21:50:09 +0000 (16:50 -0500)]
render plain text file if the LFS object doesn't exist (#31812)
We had an issue where a repo was using LFS to store a file, but the user
did not push the file. When trying to view the file, Gitea returned a
500 HTTP status code referencing `ErrLFSObjectNotExist`. It appears the
intent was the render this file as plain text, but the conditional was
flipped. I've also added a test to verify that the file is rendered as
plain text.
Edip Emre Bodur [Tue, 13 Aug 2024 02:53:43 +0000 (05:53 +0300)]
Fixes for unreachable project issues when transfer repository from organization (#31770)
When transferring repositories that have issues linked to a project
board to another organization, the issues remain associated with the
original project board. This causes the columns in the project board to
become bugged, making it difficult to move other issues in or out of the
affected columns. As a solution, I removed the issue relations since the
other organization does not have this project table.
Simon Priet [Mon, 12 Aug 2024 23:36:28 +0000 (01:36 +0200)]
Scroll images in project issues separately from the remaining issue (#31683)
As discussed in #31667 & #26561, when a card on a Project contains
images, they can overflow the card on its containing column. This aims
to fix this issue via snapping scrollbars.
---
Issue #31667 is open to discussion as there should be room for
improvement.
silverwind [Mon, 12 Aug 2024 00:40:18 +0000 (02:40 +0200)]
Add `:focus-visible` style to buttons (#31799)
Buttons now show a focus style via
[`:focus-visible`](https://developer.mozilla.org/en-US/docs/Web/CSS/:focus-visible)
when the browser deems the focus to be important, like for example when
the button is focused via keyboard navigation.
Fábio Barkoski [Sun, 11 Aug 2024 04:50:54 +0000 (01:50 -0300)]
Move repository visibility to danger zone in the settings area (#31126)
Moved repository visibility to the danger zone in the settings area. To
change the visibility, it is necessary to go to the danger zone, click
on the private/public button, and accept the change in the modal.
silverwind [Sat, 10 Aug 2024 09:46:48 +0000 (11:46 +0200)]
Add types to various low-level functions (#31781)
Adds types to various low-level modules. All changes are type-only, no
runtime changes. `tsc` now reports 38 less errors.
One problem was that `@types/sortablejs` does not accept promise return
in its functions which triggered the linter, so I disabled the rules on
those line.
Jason Song [Fri, 9 Aug 2024 02:40:45 +0000 (10:40 +0800)]
Fix `IsObjectExist` with gogit (#31790)
Fix #31271.
When gogit is enabled, `IsObjectExist` calls
`repo.gogitRepo.ResolveRevision`, which is not correct. It's for
checking references not objects, it could work with commit hash since
it's both a valid reference and a commit object, but it doesn't work
with blob objects.
So it causes #31271 because it reports that all blob objects do not
exist.
Jason Song [Fri, 9 Aug 2024 02:10:30 +0000 (10:10 +0800)]
Support compression for Actions logs (#31761)
Support compression for Actions logs to save storage space and
bandwidth. Inspired by
https://github.com/go-gitea/gitea/issues/24256#issuecomment-1521153015
The biggest challenge is that the compression format should support
[seekable](https://github.com/facebook/zstd/blob/dev/contrib/seekable_format/zstd_seekable_compression_format.md).
So when users are viewing a part of the log lines, Gitea doesn't need to
download the whole compressed file and decompress it.
That means gzip cannot help here. And I did research, there aren't too
many choices, like bgzip and xz, but I think zstd is the most popular
one. It has an implementation in Golang with
[zstd](https://github.com/klauspost/compress/tree/master/zstd) and
[zstd-seekable-format-go](https://github.com/SaveTheRbtz/zstd-seekable-format-go),
and what is better is that it has good compatibility: a seekable format
zstd file can be read by a regular zstd reader.
This PR introduces a new package `zstd` to combine and wrap the two
packages, to provide a unified and easy-to-use API.
And a new setting `LOG_COMPRESSION` is added to the config, although I
don't see any reason why not to use compression, I think's it's a good
idea to keep the default with `none` to be consistent with old versions.
`LOG_COMPRESSION` takes effect for only new log files, it adds `.zst` as
an extension to the file name, so Gitea can determine if it needs
decompression according to the file name when reading. Old files will
keep the format since it's not worth converting them, as they will be
cleared after #31735.
Jason Song [Wed, 7 Aug 2024 15:29:08 +0000 (23:29 +0800)]
Add `TAGS` to `TEST_TAGS` and fix bugs found with gogit (#31791)
Found at
https://github.com/go-gitea/gitea/pull/31790#issuecomment-2272898915
`unit-tests-gogit` never work since the workflow set `TAGS` with
`gogit`, but the Makefile use `TEST_TAGS`.
This PR adds the values of `TAGS` to `TEST_TAGS`, ensuring that setting
`TAGS` is always acceptable and avoiding confusion about which one
should be set.
Lunny Xiao [Tue, 6 Aug 2024 13:32:49 +0000 (21:32 +0800)]
Fix protected branch files detection on pre_receive hook (#31778)
Fix #31738
When pushing a new branch, the old commit is zero. Most git commands
cannot recognize the zero commit id. To get the changed files in the
push, we need to get the first diverge commit of this branch. In most
situations, we could check commits one by one until one commit is
contained by another branch. Then we will think that commit is the
diverge point.
And in a pre-receive hook, this will be more difficult because all
commits haven't been merged and they actually stored in a temporary
place by git. So we need to bring some envs to let git know the commit
exist.
Edip Emre Bodur [Mon, 5 Aug 2024 10:59:53 +0000 (13:59 +0300)]
Fix null requested_reviewer from API (#31773)
If the assign the pull request review to a team, it did not show the
members of the team in the "requested_reviewers" field, so the field was
null. As a solution, I added the team members to the array.
Jason Song [Thu, 1 Aug 2024 10:02:46 +0000 (18:02 +0800)]
Use UTC as default timezone when schedule Actions cron tasks (#31742)
Fix #31657.
According to the
[doc](https://docs.github.com/en/actions/writing-workflows/workflow-syntax-for-github-actions#onschedule)
of GitHub Actions, The timezone for cron should be UTC, not the local
timezone. And Gitea Actions doesn't have any reasons to change this, so
I think it's a bug.
However, Gitea Actions has extended the syntax, as it supports
descriptors like `@weekly` and `@every 5m`, and supports specifying the
timezone like `TZ=UTC 0 10 * * *`. So we can make it use UTC only when
the timezone is not specified, to be compatible with GitHub Actions, and
also respect the user's specified.
It does break the feature because the times to run tasks would be
changed, and it may confuse users. So I don't think we should backport
this.
## ⚠️ BREAKING ⚠️
If the server's local time zone is not UTC, a scheduled task would run
at a different time after upgrading Gitea to this version.
Jason Song [Thu, 1 Aug 2024 09:04:04 +0000 (17:04 +0800)]
Clarify Actions resources ownership (#31724)
Fix #31707.
Also related to #31715.
Some Actions resources could has different types of ownership. It could
be:
- global: all repos and orgs/users can use it.
- org/user level: only the org/user can use it.
- repo level: only the repo can use it.
There are two ways to distinguish org/user level from repo level:
1. `{owner_id: 1, repo_id: 2}` for repo level, and `{owner_id: 1,
repo_id: 0}` for org level.
2. `{owner_id: 0, repo_id: 2}` for repo level, and `{owner_id: 1,
repo_id: 0}` for org level.
The first way seems more reasonable, but it may not be true. The point
is that although a resource, like a runner, belongs to a repo (it can be
used by the repo), the runner doesn't belong to the repo's org (other
repos in the same org cannot use the runner). So, the second method
makes more sense.
And the first way is not user-friendly to query, we must set the repo id
to zero to avoid wrong results.
So, #31715 should be right. And the most simple way to fix #31707 is
just:
However, it is quite intuitive to set both owner id and repo id since
the repo belongs to the owner. So I prefer to be compatible with it. If
we get both owner id and repo id not zero when creating or finding, it's
very clear that the caller want one with repo level, but set owner id
accidentally. So it's OK to accept it but fix the owner id to zero.
Jason Song [Wed, 31 Jul 2024 10:29:48 +0000 (18:29 +0800)]
Distinguish LFS object errors to ignore missing objects during migration (#31702)
Fix #31137.
Replace #31623 #31697.
When migrating LFS objects, if there's any object that failed (like some
objects are losted, which is not really critical), Gitea will stop
migrating LFS immediately but treat the migration as successful.
This PR checks the error according to the [LFS api
doc](https://github.com/git-lfs/git-lfs/blob/main/docs/api/batch.md#successful-responses).
> LFS object error codes should match HTTP status codes where possible:
>
> - 404 - The object does not exist on the server.
> - 409 - The specified hash algorithm disagrees with the server's
acceptable options.
> - 410 - The object was removed by the owner.
> - 422 - Validation error.
If the error is `404`, it's safe to ignore it and continue migration.
Otherwise, stop the migration and mark it as failed to ensure data
integrity of LFS objects.
And maybe we should also ignore others errors (maybe `410`? I'm not sure
what's the difference between "does not exist" and "removed by the
owner".), we can add it later when some users report that they have
failed to migrate LFS because of an error which should be ignored.
Hide the "Details" link of commit status when the user cannot access actions (#30156)
Fix #26685
If a commit status comes from Gitea Actions and the user cannot access
the repo's actions unit (the user does not have the permission or the
actions unit is disabled), a 404 page will occur after clicking the
"Details" link. We should hide the "Details" link in this case.