aboutsummaryrefslogtreecommitdiffstats
path: root/modules/git
Commit message (Collapse)AuthorAgeFilesLines
...
* Remove legacy `+build:` constraint (#19582)wxiaoguang2022-05-0245-45/+0
| | | Go 1.17 and later use modern `//go:build` constraints, the old `// +build:` constraints should be removed.
* Upgrade required git version to 2.0 (#19577)wxiaoguang2022-05-021-6/+16
| | | | | * Upgrade required git version to 2.0 * update document
* Prevent intermittent race in attribute reader close (#19537)zeripath2022-04-281-16/+1
| | | | | | | | | | | | There is a potential rare race possible whereby the c.running channel could be closed twice. Looking at the code I do not see a need for this c.running channel and therefore I think we can remove this. (I think the c.running might have been some attempt to prevent a hang but the use of os.Pipes should prevent that.) Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com> Co-authored-by: 6543 <6543@obermui.de>
* Pass gitRepo down to GetRawDiff, since its used for main repo and wiki (#19461)65432022-04-251-13/+2
| | | | | as per https://github.com/go-gitea/gitea/pull/19449#issuecomment-1105283931 pass gitRepo down to GetRawDiff, since its used for main repo and wiki
* Prevent dangling cat-file calls (goroutine alternative) (#19454)zeripath2022-04-221-0/+12
| | | | | | | | | | | | | | | | | | | If an `os/exec.Command` is passed non `*os.File` as an input/output, go will create `os.Pipe`s and wait for their closure in `cmd.Wait()`. If the code following this is responsible for closing `io.Pipe`s or other handlers then on process death from context cancellation the `Wait` can hang. There are two possible solutions: 1. use `os.Pipe` as the input/output as `cmd.Wait` does not wait for these. 2. create a goroutine waiting on the context cancellation that will close the inputs. This PR provides the second option - which is a simpler change that can be more easily backported. Closes #19448 Signed-off-by: Andrew Thornton <art27@cantab.net>
* Add uploadpack.allowAnySHA1InWant to allow --filter=blob:none with older git ↵zeripath2022-04-201-1/+1
| | | | | | | | | clients (#19430) Older git clients need uploadpack.allowAnySHA1InWant if partial cloning is allowed. Fix #19118 Signed-off-by: Andrew Thornton <art27@cantab.net>
* Remove `git.Command.Run` and `git.Command.RunInDir*` (#19280)wxiaoguang2022-04-0134-350/+302
| | | | | | Follows #19266, #8553, Close #18553, now there are only three `Run..(&RunOpts{})` functions. * before: `stdout, err := RunInDir(path)` * now: `stdout, _, err := RunStdString(&git.RunOpts{Dir:path})`
* Improve sync performance for pull-mirrors (#19125)Peter Gardfjäll2022-03-316-19/+774
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This addresses https://github.com/go-gitea/gitea/issues/18352 It aims to improve performance (and resource use) of the `SyncReleasesWithTags` operation for pull-mirrors. For large repositories with many tags, `SyncReleasesWithTags` can be a costly operation (taking several minutes to complete). The reason is two-fold: 1. on sync, every upstream repo tag is compared (for changes) against existing local entries in the release table to ensure that they are up-to-date. 2. the procedure for getting _each tag_ involves a series of git operations ```bash git show-ref --tags -- v8.2.4477 git cat-file -t 29ab6ce9f36660cffaad3c8789e71162e5db5d2f git cat-file -p 29ab6ce9f36660cffaad3c8789e71162e5db5d2f git rev-list --count 29ab6ce9f36660cffaad3c8789e71162e5db5d2f ``` of which the `git rev-list --count` can be particularly heavy. This PR optimizes performance for pull-mirrors. We utilize the fact that a pull-mirror is always identical to its upstream and rebuild the entire release table on every sync and use a batch `git for-each-ref .. refs/tags` call to retrieve all tags in one go. For large mirror repos, with hundreds of annotated tags, this brings down the duration of the sync operation from several minutes to a few seconds. A few unscientific examples run on my local machine: - https://github.com/spring-projects/spring-boot (223 tags) - before: `0m28,673s` - after: `0m2,244s` - https://github.com/kubernetes/kubernetes (890 tags) - before: `8m00s` - after: `0m8,520s` - https://github.com/vim/vim (13954 tags) - before: `14m20,383s` - after: `0m35,467s` I added a `foreachref` package which contains a flexible way of specifying which reference fields are of interest (`git-for-each-ref(1)`) and to produce a parser for the expected output. These could be reused in other places where `for-each-ref` is used. I'll add unit tests for those if the overall PR looks promising.
* Refactor `git.Command.Run*`, introduce `RunWithContextString` and ↵wxiaoguang2022-03-314-113/+117
| | | | | | | | | | | | | | | | | | `RunWithContextBytes` (#19266) This follows * https://github.com/go-gitea/gitea/issues/18553 Introduce `RunWithContextString` and `RunWithContextBytes` to help the refactoring. Add related unit tests. They keep the same behavior to save stderr into err.Error() as `RunInXxx` before. Remove `RunInDirTimeoutPipeline` `RunInDirTimeoutFullPipeline` `RunInDirTimeout` `RunInDirTimeoutEnv` `RunInDirPipeline` `RunInDirFullPipeline` `RunTimeout`, `RunInDirTimeoutEnvPipeline`, `RunInDirTimeoutEnvFullPipeline`, `RunInDirTimeoutEnvFullPipelineFunc`. Then remaining `RunInDir` `RunInDirBytes` `RunInDirWithEnv` can be easily refactored in next PR with a simple search & replace: * before: `stdout, err := RunInDir(path)` * next: `stdout, _, err := RunWithContextString(&git.RunContext{Dir:path})` Other changes: 1. When `timeout <= 0`, use default. Because `timeout==0` is meaningless and could cause bugs. And now many functions becomes more simple, eg: `GitGcRepos` 9 lines to 1 line. `Fsck` 6 lines to 1 line. 2. Only set defaultCommandExecutionTimeout when the option `setting.Git.Timeout.Default > 0`
* Use a more general (and faster) method to sanitize URLs with credentials ↵wxiaoguang2022-03-312-3/+3
| | | | | (#19239) Use a more general method to sanitize URLs with credentials: Simple and intuitive / Faster / Remove all credentials in all URLs
* Make git.OpenRepository accept Context (#19260)65432022-03-3020-46/+46
| | | | | * OpenRepositoryCtx -> OpenRepository * OpenRepository -> openRepositoryWithDefaultContext, only for internal usage
* Use full output of git show-ref --tags to get tags for PushUpdateAddTag (#19235)zeripath2022-03-296-88/+291
| | | | | | | | | | | | | | | | | Strangely #19038 appears to relate to an issue whereby a tag appears to be listed in `git show-ref --tags` but then does not appear when `git show-ref --tags -- short_name` is called. As a solution though I propose to stop the second call as it is unnecessary and only likely to cause problems. I've also noticed that the tags calls are wildly inefficient and aren't using the common cat-files - so these have been added. I've also noticed that the git commit-graph is not being written on mirroring - so I've also added writing this to the migration which should improve mirror rendering somewhat. Fix #19038 Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: 6543 <6543@obermui.de>
* Hide sensitive content on admin panel progress monitor (#19218)Lunny Xiao2022-03-272-1/+28
| | | | | | Sanitize urls within git process descriptions. Co-authored-by: wxiaoguang <wxiaoguang@gmail.com> Co-authored-by: Andrew Thornton <art27@cantab.net>
* Do not include global arguments in process manager (#19226)zeripath2022-03-271-8/+10
| | | | | | | | The git command by default adds a number of global arguments. These are not helpful to be displayed in the process manager and so should be skipped for default process descriptions. Signed-off-by: Andrew Thornton <art27@cantab.net>
* Make migrations SKIP_TLS_VERIFY apply to git too (#19132)zeripath2022-03-191-9/+13
| | | | | | | | Make SKIP_TLS_VERIFY apply to git data migrations too through adding the `-c http.sslVerify=false` option to the git clone command. Fix #18998 Signed-off-by: Andrew Thornton <art27@cantab.net>
* rm .sample hooks which aren't used (#19101)techknowlogick2022-03-1634-1950/+0
|
* Don't show context cancelled errors in attribute reader (#19006)zeripath2022-03-081-1/+3
| | | | | | Fix #18997 Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
* git backend ignore replace objects (#18979)65432022-03-021-0/+2
| | | | | * git backend ignore replace objects * comment
* Prevent dangling GetAttribute calls (#18754)zeripath2022-02-142-7/+12
| | | | | | It appears possible that there could be a hang due to unread data from the repo-attribute command pipes. This PR simply closes these during the defer. Signed-off-by: Andrew Thornton <art27@cantab.net>
* Fix isempty detection of git repository (#18746)Lunny Xiao2022-02-141-6/+5
| | | | | * Fix isempty detection of git repository * Fix IsEmpty check
* Change git.cmd to RunWithContext (#18693)Martin Scholz2022-02-1119-53/+209
| | | | | | | Change all `cmd...Pipeline` commands to `cmd.RunWithContext`. #18553 Co-authored-by: Martin Scholz <martin.scholz@versasec.com>
* Add apply-patch, basic revert and cherry-pick functionality (#17902)zeripath2022-02-091-2/+16
| | | | | | | | | | | | | | | This code adds a simple endpoint to apply patches to repositories and branches on gitea. This is then used along with the conflicting checking code in #18004 to provide a basic implementation of cherry-pick revert. Now because the buttons necessary for cherry-pick and revert have required us to create a dropdown next to the Browse Source button I've also implemented Create Branch and Create Tag operations. Fix #3880 Fix #17986 Signed-off-by: Andrew Thornton <art27@cantab.net>
* Delete old git.NewCommand() and use it as git.NewCommandContext() (#18552)65432022-02-0632-123/+118
|
* Stop logging an error when notes are not found (#18626)zeripath2022-02-062-0/+6
| | | | | | | This is an unnecessary logging event. Fix #18616 Signed-off-by: Andrew Thornton <art27@cantab.net>
* Remove go 1.15 support (#18511)Gusted2022-02-012-15/+0
| | | | | - Remove support for go 1.15(go.mod already requires go 1.16). Co-authored-by: John Olheiser <john.olheiser@gmail.com>
* Ensure git tag tests and others create test repos in tmpdir (#18447)zeripath2022-01-293-62/+206
| | | | | | | | | | | | | | | | | | | | * Ensure git tag tests and other create test repos in tmpdir There are a few places where tests appear to reuse testing repos which causes random CI failures. This PR simply changes these tests to ensure that cloning always happens into new temporary directories. Fix #18444 * Change log root for integration tests to use the REPO_TEST_DIR There is a potential race in the drone integration tests whereby test-mysql etc will start writing to log files causing make test-check fail. Fix #18077 Signed-off-by: Andrew Thornton <art27@cantab.net>
* Fix broken when no commits and default branch is not master (#18422)Lunny Xiao2022-01-281-7/+13
| | | | | | | | | * Fix broken when no commits and default branch is not master * Fix IsEmpty check * Improve codes * Add timeout
* Unexport git.GlobalCommandArgs (#18376)65432022-01-255-16/+32
| | | | Unexport the git.GlobalCommandArgs variable.
* Fix partial cloning a repo (#18373)Gusted2022-01-232-2/+5
| | | | | | | | - Pass the Global command args into serviceRPC. - Fixes error with partial cloning. - Add partial clone test - Include diff Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Always use git command but not os.Command (#18363)Lunny Xiao2022-01-231-18/+14
|
* format with gofumpt (#18184)65432022-01-2023-41/+38
| | | | | | | | | | | * gofumpt -w -l . * gofumpt -w -l -extra . * Add linter * manual fix * change make fmt
* Propagate context and ensure git commands run in request context (#17868)zeripath2022-01-1922-230/+147
| | | | | | | | | This PR continues the work in #17125 by progressively ensuring that git commands run within the request context. This now means that the if there is a git repo already open in the context it will be used instead of reopening it. Signed-off-by: Andrew Thornton <art27@cantab.net>
* Fix NPE on try to get tag reference via API (#18245)65432022-01-126-11/+4
| | | | | * fix npe * rm gitRepo from Tag
* Fix source typos (#18227)luzpaz2022-01-101-2/+2
| | | Follow-up to #18219
* Fix various typos (#18219)luzpaz2022-01-101-1/+1
| | | | | Found via `codespell -q 3 -S ./options/locale,./vendor -L ba,pullrequest,pullrequests,readby,te,unknwon` Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Enable partial clone by default (#18195)Gusted2022-01-061-0/+5
| | | | | - Enable partial clones(which are by default disabled from git) by default, unless configured otherwise. - Resolves #18190
* Do not read or write git reference files directly (#18079)zeripath2021-12-234-41/+29
| | | | | | | | | | Git will and can pack references into packfiles and therefore if you write/read the files directly you will get false results. Instead you should use update-ref and show-ref. To that end I have created three new functions in git/repo_commit.go that will do this correctly. Related #17191 Signed-off-by: Andrew Thornton <art27@cantab.net>
* Migrated Repository will show modifications when possible (#17191)99rgosse2021-12-234-2/+149
| | | | * Read patches to get history
* TestRepository_GetTag intermittently panics due to an NPE (#18043)zeripath2021-12-211-1/+7
| | | | | | | | | | | | | | | There are repeated panics in tests due to TestRepository_GetTag failing to run properly. This happens when we attempt to reset the internal repo for a tag which has failed to load. The problem is - the panic that this is causing is preventing us from finding what the real error is. This PR simply moves the failure out so we have a chance to see what really is failing. Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com> Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
* Simplify parameter types (#18006)Gusted2021-12-205-7/+7
| | | Remove repeated type declarations in function definitions.
* Prevent off-by-one error on comments on newly appended lines (#18029)zeripath2021-12-202-0/+53
| | | | | | | | | | | | | | | | | * Prevent off-by-one error on comments on newly appended lines There was a bug in CutDiffAroundLine whereby if a file without a terminal new line has a patch which appends lines to it and a comment is placed on one of those lines the comment diff will be a line out of place. This fixes CutDiffAroundLine to simply ignore the missing terminal newline - however, we should really improve this rendering to add a marker to say that there was a previously missing terminal newline. Fix #17875 Signed-off-by: Andrew Thornton <art27@cantab.net>
* Improve TestPatch to use git read-tree -m and implement git-merge-one-file ↵zeripath2021-12-191-1/+5
| | | | | | | | | | | | | functionality (#18004) The current TestPatch conflict code uses a plain git apply which does not properly account for 3-way merging. However, we can improve things using `git read-tree -m` to do a three-way merge then follow the algorithm used in merge-one-file. We can also use `--patience` and/or `--histogram` to generate a nicer diff for applying patches too. Fix #13679 Fix #6417 Signed-off-by: Andrew Thornton <art27@cantab.net>
* Prevent hang in git cat-file if repository is not a valid repository and ↵zeripath2021-12-163-1/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | other fixes (#17991) This PR contains multiple fixes. The most important of which is: * Prevent hang in git cat-file if the repository is not a valid repository Unfortunately it appears that if git cat-file is run in an invalid repository it will hang until stdin is closed. This will result in deadlocked /pulls pages and dangling git cat-file calls if a broken repository is tried to be reviewed or pulls exists for a broken repository. Fix #14734 Fix #9271 Fix #16113 Otherwise there are a few small other fixes included which this PR was initially intending to fix: * Fix panic on partial compares due to missing PullRequestWorkInProgressPrefixes * Fix links on pulls pages due to regression from #17551 - by making most /issues routes match /pulls too - Fix #17983 * Fix links on feeds pages due to another regression from #17551 but also fix issue with syncing tags - Fix #17943 * Add missing locale entries for oauth group claims * Prevent NPEs if ColorFormat is called on nil users, repos or teams.
* Improve checkBranchName (#17901)zeripath2021-12-084-22/+73
| | | | | | | | | | | | The current implementation of checkBranchName is highly inefficient involving opening the repository, the listing all of the branch names checking them individually before then using using opened repo to get the tags. This PR avoids this by simply walking the references from show-ref instead of opening the repository (in the nogogit case). Signed-off-by: Andrew Thornton <art27@cantab.net>
* Refactor various strings (#17784)mscherer2021-12-025-16/+25
| | | | | | | | Fixes #16478 Co-authored-by: Gusted <williamzijl7@hotmail.com> Co-authored-by: Gusted <williamzijl7@hotmail.com> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Make Requests Processes and create process hierarchy. Associate ↵zeripath2021-11-3031-149/+169
| | | | | | | | | OpenRepository with context. (#17125) This PR registers requests with the process manager and manages hierarchy within the processes. Git repos are then associated with a context, (usually the request's context) - with sub commands using this context as their base context. Signed-off-by: Andrew Thornton <art27@cantab.net>
* Simplify code for wrting SHA to name-rev (#17696)Gusted2021-11-181-17/+8
|
* Add .gitattribute assisted language detection to blame, diff and render (#17590)zeripath2021-11-174-60/+92
| | | | | | | Use check attribute code to check the assigned language of a file and send that in to chroma as a hint for the language of the file. Signed-off-by: Andrew Thornton <art27@cantab.net>
* A better go code formatter, and now `make fmt` can run in Windows (#17684)wxiaoguang2021-11-174-0/+4
| | | | * go build / format tools * re-format imports
* Multiple Escaping Improvements (#17551)zeripath2021-11-161-1/+3
| | | | | | | | | | | | | | There are multiple places where Gitea does not properly escape URLs that it is building and there are multiple places where it builds urls when there is already a simpler function available to use this. This is an extensive PR attempting to fix these issues. 1. The first commit in this PR looks through all href, src and links in the Gitea codebase and has attempted to catch all the places where there is potentially incomplete escaping. 2. Whilst doing this we will prefer to use functions that create URLs over recreating them by hand. 3. All uses of strings should be directly escaped - even if they are not currently expected to contain escaping characters. The main benefit to doing this will be that we can consider relaxing the constraints on user names and reponames in future. 4. The next commit looks at escaping in the wiki and re-considers the urls that are used there. Using the improved escaping here wiki files containing '/'. (This implementation will currently still place all of the wiki files the root directory of the repo but this would not be difficult to change.) 5. The title generation in feeds is now properly escaped. 6. EscapePound is no longer needed - urls should be PathEscaped / QueryEscaped as necessary but then re-escaped with Escape when creating html with locales Signed-off-by: Andrew Thornton <art27@cantab.net> Signed-off-by: Andrew Thornton <art27@cantab.net>
generated by cgit v1.2.3 (git 2.39.1) at 2025-03-24 06:32:27 +0000