summaryrefslogtreecommitdiffstats
path: root/modules/gitgraph
Commit message (Collapse)AuthorAgeFilesLines
* Add context cache as a request level cache (#22294)Lunny Xiao2023-02-151-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | To avoid duplicated load of the same data in an HTTP request, we can set a context cache to do that. i.e. Some pages may load a user from a database with the same id in different areas on the same page. But the code is hidden in two different deep logic. How should we share the user? As a result of this PR, now if both entry functions accept `context.Context` as the first parameter and we just need to refactor `GetUserByID` to reuse the user from the context cache. Then it will not be loaded twice on an HTTP request. But of course, sometimes we would like to reload an object from the database, that's why `RemoveContextData` is also exposed. The core context cache is here. It defines a new context ```go type cacheContext struct { ctx context.Context data map[any]map[any]any lock sync.RWMutex } var cacheContextKey = struct{}{} func WithCacheContext(ctx context.Context) context.Context { return context.WithValue(ctx, cacheContextKey, &cacheContext{ ctx: ctx, data: make(map[any]map[any]any), }) } ``` Then you can use the below 4 methods to read/write/del the data within the same context. ```go func GetContextData(ctx context.Context, tp, key any) any func SetContextData(ctx context.Context, tp, key, value any) func RemoveContextData(ctx context.Context, tp, key any) func GetWithContextCache[T any](ctx context.Context, cacheGroupKey string, cacheTargetID any, f func() (T, error)) (T, error) ``` Then let's take a look at how `system.GetString` implement it. ```go func GetSetting(ctx context.Context, key string) (string, error) { return cache.GetWithContextCache(ctx, contextCacheKey, key, func() (string, error) { return cache.GetString(genSettingCacheKey(key), func() (string, error) { res, err := GetSettingNoCache(ctx, key) if err != nil { return "", err } return res.SettingValue, nil }) }) } ``` First, it will check if context data include the setting object with the key. If not, it will query from the global cache which may be memory or a Redis cache. If not, it will get the object from the database. In the end, if the object gets from the global cache or database, it will be set into the context cache. An object stored in the context cache will only be destroyed after the context disappeared.
* Refactor git command package to improve security and maintainability (#22678)wxiaoguang2023-02-041-7/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This PR follows #21535 (and replace #22592) ## Review without space diff https://github.com/go-gitea/gitea/pull/22678/files?diff=split&w=1 ## Purpose of this PR 1. Make git module command completely safe (risky user inputs won't be passed as argument option anymore) 2. Avoid low-level mistakes like https://github.com/go-gitea/gitea/pull/22098#discussion_r1045234918 3. Remove deprecated and dirty `CmdArgCheck` function, hide the `CmdArg` type 4. Simplify code when using git command ## The main idea of this PR * Move the `git.CmdArg` to the `internal` package, then no other package except `git` could use it. Then developers could never do `AddArguments(git.CmdArg(userInput))` any more. * Introduce `git.ToTrustedCmdArgs`, it's for user-provided and already trusted arguments. It's only used in a few cases, for example: use git arguments from config file, help unit test with some arguments. * Introduce `AddOptionValues` and `AddOptionFormat`, they make code more clear and simple: * Before: `AddArguments("-m").AddDynamicArguments(message)` * After: `AddOptionValues("-m", message)` * - * Before: `AddArguments(git.CmdArg(fmt.Sprintf("--author='%s <%s>'", sig.Name, sig.Email)))` * After: `AddOptionFormat("--author='%s <%s>'", sig.Name, sig.Email)` ## FAQ ### Why these changes were not done in #21535 ? #21535 is mainly a search&replace, it did its best to not change too much logic. Making the framework better needs a lot of changes, so this separate PR is needed as the second step. ### The naming of `AddOptionXxx` According to git's manual, the `--xxx` part is called `option`. ### How can it guarantee that `internal.CmdArg` won't be not misused? Go's specification guarantees that. Trying to access other package's internal package causes compilation error. And, `golangci-lint` also denies the git/internal package. Only the `git/command.go` can use it carefully. ### There is still a `ToTrustedCmdArgs`, will it still allow developers to make mistakes and pass untrusted arguments? Generally speaking, no. Because when using `ToTrustedCmdArgs`, the code will be very complex (see the changes for examples). Then developers and reviewers can know that something might be unreasonable. ### Why there was a `CmdArgCheck` and why it's removed? At the moment of #21535, to reduce unnecessary changes, `CmdArgCheck` was introduced as a hacky patch. Now, almost all code could be written as `cmd := NewCommand(); cmd.AddXxx(...)`, then there is no need for `CmdArgCheck` anymore. ### Why many codes for `signArg == ""` is deleted? Because in the old code, `signArg` could never be empty string, it's either `-S[key-id]` or `--no-gpg-sign`. So the `signArg == ""` is just dead code. --------- Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Implement FSFE REUSE for golang files (#21840)flynnnnnnnnnn2022-11-274-8/+4
| | | | | | | | | Change all license headers to comply with REUSE specification. Fix #16132 Co-authored-by: flynnnnnnnnnn <flynnnnnnnnnn@github> Co-authored-by: John Olheiser <john.olheiser@gmail.com>
* Refactor git command arguments and make all arguments to be safe to be used ↵wxiaoguang2022-10-231-4/+3
| | | | | | | (#21535) Follow #21464 Make all git command arguments strictly safe. Most changes are one-to-one replacing, keep all existing logic.
* alternative to PR "improve code quality" (#21464)wxiaoguang2022-10-151-12/+7
| | | | | This PR doesn't require new git version, and can be backported easily. Co-authored-by: 6543 <6543@obermui.de>
* Add more linters to improve code readability (#19989)Wim2022-06-201-1/+1
| | | | | | | | | | Add nakedret, unconvert, wastedassign, stylecheck and nolintlint linters to improve code readability - nakedret - https://github.com/alexkohler/nakedret - nakedret is a Go static analysis tool to find naked returns in functions greater than a specified function length. - unconvert - https://github.com/mdempsky/unconvert - Remove unnecessary type conversions - wastedassign - https://github.com/sanposhiho/wastedassign - wastedassign finds wasted assignment statements. - notlintlint - Reports ill-formed or insufficient nolint directives - stylecheck - https://staticcheck.io/docs/checks/#ST - keep style consistent - excluded: [ST1003 - Poorly chosen identifier](https://staticcheck.io/docs/checks/#ST1003) and [ST1005 - Incorrectly formatted error string](https://staticcheck.io/docs/checks/#ST1005)
* Move some code into models/git (#19879)Lunny Xiao2022-06-121-5/+5
| | | | | | | | | | | | | | | | | | | * Move access and repo permission to models/perm/access * fix test * Move some git related files into sub package models/git * Fix build * fix git test * move lfs to sub package * move more git related functions to models/git * Move functions sequence * Some improvements per @KN4CK3R and @delvh
* Move almost all functions' parameter db.Engine to context.Context (#19748)Lunny Xiao2022-05-201-1/+1
| | | | * Move almost all functions' parameter db.Engine to context.Context * remove some unnecessary wrap functions
* Remove `git.Command.Run` and `git.Command.RunInDir*` (#19280)wxiaoguang2022-04-011-5/+4
| | | | | | Follows #19266, #8553, Close #18553, now there are only three `Run..(&RunOpts{})` functions. * before: `stdout, err := RunInDir(path)` * now: `stdout, _, err := RunStdString(&git.RunOpts{Dir:path})`
* Make git.OpenRepository accept Context (#19260)65432022-03-301-1/+1
| | | | | * OpenRepositoryCtx -> OpenRepository * OpenRepository -> openRepositoryWithDefaultContext, only for internal usage
* Change git.cmd to RunWithContext (#18693)Martin Scholz2022-02-111-42/+48
| | | | | | | Change all `cmd...Pipeline` commands to `cmd.RunWithContext`. #18553 Co-authored-by: Martin Scholz <martin.scholz@versasec.com>
* Delete old git.NewCommand() and use it as git.NewCommandContext() (#18552)65432022-02-061-1/+1
|
* Collaborator trust model should trust collaborators (#18539)zeripath2022-02-021-1/+1
| | | | | | | | | | * Collaborator trust model should trust collaborators There was an unintended regression in #17917 which leads to only repository admin commits being trusted. This PR restores the old logic. Fix #18501 Signed-off-by: Andrew Thornton <art27@cantab.net>
* format with gofumpt (#18184)65432022-01-202-3/+0
| | | | | | | | | | | * gofumpt -w -l . * gofumpt -w -l -extra . * Add linter * manual fix * change make fmt
* Propagate context and ensure git commands run in request context (#17868)zeripath2022-01-191-1/+1
| | | | | | | | | This PR continues the work in #17125 by progressively ensuring that git commands run within the request context. This now means that the if there is a git repo already open in the context it will be used instead of reopening it. Signed-off-by: Andrew Thornton <art27@cantab.net>
* Simplify parameter types (#18006)Gusted2021-12-201-1/+1
| | | Remove repeated type declarations in function definitions.
* Add missing `X-Total-Count` and fix some related bugs (#17968)qwerty2872021-12-151-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Add missing `X-Total-Count` and fix some related bugs Adds `X-Total-Count` header to APIs that return a list but doesn't have it yet. Fixed bugs: * not returned after reporting error (https://github.com/qwerty287/gitea/blob/39eb82446c6fe5da3d79124e1f701f3795625b69/routers/api/v1/user/star.go#L70) * crash with index out of bounds, API issue/issueSubscriptions I also found various endpoints that return lists but do not apply/support pagination yet: ``` /repos/{owner}/{repo}/issues/{index}/labels /repos/{owner}/{repo}/issues/comments/{id}/reactions /repos/{owner}/{repo}/branch_protections /repos/{owner}/{repo}/contents /repos/{owner}/{repo}/hooks/git /repos/{owner}/{repo}/issue_templates /repos/{owner}/{repo}/releases/{id}/assets /repos/{owner}/{repo}/reviewers /repos/{owner}/{repo}/teams /user/emails /users/{username}/heatmap ``` If this is not expected, an new issue should be opened. Closes #13043 * fmt * Update routers/api/v1/repo/issue_subscription.go Co-authored-by: KN4CK3R <admin@oldschoolhack.me> * Use FindAndCount Co-authored-by: KN4CK3R <admin@oldschoolhack.me> Co-authored-by: 6543 <6543@obermui.de>
* Move keys to models/asymkey (#17917)Lunny Xiao2021-12-101-3/+6
| | | | | | | | | | | | | | | | | | | * Move keys to models/keys * Rename models/keys -> models/asymkey * change the missed package name * Fix package alias * Fix test * Fix docs * Fix test * Fix test * merge
* Move repository model into models/repo (#17933)Lunny Xiao2021-12-101-1/+2
| | | | | | | | | | | | | | | * Some refactors related repository model * Move more methods out of repository * Move repository into models/repo * Fix test * Fix test * some improvements * Remove unnecessary function
* Refactor various strings (#17784)mscherer2021-12-021-1/+1
| | | | | | | | Fixes #16478 Co-authored-by: Gusted <williamzijl7@hotmail.com> Co-authored-by: Gusted <williamzijl7@hotmail.com> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Move user related model into models/user (#17781)Lunny Xiao2021-11-241-3/+4
| | | | | | | | | | | | | * Move user related model into models/user * Fix lint for windows * Fix windows lint * Fix windows lint * Move some tests in models * Merge
* Remove unnecessary variable assignments (#17695)Gusted2021-11-181-2/+1
| | | | | | | | | | * Remove unnecessary variable assignments As title * enable ineffassign Co-authored-by: zeripath <art27@cantab.net> Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
* Fix some lints (#17337)Lunny Xiao2021-10-171-1/+1
| | | Fix some linting problems.
* Move login related structs and functions to models/login (#17093)Lunny Xiao2021-09-241-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Move login related structs and functions to models/login * Fix test * Fix lint * Fix lint * Fix lint of windows * Fix lint * Fix test * Fix test * Only load necessary fixtures when preparing unit tests envs * Fix lint * Fix test * Fix test * Fix error log * Fix error log * Fix error log * remove unnecessary change * fix error log * merge main branch
* Fix bug on commit graph (#15517)Lunny Xiao2021-04-171-4/+5
|
* [Refactor] CombinedStatus and CommitStatus related functions & structs (#14026)65432020-12-181-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * RM unused struct * rename (*CommitStatus) loadRepo() -> loadAttributes() * move ToCommitStatus into its own file * use CommitStatusState instead of StatusState * move CombinedStatus convertion into convert package * let models.GetLatestCommitStatus use repoID direct and accept ListOptions * update swagger docs * fix tests * Fix swagger docs * rm page * fix swagger docs!!! * return json null * always return json * rename api.Status to api.CommitStatus * fix swagger docs * sec swagger fix
* Multiple GitGraph improvements: Exclude PR heads, Add branch/PR links, Show ↵zeripath2020-11-083-33/+116
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | only certain branches, (#12766) * Multiple GitGraph improvements. Add backend support for excluding PRs, selecting branches and files. Fix #10327 Signed-off-by: Andrew Thornton <art27@cantab.net> * as per @silverwind Signed-off-by: Andrew Thornton <art27@cantab.net> * as per @silverwind Signed-off-by: Andrew Thornton <art27@cantab.net> * Only show refs in dropdown we display on the graph Signed-off-by: Andrew Thornton <art27@cantab.net> * as per @silverwind Signed-off-by: Andrew Thornton <art27@cantab.net> * use flexbox for ui header Signed-off-by: Andrew Thornton <art27@cantab.net> * Move Hide Pull Request button to the dropdown Signed-off-by: Andrew Thornton <art27@cantab.net> * Add SHA and user pictures Signed-off-by: Andrew Thornton <art27@cantab.net> * fix test Signed-off-by: Andrew Thornton <art27@cantab.net> * fix test 2 Signed-off-by: Andrew Thornton <art27@cantab.net> * fixes * async * more tweaks * use tabs in tmpl Signed-off-by: Andrew Thornton <art27@cantab.net> * remove commented thing Signed-off-by: Andrew Thornton <art27@cantab.net> * fix linting Signed-off-by: Andrew Thornton <art27@cantab.net> * Update web_src/js/features/gitgraph.js Co-authored-by: silverwind <me@silverwind.io> * graph tweaks * more tweaks * add title Signed-off-by: Andrew Thornton <art27@cantab.net> * fix loading indicator z-index and position Co-authored-by: silverwind <me@silverwind.io> Co-authored-by: techknowlogick <techknowlogick@gitea.io> Co-authored-by: Lauris BH <lauris@nix.lv>
* Render the git graph on the server (#12333)zeripath2020-08-064-89/+1209
| | | | | | | | | | | | | Rendering the git graph on the server means that we can properly track flows and switch from the Canvas implementation to a SVG implementation. * This implementation provides a 16 limited color selection * The uniqued color numbers are also provided * And there is also a monochrome version *In addition is a hover highlight that allows users to highlight commits on the same flow. Closes #12209 Signed-off-by: Andrew Thornton art27@cantab.net Co-authored-by: silverwind <me@silverwind.io>
* Ensure that git commit tree continues properly over the page (#12142)zeripath2020-07-161-14/+53
| | | | | | | | | | | | | | | * Ensure that git commit tree continues properly over the page Signed-off-by: Andrew Thornton <art27@cantab.net> * Avoid generating strings when skipping Signed-off-by: Andrew Thornton <art27@cantab.net> * skip initial non-commit-lines Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: Lauris BH <lauris@nix.lv>
* Move git graph from models to modules/graph (#9027)Lunny Xiao2019-11-162-0/+186