summaryrefslogtreecommitdiffstats
path: root/services/cron/tasks_basic.go
Commit message (Collapse)AuthorAgeFilesLines
* Artifacts retention and auto clean up (#26131)FuXiaoHei2023-09-061-0/+18
| | | | | | | | | | | | | | Currently, Artifact does not have an expiration and automatic cleanup mechanism, and this feature needs to be added. It contains the following key points: - [x] add global artifact retention days option in config file. Default value is 90 days. - [x] add cron task to clean up expired artifacts. It should run once a day. - [x] support custom retention period from `retention-days: 5` in `upload-artifact@v3`. - [x] artifacts link in actions view should be non-clickable text when expired.
* Allow package cleanup from admin page (#25307)KN4CK3R2023-08-081-1/+1
| | | | | | | | | | | | | | | Until now expired package data gets deleted daily by a cronjob. The admin page shows the size of all packages and the size of unreferenced data. The users (#25035, #20631) expect the deletion of this data if they run the cronjob from the admin page but the job only deletes data older than 24h. This PR adds a new button which deletes all expired data. ![grafik](https://github.com/go-gitea/gitea/assets/1666336/b3e35d73-9496-4fa7-a20c-e5d30b1f6850) --------- Co-authored-by: silverwind <me@silverwind.io>
* Add Cargo package registry (#21888)KN4CK3R2023-02-051-2/+2
| | | | | | | | | | | | | | | | | | This PR implements a [Cargo registry](https://doc.rust-lang.org/cargo/) to manage Rust packages. This package type was a little bit more complicated because Cargo needs an additional Git repository to store its package index. Screenshots: ![grafik](https://user-images.githubusercontent.com/1666336/203102004-08d812ac-c066-4969-9bda-2fed818554eb.png) ![grafik](https://user-images.githubusercontent.com/1666336/203102141-d9970f14-dca6-4174-b17a-50ba1bd79087.png) ![grafik](https://user-images.githubusercontent.com/1666336/203102244-dc05743b-78b6-4d97-998e-ef76341a978f.png) --------- Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Refactor git command package to improve security and maintainability (#22678)wxiaoguang2023-02-041-5/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This PR follows #21535 (and replace #22592) ## Review without space diff https://github.com/go-gitea/gitea/pull/22678/files?diff=split&w=1 ## Purpose of this PR 1. Make git module command completely safe (risky user inputs won't be passed as argument option anymore) 2. Avoid low-level mistakes like https://github.com/go-gitea/gitea/pull/22098#discussion_r1045234918 3. Remove deprecated and dirty `CmdArgCheck` function, hide the `CmdArg` type 4. Simplify code when using git command ## The main idea of this PR * Move the `git.CmdArg` to the `internal` package, then no other package except `git` could use it. Then developers could never do `AddArguments(git.CmdArg(userInput))` any more. * Introduce `git.ToTrustedCmdArgs`, it's for user-provided and already trusted arguments. It's only used in a few cases, for example: use git arguments from config file, help unit test with some arguments. * Introduce `AddOptionValues` and `AddOptionFormat`, they make code more clear and simple: * Before: `AddArguments("-m").AddDynamicArguments(message)` * After: `AddOptionValues("-m", message)` * - * Before: `AddArguments(git.CmdArg(fmt.Sprintf("--author='%s <%s>'", sig.Name, sig.Email)))` * After: `AddOptionFormat("--author='%s <%s>'", sig.Name, sig.Email)` ## FAQ ### Why these changes were not done in #21535 ? #21535 is mainly a search&replace, it did its best to not change too much logic. Making the framework better needs a lot of changes, so this separate PR is needed as the second step. ### The naming of `AddOptionXxx` According to git's manual, the `--xxx` part is called `option`. ### How can it guarantee that `internal.CmdArg` won't be not misused? Go's specification guarantees that. Trying to access other package's internal package causes compilation error. And, `golangci-lint` also denies the git/internal package. Only the `git/command.go` can use it carefully. ### There is still a `ToTrustedCmdArgs`, will it still allow developers to make mistakes and pass untrusted arguments? Generally speaking, no. Because when using `ToTrustedCmdArgs`, the code will be very complex (see the changes for examples). Then developers and reviewers can know that something might be unreasonable. ### Why there was a `CmdArgCheck` and why it's removed? At the moment of #21535, to reduce unnecessary changes, `CmdArgCheck` was introduced as a hacky patch. Now, almost all code could be written as `cmd := NewCommand(); cmd.AddXxx(...)`, then there is no need for `CmdArgCheck` anymore. ### Why many codes for `signArg == ""` is deleted? Because in the old code, `signArg` could never be empty string, it's either `-S[key-id]` or `--no-gpg-sign`. So the `signArg == ""` is just dead code. --------- Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Add doctor command for full GC of LFS (#21978)zeripath2022-12-151-1/+1
| | | | | | | | | | | | | | The recent PR adding orphaned checks to the LFS storage is not sufficient to completely GC LFS, as it is possible for LFSMetaObjects to remain associated with repos but still need to be garbage collected. Imagine a situation where a branch is uploaded containing LFS files but that branch is later completely deleted. The LFSMetaObjects will remain associated with the Repository but the Repository will no longer contain any pointers to the object. This PR adds a second doctor command to perform a full GC. Signed-off-by: Andrew Thornton <art27@cantab.net>
* Implement FSFE REUSE for golang files (#21840)flynnnnnnnnnn2022-11-271-2/+1
| | | | | | | | | Change all license headers to comply with REUSE specification. Fix #16132 Co-authored-by: flynnnnnnnnnn <flynnnnnnnnnn@github> Co-authored-by: John Olheiser <john.olheiser@gmail.com>
* Refactor git command arguments and make all arguments to be safe to be used ↵wxiaoguang2022-10-231-1/+7
| | | | | | | (#21535) Follow #21464 Make all git command arguments strictly safe. Most changes are one-to-one replacing, keep all existing logic.
* Move some code into models/git (#19879)Lunny Xiao2022-06-121-1/+2
| | | | | | | | | | | | | | | | | | | * Move access and repo permission to models/perm/access * fix test * Move some git related files into sub package models/git * Fix build * fix git test * move lfs to sub package * move more git related functions to models/git * Move functions sequence * Some improvements per @KN4CK3R and @delvh
* Disable unnecessary mirroring elements (#18527)Paweł Bogusławski2022-06-041-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Disable unnecessary mirroring elements This mod fixes disabling unnecessary mirroring elements. Related: https://github.com/go-gitea/gitea/pull/16957 Related: https://github.com/go-gitea/gitea/pull/13084 Author-Change-Id: IB#1105104 * Checkbox rendering disabled instead of hiding it Fixes: 02b45051503d4330da9757ff084c9cc5e6e60d84 Related: https://github.com/go-gitea/gitea/pull/18527#pullrequestreview-878061913 Author-Change-Id: IB#1105104 * Update custom/conf/app.example.ini Co-authored-by: silverwind <me@silverwind.io> * Update docs/content/doc/advanced/config-cheat-sheet.en-us.md Co-authored-by: silverwind <me@silverwind.io> * Mirror filter removed only when whole mirroring feature is disabled Fixes: 02b45051503d4330da9757ff084c9cc5e6e60d84 Related: https://github.com/go-gitea/gitea/pull/18527#discussion_r883268890 Author-Change-Id: IB#1105104 Co-authored-by: silverwind <me@silverwind.io>
* Add Package Registry (#16510)KN4CK3R2022-03-301-0/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Added package store settings. * Added models. * Added generic package registry. * Added tests. * Added NuGet package registry. * Moved service index to api file. * Added NPM package registry. * Added Maven package registry. * Added PyPI package registry. * Summary is deprecated. * Changed npm name. * Sanitize project url. * Allow only scoped packages. * Added user interface. * Changed method name. * Added missing migration file. * Set page info. * Added documentation. * Added documentation links. * Fixed wrong error message. * Lint template files. * Fixed merge errors. * Fixed unit test storage path. * Switch to json module. * Added suggestions. * Added package webhook. * Add package api. * Fixed swagger file. * Fixed enum and comments. * Fixed NuGet pagination. * Print test names. * Added api tests. * Fixed access level. * Fix User unmarshal. * Added RubyGems package registry. * Fix lint. * Implemented io.Writer. * Added support for sha256/sha512 checksum files. * Improved maven-metadata.xml support. * Added support for symbol package uploads. * Added tests. * Added overview docs. * Added npm dependencies and keywords. * Added no-packages information. * Display file size. * Display asset count. * Fixed filter alignment. * Added package icons. * Formatted instructions. * Allow anonymous package downloads. * Fixed comments. * Fixed postgres test. * Moved file. * Moved models to models/packages. * Use correct error response format per client. * Use simpler search form. * Fixed IsProd. * Restructured data model. * Prevent empty filename. * Fix swagger. * Implemented user/org registry. * Implemented UI. * Use GetUserByIDCtx. * Use table for dependencies. * make svg * Added support for unscoped npm packages. * Add support for npm dist tags. * Added tests for npm tags. * Unlink packages if repository gets deleted. * Prevent user/org delete if a packages exist. * Use package unlink in repository service. * Added support for composer packages. * Restructured package docs. * Added missing tests. * Fixed generic content page. * Fixed docs. * Fixed swagger. * Added missing type. * Fixed ambiguous column. * Organize content store by sha256 hash. * Added admin package management. * Added support for sorting. * Add support for multiple identical versions/files. * Added missing repository unlink. * Added file properties. * make fmt * lint * Added Conan package registry. * Updated docs. * Unify package names. * Added swagger enum. * Use longer TEXT column type. * Removed version composite key. * Merged package and container registry. * Removed index. * Use dedicated package router. * Moved files to new location. * Updated docs. * Fixed JOIN order. * Fixed GROUP BY statement. * Fixed GROUP BY #2. * Added symbol server support. * Added more tests. * Set NOT NULL. * Added setting to disable package registries. * Moved auth into service. * refactor * Use ctx everywhere. * Added package cleanup task. * Changed packages path. * Added container registry. * Refactoring * Updated comparison. * Fix swagger. * Fixed table order. * Use token auth for npm routes. * Enabled ReverseProxy auth. * Added packages link for orgs. * Fixed anonymous org access. * Enable copy button for setup instructions. * Merge error * Added suggestions. * Fixed merge. * Handle "generic". * Added link for TODO. * Added suggestions. * Changed temporary buffer filename. * Added suggestions. * Apply suggestions from code review Co-authored-by: Thomas Boerger <thomas@webhippie.de> * Update docs/content/doc/packages/nuget.en-us.md Co-authored-by: wxiaoguang <wxiaoguang@gmail.com> Co-authored-by: Thomas Boerger <thomas@webhippie.de>
* Make cron task no notice on success (#19221)zeripath2022-03-261-4/+3
| | | | | | | | | | | | | | | | | | Change all cron tasks to make them no notice on success default. Instead if a user wants notices on success they need to add NOTICE_ON_SUCCESS=true instead. ## :warning: BREAKING :warning: This changes the cron config so that notices on success are no longer set by default and breaks NO_SUCCESS_NOTICE settings. Instead users who want notices on success must set NOTICE_ON_SUCCESS=true instead. Signed-off-by: Andrew Thornton <art27@cantab.net> * Update custom/conf/app.example.ini Co-authored-by: Norwin <noerw@users.noreply.github.com> Co-authored-by: Norwin <noerw@users.noreply.github.com>
* Move repo archiver to models/repo (#17913)Lunny Xiao2021-12-061-3/+4
| | | | | | | | | | | * Move repo archiver to models/repo * Move archiver service into services/repository/ * Fix imports * Fix test * Fix test
* Move user related model into models/user (#17781)Lunny Xiao2021-11-241-8/+9
| | | | | | | | | | | | | * Move user related model into models/user * Fix lint for windows * Fix windows lint * Fix windows lint * Move some tests in models * Merge
* Move repofiles from modules/repofiles to services/repository/files (#17774)Lunny Xiao2021-11-241-1/+1
| | | | | | | | | * Move repofiles from modules to services * rename services/repository/repofiles -> services/repository/files * Fix test Co-authored-by: 6543 <6543@obermui.de>
* Add `PULL_LIMIT` and `PUSH_LIMIT` to cron.update_mirror task (#17568)zeripath2021-11-221-7/+18
|
* Move migrations into services and base into modules/migration (#17663)Lunny Xiao2021-11-161-1/+1
| | | | | | | * Move migrtions into services and base into modules/migration * Fix imports * Fix lint
* Move some functions into services/repository (#17660)Lunny Xiao2021-11-161-0/+141