You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

repo.go 28KB

Add Package Registry (#16510) * Added package store settings. * Added models. * Added generic package registry. * Added tests. * Added NuGet package registry. * Moved service index to api file. * Added NPM package registry. * Added Maven package registry. * Added PyPI package registry. * Summary is deprecated. * Changed npm name. * Sanitize project url. * Allow only scoped packages. * Added user interface. * Changed method name. * Added missing migration file. * Set page info. * Added documentation. * Added documentation links. * Fixed wrong error message. * Lint template files. * Fixed merge errors. * Fixed unit test storage path. * Switch to json module. * Added suggestions. * Added package webhook. * Add package api. * Fixed swagger file. * Fixed enum and comments. * Fixed NuGet pagination. * Print test names. * Added api tests. * Fixed access level. * Fix User unmarshal. * Added RubyGems package registry. * Fix lint. * Implemented io.Writer. * Added support for sha256/sha512 checksum files. * Improved maven-metadata.xml support. * Added support for symbol package uploads. * Added tests. * Added overview docs. * Added npm dependencies and keywords. * Added no-packages information. * Display file size. * Display asset count. * Fixed filter alignment. * Added package icons. * Formatted instructions. * Allow anonymous package downloads. * Fixed comments. * Fixed postgres test. * Moved file. * Moved models to models/packages. * Use correct error response format per client. * Use simpler search form. * Fixed IsProd. * Restructured data model. * Prevent empty filename. * Fix swagger. * Implemented user/org registry. * Implemented UI. * Use GetUserByIDCtx. * Use table for dependencies. * make svg * Added support for unscoped npm packages. * Add support for npm dist tags. * Added tests for npm tags. * Unlink packages if repository gets deleted. * Prevent user/org delete if a packages exist. * Use package unlink in repository service. * Added support for composer packages. * Restructured package docs. * Added missing tests. * Fixed generic content page. * Fixed docs. * Fixed swagger. * Added missing type. * Fixed ambiguous column. * Organize content store by sha256 hash. * Added admin package management. * Added support for sorting. * Add support for multiple identical versions/files. * Added missing repository unlink. * Added file properties. * make fmt * lint * Added Conan package registry. * Updated docs. * Unify package names. * Added swagger enum. * Use longer TEXT column type. * Removed version composite key. * Merged package and container registry. * Removed index. * Use dedicated package router. * Moved files to new location. * Updated docs. * Fixed JOIN order. * Fixed GROUP BY statement. * Fixed GROUP BY #2. * Added symbol server support. * Added more tests. * Set NOT NULL. * Added setting to disable package registries. * Moved auth into service. * refactor * Use ctx everywhere. * Added package cleanup task. * Changed packages path. * Added container registry. * Refactoring * Updated comparison. * Fix swagger. * Fixed table order. * Use token auth for npm routes. * Enabled ReverseProxy auth. * Added packages link for orgs. * Fixed anonymous org access. * Enable copy button for setup instructions. * Merge error * Added suggestions. * Fixed merge. * Handle "generic". * Added link for TODO. * Added suggestions. * Changed temporary buffer filename. * Added suggestions. * Apply suggestions from code review Co-authored-by: Thomas Boerger <thomas@webhippie.de> * Update docs/content/doc/packages/nuget.en-us.md Co-authored-by: wxiaoguang <wxiaoguang@gmail.com> Co-authored-by: Thomas Boerger <thomas@webhippie.de>
2 years ago
Implement actions (#21937) Close #13539. Co-authored by: @lunny @appleboy @fuxiaohei and others. Related projects: - https://gitea.com/gitea/actions-proto-def - https://gitea.com/gitea/actions-proto-go - https://gitea.com/gitea/act - https://gitea.com/gitea/act_runner ### Summary The target of this PR is to bring a basic implementation of "Actions", an internal CI/CD system of Gitea. That means even though it has been merged, the state of the feature is **EXPERIMENTAL**, and please note that: - It is disabled by default; - It shouldn't be used in a production environment currently; - It shouldn't be used in a public Gitea instance currently; - Breaking changes may be made before it's stable. **Please comment on #13539 if you have any different product design ideas**, all decisions reached there will be adopted here. But in this PR, we don't talk about **naming, feature-creep or alternatives**. ### ⚠️ Breaking `gitea-actions` will become a reserved user name. If a user with the name already exists in the database, it is recommended to rename it. ### Some important reviews - What is `DEFAULT_ACTIONS_URL` in `app.ini` for? - https://github.com/go-gitea/gitea/pull/21937#discussion_r1055954954 - Why the api for runners is not under the normal `/api/v1` prefix? - https://github.com/go-gitea/gitea/pull/21937#discussion_r1061173592 - Why DBFS? - https://github.com/go-gitea/gitea/pull/21937#discussion_r1061301178 - Why ignore events triggered by `gitea-actions` bot? - https://github.com/go-gitea/gitea/pull/21937#discussion_r1063254103 - Why there's no permission control for actions? - https://github.com/go-gitea/gitea/pull/21937#discussion_r1090229868 ### What it looks like <details> #### Manage runners <img width="1792" alt="image" src="https://user-images.githubusercontent.com/9418365/205870657-c72f590e-2e08-4cd4-be7f-2e0abb299bbf.png"> #### List runs <img width="1792" alt="image" src="https://user-images.githubusercontent.com/9418365/205872794-50fde990-2b45-48c1-a178-908e4ec5b627.png"> #### View logs <img width="1792" alt="image" src="https://user-images.githubusercontent.com/9418365/205872501-9b7b9000-9542-4991-8f55-18ccdada77c3.png"> </details> ### How to try it <details> #### 1. Start Gitea Clone this branch and [install from source](https://docs.gitea.io/en-us/install-from-source). Add additional configurations in `app.ini` to enable Actions: ```ini [actions] ENABLED = true ``` Start it. If all is well, you'll see the management page of runners: <img width="1792" alt="image" src="https://user-images.githubusercontent.com/9418365/205877365-8e30a780-9b10-4154-b3e8-ee6c3cb35a59.png"> #### 2. Start runner Clone the [act_runner](https://gitea.com/gitea/act_runner), and follow the [README](https://gitea.com/gitea/act_runner/src/branch/main/README.md) to start it. If all is well, you'll see a new runner has been added: <img width="1792" alt="image" src="https://user-images.githubusercontent.com/9418365/205878000-216f5937-e696-470d-b66c-8473987d91c3.png"> #### 3. Enable actions for a repo Create a new repo or open an existing one, check the `Actions` checkbox in settings and submit. <img width="1792" alt="image" src="https://user-images.githubusercontent.com/9418365/205879705-53e09208-73c0-4b3e-a123-2dcf9aba4b9c.png"> <img width="1792" alt="image" src="https://user-images.githubusercontent.com/9418365/205879383-23f3d08f-1a85-41dd-a8b3-54e2ee6453e8.png"> If all is well, you'll see a new tab "Actions": <img width="1792" alt="image" src="https://user-images.githubusercontent.com/9418365/205881648-a8072d8c-5803-4d76-b8a8-9b2fb49516c1.png"> #### 4. Upload workflow files Upload some workflow files to `.gitea/workflows/xxx.yaml`, you can follow the [quickstart](https://docs.github.com/en/actions/quickstart) of GitHub Actions. Yes, Gitea Actions is compatible with GitHub Actions in most cases, you can use the same demo: ```yaml name: GitHub Actions Demo run-name: ${{ github.actor }} is testing out GitHub Actions 🚀 on: [push] jobs: Explore-GitHub-Actions: runs-on: ubuntu-latest steps: - run: echo "🎉 The job was automatically triggered by a ${{ github.event_name }} event." - run: echo "🐧 This job is now running on a ${{ runner.os }} server hosted by GitHub!" - run: echo "🔎 The name of your branch is ${{ github.ref }} and your repository is ${{ github.repository }}." - name: Check out repository code uses: actions/checkout@v3 - run: echo "💡 The ${{ github.repository }} repository has been cloned to the runner." - run: echo "🖥️ The workflow is now ready to test your code on the runner." - name: List files in the repository run: | ls ${{ github.workspace }} - run: echo "🍏 This job's status is ${{ job.status }}." ``` If all is well, you'll see a new run in `Actions` tab: <img width="1792" alt="image" src="https://user-images.githubusercontent.com/9418365/205884473-79a874bc-171b-4aaf-acd5-0241a45c3b53.png"> #### 5. Check the logs of jobs Click a run and you'll see the logs: <img width="1792" alt="image" src="https://user-images.githubusercontent.com/9418365/205884800-994b0374-67f7-48ff-be9a-4c53f3141547.png"> #### 6. Go on You can try more examples in [the documents](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions) of GitHub Actions, then you might find a lot of bugs. Come on, PRs are welcome. </details> See also: [Feature Preview: Gitea Actions](https://blog.gitea.io/2022/12/feature-preview-gitea-actions/) --------- Co-authored-by: a1012112796 <1012112796@qq.com> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com> Co-authored-by: delvh <dev.lh@web.de> Co-authored-by: ChristopherHX <christopher.homberger@web.de> Co-authored-by: John Olheiser <john.olheiser@gmail.com>
1 year ago
Rewrite logger system (#24726) ## ⚠️ Breaking The `log.<mode>.<logger>` style config has been dropped. If you used it, please check the new config manual & app.example.ini to make your instance output logs as expected. Although many legacy options still work, it's encouraged to upgrade to the new options. The SMTP logger is deleted because SMTP is not suitable to collect logs. If you have manually configured Gitea log options, please confirm the logger system works as expected after upgrading. ## Description Close #12082 and maybe more log-related issues, resolve some related FIXMEs in old code (which seems unfixable before) Just like rewriting queue #24505 : make code maintainable, clear legacy bugs, and add the ability to support more writers (eg: JSON, structured log) There is a new document (with examples): `logging-config.en-us.md` This PR is safer than the queue rewriting, because it's just for logging, it won't break other logic. ## The old problems The logging system is quite old and difficult to maintain: * Unclear concepts: Logger, NamedLogger, MultiChannelledLogger, SubLogger, EventLogger, WriterLogger etc * Some code is diffuclt to konw whether it is right: `log.DelNamedLogger("console")` vs `log.DelNamedLogger(log.DEFAULT)` vs `log.DelLogger("console")` * The old system heavily depends on ini config system, it's difficult to create new logger for different purpose, and it's very fragile. * The "color" trick is difficult to use and read, many colors are unnecessary, and in the future structured log could help * It's difficult to add other log formats, eg: JSON format * The log outputer doesn't have full control of its goroutine, it's difficult to make outputer have advanced behaviors * The logs could be lost in some cases: eg: no Fatal error when using CLI. * Config options are passed by JSON, which is quite fragile. * INI package makes the KEY in `[log]` section visible in `[log.sub1]` and `[log.sub1.subA]`, this behavior is quite fragile and would cause more unclear problems, and there is no strong requirement to support `log.<mode>.<logger>` syntax. ## The new design See `logger.go` for documents. ## Screenshot <details> ![image](https://github.com/go-gitea/gitea/assets/2114189/4462d713-ba39-41f5-bb08-de912e67e1ff) ![image](https://github.com/go-gitea/gitea/assets/2114189/b188035e-f691-428b-8b2d-ff7b2199b2f9) ![image](https://github.com/go-gitea/gitea/assets/2114189/132e9745-1c3b-4e00-9e0d-15eaea495dee) </details> ## TODO * [x] add some new tests * [x] fix some tests * [x] test some sub-commands (manually ....) --------- Co-authored-by: Jason Song <i@wolfogre.com> Co-authored-by: delvh <dev.lh@web.de> Co-authored-by: Giteabot <teabot@gitea.io>
1 year ago
Rewrite logger system (#24726) ## ⚠️ Breaking The `log.<mode>.<logger>` style config has been dropped. If you used it, please check the new config manual & app.example.ini to make your instance output logs as expected. Although many legacy options still work, it's encouraged to upgrade to the new options. The SMTP logger is deleted because SMTP is not suitable to collect logs. If you have manually configured Gitea log options, please confirm the logger system works as expected after upgrading. ## Description Close #12082 and maybe more log-related issues, resolve some related FIXMEs in old code (which seems unfixable before) Just like rewriting queue #24505 : make code maintainable, clear legacy bugs, and add the ability to support more writers (eg: JSON, structured log) There is a new document (with examples): `logging-config.en-us.md` This PR is safer than the queue rewriting, because it's just for logging, it won't break other logic. ## The old problems The logging system is quite old and difficult to maintain: * Unclear concepts: Logger, NamedLogger, MultiChannelledLogger, SubLogger, EventLogger, WriterLogger etc * Some code is diffuclt to konw whether it is right: `log.DelNamedLogger("console")` vs `log.DelNamedLogger(log.DEFAULT)` vs `log.DelLogger("console")` * The old system heavily depends on ini config system, it's difficult to create new logger for different purpose, and it's very fragile. * The "color" trick is difficult to use and read, many colors are unnecessary, and in the future structured log could help * It's difficult to add other log formats, eg: JSON format * The log outputer doesn't have full control of its goroutine, it's difficult to make outputer have advanced behaviors * The logs could be lost in some cases: eg: no Fatal error when using CLI. * Config options are passed by JSON, which is quite fragile. * INI package makes the KEY in `[log]` section visible in `[log.sub1]` and `[log.sub1.subA]`, this behavior is quite fragile and would cause more unclear problems, and there is no strong requirement to support `log.<mode>.<logger>` syntax. ## The new design See `logger.go` for documents. ## Screenshot <details> ![image](https://github.com/go-gitea/gitea/assets/2114189/4462d713-ba39-41f5-bb08-de912e67e1ff) ![image](https://github.com/go-gitea/gitea/assets/2114189/b188035e-f691-428b-8b2d-ff7b2199b2f9) ![image](https://github.com/go-gitea/gitea/assets/2114189/132e9745-1c3b-4e00-9e0d-15eaea495dee) </details> ## TODO * [x] add some new tests * [x] fix some tests * [x] test some sub-commands (manually ....) --------- Co-authored-by: Jason Song <i@wolfogre.com> Co-authored-by: delvh <dev.lh@web.de> Co-authored-by: Giteabot <teabot@gitea.io>
1 year ago
Rewrite logger system (#24726) ## ⚠️ Breaking The `log.<mode>.<logger>` style config has been dropped. If you used it, please check the new config manual & app.example.ini to make your instance output logs as expected. Although many legacy options still work, it's encouraged to upgrade to the new options. The SMTP logger is deleted because SMTP is not suitable to collect logs. If you have manually configured Gitea log options, please confirm the logger system works as expected after upgrading. ## Description Close #12082 and maybe more log-related issues, resolve some related FIXMEs in old code (which seems unfixable before) Just like rewriting queue #24505 : make code maintainable, clear legacy bugs, and add the ability to support more writers (eg: JSON, structured log) There is a new document (with examples): `logging-config.en-us.md` This PR is safer than the queue rewriting, because it's just for logging, it won't break other logic. ## The old problems The logging system is quite old and difficult to maintain: * Unclear concepts: Logger, NamedLogger, MultiChannelledLogger, SubLogger, EventLogger, WriterLogger etc * Some code is diffuclt to konw whether it is right: `log.DelNamedLogger("console")` vs `log.DelNamedLogger(log.DEFAULT)` vs `log.DelLogger("console")` * The old system heavily depends on ini config system, it's difficult to create new logger for different purpose, and it's very fragile. * The "color" trick is difficult to use and read, many colors are unnecessary, and in the future structured log could help * It's difficult to add other log formats, eg: JSON format * The log outputer doesn't have full control of its goroutine, it's difficult to make outputer have advanced behaviors * The logs could be lost in some cases: eg: no Fatal error when using CLI. * Config options are passed by JSON, which is quite fragile. * INI package makes the KEY in `[log]` section visible in `[log.sub1]` and `[log.sub1.subA]`, this behavior is quite fragile and would cause more unclear problems, and there is no strong requirement to support `log.<mode>.<logger>` syntax. ## The new design See `logger.go` for documents. ## Screenshot <details> ![image](https://github.com/go-gitea/gitea/assets/2114189/4462d713-ba39-41f5-bb08-de912e67e1ff) ![image](https://github.com/go-gitea/gitea/assets/2114189/b188035e-f691-428b-8b2d-ff7b2199b2f9) ![image](https://github.com/go-gitea/gitea/assets/2114189/132e9745-1c3b-4e00-9e0d-15eaea495dee) </details> ## TODO * [x] add some new tests * [x] fix some tests * [x] test some sub-commands (manually ....) --------- Co-authored-by: Jason Song <i@wolfogre.com> Co-authored-by: delvh <dev.lh@web.de> Co-authored-by: Giteabot <teabot@gitea.io>
1 year ago
Implement actions (#21937) Close #13539. Co-authored by: @lunny @appleboy @fuxiaohei and others. Related projects: - https://gitea.com/gitea/actions-proto-def - https://gitea.com/gitea/actions-proto-go - https://gitea.com/gitea/act - https://gitea.com/gitea/act_runner ### Summary The target of this PR is to bring a basic implementation of "Actions", an internal CI/CD system of Gitea. That means even though it has been merged, the state of the feature is **EXPERIMENTAL**, and please note that: - It is disabled by default; - It shouldn't be used in a production environment currently; - It shouldn't be used in a public Gitea instance currently; - Breaking changes may be made before it's stable. **Please comment on #13539 if you have any different product design ideas**, all decisions reached there will be adopted here. But in this PR, we don't talk about **naming, feature-creep or alternatives**. ### ⚠️ Breaking `gitea-actions` will become a reserved user name. If a user with the name already exists in the database, it is recommended to rename it. ### Some important reviews - What is `DEFAULT_ACTIONS_URL` in `app.ini` for? - https://github.com/go-gitea/gitea/pull/21937#discussion_r1055954954 - Why the api for runners is not under the normal `/api/v1` prefix? - https://github.com/go-gitea/gitea/pull/21937#discussion_r1061173592 - Why DBFS? - https://github.com/go-gitea/gitea/pull/21937#discussion_r1061301178 - Why ignore events triggered by `gitea-actions` bot? - https://github.com/go-gitea/gitea/pull/21937#discussion_r1063254103 - Why there's no permission control for actions? - https://github.com/go-gitea/gitea/pull/21937#discussion_r1090229868 ### What it looks like <details> #### Manage runners <img width="1792" alt="image" src="https://user-images.githubusercontent.com/9418365/205870657-c72f590e-2e08-4cd4-be7f-2e0abb299bbf.png"> #### List runs <img width="1792" alt="image" src="https://user-images.githubusercontent.com/9418365/205872794-50fde990-2b45-48c1-a178-908e4ec5b627.png"> #### View logs <img width="1792" alt="image" src="https://user-images.githubusercontent.com/9418365/205872501-9b7b9000-9542-4991-8f55-18ccdada77c3.png"> </details> ### How to try it <details> #### 1. Start Gitea Clone this branch and [install from source](https://docs.gitea.io/en-us/install-from-source). Add additional configurations in `app.ini` to enable Actions: ```ini [actions] ENABLED = true ``` Start it. If all is well, you'll see the management page of runners: <img width="1792" alt="image" src="https://user-images.githubusercontent.com/9418365/205877365-8e30a780-9b10-4154-b3e8-ee6c3cb35a59.png"> #### 2. Start runner Clone the [act_runner](https://gitea.com/gitea/act_runner), and follow the [README](https://gitea.com/gitea/act_runner/src/branch/main/README.md) to start it. If all is well, you'll see a new runner has been added: <img width="1792" alt="image" src="https://user-images.githubusercontent.com/9418365/205878000-216f5937-e696-470d-b66c-8473987d91c3.png"> #### 3. Enable actions for a repo Create a new repo or open an existing one, check the `Actions` checkbox in settings and submit. <img width="1792" alt="image" src="https://user-images.githubusercontent.com/9418365/205879705-53e09208-73c0-4b3e-a123-2dcf9aba4b9c.png"> <img width="1792" alt="image" src="https://user-images.githubusercontent.com/9418365/205879383-23f3d08f-1a85-41dd-a8b3-54e2ee6453e8.png"> If all is well, you'll see a new tab "Actions": <img width="1792" alt="image" src="https://user-images.githubusercontent.com/9418365/205881648-a8072d8c-5803-4d76-b8a8-9b2fb49516c1.png"> #### 4. Upload workflow files Upload some workflow files to `.gitea/workflows/xxx.yaml`, you can follow the [quickstart](https://docs.github.com/en/actions/quickstart) of GitHub Actions. Yes, Gitea Actions is compatible with GitHub Actions in most cases, you can use the same demo: ```yaml name: GitHub Actions Demo run-name: ${{ github.actor }} is testing out GitHub Actions 🚀 on: [push] jobs: Explore-GitHub-Actions: runs-on: ubuntu-latest steps: - run: echo "🎉 The job was automatically triggered by a ${{ github.event_name }} event." - run: echo "🐧 This job is now running on a ${{ runner.os }} server hosted by GitHub!" - run: echo "🔎 The name of your branch is ${{ github.ref }} and your repository is ${{ github.repository }}." - name: Check out repository code uses: actions/checkout@v3 - run: echo "💡 The ${{ github.repository }} repository has been cloned to the runner." - run: echo "🖥️ The workflow is now ready to test your code on the runner." - name: List files in the repository run: | ls ${{ github.workspace }} - run: echo "🍏 This job's status is ${{ job.status }}." ``` If all is well, you'll see a new run in `Actions` tab: <img width="1792" alt="image" src="https://user-images.githubusercontent.com/9418365/205884473-79a874bc-171b-4aaf-acd5-0241a45c3b53.png"> #### 5. Check the logs of jobs Click a run and you'll see the logs: <img width="1792" alt="image" src="https://user-images.githubusercontent.com/9418365/205884800-994b0374-67f7-48ff-be9a-4c53f3141547.png"> #### 6. Go on You can try more examples in [the documents](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions) of GitHub Actions, then you might find a lot of bugs. Come on, PRs are welcome. </details> See also: [Feature Preview: Gitea Actions](https://blog.gitea.io/2022/12/feature-preview-gitea-actions/) --------- Co-authored-by: a1012112796 <1012112796@qq.com> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com> Co-authored-by: delvh <dev.lh@web.de> Co-authored-by: ChristopherHX <christopher.homberger@web.de> Co-authored-by: John Olheiser <john.olheiser@gmail.com>
1 year ago
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571572573574575576577578579580581582583584585586587588589590591592593594595596597598599600601602603604605606607608609610611612613614615616617618619620621622623624625626627628629630631632633634635636637638639640641642643644645646647648649650651652653654655656657658659660661662663664665666667668669670671672673674675676677678679680681682683684685686687688689690691692693694695696697698699700701702703704705706707708709710711712713714715716717718719720721722723724725726727728729730731732733734735736737738739740741742743744745746747748749750751752753754755756757758759760761762763764765766767768769770771772773774775776777778779780781782783784785786787788789790791792793794795796797798799800801802803804805806807808809810811812813814815816817818819820821822823824825826827828829830831832833834835836837838839840841842843844845846847848849850851852853854855856857858859860861862863864865866867868869870871872873874875876877878879880881882883884885886887888889890891892893894895896897898899900
  1. // Copyright 2021 The Gitea Authors. All rights reserved.
  2. // SPDX-License-Identifier: MIT
  3. package repo
  4. import (
  5. "context"
  6. "fmt"
  7. "html/template"
  8. "net"
  9. "net/url"
  10. "path/filepath"
  11. "strconv"
  12. "strings"
  13. "code.gitea.io/gitea/models/db"
  14. "code.gitea.io/gitea/models/unit"
  15. user_model "code.gitea.io/gitea/models/user"
  16. "code.gitea.io/gitea/modules/base"
  17. "code.gitea.io/gitea/modules/log"
  18. "code.gitea.io/gitea/modules/markup"
  19. "code.gitea.io/gitea/modules/setting"
  20. api "code.gitea.io/gitea/modules/structs"
  21. "code.gitea.io/gitea/modules/timeutil"
  22. "code.gitea.io/gitea/modules/util"
  23. "xorm.io/builder"
  24. )
  25. // ErrUserDoesNotHaveAccessToRepo represents an error where the user doesn't has access to a given repo.
  26. type ErrUserDoesNotHaveAccessToRepo struct {
  27. UserID int64
  28. RepoName string
  29. }
  30. // IsErrUserDoesNotHaveAccessToRepo checks if an error is a ErrRepoFileAlreadyExists.
  31. func IsErrUserDoesNotHaveAccessToRepo(err error) bool {
  32. _, ok := err.(ErrUserDoesNotHaveAccessToRepo)
  33. return ok
  34. }
  35. func (err ErrUserDoesNotHaveAccessToRepo) Error() string {
  36. return fmt.Sprintf("user doesn't have access to repo [user_id: %d, repo_name: %s]", err.UserID, err.RepoName)
  37. }
  38. func (err ErrUserDoesNotHaveAccessToRepo) Unwrap() error {
  39. return util.ErrPermissionDenied
  40. }
  41. type ErrRepoIsArchived struct {
  42. Repo *Repository
  43. }
  44. func (err ErrRepoIsArchived) Error() string {
  45. return fmt.Sprintf("%s is archived", err.Repo.LogString())
  46. }
  47. var (
  48. reservedRepoNames = []string{".", "..", "-"}
  49. reservedRepoPatterns = []string{"*.git", "*.wiki", "*.rss", "*.atom"}
  50. )
  51. // IsUsableRepoName returns true when repository is usable
  52. func IsUsableRepoName(name string) error {
  53. if db.AlphaDashDotPattern.MatchString(name) {
  54. // Note: usually this error is normally caught up earlier in the UI
  55. return db.ErrNameCharsNotAllowed{Name: name}
  56. }
  57. return db.IsUsableName(reservedRepoNames, reservedRepoPatterns, name)
  58. }
  59. // TrustModelType defines the types of trust model for this repository
  60. type TrustModelType int
  61. // kinds of TrustModel
  62. const (
  63. DefaultTrustModel TrustModelType = iota // default trust model
  64. CommitterTrustModel
  65. CollaboratorTrustModel
  66. CollaboratorCommitterTrustModel
  67. )
  68. // String converts a TrustModelType to a string
  69. func (t TrustModelType) String() string {
  70. switch t {
  71. case DefaultTrustModel:
  72. return "default"
  73. case CommitterTrustModel:
  74. return "committer"
  75. case CollaboratorTrustModel:
  76. return "collaborator"
  77. case CollaboratorCommitterTrustModel:
  78. return "collaboratorcommitter"
  79. }
  80. return "default"
  81. }
  82. // ToTrustModel converts a string to a TrustModelType
  83. func ToTrustModel(model string) TrustModelType {
  84. switch strings.ToLower(strings.TrimSpace(model)) {
  85. case "default":
  86. return DefaultTrustModel
  87. case "collaborator":
  88. return CollaboratorTrustModel
  89. case "committer":
  90. return CommitterTrustModel
  91. case "collaboratorcommitter":
  92. return CollaboratorCommitterTrustModel
  93. }
  94. return DefaultTrustModel
  95. }
  96. // RepositoryStatus defines the status of repository
  97. type RepositoryStatus int
  98. // all kinds of RepositoryStatus
  99. const (
  100. RepositoryReady RepositoryStatus = iota // a normal repository
  101. RepositoryBeingMigrated // repository is migrating
  102. RepositoryPendingTransfer // repository pending in ownership transfer state
  103. RepositoryBroken // repository is in a permanently broken state
  104. )
  105. // Repository represents a git repository.
  106. type Repository struct {
  107. ID int64 `xorm:"pk autoincr"`
  108. OwnerID int64 `xorm:"UNIQUE(s) index"`
  109. OwnerName string
  110. Owner *user_model.User `xorm:"-"`
  111. LowerName string `xorm:"UNIQUE(s) INDEX NOT NULL"`
  112. Name string `xorm:"INDEX NOT NULL"`
  113. Description string `xorm:"TEXT"`
  114. Website string `xorm:"VARCHAR(2048)"`
  115. OriginalServiceType api.GitServiceType `xorm:"index"`
  116. OriginalURL string `xorm:"VARCHAR(2048)"`
  117. DefaultBranch string
  118. NumWatches int
  119. NumStars int
  120. NumForks int
  121. NumIssues int
  122. NumClosedIssues int
  123. NumOpenIssues int `xorm:"-"`
  124. NumPulls int
  125. NumClosedPulls int
  126. NumOpenPulls int `xorm:"-"`
  127. NumMilestones int `xorm:"NOT NULL DEFAULT 0"`
  128. NumClosedMilestones int `xorm:"NOT NULL DEFAULT 0"`
  129. NumOpenMilestones int `xorm:"-"`
  130. NumProjects int `xorm:"NOT NULL DEFAULT 0"`
  131. NumClosedProjects int `xorm:"NOT NULL DEFAULT 0"`
  132. NumOpenProjects int `xorm:"-"`
  133. NumActionRuns int `xorm:"NOT NULL DEFAULT 0"`
  134. NumClosedActionRuns int `xorm:"NOT NULL DEFAULT 0"`
  135. NumOpenActionRuns int `xorm:"-"`
  136. IsPrivate bool `xorm:"INDEX"`
  137. IsEmpty bool `xorm:"INDEX"`
  138. IsArchived bool `xorm:"INDEX"`
  139. IsMirror bool `xorm:"INDEX"`
  140. Status RepositoryStatus `xorm:"NOT NULL DEFAULT 0"`
  141. RenderingMetas map[string]string `xorm:"-"`
  142. DocumentRenderingMetas map[string]string `xorm:"-"`
  143. Units []*RepoUnit `xorm:"-"`
  144. PrimaryLanguage *LanguageStat `xorm:"-"`
  145. IsFork bool `xorm:"INDEX NOT NULL DEFAULT false"`
  146. ForkID int64 `xorm:"INDEX"`
  147. BaseRepo *Repository `xorm:"-"`
  148. IsTemplate bool `xorm:"INDEX NOT NULL DEFAULT false"`
  149. TemplateID int64 `xorm:"INDEX"`
  150. Size int64 `xorm:"NOT NULL DEFAULT 0"`
  151. GitSize int64 `xorm:"NOT NULL DEFAULT 0"`
  152. LFSSize int64 `xorm:"NOT NULL DEFAULT 0"`
  153. CodeIndexerStatus *RepoIndexerStatus `xorm:"-"`
  154. StatsIndexerStatus *RepoIndexerStatus `xorm:"-"`
  155. IsFsckEnabled bool `xorm:"NOT NULL DEFAULT true"`
  156. CloseIssuesViaCommitInAnyBranch bool `xorm:"NOT NULL DEFAULT false"`
  157. Topics []string `xorm:"TEXT JSON"`
  158. TrustModel TrustModelType
  159. // Avatar: ID(10-20)-md5(32) - must fit into 64 symbols
  160. Avatar string `xorm:"VARCHAR(64)"`
  161. CreatedUnix timeutil.TimeStamp `xorm:"INDEX created"`
  162. UpdatedUnix timeutil.TimeStamp `xorm:"INDEX updated"`
  163. ArchivedUnix timeutil.TimeStamp `xorm:"DEFAULT 0"`
  164. }
  165. func init() {
  166. db.RegisterModel(new(Repository))
  167. }
  168. // SanitizedOriginalURL returns a sanitized OriginalURL
  169. func (repo *Repository) SanitizedOriginalURL() string {
  170. if repo.OriginalURL == "" {
  171. return ""
  172. }
  173. u, _ := util.SanitizeURL(repo.OriginalURL)
  174. return u
  175. }
  176. // text representations to be returned in SizeDetail.Name
  177. const (
  178. SizeDetailNameGit = "git"
  179. SizeDetailNameLFS = "lfs"
  180. )
  181. type SizeDetail struct {
  182. Name string
  183. Size int64
  184. }
  185. // SizeDetails forms a struct with various size details about repository
  186. func (repo *Repository) SizeDetails() []SizeDetail {
  187. sizeDetails := []SizeDetail{
  188. {
  189. Name: SizeDetailNameGit,
  190. Size: repo.GitSize,
  191. },
  192. {
  193. Name: SizeDetailNameLFS,
  194. Size: repo.LFSSize,
  195. },
  196. }
  197. return sizeDetails
  198. }
  199. // SizeDetailsString returns a concatenation of all repository size details as a string
  200. func (repo *Repository) SizeDetailsString() string {
  201. var str strings.Builder
  202. sizeDetails := repo.SizeDetails()
  203. for _, detail := range sizeDetails {
  204. str.WriteString(fmt.Sprintf("%s: %s, ", detail.Name, base.FileSize(detail.Size)))
  205. }
  206. return strings.TrimSuffix(str.String(), ", ")
  207. }
  208. func (repo *Repository) LogString() string {
  209. if repo == nil {
  210. return "<Repository nil>"
  211. }
  212. return fmt.Sprintf("<Repository %d:%s/%s>", repo.ID, repo.OwnerName, repo.Name)
  213. }
  214. // IsBeingMigrated indicates that repository is being migrated
  215. func (repo *Repository) IsBeingMigrated() bool {
  216. return repo.Status == RepositoryBeingMigrated
  217. }
  218. // IsBeingCreated indicates that repository is being migrated or forked
  219. func (repo *Repository) IsBeingCreated() bool {
  220. return repo.IsBeingMigrated()
  221. }
  222. // IsBroken indicates that repository is broken
  223. func (repo *Repository) IsBroken() bool {
  224. return repo.Status == RepositoryBroken
  225. }
  226. // MarkAsBrokenEmpty marks the repo as broken and empty
  227. func (repo *Repository) MarkAsBrokenEmpty() {
  228. repo.Status = RepositoryBroken
  229. repo.IsEmpty = true
  230. }
  231. // AfterLoad is invoked from XORM after setting the values of all fields of this object.
  232. func (repo *Repository) AfterLoad() {
  233. repo.NumOpenIssues = repo.NumIssues - repo.NumClosedIssues
  234. repo.NumOpenPulls = repo.NumPulls - repo.NumClosedPulls
  235. repo.NumOpenMilestones = repo.NumMilestones - repo.NumClosedMilestones
  236. repo.NumOpenProjects = repo.NumProjects - repo.NumClosedProjects
  237. repo.NumOpenActionRuns = repo.NumActionRuns - repo.NumClosedActionRuns
  238. }
  239. // LoadAttributes loads attributes of the repository.
  240. func (repo *Repository) LoadAttributes(ctx context.Context) error {
  241. // Load owner
  242. if err := repo.LoadOwner(ctx); err != nil {
  243. return fmt.Errorf("load owner: %w", err)
  244. }
  245. // Load primary language
  246. stats := make(LanguageStatList, 0, 1)
  247. if err := db.GetEngine(ctx).
  248. Where("`repo_id` = ? AND `is_primary` = ? AND `language` != ?", repo.ID, true, "other").
  249. Find(&stats); err != nil {
  250. return fmt.Errorf("find primary languages: %w", err)
  251. }
  252. stats.LoadAttributes()
  253. for _, st := range stats {
  254. if st.RepoID == repo.ID {
  255. repo.PrimaryLanguage = st
  256. break
  257. }
  258. }
  259. return nil
  260. }
  261. // FullName returns the repository full name
  262. func (repo *Repository) FullName() string {
  263. return repo.OwnerName + "/" + repo.Name
  264. }
  265. // HTMLURL returns the repository HTML URL
  266. func (repo *Repository) HTMLURL() string {
  267. return setting.AppURL + url.PathEscape(repo.OwnerName) + "/" + url.PathEscape(repo.Name)
  268. }
  269. // CommitLink make link to by commit full ID
  270. // note: won't check whether it's an right id
  271. func (repo *Repository) CommitLink(commitID string) (result string) {
  272. if commitID == "" || commitID == "0000000000000000000000000000000000000000" {
  273. result = ""
  274. } else {
  275. result = repo.Link() + "/commit/" + url.PathEscape(commitID)
  276. }
  277. return result
  278. }
  279. // APIURL returns the repository API URL
  280. func (repo *Repository) APIURL() string {
  281. return setting.AppURL + "api/v1/repos/" + url.PathEscape(repo.OwnerName) + "/" + url.PathEscape(repo.Name)
  282. }
  283. // GetCommitsCountCacheKey returns cache key used for commits count caching.
  284. func (repo *Repository) GetCommitsCountCacheKey(contextName string, isRef bool) string {
  285. var prefix string
  286. if isRef {
  287. prefix = "ref"
  288. } else {
  289. prefix = "commit"
  290. }
  291. return fmt.Sprintf("commits-count-%d-%s-%s", repo.ID, prefix, contextName)
  292. }
  293. // LoadUnits loads repo units into repo.Units
  294. func (repo *Repository) LoadUnits(ctx context.Context) (err error) {
  295. if repo.Units != nil {
  296. return nil
  297. }
  298. repo.Units, err = getUnitsByRepoID(ctx, repo.ID)
  299. if log.IsTrace() {
  300. unitTypeStrings := make([]string, len(repo.Units))
  301. for i, unit := range repo.Units {
  302. unitTypeStrings[i] = unit.Type.String()
  303. }
  304. log.Trace("repo.Units, ID=%d, Types: [%s]", repo.ID, strings.Join(unitTypeStrings, ", "))
  305. }
  306. return err
  307. }
  308. // UnitEnabled if this repository has the given unit enabled
  309. func (repo *Repository) UnitEnabled(ctx context.Context, tp unit.Type) bool {
  310. if err := repo.LoadUnits(ctx); err != nil {
  311. log.Warn("Error loading repository (ID: %d) units: %s", repo.ID, err.Error())
  312. }
  313. for _, unit := range repo.Units {
  314. if unit.Type == tp {
  315. return true
  316. }
  317. }
  318. return false
  319. }
  320. // MustGetUnit always returns a RepoUnit object
  321. func (repo *Repository) MustGetUnit(ctx context.Context, tp unit.Type) *RepoUnit {
  322. ru, err := repo.GetUnit(ctx, tp)
  323. if err == nil {
  324. return ru
  325. }
  326. if tp == unit.TypeExternalWiki {
  327. return &RepoUnit{
  328. Type: tp,
  329. Config: new(ExternalWikiConfig),
  330. }
  331. } else if tp == unit.TypeExternalTracker {
  332. return &RepoUnit{
  333. Type: tp,
  334. Config: new(ExternalTrackerConfig),
  335. }
  336. } else if tp == unit.TypePullRequests {
  337. return &RepoUnit{
  338. Type: tp,
  339. Config: new(PullRequestsConfig),
  340. }
  341. } else if tp == unit.TypeIssues {
  342. return &RepoUnit{
  343. Type: tp,
  344. Config: new(IssuesConfig),
  345. }
  346. } else if tp == unit.TypeActions {
  347. return &RepoUnit{
  348. Type: tp,
  349. Config: new(ActionsConfig),
  350. }
  351. }
  352. return &RepoUnit{
  353. Type: tp,
  354. Config: new(UnitConfig),
  355. }
  356. }
  357. // GetUnit returns a RepoUnit object
  358. func (repo *Repository) GetUnit(ctx context.Context, tp unit.Type) (*RepoUnit, error) {
  359. if err := repo.LoadUnits(ctx); err != nil {
  360. return nil, err
  361. }
  362. for _, unit := range repo.Units {
  363. if unit.Type == tp {
  364. return unit, nil
  365. }
  366. }
  367. return nil, ErrUnitTypeNotExist{tp}
  368. }
  369. // LoadOwner loads owner user
  370. func (repo *Repository) LoadOwner(ctx context.Context) (err error) {
  371. if repo.Owner != nil {
  372. return nil
  373. }
  374. repo.Owner, err = user_model.GetUserByID(ctx, repo.OwnerID)
  375. return err
  376. }
  377. // MustOwner always returns a valid *user_model.User object to avoid
  378. // conceptually impossible error handling.
  379. // It creates a fake object that contains error details
  380. // when error occurs.
  381. func (repo *Repository) MustOwner(ctx context.Context) *user_model.User {
  382. if err := repo.LoadOwner(ctx); err != nil {
  383. return &user_model.User{
  384. Name: "error",
  385. FullName: err.Error(),
  386. }
  387. }
  388. return repo.Owner
  389. }
  390. // ComposeMetas composes a map of metas for properly rendering issue links and external issue trackers.
  391. func (repo *Repository) ComposeMetas() map[string]string {
  392. if len(repo.RenderingMetas) == 0 {
  393. metas := map[string]string{
  394. "user": repo.OwnerName,
  395. "repo": repo.Name,
  396. "repoPath": repo.RepoPath(),
  397. "mode": "comment",
  398. }
  399. unit, err := repo.GetUnit(db.DefaultContext, unit.TypeExternalTracker)
  400. if err == nil {
  401. metas["format"] = unit.ExternalTrackerConfig().ExternalTrackerFormat
  402. switch unit.ExternalTrackerConfig().ExternalTrackerStyle {
  403. case markup.IssueNameStyleAlphanumeric:
  404. metas["style"] = markup.IssueNameStyleAlphanumeric
  405. case markup.IssueNameStyleRegexp:
  406. metas["style"] = markup.IssueNameStyleRegexp
  407. metas["regexp"] = unit.ExternalTrackerConfig().ExternalTrackerRegexpPattern
  408. default:
  409. metas["style"] = markup.IssueNameStyleNumeric
  410. }
  411. }
  412. repo.MustOwner(db.DefaultContext)
  413. if repo.Owner.IsOrganization() {
  414. teams := make([]string, 0, 5)
  415. _ = db.GetEngine(db.DefaultContext).Table("team_repo").
  416. Join("INNER", "team", "team.id = team_repo.team_id").
  417. Where("team_repo.repo_id = ?", repo.ID).
  418. Select("team.lower_name").
  419. OrderBy("team.lower_name").
  420. Find(&teams)
  421. metas["teams"] = "," + strings.Join(teams, ",") + ","
  422. metas["org"] = strings.ToLower(repo.OwnerName)
  423. }
  424. repo.RenderingMetas = metas
  425. }
  426. return repo.RenderingMetas
  427. }
  428. // ComposeDocumentMetas composes a map of metas for properly rendering documents
  429. func (repo *Repository) ComposeDocumentMetas() map[string]string {
  430. if len(repo.DocumentRenderingMetas) == 0 {
  431. metas := map[string]string{}
  432. for k, v := range repo.ComposeMetas() {
  433. metas[k] = v
  434. }
  435. metas["mode"] = "document"
  436. repo.DocumentRenderingMetas = metas
  437. }
  438. return repo.DocumentRenderingMetas
  439. }
  440. // GetBaseRepo populates repo.BaseRepo for a fork repository and
  441. // returns an error on failure (NOTE: no error is returned for
  442. // non-fork repositories, and BaseRepo will be left untouched)
  443. func (repo *Repository) GetBaseRepo(ctx context.Context) (err error) {
  444. if !repo.IsFork {
  445. return nil
  446. }
  447. repo.BaseRepo, err = GetRepositoryByID(ctx, repo.ForkID)
  448. return err
  449. }
  450. // IsGenerated returns whether _this_ repository was generated from a template
  451. func (repo *Repository) IsGenerated() bool {
  452. return repo.TemplateID != 0
  453. }
  454. // RepoPath returns repository path by given user and repository name.
  455. func RepoPath(userName, repoName string) string { //revive:disable-line:exported
  456. return filepath.Join(user_model.UserPath(userName), strings.ToLower(repoName)+".git")
  457. }
  458. // RepoPath returns the repository path
  459. func (repo *Repository) RepoPath() string {
  460. return RepoPath(repo.OwnerName, repo.Name)
  461. }
  462. // Link returns the repository relative url
  463. func (repo *Repository) Link() string {
  464. return setting.AppSubURL + "/" + url.PathEscape(repo.OwnerName) + "/" + url.PathEscape(repo.Name)
  465. }
  466. // ComposeCompareURL returns the repository comparison URL
  467. func (repo *Repository) ComposeCompareURL(oldCommitID, newCommitID string) string {
  468. return fmt.Sprintf("%s/%s/compare/%s...%s", url.PathEscape(repo.OwnerName), url.PathEscape(repo.Name), util.PathEscapeSegments(oldCommitID), util.PathEscapeSegments(newCommitID))
  469. }
  470. func (repo *Repository) ComposeBranchCompareURL(baseRepo *Repository, branchName string) string {
  471. if baseRepo == nil {
  472. baseRepo = repo
  473. }
  474. var cmpBranchEscaped string
  475. if repo.ID != baseRepo.ID {
  476. cmpBranchEscaped = fmt.Sprintf("%s/%s:", url.PathEscape(repo.OwnerName), url.PathEscape(repo.Name))
  477. }
  478. cmpBranchEscaped = fmt.Sprintf("%s%s", cmpBranchEscaped, util.PathEscapeSegments(branchName))
  479. return fmt.Sprintf("%s/compare/%s...%s", baseRepo.Link(), util.PathEscapeSegments(baseRepo.DefaultBranch), cmpBranchEscaped)
  480. }
  481. // IsOwnedBy returns true when user owns this repository
  482. func (repo *Repository) IsOwnedBy(userID int64) bool {
  483. return repo.OwnerID == userID
  484. }
  485. // CanCreateBranch returns true if repository meets the requirements for creating new branches.
  486. func (repo *Repository) CanCreateBranch() bool {
  487. return !repo.IsMirror
  488. }
  489. // CanEnablePulls returns true if repository meets the requirements of accepting pulls.
  490. func (repo *Repository) CanEnablePulls() bool {
  491. return !repo.IsMirror && !repo.IsEmpty
  492. }
  493. // AllowsPulls returns true if repository meets the requirements of accepting pulls and has them enabled.
  494. func (repo *Repository) AllowsPulls() bool {
  495. return repo.CanEnablePulls() && repo.UnitEnabled(db.DefaultContext, unit.TypePullRequests)
  496. }
  497. // CanEnableEditor returns true if repository meets the requirements of web editor.
  498. func (repo *Repository) CanEnableEditor() bool {
  499. return !repo.IsMirror
  500. }
  501. // DescriptionHTML does special handles to description and return HTML string.
  502. func (repo *Repository) DescriptionHTML(ctx context.Context) template.HTML {
  503. desc, err := markup.RenderDescriptionHTML(&markup.RenderContext{
  504. Ctx: ctx,
  505. // Don't use Metas to speedup requests
  506. }, repo.Description)
  507. if err != nil {
  508. log.Error("Failed to render description for %s (ID: %d): %v", repo.Name, repo.ID, err)
  509. return template.HTML(markup.Sanitize(repo.Description))
  510. }
  511. return template.HTML(markup.Sanitize(desc))
  512. }
  513. // CloneLink represents different types of clone URLs of repository.
  514. type CloneLink struct {
  515. SSH string
  516. HTTPS string
  517. }
  518. // ComposeHTTPSCloneURL returns HTTPS clone URL based on given owner and repository name.
  519. func ComposeHTTPSCloneURL(owner, repo string) string {
  520. return fmt.Sprintf("%s%s/%s.git", setting.AppURL, url.PathEscape(owner), url.PathEscape(repo))
  521. }
  522. func ComposeSSHCloneURL(ownerName, repoName string) string {
  523. sshUser := setting.SSH.User
  524. sshDomain := setting.SSH.Domain
  525. // non-standard port, it must use full URI
  526. if setting.SSH.Port != 22 {
  527. sshHost := net.JoinHostPort(sshDomain, strconv.Itoa(setting.SSH.Port))
  528. return fmt.Sprintf("ssh://%s@%s/%s/%s.git", sshUser, sshHost, url.PathEscape(ownerName), url.PathEscape(repoName))
  529. }
  530. // for standard port, it can use a shorter URI (without the port)
  531. sshHost := sshDomain
  532. if ip := net.ParseIP(sshHost); ip != nil && ip.To4() == nil {
  533. sshHost = "[" + sshHost + "]" // for IPv6 address, wrap it with brackets
  534. }
  535. if setting.Repository.UseCompatSSHURI {
  536. return fmt.Sprintf("ssh://%s@%s/%s/%s.git", sshUser, sshHost, url.PathEscape(ownerName), url.PathEscape(repoName))
  537. }
  538. return fmt.Sprintf("%s@%s:%s/%s.git", sshUser, sshHost, url.PathEscape(ownerName), url.PathEscape(repoName))
  539. }
  540. func (repo *Repository) cloneLink(isWiki bool) *CloneLink {
  541. repoName := repo.Name
  542. if isWiki {
  543. repoName += ".wiki"
  544. }
  545. cl := new(CloneLink)
  546. cl.SSH = ComposeSSHCloneURL(repo.OwnerName, repoName)
  547. cl.HTTPS = ComposeHTTPSCloneURL(repo.OwnerName, repoName)
  548. return cl
  549. }
  550. // CloneLink returns clone URLs of repository.
  551. func (repo *Repository) CloneLink() (cl *CloneLink) {
  552. return repo.cloneLink(false)
  553. }
  554. // GetOriginalURLHostname returns the hostname of a URL or the URL
  555. func (repo *Repository) GetOriginalURLHostname() string {
  556. u, err := url.Parse(repo.OriginalURL)
  557. if err != nil {
  558. return repo.OriginalURL
  559. }
  560. return u.Host
  561. }
  562. // GetTrustModel will get the TrustModel for the repo or the default trust model
  563. func (repo *Repository) GetTrustModel() TrustModelType {
  564. trustModel := repo.TrustModel
  565. if trustModel == DefaultTrustModel {
  566. trustModel = ToTrustModel(setting.Repository.Signing.DefaultTrustModel)
  567. if trustModel == DefaultTrustModel {
  568. return CollaboratorTrustModel
  569. }
  570. }
  571. return trustModel
  572. }
  573. // MustNotBeArchived returns ErrRepoIsArchived if the repo is archived
  574. func (repo *Repository) MustNotBeArchived() error {
  575. if repo.IsArchived {
  576. return ErrRepoIsArchived{Repo: repo}
  577. }
  578. return nil
  579. }
  580. // __________ .__ __
  581. // \______ \ ____ ______ ____ _____|__|/ |_ ___________ ___.__.
  582. // | _// __ \\____ \ / _ \/ ___/ \ __\/ _ \_ __ < | |
  583. // | | \ ___/| |_> > <_> )___ \| || | ( <_> ) | \/\___ |
  584. // |____|_ /\___ > __/ \____/____ >__||__| \____/|__| / ____|
  585. // \/ \/|__| \/ \/
  586. // ErrRepoNotExist represents a "RepoNotExist" kind of error.
  587. type ErrRepoNotExist struct {
  588. ID int64
  589. UID int64
  590. OwnerName string
  591. Name string
  592. }
  593. // IsErrRepoNotExist checks if an error is a ErrRepoNotExist.
  594. func IsErrRepoNotExist(err error) bool {
  595. _, ok := err.(ErrRepoNotExist)
  596. return ok
  597. }
  598. func (err ErrRepoNotExist) Error() string {
  599. return fmt.Sprintf("repository does not exist [id: %d, uid: %d, owner_name: %s, name: %s]",
  600. err.ID, err.UID, err.OwnerName, err.Name)
  601. }
  602. // Unwrap unwraps this error as a ErrNotExist error
  603. func (err ErrRepoNotExist) Unwrap() error {
  604. return util.ErrNotExist
  605. }
  606. // GetRepositoryByOwnerAndName returns the repository by given owner name and repo name
  607. func GetRepositoryByOwnerAndName(ctx context.Context, ownerName, repoName string) (*Repository, error) {
  608. var repo Repository
  609. has, err := db.GetEngine(ctx).Table("repository").Select("repository.*").
  610. Join("INNER", "`user`", "`user`.id = repository.owner_id").
  611. Where("repository.lower_name = ?", strings.ToLower(repoName)).
  612. And("`user`.lower_name = ?", strings.ToLower(ownerName)).
  613. Get(&repo)
  614. if err != nil {
  615. return nil, err
  616. } else if !has {
  617. return nil, ErrRepoNotExist{0, 0, ownerName, repoName}
  618. }
  619. return &repo, nil
  620. }
  621. // GetRepositoryByName returns the repository by given name under user if exists.
  622. func GetRepositoryByName(ownerID int64, name string) (*Repository, error) {
  623. repo := &Repository{
  624. OwnerID: ownerID,
  625. LowerName: strings.ToLower(name),
  626. }
  627. has, err := db.GetEngine(db.DefaultContext).Get(repo)
  628. if err != nil {
  629. return nil, err
  630. } else if !has {
  631. return nil, ErrRepoNotExist{0, ownerID, "", name}
  632. }
  633. return repo, err
  634. }
  635. // getRepositoryURLPathSegments returns segments (owner, reponame) extracted from a url
  636. func getRepositoryURLPathSegments(repoURL string) []string {
  637. if strings.HasPrefix(repoURL, setting.AppURL) {
  638. return strings.Split(strings.TrimPrefix(repoURL, setting.AppURL), "/")
  639. }
  640. sshURLVariants := [4]string{
  641. setting.SSH.Domain + ":",
  642. setting.SSH.User + "@" + setting.SSH.Domain + ":",
  643. "git+ssh://" + setting.SSH.Domain + "/",
  644. "git+ssh://" + setting.SSH.User + "@" + setting.SSH.Domain + "/",
  645. }
  646. for _, sshURL := range sshURLVariants {
  647. if strings.HasPrefix(repoURL, sshURL) {
  648. return strings.Split(strings.TrimPrefix(repoURL, sshURL), "/")
  649. }
  650. }
  651. return nil
  652. }
  653. // GetRepositoryByURL returns the repository by given url
  654. func GetRepositoryByURL(ctx context.Context, repoURL string) (*Repository, error) {
  655. // possible urls for git:
  656. // https://my.domain/sub-path/<owner>/<repo>.git
  657. // https://my.domain/sub-path/<owner>/<repo>
  658. // git+ssh://user@my.domain/<owner>/<repo>.git
  659. // git+ssh://user@my.domain/<owner>/<repo>
  660. // user@my.domain:<owner>/<repo>.git
  661. // user@my.domain:<owner>/<repo>
  662. pathSegments := getRepositoryURLPathSegments(repoURL)
  663. if len(pathSegments) != 2 {
  664. return nil, fmt.Errorf("unknown or malformed repository URL")
  665. }
  666. ownerName := pathSegments[0]
  667. repoName := strings.TrimSuffix(pathSegments[1], ".git")
  668. return GetRepositoryByOwnerAndName(ctx, ownerName, repoName)
  669. }
  670. // GetRepositoryByID returns the repository by given id if exists.
  671. func GetRepositoryByID(ctx context.Context, id int64) (*Repository, error) {
  672. repo := new(Repository)
  673. has, err := db.GetEngine(ctx).ID(id).Get(repo)
  674. if err != nil {
  675. return nil, err
  676. } else if !has {
  677. return nil, ErrRepoNotExist{id, 0, "", ""}
  678. }
  679. return repo, nil
  680. }
  681. // GetRepositoriesMapByIDs returns the repositories by given id slice.
  682. func GetRepositoriesMapByIDs(ids []int64) (map[int64]*Repository, error) {
  683. repos := make(map[int64]*Repository, len(ids))
  684. return repos, db.GetEngine(db.DefaultContext).In("id", ids).Find(&repos)
  685. }
  686. // IsRepositoryModelOrDirExist returns true if the repository with given name under user has already existed.
  687. func IsRepositoryModelOrDirExist(ctx context.Context, u *user_model.User, repoName string) (bool, error) {
  688. has, err := IsRepositoryModelExist(ctx, u, repoName)
  689. if err != nil {
  690. return false, err
  691. }
  692. isDir, err := util.IsDir(RepoPath(u.Name, repoName))
  693. return has || isDir, err
  694. }
  695. func IsRepositoryModelExist(ctx context.Context, u *user_model.User, repoName string) (bool, error) {
  696. return db.GetEngine(ctx).Get(&Repository{
  697. OwnerID: u.ID,
  698. LowerName: strings.ToLower(repoName),
  699. })
  700. }
  701. // GetTemplateRepo populates repo.TemplateRepo for a generated repository and
  702. // returns an error on failure (NOTE: no error is returned for
  703. // non-generated repositories, and TemplateRepo will be left untouched)
  704. func GetTemplateRepo(ctx context.Context, repo *Repository) (*Repository, error) {
  705. if !repo.IsGenerated() {
  706. return nil, nil
  707. }
  708. return GetRepositoryByID(ctx, repo.TemplateID)
  709. }
  710. // TemplateRepo returns the repository, which is template of this repository
  711. func (repo *Repository) TemplateRepo() *Repository {
  712. repo, err := GetTemplateRepo(db.DefaultContext, repo)
  713. if err != nil {
  714. log.Error("TemplateRepo: %v", err)
  715. return nil
  716. }
  717. return repo
  718. }
  719. type CountRepositoryOptions struct {
  720. OwnerID int64
  721. Private util.OptionalBool
  722. }
  723. // CountRepositories returns number of repositories.
  724. // Argument private only takes effect when it is false,
  725. // set it true to count all repositories.
  726. func CountRepositories(ctx context.Context, opts CountRepositoryOptions) (int64, error) {
  727. sess := db.GetEngine(ctx).Where("id > 0")
  728. if opts.OwnerID > 0 {
  729. sess.And("owner_id = ?", opts.OwnerID)
  730. }
  731. if !opts.Private.IsNone() {
  732. sess.And("is_private=?", opts.Private.IsTrue())
  733. }
  734. count, err := sess.Count(new(Repository))
  735. if err != nil {
  736. return 0, fmt.Errorf("countRepositories: %w", err)
  737. }
  738. return count, nil
  739. }
  740. // UpdateRepoIssueNumbers updates one of a repositories amount of (open|closed) (issues|PRs) with the current count
  741. func UpdateRepoIssueNumbers(ctx context.Context, repoID int64, isPull, isClosed bool) error {
  742. field := "num_"
  743. if isClosed {
  744. field += "closed_"
  745. }
  746. if isPull {
  747. field += "pulls"
  748. } else {
  749. field += "issues"
  750. }
  751. subQuery := builder.Select("count(*)").
  752. From("issue").Where(builder.Eq{
  753. "repo_id": repoID,
  754. "is_pull": isPull,
  755. }.And(builder.If(isClosed, builder.Eq{"is_closed": isClosed})))
  756. // builder.Update(cond) will generate SQL like UPDATE ... SET cond
  757. query := builder.Update(builder.Eq{field: subQuery}).
  758. From("repository").
  759. Where(builder.Eq{"id": repoID})
  760. _, err := db.Exec(ctx, query)
  761. return err
  762. }
  763. // CountNullArchivedRepository counts the number of repositories with is_archived is null
  764. func CountNullArchivedRepository(ctx context.Context) (int64, error) {
  765. return db.GetEngine(ctx).Where(builder.IsNull{"is_archived"}).Count(new(Repository))
  766. }
  767. // FixNullArchivedRepository sets is_archived to false where it is null
  768. func FixNullArchivedRepository(ctx context.Context) (int64, error) {
  769. return db.GetEngine(ctx).Where(builder.IsNull{"is_archived"}).Cols("is_archived").NoAutoTime().Update(&Repository{
  770. IsArchived: false,
  771. })
  772. }
  773. // UpdateRepositoryOwnerName updates the owner name of all repositories owned by the user
  774. func UpdateRepositoryOwnerName(ctx context.Context, oldUserName, newUserName string) error {
  775. if _, err := db.GetEngine(ctx).Exec("UPDATE `repository` SET owner_name=? WHERE owner_name=?", newUserName, oldUserName); err != nil {
  776. return fmt.Errorf("change repo owner name: %w", err)
  777. }
  778. return nil
  779. }