You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

sqlite.ini.tmpl 3.6KB

Rewrite queue (#24505) # ⚠️ Breaking Many deprecated queue config options are removed (actually, they should have been removed in 1.18/1.19). If you see the fatal message when starting Gitea: "Please update your app.ini to remove deprecated config options", please follow the error messages to remove these options from your app.ini. Example: ``` 2023/05/06 19:39:22 [E] Removed queue option: `[indexer].ISSUE_INDEXER_QUEUE_TYPE`. Use new options in `[queue.issue_indexer]` 2023/05/06 19:39:22 [E] Removed queue option: `[indexer].UPDATE_BUFFER_LEN`. Use new options in `[queue.issue_indexer]` 2023/05/06 19:39:22 [F] Please update your app.ini to remove deprecated config options ``` Many options in `[queue]` are are dropped, including: `WRAP_IF_NECESSARY`, `MAX_ATTEMPTS`, `TIMEOUT`, `WORKERS`, `BLOCK_TIMEOUT`, `BOOST_TIMEOUT`, `BOOST_WORKERS`, they can be removed from app.ini. # The problem The old queue package has some legacy problems: * complexity: I doubt few people could tell how it works. * maintainability: Too many channels and mutex/cond are mixed together, too many different structs/interfaces depends each other. * stability: due to the complexity & maintainability, sometimes there are strange bugs and difficult to debug, and some code doesn't have test (indeed some code is difficult to test because a lot of things are mixed together). * general applicability: although it is called "queue", its behavior is not a well-known queue. * scalability: it doesn't seem easy to make it work with a cluster without breaking its behaviors. It came from some very old code to "avoid breaking", however, its technical debt is too heavy now. It's a good time to introduce a better "queue" package. # The new queue package It keeps using old config and concept as much as possible. * It only contains two major kinds of concepts: * The "base queue": channel, levelqueue, redis * They have the same abstraction, the same interface, and they are tested by the same testing code. * The "WokerPoolQueue", it uses the "base queue" to provide "worker pool" function, calls the "handler" to process the data in the base queue. * The new code doesn't do "PushBack" * Think about a queue with many workers, the "PushBack" can't guarantee the order for re-queued unhandled items, so in new code it just does "normal push" * The new code doesn't do "pause/resume" * The "pause/resume" was designed to handle some handler's failure: eg: document indexer (elasticsearch) is down * If a queue is paused for long time, either the producers blocks or the new items are dropped. * The new code doesn't do such "pause/resume" trick, it's not a common queue's behavior and it doesn't help much. * If there are unhandled items, the "push" function just blocks for a few seconds and then re-queue them and retry. * The new code doesn't do "worker booster" * Gitea's queue's handlers are light functions, the cost is only the go-routine, so it doesn't make sense to "boost" them. * The new code only use "max worker number" to limit the concurrent workers. * The new "Push" never blocks forever * Instead of creating more and more blocking goroutines, return an error is more friendly to the server and to the end user. There are more details in code comments: eg: the "Flush" problem, the strange "code.index" hanging problem, the "immediate" queue problem. Almost ready for review. TODO: * [x] add some necessary comments during review * [x] add some more tests if necessary * [x] update documents and config options * [x] test max worker / active worker * [x] re-run the CI tasks to see whether any test is flaky * [x] improve the `handleOldLengthConfiguration` to provide more friendly messages * [x] fine tune default config values (eg: length?) ## Code coverage: ![image](https://user-images.githubusercontent.com/2114189/236620635-55576955-f95d-4810-b12f-879026a3afdf.png)
1 year ago
Sign merges, CRUD, Wiki and Repository initialisation with gpg key (#7631) This PR fixes #7598 by providing a configurable way of signing commits across the Gitea instance. Per repository configurability and import/generation of trusted secure keys is not provided by this PR - from a security PoV that's probably impossible to do properly. Similarly web-signing, that is asking the user to sign something, is not implemented - this could be done at a later stage however. ## Features - [x] If commit.gpgsign is set in .gitconfig sign commits and files created through repofiles. (merges should already have been signed.) - [x] Verify commits signed with the default gpg as valid - [x] Signer, Committer and Author can all be different - [x] Allow signer to be arbitrarily different - We still require the key to have an activated email on Gitea. A more complete implementation would be to use a keyserver and mark external-or-unactivated with an "unknown" trust level icon. - [x] Add a signing-key.gpg endpoint to get the default gpg pub key if available - Rather than add a fake web-flow user I've added this as an endpoint on /api/v1/signing-key.gpg - [x] Try to match the default key with a user on gitea - this is done at verification time - [x] Make things configurable? - app.ini configuration done - [x] when checking commits are signed need to check if they're actually verifiable too - [x] Add documentation I have decided that adjusting the docker to create a default gpg key is not the correct thing to do and therefore have not implemented this.
4 years ago
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118
  1. APP_NAME = Gitea: Git with a cup of tea
  2. RUN_MODE = prod
  3. [database]
  4. DB_TYPE = sqlite3
  5. PATH = tests/{{TEST_TYPE}}/gitea-{{TEST_TYPE}}-sqlite/gitea.db
  6. [indexer]
  7. REPO_INDEXER_ENABLED = true
  8. REPO_INDEXER_PATH = tests/{{TEST_TYPE}}/gitea-{{TEST_TYPE}}-sqlite/indexers/repos.bleve
  9. [queue.issue_indexer]
  10. TYPE = level
  11. DATADIR = tests/{{TEST_TYPE}}/gitea-{{TEST_TYPE}}-sqlite/indexers/issues.queue
  12. [queue]
  13. TYPE = immediate
  14. [queue.code_indexer]
  15. TYPE = immediate
  16. [queue.push_update]
  17. TYPE = immediate
  18. [repository]
  19. ROOT = {{REPO_TEST_DIR}}tests/{{TEST_TYPE}}/gitea-{{TEST_TYPE}}-sqlite/gitea-repositories
  20. [repository.local]
  21. LOCAL_COPY_PATH = tests/{{TEST_TYPE}}/gitea-{{TEST_TYPE}}-sqlite/tmp/local-repo
  22. [repository.upload]
  23. TEMP_PATH = tests/{{TEST_TYPE}}/gitea-{{TEST_TYPE}}-sqlite/tmp/uploads
  24. [repository.signing]
  25. SIGNING_KEY = none
  26. [server]
  27. SSH_DOMAIN = localhost
  28. HTTP_PORT = 3003
  29. ROOT_URL = http://localhost:3003/
  30. DISABLE_SSH = false
  31. SSH_LISTEN_HOST = localhost
  32. SSH_PORT = 2203
  33. START_SSH_SERVER = true
  34. LFS_START_SERVER = true
  35. OFFLINE_MODE = false
  36. LFS_JWT_SECRET = Tv_MjmZuHqpIY6GFl12ebgkRAMt4RlWt0v4EHKSXO0w
  37. APP_DATA_PATH = tests/{{TEST_TYPE}}/gitea-{{TEST_TYPE}}-sqlite/data
  38. ENABLE_GZIP = true
  39. BUILTIN_SSH_SERVER_USER = git
  40. SSH_TRUSTED_USER_CA_KEYS = ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCb4DC1dMFnJ6pXWo7GMxTchtzmJHYzfN6sZ9FAPFR4ijMLfGki+olvOMO5Fql1/yGnGfbELQa1S6y4shSvj/5K+zUFScmEXYf3Gcr87RqilLkyk16RS+cHNB1u87xTHbETaa3nyCJeGQRpd4IQ4NKob745mwDZ7jQBH8AZEng50Oh8y8fi8skBBBzaYp1ilgvzG740L7uex6fHV62myq0SXeCa+oJUjq326FU8y+Vsa32H8A3e7tOgXZPdt2TVNltx2S9H2WO8RMi7LfaSwARNfy1zu+bfR50r6ef8Yx5YKCMz4wWb1SHU1GS800mjOjlInLQORYRNMlSwR1+vLlVDciOqFapDSbj+YOVOawR0R1aqlSKpZkt33DuOBPx9qe6CVnIi7Z+Px/KqM+OLCzlLY/RS+LbxQpDWcfTVRiP+S5qRTcE3M3UioN/e0BE/1+MpX90IGpvVkA63ILYbKEa4bM3ASL7ChTCr6xN5XT+GpVJveFKK1cfNx9ExHI4rzYE=
  41. [attachment]
  42. PATH = tests/{{TEST_TYPE}}/gitea-{{TEST_TYPE}}-sqlite/data/attachments
  43. [mailer]
  44. ENABLED = true
  45. PROTOCOL = dummy
  46. FROM = sqlite-{{TEST_TYPE}}-test@gitea.io
  47. [service]
  48. REGISTER_EMAIL_CONFIRM = false
  49. REGISTER_MANUAL_CONFIRM = false
  50. ENABLE_NOTIFY_MAIL = true
  51. DISABLE_REGISTRATION = false
  52. ENABLE_CAPTCHA = false
  53. REQUIRE_SIGNIN_VIEW = false
  54. DEFAULT_KEEP_EMAIL_PRIVATE = false
  55. DEFAULT_ALLOW_CREATE_ORGANIZATION = true
  56. NO_REPLY_ADDRESS = noreply.example.org
  57. [picture]
  58. DISABLE_GRAVATAR = false
  59. ENABLE_FEDERATED_AVATAR = false
  60. AVATAR_UPLOAD_PATH = tests/{{TEST_TYPE}}/gitea-{{TEST_TYPE}}-sqlite/data/avatars
  61. REPOSITORY_AVATAR_UPLOAD_PATH = tests/{{TEST_TYPE}}/gitea-{{TEST_TYPE}}-sqlite/data/repo-avatars
  62. [session]
  63. PROVIDER = file
  64. PROVIDER_CONFIG = tests/{{TEST_TYPE}}/gitea-{{TEST_TYPE}}-sqlite/data/sessions
  65. [log]
  66. MODE = {{TEST_LOGGER}}
  67. ROOT_PATH = {{REPO_TEST_DIR}}sqlite-log
  68. ENABLE_SSH_LOG = true
  69. logger.xorm.MODE = file
  70. [log.test]
  71. LEVEL = Info
  72. COLORIZE = true
  73. [log.file]
  74. LEVEL = Debug
  75. [security]
  76. DISABLE_GIT_HOOKS = false
  77. INSTALL_LOCK = true
  78. SECRET_KEY = 9pCviYTWSb
  79. INTERNAL_TOKEN = eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJuYmYiOjE0OTI3OTU5ODN9.OQkH5UmzID2XBdwQ9TAI6Jj2t1X-wElVTjbE7aoN4I8
  80. [oauth2]
  81. JWT_SECRET = KZb_QLUd4fYVyxetjxC4eZkrBgWM2SndOOWDNtgUUko
  82. [lfs]
  83. PATH = tests/{{TEST_TYPE}}/gitea-{{TEST_TYPE}}-sqlite/data/lfs
  84. [packages]
  85. ENABLED = true
  86. [markup.html]
  87. ENABLED = true
  88. FILE_EXTENSIONS = .html
  89. RENDER_COMMAND = `go run build/test-echo.go`
  90. IS_INPUT_FILE = false
  91. RENDER_CONTENT_MODE=sanitized
  92. [actions]
  93. ENABLED = true