summaryrefslogtreecommitdiffstats
path: root/modules/util
Commit message (Collapse)AuthorAgeFilesLines
* Handle base64 decoding correctly to avoid panic (#26483)wxiaoguang2023-08-142-0/+25
| | | Fix the panic if the "base64 secret" is too long.
* Less naked returns (#25713)65432023-07-072-0/+14
| | | | | just a step towards #25655 and some related refactoring
* Replace `interface{}` with `any` (#25686)silverwind2023-07-045-12/+12
| | | | | Result of running `perl -p -i -e 's#interface\{\}#any#g' **/*` and `make fmt`. Basically the same [as golang did](https://github.com/golang/go/commit/2580d0e08d5e9f979b943758d3c49877fb2324cb).
* Store and use seconds for timeline time comments (#25392)65432023-06-231-1/+3
| | | | | | | | | | | this will allow us to fully localize it later PS: we can not migrate back as the old value was a one-way conversion prepare for #25213 --- *Sponsored by Kithara Software GmbH*
* Move some regexp out of functions (#25430)John Olheiser2023-06-211-2/+3
| | | | | | | | | | /cc @KN4CK3R https://github.com/go-gitea/gitea/pull/25294#discussion_r1237425343 I also searched the codebase and found a few more. --------- Signed-off-by: jolheiser <john.olheiser@gmail.com>
* Fix profile render when the README.md size is larger than 1024 bytes (#25131)yp053272023-06-132-1/+104
| | | | | | | | | | | | | | | | | Fixes https://github.com/go-gitea/gitea/issues/25094 `GetBlobContent` will only get the first 1024 bytes, if the README.md size is larger than 1024 bytes, We can not render the rest of them. After this fix, we should provide the limited size to read when call `GetBlobContent`. After: ![image](https://github.com/go-gitea/gitea/assets/18380374/22a42936-4cf8-40b4-a5c7-e384082beb0d) --------- Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
* Add ability to set multiple redirect URIs in OAuth application UI (#25072)Denys Konovalov2023-06-051-1/+18
| | | | | | | | | | | | | | | | | | | | OAuth applications can already have multiple redirect URIs if created/edited over API. This change allows for setting multiple redirect URIs through the UI as a comma-separated list (e. g. `https://example.org/redirect,https://redirect.example.org`) <details> <summary>Screenshots</summary> ![Bildschirmfoto vom 2023-06-04 17-14-40](https://github.com/go-gitea/gitea/assets/47871822/2206dc32-e7e4-4953-9ecb-e098890b3f54) ![Bildschirmfoto vom 2023-06-04 17-14-50](https://github.com/go-gitea/gitea/assets/47871822/cd97c73c-9310-44ee-a83a-b927a1ef94da) </details> Closes #25068
* Rewrite logger system (#24726)wxiaoguang2023-05-212-0/+294
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ## ⚠️ Breaking The `log.<mode>.<logger>` style config has been dropped. If you used it, please check the new config manual & app.example.ini to make your instance output logs as expected. Although many legacy options still work, it's encouraged to upgrade to the new options. The SMTP logger is deleted because SMTP is not suitable to collect logs. If you have manually configured Gitea log options, please confirm the logger system works as expected after upgrading. ## Description Close #12082 and maybe more log-related issues, resolve some related FIXMEs in old code (which seems unfixable before) Just like rewriting queue #24505 : make code maintainable, clear legacy bugs, and add the ability to support more writers (eg: JSON, structured log) There is a new document (with examples): `logging-config.en-us.md` This PR is safer than the queue rewriting, because it's just for logging, it won't break other logic. ## The old problems The logging system is quite old and difficult to maintain: * Unclear concepts: Logger, NamedLogger, MultiChannelledLogger, SubLogger, EventLogger, WriterLogger etc * Some code is diffuclt to konw whether it is right: `log.DelNamedLogger("console")` vs `log.DelNamedLogger(log.DEFAULT)` vs `log.DelLogger("console")` * The old system heavily depends on ini config system, it's difficult to create new logger for different purpose, and it's very fragile. * The "color" trick is difficult to use and read, many colors are unnecessary, and in the future structured log could help * It's difficult to add other log formats, eg: JSON format * The log outputer doesn't have full control of its goroutine, it's difficult to make outputer have advanced behaviors * The logs could be lost in some cases: eg: no Fatal error when using CLI. * Config options are passed by JSON, which is quite fragile. * INI package makes the KEY in `[log]` section visible in `[log.sub1]` and `[log.sub1.subA]`, this behavior is quite fragile and would cause more unclear problems, and there is no strong requirement to support `log.<mode>.<logger>` syntax. ## The new design See `logger.go` for documents. ## Screenshot <details> ![image](https://github.com/go-gitea/gitea/assets/2114189/4462d713-ba39-41f5-bb08-de912e67e1ff) ![image](https://github.com/go-gitea/gitea/assets/2114189/b188035e-f691-428b-8b2d-ff7b2199b2f9) ![image](https://github.com/go-gitea/gitea/assets/2114189/132e9745-1c3b-4e00-9e0d-15eaea495dee) </details> ## TODO * [x] add some new tests * [x] fix some tests * [x] test some sub-commands (manually ....) --------- Co-authored-by: Jason Song <i@wolfogre.com> Co-authored-by: delvh <dev.lh@web.de> Co-authored-by: Giteabot <teabot@gitea.io>
* Add Alpine package registry (#23714)KN4CK3R2023-05-121-0/+16
| | | | | | | | | | | | | | | | | | This PR adds an Alpine package registry. You can follow [this tutorial](https://wiki.alpinelinux.org/wiki/Creating_an_Alpine_package) to build a *.apk package for testing. This functionality is similar to the Debian registry (#22854) and therefore shares some methods. I marked this PR as blocked because it should be merged after #22854. ![grafik](https://user-images.githubusercontent.com/1666336/227779595-b76163aa-eea1-4a79-9583-775c24ad74e8.png) --------- Co-authored-by: techknowlogick <techknowlogick@gitea.io> Co-authored-by: silverwind <me@silverwind.io> Co-authored-by: Giteabot <teabot@gitea.io>
* Modify luminance calculation and extract related functions into single files ↵Hester Gong2023-05-102-0/+130
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | (#24586) Close #24508 Main changes: As discussed in the issue 1. Change luminance calculation function to use [Relative Luminance](https://www.w3.org/WAI/GL/wiki/Relative_luminance) 2. Move the luminance related functions into color.go/color.js 3. Add tests for both the files (Not sure if test cases are too many now) Before (tests included by `UseLightTextOnBackground` are labels started with `##`): https://try.gitea.io/HesterG/testrepo/labels After: <img width="1307" alt="Screen Shot 2023-05-08 at 13 37 55" src="https://user-images.githubusercontent.com/17645053/236742562-fdfc3a4d-2fab-466b-9613-96f2bf96b4bc.png"> <img width="1289" alt="Screen Shot 2023-05-08 at 13 38 06" src="https://user-images.githubusercontent.com/17645053/236742570-022db68e-cec0-43bb-888a-fc54f5332cc3.png"> <img width="1299" alt="Screen Shot 2023-05-08 at 13 38 20" src="https://user-images.githubusercontent.com/17645053/236742572-9af1de45-fb7f-460b-828d-ba25fae20f51.png"> --------- Co-authored-by: silverwind <me@silverwind.io> Co-authored-by: Giteabot <teabot@gitea.io>
* Rewrite queue (#24505)wxiaoguang2023-05-081-12/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | # ⚠️ Breaking Many deprecated queue config options are removed (actually, they should have been removed in 1.18/1.19). If you see the fatal message when starting Gitea: "Please update your app.ini to remove deprecated config options", please follow the error messages to remove these options from your app.ini. Example: ``` 2023/05/06 19:39:22 [E] Removed queue option: `[indexer].ISSUE_INDEXER_QUEUE_TYPE`. Use new options in `[queue.issue_indexer]` 2023/05/06 19:39:22 [E] Removed queue option: `[indexer].UPDATE_BUFFER_LEN`. Use new options in `[queue.issue_indexer]` 2023/05/06 19:39:22 [F] Please update your app.ini to remove deprecated config options ``` Many options in `[queue]` are are dropped, including: `WRAP_IF_NECESSARY`, `MAX_ATTEMPTS`, `TIMEOUT`, `WORKERS`, `BLOCK_TIMEOUT`, `BOOST_TIMEOUT`, `BOOST_WORKERS`, they can be removed from app.ini. # The problem The old queue package has some legacy problems: * complexity: I doubt few people could tell how it works. * maintainability: Too many channels and mutex/cond are mixed together, too many different structs/interfaces depends each other. * stability: due to the complexity & maintainability, sometimes there are strange bugs and difficult to debug, and some code doesn't have test (indeed some code is difficult to test because a lot of things are mixed together). * general applicability: although it is called "queue", its behavior is not a well-known queue. * scalability: it doesn't seem easy to make it work with a cluster without breaking its behaviors. It came from some very old code to "avoid breaking", however, its technical debt is too heavy now. It's a good time to introduce a better "queue" package. # The new queue package It keeps using old config and concept as much as possible. * It only contains two major kinds of concepts: * The "base queue": channel, levelqueue, redis * They have the same abstraction, the same interface, and they are tested by the same testing code. * The "WokerPoolQueue", it uses the "base queue" to provide "worker pool" function, calls the "handler" to process the data in the base queue. * The new code doesn't do "PushBack" * Think about a queue with many workers, the "PushBack" can't guarantee the order for re-queued unhandled items, so in new code it just does "normal push" * The new code doesn't do "pause/resume" * The "pause/resume" was designed to handle some handler's failure: eg: document indexer (elasticsearch) is down * If a queue is paused for long time, either the producers blocks or the new items are dropped. * The new code doesn't do such "pause/resume" trick, it's not a common queue's behavior and it doesn't help much. * If there are unhandled items, the "push" function just blocks for a few seconds and then re-queue them and retry. * The new code doesn't do "worker booster" * Gitea's queue's handlers are light functions, the cost is only the go-routine, so it doesn't make sense to "boost" them. * The new code only use "max worker number" to limit the concurrent workers. * The new "Push" never blocks forever * Instead of creating more and more blocking goroutines, return an error is more friendly to the server and to the end user. There are more details in code comments: eg: the "Flush" problem, the strange "code.index" hanging problem, the "immediate" queue problem. Almost ready for review. TODO: * [x] add some necessary comments during review * [x] add some more tests if necessary * [x] update documents and config options * [x] test max worker / active worker * [x] re-run the CI tasks to see whether any test is flaky * [x] improve the `handleOldLengthConfiguration` to provide more friendly messages * [x] fine tune default config values (eg: length?) ## Code coverage: ![image](https://user-images.githubusercontent.com/2114189/236620635-55576955-f95d-4810-b12f-879026a3afdf.png)
* Add Debian package registry (#24426)KN4CK3R2023-05-021-3/+2
| | | | | | | | | | | | | | | | | | | | | | | | Co-authored-by: @awkwardbunny This PR adds a Debian package registry. You can follow [this tutorial](https://www.baeldung.com/linux/create-debian-package) to build a *.deb package for testing. Source packages are not supported at the moment and I did not find documentation of the architecture "all" and how these packages should be treated. ![grafik](https://user-images.githubusercontent.com/1666336/218126879-eb80a866-775c-4c8e-8529-5797203a64e6.png) Part of #20751. Revised copy of #22854. --------- Co-authored-by: Brian Hong <brian@hongs.me> Co-authored-by: techknowlogick <techknowlogick@gitea.io> Co-authored-by: Giteabot <teabot@gitea.io>
* Revert "Add Debian package registry" (#24412)Yarden Shoham2023-04-281-2/+3
| | | Reverts go-gitea/gitea#22854
* Add Debian package registry (#22854)KN4CK3R2023-04-281-3/+2
| | | | | | | | | | | | | | Co-authored-by: @awkwardbunny This PR adds a Debian package registry. You can follow [this tutorial](https://www.baeldung.com/linux/create-debian-package) to build a *.deb package for testing. Source packages are not supported at the moment and I did not find documentation of the architecture "all" and how these packages should be treated. --------- Co-authored-by: Brian Hong <brian@hongs.me> Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Use more specific test methods (#24265)KN4CK3R2023-04-222-12/+12
| | | | Co-authored-by: silverwind <me@silverwind.io> Co-authored-by: Giteabot <teabot@gitea.io>
* Use a general approach to access custom/static/builtin assets (#24022)wxiaoguang2023-04-124-11/+68
| | | | | | | | | | | | | | | | | | | The idea is to use a Layered Asset File-system (modules/assetfs/layered.go) For example: when there are 2 layers: "custom", "builtin", when access to asset "my/page.tmpl", the Layered Asset File-system will first try to use "custom" assets, if not found, then use "builtin" assets. This approach will hugely simplify a lot of code, make them testable. Other changes: * Simplify the AssetsHandlerFunc code * Simplify the `gitea embedded` sub-command code --------- Co-authored-by: Jason Song <i@wolfogre.com> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Merge template functions "dict/Dict/mergeinto" (#23932)wxiaoguang2023-04-071-24/+0
| | | | | | | | | | | | | | | | | | One of the steps in #23328 Before there were 3 different but similar functions: dict/Dict/mergeinto The code was just copied & pasted, no test. This PR defines a new stable `dict` function, it covers all the 3 old functions behaviors, only +160 -171 Future developers do not need to think about or guess the different dict functions, just use one: `dict` Why use `dict` but not `Dict`? Because there are far more `dict` than `Dict` in code already ......
* Use a general Eval function for expressions in templates. (#23927)wxiaoguang2023-04-071-11/+53
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | One of the proposals in #23328 This PR introduces a simple expression calculator (templates/eval/eval.go), it can do basic expression calculations. Many untested template helper functions like `Mul` `Add` can be replaced by this new approach. Then these `Add` / `Mul` / `percentage` / `Subtract` / `DiffStatsWidth` could all use this `Eval`. And it provides enhancements for Golang templates, and improves readability. Some examples: ---- * Before: `{{Add (Mul $glyph.Row 12) 12}}` * After: `{{Eval $glyph.Row "*" 12 "+" 12}}` ---- * Before: `{{if lt (Add $i 1) (len $.Topics)}}` * After: `{{if Eval $i "+" 1 "<" (len $.Topics)}}` ## FAQ ### Why not use an existing expression package? We need a highly customized expression engine: * do the calculation on the fly, without pre-compiling * deal with int/int64/float64 types, to make the result could be used in Golang template. * make the syntax could be used in the Golang template directly * do not introduce too much complex or strange syntax, we just need a simple calculator. * it needs to strictly follow Golang template's behavior, for example, Golang template treats all non-zero values as truth, but many 3rd packages don't do so. ### What's the benefit? * Developers don't need to add more `Add`/`Mul`/`Sub`-like functions, they were getting more and more. Now, only one `Eval` is enough for all cases. * The new code reads better than old `{{Add (Mul $glyph.Row 12) 12}}`, the old one isn't familiar to most procedural programming developers (eg, the Golang expression syntax). * The `Eval` is fully covered by tests, many old `Add`/`Mul`-like functions were never tested. ### The performance? It doesn't use `reflect`, it doesn't need to parse or compile when used in Golang template, the performance is as fast as native Go template. ### Is it too complex? Could it be unstable? The expression calculator program is a common homework for computer science students, and it's widely used as a teaching and practicing purpose for developers. The algorithm is pretty well-known. The behavior can be clearly defined, it is stable.
* Fix `cases.Title` crash for concurrency (#23885)wxiaoguang2023-04-031-8/+5
| | | | | | | | Regression of #19676 and #21814 Fix #23872 `cases.Title` is not thread-safe, it has internal state, so it can't be used as a global shared variable.
* Introduce GiteaLocaleNumber custom element to handle number localization on ↵wxiaoguang2023-04-034-78/+25
| | | | | | | | | | | | | | | | | | | | | | | pages. (#23861) Follow #21429 & #22861 Use `<gitea-locale-number>` instead of backend `PrettyNumber`. All old `PrettyNumber` related functions are removed. A lot of code could be simplified. And some functions haven't been used for long time (dead code), so they are also removed by the way (eg: `SplitStringAtRuneN`, `Dedent`) This PR only tries to improve the `PrettyNumber` rendering problem, it doesn't touch the "plural" problem. Screenshot: ![image](https://user-images.githubusercontent.com/2114189/229290804-1f63db65-1e34-4a54-84ba-e00b44331b17.png) ![image](https://user-images.githubusercontent.com/2114189/229290911-c88dea00-b11d-48dd-accb-9f52edd73ce4.png)
* Introduce path Clean/Join helper functions (#23495)wxiaoguang2023-03-212-17/+151
| | | | | | | | | | | | | | | Since #23493 has conflicts with latest commits, this PR is my proposal for fixing #23371 Details are in the comments And refactor the `modules/options` module, to make it always use "filepath" to access local files. Benefits: * No need to do `util.CleanPath(strings.ReplaceAll(p, "\\", "/"))), "/")` any more (not only one before) * The function behaviors are clearly defined
* Use CleanPath instead of path.Clean (#23371)Lunny Xiao2023-03-082-0/+20
| | | As title.
* Use minio/sha256-simd for accelerated SHA256 (#23052)zeripath2023-02-221-1/+1
| | | | | | | | | | | | minio/sha256-simd provides additional acceleration for SHA256 using AVX512, SHA Extensions for x86 and ARM64 for ARM. It provides a drop-in replacement for crypto/sha256 and if the extensions are not available it falls back to standard crypto/sha256. --------- Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: John Olheiser <john.olheiser@gmail.com>
* Move `IsReadmeFile*` from `modules/markup/` to `modules/util` (#22877)Nick2023-02-132-0/+120
| | | | | | | | | These functions don't examine contents, just filenames, so they don't fit in well in a markup module. This was originally part of https://github.com/go-gitea/gitea/pull/22177. Signed-off-by: Nick Guenther <nick.guenther@polymtl.ca>
* Add Chef package registry (#22554)KN4CK3R2023-02-062-0/+106
| | | | | | | | | | | | | This PR implements a [Chef registry](https://chef.io/) to manage cookbooks. This package type was a bit complicated because Chef uses RSA signed requests as authentication with the registry. ![grafik](https://user-images.githubusercontent.com/1666336/213747995-46819fd8-c3d6-45a2-afd4-a4c3c8505a4a.png) ![grafik](https://user-images.githubusercontent.com/1666336/213748145-d01c9e81-d4dd-41e3-a3cc-8241862c3166.png) Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Improve checkIfPRContentChanged (#22611)zeripath2023-01-281-0/+22
| | | | | | | | | | | | | | | | The code for checking if a commit has caused a change in a PR is extremely inefficient and affects the head repository instead of using a temporary repository. This PR therefore makes several significant improvements: * A temporary repo like that used in merging. * The diff code is then significant improved to use a three-way diff instead of comparing diffs (possibly binary) line-by-line - in memory... Ref #22578 Signed-off-by: Andrew Thornton <art27@cantab.net>
* Add support for incoming emails (#22056)KN4CK3R2023-01-142-0/+61
| | | | | | | | | | | | | | | | | | | | | | | | | | | | closes #13585 fixes #9067 fixes #2386 ref #6226 ref #6219 fixes #745 This PR adds support to process incoming emails to perform actions. Currently I added handling of replies and unsubscribing from issues/pulls. In contrast to #13585 the IMAP IDLE command is used instead of polling which results (in my opinion 😉) in cleaner code. Procedure: - When sending an issue/pull reply email, a token is generated which is present in the Reply-To and References header. - IMAP IDLE waits until a new email arrives - The token tells which action should be performed A possible signature and/or reply gets stripped from the content. I added a new service to the drone pipeline to test the receiving of incoming mails. If we keep this in, we may test our outgoing emails too in future. Co-authored-by: silverwind <me@silverwind.io> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Fix error when calculate the repository size (#22392)Lunny Xiao2023-01-131-14/+0
| | | | | | | | Fix #22386 `GetDirectorySize` moved as `getDirectorySize` because it becomes a special function which should not be put in `util`. Co-authored-by: Jason Song <i@wolfogre.com>
* Improve utils of slices (#22379)Jason Song2023-01-113-101/+170
| | | | | | | | | | | | | | | | | | | - Move the file `compare.go` and `slice.go` to `slice.go`. - Fix `ExistsInSlice`, it's buggy - It uses `sort.Search`, so it assumes that the input slice is sorted. - It passes `func(i int) bool { return slice[i] == target })` to `sort.Search`, that's incorrect, check the doc of `sort.Search`. - Conbine `IsInt64InSlice(int64, []int64)` and `ExistsInSlice(string, []string)` to `SliceContains[T]([]T, T)`. - Conbine `IsSliceInt64Eq([]int64, []int64)` and `IsEqualSlice([]string, []string)` to `SliceSortedEqual[T]([]T, T)`. - Add `SliceEqual[T]([]T, T)` as a distinction from `SliceSortedEqual[T]([]T, T)`. - Redesign `RemoveIDFromList([]int64, int64) ([]int64, bool)` to `SliceRemoveAll[T]([]T, T) []T`. - Add `SliceContainsFunc[T]([]T, func(T) bool)` and `SliceRemoveAllFunc[T]([]T, func(T) bool)` for general use. - Add comments to explain why not `golang.org/x/exp/slices`. - Add unit tests.
* Use ErrInvalidArgument in packages (#22268)KN4CK3R2022-12-311-0/+29
| | | | | | | | | Related to https://github.com/go-gitea/gitea/pull/22262#discussion_r1059010774 Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com> Co-authored-by: zeripath <art27@cantab.net> Co-authored-by: Lauris BH <lauris@nix.lv>
* Implement FSFE REUSE for golang files (#21840)flynnnnnnnnnn2022-11-2730-60/+30
| | | | | | | | | Change all license headers to comply with REUSE specification. Fix #16132 Co-authored-by: flynnnnnnnnnn <flynnnnnnnnnn@github> Co-authored-by: John Olheiser <john.olheiser@gmail.com>
* Show syntax lexer name in file view/blame (#21814)silverwind2022-11-191-1/+9
| | | | | | | | | | | | | | | | | | | | | | Show which Chroma Lexer is used to highlight the file in the file header. It's useful for development to see what was detected, and I think it's not bad info to have for the user: <img width="233" alt="Screenshot 2022-11-14 at 22 31 16" src="https://user-images.githubusercontent.com/115237/201770854-44933dfc-70a4-487c-8457-1bb3cc43ea62.png"> <img width="226" alt="Screenshot 2022-11-14 at 22 36 06" src="https://user-images.githubusercontent.com/115237/201770856-9260ce6f-6c0f-442c-92b5-201e5b113188.png"> <img width="194" alt="Screenshot 2022-11-14 at 22 36 26" src="https://user-images.githubusercontent.com/115237/201770857-6f56591b-80ea-42cc-8ea5-21b9156c018b.png"> Also, I improved the way this header overflows on small screens: <img width="354" alt="Screenshot 2022-11-14 at 22 44 36" src="https://user-images.githubusercontent.com/115237/201774828-2ddbcde1-da15-403f-bf7a-6248449fa2c5.png"> Co-authored-by: delvh <dev.lh@web.de> Co-authored-by: Lauris BH <lauris@nix.lv> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com> Co-authored-by: John Olheiser <john.olheiser@gmail.com>
* Make every not exist error unwrappable to a fs.ErrNotExist (#20891)zeripath2022-10-181-0/+37
| | | | | | | | | | | | | | | | | | | | A lot of our code is repeatedly testing if individual errors are specific types of Not Exist errors. This is repetitative and unnecesary. `Unwrap() error` provides a common way of labelling an error as a NotExist error and we can/should use this. This PR has chosen to use the common `io/fs` errors e.g. `fs.ErrNotExist` for our errors. This is in some ways not completely correct as these are not filesystem errors but it seems like a reasonable thing to do and would allow us to simplify a lot of our code to `errors.Is(err, fs.ErrNotExist)` instead of `package.IsErr...NotExist(err)` I am open to suggestions to use a different base error - perhaps `models/db.ErrNotExist` if that would be felt to be better. Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: delvh <dev.lh@web.de>
* Save files in local storage as umask (#21198)Nulo (he/him)2022-09-243-0/+80
| | | | | | | | Go creates temporary files as 600, but sometimes we want the group to be able to read them (for example, for another user to back up the storage) This PR applies the umask to the renamed tmp files in local storage. Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
* Go 1.19 format (#20758)John Olheiser2022-08-301-6/+6
| | | | | | | | | | | | | | | | | | | | | | | | * 1.19 gofumpt Signed-off-by: jolheiser <john.olheiser@gmail.com> * Change CSV test Signed-off-by: jolheiser <john.olheiser@gmail.com> * Commit whitespace fixes from @zeripath Co-authored-by: zeripath <art27@cantab.net> * Update emoji Signed-off-by: jolheiser <john.olheiser@gmail.com> * bump swagger & fix generate-swagger Signed-off-by: jolheiser <john.olheiser@gmail.com> Co-authored-by: zeripath <art27@cantab.net> Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: Lauris BH <lauris@nix.lv>
* Share HTML template renderers and create a watcher framework (#20218)zeripath2022-08-281-2/+19
| | | | | | | | | | | | | | | | | | | | | | | | | The recovery, API, Web and package frameworks all create their own HTML Renderers. This increases the memory requirements of Gitea unnecessarily with duplicate templates being kept in memory. Further the reloading framework in dev mode for these involves locking and recompiling all of the templates on each load. This will potentially hide concurrency issues and it is inefficient. This PR stores the templates renderer in the context and stores this context in the NormalRoutes, it then creates a fsnotify.Watcher framework to watch files. The watching framework is then extended to the mailer templates which were previously not being reloaded in dev. Then the locales are simplified to a similar structure. Fix #20210 Fix #20211 Fix #20217 Signed-off-by: Andrew Thornton <art27@cantab.net>
* Fix SecToTime edge-cases (#20610)Gusted2022-08-082-10/+33
|
* Fix package upload for files >32mb (#20622)KN4CK3R2022-08-022-5/+51
| | | | | | * Rewind file before first read. * Added tests. Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Add more linters to improve code readability (#19989)Wim2022-06-201-1/+1
| | | | | | | | | | Add nakedret, unconvert, wastedassign, stylecheck and nolintlint linters to improve code readability - nakedret - https://github.com/alexkohler/nakedret - nakedret is a Go static analysis tool to find naked returns in functions greater than a specified function length. - unconvert - https://github.com/mdempsky/unconvert - Remove unnecessary type conversions - wastedassign - https://github.com/sanposhiho/wastedassign - wastedassign finds wasted assignment statements. - notlintlint - Reports ill-formed or insufficient nolint directives - stylecheck - https://staticcheck.io/docs/checks/#ST - keep style consistent - excluded: [ST1003 - Poorly chosen identifier](https://staticcheck.io/docs/checks/#ST1003) and [ST1005 - Incorrectly formatted error string](https://staticcheck.io/docs/checks/#ST1005)
* Prettify number of issues (#17760)Gusted2022-06-121-0/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Prettify number of issues - Use the PrettyNumber function to add commas in large amount of issues. * Use client-side formatting * prettify on both server and client * remove unused i18n entries * handle more cases, support other int types in PrettyNumber * specify locale to avoid issues with node default locale * remove superfluos argument * introduce template helper, octicon tweaks, js refactor * Update modules/templates/helper.go * Apply some suggestions. * Add comment * Update templates/user/dashboard/issues.tmpl Co-authored-by: silverwind <me@silverwind.io> Co-authored-by: silverwind <me@silverwind.io> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Fix copy/paste of empty lines (#19798)silverwind2022-06-102-0/+40
| | | | | | | | | | | | | | | | | | | | | | | | | | | * Fix copy/paste of empty newlines again Fixes: https://github.com/go-gitea/gitea/issues/19331 Regressed by: https://github.com/go-gitea/gitea/pull/18270 Needed to do another newline addition to the Chroma output HTML to get copy/paste work again. The previous replacement conditions are probably obsolete, but as I'm not 100% sure, I opted to keep them. Specifically, the Chroma HTML change mentioned in https://github.com/go-gitea/gitea/pull/18270#issuecomment-1013350246 broke our previous newline replacement for such empty lines. Also included are a few changes to make the test more pleasant to work with. * run go mod tidy * add util.Dedent * copy in the code Co-authored-by: techknowlogick <techknowlogick@gitea.io> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com> Co-authored-by: Lauris BH <lauris@nix.lv>
* Refactor git module, make Gitea use internal git config (#19732)wxiaoguang2022-06-101-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Refactor git module, make Gitea use internal git config, add safe.directory config * introduce git.InitSimple and git.InitWithConfigSync, make serv cmd use gitconfig * use HOME instead of GIT_CONFIG_GLOBAL, because git always needs a correct HOME * fix cmd env in cmd/serv.go * fine tune error message * Fix a incorrect test case * fix configAddNonExist * fix configAddNonExist logic, add `--fixed-value` flag, add tests * add configSetNonExist function in case it's needed. * use configSetNonExist for `user.name` and `user.email` * add some comments * Update cmd/serv.go Co-authored-by: zeripath <art27@cantab.net> * Update cmd/serv.go Co-authored-by: zeripath <art27@cantab.net> * Update modules/git/git.go Co-authored-by: zeripath <art27@cantab.net> * Update modules/setting/setting.go Co-authored-by: zeripath <art27@cantab.net> * Update modules/git/repo_attribute.go Co-authored-by: zeripath <art27@cantab.net> * fix spaces in messages * use `configSet("core.protectNTFS", ...)` instead of `globalCommandArgs` * remove GIT_CONFIG_NOSYSTEM, continue to use system's git config * Update cmd/serv.go Co-authored-by: zeripath <art27@cantab.net> * fix merge * remove code for safe.directory * separate git.CommonEnvs to CommonGitCmdEnvs and CommonCmdServEnvs * avoid Golang's data race error Co-authored-by: zeripath <art27@cantab.net> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Update go tool dependencies (#19676)silverwind2022-05-102-0/+15
| | | | | | | | | | | * Update go tool dependencies Updated all tool dependencies to latest tags, hoping CI will like it. * fix new lint errors * handle more strings.Title cases * remove lint skip
* Remove legacy `unknwon/com` package (#19298)wxiaoguang2022-04-024-22/+187
| | | | | | | Follows: #19284 * The `CopyDir` is only used inside test code * Rewrite `ToSnakeCase` with more test cases * The `RedisCacher` only put strings into cache, here we use internal `toStr` to replace the legacy `ToStr` * The `UniqueQueue` can use string as ID directly, no need to call `ToStr`
* Refactor legacy `unknwon/com` package, improve golangci lint (#19284)wxiaoguang2022-04-015-22/+148
| | | | | | | | The main purpose is to refactor the legacy `unknwon/com` package. 1. Remove most imports of `unknwon/com`, only `util/legacy.go` imports the legacy `unknwon/com` 2. Use golangci's depguard to process denied packages 3. Fix some incorrect values in golangci.yml, eg, the version should be quoted string `"1.18"` 4. Use correctly escaped content for `go-import` and `go-source` meta tags 5. Refactor `com.Expand` to our stable (and the same fast) `vars.Expand`, our `vars.Expand` can still return partially rendered content even if the template is not good (eg: key mistach).
* Use a more general (and faster) method to sanitize URLs with credentials ↵wxiaoguang2022-03-312-152/+75
| | | | | (#19239) Use a more general method to sanitize URLs with credentials: Simple and intuitive / Faster / Remove all credentials in all URLs
* Add Package Registry (#16510)KN4CK3R2022-03-301-0/+147
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Added package store settings. * Added models. * Added generic package registry. * Added tests. * Added NuGet package registry. * Moved service index to api file. * Added NPM package registry. * Added Maven package registry. * Added PyPI package registry. * Summary is deprecated. * Changed npm name. * Sanitize project url. * Allow only scoped packages. * Added user interface. * Changed method name. * Added missing migration file. * Set page info. * Added documentation. * Added documentation links. * Fixed wrong error message. * Lint template files. * Fixed merge errors. * Fixed unit test storage path. * Switch to json module. * Added suggestions. * Added package webhook. * Add package api. * Fixed swagger file. * Fixed enum and comments. * Fixed NuGet pagination. * Print test names. * Added api tests. * Fixed access level. * Fix User unmarshal. * Added RubyGems package registry. * Fix lint. * Implemented io.Writer. * Added support for sha256/sha512 checksum files. * Improved maven-metadata.xml support. * Added support for symbol package uploads. * Added tests. * Added overview docs. * Added npm dependencies and keywords. * Added no-packages information. * Display file size. * Display asset count. * Fixed filter alignment. * Added package icons. * Formatted instructions. * Allow anonymous package downloads. * Fixed comments. * Fixed postgres test. * Moved file. * Moved models to models/packages. * Use correct error response format per client. * Use simpler search form. * Fixed IsProd. * Restructured data model. * Prevent empty filename. * Fix swagger. * Implemented user/org registry. * Implemented UI. * Use GetUserByIDCtx. * Use table for dependencies. * make svg * Added support for unscoped npm packages. * Add support for npm dist tags. * Added tests for npm tags. * Unlink packages if repository gets deleted. * Prevent user/org delete if a packages exist. * Use package unlink in repository service. * Added support for composer packages. * Restructured package docs. * Added missing tests. * Fixed generic content page. * Fixed docs. * Fixed swagger. * Added missing type. * Fixed ambiguous column. * Organize content store by sha256 hash. * Added admin package management. * Added support for sorting. * Add support for multiple identical versions/files. * Added missing repository unlink. * Added file properties. * make fmt * lint * Added Conan package registry. * Updated docs. * Unify package names. * Added swagger enum. * Use longer TEXT column type. * Removed version composite key. * Merged package and container registry. * Removed index. * Use dedicated package router. * Moved files to new location. * Updated docs. * Fixed JOIN order. * Fixed GROUP BY statement. * Fixed GROUP BY #2. * Added symbol server support. * Added more tests. * Set NOT NULL. * Added setting to disable package registries. * Moved auth into service. * refactor * Use ctx everywhere. * Added package cleanup task. * Changed packages path. * Added container registry. * Refactoring * Updated comparison. * Fix swagger. * Fixed table order. * Use token auth for npm routes. * Enabled ReverseProxy auth. * Added packages link for orgs. * Fixed anonymous org access. * Enable copy button for setup instructions. * Merge error * Added suggestions. * Fixed merge. * Handle "generic". * Added link for TODO. * Added suggestions. * Changed temporary buffer filename. * Added suggestions. * Apply suggestions from code review Co-authored-by: Thomas Boerger <thomas@webhippie.de> * Update docs/content/doc/packages/nuget.en-us.md Co-authored-by: wxiaoguang <wxiaoguang@gmail.com> Co-authored-by: Thomas Boerger <thomas@webhippie.de>
* Cleanup protected branches when deleting users & teams (#19158)Norwin2022-03-221-0/+18
| | | | | | | | | | | * Clean up protected_branches when deleting user fixes #19094 * Clean up protected_branches when deleting teams * fix issue Co-authored-by: Lauris BH <lauris@nix.lv>
* remove not needed (#19128)65432022-03-182-69/+0
|
* use go1.18 to build gitea (#19099)techknowlogick2022-03-162-7/+57
| | | | | | | | | | | | | | | | | | | | | | | | | | | * use go1.18 to build gitea& update min go version to 1.17 * bump in a few more places * add a few simple tests for isipprivate * update go.mod * update URL to https://go.dev/dl/ * golangci-lint * attempt golangci-lint workaround * change version * bump fumpt version * skip strings.title test * go mod tidy * update tests as some aren't private?? * update tests
generated by cgit v1.2.3 (git 2.39.1) at 2025-01-08 14:45:39 +0000