summaryrefslogtreecommitdiffstats
path: root/modules/indexer/code
Commit message (Collapse)AuthorAgeFilesLines
* Fix data race in bleve indexer (#16474) (#16509)65432021-07-221-4/+6
| | | | | * Fix data race in bleve indexer Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Fix various documentation, user-facing, and source comment typos (#16367)luzpaz2021-07-081-3/+3
| | | | | * Fix various doc, user-facing, and source comment typos Found via `codespell -q 3 -S ./options/locale,./vendor -L ba,pullrequest,pullrequests,readby`
* Use git log name-status in get last commit (#16059)zeripath2021-06-212-0/+6
| | | | | | | | | | | | | | | | | * Improve get last commit using git log --name-status git log --name-status -c provides information about the diff between a commit and its parents. Using this and adjusting the algorithm to use the first change to a path allows for a much faster generation of commit info. There is a subtle change in the results generated but this will cause the results to more closely match those from elsewhere. Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: techknowlogick <techknowlogick@gitea.io> Co-authored-by: Lauris BH <lauris@nix.lv>
* Fixed assert statements. (#16089)KN4CK3R2021-06-071-1/+1
|
* Add Image Diff for SVG files (#14867)KN4CK3R2021-06-052-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | * Added type sniffer. * Switched content detection from base to typesniffer. * Added GuessContentType to Blob. * Moved image info logic to client. Added support for SVG images in diff. * Restore old blocked svg behaviour. * Added missing image formats. * Execute image diff only when container is visible. * add margin to spinner * improve BIN tag on image diffs * Default to render view. * Show image diff on incomplete diff. Co-authored-by: silverwind <me@silverwind.io> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com> Co-authored-by: Lauris BH <lauris@nix.lv>
* Multiple Queue improvements: LevelDB Wait on empty, shutdown empty shadow ↵zeripath2021-05-151-1/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | level queue, reduce goroutines etc (#15693) * move shutdownfns, terminatefns and hammerfns out of separate goroutines Coalesce the shutdownfns etc into a list of functions that get run at shutdown rather then have them run at goroutines blocked on selects. This may help reduce the background select/poll load in certain configurations. * The LevelDB queues can actually wait on empty instead of polling Slight refactor to cause leveldb queues to wait on empty instead of polling. * Shutdown the shadow level queue once it is empty * Remove bytefifo additional goroutine for readToChan as it can just be run in run * Remove additional removeWorkers goroutine for workers * Simplify the AtShutdown and AtTerminate functions and add Channel Flusher * Add shutdown flusher to CUQ * move persistable channel shutdown stuff to Shutdown Fn * Ensure that UPCQ has the correct config * handle shutdown during the flushing * reduce risk of race between zeroBoost and addWorkers * prevent double shutdown Signed-off-by: Andrew Thornton <art27@cantab.net>
* On open repository open common cat file batch and batch-check (#15667)zeripath2021-05-102-2/+2
| | | | | | | Use common git cat-file --batch and git cat-file --batch-check to significantly reduce calls to git. Signed-off-by: Andrew Thornton <art27@cantab.net>
* Speed up `enry.IsVendor` (#15213)zeripath2021-04-012-2/+2
| | | | | | | | | `enry.IsVendor` is kinda slow as it simply iterates across all regexps. This PR ajdusts the regexps to combine them to make this process a little quicker. Related #15143 Signed-off-by: Andrew Thornton <art27@cantab.net>
* Move Bleve and Elastic code indexers to use a common cat-file --batch (#14781)zeripath2021-03-032-16/+52
| | | | | | | | | | | | | | | | * Extract out the common cat-file batch calls Signed-off-by: Andrew Thornton <art27@cantab.net> * Move bleve and elastic indexers to use a common cat-file --batch when indexing Signed-off-by: Andrew Thornton <art27@cantab.net> * move catfilebatch to batch_reader and rename to batch_reader.go Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: Lauris BH <lauris@nix.lv>
* Migrate to use jsoniter instead of encoding/json (#14841)zeripath2021-03-011-1/+2
| | | | | | | | | | * Migrate to use jsoniter * fix tests * update gitea.com/go-chi/binding Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: 6543 <6543@obermui.de>
* Reduce calls to git cat-file -s (#14682)zeripath2021-02-173-16/+32
| | | | | | | | | | | | | * Reduce calls to git cat-file -s There are multiple places where there are repeated calls to git cat-file -s due to the blobs not being created with their size. Through judicious use of git ls-tree -l and slight adjustments to the indexer code we can avoid a lot of these calls. * simplify by always expecting the long format * Also always set the sized field and tell the indexer the update is sized
* [Feature] add precise search type for Elastic Search (#12869)Jui-Nan Lin2021-01-276-15/+35
| | | | | | | | * feat: add type query parameters for specifying precise search * feat: add select dropdown in search box Co-authored-by: Lauris BH <lauris@nix.lv> Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Upgrade blevesearch dependency to v2.0.1 (#14346)Lauris BH2021-01-171-9/+9
| | | | | | | * Upgrade blevesearch dependency to v2.0.1 * Update rupture to v1.0.0 * Fix test
* Refactor Logger (#13294)zeripath2020-10-311-1/+1
| | | | | | Refactor Logger to make a logger interface and make it possible to wrap loggers for specific purposes. Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* fix: use Base36 for all code indexers (#12830)Jui-Nan Lin2020-09-142-6/+5
|
* Fix bug on elastic search (#12811)Lunny Xiao2020-09-122-15/+42
| | | | | | | | | | | * Fix bug on elastic search * Add more comments for elastic search result startIndex and endIndex * refactor indexPos * refactor indexPos * Fix bug
* Add queue for code indexer (#10332)Lunny Xiao2020-09-073-159/+158
| | | | | | | | | | | | | | | | | | | | | * Add queue for code indexer * Fix lint * Fix test * Fix lint * Fix bug * Fix bug * Fix lint * Add noqueue * Fix tests * Rename noqueue to immediate
* Support elastic search for code search (#10273)Lunny Xiao2020-08-308-157/+658
| | | | | | | | | | | | | | | | | | | | | | | * Support elastic search for code search * Finished elastic search implementation and add some tests * Enable test on drone and added docs * Add new fields to elastic search * Fix bug * remove unused changes * Use indexer alias to keep the gitea indexer version * Improve codes * Some code improvements * The real indexer name changed to xxx.v1 Co-authored-by: zeripath <art27@cantab.net>
* Re-attempt to delete temporary upload if the file is locked by another ↵zeripath2020-08-112-4/+5
| | | | | | | | | | | process (#12447) Replace all calls to os.Remove/os.RemoveAll by retrying util.Remove/util.RemoveAll and remove circular dependencies from util. Fix #12339 Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: silverwind <me@silverwind.io>
* Server-side syntax highlighting for all code (#12047)mrsdizzie2020-07-011-16/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Server-side syntax hilighting for all code This PR does a few things: * Remove all traces of highlight.js * Use chroma library to provide fast syntax hilighting directly on the server * Provide syntax hilighting for diffs * Re-style both unified and split diffs views * Add custom syntax hilighting styling for both regular and arc-green Fixes #7729 Fixes #10157 Fixes #11825 Fixes #7728 Fixes #3872 Fixes #3682 And perhaps gets closer to #9553 * fix line marker * fix repo search * Fix single line select * properly load settings * npm uninstall highlight.js * review suggestion * code review * forgot to call function * fix test * Apply suggestions from code review suggestions from @silverwind thanks Co-authored-by: silverwind <me@silverwind.io> * code review * copy/paste error * Use const for highlight size limit * Update web_src/less/_repository.less Co-authored-by: Lauris BH <lauris@nix.lv> * update size limit to 1MB and other styling tweaks * fix highlighting for certain diff sections * fix test * add worker back as suggested Co-authored-by: silverwind <me@silverwind.io> Co-authored-by: Lauris BH <lauris@nix.lv>
* Log the indexer path on failure (#11172)zeripath2020-04-221-1/+1
| | | | | | Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com> Co-authored-by: Lauris BH <lauris@nix.lv>
* Migrate to go-enry new version (#10906)Lauris BH2020-04-151-1/+1
|
* Handle panic in indexer initialisation better (#10534)zeripath2020-02-281-0/+11
| | | | | * Handle panic in indexer initialisation better * as per @guillep2k
* Add detected file language to code search (#10256)Lauris BH2020-02-205-34/+140
| | | | | | | Move langauge detection to separate module to be more reusable Add option to disable vendored file exclusion from file search Allways show all language stats for search
* Language statistics bar for repositories (#8037)Lauris BH2020-02-113-5/+6
| | | | | | | | | | | | | | | | | | * Implementation for calculating language statistics Impement saving code language statistics to database Implement rendering langauge stats Add primary laguage to show in repository list Implement repository stats indexer queue Add indexer test Refactor to use queue module * Do not timeout for queues
* Fix file rename/copy not supported by indexer (#9965)guillep2k2020-01-241-2/+27
| | | | Co-authored-by: zeripath <art27@cantab.net>
* Stop various tests from adding to the source tree (#9515)zeripath2019-12-281-5/+14
| | | | | | | Instead of just adding test generated files to .gitignore prevent them from being produced in the first place. Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* fix datarace on issue indexer queue (#9490)Lunny Xiao2019-12-252-1/+6
|
* Wrap the code indexer (#9476)zeripath2019-12-243-13/+132
| | | | | | | | | | | | | | | | | | | | * Wrap the code indexer In order to prevent a data race in the code indexer it must be wrapped with a holder otherwise it is possible to Search/Index on an incompletely initialised indexer, and search will fail with a nil pointer until the repository indexer is initialised. Further a completely initialised repository indexer should not be closed until Termination otherwise actions in Hammer/Shutdown phases could block or be lost. Finally, there is a complex dance of shutdown etiquette should the index initialisation fail. This PR restores that. * Always return err if closed whilst waiting Co-authored-by: techknowlogick <matti@mdranta.net>
* Refactor code indexer (#9313)Lunny Xiao2019-12-237-625/+765
| | | | | | | | | | | | | | | | | | | | * Refactor code indexer * fix test * fix test * refactor code indexer * fix import * improve code * fix typo * fix test and make code clean * fix lint
* Graceful: Xorm, RepoIndexer, Cron and Others (#9282)zeripath2019-12-152-34/+83
| | | | | | | | | | | | * Change graceful to use a singleton obtained through GetManager instead of a global. * Graceful: Make TestPullRequests shutdownable * Graceful: Make the cron tasks graceful * Graceful: AddTestPullRequest run in graceful ctx * Graceful: SyncMirrors shutdown * Graceful: SetDefaultContext for Xorm to be HammerContext * Avoid starting graceful for migrate commands and checkout * Graceful: DeliverHooks now can be shutdown * Fix multiple syncing errors in modules/sync/UniqueQueue & Make UniqueQueue closable * Begin the process of making the repo indexer shutdown gracefully
* DefaultBranch needs to be prefixed by BranchPrefix (#9356)zeripath2019-12-141-1/+1
|
* Move modules/indexer to modules/indexer/code (#9301)Lunny Xiao2019-12-103-12/+344
|
* Move code indexer related code to a new package (#9191)Lunny Xiao2019-12-082-0/+381
* move code indexer related code to a new package * fix lint * fix tests * fix fmt * GetMaxID support interface parameter
generated by cgit v1.2.3 (git 2.39.1) at 2025-08-10 11:08:40 +0000