aboutsummaryrefslogtreecommitdiffstats
path: root/modules/lfs
Commit message (Collapse)AuthorAgeFilesLines
* Removed some vestigial code related to Range bounds checks (#20312)Philip Peterson2022-07-281-16/+0
|
* Move some code into models/git (#19879)Lunny Xiao2022-06-121-1/+1
| | | | | | | | | | | | | | | | | | | * Move access and repo permission to models/perm/access * fix test * Move some git related files into sub package models/git * Fix build * fix git test * move lfs to sub package * move more git related functions to models/git * Move functions sequence * Some improvements per @KN4CK3R and @delvh
* Remove legacy `+build:` constraint (#19582)wxiaoguang2022-05-022-2/+0
| | | Go 1.17 and later use modern `//go:build` constraints, the old `// +build:` constraints should be removed.
* Update HTTP status codes to modern codes (#18063)KN4CK3R2022-03-231-1/+1
| | | | | | * 2xx/3xx/4xx/5xx -> http.Status... * http.StatusFound -> http.StatusTemporaryRedirect * http.StatusMovedPermanently -> http.StatusPermanentRedirect
* Improve SyncMirrors logging (#19045)zeripath2022-03-101-0/+13
| | | | | | | | | Yet another issue has come up where the logging from SyncMirrors does not provide enough context. This PR adds more context to these logging events. Related #19038 Signed-off-by: Andrew Thornton <art27@cantab.net>
* format with gofumpt (#18184)65432022-01-203-6/+4
| | | | | | | | | | | * gofumpt -w -l . * gofumpt -w -l -extra . * Add linter * manual fix * change make fmt
* Propagate context and ensure git commands run in request context (#17868)zeripath2022-01-191-4/+4
| | | | | | | | | This PR continues the work in #17125 by progressively ensuring that git commands run within the request context. This now means that the if there is a git repo already open in the context it will be used instead of reopening it. Signed-off-by: Andrew Thornton <art27@cantab.net>
* Fix source typos (#18227)luzpaz2022-01-101-2/+2
| | | Follow-up to #18219
* Use `hostmatcher` to replace `matchlist`, improve security (#17605)wxiaoguang2021-11-203-10/+13
| | | | | Use hostmacher to replace matchlist. And we introduce a better DialContext to do a full host/IP check, otherwise the attackers can still bypass the allow/block list by a 302 redirection.
* Fix some lints (#17337)Lunny Xiao2021-10-171-3/+1
| | | Fix some linting problems.
* refactor: move from io/ioutil to io and os package (#17109)Eng Zer Jun2021-09-222-8/+6
| | | | | | | | | The io/ioutil package has been deprecated as of Go 1.16, see https://golang.org/doc/go1.16#ioutil. This commit replaces the existing io/ioutil functions with their new definitions in io and os packages. Signed-off-by: Eng Zer Jun <engzerjun@gmail.com> Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Add bundle download for repository (#14538)John Olheiser2021-08-242-0/+2
| | | | | | | | | | | | | | | | | * Add bundle download Signed-off-by: jolheiser <john.olheiser@gmail.com> * Fix fmt Signed-off-by: jolheiser <john.olheiser@gmail.com> * Fix build tags Signed-off-by: jolheiser <john.olheiser@gmail.com> * Download specific commit Signed-off-by: jolheiser <john.olheiser@gmail.com>
* Add proxy settings and support for migration and webhook (#16704)Lunny Xiao2021-08-183-6/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | * Add proxy settings and support for migration and webhook * Fix default value * Add newline for example ini * Add lfs proxy support * Fix lint * Follow @zeripath's review * Fix git clone * Fix test * missgin http requests for proxy * use empty Co-authored-by: zeripath <art27@cantab.net> Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: zeripath <art27@cantab.net>
* Add an abstract json layout to make it's easier to change json library (#16528)Lunny Xiao2021-07-244-15/+15
| | | | | | | | | | | * Add an abstract json layout to make it's easier to change json library * Fix import * Fix import sequence * Fix blank lines * Fix blank lines
* Fix various documentation, user-facing, and source comment typos (#16367)luzpaz2021-07-081-1/+1
| | | | | * Fix various doc, user-facing, and source comment typos Found via `codespell -q 3 -S ./options/locale,./vendor -L ba,pullrequest,pullrequests,readby`
* Add push to remote mirror repository (#15157)KN4CK3R2021-06-148-138/+662
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Added push mirror model. * Integrated push mirror into queue. * Moved methods into own file. * Added basic implementation. * Mirror wiki too. * Removed duplicated method. * Get url for different remotes. * Added migration. * Unified remote url access. * Add/Remove push mirror remotes. * Prevent hangs with missing credentials. * Moved code between files. * Changed sanitizer interface. * Added push mirror backend methods. * Only update the mirror remote. * Limit refs on push. * Added UI part. * Added missing table. * Delete mirror if repository gets removed. * Changed signature. Handle object errors. * Added upload method. * Added "upload" unit tests. * Added transfer adapter unit tests. * Send correct headers. * Added pushing of LFS objects. * Added more logging. * Simpler body handling. * Process files in batches to reduce HTTP calls. * Added created timestamp. * Fixed invalid column name. * Changed name to prevent xorm auto setting. * Remove table header im empty. * Strip exit code from error message. * Added docs page about mirroring. * Fixed date. * Fixed merge errors. * Moved test to integrations. * Added push mirror test. * Added test.
* Fixed assert statements. (#16089)KN4CK3R2021-06-072-12/+12
|
* Rewrite of the LFS server (#15523)KN4CK3R2021-06-061-2/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Restructured code. Moved static checks out of loop. * Restructured batch api. Add support for individual errors. * Let router decide if LFS is enabled. * Renamed methods. * Return correct status from verify handler. * Unified media type check in router. * Changed error code according to spec. * Moved checks into router. * Removed invalid v1 api methods. * Unified methods. * Display better error messages. * Added size parameter. Create meta object on upload. * Use object error on invalid size. * Skip upload if object exists. * Moved methods. * Suppress fields in response. * Changed error on accept. * Added tests. * Use ErrorResponse object. * Test against message property. * Add support for the old invalid lfs client. * Fixed the check because MinIO wraps the error. * Use individual repositories. Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: Lauris BH <lauris@nix.lv>
* Fix lfs management find (#15537)Lunny Xiao2021-04-211-0/+2
| | | | | | | | | Fix #15236 * Do not do 40byte conversion within ParseTreeLine * Missed a to40ByteSHA Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: Andrew Thornton <art27@cantab.net>
* Add LFS Migration and Mirror (#14726)KN4CK3R2021-04-0818-1157/+1195
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Implemented LFS client. * Implemented scanning for pointer files. * Implemented downloading of lfs files. * Moved model-dependent code into services. * Removed models dependency. Added TryReadPointerFromBuffer. * Migrated code from service to module. * Centralised storage creation. * Removed dependency from models. * Moved ContentStore into modules. * Share structs between server and client. * Moved method to services. * Implemented lfs download on clone. * Implemented LFS sync on clone and mirror update. * Added form fields. * Updated templates. * Fixed condition. * Use alternate endpoint. * Added missing methods. * Fixed typo and make linter happy. * Detached pointer parser from gogit dependency. * Fixed TestGetLFSRange test. * Added context to support cancellation. * Use ReadFull to probably read more data. * Removed duplicated code from models. * Moved scan implementation into pointer_scanner_nogogit. * Changed method name. * Added comments. * Added more/specific log/error messages. * Embedded lfs.Pointer into models.LFSMetaObject. * Moved code from models to module. * Moved code from models to module. * Moved code from models to module. * Reduced pointer usage. * Embedded type. * Use promoted fields. * Fixed unexpected eof. * Added unit tests. * Implemented migration of local file paths. * Show an error on invalid LFS endpoints. * Hide settings if not used. * Added LFS info to mirror struct. * Fixed comment. * Check LFS endpoint. * Manage LFS settings from mirror page. * Fixed selector. * Adjusted selector. * Added more tests. * Added local filesystem migration test. * Fixed typo. * Reset settings. * Added special windows path handling. * Added unit test for HTTPClient. * Added unit test for BasicTransferAdapter. * Moved into util package. * Test if LFS endpoint is allowed. * Added support for git:// * Just use a static placeholder as the displayed url may be invalid. * Reverted to original code. * Added "Advanced Settings". * Updated wording. * Added discovery info link. * Implemented suggestion. * Fixed missing format parameter. * Added Pointer.IsValid(). * Always remove model on error. * Added suggestions. * Use channel instead of array. * Update routers/repo/migrate.go * fmt Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: zeripath <art27@cantab.net>
* Close file on invalid range (Addition to #15166) (#15268)KN4CK3R2021-04-063-29/+21
| | | | | | | | | | | | | | | * Close file on invalid range. * Close on seek error Signed-off-by: Andrew Thornton <art27@cantab.net> * Moved 'Seek' into server. * io.ReadSeekCloser is only available in Go 1.16 Co-authored-by: Andrew Thornton <art27@cantab.net> Co-authored-by: Lauris BH <lauris@nix.lv> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* [refactor] replace int with httpStatusCodes (#15282)65432021-04-051-23/+24
| | | | | | | | | | | * replace "200" (int) with "http.StatusOK" (const) * ctx.Error & ctx.HTML * ctx.JSON Part1 * ctx.JSON Part2 * ctx.JSON Part3
* Add size to Save function (#15264)zeripath2021-04-031-1/+1
| | | | | Fix #15253 Signed-off-by: Andrew Thornton <art27@cantab.net>
* Close file on invalid range (#15166)KN4CK3R2021-04-031-0/+8
| | | | | | | | | * Close file on invalid range. * Close on seek error Co-authored-by: Andrew Thornton <art27@cantab.net> Co-authored-by: Lauris BH <lauris@nix.lv> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Fix race in LFS ContentStore.Put(...) (#14895)zeripath2021-03-061-11/+51
| | | | | | | | | | | Continuing on from #14888 The previous implementation has race whereby an incomplete upload or hash mismatch upload can end up in the ContentStore. This PR moves the validation into the reader so that if there is a hash error or size mismatch the reader will return with an error instead of an io.EOF causing the storage to abort the storage. Signed-off-by: Andrew Thornton <art27@cantab.net>
* Migrate to use jsoniter instead of encoding/json (#14841)zeripath2021-03-012-2/+9
| | | | | | | | | | * Migrate to use jsoniter * fix tests * update gitea.com/go-chi/binding Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: 6543 <6543@obermui.de>
* Move macaron to chi (#14293)Lunny Xiao2021-01-262-9/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | Use [chi](https://github.com/go-chi/chi) instead of the forked [macaron](https://gitea.com/macaron/macaron). Since macaron and chi have conflicts with session share, this big PR becomes a have-to thing. According my previous idea, we can replace macaron step by step but I'm wrong. :( Below is a list of big changes on this PR. - [x] Define `context.ResponseWriter` interface with an implementation `context.Response`. - [x] Use chi instead of macaron, and also a customize `Route` to wrap chi so that the router usage is similar as before. - [x] Create different routers for `web`, `api`, `internal` and `install` so that the codes will be more clear and no magic . - [x] Use https://github.com/unrolled/render instead of macaron's internal render - [x] Use https://github.com/NYTimes/gziphandler instead of https://gitea.com/macaron/gzip - [x] Use https://gitea.com/go-chi/session which is a modified version of https://gitea.com/macaron/session and removed `nodb` support since it will not be maintained. **BREAK** - [x] Use https://gitea.com/go-chi/captcha which is a modified version of https://gitea.com/macaron/captcha - [x] Use https://gitea.com/go-chi/cache which is a modified version of https://gitea.com/macaron/cache - [x] Use https://gitea.com/go-chi/binding which is a modified version of https://gitea.com/macaron/binding - [x] Use https://github.com/go-chi/cors instead of https://gitea.com/macaron/cors - [x] Dropped https://gitea.com/macaron/i18n and make a new one in `code.gitea.io/gitea/modules/translation` - [x] Move validation form structs from `code.gitea.io/gitea/modules/auth` to `code.gitea.io/gitea/modules/forms` to avoid dependency cycle. - [x] Removed macaron log service because it's not need any more. **BREAK** - [x] All form structs have to be get by `web.GetForm(ctx)` in the route function but not as a function parameter on routes definition. - [x] Move Git HTTP protocol implementation to use routers directly. - [x] Fix the problem that chi routes don't support trailing slash but macaron did. - [x] `/api/v1/swagger` now will be redirect to `/api/swagger` but not render directly so that `APIContext` will not create a html render. Notices: - Chi router don't support request with trailing slash - Integration test `TestUserHeatmap` maybe mysql version related. It's failed on my macOS(mysql 5.7.29 installed via brew) but succeed on CI. Co-authored-by: 6543 <6543@obermui.de>
* Some code improvements (#14266)Lunny Xiao2021-01-061-3/+2
|
* Move LFSLock APIFormat into convert package (#13808)65432020-12-031-7/+8
|
* Storage configuration support `[storage]` (#13314)Lunny Xiao2020-10-312-3/+28
| | | | | | | | | | | * Fix minio bug * Add tests for storage configuration * Change the Seek flag to keep compitable minio? * Fix test when first-byte-pos of all ranges is greater than the resource length Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* LFS support to be stored on minio (#12518)Lunny Xiao2020-09-083-69/+62
| | | | | | | | | | | | | | | | | * LFS support to be stored on minio * Fix test * Fix lint * Fix lint * Fix check * Fix test * Update documents and add migration for LFS * Fix some bugs
* Add Access-Control-Expose-Headers (#12446)zeripath2020-08-131-0/+2
| | | | | | Fix #12424 Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: silverwind <me@silverwind.io>
* Re-attempt to delete temporary upload if the file is locked by another ↵zeripath2020-08-111-1/+6
| | | | | | | | | | | process (#12447) Replace all calls to os.Remove/os.RemoveAll by retrying util.Remove/util.RemoveAll and remove circular dependencies from util. Fix #12339 Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: silverwind <me@silverwind.io>
* Support Range header end in lfs (#11314)burbon2020-05-111-5/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Initial support of end Range header option in lfs * Allow end range option to be unspecified * Declare toByte for consistency * Factor out content encoding tests from doLfs This is so Range tests could still use doLfs but without repeating not related tests * Add Range header test * implemented extraHeader * parametrized expectedStatus * Add more test cases of Range header * Fix S1030: should use resp.Body.String() * Add more tests for edge cases * Fix tests Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: zeripath <art27@cantab.net>
* Add gitea-vet (#10948)John Olheiser2020-04-052-0/+8
| | | | | | | | | | | | | | | | | * Add copyright Signed-off-by: jolheiser <john.olheiser@gmail.com> * Add gitea-vet and fix non-compliance Signed-off-by: jolheiser <john.olheiser@gmail.com> * Combine tools.go into build.go and clean up Signed-off-by: jolheiser <john.olheiser@gmail.com> * Remove extra GO111MODULE=on Signed-off-by: jolheiser <john.olheiser@gmail.com>
* Multiple LFS improvements (#10667)zeripath2020-03-093-46/+149
| | | | | | | | | | | | | | * Add more logging in the LFS server Adds more logging in the LFS server and stops sending internal server error information to the client * Add LFS Lock cursor implementation * Simplify Claims in LFS and remove the float64 casts Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: Lauris BH <lauris@nix.lv>
* Fix LFS max file size enforcement (#10593)zeripath2020-03-031-0/+6
| | | | | Unfortunately the batch handler also needs to enforce max file size. Enforce max file size in batch handler
* Add max-file-size to LFS (#10463)zeripath2020-02-281-0/+6
| | | | | | | | | | | | | | | * Add max-file-size to LFS * Update modules/lfs/server.go * As per @silverwind Co-Authored-By: silverwind <me@silverwind.io> Co-authored-by: silverwind <me@silverwind.io> Co-authored-by: Lauris BH <lauris@nix.lv> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com> Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
* Make repository management section handle lfs locks (#8726)zeripath2019-12-121-2/+2
| | | | | | | | | | | | | | | | | | | | * Make repository maangement section handle lfs locks * Add check attribute handling and handle locking paths better * More cleanly check-attributes * handle error * Check if file exists in default branch before linking to it. * fixup * Properly cleanPath * Use cleanPath * Sigh
* Add basic repository lfs management (#7199)zeripath2019-10-281-1/+1
| | | | | | | | | | | | This PR adds basic repository LFS management UI including the ability to find all possible pointers within the repository. Locks are not managed at present but would be addable through some simple additions. * Add basic repository lfs management * add auto-associate function * Add functionality to find commits with this lfs file * Add link to find commits on the lfs file view * Adjust commit view to state the likely branch causing the commit * Only read Oid from database
* Ensure Request Body Readers are closed in LFS server (#8454)zeripath2019-10-102-5/+15
|
* Use gitea forked macaron (#7933)Tamal Saha2019-08-231-1/+1
| | | Signed-off-by: Tamal Saha <tamal@appscode.com>
* Removed unnecessary conversions (#7557)Christian Muehlhaeuser2019-07-231-1/+1
| | | No need to convert to the same type.
* Add golangci (#6418)kolaente2019-06-122-18/+11
|
* Fix LFS Locks over SSH (#6999)zeripath2019-05-281-32/+83
| | | | | * Fix LFS Locks over SSH * Mark test as skipped
* Fix /verify LFS handler expecting wrong content-type (#7015)Marat Radchenko2019-05-241-18/+15
| | | | | | | | | | | | Fixes #6960 According to [spec][1], /verify requests must have `Accept: application/vnd.git-lfs+json` Previous code works because `git-lfs` also [violates spec and doesn't send any Accept header at all][2] For other clients that DO set `Accept: application/vnd.git-lfs+json`, addition of `Accept: application/vnd.git-lfs` either forces them to violate the spec or is ignored, depending on order in what they create header list. [1]: https://github.com/git-lfs/git-lfs/blob/master/docs/api/basic-transfers.md#verification [2]: https://github.com/git-lfs/git-lfs/issues/3662
* Move sdk structs to modules/structs (#6905)Lunny Xiao2019-05-111-1/+1
| | | | | | | | | | | | * move sdk structs to moduels/structs * fix tests * fix fmt * fix swagger * fix vendor
* Better logging (#6038) (#6095)zeripath2019-04-021-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Panic don't fatal on create new logger Fixes #5854 Signed-off-by: Andrew Thornton <art27@cantab.net> * partial broken * Update the logging infrastrcture Signed-off-by: Andrew Thornton <art27@cantab.net> * Reset the skip levels for Fatal and Error Signed-off-by: Andrew Thornton <art27@cantab.net> * broken ncsa * More log.Error fixes Signed-off-by: Andrew Thornton <art27@cantab.net> * Remove nal * set log-levels to lowercase * Make console_test test all levels * switch to lowercased levels * OK now working * Fix vetting issues * Fix lint * Fix tests * change default logging to match current gitea * Improve log testing Signed-off-by: Andrew Thornton <art27@cantab.net> * reset error skip levels to 0 * Update documentation and access logger configuration * Redirect the router log back to gitea if redirect macaron log but also allow setting the log level - i.e. TRACE * Fix broken level caching * Refactor the router log * Add Router logger * Add colorizing options * Adjust router colors * Only create logger if they will be used * update app.ini.sample * rename Attribute ColorAttribute * Change from white to green for function * Set fatal/error levels * Restore initial trace logger * Fix Trace arguments in modules/auth/auth.go * Properly handle XORMLogger * Improve admin/config page * fix fmt * Add auto-compression of old logs * Update error log levels * Remove the unnecessary skip argument from Error, Fatal and Critical * Add stacktrace support * Fix tests * Remove x/sync from vendors? * Add stderr option to console logger * Use filepath.ToSlash to protect against Windows in tests * Remove prefixed underscores from names in colors.go * Remove not implemented database logger This was removed from Gogs on 4 Mar 2016 but left in the configuration since then. * Ensure that log paths are relative to ROOT_PATH * use path.Join * rename jsonConfig to logConfig * Rename "config" to "jsonConfig" to make it clearer * Requested changes * Requested changes: XormLogger * Try to color the windows terminal If successful default to colorizing the console logs * fixup * Colorize initially too * update vendor * Colorize logs on default and remove if this is not a colorizing logger * Fix documentation * fix test * Use go-isatty to detect if on windows we are on msys or cygwin * Fix spelling mistake * Add missing vendors * More changes * Rationalise the ANSI writer protection * Adjust colors on advice from @0x5c * Make Flags a comma separated list * Move to use the windows constant for ENABLE_VIRTUAL_TERMINAL_PROCESSING * Ensure matching is done on the non-colored message - to simpify EXPRESSION
* Allow markdown files to read from the LFS (#5787)zeripath2019-02-121-0/+69
| | | | | | | This PR makes it possible for the markdown renderer to render images and media straight from the LFS. Fix #5746 Signed-off-by: Andrew Thornton [art27@cantab.net](mailto:art27@cantab.net)
* Fix bug when read public repo lfs file (#5912)Lunny Xiao2019-01-311-2/+5
| | | | | | * fix bug when read public repo lfs file * add comment on lfs permission check
generated by cgit v1.2.3 (git 2.39.1) at 2025-01-01 14:52:20 +0000