summaryrefslogtreecommitdiffstats
path: root/modules/lfs/content_store.go
Commit message (Collapse)AuthorAgeFilesLines
* Use `crypto/sha256` (#29386)KN4CK3R2024-02-251-2/+1
| | | | | | | | Go 1.21 improved the performance of `crypto/sha256`. It's now similar to `minio/sha256-simd`, so we should just use the standard libs. https://go.dev/doc/go1.21#crypto/sha256 https://go-review.googlesource.com/c/go/+/408795 https://github.com/multiformats/go-multihash/pull/173
* Make repository response support HTTP range request (#24592)wxiaoguang2023-05-091-3/+3
| | | | | | Replace #20480 Replace #18448 Close #16414
* Make minio package support legacy MD5 checksum (#23768)wxiaoguang2023-03-281-11/+29
| | | | | | | | | | | | | | | | | | | | | | | | | | | A feedback from discord: https://discord.com/channels/322538954119184384/561007778139734027/1090185427115319386 Some storages like: * https://developers.cloudflare.com/r2/api/s3/api/ * https://www.backblaze.com/b2/docs/s3_compatible_api.html They do not support "x-amz-checksum-algorithm" header But minio recently uses that header with CRC32C by default. So we have to tell minio to use legacy MD5 checksum. I guess this needs to be backported because IIRC we 1.19 and 1.20 are using similar minio package. The minio package code for SendContentMD5 looks like this: <details> <img width="755" alt="image" src="https://user-images.githubusercontent.com/2114189/228186768-4f2f6f67-62b9-4aee-9251-5af714ad9674.png"> </details>
* Use minio/sha256-simd for accelerated SHA256 (#23052)zeripath2023-02-221-1/+2
| | | | | | | | | | | | minio/sha256-simd provides additional acceleration for SHA256 using AVX512, SHA Extensions for x86 and ARM64 for ARM. It provides a drop-in replacement for crypto/sha256 and if the extensions are not available it falls back to standard crypto/sha256. --------- Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: John Olheiser <john.olheiser@gmail.com>
* Implement FSFE REUSE for golang files (#21840)flynnnnnnnnnn2022-11-271-2/+1
| | | | | | | | | Change all license headers to comply with REUSE specification. Fix #16132 Co-authored-by: flynnnnnnnnnn <flynnnnnnnnnn@github> Co-authored-by: John Olheiser <john.olheiser@gmail.com>
* Removed some vestigial code related to Range bounds checks (#20312)Philip Peterson2022-07-281-16/+0
|
* Move some code into models/git (#19879)Lunny Xiao2022-06-121-1/+1
| | | | | | | | | | | | | | | | | | | * Move access and repo permission to models/perm/access * fix test * Move some git related files into sub package models/git * Fix build * fix git test * move lfs to sub package * move more git related functions to models/git * Move functions sequence * Some improvements per @KN4CK3R and @delvh
* Add LFS Migration and Mirror (#14726)KN4CK3R2021-04-081-26/+39
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Implemented LFS client. * Implemented scanning for pointer files. * Implemented downloading of lfs files. * Moved model-dependent code into services. * Removed models dependency. Added TryReadPointerFromBuffer. * Migrated code from service to module. * Centralised storage creation. * Removed dependency from models. * Moved ContentStore into modules. * Share structs between server and client. * Moved method to services. * Implemented lfs download on clone. * Implemented LFS sync on clone and mirror update. * Added form fields. * Updated templates. * Fixed condition. * Use alternate endpoint. * Added missing methods. * Fixed typo and make linter happy. * Detached pointer parser from gogit dependency. * Fixed TestGetLFSRange test. * Added context to support cancellation. * Use ReadFull to probably read more data. * Removed duplicated code from models. * Moved scan implementation into pointer_scanner_nogogit. * Changed method name. * Added comments. * Added more/specific log/error messages. * Embedded lfs.Pointer into models.LFSMetaObject. * Moved code from models to module. * Moved code from models to module. * Moved code from models to module. * Reduced pointer usage. * Embedded type. * Use promoted fields. * Fixed unexpected eof. * Added unit tests. * Implemented migration of local file paths. * Show an error on invalid LFS endpoints. * Hide settings if not used. * Added LFS info to mirror struct. * Fixed comment. * Check LFS endpoint. * Manage LFS settings from mirror page. * Fixed selector. * Adjusted selector. * Added more tests. * Added local filesystem migration test. * Fixed typo. * Reset settings. * Added special windows path handling. * Added unit test for HTTPClient. * Added unit test for BasicTransferAdapter. * Moved into util package. * Test if LFS endpoint is allowed. * Added support for git:// * Just use a static placeholder as the displayed url may be invalid. * Reverted to original code. * Added "Advanced Settings". * Updated wording. * Added discovery info link. * Implemented suggestion. * Fixed missing format parameter. * Added Pointer.IsValid(). * Always remove model on error. * Added suggestions. * Use channel instead of array. * Update routers/repo/migrate.go * fmt Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: zeripath <art27@cantab.net>
* Close file on invalid range (Addition to #15166) (#15268)KN4CK3R2021-04-061-21/+2
| | | | | | | | | | | | | | | * Close file on invalid range. * Close on seek error Signed-off-by: Andrew Thornton <art27@cantab.net> * Moved 'Seek' into server. * io.ReadSeekCloser is only available in Go 1.16 Co-authored-by: Andrew Thornton <art27@cantab.net> Co-authored-by: Lauris BH <lauris@nix.lv> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Add size to Save function (#15264)zeripath2021-04-031-1/+1
| | | | | Fix #15253 Signed-off-by: Andrew Thornton <art27@cantab.net>
* Close file on invalid range (#15166)KN4CK3R2021-04-031-0/+8
| | | | | | | | | * Close file on invalid range. * Close on seek error Co-authored-by: Andrew Thornton <art27@cantab.net> Co-authored-by: Lauris BH <lauris@nix.lv> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Fix race in LFS ContentStore.Put(...) (#14895)zeripath2021-03-061-11/+51
| | | | | | | | | | | Continuing on from #14888 The previous implementation has race whereby an incomplete upload or hash mismatch upload can end up in the ContentStore. This PR moves the validation into the reader so that if there is a hash error or size mismatch the reader will return with an error instead of an io.EOF causing the storage to abort the storage. Signed-off-by: Andrew Thornton <art27@cantab.net>
* Storage configuration support `[storage]` (#13314)Lunny Xiao2020-10-311-1/+22
| | | | | | | | | | | * Fix minio bug * Add tests for storage configuration * Change the Seek flag to keep compitable minio? * Fix test when first-byte-pos of all ranges is greater than the resource length Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* LFS support to be stored on minio (#12518)Lunny Xiao2020-09-081-56/+26
| | | | | | | | | | | | | | | | | * LFS support to be stored on minio * Fix test * Fix lint * Fix lint * Fix check * Fix test * Update documents and add migration for LFS * Fix some bugs
* Re-attempt to delete temporary upload if the file is locked by another ↵zeripath2020-08-111-1/+6
| | | | | | | | | | | process (#12447) Replace all calls to os.Remove/os.RemoveAll by retrying util.Remove/util.RemoveAll and remove circular dependencies from util. Fix #12339 Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: silverwind <me@silverwind.io>
* Add gitea-vet (#10948)John Olheiser2020-04-051-0/+4
| | | | | | | | | | | | | | | | | * Add copyright Signed-off-by: jolheiser <john.olheiser@gmail.com> * Add gitea-vet and fix non-compliance Signed-off-by: jolheiser <john.olheiser@gmail.com> * Combine tools.go into build.go and clean up Signed-off-by: jolheiser <john.olheiser@gmail.com> * Remove extra GO111MODULE=on Signed-off-by: jolheiser <john.olheiser@gmail.com>
* Multiple LFS improvements (#10667)zeripath2020-03-091-1/+15
| | | | | | | | | | | | | | * Add more logging in the LFS server Adds more logging in the LFS server and stops sending internal server error information to the client * Add LFS Lock cursor implementation * Simplify Claims in LFS and remove the float64 casts Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: Lauris BH <lauris@nix.lv>
* Add LFS object verification step after upload (#2868)Lauris BH2017-11-081-1/+16
| | | | | | | | | | | | * Add LFS object verification step after upload * Fix file verification condition and small refactor * Fix URLs * Remove newline and return status 422 on failed verification * Better error hadling
* Fix lint errors (#2547)Ethan Koenig2017-09-191-4/+1
|
* Fix typos in models/ and modules/ (#1248)Ethan Koenig2017-03-151-1/+1
|
* fix gofmt errorBo-Yi Wu2016-12-301-1/+1
| | | | Signed-off-by: Bo-Yi Wu <appleboy.tw@gmail.com>
* Git LFS support v2 (#122)Fabian Zaremba2016-12-261-0/+94
* Import github.com/git-lfs/lfs-test-server as lfs module base Imported commit is 3968aac269a77b73924649b9412ae03f7ccd3198 Removed: Dockerfile CONTRIBUTING.md mgmt* script/ vendor/ kvlogger.go .dockerignore .gitignore README.md * Remove config, add JWT support from github.com/mgit-at/lfs-test-server Imported commit f0cdcc5a01599c5a955dc1bbf683bb4acecdba83 * Add LFS settings * Add LFS meta object model * Add LFS routes and initialization * Import github.com/dgrijalva/jwt-go into vendor/ * Adapt LFS module: handlers, routing, meta store * Move LFS routes to /user/repo/info/lfs/* * Add request header checks to LFS BatchHandler / PostHandler * Implement LFS basic authentication * Rework JWT secret generation / load * Implement LFS SSH token authentication with JWT Specification: https://github.com/github/git-lfs/tree/master/docs/api * Integrate LFS settings into install process * Remove LFS objects when repository is deleted Only removes objects from content store when deleted repo is the only referencing repository * Make LFS module stateless Fixes bug where LFS would not work after installation without restarting Gitea * Change 500 'Internal Server Error' to 400 'Bad Request' * Change sql query to xorm call * Remove unneeded type from LFS module * Change internal imports to code.gitea.io/gitea/ * Add Gitea authors copyright * Change basic auth realm to "gitea-lfs" * Add unique indexes to LFS model * Use xorm count function in LFS check on repository delete * Return io.ReadCloser from content store and close after usage * Add LFS info to runWeb() * Export LFS content store base path * LFS file download from UI * Work around git-lfs client issue with unauthenticated requests Returning a dummy Authorization header for unauthenticated requests lets git-lfs client skip asking for auth credentials See: https://github.com/github/git-lfs/issues/1088 * Fix unauthenticated UI downloads from public repositories * Authentication check order, Finish LFS file view logic * Ignore LFS hooks if installed for current OS user Fixes Gitea UI actions for repositories tracking LFS files. Checks for minimum needed git version by parsing the semantic version string. * Hide LFS metafile diff from commit view, marking as binary * Show LFS notice if file in commit view is tracked * Add notbefore/nbf JWT claim * Correct lint suggestions - comments for structs and functions - Add comments to LFS model - Function comment for GetRandomBytesAsBase64 - LFS server function comments and lint variable suggestion * Move secret generation code out of conditional Ensures no LFS code may run with an empty secret * Do not hand out JWT tokens if LFS server support is disabled
generated by cgit v1.2.3 (git 2.39.1) at 2025-01-12 02:51:41 +0000