Use minio/sha256-simd for accelerated SHA256 (#23052)
minio/sha256-simd provides additional acceleration for SHA256 using
AVX512, SHA Extensions for x86 and ARM64 for ARM.
It provides a drop-in replacement for crypto/sha256 and if the
extensions are not available it falls back to standard crypto/sha256.
---------
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: John Olheiser <john.olheiser@gmail.com>
Change all license headers to comply with REUSE specification.
Fix #16132
Co-authored-by: flynnnnnnnnnn <flynnnnnnnnnn@github>
Co-authored-by: John Olheiser <john.olheiser@gmail.com>
* Move access and repo permission to models/perm/access
* fix test
* Move some git related files into sub package models/git
* Fix build
* fix git test
* move lfs to sub package
* move more git related functions to models/git
* Move functions sequence
* Some improvements per @KN4CK3R and @delvh
* Implemented LFS client.
* Implemented scanning for pointer files.
* Implemented downloading of lfs files.
* Moved model-dependent code into services.
* Removed models dependency. Added TryReadPointerFromBuffer.
* Migrated code from service to module.
* Centralised storage creation.
* Removed dependency from models.
* Moved ContentStore into modules.
* Share structs between server and client.
* Moved method to services.
* Implemented lfs download on clone.
* Implemented LFS sync on clone and mirror update.
* Added form fields.
* Updated templates.
* Fixed condition.
* Use alternate endpoint.
* Added missing methods.
* Fixed typo and make linter happy.
* Detached pointer parser from gogit dependency.
* Fixed TestGetLFSRange test.
* Added context to support cancellation.
* Use ReadFull to probably read more data.
* Removed duplicated code from models.
* Moved scan implementation into pointer_scanner_nogogit.
* Changed method name.
* Added comments.
* Added more/specific log/error messages.
* Embedded lfs.Pointer into models.LFSMetaObject.
* Moved code from models to module.
* Moved code from models to module.
* Moved code from models to module.
* Reduced pointer usage.
* Embedded type.
* Use promoted fields.
* Fixed unexpected eof.
* Added unit tests.
* Implemented migration of local file paths.
* Show an error on invalid LFS endpoints.
* Hide settings if not used.
* Added LFS info to mirror struct.
* Fixed comment.
* Check LFS endpoint.
* Manage LFS settings from mirror page.
* Fixed selector.
* Adjusted selector.
* Added more tests.
* Added local filesystem migration test.
* Fixed typo.
* Reset settings.
* Added special windows path handling.
* Added unit test for HTTPClient.
* Added unit test for BasicTransferAdapter.
* Moved into util package.
* Test if LFS endpoint is allowed.
* Added support for git://
* Just use a static placeholder as the displayed url may be invalid.
* Reverted to original code.
* Added "Advanced Settings".
* Updated wording.
* Added discovery info link.
* Implemented suggestion.
* Fixed missing format parameter.
* Added Pointer.IsValid().
* Always remove model on error.
* Added suggestions.
* Use channel instead of array.
* Update routers/repo/migrate.go
* fmt
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: zeripath <art27@cantab.net>
Close file on invalid range (Addition to #15166) (#15268)
* Close file on invalid range.
* Close on seek error
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Moved 'Seek' into server.
* io.ReadSeekCloser is only available in Go 1.16
Co-authored-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Close file on invalid range.
* Close on seek error
Co-authored-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Continuing on from #14888
The previous implementation has race whereby an incomplete upload or
hash mismatch upload can end up in the ContentStore. This PR moves the
validation into the reader so that if there is a hash error or size
mismatch the reader will return with an error instead of an io.EOF
causing the storage to abort the storage.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Storage configuration support `[storage]` (#13314)
* Fix minio bug
* Add tests for storage configuration
* Change the Seek flag to keep compitable minio?
* Fix test when first-byte-pos of all ranges is greater than the resource length
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* LFS support to be stored on minio
* Fix test
* Fix lint
* Fix lint
* Fix check
* Fix test
* Update documents and add migration for LFS
* Fix some bugs
Re-attempt to delete temporary upload if the file is locked by another process (#12447)
Replace all calls to os.Remove/os.RemoveAll by retrying util.Remove/util.RemoveAll and remove circular dependencies from util.
Fix #12339
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: silverwind <me@silverwind.io>
* Add more logging in the LFS server
Adds more logging in the LFS server and stops sending internal server
error information to the client
* Add LFS Lock cursor implementation
* Simplify Claims in LFS and remove the float64 casts
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
* Import github.com/git-lfs/lfs-test-server as lfs module base
Imported commit is 3968aac269
Removed:
Dockerfile CONTRIBUTING.md mgmt* script/ vendor/ kvlogger.go
.dockerignore .gitignore README.md
* Remove config, add JWT support from github.com/mgit-at/lfs-test-server
Imported commit f0cdcc5a01
* Add LFS settings
* Add LFS meta object model
* Add LFS routes and initialization
* Import github.com/dgrijalva/jwt-go into vendor/
* Adapt LFS module: handlers, routing, meta store
* Move LFS routes to /user/repo/info/lfs/*
* Add request header checks to LFS BatchHandler / PostHandler
* Implement LFS basic authentication
* Rework JWT secret generation / load
* Implement LFS SSH token authentication with JWT
Specification: https://github.com/github/git-lfs/tree/master/docs/api
* Integrate LFS settings into install process
* Remove LFS objects when repository is deleted
Only removes objects from content store when deleted repo is the only
referencing repository
* Make LFS module stateless
Fixes bug where LFS would not work after installation without
restarting Gitea
* Change 500 'Internal Server Error' to 400 'Bad Request'
* Change sql query to xorm call
* Remove unneeded type from LFS module
* Change internal imports to code.gitea.io/gitea/
* Add Gitea authors copyright
* Change basic auth realm to "gitea-lfs"
* Add unique indexes to LFS model
* Use xorm count function in LFS check on repository delete
* Return io.ReadCloser from content store and close after usage
* Add LFS info to runWeb()
* Export LFS content store base path
* LFS file download from UI
* Work around git-lfs client issue with unauthenticated requests
Returning a dummy Authorization header for unauthenticated requests
lets git-lfs client skip asking for auth credentials
See: https://github.com/github/git-lfs/issues/1088
* Fix unauthenticated UI downloads from public repositories
* Authentication check order, Finish LFS file view logic
* Ignore LFS hooks if installed for current OS user
Fixes Gitea UI actions for repositories tracking LFS files.
Checks for minimum needed git version by parsing the semantic version
string.
* Hide LFS metafile diff from commit view, marking as binary
* Show LFS notice if file in commit view is tracked
* Add notbefore/nbf JWT claim
* Correct lint suggestions - comments for structs and functions
- Add comments to LFS model
- Function comment for GetRandomBytesAsBase64
- LFS server function comments and lint variable suggestion
* Move secret generation code out of conditional
Ensures no LFS code may run with an empty secret
* Do not hand out JWT tokens if LFS server support is disabled
wxiaoguang
zeripath
flynnnnnnnnnn
Philip Peterson
Lunny Xiao
KN4CK3R
John Olheiser
Lauris BH
Ethan Koenig
Ethan Koenig
Bo-Yi Wu
Fabian Zaremba
