aboutsummaryrefslogtreecommitdiffstats
path: root/modules/storage/local.go
Commit message (Collapse)AuthorAgeFilesLines
* fix minio storage iterator path (#24691)FuXiaoHei2023-05-131-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | minio storage iterator shows different behavior with local fs iterator. in local fs storage: ``` go s.IterateObjects("prefix", func(path,obj) println(path) // show "prefix/xxx.file" }) ``` in minio storage: ```go s.IterateObjects("prefix", func(path,obj) println(path) // show "xxx.file" }) ``` I think local fs is correct, minio use wrong `basePath` to trim storage path prefix. --------- Co-authored-by: Giteabot <teabot@gitea.io>
* Introduce path Clean/Join helper functions (#23495)wxiaoguang2023-03-211-7/+10
| | | | | | | | | | | | | | | Since #23493 has conflicts with latest commits, this PR is my proposal for fixing #23371 Details are in the comments And refactor the `modules/options` module, to make it always use "filepath" to access local files. Benefits: * No need to do `util.CleanPath(strings.ReplaceAll(p, "\\", "/"))), "/")` any more (not only one before) * The function behaviors are clearly defined
* add path prefix to ObjectStorage.Iterator (#23332)FuXiaoHei2023-03-131-2/+6
| | | | | | | | | | | | Support to iterator subdirectory in ObjectStorage for ObjectStorage.Iterator method. It's required for https://github.com/go-gitea/gitea/pull/22738 to make artifact files cleanable. --------- Co-authored-by: Jason Song <i@wolfogre.com> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Use CleanPath instead of path.Clean (#23371)Lunny Xiao2023-03-081-2/+1
| | | As title.
* Change use of Walk to WalkDir to improve disk performance (#22462)Felipe Leopoldo Sologuren Gutiérrez2023-01-161-2/+2
| | | | | | | | | | | | | As suggest by Go developers, use `filepath.WalkDir` instead of `filepath.Walk` because [*Walk is less efficient than WalkDir, introduced in Go 1.16, which avoids calling `os.Lstat` on every file or directory visited](https://pkg.go.dev/path/filepath#Walk). This proposition address that, in a similar way as https://github.com/go-gitea/gitea/pull/22392 did. Co-authored-by: zeripath <art27@cantab.net> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Local storage should not store files as executable (#22162)zeripath2022-12-191-1/+2
| | | | | | | | | | The PR #21198 introduced a probable security vulnerability which resulted in making all storage files be marked as executable. This PR ensures that these are forcibly marked as non-executable. Fix #22161 Signed-off-by: Andrew Thornton <art27@cantab.net>
* Implement FSFE REUSE for golang files (#21840)flynnnnnnnnnn2022-11-271-2/+1
| | | | | | | | | Change all license headers to comply with REUSE specification. Fix #16132 Co-authored-by: flynnnnnnnnnn <flynnnnnnnnnn@github> Co-authored-by: John Olheiser <john.olheiser@gmail.com>
* Save files in local storage as umask (#21198)Nulo (he/him)2022-09-241-0/+4
| | | | | | | | Go creates temporary files as 600, but sometimes we want the group to be able to read them (for example, for another user to back up the storage) This PR applies the umask to the renamed tmp files in local storage. Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
* Clean paths when looking in Storage (#19124)zeripath2022-03-221-26/+8
| | | | | | | | | | | | | * Clean paths when looking in Storage Ensure paths are clean for minio aswell as local storage. Use url.Path not RequestURI/EscapedPath in storageHandler. Signed-off-by: Andrew Thornton <art27@cantab.net> * Apply suggestions from code review Co-authored-by: Lauris BH <lauris@nix.lv>
* Fix lfs bug (#19072)Lunny Xiao2022-03-141-0/+23
| | | * Fix lfs bug
* format with gofumpt (#18184)65432022-01-201-3/+1
| | | | | | | | | | | * gofumpt -w -l . * gofumpt -w -l -extra . * Add linter * manual fix * change make fmt
* refactor: move from io/ioutil to io and os package (#17109)Eng Zer Jun2021-09-221-2/+1
| | | | | | | | | The io/ioutil package has been deprecated as of Go 1.16, see https://golang.org/doc/go1.16#ioutil. This commit replaces the existing io/ioutil functions with their new definitions in io and os packages. Signed-off-by: Eng Zer Jun <engzerjun@gmail.com> Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Retry rename on lock induced failures (#16435)zeripath2021-07-151-1/+1
| | | | | | | | | | | | | | | | | | * Retry rename on lock induced failures Due to external locking on Windows it is possible for an os.Rename to fail if the files or directories are being used elsewhere. This PR simply suggests retrying the rename again similar to how we handle the os.Remove problems. Fix #16427 Signed-off-by: Andrew Thornton <art27@cantab.net> * resolve CI fail Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Add size to Save function (#15264)zeripath2021-04-031-1/+1
| | | | | Fix #15253 Signed-off-by: Andrew Thornton <art27@cantab.net>
* Fix race in local storage (#14888)zeripath2021-03-051-10/+38
| | | | | LocalStorage should only put completed files in position Signed-off-by: Andrew Thornton <art27@cantab.net>
* Prevent panics with missing storage (#13164)zeripath2020-10-151-0/+2
| | | | | | | | | | | | | | | | * The `.Use` of storageHandler before setting up the template renderer causes a panic if there is an error to log. * The error passed to `ctx.Error` in that case may contain sensitive information and should not be rendered to the end user. We should instead log the error and render a simple error message. * There is no handling of missing avatars and this needs a 404. Minio errors need to be mapped to standard golang errors such as os.ErrNotExist. * There is no logging when storage is set up. Related #13159 Signed-off-by: Andrew Thornton <art27@cantab.net>
* Provide self-registering storage system (#12978)zeripath2020-10-121-3/+29
| | | | | | | | | | | | | | | | | | | * Provide self-registering storage system Signed-off-by: Andrew Thornton <art27@cantab.net> * More simplification Signed-off-by: Andrew Thornton <art27@cantab.net> * Remove old strings from setting Signed-off-by: Andrew Thornton <art27@cantab.net> * oops attachments not attachment Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Add default storage configurations (#12813)Lunny Xiao2020-09-291-1/+26
| | | | Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: zeripath <art27@cantab.net>
* LFS support to be stored on minio (#12518)Lunny Xiao2020-09-081-1/+6
| | | | | | | | | | | | | | | | | * LFS support to be stored on minio * Fix test * Fix lint * Fix lint * Fix check * Fix test * Update documents and add migration for LFS * Fix some bugs
* Add a storage layer for attachments (#11387)Lunny Xiao2020-08-181-0/+70
* Add a storage layer for attachments * Fix some bug * fix test * Fix copyright head and lint * Fix bug * Add setting for minio and flags for migrate-storage * Add documents * fix lint * Add test for minio store type on attachments * fix test * fix test * Apply suggestions from code review Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com> * Add warning when storage migrated successfully * Fix drone * fix test * rebase * Fix test * display the error on console * Move minio test to amd64 since minio docker don't support arm64 * refactor the codes * add trace * Fix test * remove log on xorm * Fi download bug * Add a storage layer for attachments * Add setting for minio and flags for migrate-storage * fix lint * Add test for minio store type on attachments * Apply suggestions from code review Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com> * Fix drone * fix test * Fix test * display the error on console * Move minio test to amd64 since minio docker don't support arm64 * refactor the codes * add trace * Fix test * Add URL function to serve attachments directly from S3/Minio * Add ability to enable/disable redirection in attachment configuration * Fix typo * Add a storage layer for attachments * Add setting for minio and flags for migrate-storage * fix lint * Add test for minio store type on attachments * Apply suggestions from code review Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com> * Fix drone * fix test * Fix test * display the error on console * Move minio test to amd64 since minio docker don't support arm64 * don't change unrelated files * Fix lint * Fix build * update go.mod and go.sum * Use github.com/minio/minio-go/v6 * Remove unused function * Upgrade minio to v7 and some other improvements * fix lint * Fix go mod Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com> Co-authored-by: Tyler <tystuyfzand@gmail.com>
generated by cgit v1.2.3 (git 2.39.1) at 2025-01-15 20:08:33 +0000