Exclude the current dump file from the dump (#14606)
* Exclude the current dump file from the dump
Always prevent the current file from being added to the dump.
Fix #13618
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Add skip custom directory option
Signed-off-by: Andrew Thornton <art27@cantab.net>
* placate lint
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Use [chi](https://github.com/go-chi/chi) instead of the forked [macaron](https://gitea.com/macaron/macaron). Since macaron and chi have conflicts with session share, this big PR becomes a have-to thing. According my previous idea, we can replace macaron step by step but I'm wrong. :( Below is a list of big changes on this PR.
- [x] Define `context.ResponseWriter` interface with an implementation `context.Response`.
- [x] Use chi instead of macaron, and also a customize `Route` to wrap chi so that the router usage is similar as before.
- [x] Create different routers for `web`, `api`, `internal` and `install` so that the codes will be more clear and no magic .
- [x] Use https://github.com/unrolled/render instead of macaron's internal render
- [x] Use https://github.com/NYTimes/gziphandler instead of https://gitea.com/macaron/gzip
- [x] Use https://gitea.com/go-chi/session which is a modified version of https://gitea.com/macaron/session and removed `nodb` support since it will not be maintained. **BREAK**
- [x] Use https://gitea.com/go-chi/captcha which is a modified version of https://gitea.com/macaron/captcha
- [x] Use https://gitea.com/go-chi/cache which is a modified version of https://gitea.com/macaron/cache
- [x] Use https://gitea.com/go-chi/binding which is a modified version of https://gitea.com/macaron/binding
- [x] Use https://github.com/go-chi/cors instead of https://gitea.com/macaron/cors
- [x] Dropped https://gitea.com/macaron/i18n and make a new one in `code.gitea.io/gitea/modules/translation`
- [x] Move validation form structs from `code.gitea.io/gitea/modules/auth` to `code.gitea.io/gitea/modules/forms` to avoid dependency cycle.
- [x] Removed macaron log service because it's not need any more. **BREAK**
- [x] All form structs have to be get by `web.GetForm(ctx)` in the route function but not as a function parameter on routes definition.
- [x] Move Git HTTP protocol implementation to use routers directly.
- [x] Fix the problem that chi routes don't support trailing slash but macaron did.
- [x] `/api/v1/swagger` now will be redirect to `/api/swagger` but not render directly so that `APIContext` will not create a html render.
Notices:
- Chi router don't support request with trailing slash
- Integration test `TestUserHeatmap` maybe mysql version related. It's failed on my macOS(mysql 5.7.29 installed via brew) but succeed on CI.
Co-authored-by: 6543 <6543@obermui.de>
Re-attempt to delete temporary upload if the file is locked by another process (#12447)
Replace all calls to os.Remove/os.RemoveAll by retrying util.Remove/util.RemoveAll and remove circular dependencies from util.
Fix #12339
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: silverwind <me@silverwind.io>
Dump: add output format tar and output to stdout (#10376)
* Dump: Use mholt/archive/v3 to support tar including many compressions
Signed-off-by: Philipp Homann <homann.philipp@googlemail.com>
* Dump: Allow dump output to stdout
Signed-off-by: Philipp Homann <homann.philipp@googlemail.com>
* Dump: Fixed bug present since #6677 where SessionConfig.Provider is never "file"
Signed-off-by: Philipp Homann <homann.philipp@googlemail.com>
* Dump: never pack RepoRootPath, LFS.ContentPath and LogRootPath when they are below AppDataPath
Signed-off-by: Philipp Homann <homann.philipp@googlemail.com>
* Dump: also dump LFS (fixes #10058)
Signed-off-by: Philipp Homann <homann.philipp@googlemail.com>
* Dump: never dump CustomPath if CustomPath is a subdir of or equal to AppDataPath (fixes #10365)
Signed-off-by: Philipp Homann <homann.philipp@googlemail.com>
* Use log.Info instead of fmt.Fprintf
Signed-off-by: Philipp Homann <homann.philipp@googlemail.com>
* import ordering
* make fmt
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: Matti R <matti@mdranta.net>
cmd: dump: check value of skip-repository flag (#11254)
This is a boolean flag; simply checking if it's set isn't enough, we must check the value as well.
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Not all dumps need to include the logs, in a similar vain to not all dumps
needing to include repositories; these may be subject to different backup
mechanisms/constraints. Add a simple option to let them be excluded from the
dump to simplify workflows that need to exclude them or not collect in the
first place.
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: zeripath <art27@cantab.net>
* Fixed errors logging in dump.go
* Replaced overlooked log.Fatal() with fatal()
* Run make fmt on dump.go
* Added missing new line to the fatal log
* Run make fmt on dump.go
When running the dump command, include the specific configuration file
provided as part of the backup. Currently we only zip up custom/ and this
misses the config file if it is somewhere else, like /etc/gitea/app.ini
Fixes #6034
Set session and indexers' data files rel to AppDataPath (#2192)
* Set session and indexers' data files rel to AppDataPath
The setting AppDataPath is now relative to the working directory.
The session svc's PROVIDER_CONFIG now defaults to AppDataPath/data/sessions.
The issue indexer's IssuePath now defaults to AppDataPath/indexers/issues.bleves.
* fix bug
This change corrects a few logging issues:
* Standardized formatting errors with '%v'.
* Standardized failure warning word usage.
* Corrected an instance of using the standard log library when
the gitea log library should be used instead.
Now, the dump cmd uses setting.CustomPath and setting.LogRootPath
instead of setting.WorkDir which was kind of broken if the gogs
binary was in a different directory than gogs data.
Additionally, the backup of setting.CustomPath directory is only done
if it exists.
Using a tmp dir makes gogs dump more robust to concurrent runs.
It also permits an easier cleaning of the tmp files (gogs-db.sql and
gog-repo.zip) by just removing the tmp dir.
As a side effect, it partially fix bugs on workdir.
Previously, 'gogs dump' created the archives in the current directory,
and tried to include these archives from the directory where the
gogs binary lies.
ex: if gogs binary is in /usr/bin/gogs, and gogs dump is run from /tmp/,
/tmp/gog-repo.zip is created, but gogs dump tried to include
/usr/bin/gogs-repo.zip.
The dump cmd did not check the return value of the z.AddFile or
z.AddDir when building the final archive.
It caused the dump command to succeed even if an error occurred.
The resulted dump archive could be corrupted/empty.
(errors could be various: removal by a concurrent process, disk full,
bugs in the dump cmd itself)
Johan Van de Wauw
zeripath
Lunny Xiao
6543
PhilippHomann
Kyle Evans
Rychu
Tamal Saha
John Olheiser
mrsdizzie
glaszig
Zsombor
Morgan Bazalgette
Guillaume Dube
Jonas Östanbäck
Bo-Yi Wu
Gabriel Jackson
Manuel Kuhlmann
Joubert RedRat
Bwko
Sandro Santilli
Matthias Loibl
Thomas Boerger
Rémy Boulanouar
Kim "BKC" Carlbäcker
Thibault Meyer
Alex Myasoedov
Unknwon
kakwa
Unknwon
Peter Smit
chai2010