You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

9 years ago
Git LFS support v2 (#122) * Import github.com/git-lfs/lfs-test-server as lfs module base Imported commit is 3968aac269a77b73924649b9412ae03f7ccd3198 Removed: Dockerfile CONTRIBUTING.md mgmt* script/ vendor/ kvlogger.go .dockerignore .gitignore README.md * Remove config, add JWT support from github.com/mgit-at/lfs-test-server Imported commit f0cdcc5a01599c5a955dc1bbf683bb4acecdba83 * Add LFS settings * Add LFS meta object model * Add LFS routes and initialization * Import github.com/dgrijalva/jwt-go into vendor/ * Adapt LFS module: handlers, routing, meta store * Move LFS routes to /user/repo/info/lfs/* * Add request header checks to LFS BatchHandler / PostHandler * Implement LFS basic authentication * Rework JWT secret generation / load * Implement LFS SSH token authentication with JWT Specification: https://github.com/github/git-lfs/tree/master/docs/api * Integrate LFS settings into install process * Remove LFS objects when repository is deleted Only removes objects from content store when deleted repo is the only referencing repository * Make LFS module stateless Fixes bug where LFS would not work after installation without restarting Gitea * Change 500 'Internal Server Error' to 400 'Bad Request' * Change sql query to xorm call * Remove unneeded type from LFS module * Change internal imports to code.gitea.io/gitea/ * Add Gitea authors copyright * Change basic auth realm to "gitea-lfs" * Add unique indexes to LFS model * Use xorm count function in LFS check on repository delete * Return io.ReadCloser from content store and close after usage * Add LFS info to runWeb() * Export LFS content store base path * LFS file download from UI * Work around git-lfs client issue with unauthenticated requests Returning a dummy Authorization header for unauthenticated requests lets git-lfs client skip asking for auth credentials See: https://github.com/github/git-lfs/issues/1088 * Fix unauthenticated UI downloads from public repositories * Authentication check order, Finish LFS file view logic * Ignore LFS hooks if installed for current OS user Fixes Gitea UI actions for repositories tracking LFS files. Checks for minimum needed git version by parsing the semantic version string. * Hide LFS metafile diff from commit view, marking as binary * Show LFS notice if file in commit view is tracked * Add notbefore/nbf JWT claim * Correct lint suggestions - comments for structs and functions - Add comments to LFS model - Function comment for GetRandomBytesAsBase64 - LFS server function comments and lint variable suggestion * Move secret generation code out of conditional Ensures no LFS code may run with an empty secret * Do not hand out JWT tokens if LFS server support is disabled
7 years ago
10 years ago
Better logging (#6038) (#6095) * Panic don't fatal on create new logger Fixes #5854 Signed-off-by: Andrew Thornton <art27@cantab.net> * partial broken * Update the logging infrastrcture Signed-off-by: Andrew Thornton <art27@cantab.net> * Reset the skip levels for Fatal and Error Signed-off-by: Andrew Thornton <art27@cantab.net> * broken ncsa * More log.Error fixes Signed-off-by: Andrew Thornton <art27@cantab.net> * Remove nal * set log-levels to lowercase * Make console_test test all levels * switch to lowercased levels * OK now working * Fix vetting issues * Fix lint * Fix tests * change default logging to match current gitea * Improve log testing Signed-off-by: Andrew Thornton <art27@cantab.net> * reset error skip levels to 0 * Update documentation and access logger configuration * Redirect the router log back to gitea if redirect macaron log but also allow setting the log level - i.e. TRACE * Fix broken level caching * Refactor the router log * Add Router logger * Add colorizing options * Adjust router colors * Only create logger if they will be used * update app.ini.sample * rename Attribute ColorAttribute * Change from white to green for function * Set fatal/error levels * Restore initial trace logger * Fix Trace arguments in modules/auth/auth.go * Properly handle XORMLogger * Improve admin/config page * fix fmt * Add auto-compression of old logs * Update error log levels * Remove the unnecessary skip argument from Error, Fatal and Critical * Add stacktrace support * Fix tests * Remove x/sync from vendors? * Add stderr option to console logger * Use filepath.ToSlash to protect against Windows in tests * Remove prefixed underscores from names in colors.go * Remove not implemented database logger This was removed from Gogs on 4 Mar 2016 but left in the configuration since then. * Ensure that log paths are relative to ROOT_PATH * use path.Join * rename jsonConfig to logConfig * Rename "config" to "jsonConfig" to make it clearer * Requested changes * Requested changes: XormLogger * Try to color the windows terminal If successful default to colorizing the console logs * fixup * Colorize initially too * update vendor * Colorize logs on default and remove if this is not a colorizing logger * Fix documentation * fix test * Use go-isatty to detect if on windows we are on msys or cygwin * Fix spelling mistake * Add missing vendors * More changes * Rationalise the ANSI writer protection * Adjust colors on advice from @0x5c * Make Flags a comma separated list * Move to use the windows constant for ENABLE_VIRTUAL_TERMINAL_PROCESSING * Ensure matching is done on the non-colored message - to simpify EXPRESSION
5 years ago
Refactor path & config system (#25330) # The problem There were many "path tricks": * By default, Gitea uses its program directory as its work path * Gitea tries to use the "work path" to guess its "custom path" and "custom conf (app.ini)" * Users might want to use other directories as work path * The non-default work path should be passed to Gitea by GITEA_WORK_DIR or "--work-path" * But some Gitea processes are started without these values * The "serv" process started by OpenSSH server * The CLI sub-commands started by site admin * The paths are guessed by SetCustomPathAndConf again and again * The default values of "work path / custom path / custom conf" can be changed when compiling # The solution * Use `InitWorkPathAndCommonConfig` to handle these path tricks, and use test code to cover its behaviors. * When Gitea's web server runs, write the WORK_PATH to "app.ini", this value must be the most correct one, because if this value is not right, users would find that the web UI doesn't work and then they should be able to fix it. * Then all other sub-commands can use the WORK_PATH in app.ini to initialize their paths. * By the way, when Gitea starts for git protocol, it shouldn't output any log, otherwise the git protocol gets broken and client blocks forever. The "work path" priority is: WORK_PATH in app.ini > cmd arg --work-path > env var GITEA_WORK_DIR > builtin default The "app.ini" searching order is: cmd arg --config > cmd arg "work path / custom path" > env var "work path / custom path" > builtin default ## ⚠️ BREAKING If your instance's "work path / custom path / custom conf" doesn't meet the requirements (eg: work path must be absolute), Gitea will report a fatal error and exit. You need to set these values according to the error log. ---- Close #24818 Close #24222 Close #21606 Close #21498 Close #25107 Close #24981 Maybe close #24503 Replace #23301 Replace #22754 And maybe more
1 year ago
Refactor path & config system (#25330) # The problem There were many "path tricks": * By default, Gitea uses its program directory as its work path * Gitea tries to use the "work path" to guess its "custom path" and "custom conf (app.ini)" * Users might want to use other directories as work path * The non-default work path should be passed to Gitea by GITEA_WORK_DIR or "--work-path" * But some Gitea processes are started without these values * The "serv" process started by OpenSSH server * The CLI sub-commands started by site admin * The paths are guessed by SetCustomPathAndConf again and again * The default values of "work path / custom path / custom conf" can be changed when compiling # The solution * Use `InitWorkPathAndCommonConfig` to handle these path tricks, and use test code to cover its behaviors. * When Gitea's web server runs, write the WORK_PATH to "app.ini", this value must be the most correct one, because if this value is not right, users would find that the web UI doesn't work and then they should be able to fix it. * Then all other sub-commands can use the WORK_PATH in app.ini to initialize their paths. * By the way, when Gitea starts for git protocol, it shouldn't output any log, otherwise the git protocol gets broken and client blocks forever. The "work path" priority is: WORK_PATH in app.ini > cmd arg --work-path > env var GITEA_WORK_DIR > builtin default The "app.ini" searching order is: cmd arg --config > cmd arg "work path / custom path" > env var "work path / custom path" > builtin default ## ⚠️ BREAKING If your instance's "work path / custom path / custom conf" doesn't meet the requirements (eg: work path must be absolute), Gitea will report a fatal error and exit. You need to set these values according to the error log. ---- Close #24818 Close #24222 Close #21606 Close #21498 Close #25107 Close #24981 Maybe close #24503 Replace #23301 Replace #22754 And maybe more
1 year ago
Refactor path & config system (#25330) # The problem There were many "path tricks": * By default, Gitea uses its program directory as its work path * Gitea tries to use the "work path" to guess its "custom path" and "custom conf (app.ini)" * Users might want to use other directories as work path * The non-default work path should be passed to Gitea by GITEA_WORK_DIR or "--work-path" * But some Gitea processes are started without these values * The "serv" process started by OpenSSH server * The CLI sub-commands started by site admin * The paths are guessed by SetCustomPathAndConf again and again * The default values of "work path / custom path / custom conf" can be changed when compiling # The solution * Use `InitWorkPathAndCommonConfig` to handle these path tricks, and use test code to cover its behaviors. * When Gitea's web server runs, write the WORK_PATH to "app.ini", this value must be the most correct one, because if this value is not right, users would find that the web UI doesn't work and then they should be able to fix it. * Then all other sub-commands can use the WORK_PATH in app.ini to initialize their paths. * By the way, when Gitea starts for git protocol, it shouldn't output any log, otherwise the git protocol gets broken and client blocks forever. The "work path" priority is: WORK_PATH in app.ini > cmd arg --work-path > env var GITEA_WORK_DIR > builtin default The "app.ini" searching order is: cmd arg --config > cmd arg "work path / custom path" > env var "work path / custom path" > builtin default ## ⚠️ BREAKING If your instance's "work path / custom path / custom conf" doesn't meet the requirements (eg: work path must be absolute), Gitea will report a fatal error and exit. You need to set these values according to the error log. ---- Close #24818 Close #24222 Close #21606 Close #21498 Close #25107 Close #24981 Maybe close #24503 Replace #23301 Replace #22754 And maybe more
1 year ago
Refactor path & config system (#25330) # The problem There were many "path tricks": * By default, Gitea uses its program directory as its work path * Gitea tries to use the "work path" to guess its "custom path" and "custom conf (app.ini)" * Users might want to use other directories as work path * The non-default work path should be passed to Gitea by GITEA_WORK_DIR or "--work-path" * But some Gitea processes are started without these values * The "serv" process started by OpenSSH server * The CLI sub-commands started by site admin * The paths are guessed by SetCustomPathAndConf again and again * The default values of "work path / custom path / custom conf" can be changed when compiling # The solution * Use `InitWorkPathAndCommonConfig` to handle these path tricks, and use test code to cover its behaviors. * When Gitea's web server runs, write the WORK_PATH to "app.ini", this value must be the most correct one, because if this value is not right, users would find that the web UI doesn't work and then they should be able to fix it. * Then all other sub-commands can use the WORK_PATH in app.ini to initialize their paths. * By the way, when Gitea starts for git protocol, it shouldn't output any log, otherwise the git protocol gets broken and client blocks forever. The "work path" priority is: WORK_PATH in app.ini > cmd arg --work-path > env var GITEA_WORK_DIR > builtin default The "app.ini" searching order is: cmd arg --config > cmd arg "work path / custom path" > env var "work path / custom path" > builtin default ## ⚠️ BREAKING If your instance's "work path / custom path / custom conf" doesn't meet the requirements (eg: work path must be absolute), Gitea will report a fatal error and exit. You need to set these values according to the error log. ---- Close #24818 Close #24222 Close #21606 Close #21498 Close #25107 Close #24981 Maybe close #24503 Replace #23301 Replace #22754 And maybe more
1 year ago
Refactor path & config system (#25330) # The problem There were many "path tricks": * By default, Gitea uses its program directory as its work path * Gitea tries to use the "work path" to guess its "custom path" and "custom conf (app.ini)" * Users might want to use other directories as work path * The non-default work path should be passed to Gitea by GITEA_WORK_DIR or "--work-path" * But some Gitea processes are started without these values * The "serv" process started by OpenSSH server * The CLI sub-commands started by site admin * The paths are guessed by SetCustomPathAndConf again and again * The default values of "work path / custom path / custom conf" can be changed when compiling # The solution * Use `InitWorkPathAndCommonConfig` to handle these path tricks, and use test code to cover its behaviors. * When Gitea's web server runs, write the WORK_PATH to "app.ini", this value must be the most correct one, because if this value is not right, users would find that the web UI doesn't work and then they should be able to fix it. * Then all other sub-commands can use the WORK_PATH in app.ini to initialize their paths. * By the way, when Gitea starts for git protocol, it shouldn't output any log, otherwise the git protocol gets broken and client blocks forever. The "work path" priority is: WORK_PATH in app.ini > cmd arg --work-path > env var GITEA_WORK_DIR > builtin default The "app.ini" searching order is: cmd arg --config > cmd arg "work path / custom path" > env var "work path / custom path" > builtin default ## ⚠️ BREAKING If your instance's "work path / custom path / custom conf" doesn't meet the requirements (eg: work path must be absolute), Gitea will report a fatal error and exit. You need to set these values according to the error log. ---- Close #24818 Close #24222 Close #21606 Close #21498 Close #25107 Close #24981 Maybe close #24503 Replace #23301 Replace #22754 And maybe more
1 year ago
Refactor path & config system (#25330) # The problem There were many "path tricks": * By default, Gitea uses its program directory as its work path * Gitea tries to use the "work path" to guess its "custom path" and "custom conf (app.ini)" * Users might want to use other directories as work path * The non-default work path should be passed to Gitea by GITEA_WORK_DIR or "--work-path" * But some Gitea processes are started without these values * The "serv" process started by OpenSSH server * The CLI sub-commands started by site admin * The paths are guessed by SetCustomPathAndConf again and again * The default values of "work path / custom path / custom conf" can be changed when compiling # The solution * Use `InitWorkPathAndCommonConfig` to handle these path tricks, and use test code to cover its behaviors. * When Gitea's web server runs, write the WORK_PATH to "app.ini", this value must be the most correct one, because if this value is not right, users would find that the web UI doesn't work and then they should be able to fix it. * Then all other sub-commands can use the WORK_PATH in app.ini to initialize their paths. * By the way, when Gitea starts for git protocol, it shouldn't output any log, otherwise the git protocol gets broken and client blocks forever. The "work path" priority is: WORK_PATH in app.ini > cmd arg --work-path > env var GITEA_WORK_DIR > builtin default The "app.ini" searching order is: cmd arg --config > cmd arg "work path / custom path" > env var "work path / custom path" > builtin default ## ⚠️ BREAKING If your instance's "work path / custom path / custom conf" doesn't meet the requirements (eg: work path must be absolute), Gitea will report a fatal error and exit. You need to set these values according to the error log. ---- Close #24818 Close #24222 Close #21606 Close #21498 Close #25107 Close #24981 Maybe close #24503 Replace #23301 Replace #22754 And maybe more
1 year ago
Refactor path & config system (#25330) # The problem There were many "path tricks": * By default, Gitea uses its program directory as its work path * Gitea tries to use the "work path" to guess its "custom path" and "custom conf (app.ini)" * Users might want to use other directories as work path * The non-default work path should be passed to Gitea by GITEA_WORK_DIR or "--work-path" * But some Gitea processes are started without these values * The "serv" process started by OpenSSH server * The CLI sub-commands started by site admin * The paths are guessed by SetCustomPathAndConf again and again * The default values of "work path / custom path / custom conf" can be changed when compiling # The solution * Use `InitWorkPathAndCommonConfig` to handle these path tricks, and use test code to cover its behaviors. * When Gitea's web server runs, write the WORK_PATH to "app.ini", this value must be the most correct one, because if this value is not right, users would find that the web UI doesn't work and then they should be able to fix it. * Then all other sub-commands can use the WORK_PATH in app.ini to initialize their paths. * By the way, when Gitea starts for git protocol, it shouldn't output any log, otherwise the git protocol gets broken and client blocks forever. The "work path" priority is: WORK_PATH in app.ini > cmd arg --work-path > env var GITEA_WORK_DIR > builtin default The "app.ini" searching order is: cmd arg --config > cmd arg "work path / custom path" > env var "work path / custom path" > builtin default ## ⚠️ BREAKING If your instance's "work path / custom path / custom conf" doesn't meet the requirements (eg: work path must be absolute), Gitea will report a fatal error and exit. You need to set these values according to the error log. ---- Close #24818 Close #24222 Close #21606 Close #21498 Close #25107 Close #24981 Maybe close #24503 Replace #23301 Replace #22754 And maybe more
1 year ago
Refactor path & config system (#25330) # The problem There were many "path tricks": * By default, Gitea uses its program directory as its work path * Gitea tries to use the "work path" to guess its "custom path" and "custom conf (app.ini)" * Users might want to use other directories as work path * The non-default work path should be passed to Gitea by GITEA_WORK_DIR or "--work-path" * But some Gitea processes are started without these values * The "serv" process started by OpenSSH server * The CLI sub-commands started by site admin * The paths are guessed by SetCustomPathAndConf again and again * The default values of "work path / custom path / custom conf" can be changed when compiling # The solution * Use `InitWorkPathAndCommonConfig` to handle these path tricks, and use test code to cover its behaviors. * When Gitea's web server runs, write the WORK_PATH to "app.ini", this value must be the most correct one, because if this value is not right, users would find that the web UI doesn't work and then they should be able to fix it. * Then all other sub-commands can use the WORK_PATH in app.ini to initialize their paths. * By the way, when Gitea starts for git protocol, it shouldn't output any log, otherwise the git protocol gets broken and client blocks forever. The "work path" priority is: WORK_PATH in app.ini > cmd arg --work-path > env var GITEA_WORK_DIR > builtin default The "app.ini" searching order is: cmd arg --config > cmd arg "work path / custom path" > env var "work path / custom path" > builtin default ## ⚠️ BREAKING If your instance's "work path / custom path / custom conf" doesn't meet the requirements (eg: work path must be absolute), Gitea will report a fatal error and exit. You need to set these values according to the error log. ---- Close #24818 Close #24222 Close #21606 Close #21498 Close #25107 Close #24981 Maybe close #24503 Replace #23301 Replace #22754 And maybe more
1 year ago
Refactor path & config system (#25330) # The problem There were many "path tricks": * By default, Gitea uses its program directory as its work path * Gitea tries to use the "work path" to guess its "custom path" and "custom conf (app.ini)" * Users might want to use other directories as work path * The non-default work path should be passed to Gitea by GITEA_WORK_DIR or "--work-path" * But some Gitea processes are started without these values * The "serv" process started by OpenSSH server * The CLI sub-commands started by site admin * The paths are guessed by SetCustomPathAndConf again and again * The default values of "work path / custom path / custom conf" can be changed when compiling # The solution * Use `InitWorkPathAndCommonConfig` to handle these path tricks, and use test code to cover its behaviors. * When Gitea's web server runs, write the WORK_PATH to "app.ini", this value must be the most correct one, because if this value is not right, users would find that the web UI doesn't work and then they should be able to fix it. * Then all other sub-commands can use the WORK_PATH in app.ini to initialize their paths. * By the way, when Gitea starts for git protocol, it shouldn't output any log, otherwise the git protocol gets broken and client blocks forever. The "work path" priority is: WORK_PATH in app.ini > cmd arg --work-path > env var GITEA_WORK_DIR > builtin default The "app.ini" searching order is: cmd arg --config > cmd arg "work path / custom path" > env var "work path / custom path" > builtin default ## ⚠️ BREAKING If your instance's "work path / custom path / custom conf" doesn't meet the requirements (eg: work path must be absolute), Gitea will report a fatal error and exit. You need to set these values according to the error log. ---- Close #24818 Close #24222 Close #21606 Close #21498 Close #25107 Close #24981 Maybe close #24503 Replace #23301 Replace #22754 And maybe more
1 year ago
Refactor path & config system (#25330) # The problem There were many "path tricks": * By default, Gitea uses its program directory as its work path * Gitea tries to use the "work path" to guess its "custom path" and "custom conf (app.ini)" * Users might want to use other directories as work path * The non-default work path should be passed to Gitea by GITEA_WORK_DIR or "--work-path" * But some Gitea processes are started without these values * The "serv" process started by OpenSSH server * The CLI sub-commands started by site admin * The paths are guessed by SetCustomPathAndConf again and again * The default values of "work path / custom path / custom conf" can be changed when compiling # The solution * Use `InitWorkPathAndCommonConfig` to handle these path tricks, and use test code to cover its behaviors. * When Gitea's web server runs, write the WORK_PATH to "app.ini", this value must be the most correct one, because if this value is not right, users would find that the web UI doesn't work and then they should be able to fix it. * Then all other sub-commands can use the WORK_PATH in app.ini to initialize their paths. * By the way, when Gitea starts for git protocol, it shouldn't output any log, otherwise the git protocol gets broken and client blocks forever. The "work path" priority is: WORK_PATH in app.ini > cmd arg --work-path > env var GITEA_WORK_DIR > builtin default The "app.ini" searching order is: cmd arg --config > cmd arg "work path / custom path" > env var "work path / custom path" > builtin default ## ⚠️ BREAKING If your instance's "work path / custom path / custom conf" doesn't meet the requirements (eg: work path must be absolute), Gitea will report a fatal error and exit. You need to set these values according to the error log. ---- Close #24818 Close #24222 Close #21606 Close #21498 Close #25107 Close #24981 Maybe close #24503 Replace #23301 Replace #22754 And maybe more
1 year ago
Refactor path & config system (#25330) # The problem There were many "path tricks": * By default, Gitea uses its program directory as its work path * Gitea tries to use the "work path" to guess its "custom path" and "custom conf (app.ini)" * Users might want to use other directories as work path * The non-default work path should be passed to Gitea by GITEA_WORK_DIR or "--work-path" * But some Gitea processes are started without these values * The "serv" process started by OpenSSH server * The CLI sub-commands started by site admin * The paths are guessed by SetCustomPathAndConf again and again * The default values of "work path / custom path / custom conf" can be changed when compiling # The solution * Use `InitWorkPathAndCommonConfig` to handle these path tricks, and use test code to cover its behaviors. * When Gitea's web server runs, write the WORK_PATH to "app.ini", this value must be the most correct one, because if this value is not right, users would find that the web UI doesn't work and then they should be able to fix it. * Then all other sub-commands can use the WORK_PATH in app.ini to initialize their paths. * By the way, when Gitea starts for git protocol, it shouldn't output any log, otherwise the git protocol gets broken and client blocks forever. The "work path" priority is: WORK_PATH in app.ini > cmd arg --work-path > env var GITEA_WORK_DIR > builtin default The "app.ini" searching order is: cmd arg --config > cmd arg "work path / custom path" > env var "work path / custom path" > builtin default ## ⚠️ BREAKING If your instance's "work path / custom path / custom conf" doesn't meet the requirements (eg: work path must be absolute), Gitea will report a fatal error and exit. You need to set these values according to the error log. ---- Close #24818 Close #24222 Close #21606 Close #21498 Close #25107 Close #24981 Maybe close #24503 Replace #23301 Replace #22754 And maybe more
1 year ago
Git LFS support v2 (#122) * Import github.com/git-lfs/lfs-test-server as lfs module base Imported commit is 3968aac269a77b73924649b9412ae03f7ccd3198 Removed: Dockerfile CONTRIBUTING.md mgmt* script/ vendor/ kvlogger.go .dockerignore .gitignore README.md * Remove config, add JWT support from github.com/mgit-at/lfs-test-server Imported commit f0cdcc5a01599c5a955dc1bbf683bb4acecdba83 * Add LFS settings * Add LFS meta object model * Add LFS routes and initialization * Import github.com/dgrijalva/jwt-go into vendor/ * Adapt LFS module: handlers, routing, meta store * Move LFS routes to /user/repo/info/lfs/* * Add request header checks to LFS BatchHandler / PostHandler * Implement LFS basic authentication * Rework JWT secret generation / load * Implement LFS SSH token authentication with JWT Specification: https://github.com/github/git-lfs/tree/master/docs/api * Integrate LFS settings into install process * Remove LFS objects when repository is deleted Only removes objects from content store when deleted repo is the only referencing repository * Make LFS module stateless Fixes bug where LFS would not work after installation without restarting Gitea * Change 500 'Internal Server Error' to 400 'Bad Request' * Change sql query to xorm call * Remove unneeded type from LFS module * Change internal imports to code.gitea.io/gitea/ * Add Gitea authors copyright * Change basic auth realm to "gitea-lfs" * Add unique indexes to LFS model * Use xorm count function in LFS check on repository delete * Return io.ReadCloser from content store and close after usage * Add LFS info to runWeb() * Export LFS content store base path * LFS file download from UI * Work around git-lfs client issue with unauthenticated requests Returning a dummy Authorization header for unauthenticated requests lets git-lfs client skip asking for auth credentials See: https://github.com/github/git-lfs/issues/1088 * Fix unauthenticated UI downloads from public repositories * Authentication check order, Finish LFS file view logic * Ignore LFS hooks if installed for current OS user Fixes Gitea UI actions for repositories tracking LFS files. Checks for minimum needed git version by parsing the semantic version string. * Hide LFS metafile diff from commit view, marking as binary * Show LFS notice if file in commit view is tracked * Add notbefore/nbf JWT claim * Correct lint suggestions - comments for structs and functions - Add comments to LFS model - Function comment for GetRandomBytesAsBase64 - LFS server function comments and lint variable suggestion * Move secret generation code out of conditional Ensures no LFS code may run with an empty secret * Do not hand out JWT tokens if LFS server support is disabled
7 years ago
10 years ago
10 years ago
10 years ago
Better logging (#6038) (#6095) * Panic don't fatal on create new logger Fixes #5854 Signed-off-by: Andrew Thornton <art27@cantab.net> * partial broken * Update the logging infrastrcture Signed-off-by: Andrew Thornton <art27@cantab.net> * Reset the skip levels for Fatal and Error Signed-off-by: Andrew Thornton <art27@cantab.net> * broken ncsa * More log.Error fixes Signed-off-by: Andrew Thornton <art27@cantab.net> * Remove nal * set log-levels to lowercase * Make console_test test all levels * switch to lowercased levels * OK now working * Fix vetting issues * Fix lint * Fix tests * change default logging to match current gitea * Improve log testing Signed-off-by: Andrew Thornton <art27@cantab.net> * reset error skip levels to 0 * Update documentation and access logger configuration * Redirect the router log back to gitea if redirect macaron log but also allow setting the log level - i.e. TRACE * Fix broken level caching * Refactor the router log * Add Router logger * Add colorizing options * Adjust router colors * Only create logger if they will be used * update app.ini.sample * rename Attribute ColorAttribute * Change from white to green for function * Set fatal/error levels * Restore initial trace logger * Fix Trace arguments in modules/auth/auth.go * Properly handle XORMLogger * Improve admin/config page * fix fmt * Add auto-compression of old logs * Update error log levels * Remove the unnecessary skip argument from Error, Fatal and Critical * Add stacktrace support * Fix tests * Remove x/sync from vendors? * Add stderr option to console logger * Use filepath.ToSlash to protect against Windows in tests * Remove prefixed underscores from names in colors.go * Remove not implemented database logger This was removed from Gogs on 4 Mar 2016 but left in the configuration since then. * Ensure that log paths are relative to ROOT_PATH * use path.Join * rename jsonConfig to logConfig * Rename "config" to "jsonConfig" to make it clearer * Requested changes * Requested changes: XormLogger * Try to color the windows terminal If successful default to colorizing the console logs * fixup * Colorize initially too * update vendor * Colorize logs on default and remove if this is not a colorizing logger * Fix documentation * fix test * Use go-isatty to detect if on windows we are on msys or cygwin * Fix spelling mistake * Add missing vendors * More changes * Rationalise the ANSI writer protection * Adjust colors on advice from @0x5c * Make Flags a comma separated list * Move to use the windows constant for ENABLE_VIRTUAL_TERMINAL_PROCESSING * Ensure matching is done on the non-colored message - to simpify EXPRESSION
5 years ago
10 years ago
10 years ago
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357
  1. // Copyright 2014 The Gogs Authors. All rights reserved.
  2. // SPDX-License-Identifier: MIT
  3. package cmd
  4. import (
  5. "context"
  6. "fmt"
  7. "net"
  8. "net/http"
  9. "os"
  10. "path/filepath"
  11. "strconv"
  12. "strings"
  13. _ "net/http/pprof" // Used for debugging if enabled and a web server is running
  14. "code.gitea.io/gitea/modules/container"
  15. "code.gitea.io/gitea/modules/graceful"
  16. "code.gitea.io/gitea/modules/log"
  17. "code.gitea.io/gitea/modules/process"
  18. "code.gitea.io/gitea/modules/public"
  19. "code.gitea.io/gitea/modules/setting"
  20. "code.gitea.io/gitea/routers"
  21. "code.gitea.io/gitea/routers/install"
  22. "github.com/felixge/fgprof"
  23. "github.com/urfave/cli/v2"
  24. )
  25. // PIDFile could be set from build tag
  26. var PIDFile = "/run/gitea.pid"
  27. // CmdWeb represents the available web sub-command.
  28. var CmdWeb = &cli.Command{
  29. Name: "web",
  30. Usage: "Start Gitea web server",
  31. Description: `Gitea web server is the only thing you need to run,
  32. and it takes care of all the other things for you`,
  33. Before: PrepareConsoleLoggerLevel(log.INFO),
  34. Action: runWeb,
  35. Flags: []cli.Flag{
  36. &cli.StringFlag{
  37. Name: "port",
  38. Aliases: []string{"p"},
  39. Value: "3000",
  40. Usage: "Temporary port number to prevent conflict",
  41. },
  42. &cli.StringFlag{
  43. Name: "install-port",
  44. Value: "3000",
  45. Usage: "Temporary port number to run the install page on to prevent conflict",
  46. },
  47. &cli.StringFlag{
  48. Name: "pid",
  49. Aliases: []string{"P"},
  50. Value: PIDFile,
  51. Usage: "Custom pid file path",
  52. },
  53. &cli.BoolFlag{
  54. Name: "quiet",
  55. Aliases: []string{"q"},
  56. Usage: "Only display Fatal logging errors until logging is set-up",
  57. },
  58. &cli.BoolFlag{
  59. Name: "verbose",
  60. Usage: "Set initial logging to TRACE level until logging is properly set-up",
  61. },
  62. },
  63. }
  64. func runHTTPRedirector() {
  65. _, _, finished := process.GetManager().AddTypedContext(graceful.GetManager().HammerContext(), "Web: HTTP Redirector", process.SystemProcessType, true)
  66. defer finished()
  67. source := fmt.Sprintf("%s:%s", setting.HTTPAddr, setting.PortToRedirect)
  68. dest := strings.TrimSuffix(setting.AppURL, "/")
  69. log.Info("Redirecting: %s to %s", source, dest)
  70. handler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
  71. target := dest + r.URL.Path
  72. if len(r.URL.RawQuery) > 0 {
  73. target += "?" + r.URL.RawQuery
  74. }
  75. http.Redirect(w, r, target, http.StatusTemporaryRedirect)
  76. })
  77. err := runHTTP("tcp", source, "HTTP Redirector", handler, setting.RedirectorUseProxyProtocol)
  78. if err != nil {
  79. log.Fatal("Failed to start port redirection: %v", err)
  80. }
  81. }
  82. func createPIDFile(pidPath string) {
  83. currentPid := os.Getpid()
  84. if err := os.MkdirAll(filepath.Dir(pidPath), os.ModePerm); err != nil {
  85. log.Fatal("Failed to create PID folder: %v", err)
  86. }
  87. file, err := os.Create(pidPath)
  88. if err != nil {
  89. log.Fatal("Failed to create PID file: %v", err)
  90. }
  91. defer file.Close()
  92. if _, err := file.WriteString(strconv.FormatInt(int64(currentPid), 10)); err != nil {
  93. log.Fatal("Failed to write PID information: %v", err)
  94. }
  95. }
  96. func showWebStartupMessage(msg string) {
  97. log.Info("Gitea version: %s%s", setting.AppVer, setting.AppBuiltWith)
  98. log.Info("* RunMode: %s", setting.RunMode)
  99. log.Info("* AppPath: %s", setting.AppPath)
  100. log.Info("* WorkPath: %s", setting.AppWorkPath)
  101. log.Info("* CustomPath: %s", setting.CustomPath)
  102. log.Info("* ConfigFile: %s", setting.CustomConf)
  103. log.Info("%s", msg)
  104. }
  105. func serveInstall(ctx *cli.Context) error {
  106. showWebStartupMessage("Prepare to run install page")
  107. routers.InitWebInstallPage(graceful.GetManager().HammerContext())
  108. // Flag for port number in case first time run conflict
  109. if ctx.IsSet("port") {
  110. if err := setPort(ctx.String("port")); err != nil {
  111. return err
  112. }
  113. }
  114. if ctx.IsSet("install-port") {
  115. if err := setPort(ctx.String("install-port")); err != nil {
  116. return err
  117. }
  118. }
  119. c := install.Routes()
  120. err := listen(c, false)
  121. if err != nil {
  122. log.Critical("Unable to open listener for installer. Is Gitea already running?")
  123. graceful.GetManager().DoGracefulShutdown()
  124. }
  125. select {
  126. case <-graceful.GetManager().IsShutdown():
  127. <-graceful.GetManager().Done()
  128. log.Info("PID: %d Gitea Web Finished", os.Getpid())
  129. log.GetManager().Close()
  130. return err
  131. default:
  132. }
  133. return nil
  134. }
  135. func serveInstalled(ctx *cli.Context) error {
  136. setting.InitCfgProvider(setting.CustomConf)
  137. setting.LoadCommonSettings()
  138. setting.MustInstalled()
  139. showWebStartupMessage("Prepare to run web server")
  140. if setting.AppWorkPathMismatch {
  141. log.Error("WORK_PATH from config %q doesn't match other paths from environment variables or command arguments. "+
  142. "Only WORK_PATH in config should be set and used. Please make sure the path in config file is correct, "+
  143. "remove the other outdated work paths from environment variables and command arguments", setting.CustomConf)
  144. }
  145. rootCfg := setting.CfgProvider
  146. if rootCfg.Section("").Key("WORK_PATH").String() == "" {
  147. saveCfg, err := rootCfg.PrepareSaving()
  148. if err != nil {
  149. log.Error("Unable to prepare saving WORK_PATH=%s to config %q: %v\nYou should set it manually, otherwise there might be bugs when accessing the git repositories.", setting.AppWorkPath, setting.CustomConf, err)
  150. } else {
  151. rootCfg.Section("").Key("WORK_PATH").SetValue(setting.AppWorkPath)
  152. saveCfg.Section("").Key("WORK_PATH").SetValue(setting.AppWorkPath)
  153. if err = saveCfg.Save(); err != nil {
  154. log.Error("Unable to update WORK_PATH=%s to config %q: %v\nYou should set it manually, otherwise there might be bugs when accessing the git repositories.", setting.AppWorkPath, setting.CustomConf, err)
  155. }
  156. }
  157. }
  158. // in old versions, user's custom web files are placed in "custom/public", and they were served as "http://domain.com/assets/xxx"
  159. // now, Gitea only serves pre-defined files in the "custom/public" folder basing on the web root, the user should move their custom files to "custom/public/assets"
  160. publicFiles, _ := public.AssetFS().ListFiles(".")
  161. publicFilesSet := container.SetOf(publicFiles...)
  162. publicFilesSet.Remove(".well-known")
  163. publicFilesSet.Remove("assets")
  164. publicFilesSet.Remove("robots.txt")
  165. for _, fn := range publicFilesSet.Values() {
  166. log.Error("Found legacy public asset %q in CustomPath. Please move it to %s/public/assets/%s", fn, setting.CustomPath, fn)
  167. }
  168. if _, err := os.Stat(filepath.Join(setting.CustomPath, "robots.txt")); err == nil {
  169. log.Error(`Found legacy public asset "robots.txt" in CustomPath. Please move it to %s/public/robots.txt`, setting.CustomPath)
  170. }
  171. routers.InitWebInstalled(graceful.GetManager().HammerContext())
  172. // We check that AppDataPath exists here (it should have been created during installation)
  173. // We can't check it in `InitWebInstalled`, because some integration tests
  174. // use cmd -> InitWebInstalled, but the AppDataPath doesn't exist during those tests.
  175. if _, err := os.Stat(setting.AppDataPath); err != nil {
  176. log.Fatal("Can not find APP_DATA_PATH %q", setting.AppDataPath)
  177. }
  178. // Override the provided port number within the configuration
  179. if ctx.IsSet("port") {
  180. if err := setPort(ctx.String("port")); err != nil {
  181. return err
  182. }
  183. }
  184. // Set up Chi routes
  185. webRoutes := routers.NormalRoutes()
  186. err := listen(webRoutes, true)
  187. <-graceful.GetManager().Done()
  188. log.Info("PID: %d Gitea Web Finished", os.Getpid())
  189. log.GetManager().Close()
  190. return err
  191. }
  192. func servePprof() {
  193. http.DefaultServeMux.Handle("/debug/fgprof", fgprof.Handler())
  194. _, _, finished := process.GetManager().AddTypedContext(context.Background(), "Web: PProf Server", process.SystemProcessType, true)
  195. // The pprof server is for debug purpose only, it shouldn't be exposed on public network. At the moment it's not worth to introduce a configurable option for it.
  196. log.Info("Starting pprof server on localhost:6060")
  197. log.Info("Stopped pprof server: %v", http.ListenAndServe("localhost:6060", nil))
  198. finished()
  199. }
  200. func runWeb(ctx *cli.Context) error {
  201. defer func() {
  202. if panicked := recover(); panicked != nil {
  203. log.Fatal("PANIC: %v\n%s", panicked, log.Stack(2))
  204. }
  205. }()
  206. managerCtx, cancel := context.WithCancel(context.Background())
  207. graceful.InitManager(managerCtx)
  208. defer cancel()
  209. if os.Getppid() > 1 && len(os.Getenv("LISTEN_FDS")) > 0 {
  210. log.Info("Restarting Gitea on PID: %d from parent PID: %d", os.Getpid(), os.Getppid())
  211. } else {
  212. log.Info("Starting Gitea on PID: %d", os.Getpid())
  213. }
  214. // Set pid file setting
  215. if ctx.IsSet("pid") {
  216. createPIDFile(ctx.String("pid"))
  217. }
  218. if !setting.InstallLock {
  219. if err := serveInstall(ctx); err != nil {
  220. return err
  221. }
  222. } else {
  223. NoInstallListener()
  224. }
  225. if setting.EnablePprof {
  226. go servePprof()
  227. }
  228. return serveInstalled(ctx)
  229. }
  230. func setPort(port string) error {
  231. setting.AppURL = strings.Replace(setting.AppURL, setting.HTTPPort, port, 1)
  232. setting.HTTPPort = port
  233. switch setting.Protocol {
  234. case setting.HTTPUnix:
  235. case setting.FCGI:
  236. case setting.FCGIUnix:
  237. default:
  238. defaultLocalURL := string(setting.Protocol) + "://"
  239. if setting.HTTPAddr == "0.0.0.0" {
  240. defaultLocalURL += "localhost"
  241. } else {
  242. defaultLocalURL += setting.HTTPAddr
  243. }
  244. defaultLocalURL += ":" + setting.HTTPPort + "/"
  245. // Save LOCAL_ROOT_URL if port changed
  246. rootCfg := setting.CfgProvider
  247. saveCfg, err := rootCfg.PrepareSaving()
  248. if err != nil {
  249. return fmt.Errorf("failed to save config file: %v", err)
  250. }
  251. rootCfg.Section("server").Key("LOCAL_ROOT_URL").SetValue(defaultLocalURL)
  252. saveCfg.Section("server").Key("LOCAL_ROOT_URL").SetValue(defaultLocalURL)
  253. if err = saveCfg.Save(); err != nil {
  254. return fmt.Errorf("failed to save config file: %v", err)
  255. }
  256. }
  257. return nil
  258. }
  259. func listen(m http.Handler, handleRedirector bool) error {
  260. listenAddr := setting.HTTPAddr
  261. if setting.Protocol != setting.HTTPUnix && setting.Protocol != setting.FCGIUnix {
  262. listenAddr = net.JoinHostPort(listenAddr, setting.HTTPPort)
  263. }
  264. _, _, finished := process.GetManager().AddTypedContext(graceful.GetManager().HammerContext(), "Web: Gitea Server", process.SystemProcessType, true)
  265. defer finished()
  266. log.Info("Listen: %v://%s%s", setting.Protocol, listenAddr, setting.AppSubURL)
  267. // This can be useful for users, many users do wrong to their config and get strange behaviors behind a reverse-proxy.
  268. // A user may fix the configuration mistake when he sees this log.
  269. // And this is also very helpful to maintainers to provide help to users to resolve their configuration problems.
  270. log.Info("AppURL(ROOT_URL): %s", setting.AppURL)
  271. if setting.LFS.StartServer {
  272. log.Info("LFS server enabled")
  273. }
  274. var err error
  275. switch setting.Protocol {
  276. case setting.HTTP:
  277. if handleRedirector {
  278. NoHTTPRedirector()
  279. }
  280. err = runHTTP("tcp", listenAddr, "Web", m, setting.UseProxyProtocol)
  281. case setting.HTTPS:
  282. if setting.EnableAcme {
  283. err = runACME(listenAddr, m)
  284. break
  285. }
  286. if handleRedirector {
  287. if setting.RedirectOtherPort {
  288. go runHTTPRedirector()
  289. } else {
  290. NoHTTPRedirector()
  291. }
  292. }
  293. err = runHTTPS("tcp", listenAddr, "Web", setting.CertFile, setting.KeyFile, m, setting.UseProxyProtocol, setting.ProxyProtocolTLSBridging)
  294. case setting.FCGI:
  295. if handleRedirector {
  296. NoHTTPRedirector()
  297. }
  298. err = runFCGI("tcp", listenAddr, "FCGI Web", m, setting.UseProxyProtocol)
  299. case setting.HTTPUnix:
  300. if handleRedirector {
  301. NoHTTPRedirector()
  302. }
  303. err = runHTTP("unix", listenAddr, "Web", m, setting.UseProxyProtocol)
  304. case setting.FCGIUnix:
  305. if handleRedirector {
  306. NoHTTPRedirector()
  307. }
  308. err = runFCGI("unix", listenAddr, "Web", m, setting.UseProxyProtocol)
  309. default:
  310. log.Fatal("Invalid protocol: %s", setting.Protocol)
  311. }
  312. if err != nil {
  313. log.Critical("Failed to start server: %v", err)
  314. }
  315. log.Info("HTTP Listener: %s Closed", listenAddr)
  316. return err
  317. }