diff options
| author | zeripath <art27@cantab.net> | 2022-03-31 18:01:43 +0100 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2022-03-31 19:01:43 +0200 |
| commit | c88547ce71a554091930e129c20776daf6da35ac (patch) | |
| tree | 9232a7b0f07686698a9adbb51a3d3d72ebeaf12b /modules/web | |
| parent | 9c349a4277926bfd3ff0360301765ad7abd9f10b (diff) | |
| download | gitea-c88547ce71a554091930e129c20776daf6da35ac.tar.gz gitea-c88547ce71a554091930e129c20776daf6da35ac.zip | |
Add Goroutine stack inspector to admin/monitor (#19207)
Continues on from #19202.
Following the addition of pprof labels we can now more easily understand the relationship between a goroutine and the requests that spawn them.
This PR takes advantage of the labels and adds a few others, then provides a mechanism for the monitoring page to query the pprof goroutine profile.
The binary profile that results from this profile is immediately piped in to the google library for parsing this and then stack traces are formed for the goroutines.
If the goroutine is within a context or has been created from a goroutine within a process context it will acquire the process description labels for that process.
The goroutines are mapped with there associate pids and any that do not have an associated pid are placed in a group at the bottom as unbound.
In this way we should be able to more easily examine goroutines that have been stuck.
A manager command `gitea manager processes` is also provided that can export the processes (with or without stacktraces) to the command line.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Diffstat (limited to 'modules/web')
| -rw-r--r-- | modules/web/routing/logger_manager.go | 7 |
1 files changed, 5 insertions, 2 deletions
diff --git a/modules/web/routing/logger_manager.go b/modules/web/routing/logger_manager.go index cc434c338d..7715b0b5d3 100644 --- a/modules/web/routing/logger_manager.go +++ b/modules/web/routing/logger_manager.go @@ -11,6 +11,7 @@ import ( "time" "code.gitea.io/gitea/modules/graceful" + "code.gitea.io/gitea/modules/process" ) // Event indicates when the printer is triggered @@ -40,7 +41,9 @@ type requestRecordsManager struct { } func (manager *requestRecordsManager) startSlowQueryDetector(threshold time.Duration) { - go graceful.GetManager().RunWithShutdownContext(func(baseCtx context.Context) { + go graceful.GetManager().RunWithShutdownContext(func(ctx context.Context) { + ctx, _, finished := process.GetManager().AddTypedContext(ctx, "Service: SlowQueryDetector", process.SystemProcessType, true) + defer finished() // This go-routine checks all active requests every second. // If a request has been running for a long time (eg: /user/events), we also print a log with "still-executing" message // After the "still-executing" log is printed, the record will be removed from the map to prevent from duplicated logs in future @@ -49,7 +52,7 @@ func (manager *requestRecordsManager) startSlowQueryDetector(threshold time.Dura t := time.NewTicker(time.Second) for { select { - case <-baseCtx.Done(): + case <-ctx.Done(): return case <-t.C: now := time.Now() |