You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Rewrite logger system (#24726) ## ⚠️ Breaking The `log.<mode>.<logger>` style config has been dropped. If you used it, please check the new config manual & app.example.ini to make your instance output logs as expected. Although many legacy options still work, it's encouraged to upgrade to the new options. The SMTP logger is deleted because SMTP is not suitable to collect logs. If you have manually configured Gitea log options, please confirm the logger system works as expected after upgrading. ## Description Close #12082 and maybe more log-related issues, resolve some related FIXMEs in old code (which seems unfixable before) Just like rewriting queue #24505 : make code maintainable, clear legacy bugs, and add the ability to support more writers (eg: JSON, structured log) There is a new document (with examples): `logging-config.en-us.md` This PR is safer than the queue rewriting, because it's just for logging, it won't break other logic. ## The old problems The logging system is quite old and difficult to maintain: * Unclear concepts: Logger, NamedLogger, MultiChannelledLogger, SubLogger, EventLogger, WriterLogger etc * Some code is diffuclt to konw whether it is right: `log.DelNamedLogger("console")` vs `log.DelNamedLogger(log.DEFAULT)` vs `log.DelLogger("console")` * The old system heavily depends on ini config system, it's difficult to create new logger for different purpose, and it's very fragile. * The "color" trick is difficult to use and read, many colors are unnecessary, and in the future structured log could help * It's difficult to add other log formats, eg: JSON format * The log outputer doesn't have full control of its goroutine, it's difficult to make outputer have advanced behaviors * The logs could be lost in some cases: eg: no Fatal error when using CLI. * Config options are passed by JSON, which is quite fragile. * INI package makes the KEY in `[log]` section visible in `[log.sub1]` and `[log.sub1.subA]`, this behavior is quite fragile and would cause more unclear problems, and there is no strong requirement to support `log.<mode>.<logger>` syntax. ## The new design See `logger.go` for documents. ## Screenshot <details> ![image](https://github.com/go-gitea/gitea/assets/2114189/4462d713-ba39-41f5-bb08-de912e67e1ff) ![image](https://github.com/go-gitea/gitea/assets/2114189/b188035e-f691-428b-8b2d-ff7b2199b2f9) ![image](https://github.com/go-gitea/gitea/assets/2114189/132e9745-1c3b-4e00-9e0d-15eaea495dee) </details> ## TODO * [x] add some new tests * [x] fix some tests * [x] test some sub-commands (manually ....) --------- Co-authored-by: Jason Song <i@wolfogre.com> Co-authored-by: delvh <dev.lh@web.de> Co-authored-by: Giteabot <teabot@gitea.io>
1 anno fa
Rewrite logger system (#24726) ## ⚠️ Breaking The `log.<mode>.<logger>` style config has been dropped. If you used it, please check the new config manual & app.example.ini to make your instance output logs as expected. Although many legacy options still work, it's encouraged to upgrade to the new options. The SMTP logger is deleted because SMTP is not suitable to collect logs. If you have manually configured Gitea log options, please confirm the logger system works as expected after upgrading. ## Description Close #12082 and maybe more log-related issues, resolve some related FIXMEs in old code (which seems unfixable before) Just like rewriting queue #24505 : make code maintainable, clear legacy bugs, and add the ability to support more writers (eg: JSON, structured log) There is a new document (with examples): `logging-config.en-us.md` This PR is safer than the queue rewriting, because it's just for logging, it won't break other logic. ## The old problems The logging system is quite old and difficult to maintain: * Unclear concepts: Logger, NamedLogger, MultiChannelledLogger, SubLogger, EventLogger, WriterLogger etc * Some code is diffuclt to konw whether it is right: `log.DelNamedLogger("console")` vs `log.DelNamedLogger(log.DEFAULT)` vs `log.DelLogger("console")` * The old system heavily depends on ini config system, it's difficult to create new logger for different purpose, and it's very fragile. * The "color" trick is difficult to use and read, many colors are unnecessary, and in the future structured log could help * It's difficult to add other log formats, eg: JSON format * The log outputer doesn't have full control of its goroutine, it's difficult to make outputer have advanced behaviors * The logs could be lost in some cases: eg: no Fatal error when using CLI. * Config options are passed by JSON, which is quite fragile. * INI package makes the KEY in `[log]` section visible in `[log.sub1]` and `[log.sub1.subA]`, this behavior is quite fragile and would cause more unclear problems, and there is no strong requirement to support `log.<mode>.<logger>` syntax. ## The new design See `logger.go` for documents. ## Screenshot <details> ![image](https://github.com/go-gitea/gitea/assets/2114189/4462d713-ba39-41f5-bb08-de912e67e1ff) ![image](https://github.com/go-gitea/gitea/assets/2114189/b188035e-f691-428b-8b2d-ff7b2199b2f9) ![image](https://github.com/go-gitea/gitea/assets/2114189/132e9745-1c3b-4e00-9e0d-15eaea495dee) </details> ## TODO * [x] add some new tests * [x] fix some tests * [x] test some sub-commands (manually ....) --------- Co-authored-by: Jason Song <i@wolfogre.com> Co-authored-by: delvh <dev.lh@web.de> Co-authored-by: Giteabot <teabot@gitea.io>
1 anno fa
Rewrite logger system (#24726) ## ⚠️ Breaking The `log.<mode>.<logger>` style config has been dropped. If you used it, please check the new config manual & app.example.ini to make your instance output logs as expected. Although many legacy options still work, it's encouraged to upgrade to the new options. The SMTP logger is deleted because SMTP is not suitable to collect logs. If you have manually configured Gitea log options, please confirm the logger system works as expected after upgrading. ## Description Close #12082 and maybe more log-related issues, resolve some related FIXMEs in old code (which seems unfixable before) Just like rewriting queue #24505 : make code maintainable, clear legacy bugs, and add the ability to support more writers (eg: JSON, structured log) There is a new document (with examples): `logging-config.en-us.md` This PR is safer than the queue rewriting, because it's just for logging, it won't break other logic. ## The old problems The logging system is quite old and difficult to maintain: * Unclear concepts: Logger, NamedLogger, MultiChannelledLogger, SubLogger, EventLogger, WriterLogger etc * Some code is diffuclt to konw whether it is right: `log.DelNamedLogger("console")` vs `log.DelNamedLogger(log.DEFAULT)` vs `log.DelLogger("console")` * The old system heavily depends on ini config system, it's difficult to create new logger for different purpose, and it's very fragile. * The "color" trick is difficult to use and read, many colors are unnecessary, and in the future structured log could help * It's difficult to add other log formats, eg: JSON format * The log outputer doesn't have full control of its goroutine, it's difficult to make outputer have advanced behaviors * The logs could be lost in some cases: eg: no Fatal error when using CLI. * Config options are passed by JSON, which is quite fragile. * INI package makes the KEY in `[log]` section visible in `[log.sub1]` and `[log.sub1.subA]`, this behavior is quite fragile and would cause more unclear problems, and there is no strong requirement to support `log.<mode>.<logger>` syntax. ## The new design See `logger.go` for documents. ## Screenshot <details> ![image](https://github.com/go-gitea/gitea/assets/2114189/4462d713-ba39-41f5-bb08-de912e67e1ff) ![image](https://github.com/go-gitea/gitea/assets/2114189/b188035e-f691-428b-8b2d-ff7b2199b2f9) ![image](https://github.com/go-gitea/gitea/assets/2114189/132e9745-1c3b-4e00-9e0d-15eaea495dee) </details> ## TODO * [x] add some new tests * [x] fix some tests * [x] test some sub-commands (manually ....) --------- Co-authored-by: Jason Song <i@wolfogre.com> Co-authored-by: delvh <dev.lh@web.de> Co-authored-by: Giteabot <teabot@gitea.io>
1 anno fa
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275
  1. // Copyright 2018 The Gitea Authors. All rights reserved.
  2. // Copyright 2016 The Gogs Authors. All rights reserved.
  3. // SPDX-License-Identifier: MIT
  4. package organization
  5. import (
  6. "context"
  7. "fmt"
  8. "strings"
  9. "code.gitea.io/gitea/models/db"
  10. "code.gitea.io/gitea/models/perm"
  11. repo_model "code.gitea.io/gitea/models/repo"
  12. "code.gitea.io/gitea/models/unit"
  13. user_model "code.gitea.io/gitea/models/user"
  14. "code.gitea.io/gitea/modules/log"
  15. "code.gitea.io/gitea/modules/util"
  16. "xorm.io/builder"
  17. )
  18. // ___________
  19. // \__ ___/___ _____ _____
  20. // | |_/ __ \\__ \ / \
  21. // | |\ ___/ / __ \| Y Y \
  22. // |____| \___ >____ /__|_| /
  23. // \/ \/ \/
  24. // ErrTeamAlreadyExist represents a "TeamAlreadyExist" kind of error.
  25. type ErrTeamAlreadyExist struct {
  26. OrgID int64
  27. Name string
  28. }
  29. // IsErrTeamAlreadyExist checks if an error is a ErrTeamAlreadyExist.
  30. func IsErrTeamAlreadyExist(err error) bool {
  31. _, ok := err.(ErrTeamAlreadyExist)
  32. return ok
  33. }
  34. func (err ErrTeamAlreadyExist) Error() string {
  35. return fmt.Sprintf("team already exists [org_id: %d, name: %s]", err.OrgID, err.Name)
  36. }
  37. func (err ErrTeamAlreadyExist) Unwrap() error {
  38. return util.ErrAlreadyExist
  39. }
  40. // ErrTeamNotExist represents a "TeamNotExist" error
  41. type ErrTeamNotExist struct {
  42. OrgID int64
  43. TeamID int64
  44. Name string
  45. }
  46. // IsErrTeamNotExist checks if an error is a ErrTeamNotExist.
  47. func IsErrTeamNotExist(err error) bool {
  48. _, ok := err.(ErrTeamNotExist)
  49. return ok
  50. }
  51. func (err ErrTeamNotExist) Error() string {
  52. return fmt.Sprintf("team does not exist [org_id %d, team_id %d, name: %s]", err.OrgID, err.TeamID, err.Name)
  53. }
  54. func (err ErrTeamNotExist) Unwrap() error {
  55. return util.ErrNotExist
  56. }
  57. // OwnerTeamName return the owner team name
  58. const OwnerTeamName = "Owners"
  59. // Team represents a organization team.
  60. type Team struct {
  61. ID int64 `xorm:"pk autoincr"`
  62. OrgID int64 `xorm:"INDEX"`
  63. LowerName string
  64. Name string
  65. Description string
  66. AccessMode perm.AccessMode `xorm:"'authorize'"`
  67. Repos []*repo_model.Repository `xorm:"-"`
  68. Members []*user_model.User `xorm:"-"`
  69. NumRepos int
  70. NumMembers int
  71. Units []*TeamUnit `xorm:"-"`
  72. IncludesAllRepositories bool `xorm:"NOT NULL DEFAULT false"`
  73. CanCreateOrgRepo bool `xorm:"NOT NULL DEFAULT false"`
  74. }
  75. func init() {
  76. db.RegisterModel(new(Team))
  77. db.RegisterModel(new(TeamUser))
  78. db.RegisterModel(new(TeamRepo))
  79. db.RegisterModel(new(TeamUnit))
  80. db.RegisterModel(new(TeamInvite))
  81. }
  82. func (t *Team) LogString() string {
  83. if t == nil {
  84. return "<Team nil>"
  85. }
  86. return fmt.Sprintf("<Team %d:%s OrgID=%d AccessMode=%s>", t.ID, t.Name, t.OrgID, t.AccessMode.LogString())
  87. }
  88. // LoadUnits load a list of available units for a team
  89. func (t *Team) LoadUnits(ctx context.Context) (err error) {
  90. if t.Units != nil {
  91. return nil
  92. }
  93. t.Units, err = getUnitsByTeamID(ctx, t.ID)
  94. return err
  95. }
  96. // GetUnitNames returns the team units names
  97. func (t *Team) GetUnitNames() (res []string) {
  98. if t.AccessMode >= perm.AccessModeAdmin {
  99. return unit.AllUnitKeyNames()
  100. }
  101. for _, u := range t.Units {
  102. res = append(res, unit.Units[u.Type].NameKey)
  103. }
  104. return res
  105. }
  106. // GetUnitsMap returns the team units permissions
  107. func (t *Team) GetUnitsMap() map[string]string {
  108. m := make(map[string]string)
  109. if t.AccessMode >= perm.AccessModeAdmin {
  110. for _, u := range unit.Units {
  111. m[u.NameKey] = t.AccessMode.ToString()
  112. }
  113. } else {
  114. for _, u := range t.Units {
  115. m[u.Unit().NameKey] = u.AccessMode.ToString()
  116. }
  117. }
  118. return m
  119. }
  120. // IsOwnerTeam returns true if team is owner team.
  121. func (t *Team) IsOwnerTeam() bool {
  122. return t.Name == OwnerTeamName
  123. }
  124. // IsMember returns true if given user is a member of team.
  125. func (t *Team) IsMember(ctx context.Context, userID int64) bool {
  126. isMember, err := IsTeamMember(ctx, t.OrgID, t.ID, userID)
  127. if err != nil {
  128. log.Error("IsMember: %v", err)
  129. return false
  130. }
  131. return isMember
  132. }
  133. // LoadRepositories returns paginated repositories in team of organization.
  134. func (t *Team) LoadRepositories(ctx context.Context) (err error) {
  135. if t.Repos != nil {
  136. return nil
  137. }
  138. t.Repos, err = GetTeamRepositories(ctx, &SearchTeamRepoOptions{
  139. TeamID: t.ID,
  140. })
  141. return err
  142. }
  143. // LoadMembers returns paginated members in team of organization.
  144. func (t *Team) LoadMembers(ctx context.Context) (err error) {
  145. t.Members, err = GetTeamMembers(ctx, &SearchMembersOptions{
  146. TeamID: t.ID,
  147. })
  148. return err
  149. }
  150. // UnitEnabled returns true if the team has the given unit type enabled
  151. func (t *Team) UnitEnabled(ctx context.Context, tp unit.Type) bool {
  152. return t.UnitAccessMode(ctx, tp) > perm.AccessModeNone
  153. }
  154. // UnitAccessMode returns the access mode for the given unit type, "none" for non-existent units
  155. func (t *Team) UnitAccessMode(ctx context.Context, tp unit.Type) perm.AccessMode {
  156. accessMode, _ := t.UnitAccessModeEx(ctx, tp)
  157. return accessMode
  158. }
  159. func (t *Team) UnitAccessModeEx(ctx context.Context, tp unit.Type) (accessMode perm.AccessMode, exist bool) {
  160. if err := t.LoadUnits(ctx); err != nil {
  161. log.Warn("Error loading team (ID: %d) units: %s", t.ID, err.Error())
  162. }
  163. for _, u := range t.Units {
  164. if u.Type == tp {
  165. return u.AccessMode, true
  166. }
  167. }
  168. return perm.AccessModeNone, false
  169. }
  170. // IsUsableTeamName tests if a name could be as team name
  171. func IsUsableTeamName(name string) error {
  172. switch name {
  173. case "new":
  174. return db.ErrNameReserved{Name: name}
  175. default:
  176. return nil
  177. }
  178. }
  179. // GetTeam returns team by given team name and organization.
  180. func GetTeam(ctx context.Context, orgID int64, name string) (*Team, error) {
  181. t, exist, err := db.Get[Team](ctx, builder.Eq{"org_id": orgID, "lower_name": strings.ToLower(name)})
  182. if err != nil {
  183. return nil, err
  184. } else if !exist {
  185. return nil, ErrTeamNotExist{orgID, 0, name}
  186. }
  187. return t, nil
  188. }
  189. // GetTeamIDsByNames returns a slice of team ids corresponds to names.
  190. func GetTeamIDsByNames(ctx context.Context, orgID int64, names []string, ignoreNonExistent bool) ([]int64, error) {
  191. ids := make([]int64, 0, len(names))
  192. for _, name := range names {
  193. u, err := GetTeam(ctx, orgID, name)
  194. if err != nil {
  195. if ignoreNonExistent {
  196. continue
  197. } else {
  198. return nil, err
  199. }
  200. }
  201. ids = append(ids, u.ID)
  202. }
  203. return ids, nil
  204. }
  205. // GetOwnerTeam returns team by given team name and organization.
  206. func GetOwnerTeam(ctx context.Context, orgID int64) (*Team, error) {
  207. return GetTeam(ctx, orgID, OwnerTeamName)
  208. }
  209. // GetTeamByID returns team by given ID.
  210. func GetTeamByID(ctx context.Context, teamID int64) (*Team, error) {
  211. t := new(Team)
  212. has, err := db.GetEngine(ctx).ID(teamID).Get(t)
  213. if err != nil {
  214. return nil, err
  215. } else if !has {
  216. return nil, ErrTeamNotExist{0, teamID, ""}
  217. }
  218. return t, nil
  219. }
  220. // GetTeamNamesByID returns team's lower name from a list of team ids.
  221. func GetTeamNamesByID(ctx context.Context, teamIDs []int64) ([]string, error) {
  222. if len(teamIDs) == 0 {
  223. return []string{}, nil
  224. }
  225. var teamNames []string
  226. err := db.GetEngine(ctx).Table("team").
  227. Select("lower_name").
  228. In("id", teamIDs).
  229. Asc("name").
  230. Find(&teamNames)
  231. return teamNames, err
  232. }
  233. // IncrTeamRepoNum increases the number of repos for the given team by 1
  234. func IncrTeamRepoNum(ctx context.Context, teamID int64) error {
  235. _, err := db.GetEngine(ctx).Incr("num_repos").ID(teamID).Update(new(Team))
  236. return err
  237. }