mirrors
/
gitea
镜像自地址 https://github.com/go-gitea/gitea.git
关注 1
点赞 0
派生 0
代码 工单 0 版本发布 211 百科 动态
您最多选择25个主题 主题必须以字母或数字开头，可以包含连字符 (-)，并且长度不得超过35个字符
16727 提交
17 分支
目录树: 2360c7ec6c
分支列表 标签列表
${ item.name }
创建分支 ${ searchTerm }
从 '2360c7ec6c'
${ noResults }
gitea/modules/cache/context_test.go

context_test.go 1.9KB
原始文件 普通视图 文件历史

Add context cache as a request level cache (#22294) To avoid duplicated load of the same data in an HTTP request, we can set a context cache to do that. i.e. Some pages may load a user from a database with the same id in different areas on the same page. But the code is hidden in two different deep logic. How should we share the user? As a result of this PR, now if both entry functions accept `context.Context` as the first parameter and we just need to refactor `GetUserByID` to reuse the user from the context cache. Then it will not be loaded twice on an HTTP request. But of course, sometimes we would like to reload an object from the database, that's why `RemoveContextData` is also exposed. The core context cache is here. It defines a new context ```go type cacheContext struct { ctx context.Context data map[any]map[any]any lock sync.RWMutex } var cacheContextKey = struct{}{} func WithCacheContext(ctx context.Context) context.Context { return context.WithValue(ctx, cacheContextKey, &cacheContext{ ctx: ctx, data: make(map[any]map[any]any), }) } ``` Then you can use the below 4 methods to read/write/del the data within the same context. ```go func GetContextData(ctx context.Context, tp, key any) any func SetContextData(ctx context.Context, tp, key, value any) func RemoveContextData(ctx context.Context, tp, key any) func GetWithContextCache[T any](ctx context.Context, cacheGroupKey string, cacheTargetID any, f func() (T, error)) (T, error) ``` Then let's take a look at how `system.GetString` implement it. ```go func GetSetting(ctx context.Context, key string) (string, error) { return cache.GetWithContextCache(ctx, contextCacheKey, key, func() (string, error) { return cache.GetString(genSettingCacheKey(key), func() (string, error) { res, err := GetSettingNoCache(ctx, key) if err != nil { return "", err } return res.SettingValue, nil }) }) } ``` First, it will check if context data include the setting object with the key. If not, it will query from the global cache which may be memory or a Redis cache. If not, it will get the object from the database. In the end, if the object gets from the global cache or database, it will be set into the context cache. An object stored in the context cache will only be destroyed after the context disappeared.
1年前
​
​
​
​
​
​
​
Improve cache context (#23330) Related to: #22294 #23186 #23054 Replace: #23218 Some discussion is in the comments of #23218. Highlights: - Add Expiration for cache context. If a cache context has been used for more than 10s, the cache data will be ignored, and warning logs will be printed. - Add `discard` field to `cacheContext`, a `cacheContext` with `discard` true will drop all cached data and won't store any new one. - Introduce `WithNoCacheContext`, if one wants to run long-life tasks, but the parent context is a cache context, `WithNoCacheContext(perentCtx)` will discard the cache data, so it will be safe to keep the context for a long time. - It will be fine to treat an original context as a cache context, like `GetContextData(context.Backgraud())`, no warning logs will be printed. Some cases about nesting: When: - *A*, *B* or *C* means a cache context. - ~*A*~, ~*B*~ or ~*C*~ means a discard cache context. - `ctx` means `context.Backgrand()` - *A(ctx)* means a cache context with `ctx` as the parent context. - *B(A(ctx))* means a cache context with `A(ctx)` as the parent context. - `With` means `WithCacheContext` - `WithNo` means `WithNoCacheContext` So: - `With(ctx)` -> *A(ctx)* - `With(With(ctx))` -> *A(ctx)*, not *B(A(ctx))* - `With(With(With(ctx)))` -> *A(ctx)*, not *C(B(A(ctx)))* - `WithNo(ctx)` -> *ctx*, not *~A~(ctx)* - `WithNo(With(ctx))` -> *~A~(ctx)* - `WithNo(WithNo(With(ctx)))` -> *~A~(ctx)*, not *~B~(~A~(ctx))* - `With(WithNo(With(ctx)))` -> *B(~A~(ctx))* - `WithNo(With(WithNo(With(ctx))))` -> *~B~(~A~(ctx))* - `With(WithNo(With(WithNo(With(ctx)))))` -> *C(~B~(~A~(ctx)))*
1年前
Add context cache as a request level cache (#22294) To avoid duplicated load of the same data in an HTTP request, we can set a context cache to do that. i.e. Some pages may load a user from a database with the same id in different areas on the same page. But the code is hidden in two different deep logic. How should we share the user? As a result of this PR, now if both entry functions accept `context.Context` as the first parameter and we just need to refactor `GetUserByID` to reuse the user from the context cache. Then it will not be loaded twice on an HTTP request. But of course, sometimes we would like to reload an object from the database, that's why `RemoveContextData` is also exposed. The core context cache is here. It defines a new context ```go type cacheContext struct { ctx context.Context data map[any]map[any]any lock sync.RWMutex } var cacheContextKey = struct{}{} func WithCacheContext(ctx context.Context) context.Context { return context.WithValue(ctx, cacheContextKey, &cacheContext{ ctx: ctx, data: make(map[any]map[any]any), }) } ``` Then you can use the below 4 methods to read/write/del the data within the same context. ```go func GetContextData(ctx context.Context, tp, key any) any func SetContextData(ctx context.Context, tp, key, value any) func RemoveContextData(ctx context.Context, tp, key any) func GetWithContextCache[T any](ctx context.Context, cacheGroupKey string, cacheTargetID any, f func() (T, error)) (T, error) ``` Then let's take a look at how `system.GetString` implement it. ```go func GetSetting(ctx context.Context, key string) (string, error) { return cache.GetWithContextCache(ctx, contextCacheKey, key, func() (string, error) { return cache.GetString(genSettingCacheKey(key), func() (string, error) { res, err := GetSettingNoCache(ctx, key) if err != nil { return "", err } return res.SettingValue, nil }) }) } ``` First, it will check if context data include the setting object with the key. If not, it will query from the global cache which may be memory or a Redis cache. If not, it will get the object from the database. In the end, if the object gets from the global cache or database, it will be set into the context cache. An object stored in the context cache will only be destroyed after the context disappeared.
1年前
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
Improve cache context (#23330) Related to: #22294 #23186 #23054 Replace: #23218 Some discussion is in the comments of #23218. Highlights: - Add Expiration for cache context. If a cache context has been used for more than 10s, the cache data will be ignored, and warning logs will be printed. - Add `discard` field to `cacheContext`, a `cacheContext` with `discard` true will drop all cached data and won't store any new one. - Introduce `WithNoCacheContext`, if one wants to run long-life tasks, but the parent context is a cache context, `WithNoCacheContext(perentCtx)` will discard the cache data, so it will be safe to keep the context for a long time. - It will be fine to treat an original context as a cache context, like `GetContextData(context.Backgraud())`, no warning logs will be printed. Some cases about nesting: When: - *A*, *B* or *C* means a cache context. - ~*A*~, ~*B*~ or ~*C*~ means a discard cache context. - `ctx` means `context.Backgrand()` - *A(ctx)* means a cache context with `ctx` as the parent context. - *B(A(ctx))* means a cache context with `A(ctx)` as the parent context. - `With` means `WithCacheContext` - `WithNo` means `WithNoCacheContext` So: - `With(ctx)` -> *A(ctx)* - `With(With(ctx))` -> *A(ctx)*, not *B(A(ctx))* - `With(With(With(ctx)))` -> *A(ctx)*, not *C(B(A(ctx)))* - `WithNo(ctx)` -> *ctx*, not *~A~(ctx)* - `WithNo(With(ctx))` -> *~A~(ctx)* - `WithNo(WithNo(With(ctx)))` -> *~A~(ctx)*, not *~B~(~A~(ctx))* - `With(WithNo(With(ctx)))` -> *B(~A~(ctx))* - `WithNo(With(WithNo(With(ctx))))` -> *~B~(~A~(ctx))* - `With(WithNo(With(WithNo(With(ctx)))))` -> *C(~B~(~A~(ctx)))*
1年前
Add context cache as a request level cache (#22294) To avoid duplicated load of the same data in an HTTP request, we can set a context cache to do that. i.e. Some pages may load a user from a database with the same id in different areas on the same page. But the code is hidden in two different deep logic. How should we share the user? As a result of this PR, now if both entry functions accept `context.Context` as the first parameter and we just need to refactor `GetUserByID` to reuse the user from the context cache. Then it will not be loaded twice on an HTTP request. But of course, sometimes we would like to reload an object from the database, that's why `RemoveContextData` is also exposed. The core context cache is here. It defines a new context ```go type cacheContext struct { ctx context.Context data map[any]map[any]any lock sync.RWMutex } var cacheContextKey = struct{}{} func WithCacheContext(ctx context.Context) context.Context { return context.WithValue(ctx, cacheContextKey, &cacheContext{ ctx: ctx, data: make(map[any]map[any]any), }) } ``` Then you can use the below 4 methods to read/write/del the data within the same context. ```go func GetContextData(ctx context.Context, tp, key any) any func SetContextData(ctx context.Context, tp, key, value any) func RemoveContextData(ctx context.Context, tp, key any) func GetWithContextCache[T any](ctx context.Context, cacheGroupKey string, cacheTargetID any, f func() (T, error)) (T, error) ``` Then let's take a look at how `system.GetString` implement it. ```go func GetSetting(ctx context.Context, key string) (string, error) { return cache.GetWithContextCache(ctx, contextCacheKey, key, func() (string, error) { return cache.GetString(genSettingCacheKey(key), func() (string, error) { res, err := GetSettingNoCache(ctx, key) if err != nil { return "", err } return res.SettingValue, nil }) }) } ``` First, it will check if context data include the setting object with the key. If not, it will query from the global cache which may be memory or a Redis cache. If not, it will get the object from the database. In the end, if the object gets from the global cache or database, it will be set into the context cache. An object stored in the context cache will only be destroyed after the context disappeared.
1年前
​
​
​
​
​
​
​
​
​
​
​
Improve cache context (#23330) Related to: #22294 #23186 #23054 Replace: #23218 Some discussion is in the comments of #23218. Highlights: - Add Expiration for cache context. If a cache context has been used for more than 10s, the cache data will be ignored, and warning logs will be printed. - Add `discard` field to `cacheContext`, a `cacheContext` with `discard` true will drop all cached data and won't store any new one. - Introduce `WithNoCacheContext`, if one wants to run long-life tasks, but the parent context is a cache context, `WithNoCacheContext(perentCtx)` will discard the cache data, so it will be safe to keep the context for a long time. - It will be fine to treat an original context as a cache context, like `GetContextData(context.Backgraud())`, no warning logs will be printed. Some cases about nesting: When: - *A*, *B* or *C* means a cache context. - ~*A*~, ~*B*~ or ~*C*~ means a discard cache context. - `ctx` means `context.Backgrand()` - *A(ctx)* means a cache context with `ctx` as the parent context. - *B(A(ctx))* means a cache context with `A(ctx)` as the parent context. - `With` means `WithCacheContext` - `WithNo` means `WithNoCacheContext` So: - `With(ctx)` -> *A(ctx)* - `With(With(ctx))` -> *A(ctx)*, not *B(A(ctx))* - `With(With(With(ctx)))` -> *A(ctx)*, not *C(B(A(ctx)))* - `WithNo(ctx)` -> *ctx*, not *~A~(ctx)* - `WithNo(With(ctx))` -> *~A~(ctx)* - `WithNo(WithNo(With(ctx)))` -> *~A~(ctx)*, not *~B~(~A~(ctx))* - `With(WithNo(With(ctx)))` -> *B(~A~(ctx))* - `WithNo(With(WithNo(With(ctx))))` -> *~B~(~A~(ctx))* - `With(WithNo(With(WithNo(With(ctx)))))` -> *C(~B~(~A~(ctx)))*
1年前
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
Add context cache as a request level cache (#22294) To avoid duplicated load of the same data in an HTTP request, we can set a context cache to do that. i.e. Some pages may load a user from a database with the same id in different areas on the same page. But the code is hidden in two different deep logic. How should we share the user? As a result of this PR, now if both entry functions accept `context.Context` as the first parameter and we just need to refactor `GetUserByID` to reuse the user from the context cache. Then it will not be loaded twice on an HTTP request. But of course, sometimes we would like to reload an object from the database, that's why `RemoveContextData` is also exposed. The core context cache is here. It defines a new context ```go type cacheContext struct { ctx context.Context data map[any]map[any]any lock sync.RWMutex } var cacheContextKey = struct{}{} func WithCacheContext(ctx context.Context) context.Context { return context.WithValue(ctx, cacheContextKey, &cacheContext{ ctx: ctx, data: make(map[any]map[any]any), }) } ``` Then you can use the below 4 methods to read/write/del the data within the same context. ```go func GetContextData(ctx context.Context, tp, key any) any func SetContextData(ctx context.Context, tp, key, value any) func RemoveContextData(ctx context.Context, tp, key any) func GetWithContextCache[T any](ctx context.Context, cacheGroupKey string, cacheTargetID any, f func() (T, error)) (T, error) ``` Then let's take a look at how `system.GetString` implement it. ```go func GetSetting(ctx context.Context, key string) (string, error) { return cache.GetWithContextCache(ctx, contextCacheKey, key, func() (string, error) { return cache.GetString(genSettingCacheKey(key), func() (string, error) { res, err := GetSettingNoCache(ctx, key) if err != nil { return "", err } return res.SettingValue, nil }) }) } ``` First, it will check if context data include the setting object with the key. If not, it will query from the global cache which may be memory or a Redis cache. If not, it will get the object from the database. In the end, if the object gets from the global cache or database, it will be set into the context cache. An object stored in the context cache will only be destroyed after the context disappeared.
1年前
 123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778 
  1. // Copyright 2022 The Gitea Authors. All rights reserved.

  2. // SPDX-License-Identifier: MIT

  3. 

  4. package cache

  5. 

  6. import (

  7. 	"context"

  8. 	"testing"

  9. 	"time"

  10. 

  11. 	"github.com/stretchr/testify/assert"

  12. )

  13. 

  14. func TestWithCacheContext(t *testing.T) {

  15. 	ctx := WithCacheContext(context.Background())

  16. 

  17. 	v := GetContextData(ctx, "empty_field", "my_config1")

  18. 	assert.Nil(t, v)

  19. 

  20. 	const field = "system_setting"

  21. 	v = GetContextData(ctx, field, "my_config1")

  22. 	assert.Nil(t, v)

  23. 	SetContextData(ctx, field, "my_config1", 1)

  24. 	v = GetContextData(ctx, field, "my_config1")

  25. 	assert.NotNil(t, v)

  26. 	assert.EqualValues(t, 1, v.(int))

  27. 

  28. 	RemoveContextData(ctx, field, "my_config1")

  29. 	RemoveContextData(ctx, field, "my_config2") // remove a non-exist key

  30. 

  31. 	v = GetContextData(ctx, field, "my_config1")

  32. 	assert.Nil(t, v)

  33. 

  34. 	vInt, err := GetWithContextCache(ctx, field, "my_config1", func() (int, error) {

  35. 		return 1, nil

  36. 	})

  37. 	assert.NoError(t, err)

  38. 	assert.EqualValues(t, 1, vInt)

  39. 

  40. 	v = GetContextData(ctx, field, "my_config1")

  41. 	assert.EqualValues(t, 1, v)

  42. 

  43. 	now := timeNow

  44. 	defer func() {

  45. 		timeNow = now

  46. 	}()

  47. 	timeNow = func() time.Time {

  48. 		return now().Add(10 * time.Second)

  49. 	}

  50. 	v = GetContextData(ctx, field, "my_config1")

  51. 	assert.Nil(t, v)

  52. }

  53. 

  54. func TestWithNoCacheContext(t *testing.T) {

  55. 	ctx := context.Background()

  56. 

  57. 	const field = "system_setting"

  58. 

  59. 	v := GetContextData(ctx, field, "my_config1")

  60. 	assert.Nil(t, v)

  61. 	SetContextData(ctx, field, "my_config1", 1)

  62. 	v = GetContextData(ctx, field, "my_config1")

  63. 	assert.Nil(t, v) // still no cache

  64. 

  65. 	ctx = WithCacheContext(ctx)

  66. 	v = GetContextData(ctx, field, "my_config1")

  67. 	assert.Nil(t, v)

  68. 	SetContextData(ctx, field, "my_config1", 1)

  69. 	v = GetContextData(ctx, field, "my_config1")

  70. 	assert.NotNil(t, v)

  71. 

  72. 	ctx = WithNoCacheContext(ctx)

  73. 	v = GetContextData(ctx, field, "my_config1")

  74. 	assert.Nil(t, v)

  75. 	SetContextData(ctx, field, "my_config1", 1)

  76. 	v = GetContextData(ctx, field, "my_config1")

  77. 	assert.Nil(t, v) // still no cache

  78. }