* move migrating to backend * add loading image when migrating and fix tests * fix format * fix lint * add redis task queue support and improve docs * add redis vendor * fix vet * add database migrations and fix app.ini sample * add comments for task section on app.ini.sample * Update models/migrations/v84.go Co-Authored-By: lunny <xiaolunwen@gmail.com> * Update models/repo.go Co-Authored-By: lunny <xiaolunwen@gmail.com> * move migrating to backend * add loading image when migrating and fix tests * fix fmt * add redis task queue support and improve docs * fix fixtures * fix fixtures * fix duplicate function on index.js * fix tests * rename repository statuses * check if repository is being create when SSH request * fix lint * fix template * some improvements * fix template * unified migrate options * fix lint * fix loading page * refactor * When gitea restart, don't restart the running tasks because we may have servel gitea instances, that may break the migration * fix js * Update models/repo.go Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com> * Update docs/content/doc/advanced/config-cheat-sheet.en-us.md Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com> * fix tests * rename ErrTaskIsNotExist to ErrTaskDoesNotExist * delete release after add one on tests to make it run happy * fix tests * fix tests * improve codes * fix lint * fix lint * fix migrationstags/v1.10.0-rc1
*.snap | *.snap | ||||
*.snap-build | *.snap-build | ||||
*_source.tar.bz2 | *_source.tar.bz2 | ||||
.DS_Store |
ENABLED = false | ENABLED = false | ||||
; If you want to add authorization, specify a token here | ; If you want to add authorization, specify a token here | ||||
TOKEN = | TOKEN = | ||||
[task] | |||||
; Task queue type, could be `channel` or `redis`. | |||||
QUEUE_TYPE = channel | |||||
; Task queue length, available only when `QUEUE_TYPE` is `channel`. | |||||
QUEUE_LENGTH = 1000 | |||||
; Task queue connction string, available only when `QUEUE_TYPE` is `redis`. | |||||
; If there is a password of redis, use `addrs=127.0.0.1:6379 password=123 db=0`. | |||||
QUEUE_CONN_STR = "addrs=127.0.0.1:6379 db=0" |
- `GITEA_PREFIX_RAW`, which contains the current URL prefix in the `raw` path tree. To be used as prefix for image paths. | - `GITEA_PREFIX_RAW`, which contains the current URL prefix in the `raw` path tree. To be used as prefix for image paths. | ||||
## Time (`time`) | ## Time (`time`) | ||||
- `FORMAT`: Time format to diplay on UI. i.e. RFC1123 or 2006-01-02 15:04:05 | - `FORMAT`: Time format to diplay on UI. i.e. RFC1123 or 2006-01-02 15:04:05 | ||||
- `DEFAULT_UI_LOCATION`: Default location of time on the UI, so that we can display correct user's time on UI. i.e. Shanghai/Asia | - `DEFAULT_UI_LOCATION`: Default location of time on the UI, so that we can display correct user's time on UI. i.e. Shanghai/Asia | ||||
## Task (`task`) | |||||
- `QUEUE_TYPE`: **channel**: Task queue type, could be `channel` or `redis`. | |||||
- `QUEUE_LENGTH`: **1000**: Task queue length, available only when `QUEUE_TYPE` is `channel`. | |||||
- `QUEUE_CONN_STR`: **addrs=127.0.0.1:6379 db=0**: Task queue connection string, available only when `QUEUE_TYPE` is `redis`. If there redis needs a password, use `addrs=127.0.0.1:6379 password=123 db=0`. | |||||
## Other (`other`) | ## Other (`other`) | ||||
- `SHOW_FOOTER_BRANDING`: **false**: Show Gitea branding in the footer. | - `SHOW_FOOTER_BRANDING`: **false**: Show Gitea branding in the footer. |
- IS_INPUT_FILE: 输入方式是最后一个参数为文件路径还是从标准输入读取。 | - IS_INPUT_FILE: 输入方式是最后一个参数为文件路径还是从标准输入读取。 | ||||
## Time (`time`) | ## Time (`time`) | ||||
- `FORMAT`: 显示在界面上的时间格式。比如: RFC1123 或者 2006-01-02 15:04:05 | - `FORMAT`: 显示在界面上的时间格式。比如: RFC1123 或者 2006-01-02 15:04:05 | ||||
- `DEFAULT_UI_LOCATION`: 默认显示在界面上的时区,默认为本地时区。比如: Asia/Shanghai | - `DEFAULT_UI_LOCATION`: 默认显示在界面上的时区,默认为本地时区。比如: Asia/Shanghai | ||||
## Task (`task`) | |||||
- `QUEUE_TYPE`: **channel**: 任务队列类型,可以为 `channel` 或 `redis`。 | |||||
- `QUEUE_LENGTH`: **1000**: 任务队列长度,当 `QUEUE_TYPE` 为 `channel` 时有效。 | |||||
- `QUEUE_CONN_STR`: **addrs=127.0.0.1:6379 db=0**: 任务队列连接字符串,当 `QUEUE_TYPE` 为 `redis` 时有效。如果redis有密码,则可以 `addrs=127.0.0.1:6379 password=123 db=0`。 | |||||
## Other (`other`) | ## Other (`other`) | ||||
- `SHOW_FOOTER_BRANDING`: 为真则在页面底部显示Gitea的字样。 | - `SHOW_FOOTER_BRANDING`: 为真则在页面底部显示Gitea的字样。 |
num_milestones: 3 | num_milestones: 3 | ||||
num_closed_milestones: 1 | num_closed_milestones: 1 | ||||
num_watches: 3 | num_watches: 3 | ||||
status: 0 | |||||
- | - | ||||
id: 2 | id: 2 | ||||
num_closed_pulls: 0 | num_closed_pulls: 0 | ||||
num_stars: 1 | num_stars: 1 | ||||
close_issues_via_commit_in_any_branch: true | close_issues_via_commit_in_any_branch: true | ||||
status: 0 | |||||
- | - | ||||
id: 3 | id: 3 | ||||
num_pulls: 0 | num_pulls: 0 | ||||
num_closed_pulls: 0 | num_closed_pulls: 0 | ||||
num_watches: 0 | num_watches: 0 | ||||
status: 0 | |||||
- | - | ||||
id: 4 | id: 4 | ||||
num_pulls: 0 | num_pulls: 0 | ||||
num_closed_pulls: 0 | num_closed_pulls: 0 | ||||
num_stars: 1 | num_stars: 1 | ||||
status: 0 | |||||
- | - | ||||
id: 5 | id: 5 | ||||
num_closed_pulls: 0 | num_closed_pulls: 0 | ||||
num_watches: 0 | num_watches: 0 | ||||
is_mirror: true | is_mirror: true | ||||
status: 0 | |||||
- | - | ||||
id: 6 | id: 6 | ||||
num_pulls: 0 | num_pulls: 0 | ||||
num_closed_pulls: 0 | num_closed_pulls: 0 | ||||
is_mirror: false | is_mirror: false | ||||
status: 0 | |||||
- | - | ||||
id: 7 | id: 7 | ||||
num_pulls: 0 | num_pulls: 0 | ||||
num_closed_pulls: 0 | num_closed_pulls: 0 | ||||
is_mirror: false | is_mirror: false | ||||
status: 0 | |||||
- | - | ||||
id: 8 | id: 8 | ||||
num_pulls: 0 | num_pulls: 0 | ||||
num_closed_pulls: 0 | num_closed_pulls: 0 | ||||
is_mirror: false | is_mirror: false | ||||
status: 0 | |||||
- | - | ||||
id: 9 | id: 9 | ||||
num_pulls: 0 | num_pulls: 0 | ||||
num_closed_pulls: 0 | num_closed_pulls: 0 | ||||
is_mirror: false | is_mirror: false | ||||
status: 0 | |||||
- | - | ||||
id: 10 | id: 10 | ||||
num_closed_pulls: 0 | num_closed_pulls: 0 | ||||
is_mirror: false | is_mirror: false | ||||
num_forks: 1 | num_forks: 1 | ||||
status: 0 | |||||
- | - | ||||
id: 11 | id: 11 | ||||
num_pulls: 0 | num_pulls: 0 | ||||
num_closed_pulls: 0 | num_closed_pulls: 0 | ||||
is_mirror: false | is_mirror: false | ||||
status: 0 | |||||
- | - | ||||
id: 12 | id: 12 | ||||
num_pulls: 0 | num_pulls: 0 | ||||
num_closed_pulls: 0 | num_closed_pulls: 0 | ||||
is_mirror: false | is_mirror: false | ||||
status: 0 | |||||
- | - | ||||
id: 13 | id: 13 | ||||
num_pulls: 0 | num_pulls: 0 | ||||
num_closed_pulls: 0 | num_closed_pulls: 0 | ||||
is_mirror: false | is_mirror: false | ||||
status: 0 | |||||
- | - | ||||
id: 14 | id: 14 | ||||
num_pulls: 0 | num_pulls: 0 | ||||
num_closed_pulls: 0 | num_closed_pulls: 0 | ||||
is_mirror: false | is_mirror: false | ||||
status: 0 | |||||
- | - | ||||
id: 15 | id: 15 | ||||
lower_name: repo15 | lower_name: repo15 | ||||
name: repo15 | name: repo15 | ||||
is_empty: true | is_empty: true | ||||
status: 0 | |||||
- | - | ||||
id: 16 | id: 16 | ||||
num_pulls: 0 | num_pulls: 0 | ||||
num_closed_pulls: 0 | num_closed_pulls: 0 | ||||
num_watches: 0 | num_watches: 0 | ||||
status: 0 | |||||
- | - | ||||
id: 17 | id: 17 | ||||
num_watches: 0 | num_watches: 0 | ||||
is_mirror: false | is_mirror: false | ||||
is_fork: false | is_fork: false | ||||
status: 0 | |||||
- | - | ||||
id: 18 | id: 18 | ||||
num_closed_pulls: 0 | num_closed_pulls: 0 | ||||
is_mirror: false | is_mirror: false | ||||
is_fork: false | is_fork: false | ||||
status: 0 | |||||
- | - | ||||
id: 19 | id: 19 | ||||
num_closed_pulls: 0 | num_closed_pulls: 0 | ||||
is_mirror: false | is_mirror: false | ||||
is_fork: false | is_fork: false | ||||
status: 0 | |||||
- | - | ||||
id: 20 | id: 20 | ||||
num_closed_pulls: 0 | num_closed_pulls: 0 | ||||
is_mirror: false | is_mirror: false | ||||
is_fork: false | is_fork: false | ||||
status: 0 | |||||
- | - | ||||
id: 21 | id: 21 | ||||
num_closed_pulls: 0 | num_closed_pulls: 0 | ||||
is_mirror: false | is_mirror: false | ||||
is_fork: false | is_fork: false | ||||
status: 0 | |||||
- | - | ||||
id: 22 | id: 22 | ||||
num_closed_pulls: 0 | num_closed_pulls: 0 | ||||
is_mirror: false | is_mirror: false | ||||
is_fork: false | is_fork: false | ||||
status: 0 | |||||
- | - | ||||
id: 23 | id: 23 | ||||
num_closed_pulls: 0 | num_closed_pulls: 0 | ||||
is_mirror: false | is_mirror: false | ||||
is_fork: false | is_fork: false | ||||
status: 0 | |||||
- | - | ||||
id: 24 | id: 24 | ||||
num_closed_pulls: 0 | num_closed_pulls: 0 | ||||
is_mirror: false | is_mirror: false | ||||
is_fork: false | is_fork: false | ||||
status: 0 | |||||
- | - | ||||
id: 25 | id: 25 | ||||
num_watches: 0 | num_watches: 0 | ||||
is_mirror: true | is_mirror: true | ||||
is_fork: false | is_fork: false | ||||
status: 0 | |||||
- | - | ||||
id: 26 | id: 26 | ||||
num_watches: 0 | num_watches: 0 | ||||
is_mirror: true | is_mirror: true | ||||
is_fork: false | is_fork: false | ||||
status: 0 | |||||
- | - | ||||
id: 27 | id: 27 | ||||
is_mirror: true | is_mirror: true | ||||
num_forks: 1 | num_forks: 1 | ||||
is_fork: false | is_fork: false | ||||
status: 0 | |||||
- | - | ||||
id: 28 | id: 28 | ||||
is_mirror: true | is_mirror: true | ||||
num_forks: 1 | num_forks: 1 | ||||
is_fork: false | is_fork: false | ||||
status: 0 | |||||
- | - | ||||
id: 29 | id: 29 | ||||
num_closed_pulls: 0 | num_closed_pulls: 0 | ||||
is_mirror: false | is_mirror: false | ||||
is_fork: true | is_fork: true | ||||
status: 0 | |||||
- | - | ||||
id: 30 | id: 30 | ||||
num_closed_pulls: 0 | num_closed_pulls: 0 | ||||
is_mirror: false | is_mirror: false | ||||
is_fork: true | is_fork: true | ||||
status: 0 | |||||
- | - | ||||
id: 31 | id: 31 | ||||
num_forks: 0 | num_forks: 0 | ||||
num_issues: 0 | num_issues: 0 | ||||
is_mirror: false | is_mirror: false | ||||
status: 0 | |||||
- | - | ||||
id: 32 # org public repo | id: 32 # org public repo | ||||
num_forks: 0 | num_forks: 0 | ||||
num_issues: 0 | num_issues: 0 | ||||
is_mirror: false | is_mirror: false | ||||
status: 0 | |||||
- | - | ||||
id: 33 | id: 33 | ||||
lower_name: utf8 | lower_name: utf8 | ||||
name: utf8 | name: utf8 | ||||
is_private: false | is_private: false | ||||
status: 0 | |||||
- | - | ||||
id: 34 | id: 34 | ||||
num_forks: 0 | num_forks: 0 | ||||
num_issues: 0 | num_issues: 0 | ||||
is_mirror: false | is_mirror: false | ||||
status: 0 | |||||
- | - | ||||
id: 35 | id: 35 | ||||
num_forks: 0 | num_forks: 0 | ||||
num_issues: 0 | num_issues: 0 | ||||
is_mirror: false | is_mirror: false | ||||
status: 0 | |||||
- | - | ||||
id: 36 | id: 36 | ||||
num_forks: 0 | num_forks: 0 | ||||
num_issues: 0 | num_issues: 0 | ||||
is_mirror: false | is_mirror: false | ||||
status: 0 | |||||
- | - | ||||
id: 37 | id: 37 | ||||
num_forks: 0 | num_forks: 0 | ||||
num_issues: 0 | num_issues: 0 | ||||
is_mirror: false | is_mirror: false | ||||
status: 0 | |||||
- | - | ||||
id: 38 | id: 38 | ||||
num_forks: 0 | num_forks: 0 | ||||
num_issues: 0 | num_issues: 0 | ||||
is_mirror: false | is_mirror: false | ||||
status: 0 | |||||
- | - | ||||
id: 39 | id: 39 | ||||
num_forks: 0 | num_forks: 0 | ||||
num_issues: 0 | num_issues: 0 | ||||
is_mirror: false | is_mirror: false | ||||
status: 0 | |||||
- | - | ||||
id: 40 | id: 40 | ||||
num_forks: 0 | num_forks: 0 | ||||
num_issues: 0 | num_issues: 0 | ||||
is_mirror: false | is_mirror: false | ||||
status: 0 | |||||
- | - | ||||
id: 41 | id: 41 | ||||
num_stars: 0 | num_stars: 0 | ||||
num_forks: 0 | num_forks: 0 | ||||
num_issues: 0 | num_issues: 0 | ||||
is_mirror: false | |||||
is_mirror: false | |||||
status: 0 |
NewMigration("add repo_admin_change_team_access to user", addRepoAdminChangeTeamAccessColumnForUser), | NewMigration("add repo_admin_change_team_access to user", addRepoAdminChangeTeamAccessColumnForUser), | ||||
// v98 -> v99 | // v98 -> v99 | ||||
NewMigration("add original author name and id on migrated release", addOriginalAuthorOnMigratedReleases), | NewMigration("add original author name and id on migrated release", addOriginalAuthorOnMigratedReleases), | ||||
// v99 -> v100 | |||||
NewMigration("add task table and status column for repository table", addTaskTable), | |||||
} | } | ||||
// Migrate database to current version | // Migrate database to current version |
// Copyright 2019 The Gitea Authors. All rights reserved. | |||||
// Use of this source code is governed by a MIT-style | |||||
// license that can be found in the LICENSE file. | |||||
package migrations | |||||
import ( | |||||
"code.gitea.io/gitea/modules/structs" | |||||
"code.gitea.io/gitea/modules/timeutil" | |||||
"github.com/go-xorm/xorm" | |||||
) | |||||
func addTaskTable(x *xorm.Engine) error { | |||||
type Task struct { | |||||
ID int64 | |||||
DoerID int64 `xorm:"index"` // operator | |||||
OwnerID int64 `xorm:"index"` // repo owner id, when creating, the repoID maybe zero | |||||
RepoID int64 `xorm:"index"` | |||||
Type structs.TaskType | |||||
Status structs.TaskStatus `xorm:"index"` | |||||
StartTime timeutil.TimeStamp | |||||
EndTime timeutil.TimeStamp | |||||
PayloadContent string `xorm:"TEXT"` | |||||
Errors string `xorm:"TEXT"` // if task failed, saved the error reason | |||||
Created timeutil.TimeStamp `xorm:"created"` | |||||
} | |||||
type Repository struct { | |||||
Status int `xorm:"NOT NULL DEFAULT 0"` | |||||
} | |||||
return x.Sync2(new(Task), new(Repository)) | |||||
} |
new(OAuth2Application), | new(OAuth2Application), | ||||
new(OAuth2AuthorizationCode), | new(OAuth2AuthorizationCode), | ||||
new(OAuth2Grant), | new(OAuth2Grant), | ||||
new(Task), | |||||
) | ) | ||||
gonicNames := []string{"SSL", "UID"} | gonicNames := []string{"SSL", "UID"} |
RemoveAllWithNotice("Clean up repository temporary data", filepath.Join(setting.AppDataPath, "tmp")) | RemoveAllWithNotice("Clean up repository temporary data", filepath.Join(setting.AppDataPath, "tmp")) | ||||
} | } | ||||
// RepositoryStatus defines the status of repository | |||||
type RepositoryStatus int | |||||
// all kinds of RepositoryStatus | |||||
const ( | |||||
RepositoryReady RepositoryStatus = iota // a normal repository | |||||
RepositoryBeingMigrated // repository is migrating | |||||
) | |||||
// Repository represents a git repository. | // Repository represents a git repository. | ||||
type Repository struct { | type Repository struct { | ||||
ID int64 `xorm:"pk autoincr"` | ID int64 `xorm:"pk autoincr"` | ||||
IsPrivate bool `xorm:"INDEX"` | IsPrivate bool `xorm:"INDEX"` | ||||
IsEmpty bool `xorm:"INDEX"` | IsEmpty bool `xorm:"INDEX"` | ||||
IsArchived bool `xorm:"INDEX"` | IsArchived bool `xorm:"INDEX"` | ||||
IsMirror bool `xorm:"INDEX"` | |||||
*Mirror `xorm:"-"` | |||||
IsMirror bool `xorm:"INDEX"` | |||||
*Mirror `xorm:"-"` | |||||
Status RepositoryStatus `xorm:"NOT NULL DEFAULT 0"` | |||||
ExternalMetas map[string]string `xorm:"-"` | ExternalMetas map[string]string `xorm:"-"` | ||||
Units []*RepoUnit `xorm:"-"` | Units []*RepoUnit `xorm:"-"` | ||||
repo.Name) | repo.Name) | ||||
} | } | ||||
// IsBeingMigrated indicates that repository is being migtated | |||||
func (repo *Repository) IsBeingMigrated() bool { | |||||
return repo.Status == RepositoryBeingMigrated | |||||
} | |||||
// IsBeingCreated indicates that repository is being migrated or forked | |||||
func (repo *Repository) IsBeingCreated() bool { | |||||
return repo.IsBeingMigrated() | |||||
} | |||||
// AfterLoad is invoked from XORM after setting the values of all fields of this object. | // AfterLoad is invoked from XORM after setting the values of all fields of this object. | ||||
func (repo *Repository) AfterLoad() { | func (repo *Repository) AfterLoad() { | ||||
// FIXME: use models migration to solve all at once. | // FIXME: use models migration to solve all at once. | ||||
return repo.cloneLink(x, false) | return repo.cloneLink(x, false) | ||||
} | } | ||||
// MigrateRepoOptions contains the repository migrate options | |||||
type MigrateRepoOptions struct { | |||||
Name string | |||||
Description string | |||||
OriginalURL string | |||||
IsPrivate bool | |||||
IsMirror bool | |||||
RemoteAddr string | |||||
Wiki bool // include wiki repository | |||||
SyncReleasesWithTags bool // sync releases from tags | |||||
} | |||||
/* | /* | ||||
GitHub, GitLab, Gogs: *.wiki.git | GitHub, GitLab, Gogs: *.wiki.git | ||||
BitBucket: *.git/wiki | BitBucket: *.git/wiki | ||||
return "" | return "" | ||||
} | } | ||||
// MigrateRepository migrates an existing repository from other project hosting. | |||||
func MigrateRepository(doer, u *User, opts MigrateRepoOptions) (*Repository, error) { | |||||
repo, err := CreateRepository(doer, u, CreateRepoOptions{ | |||||
Name: opts.Name, | |||||
Description: opts.Description, | |||||
OriginalURL: opts.OriginalURL, | |||||
IsPrivate: opts.IsPrivate, | |||||
IsMirror: opts.IsMirror, | |||||
}) | |||||
// CheckCreateRepository check if could created a repository | |||||
func CheckCreateRepository(doer, u *User, name string) error { | |||||
if !doer.CanCreateRepo() { | |||||
return ErrReachLimitOfRepo{u.MaxRepoCreation} | |||||
} | |||||
if err := IsUsableRepoName(name); err != nil { | |||||
return err | |||||
} | |||||
has, err := isRepositoryExist(x, u, name) | |||||
if err != nil { | if err != nil { | ||||
return nil, err | |||||
return fmt.Errorf("IsRepositoryExist: %v", err) | |||||
} else if has { | |||||
return ErrRepoAlreadyExist{u.Name, name} | |||||
} | } | ||||
return nil | |||||
} | |||||
repoPath := RepoPath(u.Name, opts.Name) | |||||
// MigrateRepositoryGitData starts migrating git related data after created migrating repository | |||||
func MigrateRepositoryGitData(doer, u *User, repo *Repository, opts api.MigrateRepoOption) (*Repository, error) { | |||||
repoPath := RepoPath(u.Name, opts.RepoName) | |||||
if u.IsOrganization() { | if u.IsOrganization() { | ||||
t, err := u.GetOwnerTeam() | t, err := u.GetOwnerTeam() | ||||
migrateTimeout := time.Duration(setting.Git.Timeout.Migrate) * time.Second | migrateTimeout := time.Duration(setting.Git.Timeout.Migrate) * time.Second | ||||
if err := os.RemoveAll(repoPath); err != nil { | |||||
var err error | |||||
if err = os.RemoveAll(repoPath); err != nil { | |||||
return repo, fmt.Errorf("Failed to remove %s: %v", repoPath, err) | return repo, fmt.Errorf("Failed to remove %s: %v", repoPath, err) | ||||
} | } | ||||
if err = git.Clone(opts.RemoteAddr, repoPath, git.CloneRepoOptions{ | |||||
if err = git.Clone(opts.CloneAddr, repoPath, git.CloneRepoOptions{ | |||||
Mirror: true, | Mirror: true, | ||||
Quiet: true, | Quiet: true, | ||||
Timeout: migrateTimeout, | Timeout: migrateTimeout, | ||||
} | } | ||||
if opts.Wiki { | if opts.Wiki { | ||||
wikiPath := WikiPath(u.Name, opts.Name) | |||||
wikiRemotePath := wikiRemoteURL(opts.RemoteAddr) | |||||
wikiPath := WikiPath(u.Name, opts.RepoName) | |||||
wikiRemotePath := wikiRemoteURL(opts.CloneAddr) | |||||
if len(wikiRemotePath) > 0 { | if len(wikiRemotePath) > 0 { | ||||
if err := os.RemoveAll(wikiPath); err != nil { | if err := os.RemoveAll(wikiPath); err != nil { | ||||
return repo, fmt.Errorf("Failed to remove %s: %v", wikiPath, err) | return repo, fmt.Errorf("Failed to remove %s: %v", wikiPath, err) | ||||
return repo, fmt.Errorf("git.IsEmpty: %v", err) | return repo, fmt.Errorf("git.IsEmpty: %v", err) | ||||
} | } | ||||
if opts.SyncReleasesWithTags && !repo.IsEmpty { | |||||
if !opts.Releases && !repo.IsEmpty { | |||||
// Try to get HEAD branch and set it as default branch. | // Try to get HEAD branch and set it as default branch. | ||||
headBranch, err := gitRepo.GetHEADBranch() | headBranch, err := gitRepo.GetHEADBranch() | ||||
if err != nil { | if err != nil { | ||||
log.Error("Failed to update size for repository: %v", err) | log.Error("Failed to update size for repository: %v", err) | ||||
} | } | ||||
if opts.IsMirror { | |||||
if opts.Mirror { | |||||
if _, err = x.InsertOne(&Mirror{ | if _, err = x.InsertOne(&Mirror{ | ||||
RepoID: repo.ID, | RepoID: repo.ID, | ||||
Interval: setting.Mirror.DefaultInterval, | Interval: setting.Mirror.DefaultInterval, | ||||
IsPrivate bool | IsPrivate bool | ||||
IsMirror bool | IsMirror bool | ||||
AutoInit bool | AutoInit bool | ||||
Status RepositoryStatus | |||||
} | } | ||||
func getRepoInitFile(tp, name string) ([]byte, error) { | func getRepoInitFile(tp, name string) ([]byte, error) { | ||||
IsPrivate: opts.IsPrivate, | IsPrivate: opts.IsPrivate, | ||||
IsFsckEnabled: !opts.IsMirror, | IsFsckEnabled: !opts.IsMirror, | ||||
CloseIssuesViaCommitInAnyBranch: setting.Repository.DefaultCloseIssuesViaCommitsInAnyBranch, | CloseIssuesViaCommitInAnyBranch: setting.Repository.DefaultCloseIssuesViaCommitsInAnyBranch, | ||||
Status: opts.Status, | |||||
} | } | ||||
sess := x.NewSession() | sess := x.NewSession() | ||||
&CommitStatus{RepoID: repoID}, | &CommitStatus{RepoID: repoID}, | ||||
&RepoIndexerStatus{RepoID: repoID}, | &RepoIndexerStatus{RepoID: repoID}, | ||||
&Comment{RefRepoID: repoID}, | &Comment{RefRepoID: repoID}, | ||||
&Task{RepoID: repoID}, | |||||
); err != nil { | ); err != nil { | ||||
return fmt.Errorf("deleteBeans: %v", err) | return fmt.Errorf("deleteBeans: %v", err) | ||||
} | } |
// Copyright 2019 Gitea. All rights reserved. | |||||
// Use of this source code is governed by a MIT-style | |||||
// license that can be found in the LICENSE file. | |||||
package models | |||||
import ( | |||||
"encoding/json" | |||||
"fmt" | |||||
"code.gitea.io/gitea/modules/log" | |||||
"code.gitea.io/gitea/modules/migrations/base" | |||||
"code.gitea.io/gitea/modules/structs" | |||||
"code.gitea.io/gitea/modules/timeutil" | |||||
"xorm.io/builder" | |||||
) | |||||
// Task represents a task | |||||
type Task struct { | |||||
ID int64 | |||||
DoerID int64 `xorm:"index"` // operator | |||||
Doer *User `xorm:"-"` | |||||
OwnerID int64 `xorm:"index"` // repo owner id, when creating, the repoID maybe zero | |||||
Owner *User `xorm:"-"` | |||||
RepoID int64 `xorm:"index"` | |||||
Repo *Repository `xorm:"-"` | |||||
Type structs.TaskType | |||||
Status structs.TaskStatus `xorm:"index"` | |||||
StartTime timeutil.TimeStamp | |||||
EndTime timeutil.TimeStamp | |||||
PayloadContent string `xorm:"TEXT"` | |||||
Errors string `xorm:"TEXT"` // if task failed, saved the error reason | |||||
Created timeutil.TimeStamp `xorm:"created"` | |||||
} | |||||
// LoadRepo loads repository of the task | |||||
func (task *Task) LoadRepo() error { | |||||
return task.loadRepo(x) | |||||
} | |||||
func (task *Task) loadRepo(e Engine) error { | |||||
if task.Repo != nil { | |||||
return nil | |||||
} | |||||
var repo Repository | |||||
has, err := e.ID(task.RepoID).Get(&repo) | |||||
if err != nil { | |||||
return err | |||||
} else if !has { | |||||
return ErrRepoNotExist{ | |||||
ID: task.RepoID, | |||||
} | |||||
} | |||||
task.Repo = &repo | |||||
return nil | |||||
} | |||||
// LoadDoer loads do user | |||||
func (task *Task) LoadDoer() error { | |||||
if task.Doer != nil { | |||||
return nil | |||||
} | |||||
var doer User | |||||
has, err := x.ID(task.DoerID).Get(&doer) | |||||
if err != nil { | |||||
return err | |||||
} else if !has { | |||||
return ErrUserNotExist{ | |||||
UID: task.DoerID, | |||||
} | |||||
} | |||||
task.Doer = &doer | |||||
return nil | |||||
} | |||||
// LoadOwner loads owner user | |||||
func (task *Task) LoadOwner() error { | |||||
if task.Owner != nil { | |||||
return nil | |||||
} | |||||
var owner User | |||||
has, err := x.ID(task.OwnerID).Get(&owner) | |||||
if err != nil { | |||||
return err | |||||
} else if !has { | |||||
return ErrUserNotExist{ | |||||
UID: task.OwnerID, | |||||
} | |||||
} | |||||
task.Owner = &owner | |||||
return nil | |||||
} | |||||
// UpdateCols updates some columns | |||||
func (task *Task) UpdateCols(cols ...string) error { | |||||
_, err := x.ID(task.ID).Cols(cols...).Update(task) | |||||
return err | |||||
} | |||||
// MigrateConfig returns task config when migrate repository | |||||
func (task *Task) MigrateConfig() (*structs.MigrateRepoOption, error) { | |||||
if task.Type == structs.TaskTypeMigrateRepo { | |||||
var opts structs.MigrateRepoOption | |||||
err := json.Unmarshal([]byte(task.PayloadContent), &opts) | |||||
if err != nil { | |||||
return nil, err | |||||
} | |||||
return &opts, nil | |||||
} | |||||
return nil, fmt.Errorf("Task type is %s, not Migrate Repo", task.Type.Name()) | |||||
} | |||||
// ErrTaskDoesNotExist represents a "TaskDoesNotExist" kind of error. | |||||
type ErrTaskDoesNotExist struct { | |||||
ID int64 | |||||
RepoID int64 | |||||
Type structs.TaskType | |||||
} | |||||
// IsErrTaskDoesNotExist checks if an error is a ErrTaskIsNotExist. | |||||
func IsErrTaskDoesNotExist(err error) bool { | |||||
_, ok := err.(ErrTaskDoesNotExist) | |||||
return ok | |||||
} | |||||
func (err ErrTaskDoesNotExist) Error() string { | |||||
return fmt.Sprintf("task is not exist [id: %d, repo_id: %d, type: %d]", | |||||
err.ID, err.RepoID, err.Type) | |||||
} | |||||
// GetMigratingTask returns the migrating task by repo's id | |||||
func GetMigratingTask(repoID int64) (*Task, error) { | |||||
var task = Task{ | |||||
RepoID: repoID, | |||||
Type: structs.TaskTypeMigrateRepo, | |||||
} | |||||
has, err := x.Get(&task) | |||||
if err != nil { | |||||
return nil, err | |||||
} else if !has { | |||||
return nil, ErrTaskDoesNotExist{0, repoID, task.Type} | |||||
} | |||||
return &task, nil | |||||
} | |||||
// FindTaskOptions find all tasks | |||||
type FindTaskOptions struct { | |||||
Status int | |||||
} | |||||
// ToConds generates conditions for database operation. | |||||
func (opts FindTaskOptions) ToConds() builder.Cond { | |||||
var cond = builder.NewCond() | |||||
if opts.Status >= 0 { | |||||
cond = cond.And(builder.Eq{"status": opts.Status}) | |||||
} | |||||
return cond | |||||
} | |||||
// FindTasks find all tasks | |||||
func FindTasks(opts FindTaskOptions) ([]*Task, error) { | |||||
var tasks = make([]*Task, 0, 10) | |||||
err := x.Where(opts.ToConds()).Find(&tasks) | |||||
return tasks, err | |||||
} | |||||
func createTask(e Engine, task *Task) error { | |||||
_, err := e.Insert(task) | |||||
return err | |||||
} | |||||
// CreateMigrateTask creates a migrate task | |||||
func CreateMigrateTask(doer, u *User, opts base.MigrateOptions) (*Task, error) { | |||||
bs, err := json.Marshal(&opts) | |||||
if err != nil { | |||||
return nil, err | |||||
} | |||||
var task = Task{ | |||||
DoerID: doer.ID, | |||||
OwnerID: u.ID, | |||||
Type: structs.TaskTypeMigrateRepo, | |||||
Status: structs.TaskStatusQueue, | |||||
PayloadContent: string(bs), | |||||
} | |||||
if err := createTask(x, &task); err != nil { | |||||
return nil, err | |||||
} | |||||
repo, err := CreateRepository(doer, u, CreateRepoOptions{ | |||||
Name: opts.RepoName, | |||||
Description: opts.Description, | |||||
OriginalURL: opts.CloneAddr, | |||||
IsPrivate: opts.Private, | |||||
IsMirror: opts.Mirror, | |||||
Status: RepositoryBeingMigrated, | |||||
}) | |||||
if err != nil { | |||||
task.EndTime = timeutil.TimeStampNow() | |||||
task.Status = structs.TaskStatusFailed | |||||
err2 := task.UpdateCols("end_time", "status") | |||||
if err2 != nil { | |||||
log.Error("UpdateCols Failed: %v", err2.Error()) | |||||
} | |||||
return nil, err | |||||
} | |||||
task.RepoID = repo.ID | |||||
if err = task.UpdateCols("repo_id"); err != nil { | |||||
return nil, err | |||||
} | |||||
return &task, nil | |||||
} | |||||
// FinishMigrateTask updates database when migrate task finished | |||||
func FinishMigrateTask(task *Task) error { | |||||
task.Status = structs.TaskStatusFinished | |||||
task.EndTime = timeutil.TimeStampNow() | |||||
sess := x.NewSession() | |||||
defer sess.Close() | |||||
if err := sess.Begin(); err != nil { | |||||
return err | |||||
} | |||||
if _, err := sess.ID(task.ID).Cols("status", "end_time").Update(task); err != nil { | |||||
return err | |||||
} | |||||
task.Repo.Status = RepositoryReady | |||||
if _, err := sess.ID(task.RepoID).Cols("status").Update(task.Repo); err != nil { | |||||
return err | |||||
} | |||||
return sess.Commit() | |||||
} |
// GetEditorconfig returns the .editorconfig definition if found in the | // GetEditorconfig returns the .editorconfig definition if found in the | ||||
// HEAD of the default repo branch. | // HEAD of the default repo branch. | ||||
func (r *Repository) GetEditorconfig() (*editorconfig.Editorconfig, error) { | func (r *Repository) GetEditorconfig() (*editorconfig.Editorconfig, error) { | ||||
if r.GitRepo == nil { | |||||
return nil, nil | |||||
} | |||||
commit, err := r.GitRepo.GetBranchCommit(r.Repository.DefaultBranch) | commit, err := r.GitRepo.GetBranchCommit(r.Repository.DefaultBranch) | ||||
if err != nil { | if err != nil { | ||||
return nil, err | return nil, err | ||||
return | return | ||||
} | } | ||||
gitRepo, err := git.OpenRepository(models.RepoPath(userName, repoName)) | |||||
if err != nil { | |||||
ctx.ServerError("RepoAssignment Invalid repo "+models.RepoPath(userName, repoName), err) | |||||
return | |||||
} | |||||
ctx.Repo.GitRepo = gitRepo | |||||
ctx.Repo.RepoLink = repo.Link() | ctx.Repo.RepoLink = repo.Link() | ||||
ctx.Data["RepoLink"] = ctx.Repo.RepoLink | ctx.Data["RepoLink"] = ctx.Repo.RepoLink | ||||
ctx.Data["RepoRelPath"] = ctx.Repo.Owner.Name + "/" + ctx.Repo.Repository.Name | ctx.Data["RepoRelPath"] = ctx.Repo.Owner.Name + "/" + ctx.Repo.Repository.Name | ||||
ctx.Data["RepoExternalIssuesLink"] = unit.ExternalTrackerConfig().ExternalTrackerURL | ctx.Data["RepoExternalIssuesLink"] = unit.ExternalTrackerConfig().ExternalTrackerURL | ||||
} | } | ||||
tags, err := ctx.Repo.GitRepo.GetTags() | |||||
if err != nil { | |||||
ctx.ServerError("GetTags", err) | |||||
return | |||||
} | |||||
ctx.Data["Tags"] = tags | |||||
count, err := models.GetReleaseCountByRepoID(ctx.Repo.Repository.ID, models.FindReleasesOptions{ | count, err := models.GetReleaseCountByRepoID(ctx.Repo.Repository.ID, models.FindReleasesOptions{ | ||||
IncludeDrafts: false, | IncludeDrafts: false, | ||||
IncludeTags: true, | IncludeTags: true, | ||||
} | } | ||||
// repo is empty and display enable | // repo is empty and display enable | ||||
if ctx.Repo.Repository.IsEmpty { | |||||
if ctx.Repo.Repository.IsEmpty || ctx.Repo.Repository.IsBeingCreated() { | |||||
ctx.Data["BranchName"] = ctx.Repo.Repository.DefaultBranch | ctx.Data["BranchName"] = ctx.Repo.Repository.DefaultBranch | ||||
return | return | ||||
} | } | ||||
ctx.Data["TagName"] = ctx.Repo.TagName | |||||
gitRepo, err := git.OpenRepository(models.RepoPath(userName, repoName)) | |||||
if err != nil { | |||||
ctx.ServerError("RepoAssignment Invalid repo "+models.RepoPath(userName, repoName), err) | |||||
return | |||||
} | |||||
ctx.Repo.GitRepo = gitRepo | |||||
tags, err := ctx.Repo.GitRepo.GetTags() | |||||
if err != nil { | |||||
ctx.ServerError("GetTags", err) | |||||
return | |||||
} | |||||
ctx.Data["Tags"] = tags | |||||
brs, err := ctx.Repo.GitRepo.GetBranches() | brs, err := ctx.Repo.GitRepo.GetBranches() | ||||
if err != nil { | if err != nil { | ||||
ctx.ServerError("GetBranches", err) | ctx.ServerError("GetBranches", err) | ||||
ctx.Data["Branches"] = brs | ctx.Data["Branches"] = brs | ||||
ctx.Data["BranchesCount"] = len(brs) | ctx.Data["BranchesCount"] = len(brs) | ||||
ctx.Data["TagName"] = ctx.Repo.TagName | |||||
// If not branch selected, try default one. | // If not branch selected, try default one. | ||||
// If default branch doesn't exists, fall back to some other branch. | // If default branch doesn't exists, fall back to some other branch. | ||||
if len(ctx.Repo.BranchName) == 0 { | if len(ctx.Repo.BranchName) == 0 { |
package base | package base | ||||
// MigrateOptions defines the way a repository gets migrated | |||||
type MigrateOptions struct { | |||||
RemoteURL string | |||||
AuthUsername string | |||||
AuthPassword string | |||||
Name string | |||||
Description string | |||||
OriginalURL string | |||||
import "code.gitea.io/gitea/modules/structs" | |||||
Wiki bool | |||||
Issues bool | |||||
Milestones bool | |||||
Labels bool | |||||
Releases bool | |||||
Comments bool | |||||
PullRequests bool | |||||
Private bool | |||||
Mirror bool | |||||
} | |||||
// MigrateOptions defines the way a repository gets migrated | |||||
type MigrateOptions = structs.MigrateRepoOption |
"code.gitea.io/gitea/modules/log" | "code.gitea.io/gitea/modules/log" | ||||
"code.gitea.io/gitea/modules/migrations/base" | "code.gitea.io/gitea/modules/migrations/base" | ||||
"code.gitea.io/gitea/modules/setting" | "code.gitea.io/gitea/modules/setting" | ||||
"code.gitea.io/gitea/modules/structs" | |||||
"code.gitea.io/gitea/modules/timeutil" | "code.gitea.io/gitea/modules/timeutil" | ||||
gouuid "github.com/satori/go.uuid" | gouuid "github.com/satori/go.uuid" | ||||
remoteAddr = u.String() | remoteAddr = u.String() | ||||
} | } | ||||
r, err := models.MigrateRepository(g.doer, owner, models.MigrateRepoOptions{ | |||||
Name: g.repoName, | |||||
Description: repo.Description, | |||||
OriginalURL: repo.OriginalURL, | |||||
IsMirror: repo.IsMirror, | |||||
RemoteAddr: remoteAddr, | |||||
IsPrivate: repo.IsPrivate, | |||||
Wiki: opts.Wiki, | |||||
SyncReleasesWithTags: !opts.Releases, // if didn't get releases, then sync them from tags | |||||
var r *models.Repository | |||||
if opts.MigrateToRepoID <= 0 { | |||||
r, err = models.CreateRepository(g.doer, owner, models.CreateRepoOptions{ | |||||
Name: g.repoName, | |||||
Description: repo.Description, | |||||
OriginalURL: repo.OriginalURL, | |||||
IsPrivate: opts.Private, | |||||
IsMirror: opts.Mirror, | |||||
Status: models.RepositoryBeingMigrated, | |||||
}) | |||||
} else { | |||||
r, err = models.GetRepositoryByID(opts.MigrateToRepoID) | |||||
} | |||||
if err != nil { | |||||
return err | |||||
} | |||||
r, err = models.MigrateRepositoryGitData(g.doer, owner, r, structs.MigrateRepoOption{ | |||||
RepoName: g.repoName, | |||||
Description: repo.Description, | |||||
Mirror: repo.IsMirror, | |||||
CloneAddr: remoteAddr, | |||||
Private: repo.IsPrivate, | |||||
Wiki: opts.Wiki, | |||||
Releases: opts.Releases, // if didn't get releases, then sync them from tags | |||||
}) | }) | ||||
g.repo = r | g.repo = r | ||||
if err != nil { | if err != nil { | ||||
return err | return err |
"time" | "time" | ||||
"code.gitea.io/gitea/models" | "code.gitea.io/gitea/models" | ||||
"code.gitea.io/gitea/modules/structs" | |||||
"code.gitea.io/gitea/modules/util" | "code.gitea.io/gitea/modules/util" | ||||
"github.com/stretchr/testify/assert" | "github.com/stretchr/testify/assert" | ||||
uploader = NewGiteaLocalUploader(user, user.Name, repoName) | uploader = NewGiteaLocalUploader(user, user.Name, repoName) | ||||
) | ) | ||||
err := migrateRepository(downloader, uploader, MigrateOptions{ | |||||
RemoteURL: "https://github.com/go-xorm/builder", | |||||
Name: repoName, | |||||
err := migrateRepository(downloader, uploader, structs.MigrateRepoOption{ | |||||
CloneAddr: "https://github.com/go-xorm/builder", | |||||
RepoName: repoName, | |||||
AuthUsername: "", | AuthUsername: "", | ||||
Wiki: true, | Wiki: true, |
// Match returns ture if the migration remote URL matched this downloader factory | // Match returns ture if the migration remote URL matched this downloader factory | ||||
func (f *GithubDownloaderV3Factory) Match(opts base.MigrateOptions) (bool, error) { | func (f *GithubDownloaderV3Factory) Match(opts base.MigrateOptions) (bool, error) { | ||||
u, err := url.Parse(opts.RemoteURL) | |||||
u, err := url.Parse(opts.CloneAddr) | |||||
if err != nil { | if err != nil { | ||||
return false, err | return false, err | ||||
} | } | ||||
// New returns a Downloader related to this factory according MigrateOptions | // New returns a Downloader related to this factory according MigrateOptions | ||||
func (f *GithubDownloaderV3Factory) New(opts base.MigrateOptions) (base.Downloader, error) { | func (f *GithubDownloaderV3Factory) New(opts base.MigrateOptions) (base.Downloader, error) { | ||||
u, err := url.Parse(opts.RemoteURL) | |||||
u, err := url.Parse(opts.CloneAddr) | |||||
if err != nil { | if err != nil { | ||||
return nil, err | return nil, err | ||||
} | } |
package migrations | package migrations | ||||
import ( | import ( | ||||
"fmt" | |||||
"code.gitea.io/gitea/models" | "code.gitea.io/gitea/models" | ||||
"code.gitea.io/gitea/modules/log" | "code.gitea.io/gitea/modules/log" | ||||
"code.gitea.io/gitea/modules/migrations/base" | "code.gitea.io/gitea/modules/migrations/base" | ||||
func MigrateRepository(doer *models.User, ownerName string, opts base.MigrateOptions) (*models.Repository, error) { | func MigrateRepository(doer *models.User, ownerName string, opts base.MigrateOptions) (*models.Repository, error) { | ||||
var ( | var ( | ||||
downloader base.Downloader | downloader base.Downloader | ||||
uploader = NewGiteaLocalUploader(doer, ownerName, opts.Name) | |||||
uploader = NewGiteaLocalUploader(doer, ownerName, opts.RepoName) | |||||
) | ) | ||||
for _, factory := range factories { | for _, factory := range factories { | ||||
opts.Comments = false | opts.Comments = false | ||||
opts.Issues = false | opts.Issues = false | ||||
opts.PullRequests = false | opts.PullRequests = false | ||||
downloader = NewPlainGitDownloader(ownerName, opts.Name, opts.RemoteURL) | |||||
log.Trace("Will migrate from git: %s", opts.RemoteURL) | |||||
downloader = NewPlainGitDownloader(ownerName, opts.RepoName, opts.CloneAddr) | |||||
log.Trace("Will migrate from git: %s", opts.CloneAddr) | |||||
} | } | ||||
if err := migrateRepository(downloader, uploader, opts); err != nil { | if err := migrateRepository(downloader, uploader, opts); err != nil { | ||||
if err1 := uploader.Rollback(); err1 != nil { | if err1 := uploader.Rollback(); err1 != nil { | ||||
log.Error("rollback failed: %v", err1) | log.Error("rollback failed: %v", err1) | ||||
} | } | ||||
if err2 := models.CreateRepositoryNotice(fmt.Sprintf("Migrate repository from %s failed: %v", opts.CloneAddr, err)); err2 != nil { | |||||
log.Error("create respotiry notice failed: ", err2) | |||||
} | |||||
return nil, err | return nil, err | ||||
} | } | ||||
newNotifyMailService() | newNotifyMailService() | ||||
newWebhookService() | newWebhookService() | ||||
newIndexerService() | newIndexerService() | ||||
newTaskService() | |||||
} | } |
// Copyright 2019 The Gitea Authors. All rights reserved. | |||||
// Use of this source code is governed by a MIT-style | |||||
// license that can be found in the LICENSE file. | |||||
package setting | |||||
var ( | |||||
// Task settings | |||||
Task = struct { | |||||
QueueType string | |||||
QueueLength int | |||||
QueueConnStr string | |||||
}{ | |||||
QueueType: ChannelQueueType, | |||||
QueueLength: 1000, | |||||
QueueConnStr: "addrs=127.0.0.1:6379 db=0", | |||||
} | |||||
) | |||||
func newTaskService() { | |||||
sec := Cfg.Section("task") | |||||
Task.QueueType = sec.Key("QUEUE_TYPE").MustString(ChannelQueueType) | |||||
Task.QueueLength = sec.Key("QUEUE_LENGTH").MustInt(1000) | |||||
Task.QueueConnStr = sec.Key("QUEUE_CONN_STR").MustString("addrs=127.0.0.1:6379 db=0") | |||||
} |
// required: true | // required: true | ||||
UID int `json:"uid" binding:"Required"` | UID int `json:"uid" binding:"Required"` | ||||
// required: true | // required: true | ||||
RepoName string `json:"repo_name" binding:"Required"` | |||||
Mirror bool `json:"mirror"` | |||||
Private bool `json:"private"` | |||||
Description string `json:"description"` | |||||
RepoName string `json:"repo_name" binding:"Required"` | |||||
Mirror bool `json:"mirror"` | |||||
Private bool `json:"private"` | |||||
Description string `json:"description"` | |||||
Wiki bool | |||||
Issues bool | |||||
Milestones bool | |||||
Labels bool | |||||
Releases bool | |||||
Comments bool | |||||
PullRequests bool | |||||
MigrateToRepoID int64 | |||||
} | } |
// Copyright 2019 Gitea. All rights reserved. | |||||
// Use of this source code is governed by a MIT-style | |||||
// license that can be found in the LICENSE file. | |||||
package structs | |||||
// TaskType defines task type | |||||
type TaskType int | |||||
// all kinds of task types | |||||
const ( | |||||
TaskTypeMigrateRepo TaskType = iota // migrate repository from external or local disk | |||||
) | |||||
// Name returns the task type name | |||||
func (taskType TaskType) Name() string { | |||||
switch taskType { | |||||
case TaskTypeMigrateRepo: | |||||
return "Migrate Repository" | |||||
} | |||||
return "" | |||||
} | |||||
// TaskStatus defines task status | |||||
type TaskStatus int | |||||
// enumerate all the kinds of task status | |||||
const ( | |||||
TaskStatusQueue TaskStatus = iota // 0 task is queue | |||||
TaskStatusRunning // 1 task is running | |||||
TaskStatusStopped // 2 task is stopped | |||||
TaskStatusFailed // 3 task is failed | |||||
TaskStatusFinished // 4 task is finished | |||||
) |
// Copyright 2019 Gitea. All rights reserved. | |||||
// Use of this source code is governed by a MIT-style | |||||
// license that can be found in the LICENSE file. | |||||
package task | |||||
import ( | |||||
"bytes" | |||||
"errors" | |||||
"fmt" | |||||
"strings" | |||||
"code.gitea.io/gitea/models" | |||||
"code.gitea.io/gitea/modules/log" | |||||
"code.gitea.io/gitea/modules/migrations" | |||||
"code.gitea.io/gitea/modules/notification" | |||||
"code.gitea.io/gitea/modules/structs" | |||||
"code.gitea.io/gitea/modules/timeutil" | |||||
"code.gitea.io/gitea/modules/util" | |||||
) | |||||
func handleCreateError(owner *models.User, err error, name string) error { | |||||
switch { | |||||
case models.IsErrReachLimitOfRepo(err): | |||||
return fmt.Errorf("You have already reached your limit of %d repositories", owner.MaxCreationLimit()) | |||||
case models.IsErrRepoAlreadyExist(err): | |||||
return errors.New("The repository name is already used") | |||||
case models.IsErrNameReserved(err): | |||||
return fmt.Errorf("The repository name '%s' is reserved", err.(models.ErrNameReserved).Name) | |||||
case models.IsErrNamePatternNotAllowed(err): | |||||
return fmt.Errorf("The pattern '%s' is not allowed in a repository name", err.(models.ErrNamePatternNotAllowed).Pattern) | |||||
default: | |||||
return err | |||||
} | |||||
} | |||||
func runMigrateTask(t *models.Task) (err error) { | |||||
defer func() { | |||||
if e := recover(); e != nil { | |||||
var buf bytes.Buffer | |||||
fmt.Fprintf(&buf, "Handler crashed with error: %v", log.Stack(2)) | |||||
err = errors.New(buf.String()) | |||||
} | |||||
if err == nil { | |||||
err = models.FinishMigrateTask(t) | |||||
if err == nil { | |||||
notification.NotifyMigrateRepository(t.Doer, t.Owner, t.Repo) | |||||
return | |||||
} | |||||
log.Error("FinishMigrateTask failed: %s", err.Error()) | |||||
} | |||||
t.EndTime = timeutil.TimeStampNow() | |||||
t.Status = structs.TaskStatusFailed | |||||
t.Errors = err.Error() | |||||
if err := t.UpdateCols("status", "errors", "end_time"); err != nil { | |||||
log.Error("Task UpdateCols failed: %s", err.Error()) | |||||
} | |||||
if t.Repo != nil { | |||||
if errDelete := models.DeleteRepository(t.Doer, t.OwnerID, t.Repo.ID); errDelete != nil { | |||||
log.Error("DeleteRepository: %v", errDelete) | |||||
} | |||||
} | |||||
}() | |||||
if err := t.LoadRepo(); err != nil { | |||||
return err | |||||
} | |||||
// if repository is ready, then just finsih the task | |||||
if t.Repo.Status == models.RepositoryReady { | |||||
return nil | |||||
} | |||||
if err := t.LoadDoer(); err != nil { | |||||
return err | |||||
} | |||||
if err := t.LoadOwner(); err != nil { | |||||
return err | |||||
} | |||||
t.StartTime = timeutil.TimeStampNow() | |||||
t.Status = structs.TaskStatusRunning | |||||
if err := t.UpdateCols("start_time", "status"); err != nil { | |||||
return err | |||||
} | |||||
var opts *structs.MigrateRepoOption | |||||
opts, err = t.MigrateConfig() | |||||
if err != nil { | |||||
return err | |||||
} | |||||
opts.MigrateToRepoID = t.RepoID | |||||
repo, err := migrations.MigrateRepository(t.Doer, t.Owner.Name, *opts) | |||||
if err == nil { | |||||
notification.NotifyMigrateRepository(t.Doer, t.Owner, repo) | |||||
log.Trace("Repository migrated [%d]: %s/%s", repo.ID, t.Owner.Name, repo.Name) | |||||
return nil | |||||
} | |||||
if models.IsErrRepoAlreadyExist(err) { | |||||
return errors.New("The repository name is already used") | |||||
} | |||||
// remoteAddr may contain credentials, so we sanitize it | |||||
err = util.URLSanitizedError(err, opts.CloneAddr) | |||||
if strings.Contains(err.Error(), "Authentication failed") || | |||||
strings.Contains(err.Error(), "could not read Username") { | |||||
return fmt.Errorf("Authentication failed: %v", err.Error()) | |||||
} else if strings.Contains(err.Error(), "fatal:") { | |||||
return fmt.Errorf("Migration failed: %v", err.Error()) | |||||
} | |||||
return handleCreateError(t.Owner, err, "MigratePost") | |||||
} |
// Copyright 2019 Gitea. All rights reserved. | |||||
// Use of this source code is governed by a MIT-style | |||||
// license that can be found in the LICENSE file. | |||||
package task | |||||
import "code.gitea.io/gitea/models" | |||||
// Queue defines an interface to run task queue | |||||
type Queue interface { | |||||
Run() error | |||||
Push(*models.Task) error | |||||
Stop() | |||||
} |
// Copyright 2019 The Gitea Authors. All rights reserved. | |||||
// Use of this source code is governed by a MIT-style | |||||
// license that can be found in the LICENSE file. | |||||
package task | |||||
import ( | |||||
"code.gitea.io/gitea/models" | |||||
"code.gitea.io/gitea/modules/log" | |||||
) | |||||
var ( | |||||
_ Queue = &ChannelQueue{} | |||||
) | |||||
// ChannelQueue implements | |||||
type ChannelQueue struct { | |||||
queue chan *models.Task | |||||
} | |||||
// NewChannelQueue create a memory channel queue | |||||
func NewChannelQueue(queueLen int) *ChannelQueue { | |||||
return &ChannelQueue{ | |||||
queue: make(chan *models.Task, queueLen), | |||||
} | |||||
} | |||||
// Run starts to run the queue | |||||
func (c *ChannelQueue) Run() error { | |||||
for task := range c.queue { | |||||
err := Run(task) | |||||
if err != nil { | |||||
log.Error("Run task failed: %s", err.Error()) | |||||
} | |||||
} | |||||
return nil | |||||
} | |||||
// Push will push the task ID to queue | |||||
func (c *ChannelQueue) Push(task *models.Task) error { | |||||
c.queue <- task | |||||
return nil | |||||
} | |||||
// Stop stop the queue | |||||
func (c *ChannelQueue) Stop() { | |||||
close(c.queue) | |||||
} |
// Copyright 2019 The Gitea Authors. All rights reserved. | |||||
// Use of this source code is governed by a MIT-style | |||||
// license that can be found in the LICENSE file. | |||||
package task | |||||
import ( | |||||
"encoding/json" | |||||
"errors" | |||||
"strconv" | |||||
"strings" | |||||
"time" | |||||
"code.gitea.io/gitea/models" | |||||
"code.gitea.io/gitea/modules/log" | |||||
"github.com/go-redis/redis" | |||||
) | |||||
var ( | |||||
_ Queue = &RedisQueue{} | |||||
) | |||||
type redisClient interface { | |||||
RPush(key string, args ...interface{}) *redis.IntCmd | |||||
LPop(key string) *redis.StringCmd | |||||
Ping() *redis.StatusCmd | |||||
} | |||||
// RedisQueue redis queue | |||||
type RedisQueue struct { | |||||
client redisClient | |||||
queueName string | |||||
closeChan chan bool | |||||
} | |||||
func parseConnStr(connStr string) (addrs, password string, dbIdx int, err error) { | |||||
fields := strings.Fields(connStr) | |||||
for _, f := range fields { | |||||
items := strings.SplitN(f, "=", 2) | |||||
if len(items) < 2 { | |||||
continue | |||||
} | |||||
switch strings.ToLower(items[0]) { | |||||
case "addrs": | |||||
addrs = items[1] | |||||
case "password": | |||||
password = items[1] | |||||
case "db": | |||||
dbIdx, err = strconv.Atoi(items[1]) | |||||
if err != nil { | |||||
return | |||||
} | |||||
} | |||||
} | |||||
return | |||||
} | |||||
// NewRedisQueue creates single redis or cluster redis queue | |||||
func NewRedisQueue(addrs string, password string, dbIdx int) (*RedisQueue, error) { | |||||
dbs := strings.Split(addrs, ",") | |||||
var queue = RedisQueue{ | |||||
queueName: "task_queue", | |||||
closeChan: make(chan bool), | |||||
} | |||||
if len(dbs) == 0 { | |||||
return nil, errors.New("no redis host found") | |||||
} else if len(dbs) == 1 { | |||||
queue.client = redis.NewClient(&redis.Options{ | |||||
Addr: strings.TrimSpace(dbs[0]), // use default Addr | |||||
Password: password, // no password set | |||||
DB: dbIdx, // use default DB | |||||
}) | |||||
} else { | |||||
// cluster will ignore db | |||||
queue.client = redis.NewClusterClient(&redis.ClusterOptions{ | |||||
Addrs: dbs, | |||||
Password: password, | |||||
}) | |||||
} | |||||
if err := queue.client.Ping().Err(); err != nil { | |||||
return nil, err | |||||
} | |||||
return &queue, nil | |||||
} | |||||
// Run starts to run the queue | |||||
func (r *RedisQueue) Run() error { | |||||
for { | |||||
select { | |||||
case <-r.closeChan: | |||||
return nil | |||||
case <-time.After(time.Millisecond * 100): | |||||
} | |||||
bs, err := r.client.LPop(r.queueName).Bytes() | |||||
if err != nil { | |||||
if err != redis.Nil { | |||||
log.Error("LPop failed: %v", err) | |||||
} | |||||
time.Sleep(time.Millisecond * 100) | |||||
continue | |||||
} | |||||
var task models.Task | |||||
err = json.Unmarshal(bs, &task) | |||||
if err != nil { | |||||
log.Error("Unmarshal task failed: %s", err.Error()) | |||||
} else { | |||||
err = Run(&task) | |||||
if err != nil { | |||||
log.Error("Run task failed: %s", err.Error()) | |||||
} | |||||
} | |||||
} | |||||
} | |||||
// Push implements Queue | |||||
func (r *RedisQueue) Push(task *models.Task) error { | |||||
bs, err := json.Marshal(task) | |||||
if err != nil { | |||||
return err | |||||
} | |||||
return r.client.RPush(r.queueName, bs).Err() | |||||
} | |||||
// Stop stop the queue | |||||
func (r *RedisQueue) Stop() { | |||||
r.closeChan <- true | |||||
} |
// Copyright 2019 Gitea. All rights reserved. | |||||
// Use of this source code is governed by a MIT-style | |||||
// license that can be found in the LICENSE file. | |||||
package task | |||||
import ( | |||||
"fmt" | |||||
"code.gitea.io/gitea/models" | |||||
"code.gitea.io/gitea/modules/log" | |||||
"code.gitea.io/gitea/modules/migrations/base" | |||||
"code.gitea.io/gitea/modules/setting" | |||||
"code.gitea.io/gitea/modules/structs" | |||||
) | |||||
// taskQueue is a global queue of tasks | |||||
var taskQueue Queue | |||||
// Run a task | |||||
func Run(t *models.Task) error { | |||||
switch t.Type { | |||||
case structs.TaskTypeMigrateRepo: | |||||
return runMigrateTask(t) | |||||
default: | |||||
return fmt.Errorf("Unknow task type: %d", t.Type) | |||||
} | |||||
} | |||||
// Init will start the service to get all unfinished tasks and run them | |||||
func Init() error { | |||||
switch setting.Task.QueueType { | |||||
case setting.ChannelQueueType: | |||||
taskQueue = NewChannelQueue(setting.Task.QueueLength) | |||||
case setting.RedisQueueType: | |||||
var err error | |||||
addrs, pass, idx, err := parseConnStr(setting.Task.QueueConnStr) | |||||
if err != nil { | |||||
return err | |||||
} | |||||
taskQueue, err = NewRedisQueue(addrs, pass, idx) | |||||
if err != nil { | |||||
return err | |||||
} | |||||
default: | |||||
return fmt.Errorf("Unsupported task queue type: %v", setting.Task.QueueType) | |||||
} | |||||
go func() { | |||||
if err := taskQueue.Run(); err != nil { | |||||
log.Error("taskQueue.Run end failed: %v", err) | |||||
} | |||||
}() | |||||
return nil | |||||
} | |||||
// MigrateRepository add migration repository to task | |||||
func MigrateRepository(doer, u *models.User, opts base.MigrateOptions) error { | |||||
task, err := models.CreateMigrateTask(doer, u, opts) | |||||
if err != nil { | |||||
return err | |||||
} | |||||
return taskQueue.Push(task) | |||||
} |
migrate.migrate_items_options = When migrating from github, input a username and migration options will be displayed. | migrate.migrate_items_options = When migrating from github, input a username and migration options will be displayed. | ||||
migrated_from = Migrated from <a href="%[1]s">%[2]s</a> | migrated_from = Migrated from <a href="%[1]s">%[2]s</a> | ||||
migrated_from_fake = Migrated From %[1]s | migrated_from_fake = Migrated From %[1]s | ||||
migrate.migrating = Migrating from <b>%s</b> ... | |||||
migrate.migrating_failed = Migrating from <b>%s</b> failed. | |||||
mirror_from = mirror of | mirror_from = mirror of | ||||
forked_from = forked from | forked_from = forked from |
}) | }) | ||||
} | } | ||||
function initRepoStatusChecker() { | |||||
const migrating = $("#repo_migrating"); | |||||
$('#repo_migrating_failed').hide(); | |||||
if (migrating) { | |||||
const repo_name = migrating.attr('repo'); | |||||
if (typeof repo_name === 'undefined') { | |||||
return | |||||
} | |||||
$.ajax({ | |||||
type: "GET", | |||||
url: suburl +"/"+repo_name+"/status", | |||||
data: { | |||||
"_csrf": csrf, | |||||
}, | |||||
complete: function(xhr) { | |||||
if (xhr.status == 200) { | |||||
if (xhr.responseJSON) { | |||||
if (xhr.responseJSON["status"] == 0) { | |||||
location.reload(); | |||||
return | |||||
} | |||||
setTimeout(function () { | |||||
initRepoStatusChecker() | |||||
}, 2000); | |||||
return | |||||
} | |||||
} | |||||
$('#repo_migrating_progress').hide(); | |||||
$('#repo_migrating_failed').show(); | |||||
} | |||||
}) | |||||
} | |||||
} | |||||
function initReactionSelector(parent) { | function initReactionSelector(parent) { | ||||
let reactions = ''; | let reactions = ''; | ||||
if (!parent) { | if (!parent) { | ||||
initIssueList(); | initIssueList(); | ||||
initWipTitle(); | initWipTitle(); | ||||
initPullRequestReview(); | initPullRequestReview(); | ||||
initRepoStatusChecker(); | |||||
// Repo clone url. | // Repo clone url. | ||||
if ($('#repo-clone-url').length > 0) { | if ($('#repo-clone-url').length > 0) { |
} | } | ||||
var opts = migrations.MigrateOptions{ | var opts = migrations.MigrateOptions{ | ||||
RemoteURL: remoteAddr, | |||||
Name: form.RepoName, | |||||
CloneAddr: remoteAddr, | |||||
RepoName: form.RepoName, | |||||
Description: form.Description, | Description: form.Description, | ||||
Private: form.Private || setting.Repository.ForcePrivate, | Private: form.Private || setting.Repository.ForcePrivate, | ||||
Mirror: form.Mirror, | Mirror: form.Mirror, |
"code.gitea.io/gitea/modules/markup/external" | "code.gitea.io/gitea/modules/markup/external" | ||||
"code.gitea.io/gitea/modules/setting" | "code.gitea.io/gitea/modules/setting" | ||||
"code.gitea.io/gitea/modules/ssh" | "code.gitea.io/gitea/modules/ssh" | ||||
"code.gitea.io/gitea/modules/task" | |||||
"code.gitea.io/gitea/services/mailer" | "code.gitea.io/gitea/services/mailer" | ||||
mirror_service "code.gitea.io/gitea/services/mirror" | mirror_service "code.gitea.io/gitea/services/mirror" | ||||
mirror_service.InitSyncMirrors() | mirror_service.InitSyncMirrors() | ||||
models.InitDeliverHooks() | models.InitDeliverHooks() | ||||
models.InitTestPullRequests() | models.InitTestPullRequests() | ||||
if err := task.Init(); err != nil { | |||||
log.Fatal("Failed to initialize task scheduler: %v", err) | |||||
} | |||||
} | } | ||||
if setting.EnableSQLite3 { | if setting.EnableSQLite3 { | ||||
log.Info("SQLite3 Supported") | log.Info("SQLite3 Supported") |
repo.OwnerName = ownerName | repo.OwnerName = ownerName | ||||
results.RepoID = repo.ID | results.RepoID = repo.ID | ||||
if repo.IsBeingCreated() { | |||||
ctx.JSON(http.StatusInternalServerError, map[string]interface{}{ | |||||
"results": results, | |||||
"type": "InternalServerError", | |||||
"err": "Repository is being created, you could retry after it finished", | |||||
}) | |||||
return | |||||
} | |||||
// We can shortcut at this point if the repo is a mirror | // We can shortcut at this point if the repo is a mirror | ||||
if mode > models.AccessModeRead && repo.IsMirror { | if mode > models.AccessModeRead && repo.IsMirror { | ||||
ctx.JSON(http.StatusUnauthorized, map[string]interface{}{ | ctx.JSON(http.StatusUnauthorized, map[string]interface{}{ |
"code.gitea.io/gitea/modules/migrations" | "code.gitea.io/gitea/modules/migrations" | ||||
"code.gitea.io/gitea/modules/notification" | "code.gitea.io/gitea/modules/notification" | ||||
"code.gitea.io/gitea/modules/setting" | "code.gitea.io/gitea/modules/setting" | ||||
"code.gitea.io/gitea/modules/task" | |||||
"code.gitea.io/gitea/modules/util" | "code.gitea.io/gitea/modules/util" | ||||
"github.com/unknwon/com" | "github.com/unknwon/com" | ||||
func handleCreateError(ctx *context.Context, owner *models.User, err error, name string, tpl base.TplName, form interface{}) { | func handleCreateError(ctx *context.Context, owner *models.User, err error, name string, tpl base.TplName, form interface{}) { | ||||
switch { | switch { | ||||
case migrations.IsRateLimitError(err): | |||||
ctx.RenderWithErr(ctx.Tr("form.visit_rate_limit"), tpl, form) | |||||
case models.IsErrReachLimitOfRepo(err): | case models.IsErrReachLimitOfRepo(err): | ||||
ctx.RenderWithErr(ctx.Tr("repo.form.reach_limit_of_creation", owner.MaxCreationLimit()), tpl, form) | ctx.RenderWithErr(ctx.Tr("repo.form.reach_limit_of_creation", owner.MaxCreationLimit()), tpl, form) | ||||
case models.IsErrRepoAlreadyExist(err): | case models.IsErrRepoAlreadyExist(err): | ||||
ctx.HTML(200, tplMigrate) | ctx.HTML(200, tplMigrate) | ||||
} | } | ||||
func handleMigrateError(ctx *context.Context, owner *models.User, err error, name string, tpl base.TplName, form *auth.MigrateRepoForm) { | |||||
switch { | |||||
case migrations.IsRateLimitError(err): | |||||
ctx.RenderWithErr(ctx.Tr("form.visit_rate_limit"), tpl, form) | |||||
case migrations.IsTwoFactorAuthError(err): | |||||
ctx.RenderWithErr(ctx.Tr("form.2fa_auth_required"), tpl, form) | |||||
case models.IsErrReachLimitOfRepo(err): | |||||
ctx.RenderWithErr(ctx.Tr("repo.form.reach_limit_of_creation", owner.MaxCreationLimit()), tpl, form) | |||||
case models.IsErrRepoAlreadyExist(err): | |||||
ctx.Data["Err_RepoName"] = true | |||||
ctx.RenderWithErr(ctx.Tr("form.repo_name_been_taken"), tpl, form) | |||||
case models.IsErrNameReserved(err): | |||||
ctx.Data["Err_RepoName"] = true | |||||
ctx.RenderWithErr(ctx.Tr("repo.form.name_reserved", err.(models.ErrNameReserved).Name), tpl, form) | |||||
case models.IsErrNamePatternNotAllowed(err): | |||||
ctx.Data["Err_RepoName"] = true | |||||
ctx.RenderWithErr(ctx.Tr("repo.form.name_pattern_not_allowed", err.(models.ErrNamePatternNotAllowed).Pattern), tpl, form) | |||||
default: | |||||
remoteAddr, _ := form.ParseRemoteAddr(owner) | |||||
err = util.URLSanitizedError(err, remoteAddr) | |||||
if strings.Contains(err.Error(), "Authentication failed") || | |||||
strings.Contains(err.Error(), "Bad credentials") || | |||||
strings.Contains(err.Error(), "could not read Username") { | |||||
ctx.Data["Err_Auth"] = true | |||||
ctx.RenderWithErr(ctx.Tr("form.auth_failed", err.Error()), tpl, form) | |||||
} else if strings.Contains(err.Error(), "fatal:") { | |||||
ctx.Data["Err_CloneAddr"] = true | |||||
ctx.RenderWithErr(ctx.Tr("repo.migrate.failed", err.Error()), tpl, form) | |||||
} else { | |||||
ctx.ServerError(name, err) | |||||
} | |||||
} | |||||
} | |||||
// MigratePost response for migrating from external git repository | // MigratePost response for migrating from external git repository | ||||
func MigratePost(ctx *context.Context, form auth.MigrateRepoForm) { | func MigratePost(ctx *context.Context, form auth.MigrateRepoForm) { | ||||
ctx.Data["Title"] = ctx.Tr("new_migrate") | ctx.Data["Title"] = ctx.Tr("new_migrate") | ||||
} | } | ||||
var opts = migrations.MigrateOptions{ | var opts = migrations.MigrateOptions{ | ||||
RemoteURL: remoteAddr, | |||||
Name: form.RepoName, | |||||
CloneAddr: remoteAddr, | |||||
RepoName: form.RepoName, | |||||
Description: form.Description, | Description: form.Description, | ||||
Private: form.Private || setting.Repository.ForcePrivate, | Private: form.Private || setting.Repository.ForcePrivate, | ||||
Mirror: form.Mirror, | Mirror: form.Mirror, | ||||
opts.Releases = false | opts.Releases = false | ||||
} | } | ||||
repo, err := migrations.MigrateRepository(ctx.User, ctxUser.Name, opts) | |||||
if err == nil { | |||||
notification.NotifyCreateRepository(ctx.User, ctxUser, repo) | |||||
log.Trace("Repository migrated [%d]: %s/%s successfully", repo.ID, ctxUser.Name, form.RepoName) | |||||
ctx.Redirect(setting.AppSubURL + "/" + ctxUser.Name + "/" + form.RepoName) | |||||
err = models.CheckCreateRepository(ctx.User, ctxUser, opts.RepoName) | |||||
if err != nil { | |||||
handleMigrateError(ctx, ctxUser, err, "MigratePost", tplMigrate, &form) | |||||
return | return | ||||
} | } | ||||
switch { | |||||
case models.IsErrReachLimitOfRepo(err): | |||||
ctx.RenderWithErr(ctx.Tr("repo.form.reach_limit_of_creation", ctxUser.MaxCreationLimit()), tplMigrate, &form) | |||||
case models.IsErrNameReserved(err): | |||||
ctx.Data["Err_RepoName"] = true | |||||
ctx.RenderWithErr(ctx.Tr("repo.form.name_reserved", err.(models.ErrNameReserved).Name), tplMigrate, &form) | |||||
case models.IsErrRepoAlreadyExist(err): | |||||
ctx.Data["Err_RepoName"] = true | |||||
ctx.RenderWithErr(ctx.Tr("form.repo_name_been_taken"), tplMigrate, &form) | |||||
case models.IsErrNamePatternNotAllowed(err): | |||||
ctx.Data["Err_RepoName"] = true | |||||
ctx.RenderWithErr(ctx.Tr("repo.form.name_pattern_not_allowed", err.(models.ErrNamePatternNotAllowed).Pattern), tplMigrate, &form) | |||||
case migrations.IsRateLimitError(err): | |||||
ctx.RenderWithErr(ctx.Tr("form.visit_rate_limit"), tplMigrate, &form) | |||||
case migrations.IsTwoFactorAuthError(err): | |||||
ctx.Data["Err_Auth"] = true | |||||
ctx.RenderWithErr(ctx.Tr("form.2fa_auth_required"), tplMigrate, &form) | |||||
default: | |||||
// remoteAddr may contain credentials, so we sanitize it | |||||
err = util.URLSanitizedError(err, remoteAddr) | |||||
if strings.Contains(err.Error(), "Authentication failed") || | |||||
strings.Contains(err.Error(), "Bad credentials") || | |||||
strings.Contains(err.Error(), "could not read Username") { | |||||
ctx.Data["Err_Auth"] = true | |||||
ctx.RenderWithErr(ctx.Tr("form.auth_failed", err.Error()), tplMigrate, &form) | |||||
} else if strings.Contains(err.Error(), "fatal:") { | |||||
ctx.Data["Err_CloneAddr"] = true | |||||
ctx.RenderWithErr(ctx.Tr("repo.migrate.failed", err.Error()), tplMigrate, &form) | |||||
} else { | |||||
ctx.ServerError("MigratePost", err) | |||||
} | |||||
err = task.MigrateRepository(ctx.User, ctxUser, opts) | |||||
if err == nil { | |||||
ctx.Redirect(setting.AppSubURL + "/" + ctxUser.Name + "/" + opts.RepoName) | |||||
return | |||||
} | } | ||||
handleMigrateError(ctx, ctxUser, err, "MigratePost", tplMigrate, &form) | |||||
} | } | ||||
// Action response for actions to a repository | // Action response for actions to a repository | ||||
ctx.ServeFile(archivePath, ctx.Repo.Repository.Name+"-"+refName+ext) | ctx.ServeFile(archivePath, ctx.Repo.Repository.Name+"-"+refName+ext) | ||||
} | } | ||||
// Status returns repository's status | |||||
func Status(ctx *context.Context) { | |||||
task, err := models.GetMigratingTask(ctx.Repo.Repository.ID) | |||||
if err != nil { | |||||
ctx.JSON(500, map[string]interface{}{ | |||||
"err": err, | |||||
}) | |||||
return | |||||
} | |||||
ctx.JSON(200, map[string]interface{}{ | |||||
"status": ctx.Repo.Repository.Status, | |||||
"err": task.Errors, | |||||
}) | |||||
} |
"fmt" | "fmt" | ||||
gotemplate "html/template" | gotemplate "html/template" | ||||
"io/ioutil" | "io/ioutil" | ||||
"net/url" | |||||
"path" | "path" | ||||
"strings" | "strings" | ||||
tplRepoHome base.TplName = "repo/home" | tplRepoHome base.TplName = "repo/home" | ||||
tplWatchers base.TplName = "repo/watchers" | tplWatchers base.TplName = "repo/watchers" | ||||
tplForks base.TplName = "repo/forks" | tplForks base.TplName = "repo/forks" | ||||
tplMigrating base.TplName = "repo/migrating" | |||||
) | ) | ||||
func renderDirectory(ctx *context.Context, treeLink string) { | func renderDirectory(ctx *context.Context, treeLink string) { | ||||
} | } | ||||
} | } | ||||
func safeURL(address string) string { | |||||
u, err := url.Parse(address) | |||||
if err != nil { | |||||
return address | |||||
} | |||||
u.User = nil | |||||
return u.String() | |||||
} | |||||
// Home render repository home page | // Home render repository home page | ||||
func Home(ctx *context.Context) { | func Home(ctx *context.Context) { | ||||
if len(ctx.Repo.Units) > 0 { | if len(ctx.Repo.Units) > 0 { | ||||
if ctx.Repo.Repository.IsBeingCreated() { | |||||
task, err := models.GetMigratingTask(ctx.Repo.Repository.ID) | |||||
if err != nil { | |||||
ctx.ServerError("models.GetMigratingTask", err) | |||||
return | |||||
} | |||||
cfg, err := task.MigrateConfig() | |||||
if err != nil { | |||||
ctx.ServerError("task.MigrateConfig", err) | |||||
return | |||||
} | |||||
ctx.Data["Repo"] = ctx.Repo | |||||
ctx.Data["MigrateTask"] = task | |||||
ctx.Data["CloneAddr"] = safeURL(cfg.CloneAddr) | |||||
ctx.HTML(200, tplMigrating) | |||||
return | |||||
} | |||||
var firstUnit *models.Unit | var firstUnit *models.Unit | ||||
for _, repoUnit := range ctx.Repo.Units { | for _, repoUnit := range ctx.Repo.Units { | ||||
if repoUnit.Type == models.UnitTypeCode { | if repoUnit.Type == models.UnitTypeCode { |
m.Get("/archive/*", repo.MustBeNotEmpty, reqRepoCodeReader, repo.Download) | m.Get("/archive/*", repo.MustBeNotEmpty, reqRepoCodeReader, repo.Download) | ||||
m.Get("/status", reqRepoCodeReader, repo.Status) | |||||
m.Group("/branches", func() { | m.Group("/branches", func() { | ||||
m.Get("", repo.Branches) | m.Get("", repo.Branches) | ||||
}, repo.MustBeNotEmpty, context.RepoRef(), reqRepoCodeReader) | }, repo.MustBeNotEmpty, context.RepoRef(), reqRepoCodeReader) |
"code.gitea.io/gitea/models" | "code.gitea.io/gitea/models" | ||||
"code.gitea.io/gitea/modules/git" | "code.gitea.io/gitea/modules/git" | ||||
"code.gitea.io/gitea/modules/structs" | |||||
release_service "code.gitea.io/gitea/services/release" | release_service "code.gitea.io/gitea/services/release" | ||||
"github.com/stretchr/testify/assert" | "github.com/stretchr/testify/assert" | ||||
repo := models.AssertExistsAndLoadBean(t, &models.Repository{ID: 1}).(*models.Repository) | repo := models.AssertExistsAndLoadBean(t, &models.Repository{ID: 1}).(*models.Repository) | ||||
repoPath := models.RepoPath(user.Name, repo.Name) | repoPath := models.RepoPath(user.Name, repo.Name) | ||||
migrationOptions := models.MigrateRepoOptions{ | |||||
Name: "test_mirror", | |||||
Description: "Test mirror", | |||||
IsPrivate: false, | |||||
IsMirror: true, | |||||
RemoteAddr: repoPath, | |||||
Wiki: true, | |||||
SyncReleasesWithTags: true, | |||||
opts := structs.MigrateRepoOption{ | |||||
RepoName: "test_mirror", | |||||
Description: "Test mirror", | |||||
Private: false, | |||||
Mirror: true, | |||||
CloneAddr: repoPath, | |||||
Wiki: true, | |||||
Releases: false, | |||||
} | } | ||||
mirror, err := models.MigrateRepository(user, user, migrationOptions) | |||||
mirrorRepo, err := models.CreateRepository(user, user, models.CreateRepoOptions{ | |||||
Name: opts.RepoName, | |||||
Description: opts.Description, | |||||
IsPrivate: opts.Private, | |||||
IsMirror: opts.Mirror, | |||||
Status: models.RepositoryBeingMigrated, | |||||
}) | |||||
assert.NoError(t, err) | |||||
mirror, err := models.MigrateRepositoryGitData(user, user, mirrorRepo, opts) | |||||
assert.NoError(t, err) | assert.NoError(t, err) | ||||
gitRepo, err := git.OpenRepository(repoPath) | gitRepo, err := git.OpenRepository(repoPath) |
{{if .IsMirror}}<div class="fork-flag">{{$.i18n.Tr "repo.mirror_from"}} <a target="_blank" rel="noopener noreferrer" href="{{MirrorAddress $.Mirror}}">{{MirrorAddress $.Mirror}}</a></div>{{end}} | {{if .IsMirror}}<div class="fork-flag">{{$.i18n.Tr "repo.mirror_from"}} <a target="_blank" rel="noopener noreferrer" href="{{MirrorAddress $.Mirror}}">{{MirrorAddress $.Mirror}}</a></div>{{end}} | ||||
{{if .IsFork}}<div class="fork-flag">{{$.i18n.Tr "repo.forked_from"}} <a href="{{.BaseRepo.Link}}">{{SubStr .BaseRepo.RelLink 1 -1}}</a></div>{{end}} | {{if .IsFork}}<div class="fork-flag">{{$.i18n.Tr "repo.forked_from"}} <a href="{{.BaseRepo.Link}}">{{SubStr .BaseRepo.RelLink 1 -1}}</a></div>{{end}} | ||||
</div> | </div> | ||||
<div class="repo-buttons"> | |||||
<div class="ui labeled button" tabindex="0"> | |||||
<a class="ui compact basic button" href="{{$.RepoLink}}/action/{{if $.IsWatchingRepo}}un{{end}}watch?redirect_to={{$.Link}}"> | |||||
<i class="icon fa-eye{{if not $.IsWatchingRepo}}-slash{{end}}"></i>{{if $.IsWatchingRepo}}{{$.i18n.Tr "repo.unwatch"}}{{else}}{{$.i18n.Tr "repo.watch"}}{{end}} | |||||
</a> | |||||
<a class="ui basic label" href="{{.Link}}/watchers"> | |||||
{{.NumWatches}} | |||||
</a> | |||||
</div> | |||||
<div class="ui labeled button" tabindex="0"> | |||||
<a class="ui compact basic button" href="{{$.RepoLink}}/action/{{if $.IsStaringRepo}}un{{end}}star?redirect_to={{$.Link}}"> | |||||
<i class="icon star{{if not $.IsStaringRepo}} outline{{end}}"></i>{{if $.IsStaringRepo}}{{$.i18n.Tr "repo.unstar"}}{{else}}{{$.i18n.Tr "repo.star"}}{{end}} | |||||
</a> | |||||
<a class="ui basic label" href="{{.Link}}/stars"> | |||||
{{.NumStars}} | |||||
</a> | |||||
</div> | |||||
{{if and (not .IsEmpty) ($.Permission.CanRead $.UnitTypeCode)}} | |||||
<div class="ui labeled button {{if and ($.IsSigned) (not $.CanSignedUserFork)}}disabled-repo-button{{end}}" tabindex="0"> | |||||
<a class="ui compact basic button {{if or (not $.IsSigned) (not $.CanSignedUserFork)}}poping up{{end}}" {{if $.CanSignedUserFork}}href="{{AppSubUrl}}/repo/fork/{{.ID}}"{{else if $.IsSigned}} data-content="{{$.i18n.Tr "repo.fork_from_self"}}" {{ else }} data-content="{{$.i18n.Tr "repo.fork_guest_user" }}" href="{{AppSubUrl}}/user/login?redirect_to={{AppSubUrl}}/repo/fork/{{.ID}}" {{end}} data-position="top center" data-variation="tiny"> | |||||
<i class="octicon octicon-repo-forked"></i>{{$.i18n.Tr "repo.fork"}} | |||||
{{if not .IsBeingCreated}} | |||||
<div class="repo-buttons"> | |||||
<div class="ui labeled button" tabindex="0"> | |||||
<a class="ui compact basic button" href="{{$.RepoLink}}/action/{{if $.IsWatchingRepo}}un{{end}}watch?redirect_to={{$.Link}}"> | |||||
<i class="icon fa-eye{{if not $.IsWatchingRepo}}-slash{{end}}"></i>{{if $.IsWatchingRepo}}{{$.i18n.Tr "repo.unwatch"}}{{else}}{{$.i18n.Tr "repo.watch"}}{{end}} | |||||
</a> | </a> | ||||
<a class="ui basic label" href="{{.Link}}/forks"> | |||||
{{.NumForks}} | |||||
<a class="ui basic label" href="{{.Link}}/watchers"> | |||||
{{.NumWatches}} | |||||
</a> | </a> | ||||
</div> | </div> | ||||
{{end}} | |||||
</div> | |||||
<div class="ui labeled button" tabindex="0"> | |||||
<a class="ui compact basic button" href="{{$.RepoLink}}/action/{{if $.IsStaringRepo}}un{{end}}star?redirect_to={{$.Link}}"> | |||||
<i class="icon star{{if not $.IsStaringRepo}} outline{{end}}"></i>{{if $.IsStaringRepo}}{{$.i18n.Tr "repo.unstar"}}{{else}}{{$.i18n.Tr "repo.star"}}{{end}} | |||||
</a> | |||||
<a class="ui basic label" href="{{.Link}}/stars"> | |||||
{{.NumStars}} | |||||
</a> | |||||
</div> | |||||
{{if and (not .IsEmpty) ($.Permission.CanRead $.UnitTypeCode)}} | |||||
<div class="ui labeled button {{if and ($.IsSigned) (not $.CanSignedUserFork)}}disabled-repo-button{{end}}" tabindex="0"> | |||||
<a class="ui compact basic button {{if or (not $.IsSigned) (not $.CanSignedUserFork)}}poping up{{end}}" {{if $.CanSignedUserFork}}href="{{AppSubUrl}}/repo/fork/{{.ID}}"{{else if $.IsSigned}} data-content="{{$.i18n.Tr "repo.fork_from_self"}}" {{ else }} data-content="{{$.i18n.Tr "repo.fork_guest_user" }}" href="{{AppSubUrl}}/user/login?redirect_to={{AppSubUrl}}/repo/fork/{{.ID}}" {{end}} data-position="top center" data-variation="tiny"> | |||||
<i class="octicon octicon-repo-forked"></i>{{$.i18n.Tr "repo.fork"}} | |||||
</a> | |||||
<a class="ui basic label" href="{{.Link}}/forks"> | |||||
{{.NumForks}} | |||||
</a> | |||||
</div> | |||||
{{end}} | |||||
</div> | |||||
{{end}} | |||||
</div><!-- end grid --> | </div><!-- end grid --> | ||||
</div><!-- end container --> | </div><!-- end container --> | ||||
{{end}} | {{end}} | ||||
<div class="ui tabs container"> | <div class="ui tabs container"> | ||||
<div class="ui tabular stackable menu navbar"> | |||||
{{if .Permission.CanRead $.UnitTypeCode}} | |||||
<a class="{{if .PageIsViewCode}}active{{end}} item" href="{{.RepoLink}}{{if (ne .BranchName .Repository.DefaultBranch)}}/src/{{.BranchNameSubURL | EscapePound}}{{end}}"> | |||||
<i class="octicon octicon-code"></i> {{.i18n.Tr "repo.code"}} | |||||
</a> | |||||
{{end}} | |||||
{{if .Permission.CanRead $.UnitTypeIssues}} | |||||
<a class="{{if .PageIsIssueList}}active{{end}} item" href="{{.RepoLink}}/issues"> | |||||
<i class="octicon octicon-issue-opened"></i> {{.i18n.Tr "repo.issues"}} <span class="ui {{if not .Repository.NumOpenIssues}}gray{{else}}blue{{end}} small label">{{.Repository.NumOpenIssues}}</span> | |||||
{{if not .Repository.IsBeingCreated}} | |||||
<div class="ui tabular stackable menu navbar"> | |||||
{{if .Permission.CanRead $.UnitTypeCode}} | |||||
<a class="{{if .PageIsViewCode}}active{{end}} item" href="{{.RepoLink}}{{if (ne .BranchName .Repository.DefaultBranch)}}/src/{{.BranchNameSubURL | EscapePound}}{{end}}"> | |||||
<i class="octicon octicon-code"></i> {{.i18n.Tr "repo.code"}} | |||||
</a> | </a> | ||||
{{end}} | |||||
{{if .Permission.CanRead $.UnitTypeExternalTracker}} | |||||
<a class="{{if .PageIsIssueList}}active{{end}} item" href="{{.RepoExternalIssuesLink}}" target="_blank" rel="noopener noreferrer"> | |||||
<i class="octicon octicon-link-external"></i> {{.i18n.Tr "repo.issues"}} </span> | |||||
</a> | |||||
{{end}} | |||||
{{end}} | |||||
{{if and .Repository.CanEnablePulls (.Permission.CanRead $.UnitTypePullRequests)}} | |||||
<a class="{{if .PageIsPullList}}active{{end}} item" href="{{.RepoLink}}/pulls"> | |||||
<i class="octicon octicon-git-pull-request"></i> {{.i18n.Tr "repo.pulls"}} <span class="ui {{if not .Repository.NumOpenPulls}}gray{{else}}blue{{end}} small label">{{.Repository.NumOpenPulls}}</span> | |||||
</a> | |||||
{{end}} | |||||
{{if .Permission.CanRead $.UnitTypeIssues}} | |||||
<a class="{{if .PageIsIssueList}}active{{end}} item" href="{{.RepoLink}}/issues"> | |||||
<i class="octicon octicon-issue-opened"></i> {{.i18n.Tr "repo.issues"}} <span class="ui {{if not .Repository.NumOpenIssues}}gray{{else}}blue{{end}} small label">{{.Repository.NumOpenIssues}}</span> | |||||
</a> | |||||
{{end}} | |||||
{{if and (.Permission.CanRead $.UnitTypeReleases) (not .IsEmptyRepo) }} | |||||
<a class="{{if .PageIsReleaseList}}active{{end}} item" href="{{.RepoLink}}/releases"> | |||||
<i class="octicon octicon-tag"></i> {{.i18n.Tr "repo.releases"}} <span class="ui {{if not .Repository.NumReleases}}gray{{else}}blue{{end}} small label">{{.Repository.NumReleases}}</span> | |||||
</a> | |||||
{{end}} | |||||
{{if .Permission.CanRead $.UnitTypeExternalTracker}} | |||||
<a class="{{if .PageIsIssueList}}active{{end}} item" href="{{.RepoExternalIssuesLink}}" target="_blank" rel="noopener noreferrer"> | |||||
<i class="octicon octicon-link-external"></i> {{.i18n.Tr "repo.issues"}} </span> | |||||
</a> | |||||
{{end}} | |||||
{{if or (.Permission.CanRead $.UnitTypeWiki) (.Permission.CanRead $.UnitTypeExternalWiki)}} | |||||
<a class="{{if .PageIsWiki}}active{{end}} item" href="{{.RepoLink}}/wiki" {{if (.Permission.CanRead $.UnitTypeExternalWiki)}} target="_blank" rel="noopener noreferrer" {{end}}> | |||||
<i class="octicon octicon-book"></i> {{.i18n.Tr "repo.wiki"}} | |||||
</a> | |||||
{{end}} | |||||
{{if and .Repository.CanEnablePulls (.Permission.CanRead $.UnitTypePullRequests)}} | |||||
<a class="{{if .PageIsPullList}}active{{end}} item" href="{{.RepoLink}}/pulls"> | |||||
<i class="octicon octicon-git-pull-request"></i> {{.i18n.Tr "repo.pulls"}} <span class="ui {{if not .Repository.NumOpenPulls}}gray{{else}}blue{{end}} small label">{{.Repository.NumOpenPulls}}</span> | |||||
</a> | |||||
{{end}} | |||||
{{if and (.Permission.CanReadAny $.UnitTypePullRequests $.UnitTypeIssues $.UnitTypeReleases) (not .IsEmptyRepo)}} | |||||
<a class="{{if .PageIsActivity}}active{{end}} item" href="{{.RepoLink}}/activity"> | |||||
<i class="octicon octicon-pulse"></i> {{.i18n.Tr "repo.activity"}} | |||||
{{if and (.Permission.CanRead $.UnitTypeReleases) (not .IsEmptyRepo) }} | |||||
<a class="{{if .PageIsReleaseList}}active{{end}} item" href="{{.RepoLink}}/releases"> | |||||
<i class="octicon octicon-tag"></i> {{.i18n.Tr "repo.releases"}} <span class="ui {{if not .Repository.NumReleases}}gray{{else}}blue{{end}} small label">{{.Repository.NumReleases}}</span> | |||||
</a> | </a> | ||||
{{end}} | |||||
{{end}} | |||||
{{template "custom/extra_tabs" .}} | |||||
{{if or (.Permission.CanRead $.UnitTypeWiki) (.Permission.CanRead $.UnitTypeExternalWiki)}} | |||||
<a class="{{if .PageIsWiki}}active{{end}} item" href="{{.RepoLink}}/wiki" {{if (.Permission.CanRead $.UnitTypeExternalWiki)}} target="_blank" rel="noopener noreferrer" {{end}}> | |||||
<i class="octicon octicon-book"></i> {{.i18n.Tr "repo.wiki"}} | |||||
</a> | |||||
{{end}} | |||||
{{if .Permission.IsAdmin}} | |||||
<div class="right menu"> | |||||
<a class="{{if .PageIsSettings}}active{{end}} item" href="{{.RepoLink}}/settings"> | |||||
<i class="octicon octicon-tools"></i> {{.i18n.Tr "repo.settings"}} | |||||
{{if and (.Permission.CanReadAny $.UnitTypePullRequests $.UnitTypeIssues $.UnitTypeReleases) (not .IsEmptyRepo)}} | |||||
<a class="{{if .PageIsActivity}}active{{end}} item" href="{{.RepoLink}}/activity"> | |||||
<i class="octicon octicon-pulse"></i> {{.i18n.Tr "repo.activity"}} | |||||
</a> | </a> | ||||
</div> | |||||
{{end}} | |||||
</div> | |||||
{{end}} | |||||
{{template "custom/extra_tabs" .}} | |||||
{{if .Permission.IsAdmin}} | |||||
<div class="right menu"> | |||||
<a class="{{if .PageIsSettings}}active{{end}} item" href="{{.RepoLink}}/settings"> | |||||
<i class="octicon octicon-tools"></i> {{.i18n.Tr "repo.settings"}} | |||||
</a> | |||||
</div> | |||||
{{end}} | |||||
</div> | |||||
{{end}} | |||||
</div> | </div> | ||||
<div class="ui tabs divider"></div> | <div class="ui tabs divider"></div> | ||||
</div> | </div> |
{{template "base/head" .}} | |||||
<div class="repository quickstart"> | |||||
{{template "repo/header" .}} | |||||
<div class="ui container"> | |||||
<div class="ui grid"> | |||||
<div class="sixteen wide column content"> | |||||
{{template "base/alert" .}} | |||||
<div class="home"> | |||||
<div class="ui stackable middle very relaxed page grid"> | |||||
<div id="repo_migrating" class="sixteen wide center aligned centered column" repo="{{.Repo.Repository.FullName}}"> | |||||
<div> | |||||
<img src="{{AppSubUrl}}/img/loading.png"/> | |||||
</div> | |||||
</div> | |||||
</div> | |||||
<div class="ui stackable middle very relaxed page grid"> | |||||
<div class="sixteen wide center aligned centered column"> | |||||
<div id="repo_migrating_progress"> | |||||
<p>{{.i18n.Tr "repo.migrate.migrating" .CloneAddr | Safe}}</p> | |||||
</div> | |||||
<div id="repo_migrating_failed"> | |||||
<p>{{.i18n.Tr "repo.migrate.migrating_failed" .CloneAddr | Safe}}</p> | |||||
</div> | |||||
</div> | |||||
</div> | |||||
</div> | |||||
</div> | |||||
</div> | |||||
</div> | |||||
</div> | |||||
{{template "base/footer" .}} |