Refactor: Move login out of models (#16199)

`models` does far too much. In particular it handles all `UserSignin`.

It shouldn't be responsible for calling LDAP, SMTP or PAM for signing in.

Therefore we should move this code out of `models`.

This code has to depend on `models` - therefore it belongs in `services`.

There is a package in `services` called `auth` and clearly this functionality belongs in there.

Plan:

- [x] Change `auth.Auth` to `auth.Method` - as they represent methods of authentication.
- [x] Move `models.UserSignIn` into `auth`
- [x] Move `models.ExternalUserLogin`
- [x] Move most of the `LoginVia*` methods to `auth` or subpackages
- [x] Move Resynchronize functionality to `auth`
  - Involved some restructuring of `models/ssh_key.go` to reduce the size of this massive file and simplify its files.
- [x] Move the rest of the LDAP functionality in to the ldap subpackage
- [x] Re-factor the login sources to express an interfaces `auth.Source`?
  - I've done this through some smaller interfaces Authenticator and Synchronizable - which would allow us to extend things in future
- [x] Now LDAP is out of models - need to think about modules/auth/ldap and I think all of that functionality might just be moveable
- [x] Similarly a lot Oauth2 functionality need not be in models too and should be moved to services/auth/source/oauth2
  - [x] modules/auth/oauth2/oauth2.go uses xorm... This is naughty - probably need to move this into models.
  - [x] models/oauth2.go - mostly should be in modules/auth/oauth2 or services/auth/source/oauth2 
- [x] More simplifications of login_source.go may need to be done
- Allow wiring in of notify registration -  *this can now easily be done - but I think we should do it in another PR*  - see #16178 
- More refactors...?
  - OpenID should probably become an auth Method but I think that can be left for another PR
  - Methods should also probably be cleaned up  - again another PR I think.
  - SSPI still needs more refactors.* Rename auth.Auth auth.Method
* Restructure ssh_key.go

- move functions from models/user.go that relate to ssh_key to ssh_key
- split ssh_key.go to try create clearer function domains for allow for
future refactors here.

Signed-off-by: Andrew Thornton <art27@cantab.net>

8 files changed, 1035 insertions, 0 deletions

diff --git a/services/auth/source/ldap/README.md b/services/auth/source/ldap/README.md
new file mode 100644
index 0000000000..3a839fa314
--- /dev/null
+++ b/services/auth/source/ldap/README.md
@@ -0,0 +1,122 @@
+# Gitea LDAP Authentication Module
+
+## About
+
+This authentication module attempts to authorize and authenticate a user
+against an LDAP server. It provides two methods of authentication: LDAP via
+BindDN, and LDAP simple authentication.
+
+LDAP via BindDN functions like most LDAP authentication systems. First, it
+queries the LDAP server using a Bind DN and searches for the user that is
+attempting to sign in. If the user is found, the module attempts to bind to the
+server using the user's supplied credentials. If this succeeds, the user has
+been authenticated, and his account information is retrieved and passed to the
+Gogs login infrastructure.
+
+LDAP simple authentication does not utilize a Bind DN. Instead, it binds
+directly with the LDAP server using the user's supplied credentials. If the bind
+succeeds and no filter rules out the user, the user is authenticated.
+
+LDAP via BindDN is recommended for most users. By using a Bind DN, the server
+can perform authorization by restricting which entries the Bind DN account can
+read. Further, using a Bind DN with reduced permissions can reduce security risk
+in the face of application bugs.
+
+## Usage
+
+To use this module, add an LDAP authentication source via the Authentications
+section in the admin panel. Both the LDAP via BindDN and the simple auth LDAP
+share the following fields:
+
+* Authorization Name **(required)**
+  * A name to assign to the new method of authorization.
+
+* Host **(required)**
+  * The address where the LDAP server can be reached.
+  * Example: mydomain.com
+
+* Port **(required)**
+  * The port to use when connecting to the server.
+  * Example: 636
+
+* Enable TLS Encryption (optional)
+  * Whether to use TLS when connecting to the LDAP server.
+
+* Admin Filter (optional)
+  * An LDAP filter specifying if a user should be given administrator
+      privileges. If a user accounts passes the filter, the user will be
+      privileged as an administrator.
+  * Example: (objectClass=adminAccount)
+
+* First name attribute (optional)
+  * The attribute of the user's LDAP record containing the user's first name.
+      This will be used to populate their account information.
+  * Example: givenName
+
+* Surname attribute (optional)
+  * The attribute of the user's LDAP record containing the user's surname This
+      will be used to populate their account information.
+  * Example: sn
+
+* E-mail attribute **(required)**
+  * The attribute of the user's LDAP record containing the user's email
+      address. This will be used to populate their account information.
+  * Example: mail
+
+**LDAP via BindDN** adds the following fields:
+
+* Bind DN (optional)
+  * The DN to bind to the LDAP server with when searching for the user. This
+      may be left blank to perform an anonymous search.
+  * Example: cn=Search,dc=mydomain,dc=com
+
+* Bind Password (optional)
+  * The password for the Bind DN specified above, if any. _Note: The password
+      is stored in plaintext at the server. As such, ensure that your Bind DN
+      has as few privileges as possible._
+
+* User Search Base **(required)**
+  * The LDAP base at which user accounts will be searched for.
+  * Example: ou=Users,dc=mydomain,dc=com
+
+* User Filter **(required)**
+  * An LDAP filter declaring how to find the user record that is attempting to
+      authenticate. The '%s' matching parameter will be substituted with the
+      user's username.
+  * Example: (&(objectClass=posixAccount)(uid=%s))
+
+**LDAP using simple auth** adds the following fields:
+
+* User DN **(required)**
+  * A template to use as the user's DN. The `%s` matching parameter will be
+      substituted with the user's username.
+  * Example: cn=%s,ou=Users,dc=mydomain,dc=com
+  * Example: uid=%s,ou=Users,dc=mydomain,dc=com
+
+* User Search Base (optional)
+  * The LDAP base at which user accounts will be searched for.
+  * Example: ou=Users,dc=mydomain,dc=com
+
+* User Filter **(required)**
+  * An LDAP filter declaring when a user should be allowed to log in. The `%s`
+      matching parameter will be substituted with the user's username.
+  * Example: (&(objectClass=posixAccount)(cn=%s))
+  * Example: (&(objectClass=posixAccount)(uid=%s))
+
+**Verify group membership in LDAP** uses the following fields:
+
+* Group Search Base (optional)
+  * The LDAP DN used for groups.
+  * Example: ou=group,dc=mydomain,dc=com
+
+* Group Name Filter (optional)
+  * An LDAP filter declaring how to find valid groups in the above DN.
+  * Example: (|(cn=gitea_users)(cn=admins))
+
+* User Attribute in Group (optional)
+  * Which user LDAP attribute is listed in the group.
+  * Example: uid
+
+* Group Attribute for User (optional)
+  * Which group LDAP attribute contains an array above user attribute names.
+  * Example: memberUid
diff --git a/services/auth/source/ldap/assert_interface_test.go b/services/auth/source/ldap/assert_interface_test.go
new file mode 100644
index 0000000000..4cf3eafe76
--- /dev/null
+++ b/services/auth/source/ldap/assert_interface_test.go
@@ -0,0 +1,27 @@
+// Copyright 2021 The Gitea Authors. All rights reserved.
+// Use of this source code is governed by a MIT-style
+// license that can be found in the LICENSE file.
+
+package ldap_test
+
+import (
+	"code.gitea.io/gitea/models"
+	"code.gitea.io/gitea/services/auth"
+	"code.gitea.io/gitea/services/auth/source/ldap"
+)
+
+// This test file exists to assert that our Source exposes the interfaces that we expect
+// It tightly binds the interfaces and implementation without breaking go import cycles
+
+type sourceInterface interface {
+	auth.PasswordAuthenticator
+	auth.SynchronizableSource
+	models.SSHKeyProvider
+	models.LoginConfig
+	models.SkipVerifiable
+	models.HasTLSer
+	models.UseTLSer
+	models.LoginSourceSettable
+}
+
+var _ (sourceInterface) = &ldap.Source{}
diff --git a/services/auth/source/ldap/security_protocol.go b/services/auth/source/ldap/security_protocol.go
new file mode 100644
index 0000000000..47c9d30e5c
--- /dev/null
+++ b/services/auth/source/ldap/security_protocol.go
@@ -0,0 +1,27 @@
+// Copyright 2021 The Gitea Authors. All rights reserved.
+// Use of this source code is governed by a MIT-style
+// license that can be found in the LICENSE file.
+
+package ldap
+
+// SecurityProtocol protocol type
+type SecurityProtocol int
+
+// Note: new type must be added at the end of list to maintain compatibility.
+const (
+	SecurityProtocolUnencrypted SecurityProtocol = iota
+	SecurityProtocolLDAPS
+	SecurityProtocolStartTLS
+)
+
+// String returns the name of the SecurityProtocol
+func (s SecurityProtocol) String() string {
+	return SecurityProtocolNames[s]
+}
+
+// SecurityProtocolNames contains the name of SecurityProtocol values.
+var SecurityProtocolNames = map[SecurityProtocol]string{
+	SecurityProtocolUnencrypted: "Unencrypted",
+	SecurityProtocolLDAPS:       "LDAPS",
+	SecurityProtocolStartTLS:    "StartTLS",
+}
diff --git a/services/auth/source/ldap/source.go b/services/auth/source/ldap/source.go
new file mode 100644
index 0000000000..87be0117ee
--- /dev/null
+++ b/services/auth/source/ldap/source.go
@@ -0,0 +1,120 @@
+// Copyright 2021 The Gitea Authors. All rights reserved.
+// Use of this source code is governed by a MIT-style
+// license that can be found in the LICENSE file.
+
+package ldap
+
+import (
+	"strings"
+
+	"code.gitea.io/gitea/models"
+	"code.gitea.io/gitea/modules/secret"
+	"code.gitea.io/gitea/modules/setting"
+
+	jsoniter "github.com/json-iterator/go"
+)
+
+// .____     ________      _____ __________
+// |    |    \______ \    /  _  \\______   \
+// |    |     |    |  \  /  /_\  \|     ___/
+// |    |___  |    `   \/    |    \    |
+// |_______ \/_______  /\____|__  /____|
+//         \/        \/         \/
+
+// Package ldap provide functions & structure to query a LDAP ldap directory
+// For now, it's mainly tested again an MS Active Directory service, see README.md for more information
+
+// Source Basic LDAP authentication service
+type Source struct {
+	Name                  string // canonical name (ie. corporate.ad)
+	Host                  string // LDAP host
+	Port                  int    // port number
+	SecurityProtocol      SecurityProtocol
+	SkipVerify            bool
+	BindDN                string // DN to bind with
+	BindPasswordEncrypt   string // Encrypted Bind BN password
+	BindPassword          string // Bind DN password
+	UserBase              string // Base search path for users
+	UserDN                string // Template for the DN of the user for simple auth
+	AttributeUsername     string // Username attribute
+	AttributeName         string // First name attribute
+	AttributeSurname      string // Surname attribute
+	AttributeMail         string // E-mail attribute
+	AttributesInBind      bool   // fetch attributes in bind context (not user)
+	AttributeSSHPublicKey string // LDAP SSH Public Key attribute
+	SearchPageSize        uint32 // Search with paging page size
+	Filter                string // Query filter to validate entry
+	AdminFilter           string // Query filter to check if user is admin
+	RestrictedFilter      string // Query filter to check if user is restricted
+	Enabled               bool   // if this source is disabled
+	AllowDeactivateAll    bool   // Allow an empty search response to deactivate all users from this source
+	GroupsEnabled         bool   // if the group checking is enabled
+	GroupDN               string // Group Search Base
+	GroupFilter           string // Group Name Filter
+	GroupMemberUID        string // Group Attribute containing array of UserUID
+	UserUID               string // User Attribute listed in Group
+
+	// reference to the loginSource
+	loginSource *models.LoginSource
+}
+
+// FromDB fills up a LDAPConfig from serialized format.
+func (source *Source) FromDB(bs []byte) error {
+	err := models.JSONUnmarshalHandleDoubleEncode(bs, &source)
+	if err != nil {
+		return err
+	}
+	if source.BindPasswordEncrypt != "" {
+		source.BindPassword, err = secret.DecryptSecret(setting.SecretKey, source.BindPasswordEncrypt)
+		source.BindPasswordEncrypt = ""
+	}
+	return err
+}
+
+// ToDB exports a LDAPConfig to a serialized format.
+func (source *Source) ToDB() ([]byte, error) {
+	var err error
+	source.BindPasswordEncrypt, err = secret.EncryptSecret(setting.SecretKey, source.BindPassword)
+	if err != nil {
+		return nil, err
+	}
+	source.BindPassword = ""
+	json := jsoniter.ConfigCompatibleWithStandardLibrary
+	return json.Marshal(source)
+}
+
+// SecurityProtocolName returns the name of configured security
+// protocol.
+func (source *Source) SecurityProtocolName() string {
+	return SecurityProtocolNames[source.SecurityProtocol]
+}
+
+// IsSkipVerify returns if SkipVerify is set
+func (source *Source) IsSkipVerify() bool {
+	return source.SkipVerify
+}
+
+// HasTLS returns if HasTLS
+func (source *Source) HasTLS() bool {
+	return source.SecurityProtocol > SecurityProtocolUnencrypted
+}
+
+// UseTLS returns if UseTLS
+func (source *Source) UseTLS() bool {
+	return source.SecurityProtocol != SecurityProtocolUnencrypted
+}
+
+// ProvidesSSHKeys returns if this source provides SSH Keys
+func (source *Source) ProvidesSSHKeys() bool {
+	return len(strings.TrimSpace(source.AttributeSSHPublicKey)) > 0
+}
+
+// SetLoginSource sets the related LoginSource
+func (source *Source) SetLoginSource(loginSource *models.LoginSource) {
+	source.loginSource = loginSource
+}
+
+func init() {
+	models.RegisterLoginTypeConfig(models.LoginLDAP, &Source{})
+	models.RegisterLoginTypeConfig(models.LoginDLDAP, &Source{})
+}
diff --git a/services/auth/source/ldap/source_authenticate.go b/services/auth/source/ldap/source_authenticate.go
new file mode 100644
index 0000000000..1d5e69539b
--- /dev/null
+++ b/services/auth/source/ldap/source_authenticate.go
@@ -0,0 +1,93 @@
+// Copyright 2021 The Gitea Authors. All rights reserved.
+// Use of this source code is governed by a MIT-style
+// license that can be found in the LICENSE file.
+
+package ldap
+
+import (
+	"fmt"
+	"strings"
+
+	"code.gitea.io/gitea/models"
+)
+
+// Authenticate queries if login/password is valid against the LDAP directory pool,
+// and create a local user if success when enabled.
+func (source *Source) Authenticate(user *models.User, login, password string) (*models.User, error) {
+	sr := source.SearchEntry(login, password, source.loginSource.Type == models.LoginDLDAP)
+	if sr == nil {
+		// User not in LDAP, do nothing
+		return nil, models.ErrUserNotExist{Name: login}
+	}
+
+	isAttributeSSHPublicKeySet := len(strings.TrimSpace(source.AttributeSSHPublicKey)) > 0
+
+	// Update User admin flag if exist
+	if isExist, err := models.IsUserExist(0, sr.Username); err != nil {
+		return nil, err
+	} else if isExist {
+		if user == nil {
+			user, err = models.GetUserByName(sr.Username)
+			if err != nil {
+				return nil, err
+			}
+		}
+		if user != nil && !user.ProhibitLogin {
+			cols := make([]string, 0)
+			if len(source.AdminFilter) > 0 && user.IsAdmin != sr.IsAdmin {
+				// Change existing admin flag only if AdminFilter option is set
+				user.IsAdmin = sr.IsAdmin
+				cols = append(cols, "is_admin")
+			}
+			if !user.IsAdmin && len(source.RestrictedFilter) > 0 && user.IsRestricted != sr.IsRestricted {
+				// Change existing restricted flag only if RestrictedFilter option is set
+				user.IsRestricted = sr.IsRestricted
+				cols = append(cols, "is_restricted")
+			}
+			if len(cols) > 0 {
+				err = models.UpdateUserCols(user, cols...)
+				if err != nil {
+					return nil, err
+				}
+			}
+		}
+	}
+
+	if user != nil {
+		if isAttributeSSHPublicKeySet && models.SynchronizePublicKeys(user, source.loginSource, sr.SSHPublicKey) {
+			return user, models.RewriteAllPublicKeys()
+		}
+
+		return user, nil
+	}
+
+	// Fallback.
+	if len(sr.Username) == 0 {
+		sr.Username = login
+	}
+
+	if len(sr.Mail) == 0 {
+		sr.Mail = fmt.Sprintf("%s@localhost", sr.Username)
+	}
+
+	user = &models.User{
+		LowerName:    strings.ToLower(sr.Username),
+		Name:         sr.Username,
+		FullName:     composeFullName(sr.Name, sr.Surname, sr.Username),
+		Email:        sr.Mail,
+		LoginType:    source.loginSource.Type,
+		LoginSource:  source.loginSource.ID,
+		LoginName:    login,
+		IsActive:     true,
+		IsAdmin:      sr.IsAdmin,
+		IsRestricted: sr.IsRestricted,
+	}
+
+	err := models.CreateUser(user)
+
+	if err == nil && isAttributeSSHPublicKeySet && models.AddPublicKeysBySource(user, source.loginSource, sr.SSHPublicKey) {
+		err = models.RewriteAllPublicKeys()
+	}
+
+	return user, err
+}
diff --git a/services/auth/source/ldap/source_search.go b/services/auth/source/ldap/source_search.go
new file mode 100644
index 0000000000..e99fc67901
--- /dev/null
+++ b/services/auth/source/ldap/source_search.go
@@ -0,0 +1,443 @@
+// Copyright 2014 The Gogs Authors. All rights reserved.
+// Copyright 2020 The Gitea Authors. All rights reserved.
+// Use of this source code is governed by a MIT-style
+// license that can be found in the LICENSE file.
+
+package ldap
+
+import (
+	"crypto/tls"
+	"fmt"
+	"strings"
+
+	"code.gitea.io/gitea/modules/log"
+
+	"github.com/go-ldap/ldap/v3"
+)
+
+// SearchResult : user data
+type SearchResult struct {
+	Username     string   // Username
+	Name         string   // Name
+	Surname      string   // Surname
+	Mail         string   // E-mail address
+	SSHPublicKey []string // SSH Public Key
+	IsAdmin      bool     // if user is administrator
+	IsRestricted bool     // if user is restricted
+}
+
+func (ls *Source) sanitizedUserQuery(username string) (string, bool) {
+	// See http://tools.ietf.org/search/rfc4515
+	badCharacters := "\x00()*\\"
+	if strings.ContainsAny(username, badCharacters) {
+		log.Debug("'%s' contains invalid query characters. Aborting.", username)
+		return "", false
+	}
+
+	return fmt.Sprintf(ls.Filter, username), true
+}
+
+func (ls *Source) sanitizedUserDN(username string) (string, bool) {
+	// See http://tools.ietf.org/search/rfc4514: "special characters"
+	badCharacters := "\x00()*\\,='\"#+;<>"
+	if strings.ContainsAny(username, badCharacters) {
+		log.Debug("'%s' contains invalid DN characters. Aborting.", username)
+		return "", false
+	}
+
+	return fmt.Sprintf(ls.UserDN, username), true
+}
+
+func (ls *Source) sanitizedGroupFilter(group string) (string, bool) {
+	// See http://tools.ietf.org/search/rfc4515
+	badCharacters := "\x00*\\"
+	if strings.ContainsAny(group, badCharacters) {
+		log.Trace("Group filter invalid query characters: %s", group)
+		return "", false
+	}
+
+	return group, true
+}
+
+func (ls *Source) sanitizedGroupDN(groupDn string) (string, bool) {
+	// See http://tools.ietf.org/search/rfc4514: "special characters"
+	badCharacters := "\x00()*\\'\"#+;<>"
+	if strings.ContainsAny(groupDn, badCharacters) || strings.HasPrefix(groupDn, " ") || strings.HasSuffix(groupDn, " ") {
+		log.Trace("Group DN contains invalid query characters: %s", groupDn)
+		return "", false
+	}
+
+	return groupDn, true
+}
+
+func (ls *Source) findUserDN(l *ldap.Conn, name string) (string, bool) {
+	log.Trace("Search for LDAP user: %s", name)
+
+	// A search for the user.
+	userFilter, ok := ls.sanitizedUserQuery(name)
+	if !ok {
+		return "", false
+	}
+
+	log.Trace("Searching for DN using filter %s and base %s", userFilter, ls.UserBase)
+	search := ldap.NewSearchRequest(
+		ls.UserBase, ldap.ScopeWholeSubtree, ldap.NeverDerefAliases, 0, 0,
+		false, userFilter, []string{}, nil)
+
+	// Ensure we found a user
+	sr, err := l.Search(search)
+	if err != nil || len(sr.Entries) < 1 {
+		log.Debug("Failed search using filter[%s]: %v", userFilter, err)
+		return "", false
+	} else if len(sr.Entries) > 1 {
+		log.Debug("Filter '%s' returned more than one user.", userFilter)
+		return "", false
+	}
+
+	userDN := sr.Entries[0].DN
+	if userDN == "" {
+		log.Error("LDAP search was successful, but found no DN!")
+		return "", false
+	}
+
+	return userDN, true
+}
+
+func dial(ls *Source) (*ldap.Conn, error) {
+	log.Trace("Dialing LDAP with security protocol (%v) without verifying: %v", ls.SecurityProtocol, ls.SkipVerify)
+
+	tlsCfg := &tls.Config{
+		ServerName:         ls.Host,
+		InsecureSkipVerify: ls.SkipVerify,
+	}
+	if ls.SecurityProtocol == SecurityProtocolLDAPS {
+		return ldap.DialTLS("tcp", fmt.Sprintf("%s:%d", ls.Host, ls.Port), tlsCfg)
+	}
+
+	conn, err := ldap.Dial("tcp", fmt.Sprintf("%s:%d", ls.Host, ls.Port))
+	if err != nil {
+		return nil, fmt.Errorf("Dial: %v", err)
+	}
+
+	if ls.SecurityProtocol == SecurityProtocolStartTLS {
+		if err = conn.StartTLS(tlsCfg); err != nil {
+			conn.Close()
+			return nil, fmt.Errorf("StartTLS: %v", err)
+		}
+	}
+
+	return conn, nil
+}
+
+func bindUser(l *ldap.Conn, userDN, passwd string) error {
+	log.Trace("Binding with userDN: %s", userDN)
+	err := l.Bind(userDN, passwd)
+	if err != nil {
+		log.Debug("LDAP auth. failed for %s, reason: %v", userDN, err)
+		return err
+	}
+	log.Trace("Bound successfully with userDN: %s", userDN)
+	return err
+}
+
+func checkAdmin(l *ldap.Conn, ls *Source, userDN string) bool {
+	if len(ls.AdminFilter) == 0 {
+		return false
+	}
+	log.Trace("Checking admin with filter %s and base %s", ls.AdminFilter, userDN)
+	search := ldap.NewSearchRequest(
+		userDN, ldap.ScopeWholeSubtree, ldap.NeverDerefAliases, 0, 0, false, ls.AdminFilter,
+		[]string{ls.AttributeName},
+		nil)
+
+	sr, err := l.Search(search)
+
+	if err != nil {
+		log.Error("LDAP Admin Search failed unexpectedly! (%v)", err)
+	} else if len(sr.Entries) < 1 {
+		log.Trace("LDAP Admin Search found no matching entries.")
+	} else {
+		return true
+	}
+	return false
+}
+
+func checkRestricted(l *ldap.Conn, ls *Source, userDN string) bool {
+	if len(ls.RestrictedFilter) == 0 {
+		return false
+	}
+	if ls.RestrictedFilter == "*" {
+		return true
+	}
+	log.Trace("Checking restricted with filter %s and base %s", ls.RestrictedFilter, userDN)
+	search := ldap.NewSearchRequest(
+		userDN, ldap.ScopeWholeSubtree, ldap.NeverDerefAliases, 0, 0, false, ls.RestrictedFilter,
+		[]string{ls.AttributeName},
+		nil)
+
+	sr, err := l.Search(search)
+
+	if err != nil {
+		log.Error("LDAP Restrictred Search failed unexpectedly! (%v)", err)
+	} else if len(sr.Entries) < 1 {
+		log.Trace("LDAP Restricted Search found no matching entries.")
+	} else {
+		return true
+	}
+	return false
+}
+
+// SearchEntry : search an LDAP source if an entry (name, passwd) is valid and in the specific filter
+func (ls *Source) SearchEntry(name, passwd string, directBind bool) *SearchResult {
+	// See https://tools.ietf.org/search/rfc4513#section-5.1.2
+	if len(passwd) == 0 {
+		log.Debug("Auth. failed for %s, password cannot be empty", name)
+		return nil
+	}
+	l, err := dial(ls)
+	if err != nil {
+		log.Error("LDAP Connect error, %s:%v", ls.Host, err)
+		ls.Enabled = false
+		return nil
+	}
+	defer l.Close()
+
+	var userDN string
+	if directBind {
+		log.Trace("LDAP will bind directly via UserDN template: %s", ls.UserDN)
+
+		var ok bool
+		userDN, ok = ls.sanitizedUserDN(name)
+
+		if !ok {
+			return nil
+		}
+
+		err = bindUser(l, userDN, passwd)
+		if err != nil {
+			return nil
+		}
+
+		if ls.UserBase != "" {
+			// not everyone has a CN compatible with input name so we need to find
+			// the real userDN in that case
+
+			userDN, ok = ls.findUserDN(l, name)
+			if !ok {
+				return nil
+			}
+		}
+	} else {
+		log.Trace("LDAP will use BindDN.")
+
+		var found bool
+
+		if ls.BindDN != "" && ls.BindPassword != "" {
+			err := l.Bind(ls.BindDN, ls.BindPassword)
+			if err != nil {
+				log.Debug("Failed to bind as BindDN[%s]: %v", ls.BindDN, err)
+				return nil
+			}
+			log.Trace("Bound as BindDN %s", ls.BindDN)
+		} else {
+			log.Trace("Proceeding with anonymous LDAP search.")
+		}
+
+		userDN, found = ls.findUserDN(l, name)
+		if !found {
+			return nil
+		}
+	}
+
+	if !ls.AttributesInBind {
+		// binds user (checking password) before looking-up attributes in user context
+		err = bindUser(l, userDN, passwd)
+		if err != nil {
+			return nil
+		}
+	}
+
+	userFilter, ok := ls.sanitizedUserQuery(name)
+	if !ok {
+		return nil
+	}
+
+	var isAttributeSSHPublicKeySet = len(strings.TrimSpace(ls.AttributeSSHPublicKey)) > 0
+
+	attribs := []string{ls.AttributeUsername, ls.AttributeName, ls.AttributeSurname, ls.AttributeMail}
+	if len(strings.TrimSpace(ls.UserUID)) > 0 {
+		attribs = append(attribs, ls.UserUID)
+	}
+	if isAttributeSSHPublicKeySet {
+		attribs = append(attribs, ls.AttributeSSHPublicKey)
+	}
+
+	log.Trace("Fetching attributes '%v', '%v', '%v', '%v', '%v', '%v' with filter '%s' and base '%s'", ls.AttributeUsername, ls.AttributeName, ls.AttributeSurname, ls.AttributeMail, ls.AttributeSSHPublicKey, ls.UserUID, userFilter, userDN)
+	search := ldap.NewSearchRequest(
+		userDN, ldap.ScopeWholeSubtree, ldap.NeverDerefAliases, 0, 0, false, userFilter,
+		attribs, nil)
+
+	sr, err := l.Search(search)
+	if err != nil {
+		log.Error("LDAP Search failed unexpectedly! (%v)", err)
+		return nil
+	} else if len(sr.Entries) < 1 {
+		if directBind {
+			log.Trace("User filter inhibited user login.")
+		} else {
+			log.Trace("LDAP Search found no matching entries.")
+		}
+
+		return nil
+	}
+
+	var sshPublicKey []string
+
+	username := sr.Entries[0].GetAttributeValue(ls.AttributeUsername)
+	firstname := sr.Entries[0].GetAttributeValue(ls.AttributeName)
+	surname := sr.Entries[0].GetAttributeValue(ls.AttributeSurname)
+	mail := sr.Entries[0].GetAttributeValue(ls.AttributeMail)
+	uid := sr.Entries[0].GetAttributeValue(ls.UserUID)
+
+	// Check group membership
+	if ls.GroupsEnabled {
+		groupFilter, ok := ls.sanitizedGroupFilter(ls.GroupFilter)
+		if !ok {
+			return nil
+		}
+		groupDN, ok := ls.sanitizedGroupDN(ls.GroupDN)
+		if !ok {
+			return nil
+		}
+
+		log.Trace("Fetching groups '%v' with filter '%s' and base '%s'", ls.GroupMemberUID, groupFilter, groupDN)
+		groupSearch := ldap.NewSearchRequest(
+			groupDN, ldap.ScopeWholeSubtree, ldap.NeverDerefAliases, 0, 0, false, groupFilter,
+			[]string{ls.GroupMemberUID},
+			nil)
+
+		srg, err := l.Search(groupSearch)
+		if err != nil {
+			log.Error("LDAP group search failed: %v", err)
+			return nil
+		} else if len(srg.Entries) < 1 {
+			log.Error("LDAP group search failed: 0 entries")
+			return nil
+		}
+
+		isMember := false
+	Entries:
+		for _, group := range srg.Entries {
+			for _, member := range group.GetAttributeValues(ls.GroupMemberUID) {
+				if (ls.UserUID == "dn" && member == sr.Entries[0].DN) || member == uid {
+					isMember = true
+					break Entries
+				}
+			}
+		}
+
+		if !isMember {
+			log.Error("LDAP group membership test failed")
+			return nil
+		}
+	}
+
+	if isAttributeSSHPublicKeySet {
+		sshPublicKey = sr.Entries[0].GetAttributeValues(ls.AttributeSSHPublicKey)
+	}
+	isAdmin := checkAdmin(l, ls, userDN)
+	var isRestricted bool
+	if !isAdmin {
+		isRestricted = checkRestricted(l, ls, userDN)
+	}
+
+	if !directBind && ls.AttributesInBind {
+		// binds user (checking password) after looking-up attributes in BindDN context
+		err = bindUser(l, userDN, passwd)
+		if err != nil {
+			return nil
+		}
+	}
+
+	return &SearchResult{
+		Username:     username,
+		Name:         firstname,
+		Surname:      surname,
+		Mail:         mail,
+		SSHPublicKey: sshPublicKey,
+		IsAdmin:      isAdmin,
+		IsRestricted: isRestricted,
+	}
+}
+
+// UsePagedSearch returns if need to use paged search
+func (ls *Source) UsePagedSearch() bool {
+	return ls.SearchPageSize > 0
+}
+
+// SearchEntries : search an LDAP source for all users matching userFilter
+func (ls *Source) SearchEntries() ([]*SearchResult, error) {
+	l, err := dial(ls)
+	if err != nil {
+		log.Error("LDAP Connect error, %s:%v", ls.Host, err)
+		ls.Enabled = false
+		return nil, err
+	}
+	defer l.Close()
+
+	if ls.BindDN != "" && ls.BindPassword != "" {
+		err := l.Bind(ls.BindDN, ls.BindPassword)
+		if err != nil {
+			log.Debug("Failed to bind as BindDN[%s]: %v", ls.BindDN, err)
+			return nil, err
+		}
+		log.Trace("Bound as BindDN %s", ls.BindDN)
+	} else {
+		log.Trace("Proceeding with anonymous LDAP search.")
+	}
+
+	userFilter := fmt.Sprintf(ls.Filter, "*")
+
+	var isAttributeSSHPublicKeySet = len(strings.TrimSpace(ls.AttributeSSHPublicKey)) > 0
+
+	attribs := []string{ls.AttributeUsername, ls.AttributeName, ls.AttributeSurname, ls.AttributeMail}
+	if isAttributeSSHPublicKeySet {
+		attribs = append(attribs, ls.AttributeSSHPublicKey)
+	}
+
+	log.Trace("Fetching attributes '%v', '%v', '%v', '%v', '%v' with filter %s and base %s", ls.AttributeUsername, ls.AttributeName, ls.AttributeSurname, ls.AttributeMail, ls.AttributeSSHPublicKey, userFilter, ls.UserBase)
+	search := ldap.NewSearchRequest(
+		ls.UserBase, ldap.ScopeWholeSubtree, ldap.NeverDerefAliases, 0, 0, false, userFilter,
+		attribs, nil)
+
+	var sr *ldap.SearchResult
+	if ls.UsePagedSearch() {
+		sr, err = l.SearchWithPaging(search, ls.SearchPageSize)
+	} else {
+		sr, err = l.Search(search)
+	}
+	if err != nil {
+		log.Error("LDAP Search failed unexpectedly! (%v)", err)
+		return nil, err
+	}
+
+	result := make([]*SearchResult, len(sr.Entries))
+
+	for i, v := range sr.Entries {
+		result[i] = &SearchResult{
+			Username: v.GetAttributeValue(ls.AttributeUsername),
+			Name:     v.GetAttributeValue(ls.AttributeName),
+			Surname:  v.GetAttributeValue(ls.AttributeSurname),
+			Mail:     v.GetAttributeValue(ls.AttributeMail),
+			IsAdmin:  checkAdmin(l, ls, v.DN),
+		}
+		if !result[i].IsAdmin {
+			result[i].IsRestricted = checkRestricted(l, ls, v.DN)
+		}
+		if isAttributeSSHPublicKeySet {
+			result[i].SSHPublicKey = v.GetAttributeValues(ls.AttributeSSHPublicKey)
+		}
+	}
+
+	return result, nil
+}
diff --git a/services/auth/source/ldap/source_sync.go b/services/auth/source/ldap/source_sync.go
new file mode 100644
index 0000000000..7e4088e571
--- /dev/null
+++ b/services/auth/source/ldap/source_sync.go
@@ -0,0 +1,184 @@
+// Copyright 2021 The Gitea Authors. All rights reserved.
+// Use of this source code is governed by a MIT-style
+// license that can be found in the LICENSE file.
+
+package ldap
+
+import (
+	"context"
+	"fmt"
+	"strings"
+
+	"code.gitea.io/gitea/models"
+	"code.gitea.io/gitea/modules/log"
+)
+
+// Sync causes this ldap source to synchronize its users with the db
+func (source *Source) Sync(ctx context.Context, updateExisting bool) error {
+	log.Trace("Doing: SyncExternalUsers[%s]", source.loginSource.Name)
+
+	var existingUsers []int64
+	isAttributeSSHPublicKeySet := len(strings.TrimSpace(source.AttributeSSHPublicKey)) > 0
+	var sshKeysNeedUpdate bool
+
+	// Find all users with this login type - FIXME: Should this be an iterator?
+	users, err := models.GetUsersBySource(source.loginSource)
+	if err != nil {
+		log.Error("SyncExternalUsers: %v", err)
+		return err
+	}
+	select {
+	case <-ctx.Done():
+		log.Warn("SyncExternalUsers: Cancelled before update of %s", source.loginSource.Name)
+		return models.ErrCancelledf("Before update of %s", source.loginSource.Name)
+	default:
+	}
+
+	sr, err := source.SearchEntries()
+	if err != nil {
+		log.Error("SyncExternalUsers LDAP source failure [%s], skipped", source.loginSource.Name)
+		return nil
+	}
+
+	if len(sr) == 0 {
+		if !source.AllowDeactivateAll {
+			log.Error("LDAP search found no entries but did not report an error. Refusing to deactivate all users")
+			return nil
+		}
+		log.Warn("LDAP search found no entries but did not report an error. All users will be deactivated as per settings")
+	}
+
+	for _, su := range sr {
+		select {
+		case <-ctx.Done():
+			log.Warn("SyncExternalUsers: Cancelled at update of %s before completed update of users", source.loginSource.Name)
+			// Rewrite authorized_keys file if LDAP Public SSH Key attribute is set and any key was added or removed
+			if sshKeysNeedUpdate {
+				err = models.RewriteAllPublicKeys()
+				if err != nil {
+					log.Error("RewriteAllPublicKeys: %v", err)
+				}
+			}
+			return models.ErrCancelledf("During update of %s before completed update of users", source.loginSource.Name)
+		default:
+		}
+		if len(su.Username) == 0 {
+			continue
+		}
+
+		if len(su.Mail) == 0 {
+			su.Mail = fmt.Sprintf("%s@localhost", su.Username)
+		}
+
+		var usr *models.User
+		// Search for existing user
+		for _, du := range users {
+			if du.LowerName == strings.ToLower(su.Username) {
+				usr = du
+				break
+			}
+		}
+
+		fullName := composeFullName(su.Name, su.Surname, su.Username)
+		// If no existing user found, create one
+		if usr == nil {
+			log.Trace("SyncExternalUsers[%s]: Creating user %s", source.loginSource.Name, su.Username)
+
+			usr = &models.User{
+				LowerName:    strings.ToLower(su.Username),
+				Name:         su.Username,
+				FullName:     fullName,
+				LoginType:    source.loginSource.Type,
+				LoginSource:  source.loginSource.ID,
+				LoginName:    su.Username,
+				Email:        su.Mail,
+				IsAdmin:      su.IsAdmin,
+				IsRestricted: su.IsRestricted,
+				IsActive:     true,
+			}
+
+			err = models.CreateUser(usr)
+
+			if err != nil {
+				log.Error("SyncExternalUsers[%s]: Error creating user %s: %v", source.loginSource.Name, su.Username, err)
+			} else if isAttributeSSHPublicKeySet {
+				log.Trace("SyncExternalUsers[%s]: Adding LDAP Public SSH Keys for user %s", source.loginSource.Name, usr.Name)
+				if models.AddPublicKeysBySource(usr, source.loginSource, su.SSHPublicKey) {
+					sshKeysNeedUpdate = true
+				}
+			}
+		} else if updateExisting {
+			existingUsers = append(existingUsers, usr.ID)
+
+			// Synchronize SSH Public Key if that attribute is set
+			if isAttributeSSHPublicKeySet && models.SynchronizePublicKeys(usr, source.loginSource, su.SSHPublicKey) {
+				sshKeysNeedUpdate = true
+			}
+
+			// Check if user data has changed
+			if (len(source.AdminFilter) > 0 && usr.IsAdmin != su.IsAdmin) ||
+				(len(source.RestrictedFilter) > 0 && usr.IsRestricted != su.IsRestricted) ||
+				!strings.EqualFold(usr.Email, su.Mail) ||
+				usr.FullName != fullName ||
+				!usr.IsActive {
+
+				log.Trace("SyncExternalUsers[%s]: Updating user %s", source.loginSource.Name, usr.Name)
+
+				usr.FullName = fullName
+				usr.Email = su.Mail
+				// Change existing admin flag only if AdminFilter option is set
+				if len(source.AdminFilter) > 0 {
+					usr.IsAdmin = su.IsAdmin
+				}
+				// Change existing restricted flag only if RestrictedFilter option is set
+				if !usr.IsAdmin && len(source.RestrictedFilter) > 0 {
+					usr.IsRestricted = su.IsRestricted
+				}
+				usr.IsActive = true
+
+				err = models.UpdateUserCols(usr, "full_name", "email", "is_admin", "is_restricted", "is_active")
+				if err != nil {
+					log.Error("SyncExternalUsers[%s]: Error updating user %s: %v", source.loginSource.Name, usr.Name, err)
+				}
+			}
+		}
+	}
+
+	// Rewrite authorized_keys file if LDAP Public SSH Key attribute is set and any key was added or removed
+	if sshKeysNeedUpdate {
+		err = models.RewriteAllPublicKeys()
+		if err != nil {
+			log.Error("RewriteAllPublicKeys: %v", err)
+		}
+	}
+
+	select {
+	case <-ctx.Done():
+		log.Warn("SyncExternalUsers: Cancelled during update of %s before delete users", source.loginSource.Name)
+		return models.ErrCancelledf("During update of %s before delete users", source.loginSource.Name)
+	default:
+	}
+
+	// Deactivate users not present in LDAP
+	if updateExisting {
+		for _, usr := range users {
+			found := false
+			for _, uid := range existingUsers {
+				if usr.ID == uid {
+					found = true
+					break
+				}
+			}
+			if !found {
+				log.Trace("SyncExternalUsers[%s]: Deactivating user %s", source.loginSource.Name, usr.Name)
+
+				usr.IsActive = false
+				err = models.UpdateUserCols(usr, "is_active")
+				if err != nil {
+					log.Error("SyncExternalUsers[%s]: Error deactivating user %s: %v", source.loginSource.Name, usr.Name, err)
+				}
+			}
+		}
+	}
+	return nil
+}
diff --git a/services/auth/source/ldap/util.go b/services/auth/source/ldap/util.go
new file mode 100644
index 0000000000..f27de37c87
--- /dev/null
+++ b/services/auth/source/ldap/util.go
@@ -0,0 +1,19 @@
+// Copyright 2021 The Gitea Authors. All rights reserved.
+// Use of this source code is governed by a MIT-style
+// license that can be found in the LICENSE file.
+
+package ldap
+
+// composeFullName composes a firstname surname or username
+func composeFullName(firstname, surname, username string) string {
+	switch {
+	case len(firstname) == 0 && len(surname) == 0:
+		return username
+	case len(firstname) == 0:
+		return surname
+	case len(surname) == 0:
+		return firstname
+	default:
+		return firstname + " " + surname
+	}
+}