Masaya Suzuki [Sun, 11 Nov 2018 20:31:22 +0000 (12:31 -0800)]
Revert C Git 1.7.5 bug workaround
This reverts the workaround introduced by 1c6c73c5a9b8dd700be45d658f165a464265dba7, which is a patch for dealing
with a buggy C Git client v1.7.5 in 2012. We'll stop supporting very old
C Git clients.
Thomas Wolf [Sat, 17 Nov 2018 17:44:27 +0000 (18:44 +0100)]
Apache MINA sshd client: don't leak HostConfigEntry
ProxyDataFactory had a parameter of type HostConfigEntry, but actually
it wasn't used anywhere. Remove it -- it was the last leaked type from
Apache MINA sshd.
Also use the logger provided by upstream SshClient instead of creating
a new Logger.
Bug: 520927
Change-Id: Iaa78bbb998a5e574fa091664b75c48a3b9cfb897 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Fri, 16 Nov 2018 16:00:25 +0000 (17:00 +0100)]
Apache MINA sshd client: test & fix password authentication
Add tests for password and keyboard-interactive authentication.
Implement password authentication; the default provided by sshd
is non-interactive, which is not useful for JGit.
Make sure the CredentialsProvider gets reset on successive password
retrieval attempts. Otherwise it might always return the same non-
accepted password from a secure storage. (That one was discovered
by actually trying this via EGit; the JGit tests don't catch this.)
Change the default order of authentication mechanisms to prefer
password over keyboard-interactive. This is a mitigation for upstream
bug SSHD-866.[1]
Thomas Wolf [Thu, 8 Nov 2018 15:58:52 +0000 (16:58 +0100)]
Move SshTestGitServer to new bundle org.eclipse.jgit.junit.ssh
Create the bundle and move the SshTestGitServer there. Verified that
the Eclipse build still works and ran JSchSshTest and ApacheSshTest as
junit tests inside Eclipse.
Update maven build and features to account for that. Verified by
running full maven build including packaging.
Update bazel build files to account for that. Verified by a
clean-slate bazel build :all, followed by running the JSchSshTest
and the ApacheSshTest via bazel.
Change-Id: Ia084942f4425b454529de148e00417e7da786a90 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Sun, 21 Oct 2018 17:44:34 +0000 (19:44 +0200)]
Apache MINA sshd client: proxy support
This is not about the ssh config ProxyCommand but about programmatic
support for HTTP and SOCKS5 proxies. Eclipse allows the user to
specify such proxies, and JSch at least contains code to connect
through proxies. So our Apache MINA sshd client also should be able
to do this.
Add interfaces and provide two implementations for HTTP and SOCKS5
proxies. Adapt the core code to be able to deal with proxy connections
at all. The built-in client-side support for this in sshd 2.0.0 is
woefully inadequate.
Tested manually by running proxies and then fetching various real-
world repositories via these proxies from different servers. Proxies
tested: ssh -D (SOCKS, anonymous), tinyproxy (HTTP, anonymous), and
3proxy (SOCKS & HTTP, username-password authentication). The GSS-API
authentication is untested since I have no Kerberos setup.
Bug: 520927
Change-Id: I1a5c34687d439b3ef8373c5d58e24004f93e63ae Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Thu, 15 Nov 2018 15:33:04 +0000 (16:33 +0100)]
Apache MINA sshd client: don't leak upstream classes and interfaces
We will get an API evolution problem if we expose as API classes and
interfaces that derive from upstream classes or interfaces. Upstream
interfaces also evolve quite erratically and evolution doesn't seem
to follow semantic versioning.
Introduce a new KeyPasswordProvider interface so that we don't have
to depend on the upstream FilePasswordProvider in our API. (We do
need _some_ abstraction for getting passwords for encrypted keys in
the API; EGit will need to provide its own implementation.)
Move some other upstream dependencies (HostConfigEntry, and various
previously protected methods in SshdSessionFactory) out of the API:
classes moved to internal space, and methods made private.
The only dependencies on upstream interfaces are thus in a few method
parameter types. Those cannot be avoided, but should also not pose
problems.
Bug: 520927
Change-Id: Idc9c6b0f237f29f46343c0fe15179242f2007bec Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Ivan Frade [Fri, 16 Nov 2018 05:27:20 +0000 (21:27 -0800)]
Remove unnecessary modifiers from interfaces
From Oracle's "Defining an interface":
"All abstract, default, and static methods in an interface are
implicitly public, so you can omit the public modifier."
(Without any modifier, the interface methods are also abstract, so we
omit also the "abstract")
"In addition, an interface can contain constant declarations. All
constant values defined in an interface are implicitly public, static,
and final. Once again, you can omit these modifiers."
This makes the code more consistent. Now all interfaces under
org.eclipse.jgit follow the guidelines.
Change-Id: I4fe6deb111899ec1b4318ab5a6050f3851fa1fd3 Signed-off-by: Ivan Frade <ifrade@google.com>
Thomas Wolf [Thu, 18 Oct 2018 21:34:04 +0000 (23:34 +0200)]
Add --ssh option to command-line commands
Enables using the new ssh client based on Apache MINA sshd instead
of the old JSch client. The default is still JSch, so unless the
command is invoked with --ssh apache, there's no change.
I prefer this over some fiddling with the GIT_SSH environment variable
since that variable is handled in the JGit core bundle, which should
remain free of any dependency to org.eclipse.jgit.ssh.apache to avoid
problems in Gerrit or other JGit users that may use a different Apache
MINA sshd version.
Bug: 520927
Change-Id: I8460759c7113ef7887520fb0d297aa312200c69f Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Wed, 17 Oct 2018 18:22:26 +0000 (20:22 +0200)]
Apache MINA sshd: use NumberOfPasswordPrompts for encrypted keys
sshd only asks exactly once for the password. C.f. upstream issue
SSHD-850.[1] So we have to work around this limitation for now.
Once we move to sshd > 2.1.0, this can be simplified somewhat.
Thomas Wolf [Fri, 5 Oct 2018 19:35:16 +0000 (21:35 +0200)]
Apache MINA sshd client: properly handle HostKeyAlgorithms config
By default sshd will use its default built-in list, which matches
the one of openssh (as far as the algorithms exist in sshd at all).
But it doesn't handle HostKeyAlgorithms from the ssh config at all.
Implement this as in openssh, including the '+' and '-' modifiers
and reordering the default if there are known host keys for a
server already.
Add tests for the reordering.
Also use a more robust reader for the known hosts file. The default
aborts on the first error.
Bug: 520927
Change-Id: Ib1684440bfe2e96140536aa1a93c4bd4a0d35916 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Wed, 3 Oct 2018 06:27:40 +0000 (08:27 +0200)]
Apache MINA sshd client: respect NumberOfPasswordPrompts
Set the internal property on the session as defined in the ssh config.
Note that NumberOfPasswordPrompts in openssh applies independently to
both user logins in keyboard-interactive authentication _and_ to
passphrases for identity files (encrypted keys). Apache MINA sshd uses
the setting only for keyboard-interactive authentication, but not for
identity file passphrase prompts. For identity files, it asks exactly
once. This has been reported as issue SSHD-850 upstream.[1]
Thomas Wolf [Tue, 2 Oct 2018 20:39:40 +0000 (22:39 +0200)]
Apache MINA sshd client: add gssapi-with-mic authentication
sshd does support gssapi-with-mic on the server side, but has no
built-in client-side support for this authentication mechanism.
Add our own implementation for it, following RFC 4462.[1] To avoid
needlessly re-trying mechanisms that aren't even configured on the
client, we disable mechanisms that fail on the very first attempt
to use them.
Since we have no real Kerberos5 test setup, this cannot be fully
tested in CI. The disabling of the authentication mechanism and
that it is skipped when not successful _is_ tested.
[1] https://www.ietf.org/rfc/rfc4462.txt
Bug: 520927
Change-Id: I5d0cdb14103588a57c52f927df541b589ab88d88 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Mon, 1 Oct 2018 21:27:10 +0000 (23:27 +0200)]
Add ssh tests for PreferredAuthentications
Tests that it works with unknown authentications in the list, and
fails if there are no common authentications between server and
client. The latter also tests that the ssh config setting is taken
into account at all.
And promptly the JGit sshd client didn't. Add a fix for this. It's
a tiny bit hacky: Apache MINA looks up a custom property set on a
hierarchy of "PropertyResolver"s starting with the session. On the
session itself this property can never be set since it's read
already in the session constructor before anyone had any chance
to set it. The next element in the resolver hierarchy is the sshd
SshClient, and so we set that property there. Since we use one
SshClient and one ClientSession per JGit SshdSession, this is OK.
Bug: 520927
Change-Id: I62446fc1fffde125a8965c030240f0918ae234b7 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Sun, 23 Sep 2018 14:45:45 +0000 (16:45 +0200)]
Apache MINA sshd client
Add a new ssh client implementation based on Apach MINA sshd 2.0.0.
This implementation uses JGit's own config file parser and host entry
resolver. Code inspection of the Apache MINA implementation revealed
a few bugs or idiosyncrasies that immediately would re-introduce bugs
already fixed in the past in JGit.
Apache MINA sshd is not without quirks either, and I had to configure
and override more than I had expected. But at least it was all doable
in clean ways.
Apache MINA boasts support for Bouncy Castle, so in theory this should
open the way to using more ssh key algorithms, such as ed25519.
The implementation is in a separate bundle and is still not used in
the core org.eclipse.jgit bundle. The tests re-use the ssh tests from
the core test bundle.
Bug: 520927
Change-Id: Ib35e73c35799140fe050d1ff4fb18d0d3596580e Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Sun, 23 Sep 2018 13:44:10 +0000 (15:44 +0200)]
Add more ssh tests: pushing, known_host file handling, etc.
Add support for git-receive-pack to the ssh git server and add two
new tests for pushing.
This actually uncovered an undocumented requirement in TransportSftp:
the FTP rename operation assumes POSIX semantics, i.e., that the
target is removed. This works as written only for servers that
support and advertise the "posix-rename@openssh.com" FTP extension.
Our little Apache MINA server does not advertise this extension.
Fix the FtpChannel implementation for Jsch to handle this case in a
meaningful way so that it can pass the new "push over sftp" test.
Add more tests to test the behavior of server host key checking.
Also refactor the tests generally to separate better the test
framework from the actual tests.
Bug: 520927
Change-Id: Ia4bb85e17ddacde7b36ee8c2d5d454bbfa66dfc3 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Mon, 17 Sep 2018 18:36:36 +0000 (20:36 +0200)]
TransportSftp: eliminate dependency on Jsch
Introduce an FtpChannel abstraction, which can be obtained from a
RemoteSession. In JSchSession, wrap a JSch ChannelSftp as such an
FtpChannel. The JSch-specific SftpException is also mapped to a
generic FtpException. Rewrite TransportSftp to use only the new
abstraction layer.
This makes it possible to provide alternate ssh/sftp implementations.
Bug: 520927
Change-Id: I379026f7d4122f34931df909a28e73c02cd8a1da Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Masaya Suzuki [Sun, 11 Nov 2018 20:27:47 +0000 (12:27 -0800)]
Call unlockPack in finally
The lock is obtained in receivePackAndCheckConnectivity. It seems to me
the structure that requres the caller to unlock the lock is wrong, but
at least by calling in finally ensures it is called even if an exception
is thrown.
Jonathan Nieder [Fri, 9 Nov 2018 02:11:57 +0000 (18:11 -0800)]
Simplify RevWalk#iterator by factoring out common code
Factor out a helper that calls next() and tunnels IOException in a
RuntimeException, similar to TunnelException.tunnel(RevWalk::next) in
Guava terms[1].
This should make the code a little more readable. No functional
change intended.
Jonathan Tan [Thu, 8 Nov 2018 22:46:10 +0000 (14:46 -0800)]
DepthGenerator: fix multi-child boundary handling
Suppose that a repository has the following commit graph:
B C
\ /
A
and it was cloned with --shallow-exclude=A. DepthGenerator does not mark
C as shallow, causing an invalid repository to be produced on the
client, because A is not sent. (A similar issue occurs when
--shallow-since is used to exclude A but neither B nor C.)
This happens whenever an excluded commit has more than one child that is
to be sent to the client. Fix DepthGenerator to handle this case
correctly.
While we're editing DepthWalk.Commit, fix the documentation of
DepthWalk.Commit#isBoundary.
Change-Id: I7068abf0fe0c864d1b0e56e1616dad1aa8719411 Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Thomas Wolf [Fri, 21 Sep 2018 20:43:34 +0000 (22:43 +0200)]
Factor out a JSch-independent ssh config parser
Move the bulk of the basic parsing and host entry handling into a
new class OpenSshConfigFile that has no dependencies on any concrete
ssh implementation. Make the existing OpenSshConfig use the new
parser.
Introduce a new class SshConstants collecting all the various ssh-
related string literals. Also use TreeMaps with a case-insensitive
key comparator instead of converting keys to uppercase. Add a test
to verify that keys are matched case-insensitively.
Most of the parsing code was simply moved, except that the new
parser supports looking up entries given host name, port, and user
name, and can thus handle more %-substitutions correctly. This
feature is not yet used and cannot be used with JSch since JSch
only has a ConfigRepository.getConfig(String) interface.
The split is still worth the trouble as it opens the way to using
another ssh client altogether. Apache MINA sshd, for instance,
resolves host entries giving host name, port, and user name.
(Apache MINA has a built-in ssh config handling, but that has
problems, too: its pattern matching is case-insensitive, and its
merging of host entries if several match is not the same as in
OpenSsh. But with this refactoring, it will be possible to plug in
OpenSshConfigFile into an Apache MINA sshd client without dragging
along JSch.)
One test case that doesn't make sense anymore has been removed. It
tested that repeatedly querying for a host entry returned the same
object. That is no longer true since the caching has been moved to
a deeper level.
Bug: 520927
Change-Id: I6381d52b29099595e6eaf8b05c786aeeaefbf9cc Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Fri, 14 Sep 2018 20:40:08 +0000 (22:40 +0200)]
Ssh tests with an Apache MINA sshd test git server
Add a simple ssh git server based on Apache MINA sshd, and use it
in new tests that verify ssh operations and in particular a number
of bugs that had cropped up over time in JSch.
The git server supports fetching only, and sftp access.
The tests are all in an abstract base class; the concrete JschSshTest
class only provides ssh-specific test setup. So the same tests could
be run easily also with some other ssh client.
Bug: 520927
Change-Id: Ide6687b717fb497a29fc83f22b07390a26dfce1d Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Thu, 1 Nov 2018 12:53:25 +0000 (13:53 +0100)]
Fix test setup for Eclipse Photon and greater
When a source folder is marked as a test folder, JDT requires that it
has an output folder different from the one used for regular sources.
Therefore give the test folders in org.eclipse.jgit.test a separate
output folder "bin-tst".
Moreover JDT reports errors if non-test classes have dependencies on
test classes. Therefore remove the "test" annotation from
org.eclipse.jgit.junit.
Change-Id: Ib527439ff5b7d7b570b8a60819ecaa70f59c63a3 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Fri, 2 Nov 2018 17:38:23 +0000 (18:38 +0100)]
Update JGit target platforms to Orbit I20181102163257
Remove the S20181031145145 Orbit definition; that one was for our
purposes broken (mistakes in the org.apache.sshd-core and
net.i2p.crypto.eddsa receipes).
Change-Id: I8246d0837bc80165dd23780236b58fd9fe2fe0bc Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Ivan Frade [Sat, 27 Oct 2018 00:03:40 +0000 (17:03 -0700)]
RepoCommand: Preserve executable bit in <copyfile>
The copyfile entry in the manifest file copies the contents of the file
but doesn't keep the executable flag. This is inconsistent with repo
tool behaviour, plus is natural to expect that the copy of a executable
file is executable.
Transfer the executable bit when copying the file, aligning the
RepoCommand with repo tool and user expectations.
Change-Id: I01b24f482d5939e01d496f032388b3a5c02a912a Signed-off-by: Ivan Frade <ifrade@google.com>
Ivan Frade [Tue, 30 Oct 2018 18:51:49 +0000 (11:51 -0700)]
RepoCommand.RemoteReader: Add method to read contents and mode of file
The RepoCommand.RemoteReader interface doesn't offer access to the mode
of a file. Caller can only default to mark the copied objects as regular
files, losing e.g. the executable bit (if set).
Add a new method readFileWithMode that returns the contents and mode of
the remote file. It supersedes the readFile method, that is marked as
deprecated.
Now callers can set correctly the file mode of the copied file.
Change-Id: I8fce01e4bc5707434c0cbc4aebbae1b6b64756f0 Signed-off-by: Ivan Frade <ifrade@google.com>
Matthias Sohn [Sun, 28 Oct 2018 23:40:00 +0000 (00:40 +0100)]
Silence API errors for new methods added to DepthWalk
237abe6a added method getDeepenNots() with a default implementation and
method getDeepenNotFlag() to the interface DepthWalk. This affects
implementers which is ok in minor release following OSGi semantic
versioning.
Change-Id: I1c872da261fc6825e1e310127761b8b8a6d397d4 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Michael Keppler [Sat, 27 Oct 2018 19:55:39 +0000 (21:55 +0200)]
Fix typos
s/occured/occurred/
s/stablished/established/
Change-Id: Ib052bef60bd18043001a30ed43754ee1a5182016 Signed-off-by: Michael Keppler <Michael.Keppler@gmx.de> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Ivan Frade [Fri, 21 Sep 2018 19:08:33 +0000 (12:08 -0700)]
ObjectIdMatcher: Custom matcher for sets of ObjectIds
Parsed requests represent object ids (SHA1) in ObjectId instances but tests
use strings for those ids because they are easier to define.
Create a custom matcher that hides the conversion from string to
ObjectId. Note that this reverses the existing code conversion (it was
transforming ObjectIds into string).
This produces more readable code, consistent with the other hamcrest
assertions.
Change-Id: I47ba1d25557d791fe74fb93c740ff7de9923cc00 Signed-off-by: Ivan Frade <ifrade@google.com>
Jonathan Tan [Mon, 1 Oct 2018 22:50:47 +0000 (15:50 -0700)]
UploadPack: Implement deepen-not for protocol v2
This allows clients to use the --shallow-exclude parameter (producing a
"deepen-not <ref>" line when communicating with the server) in their fetch
commands when fetching against a JGit server using protocol v2.
Note that the implementation in this commit is somewhat inefficient, as
described in the TODO comment in DepthGenerator.
Change-Id: I9fad3ed9276b624d8f668356ffd99a067dc67ef7 Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Jonathan Tan [Tue, 2 Oct 2018 22:18:43 +0000 (15:18 -0700)]
Throw error when deepen-since excludes all commits
In C Git, when a client fetches with "git fetch --shallow-since=<date>
origin <ref>", and all commits reachable from <ref> are older than
<date>, the server dies with a message "no commits selected for shallow
requests". That is, (1) the --shallow-since filter applies to the commit
pointed to by the ref itself, and (2) there is a check that at least one
commit is not filtered out. (The pack-protocol.txt documentation does
not describe this, but the C implementation does this.)
The implementation in commit 1bb430dc21 ("UploadPack: support
deepen-since in protocol v2", 2018-09-27) does neither (1) nor (2), so
do both of these.
Change-Id: I9946327a71627626ecce34ca2d017d2add8867fc Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Markus Keller [Tue, 16 Oct 2018 14:25:56 +0000 (16:25 +0200)]
Make PrePushHook properly terminate ref lines
All of the input lines passed to pre-push hook scripts must be properly
terminated by '\n', so that normal shell scripts like the git-supplied
pre-push.sample work properly, even when pushing just a single branch.
With the old code, hook scripts that use the following pattern didn't
process the last line, because 'read' has a non-zero exit status when
EOF is encountered:
while read local_ref local_sha remote_ref remote_sha; do ... done
Change-Id: Id899662ed3fedef6c314fc4b2ddf91a6dcb98cbb Signed-off-by: Markus Keller <markus.kell.r@gmail.com>
Ned Twigg [Tue, 21 Aug 2018 21:44:28 +0000 (14:44 -0700)]
CheckoutCommand: force flag now allows overwrite
Before this commit, a force checkout would fail if there
were any conflicting files. After this commit, a force
checkout will overwrite the conflicting files, as expected.
Making this work required fixing a bug in DirCacheCheckout.
Before this commit, when DirCacheCheckout had
failOnConflict=false, it would delete all conflicting files
from the working copy and just leave them missing. After
this commit, DirCacheCheckout overwrites conflicting files
with the merge tree.
This change in DirCacheCheckout causes "reset --hard" and
"revert --abort" to behave as expected (previously they
would simply delete conflicting files, now they will be
overwritten from the merge tree).
Change-Id: If7e328ee792ef6511ab7d9c26d8d77c39210ec9f Signed-off-by: Ned Twigg <ned.twigg@diffplug.com>
Ivan Frade [Wed, 17 Oct 2018 23:52:30 +0000 (16:52 -0700)]
UploadPack: Return correct peer user agent on v2 requests
UploadPack.getPeerUserAgent() doesn't produce the expected results for
protocol v2 requests. In v2, the agent reported in the request (in an
"agent=" line) is not in the clientCapabilities but in a field on its
own. This makes getPeerUserAgent default to the transport user agent.
Making "agent" a shared property between protocol v0/v1 and v2 fixes the
problem, simplifies the function and harmonizes the implementation
between protocol versions.
In a follow up commit the "agent" will be identified on parsing time,
instead of taking it from the client capabilities.
Change-Id: Idf9825ec4e0b81a1458c8e3701f3e28aafd8a32a Signed-off-by: Ivan Frade <ifrade@google.com>
Ivan Frade [Thu, 26 Jul 2018 22:49:09 +0000 (15:49 -0700)]
Accept protocol v2 server options on fetch and ls-refs requests
In protocol v2, a command request can be followed by server options
(lines like "agent=<>" and "server-option=<>"), but current code
doesn't accept those lines.
Advertise the "server-option" capability, parse the lines and add
them to the request objects.
Other code in JGit can see this options and act accordingly via the
protocol v2 hooks.
This should not require any change in the client side.
Change-Id: If3946390f9cc02d29644b6ca52534b6f757bda9f Signed-off-by: Ivan Frade <ifrade@google.com>
projects / jgit.git / log