This is not about the ssh config ProxyCommand but about programmatic
support for HTTP and SOCKS5 proxies. Eclipse allows the user to
specify such proxies, and JSch at least contains code to connect
through proxies. So our Apache MINA sshd client also should be able
to do this.
Add interfaces and provide two implementations for HTTP and SOCKS5
proxies. Adapt the core code to be able to deal with proxy connections
at all. The built-in client-side support for this in sshd 2.0.0 is
woefully inadequate.
Tested manually by running proxies and then fetching various real-
world repositories via these proxies from different servers. Proxies
tested: ssh -D (SOCKS, anonymous), tinyproxy (HTTP, anonymous), and
3proxy (SOCKS & HTTP, username-password authentication). The GSS-API
authentication is untested since I have no Kerberos setup.
Bug: 520927
Change-Id: I1a5c34687d439b3ef8373c5d58e24004f93e63ae
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Apache MINA sshd client: don't leak upstream classes and interfaces
We will get an API evolution problem if we expose as API classes and
interfaces that derive from upstream classes or interfaces. Upstream
interfaces also evolve quite erratically and evolution doesn't seem
to follow semantic versioning.
Introduce a new KeyPasswordProvider interface so that we don't have
to depend on the upstream FilePasswordProvider in our API. (We do
need _some_ abstraction for getting passwords for encrypted keys in
the API; EGit will need to provide its own implementation.)
Move some other upstream dependencies (HostConfigEntry, and various
previously protected methods in SshdSessionFactory) out of the API:
classes moved to internal space, and methods made private.
The only dependencies on upstream interfaces are thus in a few method
parameter types. Those cannot be avoided, but should also not pose
problems.
Bug: 520927
Change-Id: Idc9c6b0f237f29f46343c0fe15179242f2007bec
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
From Oracle's "Defining an interface":
"All abstract, default, and static methods in an interface are
implicitly public, so you can omit the public modifier."
(Without any modifier, the interface methods are also abstract, so we
omit also the "abstract")
"In addition, an interface can contain constant declarations. All
constant values defined in an interface are implicitly public, static,
and final. Once again, you can omit these modifiers."
This makes the code more consistent. Now all interfaces under
org.eclipse.jgit follow the guidelines.
Change-Id: I4fe6deb111899ec1b4318ab5a6050f3851fa1fd3
Signed-off-by: Ivan Frade <ifrade@google.com>
Enables using the new ssh client based on Apache MINA sshd instead
of the old JSch client. The default is still JSch, so unless the
command is invoked with --ssh apache, there's no change.
I prefer this over some fiddling with the GIT_SSH environment variable
since that variable is handled in the JGit core bundle, which should
remain free of any dependency to org.eclipse.jgit.ssh.apache to avoid
problems in Gerrit or other JGit users that may use a different Apache
MINA sshd version.
Bug: 520927
Change-Id: I8460759c7113ef7887520fb0d297aa312200c69f
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Apache MINA sshd: use NumberOfPasswordPrompts for encrypted keys
sshd only asks exactly once for the password. C.f. upstream issue
SSHD-850.[1] So we have to work around this limitation for now.
Once we move to sshd > 2.1.0, this can be simplified somewhat.
[1] https://issues.apache.org/jira/browse/SSHD-850
Bug: 520927
Change-Id: Id65650228486c5ed30affa9c62eac982e01ae207
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Apache MINA sshd client: properly handle HostKeyAlgorithms config
By default sshd will use its default built-in list, which matches
the one of openssh (as far as the algorithms exist in sshd at all).
But it doesn't handle HostKeyAlgorithms from the ssh config at all.
Implement this as in openssh, including the '+' and '-' modifiers
and reordering the default if there are known host keys for a
server already.
Add tests for the reordering.
Also use a more robust reader for the known hosts file. The default
aborts on the first error.
Bug: 520927
Change-Id: Ib1684440bfe2e96140536aa1a93c4bd4a0d35916
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Apache MINA sshd client: respect NumberOfPasswordPrompts
Set the internal property on the session as defined in the ssh config.
Note that NumberOfPasswordPrompts in openssh applies independently to
both user logins in keyboard-interactive authentication _and_ to
passphrases for identity files (encrypted keys). Apache MINA sshd uses
the setting only for keyboard-interactive authentication, but not for
identity file passphrase prompts. For identity files, it asks exactly
once. This has been reported as issue SSHD-850 upstream.[1]
[1] https://issues.apache.org/jira/browse/SSHD-850
Bug: 520927
Change-Id: I390ffe9e1c52b96d3e8e28fd8edbdc73dde9edb4
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Apache MINA sshd client: add gssapi-with-mic authentication
sshd does support gssapi-with-mic on the server side, but has no
built-in client-side support for this authentication mechanism.
Add our own implementation for it, following RFC 4462.[1] To avoid
needlessly re-trying mechanisms that aren't even configured on the
client, we disable mechanisms that fail on the very first attempt
to use them.
Since we have no real Kerberos5 test setup, this cannot be fully
tested in CI. The disabling of the authentication mechanism and
that it is skipped when not successful _is_ tested.
[1] https://www.ietf.org/rfc/rfc4462.txt
Bug: 520927
Change-Id: I5d0cdb14103588a57c52f927df541b589ab88d88
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Tests that it works with unknown authentications in the list, and
fails if there are no common authentications between server and
client. The latter also tests that the ssh config setting is taken
into account at all.
And promptly the JGit sshd client didn't. Add a fix for this. It's
a tiny bit hacky: Apache MINA looks up a custom property set on a
hierarchy of "PropertyResolver"s starting with the session. On the
session itself this property can never be set since it's read
already in the session constructor before anyone had any chance
to set it. The next element in the resolver hierarchy is the sshd
SshClient, and so we set that property there. Since we use one
SshClient and one ClientSession per JGit SshdSession, this is OK.
Bug: 520927
Change-Id: I62446fc1fffde125a8965c030240f0918ae234b7
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Add a new ssh client implementation based on Apach MINA sshd 2.0.0.
This implementation uses JGit's own config file parser and host entry
resolver. Code inspection of the Apache MINA implementation revealed
a few bugs or idiosyncrasies that immediately would re-introduce bugs
already fixed in the past in JGit.
Apache MINA sshd is not without quirks either, and I had to configure
and override more than I had expected. But at least it was all doable
in clean ways.
Apache MINA boasts support for Bouncy Castle, so in theory this should
open the way to using more ssh key algorithms, such as ed25519.
The implementation is in a separate bundle and is still not used in
the core org.eclipse.jgit bundle. The tests re-use the ssh tests from
the core test bundle.
Bug: 520927
Change-Id: Ib35e73c35799140fe050d1ff4fb18d0d3596580e
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Add more ssh tests: pushing, known_host file handling, etc.
Add support for git-receive-pack to the ssh git server and add two
new tests for pushing.
This actually uncovered an undocumented requirement in TransportSftp:
the FTP rename operation assumes POSIX semantics, i.e., that the
target is removed. This works as written only for servers that
support and advertise the "posix-rename@openssh.com" FTP extension.
Our little Apache MINA server does not advertise this extension.
Fix the FtpChannel implementation for Jsch to handle this case in a
meaningful way so that it can pass the new "push over sftp" test.
Add more tests to test the behavior of server host key checking.
Also refactor the tests generally to separate better the test
framework from the actual tests.
Bug: 520927
Change-Id: Ia4bb85e17ddacde7b36ee8c2d5d454bbfa66dfc3
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Introduce an FtpChannel abstraction, which can be obtained from a
RemoteSession. In JSchSession, wrap a JSch ChannelSftp as such an
FtpChannel. The JSch-specific SftpException is also mapped to a
generic FtpException. Rewrite TransportSftp to use only the new
abstraction layer.
This makes it possible to provide alternate ssh/sftp implementations.
Bug: 520927
Change-Id: I379026f7d4122f34931df909a28e73c02cd8a1da
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
The lock is obtained in receivePackAndCheckConnectivity. It seems to me
the structure that requres the caller to unlock the lock is wrong, but
at least by calling in finally ensures it is called even if an exception
is thrown.
Change-Id: I123841b017baf5acffe0064d1004ef11a0a5e6c2
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
Do not add an artificial line break to the message, since it may become
much wider due to the embedded exception messages anyway.
The layout shall be controlled by the egit supplied message dialog using
layout constraints.
Bug: 540537
Change-Id: I4257b52e5e59689dfcbab47bd7c075b3fd031837
Signed-off-by: Michael Keppler <Michael.Keppler@gmx.de>
Simplify RevWalk#iterator by factoring out common code
Factor out a helper that calls next() and tunnels IOException in a
RuntimeException, similar to TunnelException.tunnel(RevWalk::next) in
Guava terms[1].
This should make the code a little more readable. No functional
change intended.
[1] https://github.com/google/guava/issues/2828#issuecomment-304187823
Change-Id: I97c062d03a17663d5c40895fd3d2c6a7306d4f39
Signed-off-by: Jonathan Nieder <jrn@google.com>
MissingObjectException and IncorrectObjectTypeException are subclasses
of IOException.
Change-Id: Ib4e1f37ce1b0b08e69ba3375bbdb6ee82ee4f036
Signed-off-by: Jonathan Nieder <jrn@google.com>
Suppose that a repository has the following commit graph:
B C
\ /
A
and it was cloned with --shallow-exclude=A. DepthGenerator does not mark
C as shallow, causing an invalid repository to be produced on the
client, because A is not sent. (A similar issue occurs when
--shallow-since is used to exclude A but neither B nor C.)
This happens whenever an excluded commit has more than one child that is
to be sent to the client. Fix DepthGenerator to handle this case
correctly.
While we're editing DepthWalk.Commit, fix the documentation of
DepthWalk.Commit#isBoundary.
Change-Id: I7068abf0fe0c864d1b0e56e1616dad1aa8719411
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Bazel: Consume versions directly from lib:versions.bzl in skylib
Avoid loading versions from @bazel_skylib//:lib.bzl, because it is now
deprecated and is going to be removed in future skylib versions:
https://github.com/bazelbuild/bazel-skylib/blob/master/lib.bzl#L17
Change-Id: Ie722351de5254f611cf3489c8689922c8346af7f
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Move the bulk of the basic parsing and host entry handling into a
new class OpenSshConfigFile that has no dependencies on any concrete
ssh implementation. Make the existing OpenSshConfig use the new
parser.
Introduce a new class SshConstants collecting all the various ssh-
related string literals. Also use TreeMaps with a case-insensitive
key comparator instead of converting keys to uppercase. Add a test
to verify that keys are matched case-insensitively.
Most of the parsing code was simply moved, except that the new
parser supports looking up entries given host name, port, and user
name, and can thus handle more %-substitutions correctly. This
feature is not yet used and cannot be used with JSch since JSch
only has a ConfigRepository.getConfig(String) interface.
The split is still worth the trouble as it opens the way to using
another ssh client altogether. Apache MINA sshd, for instance,
resolves host entries giving host name, port, and user name.
(Apache MINA has a built-in ssh config handling, but that has
problems, too: its pattern matching is case-insensitive, and its
merging of host entries if several match is not the same as in
OpenSsh. But with this refactoring, it will be possible to plug in
OpenSshConfigFile into an Apache MINA sshd client without dragging
along JSch.)
One test case that doesn't make sense anymore has been removed. It
tested that repeatedly querying for a host entry returned the same
object. That is no longer true since the caching has been moved to
a deeper level.
Bug: 520927
Change-Id: I6381d52b29099595e6eaf8b05c786aeeaefbf9cc
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Ssh tests with an Apache MINA sshd test git server
Add a simple ssh git server based on Apache MINA sshd, and use it
in new tests that verify ssh operations and in particular a number
of bugs that had cropped up over time in JSch.
The git server supports fetching only, and sftp access.
The tests are all in an abstract base class; the concrete JschSshTest
class only provides ssh-specific test setup. So the same tests could
be run easily also with some other ssh client.
Bug: 520927
Change-Id: Ide6687b717fb497a29fc83f22b07390a26dfce1d
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Fix package exports in org.eclipse.jgit MANIFEST.MF
Do not export o.e.j.internal.transport.parser as public package;
restrict visibility to org.eclipse.jgit.test only.
Add two packages that were not listed at all (o.e.j.internal.revwalk
and o.e.j.internal.submodule) marked as x-internal:=true.
Change-Id: I9188356075515ad354b724102fbd6304b682de6a
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
When a source folder is marked as a test folder, JDT requires that it
has an output folder different from the one used for regular sources.
Therefore give the test folders in org.eclipse.jgit.test a separate
output folder "bin-tst".
Moreover JDT reports errors if non-test classes have dependencies on
test classes. Therefore remove the "test" annotation from
org.eclipse.jgit.junit.
Change-Id: Ib527439ff5b7d7b570b8a60819ecaa70f59c63a3
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Update JGit target platforms to Orbit I20181102163257
Remove the S20181031145145 Orbit definition; that one was for our
purposes broken (mistakes in the org.apache.sshd-core and
net.i2p.crypto.eddsa receipes).
Change-Id: I8246d0837bc80165dd23780236b58fd9fe2fe0bc
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Also replace native git_repository rule with Starlark rule. This is
needed because in recent Bazel versions native git_repository rule
is deprecated.
Change-Id: I2c19fd31693a930d7d35fdbb93bfa4abf21fa2aa
Signed-off-by: David Ostrovsky <david@ostrovsky.org>
The removed method and the new interface method only affect implementors
which is ok in a minor release following OSGi semantic versioning.
Change-Id: Ia5e55bd803965c7590c9278eecc6bdd36241383f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
RepoCommand: Preserve executable bit in <copyfile>
The copyfile entry in the manifest file copies the contents of the file
but doesn't keep the executable flag. This is inconsistent with repo
tool behaviour, plus is natural to expect that the copy of a executable
file is executable.
Transfer the executable bit when copying the file, aligning the
RepoCommand with repo tool and user expectations.
Change-Id: I01b24f482d5939e01d496f032388b3a5c02a912a
Signed-off-by: Ivan Frade <ifrade@google.com>
RepoCommand.RemoteReader: Add method to read contents and mode of file
The RepoCommand.RemoteReader interface doesn't offer access to the mode
of a file. Caller can only default to mark the copied objects as regular
files, losing e.g. the executable bit (if set).
Add a new method readFileWithMode that returns the contents and mode of
the remote file. It supersedes the readFile method, that is marked as
deprecated.
Now callers can set correctly the file mode of the copied file.
Change-Id: I8fce01e4bc5707434c0cbc4aebbae1b6b64756f0
Signed-off-by: Ivan Frade <ifrade@google.com>
RepoCommandTest: Extract method to assert file contents
Many tests verify the contents of files in a try-with-resources
incantation that clutters the code.
Extract that verification to an "assertContents" method, that is easier
to read.
Change-Id: If430eac6f5b9ae352e42b2d43867ceb6cd618fbb
Signed-off-by: Ivan Frade <ifrade@google.com>
Thomas Wolf
Ivan Frade
David Pursehouse
Masaya Suzuki
Michael Keppler
Jonathan Nieder
Jonathan Tan
David Ostrovsky
Matthias Sohn