TransportHttp: support preemptive Basic authentication
If the caller knows already HTTP Basic authentication will be needed
and if it also already has the username and password, preemptive
authentication is a little bit more efficient since it avoids the
initial 401 response.
Add a setPreemptiveBasicAuthentication(username, password) method
to TransportHttp. Client code could call this for instance in a
TransportConfigCallback. The method throws an IllegalStateException
if it is called after an HTTP request has already been made.
Additionally, a URI can include userinfo. Although it is not
recommended to put passwords in URIs, JGit's URIish and also the
Java URL and URI classes still allow it. The underlying HTTP
connection may omit these fields though. If present, take these
fields as additional source for preemptive Basic authentication if
setPreemptiveBasicAuthentication() has not been called.
No preemptive authentication will be done if the connection is
redirected to a different host.
Add tests.
Bug: 541327
Change-Id: Id00b975e56a15b532de96f7bbce48106d992a22b
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
TransportHttp: make the connection factory configurable
Previously, TransportHttp always used the globally set connection
factory. This is problematic if that global factory is changed in
the middle of a fetch or push operation. Initialize the factory to
use in the constructor, then use that factory for all HTTP requests
made through this transport. Provide a setter and a getter for it
so that client code can customize the factory, if needed, in a
TransportConfigCallback.
Once a factory has been used on a TransportHttp instance it cannot
be changed anymore.
Make the global static factory reference volatile.
Change-Id: I7c6ee16680407d3724e901c426db174a3125ba1c
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Make all transports request protocol V2 when fetching. Depending on
the transport, set the GIT_PROTOCOL environment variable (file and
ssh), pass the Git-Protocol header (http), or set the hidden
"\0version=2\0" (git anon). We'll fall back to V0 if the server
doesn't reply with a version 2 answer.
A user can control which protocol the client requests via the git
config protocol.version; if not set, JGit requests protocol V2 for
fetching. Pushing always uses protocol V0 still.
In the API, there is only a new Transport.openFetch() version that
takes a collection of RefSpecs plus additional patterns to construct
the Ref prefixes for the "ls-refs" command in protocol V2. If none
are given, the server will still advertise all refs, even in protocol
V2.
BasePackConnection.readAdvertisedRefs() handles falling back to
protocol V0. It newly returns true if V0 was used and the advertised
refs were read, and false if V2 is used and an explicit "ls-refs" is
needed. (This can't be done transparently inside readAdvertisedRefs()
because a "stateless RPC" transport like TransportHttp may need to
open a new connection for writing.)
BasePackFetchConnection implements the changes needed for the protocol
V2 "fetch" command (stateless protocol, simplified ACK handling,
delimiters, section headers).
In TransportHttp, change readSmartHeaders() to also recognize the
"version 2" packet line as a valid smart server indication.
Adapt tests, and run all the HTTP tests not only with both HTTP
connection factories (JDK and Apache HttpClient) but also with both
protocol V0 and V2. The SSH tests are much slower and much more
focused on the SSH protocol and SSH key handling. Factor out two
very simple cloning and pulling tests and make those run with
protocol V2.
Bug: 553083
Change-Id: I357c7f5daa7efb2872f1c64ee6f6d54229031ae1
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Add constants for parsing git wire protocol version
This would allow other JGit users to access and reuse the constants.
Change-Id: I1608802f45586af5f8582afa592e26679e9cebe3
Signed-off-by: David Ostrovsky <david@ostrovsky.org>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Revert "Client-side protocol V2 support for fetching"
This reverts commit f802f06e7f.
I had misunderstood how protocol V2 works. This implementation only
works if the negotiation during fetch is done in one round.
Fixing this is substantial work in BasePackFetchConnection. Basically
I think I'd have to change back negotiate to the V0 version, and have
a doFetch() that does
if protocol V2
doFetchV2()
else
doFetchV0()
with doFetchV0 the old code, and doFetchV2 completely new.
Plus there would need to be a HTTP test case requiring several
negotiation rounds.
This is a couple of days work at least, and I don't know when I will
have the time to revisit this. So although the rest of the code is
fine I prefer to back this out completely and not leave a only half
working implementation in the code for an indeterminate time.
Bug: 553083
Change-Id: Icbbbb09882b3b83f9897deac4a06d5f8dc99d84e
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Make all transports request protocol V2 when fetching. Depending on
the transport, set the GIT_PROTOCOL environment variable (file and
ssh), pass the Git-Protocol header (http), or set the hidden
"\0version=2\0" (git anon). We'll fall back to V0 if the server
doesn't reply with a version 2 answer.
A user can control which protocol the client requests via the git
config protocol.version; if not set, JGit requests protocol V2 for
fetching. Pushing always uses protocol V0 still.
In the API, there is only a new Transport.openFetch() version that
takes a collection of RefSpecs plus additional patterns to construct
the Ref prefixes for the "ls-refs" command in protocol V2. If none
are given, the server will still advertise all refs, even in protocol
V2.
BasePackConnection.readAdvertisedRefs() handles falling back to
protocol V0. It newly returns true if V0 was used and the advertised
refs were read, and false if V2 is used and an explicit "ls-refs" is
needed. (This can't be done transparently inside readAdvertisedRefs()
because a "stateless RPC" transport like TransportHttp may need to
open a new connection for writing.)
BasePackFetchConnection implements the changes needed for the protocol
V2 "fetch" command (simplified ACK handling, delimiters, section
headers).
In TransportHttp, change readSmartHeaders() to also recognize the
"version 2" packet line as a valid smart server indication.
Adapt tests, and run all the HTTP tests not only with both HTTP
connection factories (JDK and Apache HttpClient) but also with both
protocol V0 and V2. Do the same for the SSH transport tests.
Bug: 553083
Change-Id: Ice9866aa78020f5ca8f397cde84dc224bf5d41b4
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
TransportHttp: abort on time-out or on SocketException
Avoid trying other authentication methods on SocketException or on
InterruptedIOException. SocketException is rather fatal, such as
nothing listening on the peer's port, connection reset, or it could
be a connection time-out.
Time-outs enforced by Timeout{Input,Output}Stream may result in
InterruptedIOException being thrown.
In both cases, it makes no sense to try other authentication methods,
and doing so may wrongly report "authentication not supported" or
"cannot open git-upload-pack" or some such instead of reporting a
time-out.
Bug: 563138
Change-Id: I0191b1e784c2471035e550205abd06ec9934fd00
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
UploadPackServlet: Use uploadWithExceptionPropagation
As UploadPackErrorHandler's Javadoc says, UploadPackServlet should have
called uploadWithExceptionPropagation and let UploadPackErrorHandler to
handle the exception. Fix UploadPackServlet.
Change-Id: I1f9686495fcf3ef28598ccdff3e6f76a16c8bca3
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
Enable and fix "Statement unnecessarily nested within else clause" warnings
Since [1] the gerrit project includes jgit as a submodule, and has this
warning enabled, resulting in 100s of warnings in the console.
Also enable the warning here, and fix them.
At the same time, add missing braces around adjacent and nearby one-line
blocks.
[1] https://gerrit-review.googlesource.com/c/gerrit/+/227897
Change-Id: I81df3fc7ed6eedf6874ce1a3bedfa727a1897e4c
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Other than Maven bazel seems unable to cope with abstract
AllFactoriesHttpTestCase having no test methods, hence tag this class
with @Ignore.
Change-Id: I9dfe43f882ad073b284648e24844b51877d87776
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
This shows the class name of the HTTP factory used in each test run.
Change-Id: I7c1df20f0e138dc7e3120fe87e80d40ab17dd8c8
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Factor out the test parameterization to use both connection factories
into a common super class and use it in more tests.
This made HttpClientTests.testV2HttpSubsequentResponse() fail for
Apache HTTP. The test used the pattern
- create POST connection
- setDoOutput(true)
- connect()
- write output stream
- get & read input stream
This pattern is never used in JGit, which actually calls connect() only
in one case in LFS, and that's on a HEAD request.
The above pattern works on JDK, but fails on Apache HTTP because with
Apache HTTP a connect() actually executes the full request including
writing the entity. To work with Apache HTTP, the pattern would need
to be
- create POST connection
- setDoOutput(true)
- write output stream
- connect()
- get & read input stream
which is fine for both. JDK connects implicitly in getOutputStream()
and treats the later explicit connect() as a no-op, and Apache works
because the entity is written when connect() is called.
Because JDK connects implicitly on getOutputStream(), the following
pattern also works with JDK:
- create POST connection
- setDoOutput(true)
- write output stream
- get & read input stream
Support this with Apache HTTP too: let getInputStream() execute
the request if it wasn't executed already.
Remove explicit connect() calls from test code, since JGit doesn't do
those either.
Change-Id: Ica038c00a7b8edcc01d5660d18e961146305b87f
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Cache user global and system-wide git configurations
So far the git configuration and the system wide git configuration were
always reloaded when jgit accessed these global configuration files to
access global configuration options which are not in the context of a
single git repository. Cache these configurations in SystemReader and
only reload them if their file metadata observed using FileSnapshot
indicates a modification.
Change-Id: I092fe11a5d95f1c5799273cacfc7a415d0b7786c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
TransportHttp: resolve redirect URIs from the "Location" header
The "Location" header in a redirect response may contain a relative
URI. Resolve it against the URI the request was made.
Bug: 550033
Change-Id: I29de07dfbbbc794090821b7c190cb2cf662c5a60
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
As reported by Error Prone:
An inner class should be static unless it references members of its
enclosing class. An inner class that is made non-static unnecessarily
uses more memory and does not make the intent of the class clear.
See https://errorprone.info/bugpattern/ClassCanBeStatic
Change-Id: Ib99d120532630dba63cf400cc1c61c318286fc41
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
(cherry picked from commit ee40efcea4)
PacketLineIn: Add an iterator over strings in the input stream
Allows callers to read all lines in the input stream until the
END marker is reached, without having to explicitly check for
the END marker.
Replace all remaining usage of the END marker with the new method.
Change-Id: I51f419c7f569ab7ed01e1aaaf6b40ed8cdc2116b
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Replace trivial reference comparison of PacketLineIn.{DELIM,END}
Replace reference comparisons of PacketLineIn's DELIM and END strings
with usage of the helper methods isDelimiter() and isEnd().
Change-Id: I52dcfc4ee9097f1bd6970601c716701847d9eebd
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
SmartClientSmartServerTest: Open Repository in try-with-resource
Since 52923e9 ("LocalDiskRepositoryTestCase#createRepository: Default
auto-close to false", Jan 20, 2019) the createBareRepository method
creates repositories that do not get automatically closed in #tearDown.
Convert invocations of createBareRepository to use try-with-resource.
Change-Id: I320030c5d4438713971bee33316bff408bac47fc
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Its implementation contains
} catch (IOException e) {
// Legacy API, assume error means "no"
return false;
}
Better to use ObjectDatabase#has, which throws IOException to report
errors.
Change-Id: I7de02f7ceb8f57b2a8ebdb16d2aa4376775ff933
Signed-off-by: Jonathan Nieder <jrn@google.com>
UploadPack: Test filtering by AdvertiseRefsHook in stateless transports
AdvertiseRefsHook is used to limit the visibility of the refs in Gerrit.
If this hook is not called, then all refs are treated as visible,
causing the server to serve commits reachable from branches the client
should not be able to access, if asked to via a request naming a guessed
object id.
Until 3a529361a7 (Call AdvertiseRefsHook
before validating wants, 2018-12-18), UploadPack would invoke this hook
at ref advertisement time but not during negotiation and when serving a
pack file. Add a test to avoid regressing. Stateful bidirectional
transports were not affected, so the test uses HTTP.
[jn: split out when backporting the fix to stable-4.5. The test passes
as long as v4.9.0.201710071750-r~169 (fetch: Accept any SHA-1 on lhs of
refspec, 2017-06-04) is cherry picked along with it.]
Change-Id: I8c017107336adc7cb4c826985779676bf043e648
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
Signed-off-by: Jonathan Nieder <jrn@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
This reverts the workaround introduced by
1c6c73c5a9b8dd700be45d658f165a464265dba7, which is a patch for dealing
with a buggy C Git client v1.7.5 in 2012. We'll stop supporting very old
C Git clients.
Change-Id: I94999a39101c96f210b5eca3c2f620c15eb1ac1b
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
FileResolverTest#testNotAGitRepository: Fail when exception is not thrown
Error Prone reports:
Not calling fail() when expecting an exception masks bugs
See https://errorprone.info/bugpattern/MissingFail
Change-Id: I9ac844aa6c5a620d9b5d21d2e242347b3788b96a
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
As reported by Error Prone:
An inner class should be static unless it references members of its
enclosing class. An inner class that is made non-static unnecessarily
uses more memory and does not make the intent of the class clear.
See https://errorprone.info/bugpattern/ClassCanBeStatic
Change-Id: Ib99d120532630dba63cf400cc1c61c318286fc41
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
HttpClientConnectionTest: Throw UnsupportedOperationException for unused methods
The HttpResponseMock class is only used in a single test to assert
on the expected headers. All of its overridden methods are empty, and
this results in warnings in Eclipse:
'Empty block should be documented'
Rather than adding "// Not used" comments, change all of the methods'
implementations to throw UnsupportedOperationException. This not only
gets rid of the warnings, but also makes it explicit that we don't
intend any of them to actually be called.
Change-Id: I8fe06b155e0809bb3507e4e28b00fcc4f9333b0b
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
HttpClientTests: Use assertEquals to assert about response code
The current usage of assertThat causes a warning in Eclipse:
"The expression of type int is boxed into Integer".
Replace it with assertEquals which does not cause this warning, and is
consistent with how such assertions are done in other tests.
Change-Id: Id3de3548353bf6be069b6ede89c605d094b6d3f4
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Try to give as much information as possible. The connection's
response message might contain additional hints as to why the
connection could not be established.
Bug: 536541
Change-Id: I7230e4e0be9417be8cedeb8aaab35186fcbf00a5
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Teach UploadPack to support protocol v2 with non-bidirectional pipes,
and add support to the HTTP protocol for v2. This is only activated if
the repository's config has "protocol.version" equal to 2.
Change-Id: I093a14acd2c3850b8b98e14936a716958f35a848
Helped-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Jonathan Nieder <jrn@google.com>
Remove it from
* package private functions.
* try blocks
* for loops
this was done with the following python script:
$ cat f.py
import sys
import re
import os
def replaceFinal(m):
return m.group(1) + "(" + m.group(2).replace('final ', '') + ")"
methodDecl = re.compile(r"^([\t ]*[a-zA-Z_ ]+)\(([^)]*)\)")
def subst(fn):
input = open(fn)
os.rename(fn, fn + "~")
dest = open(fn, 'w')
for l in input:
l = methodDecl.sub(replaceFinal, l)
dest.write(l)
dest.close()
for root, dirs, files in os.walk(".", topdown=False):
for f in files:
if not f.endswith('.java'):
continue
full = os.path.join(root, f)
print full
subst(full)
Change-Id: If533a75a417594fc893e7c669d2c1f0f6caeb7ca
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>