UploadPackServlet: Use uploadWithExceptionPropagation
As UploadPackErrorHandler's Javadoc says, UploadPackServlet should have
called uploadWithExceptionPropagation and let UploadPackErrorHandler to
handle the exception. Fix UploadPackServlet.
Change-Id: I1f9686495fcf3ef28598ccdff3e6f76a16c8bca3
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
Enable and fix "Statement unnecessarily nested within else clause" warnings
Since [1] the gerrit project includes jgit as a submodule, and has this
warning enabled, resulting in 100s of warnings in the console.
Also enable the warning here, and fix them.
At the same time, add missing braces around adjacent and nearby one-line
blocks.
[1] https://gerrit-review.googlesource.com/c/gerrit/+/227897
Change-Id: I81df3fc7ed6eedf6874ce1a3bedfa727a1897e4c
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Factor out the test parameterization to use both connection factories
into a common super class and use it in more tests.
This made HttpClientTests.testV2HttpSubsequentResponse() fail for
Apache HTTP. The test used the pattern
- create POST connection
- setDoOutput(true)
- connect()
- write output stream
- get & read input stream
This pattern is never used in JGit, which actually calls connect() only
in one case in LFS, and that's on a HEAD request.
The above pattern works on JDK, but fails on Apache HTTP because with
Apache HTTP a connect() actually executes the full request including
writing the entity. To work with Apache HTTP, the pattern would need
to be
- create POST connection
- setDoOutput(true)
- write output stream
- connect()
- get & read input stream
which is fine for both. JDK connects implicitly in getOutputStream()
and treats the later explicit connect() as a no-op, and Apache works
because the entity is written when connect() is called.
Because JDK connects implicitly on getOutputStream(), the following
pattern also works with JDK:
- create POST connection
- setDoOutput(true)
- write output stream
- get & read input stream
Support this with Apache HTTP too: let getInputStream() execute
the request if it wasn't executed already.
Remove explicit connect() calls from test code, since JGit doesn't do
those either.
Change-Id: Ica038c00a7b8edcc01d5660d18e961146305b87f
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Cache user global and system-wide git configurations
So far the git configuration and the system wide git configuration were
always reloaded when jgit accessed these global configuration files to
access global configuration options which are not in the context of a
single git repository. Cache these configurations in SystemReader and
only reload them if their file metadata observed using FileSnapshot
indicates a modification.
Change-Id: I092fe11a5d95f1c5799273cacfc7a415d0b7786c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
TransportHttp: resolve redirect URIs from the "Location" header
The "Location" header in a redirect response may contain a relative
URI. Resolve it against the URI the request was made.
Bug: 550033
Change-Id: I29de07dfbbbc794090821b7c190cb2cf662c5a60
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
SmartClientSmartServerTest: Open Repository in try-with-resource
Since 52923e9 ("LocalDiskRepositoryTestCase#createRepository: Default
auto-close to false", Jan 20, 2019) the createBareRepository method
creates repositories that do not get automatically closed in #tearDown.
Convert invocations of createBareRepository to use try-with-resource.
Change-Id: I320030c5d4438713971bee33316bff408bac47fc
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Its implementation contains
} catch (IOException e) {
// Legacy API, assume error means "no"
return false;
}
Better to use ObjectDatabase#has, which throws IOException to report
errors.
Change-Id: I7de02f7ceb8f57b2a8ebdb16d2aa4376775ff933
Signed-off-by: Jonathan Nieder <jrn@google.com>
UploadPack: Test filtering by AdvertiseRefsHook in stateless transports
AdvertiseRefsHook is used to limit the visibility of the refs in Gerrit.
If this hook is not called, then all refs are treated as visible,
causing the server to serve commits reachable from branches the client
should not be able to access, if asked to via a request naming a guessed
object id.
Until 3a529361a7 (Call AdvertiseRefsHook
before validating wants, 2018-12-18), UploadPack would invoke this hook
at ref advertisement time but not during negotiation and when serving a
pack file. Add a test to avoid regressing. Stateful bidirectional
transports were not affected, so the test uses HTTP.
[jn: split out when backporting the fix to stable-4.5. The test passes
as long as v4.9.0.201710071750-r~169 (fetch: Accept any SHA-1 on lhs of
refspec, 2017-06-04) is cherry picked along with it.]
Change-Id: I8c017107336adc7cb4c826985779676bf043e648
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
Signed-off-by: Jonathan Nieder <jrn@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Open auto-closeable resources in try-with-resource
When an auto-closeable resources is not opened in try-with-resource,
the warning "should be managed by try-with-resource" is emitted by
Eclipse.
Fix the ones that can be silenced simply by moving the declaration of
the variable into a try-with-resource.
In cases where we explicitly call the close() method, for example in
tests where we are testing specific behavior caused by the close(),
suppress the warning.
Leave the ones that will require more significant refcactoring to fix.
They can be done in separate commits that can be reviewed and tested
in isolation.
Change-Id: I9682cd20fb15167d3c7f9027cecdc82bc50b83c4
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Use constants from StandardCharsets instead of hard-coded strings
Instead of hard-coding the charset strings "US-ASCII", "UTF-8", and
"ISO-8859-1", use the corresponding constants from StandardCharsets.
UnsupportedEncodingException is not thrown when the StandardCharset
constants are used, so remove the now redundant handling.
Because the encoding names are no longer hard-coded strings, also
remove redundant $NON-NLS warning suppressions.
Also replace existing usages of the constants with static imports.
Change-Id: I0a4510d3d992db5e277f009a41434276f95bda4e
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Cleanup: message reporting for HTTP redirect handling
The addition of "tooManyRedirects" in commit 7ac1bfc ("Do
authentication re-tries on HTTP POST") was an error I didn't
catch after rebasing that change. That message had been renamed
in the earlier commit e17bfc9 ("Add support to follow HTTP
redirects") to "redirectLimitExceeded".
Also make sure we always use the TransportException(URIish, ...)
constructor; it'll prefix the message given with the sanitized URI.
Change messages to remove the explicit mention of that URI inside the
message. Adapt tests that check the expected exception message text.
For the info logging of redirects, remove a potentially present
password component in the URI to avoid leaking it into the log.
Change-Id: I517112404757a9a947e92aaace743c6541dce6aa
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
There is at least one git server out there (GOGS) that does
not require authentication on the initial GET for
info/refs?service=git-receive-pack but that _does_ require
authentication for the subsequent POST to actually do the push.
This occurs on GOGS with public repositories; for private
repositories it wants authentication up front.
Handle this behavior by adding 401 handling to our POST request.
Note that this is suboptimal; we'll re-send the push data at
least twice if an authentication failure on POST occurs. It
would be much better if the server required authentication
up-front in the GET request.
Added authentication unit tests (using BASIC auth) to the
SmartClientSmartServerTest:
- clone with authentication
- clone with authentication but lacking CredentialsProvider
- clone with authentication and wrong password
- clone with authentication after redirect
- clone with authentication only on POST, but not on GET
Also tested manually in the wild using repositories at try.gogs.io.
That server offers only BASIC auth, so the other paths
(DIGEST, NEGOTIATE, fall back from DIGEST to BASIC) are untested
and I have no way to test them.
* public repository: GET unauthenticated, POST authenticated
Also tested after clearing the credentials and then entering a
wrong password: correctly asks three times during the HTTP
POST for user name and password, then gives up.
* private repository: authentication already on GET; then gets
applied correctly initially to the POST request, which succeeds.
Also fix the authentication to use the credentials for the redirected
URI if redirects had occurred. We must not present the credentials
for the original URI in that case. Consider a malicious redirect A->B:
this would allow server B to harvest the user credentials for server
A. The unit test for authentication after a redirect also tests for
this.
Bug: 513043
Change-Id: I97ee5058569efa1545a6c6f6edfd2b357c40592a
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
git-core follows HTTP redirects so JGit should also provide this.
Implement config setting http.followRedirects with possible values
"false" (= never), "true" (= always), and "initial" (only on GET, but
not on POST).[1]
We must do our own redirect handling and cannot rely on the support
that the underlying real connection may offer. At least the JDK's
HttpURLConnection has two features that get in the way:
* it does not allow cross-protocol redirects and thus fails on
http->https redirects (for instance, on Github).
* it translates a redirect after a POST to a GET unless the system
property "http.strictPostRedirect" is set to true. We don't want
to manipulate that system setting nor require it.
Additionally, git has its own rules about what redirects it accepts;[2]
for instance, it does not allow a redirect that adds query arguments.
We handle response codes 301, 302, 303, and 307 as per RFC 2616.[3]
On POST we do not handle 303, and we follow redirects only if
http.followRedirects == true.
Redirects are followed only a certain number of times. There are two
ways to control that limit:
* by default, the limit is given by the http.maxRedirects system
property that is also used by the JDK. If the system property is
not set, the default is 5. (This is much lower than the JDK default
of 20, but I don't see the value of following so many redirects.)
* this can be overwritten by a http.maxRedirects git config setting.
The JGit http.* git config settings are currently all global; JGit has
no support yet for URI-specific settings "http.<pattern>.name". Adding
support for that is well beyond the scope of this change.
Like git-core, we log every redirect attempt (LOG.info) so that users
may know about the redirection having occurred.
Extends the test framework to configure an AppServer with HTTPS support
so that we can test cloning via HTTPS and redirections involving HTTPS.
[1] https://git-scm.com/docs/git-config
[2] 6628eb41db
[3] https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html
CQ: 13987
Bug: 465167
Change-Id: I86518cb76842f7d326b51f8715e3bbf8ada89859
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Update to the latest 9.3 version until we have a solution for bug 514336
we face with 9.4.x versions.
Change-Id: I6d8d476abe8677ce865a08099bb77330effc700a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
MappedLoginService is no longer available in Jetty 9.4 therefore base
TestLoginService on AbstractLoginService.
Apparently Jetty now uses slf4j hence adapt RecordingLogger accordingly
so we can log error messages containing slf4j style formatting anchors
"{}".
Change-Id: Ibb36aba8782882936849b6102001a88b699bb65c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
(cherry picked from commit 5e8e2179b2)
Enable and fix warnings about redundant specification of type arguments
Since the introduction of generic type parameter inference in Java 7,
it's not necessary to explicitly specify the type of generic parameters.
Enable the warning in Eclipse, and fix all occurrences.
Change-Id: I9158caf1beca5e4980b6240ac401f3868520aad0
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Enable and fix 'Should be tagged with @Override' warning
Set missingOverrideAnnotation=warning in Eclipse compiler preferences
which enables the warning:
The method <method> of type <type> should be tagged with @Override
since it actually overrides a superclass method
Justification for this warning is described in:
http://stackoverflow.com/a/94411/381622
Enabling this causes in excess of 1000 warnings across the entire
code-base. They are very easy to fix automatically with Eclipse's
"Quick Fix" tool.
Fix all of them except 2 which cause compilation failure when the
project is built with mvn; add TODO comments on those for further
investigation.
Change-Id: I5772061041fd361fe93137fd8b0ad356e748a29c
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
MappedLoginService is no longer available in Jetty 9.4 therefore base
TestLoginService on AbstractLoginService.
Apparently Jetty now uses slf4j hence adapt RecordingLogger accordingly
so we can log error messages containing slf4j style formatting anchors
"{}".
Change-Id: Ibb36aba8782882936849b6102001a88b699bb65c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Fix assertion in SmartClientSmartServerTest.testPush_CreateBranch()
This assertion only defined a message but didn't assert anything.
Change-Id: I0914642b64b69dc4e3ec24acbf8052f9171613d8
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Now if refs are unreadable when serving an upload pack the handler
will fail due to the actual underlying failure. Previously all wants
would be rejected as invalid because Repository.getAllRefs() returned
an empty map.
Testing this required a new subclass of InMemoryRepository so that
an IOException could be injected at the correct time.
Signed-off-by: Michael Edgar <adgar@google.com>
Change-Id: Iac708b1db9d0ccce08c4ef5ace599ea0b57afdc0
smart HTTP server: Pass along "want X not valid" to client
If the client sends a SHA-1 that the server does not recognize echo
this back to the client with an explicit error message instead of
the generic "internal server error".
This was always the intent of the implementation but it was being
dropped on smart HTTP due to the UploadPackServlet catching the
PackProtocolException, discarding the buffered message UploadPack
meant to send, and sending along a generic message instead.
Change-Id: I8d96b064ec655aef64ac2ef3e01853625af32cd1
After creating a Transport instance callers should always call
its close() method. Use AutoCloseable to document this idiom
and allow use of try-with-resources.
Change-Id: I0c6ff3e39ebecdd7a028dbcae1856a818937b186
Repository: Introduce exactRef and findRef, deprecate getRef
The Repository class provides only one method to look up a ref by
name, getRef. If I request refs/heads/master and that ref does not
exist, getRef will look further in the search path:
ref/refs/heads/master
refs/heads/refs/heads/master
refs/remotes/refs/heads/master
This behavior is counterintuitive, needlessly inexpensive, and usually
not what the caller expects.
Allow callers to specify whether to use the search path by providing
two separate methods:
- exactRef, which looks up a ref when its exact name is known
- findRef, which looks for a ref along the search path
For backward compatibility, keep getRef as a deprecated synonym for
findRef.
This change introduces findRef and exactRef but does not update
callers outside tests to use them yet.
Change-Id: I35375d942baeb3ded15520388f8ebb9c0cc86f8c
Signed-off-by: Jonathan Nieder <jrn@google.com>
Fix LocalDiskRepositoryTestCase to create correct type of repos
In one place LocalDiskRepositoryTestCase was ignoring the specification
whether to create a bare or non-bare repository. Fix this and fix also
one test which fails now because bare repos don't write reflogs by
default.
Change-Id: I4bcf8cf97c5b46e2f3919809eaa121a8d0e47010
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Add an implementation for HttpConnection using Apache HttpClient
This change implements the http connection abstraction with the help of
org.apache.http.client.HttpClient. The default implementation used by
JGit is still the JDK HttpURLConnection. But now JGit users have the
possibility to switch completely to org.apache.httpclient. The reason
for this is that in certain (e.g. cloud) environments you are forced to
use the org.apache classes.
Change-Id: I0b357f23243ed13a014c79ba179fa327dfe318b2
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
JGit 3.0: move internal classes into an internal subpackage
This breaks all existing callers once. Applications are not supposed
to build against the internal storage API unless they can accept API
churn and make necessary updates as versions change.
Change-Id: I2ab1327c202ef2003565e1b0770a583970e432e9
Revert Jetty from 8.1.3.v20120416 to 7.6.0.v20120127
This reverts commit 24a0f47e32 and
updates JGit dependencies to use the latest available Jetty 7.x
release. We can't use Jetty 8.x since it depends on Servlet API 3.0
which requires Java 6 but JGit still wants to support Java 5.
Use one of the target platforms defined in
Ibf67a6d3539fa0708a3e5dbe44fb899c56fbd8ed to work with that in Eclipse.
Change-Id: I343273d994dc7b6e0287c604e5926ff77d5b585b
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
It's useful to have ReflogEntry refactored out so it can be
used by clients via the JGit API.
Change-Id: I03044df9af9f9547777545b7c9b93bdf5f8b7cb5
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
Smart HTTP clients may request both multi_ack_detailed and no-done in
the same request to prevent the client from needing to send a "done"
line to the server in response to a server's "ACK %s ready".
For smart HTTP, this can save 1 full HTTP RPC in the fetch exchange,
improving overall latency when incrementally updating a client that
has not diverged very far from the remote repository.
Unfortuantely this capability cannot be enabled for the traditional
bi-directional connections. multi_ack_detailed has the client sending
more "have" lines at the same time that the server is creating the
"ACK %s ready" and writing out the PACK stream, resulting in some race
conditions and/or deadlock, depending on how the pipe buffers are
implemented. For very small updates, a server might actually be able
to send "ACK %s ready", then the PACK, and disconnect before the
client even finishes sending its first batch of "have" lines. This
may cause the client to fail with a broken pipe exception. To avoid
all of these potential problems, "no-done" is restricted only to the
smart HTTP variant of the protocol.
Change-Id: Ie0d0a39320202bc096fec2e97cb58e9efd061b2d
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
When the client is clearly making a smart HTTP request to our smart
HTTP server, return any errors like RepositoryNotFoundException or
ServiceNotEnabledException inside of the payload as a Git level ERR
message, rather than an HTTP error code.
This prevents the C Git command line client from retrying a failed
"$URL/info/refs?service=git-upload-pack" request without the smart
service URL, only to fail again with "403 Forbidden" when the dumb
as-is service has been disabled by the server configuration, or is
unavailable because the repository is not on the local filesystem.
Change-Id: I57e8756d5026e885e0ca615979bfcd729703be6c
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Using a resolver and factory pattern for the anonymous git:// Daemon
class makes transport.Daemon more useful on non-file storage systems,
or in embedded applications where the caller wants more precise
control over the work tasks constructed within the daemon.
Rather than defining new interfaces, move the existing HTTP ones
into transport.resolver and make them generic on the connection
handle type. For HTTP, continue to use HttpServletRequest, and
for transport.Daemon use DaemonClient.
To remain compatible with transport.Daemon, FileResolver needs to
learn how to use multiple base directories, and how to export any
Repository instance at a fixed name.
Change-Id: I1efa6b2bd7c6567e983fbbf346947238ea2e847e
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Matthias Sohn
Masaya Suzuki
David Pursehouse
Thomas Wolf
Carsten Hammer
Jonathan Nieder
Bo Zhang
Mike Edgar
Shawn Pearce
Christian Halstrick
Robin Rosenberg
Chris Aniszczyk