Since the introduction of generic type parameter inference in Java 7,
it's not necessary to explicitly specify the type of generic parameters.

Enable the warning in Eclipse, and fix all occurrences.

Change-Id: I9158caf1beca5e4980b6240ac401f3868520aad0
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>

Set missingOverrideAnnotation=warning in Eclipse compiler preferences
which enables the warning:

  The method <method> of type <type> should be tagged with @Override
  since it actually overrides a superclass method

Justification for this warning is described in:

  http://stackoverflow.com/a/94411/381622

Enabling this causes in excess of 1000 warnings across the entire
code-base. They are very easy to fix automatically with Eclipse's
"Quick Fix" tool.

Fix all of them except 2 which cause compilation failure when the
project is built with mvn; add TODO comments on those for further
investigation.

Change-Id: I5772061041fd361fe93137fd8b0ad356e748a29c
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>

TransportHttp sets 'Accept-Encoding: gzip' to allow the server to
compress HTTP responses. When fetching a loose object over HTTP, it
uses the following code to read the response:

       InputStream in = openInputStream(c);
       int len = c.getContentLength();
       return new FileStream(in, len);

If the content is gzipped, openInputStream decompresses it and produces
the correct content for the object. Unfortunately the Content-Length
header contains the length of the compressed stream instead of the
actual content length. Use a length of -1 instead since we don't know
the actual length.

Loose objects are already compressed, so the gzip encoding typically
produces a longer compressed payload. The value from the Content-Length
is too high, producing EOFException: Short read of block.

Change-Id: I8d5284dad608e3abd8217823da2b365e8cd998b0
Signed-off-by: Zhen Chen <czhen@google.com>
Helped-by: Jonathan Nieder <jrn@google.com>

Change-Id: I7c545d06b1bced678c020fab9af1382bc4416b6e
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>

Change-Id: I268e66d2299a1a12f7ae44b67a6b947339038245
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>

The exception can be thrown in a various reason, and sometimes 403
Forbidden is not appropriate. Make the HTTP status code customizable.

Change-Id: If2ef6f454f7479158a4e28a12909837db483521c
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>

We have done this since forever with the "wanted old new ref" error,
so let's do it for other such errors thrown in the same block as well.

Change-Id: Ib3b1c7f05e31a5b3e40e85eb07b16736920a033b

When sending an error response due to ServiceNotAuthorizedException or
ServiceNotEnabledException, usually we send a default message.  In the
ServiceNotEnabledException case, we use

	403 Git access forbidden

except in a dumb-HTTP-specific filter where we use the servlet
container's default 403 response:

	403 Forbidden

In the ServiceNotAuthorizedException case, we use the servlet
container's default 401 response:

	401 Unauthorized

There is one exception: a ServiceNotEnabledException when handling a
smart HTTP /info/refs request uses the message from the exception:

	403 Service not enabled

Be more consistent by always using the message from the exception.  This
way, authors of a RepositoryResolver, UploadPackFactory, or
ReceivePackFactory can provide a more detailed message when appropriate.
The defaults are

	401 Unauthorized
	403 Service not enabled

Change-Id: Id1fe1c2042fb96487c3671c1965c8a65c4b8e1b8
Signed-off-by: Jonathan Nieder <jrn@google.com>

No functional change.

Change-Id: I945ba18879c360f433e026aa125ef3f9f6a75793
Signed-off-by: Jonathan Nieder <jrn@google.com>

See the discussion [1] in the Gerrit mailing list.

[1] https://groups.google.com/forum/#!topic/repo-discuss/RRQT_xCqz4o

Change-Id: I2c67384309c5c2e8511a7d0d4e088b4e95f819ff
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

Instead of dumping a full stack trace when a client sends an invalid
commit, record only a short line explaining the attempt:

  Cannot receive Invalid commit c0ff33...: invalid author into /tmp/jgit.git

The text alone is sufficient to explain the problem and the stack
trace does not lend any additional useful information. ObjectChecker
is quite clear about its rejection cases.

Change-Id: Ifc8cf06032489dc6431be1ba66101cf3d4299218

If UploadPack or ReceivePack has an exception record an identifier
associated with the repository as part of the log message. This can
help the HTTP admin track down the offending repository and take
action to repair the root cause.

Change-Id: I58f22b33cdb40994f044a26fba9fe965b45be51d

Since git-core ff5effd (v1.7.12.1) the native wire protocol transmits
the server and client implementation and version strings using
capability "agent=git/1.7.12.1" or similar.

Support this in JGit and hang the implementation data off UploadPack
and ReceivePack.  On HTTP transports default to the User-Agent HTTP
header until the client overrides this with the optional capability
string in the first line.

Extract the user agent string into a UserAgent class under transport
where it can be specified to a different value if the application's
build process has broken the Implementation-Version header in the
JGit package.

Change-Id: Icfc6524d84a787386d1786310b421b2f92ae9e65

When a user tried to use a service not enabled in the remote server
a misleading error  message was given:

   fatal: remote error: Git access forbidden

This patch modifies the error message to make the cause clearer
to the user. Now, when the user tries to use a not enabled service,
the message error clearly states it:

   fatal: remote error: Service not enabled

Change-Id: If096c4ddd17c5aae0e99e3ea6eea4b69bd3c5466
Signed-off-by: Hector Oswaldo Caballero <hector.caballero@ericsson.com>

This reverts commit 19f869996f.
Leaving path info encoded confuses applications like Gitiles.
Trying to fix this inside of JGit was maybe the wrong solution.

Change-Id: I8df9ab6233ff513e427701c8a1a66022c19784eb

Gitiles malfunctions in conjunction with jgit and guice
because of a recent Guice bug fix. Work around the problem
by parsing the URI directly, bypassing the unescaping
performed by the getPathInfo method.

This rest of this message is copied from
https://gerrit-review.googlesource.com/#/c/60820/ :

The fix for Guice issue #745[1] causes getPathInfo() within the
GuiceFilter to return decoded values, eliminating the difference
between "foo/bar" and "foo%2Fbar". This is in spec with the servlet
standard, whose javadoc for getPathInfo[2] states that the return
value be "decoded by the web container".

Work around this by extracting the path part directly from the request
URI, which is unmodified by the container. This is copying the Guice
behavior prior to the bugfix.

[1] https://github.com/google/guice/issues/745
[2] http://docs.oracle.com/javaee/7/api/javax/servlet/http/HttpServletRequest.html#getPathInfo()

Change-Id: I7fdb291bda377dab6160599ee537962d5f60f1e8
Signed-off-by: David Pletcher <dpletcher@google.com>

Bug: 431552
Change-Id: I469316f5645205016e1fa6b0fbd2ff3b509b14bc
Signed-off-by: Robin Stocker <robin@nibor.org>

This avoids the server from referencing the client code directly.

Change-Id: Ie6ade781b5a689646ad8b0b2988ef2b544412195

Currently, Repository.getAllRefs() and Repository.getTags() silently
ignores an IOException and instead returns an empty map. Repository
is a public API and as such cannot be changed until the next major
revision change. Where possible, update the internal jgit APIs to
use the RefDatabase directly, since it propagates the error.

Change-Id: I4e4537d8bd0fa772f388262684c5c4ca1929dc4c

This breaks all existing callers once. Applications are not supposed
to build against the internal storage API unless they can accept API
churn and make necessary updates as versions change.

Change-Id: I2ab1327c202ef2003565e1b0770a583970e432e9

Change-Id: I83ca25fb569c0dbc36eb374d5437fcf2b65a6f68

Change-Id: Ief195fb5c55f75172f0428fdac8c8874292ae566

Compressing the response with gzip causes the stream to delay
flushing until gzip has seen the entire response message, or buffers
fill up and the compressed data has to be sent. This hides the
resolving progress monitor from the client, as well as any other
progress messages the server might be trying to send.

Disable compression in receive, matching what /git-upload-pack has.

Change-Id: Ic8d8abe1f43c3f540d1ee7c43a8947a555307d94

Our rule to enforce javodocs for public members gives us a problem
because there are some patterns where javadoc make little sense so we
make the comments as small as possible, which our formatting rules do
not like, so disable it for those source files.

Change-Id: I6e3edb1e650ed45428b89cf41e6151b6536bca8a
Signed-off-by: Chris Aniszczyk <zx@twitter.com>

For streams that should not be closed, i.e. don't own an underlying
stream, and in-memory streams that do not need to be closed we just
suppress the warning. This mostly apply to test cases. GC is enough.

For streams with external resources (i.e. files) we add the necessary
call to close().

Change-Id: I4d883ba2e7d07f199fe57ccb3459ece00441a570

Change-Id: Ic7b8494713d59574ee8f064a5c1042b48aedf012

Invoke the wrapper types' valueOf via static imports.
For booleans used in asserts, add a new assert in
the JUnit utility package since out current version of JUnit
does not have the assert(boolean, boolean) method.

Change-Id: I9099bd8efbc8c133479344d51ce7dabed8958a2b

Instead of a confusing 403 Forbidden error indicating the dumb file
service is disabled on this server, use 406 Not Acceptable to mean
the client sent a request for content (the plain info/refs file)
that this server does not want to provide.

The stock C Git client will report HTTP 406 error if it predates
1.6.6 or something goes wrong with the smart request and it tried
falling back to the dumb request. This may help to debug cases where
a broken proxy server exists between the client and the server and
has mangled a prior smart info/refs response.

Change-Id: Ic2b78ba9502e4bbdff7cc3ba1fd284cf7616412b

Enable streaming compression for any response that is bigger than
the 32 KiB buffer used by SmartOutputStream. This is useful on the
info/refs file which can have many branches and tags listed, and
is often bigger than 32 KiB, but also compresses by at least 50%.

Disable streaming compression on large git-upload-pack responses,
as these are usually highly compressed Git pack data. Trying to
compress these with gzip will only waste CPU time and additional
transfer space with the gzip wrapper. Small git-upload-pack data
is usually text based negotiation responses and can be squeezed
smaller with a little bit of CPU usage.

Change-Id: Ia13e63ed334f594d5e1ab53a97240eb2e8f550e2

I have unfortunately introduced a few bugs in the native Git client
over the years. 1.7.5 is unable to send chunked requests correctly,
resulting in corrupt data at the server. Ban this client whenever
it uses chunked encoding with an error message.

Prior to some more recent versions, git push over HTTP failed to
report status information and error messages due to a race within
the client and its helper process. Check for these bad versions and
send errors as messages before the status report, enabling users
to see the failures on their terminal.

Change-Id: Ic62d6591cbd851d21dbb3e9b023d655eaecb0624

This has no effect on Git clients, but for browsers, 403 Forbidden may
be more appropriate. 500 Internal Server Error implies that there is
a problem with the server, whereas ServiceMayNotContinueException is
specifically intended to cover cases where the server is functioning
correctly but has determined that the request may not proceed.

Change-Id: I825abd2a029d372060103655eabf488a0547c1e8

This allows the use of precompiled patterns, such as those compiled with
flags.

Change-Id: I1c87fea98e246004aecbae3aabaf1d21fbf3176e

Include some behaviors that were not clear to me until I had used it a
few times.

Warn about broken behavior for capture groups that do not match. It
would be nice to support these, but even for the cases where it's
clear what the behavior should be, it would be infeasible to
implement.

For example, consider the second group of the regex "(/a)/b(/c)?"
matched against the path "/a/b". We might want getServletPath() to
return "/a/b" and getPathInfo() to return null, but this is hard to
implement: there's no easy way to say "the substring up to the point
where (/c) would have matched if it were in the string even though
it's not." And even if we could, it's not clear there is even a right
answer in the general case.

Moreover, ideally we could warn about such broken patterns at servlet
initialization time, rather than at runtime, but even answering the
question of whether there are capture groups that might not match
requires more customized regular expression parsing than we want to
embark on. Hence, the best we can do is document how it fails.

Change-Id: I7bd5011f5bd387f9345a0e79b22a4d7ed918a190

Implementations may want to send an error message to the user, which
doesn't really fit with any of the existing exception types.
ServiceMayNotContinueException, on the other hand, is documented as
always containing a user-visible error string, so use that.

Modify the git and HTTP transport mechanisms to properly relay this
message to the end user.

Change-Id: I362e67ea46102a145bf2c6284d38788537c9735f

Change-Id: Ie22ac47e315bff76f224214bc042fc483eb01550
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>

The fetch-pack/upload-pack stream usually has an LF at the
end of the first "want" line. Trim this when checking to
see if side-band or side-band-64k was used.

Perform the same trim for send-pack/receive-pack, as it is
harmless in this context to ignore an LF just before doing
an error report.

Change-Id: I6ef946bb6124fa72c52bd5320187eaac3ed906e7

When a client POSTs to /git-{upload,receive}-pack, the first line
includes their client capabilities. As soon as the C git client sends
side-band(-64k), it goes into a state where it chokes on data not sent
in a valid sideband channel.

GitSmartHttpTools.sendError() is called early in the request, likely
before a {Upload,Receive}Pack handler is assigned or, even so, before it
has read the request. In some cases we must read the first line manually
within sendError() to tell whether sideband is needed.

Change-Id: I8277fd45a4ec3b71fa8f87404b4f5d1a09e0f384

This is intended to replace the RefFilter interface (but does not yet,
for backwards compatibility). That interface required lots of extra
scanning and copying in filter cases such as only advertising a subtree
of the refs directory. Instead, provide a hook that can be executed
right before ref advertisement, using the public methods on
UploadPack/ReceivePack to explicitly set the map of advertised refs.

Change-Id: I0067019a191c8148af2cfb71a675f2258c5af0ca

The HTTP RFCs require a server to fully consume the request body before
it can return a non-error status code, which is any code below 400.

JGit returns most Git level errors inside of an HTTP 200 OK response,
and sometimes this happens before the entire request was consumed from
the servlet container. In such cases the body must be skipped or read
until EOF is reached, ensuring the HTTP keep-alive semantics will work
for the next request on the same TCP connection.

HTTP status codes >= 400 may be returned without consuming the body,
and a servlet container must set "Connection: close" in the response
headers when this happens, since the state of the request body is not
well defined with an early abort.

With the introduction of sendError() in GitSmartHttpTools there are
only a handful of locations that need to worry about the request body
being consumed, so sprinkle the call in as necessary.

Change-Id: I5381e110585f780c01a764df8e27c80aacf5146e

Error messages are typically short, below the 32 KiB in-memory buffer
size of the SmartOutputStream. When an error is queued up for sending
to a client and an exception is thrown up into the servlet handler we
discarded the message and sent nothing to the client, as the messages
were stuck inside of the SmartOutputStream buffer.

Hoist the creation of the output stream above the invocation of try
block of the service, and use close() in the few catch blocks that
assume there are buffered messages ready for transmission. This will
ensure errors from unpacking a stream in ReceivePack are sent off to
a client correctly, as previously these were causing no status report
to arrive at the client side as the data was stuck in the buffer.

Change-Id: I5534b560697731121f48979ae077aa7c95b8e39c

The stream must be closed to ensure the native resources associated
with its internal Deflater instance are cleaned up early, instead of
waiting for GC to identify the dead object and finialize it.

Change-Id: Ic31b5df563f19404ed4682556999f4332aa61562

The GitSmartHttpTools class started as utility functions to help report
useful error messages to users of the android.googlesource.com service.

Now that the GitServlet and GitFilter classes support filters before a
git-upload-pack or git-receive-pack request, server implementors may
these routines helpful to report custom messages to clients.  Using the
sendError() method to return an HTTP 200 OK with error text embedded in
the payload prevents native Git clients from retrying the action with a
dumb Git or WebDAV HTTP request.

Refactor some of the existing code to use these new error functions and
protocol constants.  The new sendError() function is very close to being
identical to the old error handling code in RepositoryFilter, however we
now use the POST Content-Type rather than the Accept HTTP header to check
if the client will accept the error data in the response body rather than
using the HTTP status code.  This is a more reliable way of checking for
native Git clients, as the Accept header was not always populated with the
correct string in older versions of Git smart HTTP.

Change-Id: I828ac2deb085af12b6689c10f86662ddd39bd1a2

If removing the leading slash results in an empty string, return
with an HTTP 404 error before trying to use the RepositoryResolver.
Moving this into a loop ahead of the length check ensures there is
no empty string passed into the resolver.

Change-Id: I80e5b7cf25ae9f2164b5c396a29773e5c7d7286e

All Git URLs operate off a suffix approach, for example the default
binding is for paths such as:

  */info/refs
  */git-upload-pack
  */git-receive-pack

These names are not common on project hosting servers, especially
one like Gerrit Code Review.

In addition to offering Git-over-HTTP as a servlet, offer it as a
filter that triggers when a matching suffix appears, but otherwise
delegates the request through the chain.  This filter would permit
Gerrit Code Review to place projects at the root of the server,
rather than within the "/p/" subdirectory, making the HTTP and SSH
URL structure exactly match each other.

To prevent breakage with existing users, the MetaServlet and
GitServlet are kept as wrappers delegating to their filters,
returning 404 Not Found when the filter has no match.

Change-Id: I2465c15c086497e0faaae5941159d80c028fa8b1

If an internal exception occurs while packing and the request
needs to abort, the HTTP response might already be committed due
to progress message having already been delivered to the client.
This prevents UploadPackServlet from resetting the response and
sending back an HTTP 500 response.

Try to catch all exceptions and report internal errors over the
sideband stream or as an ERR command during the initial ACK/NAK
negotiation phase. This allows JGit to transmit an error message
that the user will receive on their console without needing to
worry about resetting the (already gone) HTTP response.

Change-Id: Ie393fb8bb55d2b79ab1276adf71c781c1807f9fe
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>

The filter did not correctly match smart HTTP client requests,
so it always fell back on HTTP status codes for errors. This
usually causes a smart client to retry a dumb request, which
is not what the server wants.

Change-Id: I42592378dc42fbe308ef30a2923786c690f668a9
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>

Smart HTTP clients may request both multi_ack_detailed and no-done in
the same request to prevent the client from needing to send a "done"
line to the server in response to a server's "ACK %s ready".

For smart HTTP, this can save 1 full HTTP RPC in the fetch exchange,
improving overall latency when incrementally updating a client that
has not diverged very far from the remote repository.

Unfortuantely this capability cannot be enabled for the traditional
bi-directional connections.  multi_ack_detailed has the client sending
more "have" lines at the same time that the server is creating the
"ACK %s ready" and writing out the PACK stream, resulting in some race
conditions and/or deadlock, depending on how the pipe buffers are
implemented.  For very small updates, a server might actually be able
to send "ACK %s ready", then the PACK, and disconnect before the
client even finishes sending its first batch of "have" lines.  This
may cause the client to fail with a broken pipe exception.  To avoid
all of these potential problems, "no-done" is restricted only to the
smart HTTP variant of the protocol.

Change-Id: Ie0d0a39320202bc096fec2e97cb58e9efd061b2d
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>

When the client is clearly making a smart HTTP request to our smart
HTTP server, return any errors like RepositoryNotFoundException or
ServiceNotEnabledException inside of the payload as a Git level ERR
message, rather than an HTTP error code.

This prevents the C Git command line client from retrying a failed
"$URL/info/refs?service=git-upload-pack" request without the smart
service URL, only to fail again with "403 Forbidden" when the dumb
as-is service has been disabled by the server configuration, or is
unavailable because the repository is not on the local filesystem.

Change-Id: I57e8756d5026e885e0ca615979bfcd729703be6c
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>

Embedding applications can use this hook to watch actions within
UploadPack and possibly reject them. This could be useful to prevent
clones of a large repository from this server, or to stop abusive
negotiation rounds that offer thousands of objects in a single batch.

Change-Id: Id96f1885ac4d61f22c80b6418fff54184b7348ba
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>

Permit applications embedding GitServlet to wrap the
info/refs?service=$name and /$name operations with a
servlet Filter.

To help applications inspect state of the operation,
expose the UploadPack or ReceivePack object into a
request attribute.  This can be useful for logging,
or to implement throttling of requests like Gerrit
Code Review uses to prevent server overload.

Change-Id: Ib8773c14e2b7a650769bd578aad745e6651210cb
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>