AmazonS3: Buffer pushed pack content under $GIT_DIR
This applies the same filesystem permissions as the source objects.
Users may override in properties files using the tmpdir value.
Change-Id: I3ec332cf41f12eae246cfaee9fd792c52cb2908b
RecievePack already honors fsck settings for safeForWindows and
safeForMacOS. Allow those same checks to be performed during fetch
through a caller-configurable ObjectChecker.
Default the fetch fsck options to match the current platform, as
it can be reasonably assumed the repository will be accessed here.
Change-Id: I3c0f411fad209c6bd8fb9c4acf5c55a6799a6a2a
cgit has this feature for some time. This will teach JGit to send symbolic refs,
too.
Change-Id: I7cb2ab4e6d31a838a0af92eac64535fdb66ed74a
Signed-off-by: Yuxuan 'fishy' Wang <fishywang@google.com>
Fix variable name and error message for sideband testing
Actually the test only allows a range from [1,255], so let's name the
variable so.
Change-Id: Iecdb8149b83389c67e3cd2f64f4a654c175475be
Signed-off-by: Stefan Beller <sbeller@google.com>
Implement atomic refs update, if possible by database
Inspired by the series[1], this implements the possibility to
have atomic ref transactions.
If the database supports atomic ref update capabilities, we'll
advertise these. If the client wishes to use this feature, either
all refs will be updated or none at all.
[1] http://thread.gmane.org/gmane.comp.version-control.git/259019/focus=259024
Change-Id: I7b5d19c21f3b5557e41b9bcb5d359a65ff1a493d
Signed-off-by: Stefan Beller <sbeller@google.com>
The chaining credentials provider sequentially tries to obtain
credentials from a list of credential providers and returns the
credentials from the first provider which can provide them.
Change-Id: I499f304119d7066d011dbde3556dee6facee8ab0
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Do not use .netrc implicitly if no CredentialsProvider was set
Do not silently set the NetRCCredentialsProvider if no
CredentialsProvider was set explicitly since applications may want to
have full control which provider should be used.
Bug: 444338
Change-Id: Ie096983bc1caa90443a504d302bfea8f2d26ab9e
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Eliminate warnings for non-nls strings that will never be translated
Some of these eliminations just reduces the number of warnings on
lines where messages are constructed that can/will be translated.
Change-Id: I6eddb39ccc8f2488741bb58540d9ec5f5665e2c4
Prevent NPE if no CredentialsProvider is registered
If the git server requires authentication and no CredentialsProvider is
registered TransportHttp.connect() would throw an NPE since it tries to
reset the credentials provider. Instead throw a TransportException
explaining the problem.
Change-Id: Ib274e7d9c43bba301089975423de6a05ca5169f6
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
UploadPack: Always make PackWriter.Statistics available
If the packer fails, still obtain the stats and make them available
to the logger and the caller. Failures can frequently happen when
a client disconnects in the middle of a pack stream. Server admins
may still want to examine the timing metrics from counting and
compressing phases.
Change-Id: Iceae4f68b5473f4223d85c9edfb57837fc818eed
In Git 1.9 (5dbd767601 "support pushing from a shallow clone")
the git-core project intentionally broke the existing send-pack
protocol from shallow clients.
Shallow clients now transmit their shallow information during push,
ahead of the old-new command sequence. JGit must accept these lines
when presented.
To protect the server against clients sending partial history,
require the connectivity check when pushed to by a shallow client.
Change-Id: I46639366b0900052c376091e1688f07def44ab79
Throw URISyntaxException for incorrect percent-encoding
URIish.unescape() threw an ArrayIndexOutOfBoundsException if the given
url has incorrect percent-encoding (e.g. http://example.com/%gg). But an
URISyntaxException is much better to know the reason of the failure.
Change-Id: I3f40a26d43cd2eb4e32c11aba7dc2594bc1f98e2
Signed-off-by: Yi EungJun <eungjun.yi@navercorp.com>
Cleanup use of java.util.Inflater, fixing rare infinite loops
The native implementation of inflate() can set finished to return
true at the same time as it copies the last bytes into the buffer.
Check for finished on each iteration, terminating as soon as libz
knows the stream was completely inflated.
If not finished, it is likely input is required before the next
native call could do any useful work. Most invocations are passing
in a buffer large enough to store the entire result. A partial return
from inflate() will need more input before it can continue. Checking
right away that needsInput() is true saves a native call to determine
no bytes can be inflated without more input.
This should fix a rare infinite loop condition inside of inflation
when an object ends exactly at the end of a block boundary, and
the next block contains only the 20 byte trailing SHA-1.
When the stream is finished each new attempt to inflate() returns
n == 0, as no additional bytes were output. The needsInput() test
tries to add the length of the footer block to itself, but then loops
back around an reloads the same block as the block is smaller than
a full block size. A zero length input is set to the inflater,
which triggers needsInput() condition again.
Change-Id: I95d02bfeab4bf995a254d49166b4ae62d1f21346
Allow retrying connecting SshSession in case of an exception
Connecting to an SshSession may fail due to different reasons. Jsch for
example often throws an com.jcraft.jsch.JschException: verify: false.[1]
The issue is still not fixed in JSch 0.1.51.
In such a case it is worth retrying to connect. The number of connection
attempts can be configured using ssh_config parameter
"ConnectionAttempts" [2].
Don't retry if the user canceled authentication.
[1] http://sourceforge.net/p/jsch/bugs/58/
[2] http://linux.die.net/man/5/ssh_config
Bug: 437656
Change-Id: I6dd2a3786b7d3f15f5a46821d8edac987a57e381
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
0b5441a8 introduced an enum for authentication types and changed the
case of digest and basic authentication type names to all uppercase.
This broke digest authentication at least when using Gerrit as the git
server.
According to RFC2617 [1] "Basic" and "Digest" is the literal to be used
in authentication headers and not "BASIC" [1] and "DIGEST" [2].
According to RFC4559 "Negotiate" [3] is used for SPNEGO based
authentication.
[1] http://tools.ietf.org/html/rfc2617#page-5
[2] http://tools.ietf.org/html/rfc2617#page-8
[3] http://tools.ietf.org/html/rfc4559#page-3
Bug: 435866
Change-Id: I6173aff9352d7def225cafe2d73e5176ad40dff0
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Adds support for Negotiate(SPNEGO) HTTP authentication method. This method
is set to have a higher priority as Digest HTTP authentication method.
Bug: 428836
Change-Id: Ib181096d39f538df1dd7d3f36516843777bf12ae
Signed-off-by: Laurent Goujon <lgoujon@twitter.com>
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
Creates HttpAuthMethod type enum to support auth ordering
Refactors HttpAuthMethod to support more authentication methods,
still sorted by priority orders.
Bug: 428836
Change-Id: I049c1742e7afbc51f3f6033fa4d471b344813cfa
Signed-off-by: Laurent Goujon <lgoujon@twitter.com>
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
Detects background authentication and force use of jgit authentication
Sun HttpURLConnection is able to handle authentication like SPNEGO without
caller intervention. However, there are some restrictions:
- do not need user direct input (user,password for example)
- it doesn't work when request body is chunked/streamed (because it cannot be
replayed)
Unfortunately there is no real way to leverage HttpURLConnection authentication
work as the authentication header is stripped off the request before returning
to the caller. There's also no way to explicitly disable authentication in
HttpURLConnection (SPNEGO auth will always be attempted if a valid token can be
created by GSSAPI).
This is an issue for jgit since it is expected that the first request will be
used to detect authentication method, and reuse for the subsequent requests.
This patch modifies TransportHTTP to detect authentication done in the background
by HttpURLConnection and sets the jgit authentication method accordingly so it will
always work for future requests (assuming that the authentication method used by
HttpURLConnection is also supported by jgit).
Bug: 428836
Change-Id: I79f3b70ca2b8377e20da8e6a01914e43e96595ce
Signed-off-by: Laurent Goujon <lgoujon@twitter.com>
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
When JGit uses bitmaps (which is the case after a gc), the push command
doesn't go through the code where MissingObjectExceptions are caught
for remote objects not found locally.
Fixed by removing earlier non-locally-found remote objects.
This was seen withing gerrit, see:
https://code.google.com/p/gerrit/issues/detail?id=2025
Bug: 426044
Change-Id: Ieda718a0530e3680036edfa0963ab88fdd1362c0
Signed-off-by: Jean-Jacques Lafay <jeanjacques.lafay@gmail.com>
Signed-off-by: Doug Kelly <dougk.ff7@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
JGit caches the contents of the known_hosts file within JSch when
first started, and never attempts to re-read it. If the contents
change (such as when using Gerrit replication), the new host will
never be found.
On failures, try reloading the knonwn_hosts and retry the
connection, in addition to reloading the credentials.
Change-Id: I980c4f4003f2a48c71b9b47c51e640d1e7742f58
Signed-off-by: Doug Kelly <dougk.ff7@gmail.com>
Expose ReceiveCommand version of TrackingRefUpdate
If the caller uses a BatchRefUpdate to execute these (e.g. fetch)
there may be more detailed status information inside of the command
that was used to execute it. Allow the caller to obtain that status
data from the ReceiveCommand version of the TrackingRefUpdate.
Change-Id: I47209377fb8fcef0346d7678f1ab27f4790b49bc
Ensure that stored credentials aren't reset too early
Some commands are started without showing a dialog allowing to enter
credentials if needed. Hence we need to tolerate one failing HTTP
authentication to trigger loading credentials from the secure store.
Hence we should not immediately reset the stored credentials if the
first attempt to authenticate fails.
Bug: 431209
Change-Id: I1b9fa34c3d70be226bb1c59c9ebe995998d29bc8
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Retry to call credentials provider if http authentication failed
If the user provided wrong credentials or credentials changed we
shouldn't give up immediately but retry to get valid credentials from
the credentials provider. Reset the credentials provider if
authentication failed to avoid it reuses wrong credentials in
case it stored them in a persistent store.
Bug: 338048
Bug: 342592
Bug: 427735
Change-Id: Ibd62ef3da17be6454991c43f524c8bbc7ca3c37e
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Ensure that ssh authentication is retried only in JGit
JGit wants to control retries during ssh authentication. But JSch by
default already retries 6 times. Hence disable retries in JSch. Without
this the credentials provider isn't reset if e.g. wrong credentials are
specified.
Bug: 430210
Change-Id: I6fa726a14a84b06411e7010608c7a1e2bfb8bfe8
Display progress while checking connectivity on push
Verifying 100 new objects are fully connected to the existing DAG
is usually very cheap. Checking the entire Linux kernel history is
fully connected when pushing it to a new repository can take 30-60
seconds. Display a progress counter during this time so the client
knows the server is still working.
Change-Id: Iababe3ee1d35cb82f2bef2f12da7a2ecd03282b0
Allow configuration of receive pack's ObjectChecker through fsck.*
fsck.allowLeadingZeroFileMode may be set true to permit pushing
broken trees with leading '0' in the file mode.
fsck.safeForWindows may be set true to require new trees to have
only file names that are safe on the Windows platform.
fsck.safeForMacOS may be set true to require new trees to have
only file names that do not cause collisions or confusion on the
Mac OS platform.
Change-Id: I1a225c1b3cd13c0d1a0d43fffe79355c501f49b7
Correct fetch to use fetch.fsckObjects and transfer.fsckObjects
git-core configures fsck during fetch using these two variables.
JGit use of fsck in fetch predates the usage in git-core and had
reused receive.fsckobjects, which does not match behavior.
Change-Id: Ie4d5f082258c4b2928c7ecc3780c6363fa587f34
Default receive.fsckObjects to transfer.fsckObjects
ReceivePack should configure itself with receive.fsckObjects,
and if not defined, transfer.fsckObjects. This is the order
used by git-core.
Change-Id: I41f243633dacb606dbcc3132972f63bbaba174d1
Allow ReceivePack callers to configure their own ObjectChecker
PackParser permits supplying a specific ObjectChecker instance.
Allow this to be passed through ReceivePack, giving the caller
more flexibility to configure the implementation.
Change-Id: I9440dd25588008626222f33bfd697f57c05b439e
PostReceiveHooks can make use of this information to, for example,
update a cached size of the Git repository.
Change-Id: I2bf1200959a50531e2155a7609c96035ba45b10d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Revert "Add getPackFile to ReceivePack to make PostReceiveHook more
usable"
This reverts commit 2670fd427c.
By returning an instance of File from the ReceivePack.getPackFile the
abstraction of the persistence implementation was broken.
Change-Id: I28e3ebf3a659a7cbc94be51bba9e1ad338f2b786
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Add getPackFile to ReceivePack to make PostReceiveHook more usable
Having access to the pack file that was created by the ReceivePack
may be useful for post receive hooks. For example, a hook may want
to check the size of the received pack and the created index.
Change-Id: I4d51758e4565d32c9f8892242947eb72644b847d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Possibility to limit the max pack size on receive-pack
The maxPackSizeLimit, when set, will reject a pack if it exceeds
that limit.
This feature is intended to provide a mechanism to control disk space
quota on Git repositories.
Change-Id: I83d8db670875c395f8171461b402083323e623a5
CQ: 7896
Move Apache httpclient based HTTP support to a separate bundle
This move avoids that all consumers of org.eclipse.jgit depend on Apache
httpclient. Also add another feature to make this optional for OSGi
consumers as well.
Change-Id: I5ef5e00c53678b9e1d7cfd54bbca3ff6f1c1c967
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Add an implementation for HttpConnection using Apache HttpClient
This change implements the http connection abstraction with the help of
org.apache.http.client.HttpClient. The default implementation used by
JGit is still the JDK HttpURLConnection. But now JGit users have the
possibility to switch completely to org.apache.httpclient. The reason
for this is that in certain (e.g. cloud) environments you are forced to
use the org.apache classes.
Change-Id: I0b357f23243ed13a014c79ba179fa327dfe318b2
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Previously all HTTP communication was done with the help of
java.net.HttpUrlConnection. In order to make JGit usable in environments
where the direct usage of such connections is not allowed but where the
environment provides other means to get network connections an
abstraction for connections is introduced. The idea is that new
implementations of this interface will be introduced which will not use
java.net.HttpUrlConnection but use e.g.
org.apache.client.http.HttpClient to provide network connections.
One example: certain cloud infrastructures don't allow that components
in the cloud communicate directly with HttpUrlConnection. Instead they
provide services where a component can ask for a connection (given a
symbolic name for the destination) and where the infrastructure returns
a preconfigured org.apache.http.client.HttpClient. In order to allow
JGit to be running in such environments we need the abstraction
introduced in this commit.
Change-Id: I3b06629f90a118bd284e55bb3f6465fe7d10463d
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Shawn Pearce
Yuxuan 'fishy' Wang
Stefan Beller
Christian Halstrick
Dani Megert
Robin Stocker
Matthias Sohn
Robin Rosenberg
Yi EungJun
Shawn Pearce
Alexey Kuznetsov
Stefan Lay
Anton Bannykh
Laurent Goujon
Jean-Jacques Lafay
Bogdan Gheorghe
Doug Kelly
Saša Živkov