Currently, unit tests need to either close the Repository underlying a
TestRepository manually, or not close it at all. Both are error prone.
The TestRepository holds a reference to 4 AutoCloseable objects:
Repository, ObjectInserter, Git, and RevWalk. The last two can escape
the TestRepository scope, so they are not closed when TestRepository is
closed.
Change-Id: I4461bb9104d517bd6bef09c38507c7c2ef5c31d4 Signed-off-by: Jackson Toeniskoetter <jackdt@google.com>
Minh Thai [Tue, 18 Dec 2018 23:06:59 +0000 (15:06 -0800)]
DfsBlockCache: Consolidate where ReadableChannel is opened
Opening a readable channel can be expensive and the number of channels
can be limited in DFS. Ensure that caller of
BlockBasedFile.readOneBlock() is responsible for opening and closing
the file, and that the ReadableChannel is reused in the request. As a side
effect, this makes the code easier to read, with better use of
try-with-resources.
The downside is that this means a readable channel is always opened, even
when the entire pack is already available for copying from cache. This
should be an acceptable cost: it's rare enough not to overload the server
and from a client latency perspective, the latency cost is in the noise
relative to the data transfer cost involved in a clone. If this turns out
to be a problem in practice, we can reintroduce that optimization in a
followup change.
Change-Id: I340428ee4bacd2dce019d5616ef12339a0c85f0b Signed-off-by: Minh Thai <mthai@google.com> Signed-off-by: Jonathan Nieder <jrn@google.com>
Minh Thai [Sat, 22 Dec 2018 00:27:16 +0000 (16:27 -0800)]
DfsBlockCache to lock while loading object references
We see the same index being loaded by multiple threads. Each is
hundreds of MB and takes several seconds to load, causing server to
run out of memory. This change introduces a lock to avoid these
duplicate works. It uses a new set of locks similar in implementation
to the loadLocks for getOrLoad of blocks. The locks are kept separate
to prevent long-running index loading from blocking out fast block
loading. The cache instance can be configured with a consumer to
monitor the wait time of the new locks.
Fix "jgit checkout -f" to overwrite dirty worktree files
CheckoutCommand had a setForce() method. But this didn't correspond
to native git's 'git checkout -f' option. Deprecate the old setForce()
method and move its implementation to a new method setForceRefUpdate()
and use it to implement the -B option in the CLI class Checkout.
Add a setForced() method and use it to fix the associated '-f' option of
the CLI Checkout class to behave like native git's 'git checkout -f'
which overwrites dirty worktree files during checkout.
This is still not fully matching native git's behavior: updating
additionally dirty index entries is not done yet.
Bug: 530771
Change-Id: I776b78eb623b6ea0aca42f681788f2e4b1667f15 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Jonathan Nieder [Sat, 5 Jan 2019 00:02:07 +0000 (16:02 -0800)]
Don't swallow IOException
Swallowing intermittent errors and trying to recover from them
makes JGit's behavior hard to predict and difficult to debug.
Propagate the errors instead. This doesn't violate JGit's usual
backward compatibility promise for clients because in these
contexts an IOException indicates either repository corruption or
a true I/O error. Let's consider these cases one at a time.
In the case of repository corruption, falling back e.g. to an empty
set of refs or a missing ref will not serve a caller well. The
fallback does not indicate the nature of the corruption, so they are
not in a good place to recover from the error. This is analogous to
Git, which attempts to provide sufficient support to recover from
corruption (by ensuring commands like "git branch -D" cope with
corruption) but little else.
In the case of an I/O error, the best we can do is to propagate the
error so that the user sees a dialog and has an opportunity to try
again. As in the corruption case, the fallback behavior does not
provide enough information for a caller to rely on the current error
handling, and callers such as EGit already need to be able to handle
runtime exceptions.
To be conservative, keep the existing behavior for the deprecated
Repository#peel method. In this example, the fallback behavior is to
return an unpeeled ref, which is distinguishable from the ref not
existing and should thus at least be possible to debug.
Change-Id: I0eb58eb8c77519df7f50d21d1742016b978e67a3 Signed-off-by: Jonathan Nieder <jrn@google.com>
Thomas Wolf [Mon, 17 Dec 2018 12:02:19 +0000 (13:02 +0100)]
Apache MINA sshd client: less aggressive key file name caching
Don't use the ~/.ssh directory as cache key for the key provider
but the configured paths of the default keys. Otherwise changes
in that list of paths are not picked up.
This is in particular a problem for EGit, where the user can modify
this list of keys interactively in the preferences. Without this
change, Eclipse needs to be restarted to pick up such changes.
Bug: 542845
Change-Id: I63432fb10729a90b3c5e14f13e39bf482aef811b Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Michael Keppler [Thu, 3 Jan 2019 06:57:02 +0000 (07:57 +0100)]
Deprecate RawParseUtils.UTF8-CHARSET
That constant is just a redirection to a java standard constant
meanwhile. It is not referenced anymore in jgit code (and egit is just
removing it). Clients can use the redirection target directly.
Change-Id: I058d013f61da8d7b771c499d8743aafb8faa5ea8 Signed-off-by: Michael Keppler <Michael.Keppler@gmx.de> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
A checkout done directly after cloning (the "initial
checkout") has a different semantic as a default
checkout. That is defined in the documentation of
"git read-tree" [1]. JGit was detecting that it is
doing an initial checkout differently from native
git: jgit used to check that the index is empty
but native git required that the index file does
not exist [2]. Teach JGit to behave like native
git.
Michael Keppler [Mon, 31 Dec 2018 07:31:09 +0000 (08:31 +0100)]
Replace deprecated FirstLine by FirstCommand
and allow package org.eclipse.jgit.http.server to use package
org.eclipse.jgit.internal.transport.parser.
Change-Id: Ief330c3e75a735853d0a5a265a9ff56fb5128b99 Signed-off-by: Michael Keppler <Michael.Keppler@gmx.de> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Jonathan Nieder [Thu, 27 Dec 2018 00:11:07 +0000 (16:11 -0800)]
Move BaseReceivePack#advertisedRefs getter and setter to ReceivePack
Another step toward merging BaseReceivePack into ReceivePack.
Change-Id: If861e28ce512f556e574352fa7d4a0df0984693f Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Jonathan Nieder [Thu, 27 Dec 2018 00:05:34 +0000 (16:05 -0800)]
Move BaseReceivePack#walk getter to ReceivePack
Another step toward merging BaseReceivePack into ReceivePack.
Change-Id: I43cf2e36e2d5b0cd85bf23c81469909c14757b63 Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Jonathan Nieder [Wed, 26 Dec 2018 23:58:46 +0000 (15:58 -0800)]
Move BaseReceivePack#db getter to ReceivePack
Another step toward eliminating BaseReceivePack as a separate API.
Change-Id: If7b7d5c65a043607a2424211adb479fa33a9077b Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Jonathan Nieder [Wed, 26 Dec 2018 23:59:36 +0000 (15:59 -0800)]
Move BaseReceivePack#pushCert getter and setter to ReceivePack
This is a first step toward eliminating the BaseReceivePack API.
Inspired by a larger change by Dan Wang <dwwang@google.com>.
Change-Id: I5c876a67d8db24bf808823d9ea44d991b1ce5277 Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Jonathan Nieder [Thu, 12 Nov 2015 00:10:24 +0000 (16:10 -0800)]
RefDatabase: Introduce findRef synonym for getRef
Using findRef instead of getRef makes it clearer that the caller wants
to search for the ref in the search path, instead of looking for a ref
that exactly matches the input.
This change introduces the new findRef method and deprecates getRef.
It updates Repository#findRef to use the new method, ensuring some
test coverage. Other callers will be updated in followup changes.
A nice side effect of introducing the new findRef method is that it is
final and based on firstExactRef, so implementers can focus on
implementing the latter efficiently and do not have to carefully write
custom path search code respecting SEARCH_PATH.
Change-Id: Id3bb944344a9743705fd1f20193ab679298fa51c Signed-off-by: Jonathan Nieder <jrn@google.com>
Jonathan Nieder [Thu, 27 Dec 2018 04:33:55 +0000 (20:33 -0800)]
RefDirectory: Look up several exact refs in one shot
Override exactRef(String...) and firstExactRef(String...) with
implementations specific to FileRepository.
The specialized implementations are similar to the generic ones from
RefDatabase, but because these use readRef directly instead of
exactRef, they only need to call fireRefsChanged once.
This will allow replacing RefDirectory#getRef with a generic
implementation that uses firstExactRef without hurting performance.
Change-Id: I1be1948bd6121c1a1e8152e201aab97e7fb308bb Signed-off-by: Jonathan Nieder <jrn@google.com>
Jonathan Nieder [Wed, 11 Nov 2015 23:50:39 +0000 (15:50 -0800)]
RefDirectory: Do not use search path to find additional refs
Psuedorefs like FETCH_HEAD and MERGE_HEAD are supposed to be directly
under the .git directory, not in other locations in the SEARCH_PATH
like refs/ and refs/heads/. Use exactRef to access them.
Change-Id: Iab8ac47008822fa78fc0691e239e518c34d7a98e Signed-off-by: Jonathan Nieder <jrn@google.com>
Jonathan Nieder [Wed, 11 Nov 2015 01:03:06 +0000 (17:03 -0800)]
RefDirectory: Fire RefsChangedEvent on error, too
getRef and exactRef can produce recoverable exceptions --- for
example, a corrupt loose ref that cannot be parsed. If readRef was
called and updated looseRefs in the process, RefsChangedEvent should
still be fired.
Noticed while improving the implementation of getRef. This commit
only affects exactRef and getRef. Other methods might be similarly
skipping firing RefsChangedEvent in their error handling code, and
this change does not fix them.
Change-Id: I0f460f6c8d9a585ad8453a4a47c1c77e24a1fb83 Signed-off-by: Jonathan Nieder <jrn@google.com>
Ivan Frade [Thu, 20 Dec 2018 00:43:56 +0000 (16:43 -0800)]
RefDatabase/Ref: Add versioning to reference database
In DFS implementations the reference table can fall out of sync, but
it is not possible to check this situation in the current API.
Add a property to the Refs indicating the order of its updates. This
version is set only by RefDatabase implementations that support
versioning (e.g reftable based).
Caller is responsible to check if the reference db creates versioned
refs before accessing getUpdateIndex(). E.g:
Ref ref = refdb.exactRef(...);
if (refdb.hasVersioning()) {
ref.getUpdateIndex();
}
Change-Id: I0d5ec8e8df47c730301b2e12851a6bf3dac9d120 Signed-off-by: Ivan Frade <ifrade@google.com>
Masaya Suzuki [Mon, 24 Dec 2018 00:43:54 +0000 (16:43 -0800)]
Skip some tests when the runtime cannot handle Unicode file paths
When executing a test with LANG environment variable set to non UTF-8
encoding, it seems that JRE cannot handle Unicode file paths. This
happens when this test is executed in Bazel as it unsets LANG
(https://docs.bazel.build/versions/master/test-encyclopedia.html#initial-conditions).
Skip the test if the runtime cannot handle Unicode file paths.
David Pursehouse [Tue, 18 Dec 2018 10:53:26 +0000 (19:53 +0900)]
BasePackConnection: Check for expected length of ref advertisement
When a server sends a ref advertisement using protocol v2 it contains
lines other than ref names and sha1s. Attempting to get the sha1 out
of such a line using the substring method can result in a SIOOB error
when it doesn't actually contain the sha1 and ref name.
Add a check that the line is of the expected length, and subsequently
that the extracted object id is valid, and if not throw an exception.
Change-Id: Id92fe66ff8b6deb2cf987d81929f8d0602c399f4 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
David Pursehouse [Tue, 18 Dec 2018 06:44:44 +0000 (15:44 +0900)]
TransferConfig: Make constructors public
UploadPack has a setTransferConfig method which allows to set the
transfer config, however since the constructors of TransferConfig
have the default package visibility it is not possible for any
application using UploadPack, for example Gerrit, to actually set
a transfer config.
Make the constructors public. This is consistent with the public
constructors for example on PackConfig.
Change-Id: I07080255838421871403b2b2bcc294aa8f621c57 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
David Pursehouse [Tue, 18 Dec 2018 02:44:00 +0000 (11:44 +0900)]
UploadPack: Prevent setting null protocolV2Hook
The setProtocolV2Hook sets the protocolV2Hook to whatever value is
passed, which could be null, but the invocations of protocolV2Hook's
methods are not guarded by null-checks.
Annotate the parameter as @Nullable and set ProtocolV2Hook.DEFAULT
when null is passed. This makes the implementation consistent with
other similar methods that set a hook or filter with possible null
value.
Change-Id: I70919a3248d4c2658783941a37c47e437cff0baa Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Michael Keppler [Sat, 15 Dec 2018 15:17:31 +0000 (16:17 +0100)]
Upgrade Tycho to 1.3
With the upcoming Eclipse release 2018-12 a new version of Tycho has
been released. Upgrade the Tycho related build steps to the new version
in the Maven build.
Change-Id: Ifff186a9f97ed9faf70f15b20396724b0c9e801c Signed-off-by: Michael Keppler <Michael.Keppler@gmx.de>
Matthias Sohn [Sat, 8 Dec 2018 23:45:43 +0000 (00:45 +0100)]
Merge branch 'stable-5.2'
* committer:
Update list of committers
Add new ssh bundles to Maven central scripts
Update maven site reports
Prepare 5.2.1-SNAPSHOT builds
JGit v5.2.0.201812061821-r
Update Orbit to R20181128170323 for 2018-12
Change-Id: I97c6ce5f0c963bfab4d45462f555563d9c5bbe8a Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Sat, 8 Dec 2018 23:12:11 +0000 (00:12 +0100)]
Update maven site reports
- update name of reports which changed name in
maven-project-info-reports-plugin 3.0.0
- add dependency-covergence report
- add dependency-management report
- add index report
- add summary report
Change-Id: I6d406ecd9e082d96b2bd250704d5ca18e7c8f735 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Mon, 3 Dec 2018 01:32:05 +0000 (02:32 +0100)]
Merge branch 'stable-5.2'
* stable-5.2:
Format lib/BUILD with buildifier
Update Orbit to S20181128170323 for 2018-12 RC1
Include id_ed25519 in the known default identity files
Apache MINA sshd client: enable support for ed25519 keys
Prepare 5.2.0-SNAPSHOT builds
Set git environment variables for hooks
JGit v5.2.0.201811281532-m3
Change-Id: If96adcbf35ccf8d9f4da0f5d97491f502f5a72a9 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Thomas Wolf [Fri, 24 Aug 2018 23:58:09 +0000 (01:58 +0200)]
Fix IndexDiffs for git links
After cloning a repo with a submodule, non-recursively, JGit would
encounter in its TreeWalk in IndexDiff:
* first, a missing gitlink (in index & HEAD, not in working tree)
* second, the untracked folder (not in index and head, in working tree)
As a result, it would report the submodule as missing. Canonical git
reports a clean workspace.
The root cause of this is that the path of a gitlink "x" did
not compare equal to the path of a tree "x" in JGit.
Correct Paths.compare() to account for that. If two paths are otherwise
equal, then let gitlinks match both trees and files. Matching trees
solves the bug. Matching files is necessary to handle the case where
the gitlink directory was replaced by a file; see the new test case
IndexDiffSubmoduleTest.testSubmoduleReplacedByFile(). Comparisons of
unequal paths are left untouched, so the sort order is unchanged.
After the fix, another bug(?) in WorkingTreeIterator became apparent:
with core.dirNoGitLinks = true, it was no longer possible to overwrite
a gitlink in the index. This is now fixed in WorkingTreeIterator.
Add new test cases for the bug itself and for some related cases
(submodule directory deleted or replaced by a file) in
IndexDiffSubmoduleTest. Add a test for missing files in IndexDiffTest,
and adapt the PathsTest to test matching gitlinks.
Bug: 467631
Change-Id: I0549d10d46b1858e5eec3cc15095aa9f1d5f5280 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Matthias Sohn [Sun, 25 Nov 2018 22:15:11 +0000 (23:15 +0100)]
Fix DescribeCommand with multiple match options
when multiple match options are given in git describe the result must
not depend on the order of the match options. JGit wrongly picked the
first match using the match options in the order they were defined. Fix
this by concatenating the streams of matching tags for all match options
and then choosing the first match on the concatenated stream sorted in
tie break order.
See https://git-scm.com/docs/git-describe#git-describe---matchltpatterngt
Change-Id: Id01433d35fa16fb4c30526605bee041ac1d954b2 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Tue, 20 Nov 2018 00:00:30 +0000 (16:00 -0800)]
Add a method to get all values of HTTP header defined as list
According to RFC 2616 [1] header field names are case insensitive.
Header fields defined as a comma separated list can have multiple header
fields with the same field name. Add a method to HttpConnection which
retrieves all values with a given header field name with the field name
compared case insensitive.
Thomas Wolf [Tue, 20 Nov 2018 11:24:38 +0000 (12:24 +0100)]
Undo treating blobs with NULs as a single line
This partially reverts commit a551b646: revert the changes in
RawParseUtils.lineMap(). Forcing all blobs containing a NUL byte
as a single line causes blame to produce useless results as soon
as it hits any version containing a NUL byte.
Doing binary detection at this level also has the problem that the
user cannot control it. Not by setting the text attribute nor in any
other way.
This came up in bug 541036, where a Java source inadvertently
contained NUL bytes in strings. Even fixing this by using escapes
"\000" will not fix JGit's blame for this file because the past
versions will still contain the NUL byte.
Native git can blame that file from bug 541036 fine.
Added new tests verifying that blaming a text file containing a NUL
byte produces sensible results.
Bug: 541036
Change-Id: I8991bec88e9827cc096868c6026ea1890b6d0d32 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Matthias Sohn [Wed, 21 Nov 2018 14:14:35 +0000 (15:14 +0100)]
Update mockito to 2.13.0
Update mockito and its dependencies to latest version available in
Orbit:
- mockito 2.13.0.v20180426-1843
- net.bytebuddy.byte-buddy 1.7.9.v20180420-1519
- net.bytebuddy.byte-buddy-agent 1.7.9.v20180420-1519
- org.objenesis 2.6.0.v20180420-1519
Masaya Suzuki [Sun, 11 Nov 2018 20:31:22 +0000 (12:31 -0800)]
Revert C Git 1.7.5 bug workaround
This reverts the workaround introduced by 1c6c73c5a9b8dd700be45d658f165a464265dba7, which is a patch for dealing
with a buggy C Git client v1.7.5 in 2012. We'll stop supporting very old
C Git clients.
Thomas Wolf [Sat, 17 Nov 2018 17:44:27 +0000 (18:44 +0100)]
Apache MINA sshd client: don't leak HostConfigEntry
ProxyDataFactory had a parameter of type HostConfigEntry, but actually
it wasn't used anywhere. Remove it -- it was the last leaked type from
Apache MINA sshd.
Also use the logger provided by upstream SshClient instead of creating
a new Logger.
Bug: 520927
Change-Id: Iaa78bbb998a5e574fa091664b75c48a3b9cfb897 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Fri, 16 Nov 2018 16:00:25 +0000 (17:00 +0100)]
Apache MINA sshd client: test & fix password authentication
Add tests for password and keyboard-interactive authentication.
Implement password authentication; the default provided by sshd
is non-interactive, which is not useful for JGit.
Make sure the CredentialsProvider gets reset on successive password
retrieval attempts. Otherwise it might always return the same non-
accepted password from a secure storage. (That one was discovered
by actually trying this via EGit; the JGit tests don't catch this.)
Change the default order of authentication mechanisms to prefer
password over keyboard-interactive. This is a mitigation for upstream
bug SSHD-866.[1]
Thomas Wolf [Thu, 8 Nov 2018 15:58:52 +0000 (16:58 +0100)]
Move SshTestGitServer to new bundle org.eclipse.jgit.junit.ssh
Create the bundle and move the SshTestGitServer there. Verified that
the Eclipse build still works and ran JSchSshTest and ApacheSshTest as
junit tests inside Eclipse.
Update maven build and features to account for that. Verified by
running full maven build including packaging.
Update bazel build files to account for that. Verified by a
clean-slate bazel build :all, followed by running the JSchSshTest
and the ApacheSshTest via bazel.
Change-Id: Ia084942f4425b454529de148e00417e7da786a90 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Sun, 21 Oct 2018 17:44:34 +0000 (19:44 +0200)]
Apache MINA sshd client: proxy support
This is not about the ssh config ProxyCommand but about programmatic
support for HTTP and SOCKS5 proxies. Eclipse allows the user to
specify such proxies, and JSch at least contains code to connect
through proxies. So our Apache MINA sshd client also should be able
to do this.
Add interfaces and provide two implementations for HTTP and SOCKS5
proxies. Adapt the core code to be able to deal with proxy connections
at all. The built-in client-side support for this in sshd 2.0.0 is
woefully inadequate.
Tested manually by running proxies and then fetching various real-
world repositories via these proxies from different servers. Proxies
tested: ssh -D (SOCKS, anonymous), tinyproxy (HTTP, anonymous), and
3proxy (SOCKS & HTTP, username-password authentication). The GSS-API
authentication is untested since I have no Kerberos setup.
Bug: 520927
Change-Id: I1a5c34687d439b3ef8373c5d58e24004f93e63ae Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Thu, 15 Nov 2018 15:33:04 +0000 (16:33 +0100)]
Apache MINA sshd client: don't leak upstream classes and interfaces
We will get an API evolution problem if we expose as API classes and
interfaces that derive from upstream classes or interfaces. Upstream
interfaces also evolve quite erratically and evolution doesn't seem
to follow semantic versioning.
Introduce a new KeyPasswordProvider interface so that we don't have
to depend on the upstream FilePasswordProvider in our API. (We do
need _some_ abstraction for getting passwords for encrypted keys in
the API; EGit will need to provide its own implementation.)
Move some other upstream dependencies (HostConfigEntry, and various
previously protected methods in SshdSessionFactory) out of the API:
classes moved to internal space, and methods made private.
The only dependencies on upstream interfaces are thus in a few method
parameter types. Those cannot be avoided, but should also not pose
problems.
Bug: 520927
Change-Id: Idc9c6b0f237f29f46343c0fe15179242f2007bec Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Ivan Frade [Fri, 16 Nov 2018 05:27:20 +0000 (21:27 -0800)]
Remove unnecessary modifiers from interfaces
From Oracle's "Defining an interface":
"All abstract, default, and static methods in an interface are
implicitly public, so you can omit the public modifier."
(Without any modifier, the interface methods are also abstract, so we
omit also the "abstract")
"In addition, an interface can contain constant declarations. All
constant values defined in an interface are implicitly public, static,
and final. Once again, you can omit these modifiers."
This makes the code more consistent. Now all interfaces under
org.eclipse.jgit follow the guidelines.
Change-Id: I4fe6deb111899ec1b4318ab5a6050f3851fa1fd3 Signed-off-by: Ivan Frade <ifrade@google.com>
projects / jgit.git / log