Matthias Sohn [Thu, 17 Jan 2019 23:15:32 +0000 (00:15 +0100)]
pgm: Handle exceptions in Config command
This avoids we show a stacktrace on the console by default when this
type of exception is thrown during the run method is executed. Also
externalize error message.
Change-Id: I909dc77385a672d8298053b12683c0cbbf9f2aa2 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
This also includes a change to generating the jgit CLI jar. Shading is
no longer possible because it breaks the signature of BouncyCastle.
Instead, the Spring Boot Loader Maven plug-in is now used to generate an
executable jar.
Bug: 382212
Change-Id: I35ee3d4b06d9d479475ab2e51b29bed49661bbdc Also-by: Gunnar Wagenknecht <gunnar@wagenknecht.org> Signed-off-by: Gunnar Wagenknecht <gunnar@wagenknecht.org> Signed-off-by: Medha Bhargav Prabhala <mprabhala@salesforce.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
This change introduces the concept of a GpgSigner which will sign
commits. The GpgSigner will be of a specific implementation (eg.,
Bouncycastle or OpenPgP executable). The actual implementation is not
part of this change.
Matthias Sohn [Wed, 16 Jan 2019 00:13:38 +0000 (01:13 +0100)]
pgm: propagate generic exceptions when throwing Die exception
If a command is stopped by throwing a Die exception since a generic
exception was caught this exception contains additional information
which is not available in the error message. Hence chain it to the Die
exception in order to allow printing a complete stacktrace including
causes to the error stream of the JGit command line by adding option
--show-stack-trace, e.g.
$ jgit --show-stack-trace commit -m "test"
Change-Id: I7ad8c56b997ab60d2bd08f4e7b4d0b80c84021c3 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Jonathan Nieder [Mon, 14 Jan 2019 18:48:06 +0000 (10:48 -0800)]
dfs: Remove synchronization in BlockBasedFile#LazyChannel
As explained in 'The "Double-Checked Locking is Broken"
Declaration'[*], Java's memory model does not support double-checked
locking:
class LazyReadableChannel {
private ReachableChannel rc = null;
public ReadableChannel get() {
if (rc == null) {
synchronized (this) {
if (rc == null) {
rc = new ReadableChannel();
}
}
}
return rc;
}
}
With JDK 5 and newer, there is a formal memory model that ensures this
works if "rc" is volatile, but it is still not thread-safe without
that.
Fortunately, this ReadableChannelSupplier is never passed between
threads, so it does not need to be thread-safe. Simplify by removing
the synchronization.
Currently, unit tests need to either close the Repository underlying a
TestRepository manually, or not close it at all. Both are error prone.
The TestRepository holds a reference to 4 AutoCloseable objects:
Repository, ObjectInserter, Git, and RevWalk. The last two can escape
the TestRepository scope, so they are not closed when TestRepository is
closed.
Change-Id: I4461bb9104d517bd6bef09c38507c7c2ef5c31d4 Signed-off-by: Jackson Toeniskoetter <jackdt@google.com>
Minh Thai [Tue, 18 Dec 2018 23:06:59 +0000 (15:06 -0800)]
DfsBlockCache: Consolidate where ReadableChannel is opened
Opening a readable channel can be expensive and the number of channels
can be limited in DFS. Ensure that caller of
BlockBasedFile.readOneBlock() is responsible for opening and closing
the file, and that the ReadableChannel is reused in the request. As a side
effect, this makes the code easier to read, with better use of
try-with-resources.
The downside is that this means a readable channel is always opened, even
when the entire pack is already available for copying from cache. This
should be an acceptable cost: it's rare enough not to overload the server
and from a client latency perspective, the latency cost is in the noise
relative to the data transfer cost involved in a clone. If this turns out
to be a problem in practice, we can reintroduce that optimization in a
followup change.
Change-Id: I340428ee4bacd2dce019d5616ef12339a0c85f0b Signed-off-by: Minh Thai <mthai@google.com> Signed-off-by: Jonathan Nieder <jrn@google.com>
Minh Thai [Sat, 22 Dec 2018 00:27:16 +0000 (16:27 -0800)]
DfsBlockCache to lock while loading object references
We see the same index being loaded by multiple threads. Each is
hundreds of MB and takes several seconds to load, causing server to
run out of memory. This change introduces a lock to avoid these
duplicate works. It uses a new set of locks similar in implementation
to the loadLocks for getOrLoad of blocks. The locks are kept separate
to prevent long-running index loading from blocking out fast block
loading. The cache instance can be configured with a consumer to
monitor the wait time of the new locks.
Fix "jgit checkout -f" to overwrite dirty worktree files
CheckoutCommand had a setForce() method. But this didn't correspond
to native git's 'git checkout -f' option. Deprecate the old setForce()
method and move its implementation to a new method setForceRefUpdate()
and use it to implement the -B option in the CLI class Checkout.
Add a setForced() method and use it to fix the associated '-f' option of
the CLI Checkout class to behave like native git's 'git checkout -f'
which overwrites dirty worktree files during checkout.
This is still not fully matching native git's behavior: updating
additionally dirty index entries is not done yet.
Bug: 530771
Change-Id: I776b78eb623b6ea0aca42f681788f2e4b1667f15 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Jonathan Nieder [Sat, 5 Jan 2019 00:02:07 +0000 (16:02 -0800)]
Don't swallow IOException
Swallowing intermittent errors and trying to recover from them
makes JGit's behavior hard to predict and difficult to debug.
Propagate the errors instead. This doesn't violate JGit's usual
backward compatibility promise for clients because in these
contexts an IOException indicates either repository corruption or
a true I/O error. Let's consider these cases one at a time.
In the case of repository corruption, falling back e.g. to an empty
set of refs or a missing ref will not serve a caller well. The
fallback does not indicate the nature of the corruption, so they are
not in a good place to recover from the error. This is analogous to
Git, which attempts to provide sufficient support to recover from
corruption (by ensuring commands like "git branch -D" cope with
corruption) but little else.
In the case of an I/O error, the best we can do is to propagate the
error so that the user sees a dialog and has an opportunity to try
again. As in the corruption case, the fallback behavior does not
provide enough information for a caller to rely on the current error
handling, and callers such as EGit already need to be able to handle
runtime exceptions.
To be conservative, keep the existing behavior for the deprecated
Repository#peel method. In this example, the fallback behavior is to
return an unpeeled ref, which is distinguishable from the ref not
existing and should thus at least be possible to debug.
Change-Id: I0eb58eb8c77519df7f50d21d1742016b978e67a3 Signed-off-by: Jonathan Nieder <jrn@google.com>
Thomas Wolf [Mon, 17 Dec 2018 12:02:19 +0000 (13:02 +0100)]
Apache MINA sshd client: less aggressive key file name caching
Don't use the ~/.ssh directory as cache key for the key provider
but the configured paths of the default keys. Otherwise changes
in that list of paths are not picked up.
This is in particular a problem for EGit, where the user can modify
this list of keys interactively in the preferences. Without this
change, Eclipse needs to be restarted to pick up such changes.
Bug: 542845
Change-Id: I63432fb10729a90b3c5e14f13e39bf482aef811b Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Michael Keppler [Thu, 3 Jan 2019 06:57:02 +0000 (07:57 +0100)]
Deprecate RawParseUtils.UTF8-CHARSET
That constant is just a redirection to a java standard constant
meanwhile. It is not referenced anymore in jgit code (and egit is just
removing it). Clients can use the redirection target directly.
Change-Id: I058d013f61da8d7b771c499d8743aafb8faa5ea8 Signed-off-by: Michael Keppler <Michael.Keppler@gmx.de> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
A checkout done directly after cloning (the "initial
checkout") has a different semantic as a default
checkout. That is defined in the documentation of
"git read-tree" [1]. JGit was detecting that it is
doing an initial checkout differently from native
git: jgit used to check that the index is empty
but native git required that the index file does
not exist [2]. Teach JGit to behave like native
git.
Michael Keppler [Mon, 31 Dec 2018 07:31:09 +0000 (08:31 +0100)]
Replace deprecated FirstLine by FirstCommand
and allow package org.eclipse.jgit.http.server to use package
org.eclipse.jgit.internal.transport.parser.
Change-Id: Ief330c3e75a735853d0a5a265a9ff56fb5128b99 Signed-off-by: Michael Keppler <Michael.Keppler@gmx.de> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Jonathan Nieder [Thu, 27 Dec 2018 00:11:07 +0000 (16:11 -0800)]
Move BaseReceivePack#advertisedRefs getter and setter to ReceivePack
Another step toward merging BaseReceivePack into ReceivePack.
Change-Id: If861e28ce512f556e574352fa7d4a0df0984693f Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Jonathan Nieder [Thu, 27 Dec 2018 00:05:34 +0000 (16:05 -0800)]
Move BaseReceivePack#walk getter to ReceivePack
Another step toward merging BaseReceivePack into ReceivePack.
Change-Id: I43cf2e36e2d5b0cd85bf23c81469909c14757b63 Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Jonathan Nieder [Wed, 26 Dec 2018 23:58:46 +0000 (15:58 -0800)]
Move BaseReceivePack#db getter to ReceivePack
Another step toward eliminating BaseReceivePack as a separate API.
Change-Id: If7b7d5c65a043607a2424211adb479fa33a9077b Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Jonathan Nieder [Wed, 26 Dec 2018 23:59:36 +0000 (15:59 -0800)]
Move BaseReceivePack#pushCert getter and setter to ReceivePack
This is a first step toward eliminating the BaseReceivePack API.
Inspired by a larger change by Dan Wang <dwwang@google.com>.
Change-Id: I5c876a67d8db24bf808823d9ea44d991b1ce5277 Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Jonathan Nieder [Thu, 12 Nov 2015 00:10:24 +0000 (16:10 -0800)]
RefDatabase: Introduce findRef synonym for getRef
Using findRef instead of getRef makes it clearer that the caller wants
to search for the ref in the search path, instead of looking for a ref
that exactly matches the input.
This change introduces the new findRef method and deprecates getRef.
It updates Repository#findRef to use the new method, ensuring some
test coverage. Other callers will be updated in followup changes.
A nice side effect of introducing the new findRef method is that it is
final and based on firstExactRef, so implementers can focus on
implementing the latter efficiently and do not have to carefully write
custom path search code respecting SEARCH_PATH.
Change-Id: Id3bb944344a9743705fd1f20193ab679298fa51c Signed-off-by: Jonathan Nieder <jrn@google.com>
Jonathan Nieder [Thu, 27 Dec 2018 04:33:55 +0000 (20:33 -0800)]
RefDirectory: Look up several exact refs in one shot
Override exactRef(String...) and firstExactRef(String...) with
implementations specific to FileRepository.
The specialized implementations are similar to the generic ones from
RefDatabase, but because these use readRef directly instead of
exactRef, they only need to call fireRefsChanged once.
This will allow replacing RefDirectory#getRef with a generic
implementation that uses firstExactRef without hurting performance.
Change-Id: I1be1948bd6121c1a1e8152e201aab97e7fb308bb Signed-off-by: Jonathan Nieder <jrn@google.com>
Jonathan Nieder [Wed, 11 Nov 2015 23:50:39 +0000 (15:50 -0800)]
RefDirectory: Do not use search path to find additional refs
Psuedorefs like FETCH_HEAD and MERGE_HEAD are supposed to be directly
under the .git directory, not in other locations in the SEARCH_PATH
like refs/ and refs/heads/. Use exactRef to access them.
Change-Id: Iab8ac47008822fa78fc0691e239e518c34d7a98e Signed-off-by: Jonathan Nieder <jrn@google.com>
Jonathan Nieder [Wed, 11 Nov 2015 01:03:06 +0000 (17:03 -0800)]
RefDirectory: Fire RefsChangedEvent on error, too
getRef and exactRef can produce recoverable exceptions --- for
example, a corrupt loose ref that cannot be parsed. If readRef was
called and updated looseRefs in the process, RefsChangedEvent should
still be fired.
Noticed while improving the implementation of getRef. This commit
only affects exactRef and getRef. Other methods might be similarly
skipping firing RefsChangedEvent in their error handling code, and
this change does not fix them.
Change-Id: I0f460f6c8d9a585ad8453a4a47c1c77e24a1fb83 Signed-off-by: Jonathan Nieder <jrn@google.com>
Jonathan Nieder [Sat, 22 Dec 2018 21:53:10 +0000 (13:53 -0800)]
UploadPack: Filter refs used for deepen-not resolution
Clients can use --shallow-exclude to obtain information about what
commits are reachable from refs they are not supposed to be able to
see. Plug the hole by allowing the AdvertiseRefsHook and RefFilter to
take effect here, too.
Change-Id: If2b8e95344fa49e10a6a202144318b60d002490e Signed-off-by: Jonathan Nieder <jrn@google.com>
Jonathan Nieder [Wed, 26 Dec 2018 20:40:28 +0000 (12:40 -0800)]
UploadPack: Avoid calling AdvertiseRefsHook twice
The AdvertiseRefsHook can be called twice if the following conditions
hold:
1. This AdvertiseRefsHook doesn't set this.refs.
2. getAdvertisedOrDefaultRefs is called after getFilteredRefs.
For example, this can happen when fetchV2 is called after lsRefsV2
when using a stateful bidirectional transport.
The second call does not accomplish anything useful. Guard it with
'if (!advertiseRefsHookCalled)' to avoid wasted work.
Reported-by: Jonathan Tan <jonathantanmy@google.com>
Change-Id: Ib746582e4ef645b767a5b3fb969596df99ac2ab5 Signed-off-by: Jonathan Nieder <jrn@google.com>
Ivan Frade [Thu, 20 Dec 2018 00:43:56 +0000 (16:43 -0800)]
RefDatabase/Ref: Add versioning to reference database
In DFS implementations the reference table can fall out of sync, but
it is not possible to check this situation in the current API.
Add a property to the Refs indicating the order of its updates. This
version is set only by RefDatabase implementations that support
versioning (e.g reftable based).
Caller is responsible to check if the reference db creates versioned
refs before accessing getUpdateIndex(). E.g:
Ref ref = refdb.exactRef(...);
if (refdb.hasVersioning()) {
ref.getUpdateIndex();
}
Change-Id: I0d5ec8e8df47c730301b2e12851a6bf3dac9d120 Signed-off-by: Ivan Frade <ifrade@google.com>
Jonathan Nieder [Wed, 19 Dec 2018 05:00:42 +0000 (21:00 -0800)]
UploadPack: Filter refs used for want-ref resolution
In the longer term, we can add support for this to the
RequestValidator interface. In the short term, this is a minimal
band-aid to ensure any refs the client requests are visible to the
client.
Change-Id: I0683c7a00e707cf97eef6c6bb782671d0a550ffe Reported-by: Ivan Frade <ifrade@google.com> Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Jonathan Nieder [Wed, 19 Dec 2018 03:44:39 +0000 (19:44 -0800)]
UploadPack: Defer want-ref resolution to after parsing
ProtocolV2Parser explains:
// TODO(ifrade): This validation should be done after the
// protocol parsing. It is not a protocol problem asking for an
// unexisting ref and we wouldn't need the ref database here.
Do so. This way all ref database accesses are in one place, in the
UploadPack class.
No user-visible change intended --- this is just to make the code
easier to manipulate.
Change-Id: I68e87dff7b9a63ccc169bd0836e8e8baaf5d1048 Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Tue, 25 Dec 2018 22:58:53 +0000 (23:58 +0100)]
Merge branch 'stable-5.0' into stable-5.1
* stable-5.0:
Call AdvertiseRefsHook for protocol v2
Prepare 4.11.7-SNAPSHOT builds
JGit v4.11.6.201812241910-r
Prepare 4.9.9-SNAPSHOT builds
JGit v4.9.8.201812241815-r
UploadPack: Test filtering by AdvertiseRefsHook in stateless transports
Prepare 4.7.8-SNAPSHOT builds
JGit v4.7.7.201812240805-r
Fix feature versions imported by feature org.eclipse.jgit.pgm
Prepare 4.5.6-SNAPSHOT builds
JGit v4.5.5.201812240535-r
Call AdvertiseRefsHook before validating wants
Change-Id: Icdc212bf5be2485d0f8028acf6c62fb8531d0e3c Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Masaya Suzuki [Tue, 18 Dec 2018 17:20:54 +0000 (09:20 -0800)]
Call AdvertiseRefsHook for protocol v2
AdvertiseRefsHook is used to limit the visibility of the refs in Gerrit.
If this hook is not called, then all refs are treated as visible.
In protocol v2, the hook is not called, causing the server to advertise
all refs. This bug was introduced in v5.0.0.201805221745-rc1~1^2~9
(Execute AdvertiseRefsHook only for protocol v0 and v1, 2018-05-14).
Even before then, the hook was not called in requests after the
capability advertisement, so in transports like HTTP that do not retain
state between round-trips, the server would advertise all refs in
response to an ls-refs (ls-remote) request.
Fix both cases by using getAdvertisedOrDefaultRefs to retrieve the
advertised refs in lsRefs, ensuring the hook is called in all cases that
use its result.
[jn: backported to stable-5.0; split out from a larger patch that also
fixes protocol v0; avoided filtering this.refs by ref prefix]
Change-Id: I64bce0e72d15b90baccc235c067e57b6af21b55f Signed-off-by: Masaya Suzuki <masayasuzuki@google.com> Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Tue, 25 Dec 2018 22:27:13 +0000 (23:27 +0100)]
Merge branch 'stable-4.11' into stable-5.0
* stable-4.11:
Prepare 4.11.7-SNAPSHOT builds
JGit v4.11.6.201812241910-r
Prepare 4.9.9-SNAPSHOT builds
JGit v4.9.8.201812241815-r
UploadPack: Test filtering by AdvertiseRefsHook in stateless transports
Prepare 4.7.8-SNAPSHOT builds
JGit v4.7.7.201812240805-r
Fix feature versions imported by feature org.eclipse.jgit.pgm
Prepare 4.5.6-SNAPSHOT builds
JGit v4.5.5.201812240535-r
Call AdvertiseRefsHook before validating wants
Change-Id: Ie81284ca6d580b0712c49eec610393d0c0c50203 Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Mon, 24 Dec 2018 23:49:46 +0000 (00:49 +0100)]
Merge branch 'stable-4.10' into stable-4.11
* stable-4.10:
Prepare 4.9.9-SNAPSHOT builds
JGit v4.9.8.201812241815-r
UploadPack: Test filtering by AdvertiseRefsHook in stateless transports
Prepare 4.7.8-SNAPSHOT builds
JGit v4.7.7.201812240805-r
Fix feature versions imported by feature org.eclipse.jgit.pgm
Prepare 4.5.6-SNAPSHOT builds
JGit v4.5.5.201812240535-r
Call AdvertiseRefsHook before validating wants
Change-Id: I937e9a4547fc10e4de7c887163022d1ab0322d64 Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Mon, 24 Dec 2018 23:33:44 +0000 (00:33 +0100)]
Merge branch 'stable-4.9' into stable-4.10
* stable-4.9:
Prepare 4.9.9-SNAPSHOT builds
JGit v4.9.8.201812241815-r
UploadPack: Test filtering by AdvertiseRefsHook in stateless transports
Prepare 4.7.8-SNAPSHOT builds
JGit v4.7.7.201812240805-r
Fix feature versions imported by feature org.eclipse.jgit.pgm
Prepare 4.5.6-SNAPSHOT builds
JGit v4.5.5.201812240535-r
Call AdvertiseRefsHook before validating wants
Change-Id: I2e499f34b1c481af794fa9325b0dfebaccdf3cb0 Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Masaya Suzuki [Tue, 18 Dec 2018 17:20:54 +0000 (09:20 -0800)]
UploadPack: Test filtering by AdvertiseRefsHook in stateless transports
AdvertiseRefsHook is used to limit the visibility of the refs in Gerrit.
If this hook is not called, then all refs are treated as visible,
causing the server to serve commits reachable from branches the client
should not be able to access, if asked to via a request naming a guessed
object id.
Until 3a529361a76e8267467071e0b13ebb36b97d8fb2 (Call AdvertiseRefsHook
before validating wants, 2018-12-18), UploadPack would invoke this hook
at ref advertisement time but not during negotiation and when serving a
pack file. Add a test to avoid regressing. Stateful bidirectional
transports were not affected, so the test uses HTTP.
[jn: split out when backporting the fix to stable-4.5. The test passes
as long as v4.9.0.201710071750-r~169 (fetch: Accept any SHA-1 on lhs of
refspec, 2017-06-04) is cherry picked along with it.]
Change-Id: I8c017107336adc7cb4c826985779676bf043e648 Signed-off-by: Masaya Suzuki <masayasuzuki@google.com> Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Change-Id: Ib44e314a68bca2349b45f4937257aa1298c8d74b Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Change-Id: I68a21067705b580b40840f8039001ff1e5273c15 Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Mon, 24 Dec 2018 12:25:31 +0000 (13:25 +0100)]
Merge branch 'stable-4.6' into stable-4.7
* stable-4.6:
Fix feature versions imported by feature org.eclipse.jgit.pgm
Prepare 4.5.6-SNAPSHOT builds
JGit v4.5.5.201812240535-r
Call AdvertiseRefsHook before validating wants
Change-Id: If637694f80dbd1e774d60c672fe78a6500650bb8 Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Change-Id: I0fd67ddd9c4966c20d82cdfe78b2f9d4898b4665 Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
projects / jgit.git / log