Matthias Sohn [Mon, 16 Dec 2019 10:20:12 +0000 (11:20 +0100)]
Add resource preferences for top level jgit project
Define resource preferences for the root directory of the jgit
repository imported as an Eclipse project. This allows to work on files
in the root folder inside Eclipse workspace.
Change-Id: I1885841fd612a646b0b9a31f001ced5ca112e90d Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Sat, 14 Dec 2019 21:45:09 +0000 (22:45 +0100)]
Set config "extensions" option when converting ref storage format
When converting to reftable format the option extensions.refStorage must
be set to "reftable" [1]. When converting back to refdir format this
config option needs to be removed.
Introduce constants for refStorage config options, also for the
"reftree" format.
Carsten Hammer [Tue, 21 May 2019 15:04:18 +0000 (17:04 +0200)]
Replace chain of if statements with switch
and switch over strings where possible. Sometimes if statements are
chained and form a series of comparisons against constants. Using switch
statements improves readability.
Tim Neumann [Thu, 28 Nov 2019 14:56:50 +0000 (15:56 +0100)]
Add ability to redirect stderr from git hooks
This will change the behavior in the CLI to resemble that of C-Git more
closely by printing the stderr of the hooks to the CLI stderr
independently of the exit code of the hook.
This is also useful for the corresponding EGIT-Change, which will add
the ability to show the hook output in eclipse.
With this also the stderr can be shown even if the exit code is 0.
Bug: 553471
Change-Id: Ie7bc503fe39e270e9b93dd1108b5879f02a12b4c Signed-off-by: Tim Neumann <Tim.Neumann@advantest.com>
John Tipper [Mon, 2 Dec 2019 23:00:19 +0000 (23:00 +0000)]
CLI: Add support for excluding paths from Git log command
Git log supports the exclude pathspec, which allows for excluding paths
from the log command. JGit only supports adding paths to the log
command. See the following StackOverflow question for details:
https://stackoverflow.com/questions/59143934/java-jgit-how-to-get-git-
commits-not-affecting-certain-directories
This commit adds an excludePath() method to the log command. It does not
yet support regex or glob wildcards.
Change-Id: I8cd59950b87850b55a15c7e2ea5470145c9aae28 Signed-off-by: John Tipper <john_tipper@hotmail.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Demetr Starshov [Tue, 8 Oct 2019 23:53:22 +0000 (16:53 -0700)]
ReceivePack: Moves connectivity check to separate class
Move all connectivity check to separate classes. Set default to be
FullConnectivityChecker i.e. checker which will check connectivity
from all advertised refs. Add ability to set other connectivity
checker which can use different approach.
Matthias Sohn [Thu, 5 Dec 2019 00:56:31 +0000 (01:56 +0100)]
Suppress API error for RefDatabase.hasFastTipsWithSha1()
This method was added in stable-5.6 after 5.6 RC1 was tagged. It's shown
as an API error when using 5.6.0 RC1 as the API baseline for master.
This suppression can be removed when 5.6 is released which will serve as
the API baseline for 5.7.
Change-Id: Ic6359fe59ed2d2a9431583b0854ec86bbac76880 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Han-Wen Nienhuys [Mon, 18 Nov 2019 00:29:16 +0000 (16:29 -0800)]
Introduce RefDatabase#hasFastTipsWithSha1
The reftable format supports fast inverse (SHA1 => ref) queries.
If the ref database does not support fast inverse queries, it may be
advantageous to build a complete SHA1 to ref map in advance for
multiple uses. To let applications decide, this function indicates
whether the inverse map is available.
Masaya Suzuki [Fri, 8 Nov 2019 18:12:20 +0000 (10:12 -0800)]
transport: Let users specify an error handler for unpacking a pack file
Currently, there's no way for a user to customize the error handler for
the exception happened during unpacking an incoming pack file. Create an
error handler class and let them specify one.
Masaya Suzuki [Fri, 8 Nov 2019 02:52:17 +0000 (18:52 -0800)]
transport: Consolidate status reporting code
BaseReceivePack#sendStatusReport anyway needs to know
CAPABILITY_REPORT_STATUS. By moving this flag to BaseReceivePack,
simplify the status reporting code.
Masaya Suzuki [Fri, 8 Nov 2019 02:22:17 +0000 (18:22 -0800)]
transport: Add ReceiveCommandErrorHandler
This gives a chance to handle an exception for a user. For example, when
an IOException is thrown while executing
`walk.parseAny(cmd.getNewId())`, it's always handled as
REJECTED_MISSING_OBJECT. However, IOException can mean a Git storage IO
error. By introducing an error handler class, a user can add a custom
error handler for these cases.
Masaya Suzuki [Fri, 8 Nov 2019 01:40:25 +0000 (17:40 -0800)]
transport: Move exception handler to the caller
Move exception handling code to the caller side. This is a preparation
for adding a custom error handler that is similar to UploadPack.
Note that `catch (Throwable t)` will not affect the exception spec since
Java will do an analysis of rethrown exceptions. See
https://docs.oracle.com/javase/8/docs/technotes/guides/language/catch-multiple.html.
Masaya Suzuki [Fri, 8 Nov 2019 22:30:29 +0000 (14:30 -0800)]
UploadPackServlet: Use uploadWithExceptionPropagation
As UploadPackErrorHandler's Javadoc says, UploadPackServlet should have
called uploadWithExceptionPropagation and let UploadPackErrorHandler to
handle the exception. Fix UploadPackServlet.
Masaya Suzuki [Fri, 8 Nov 2019 22:40:19 +0000 (14:40 -0800)]
GitSmartHttpTools: Do not use sideband when sending an error
Unlike ReceivePack, the V0/V1 UploadPack response does not support
sideband except for the packfile parts. By sending an error in a
sideband packet, the JGit client says "Expected ACK/NACK, got: ...".
Use an error packet always. The recent Git clients will understand it
better than out-of-context sideband packets.
Ivan Frade [Tue, 19 Nov 2019 06:45:19 +0000 (22:45 -0800)]
ReceivePack: Open visibility for some methods
This partially reverts Ic6bb5e66. It made private some methods in
ReceivePack that were protected before. At least two of those methods
(#init and #receivePackAndCheckConnectivity) are used by out-of-tree
subclasses for tracing and testing.
Make ReceivePack#init() and ReceivePack#receivePackAndCheckConnectivity
protected again to allow tracing and testing.
Change-Id: I7ff22c091fbfc2d94009c449d58e7c5ac9f4f759 Signed-off-by: Ivan Frade <ifrade@google.com>
Thomas Wolf [Tue, 6 Aug 2019 16:31:35 +0000 (18:31 +0200)]
Make blame work correctly on merge conflicts
When a conflicting file was blamed, JGit would not identify lines
coming from the merge parents. The main cause for this was that
Blame and BlameCommand simply added the first DirCacheEntry found
for a file to its queue of candidates (blobs or commits) to consider.
In case of a conflict this typically is the merge base commit, and
comparing a auto-merged contents against that base would yield
incorrect results.
Such cases have to be handled specially. The candidate to be
considered by the blame must use the working tree contents, but
at the same time behave like a merge commit/candidate with HEAD
and the MERGE_HEADs as parents. Canonical git does something very
similar, see [1].
Implement that and add tests.
I first did this for the JGit pgm Blame command. When I then tried
to do the same in BlameCommand, I noticed that the latter also
included some fancy but incomplete CR-LF handling. In order to
be able to use the new BlameGenerator.prepareHead() also in
BlameCommand this CR-LF handling was also moved into BlameGenerator
and corrected in doing so.
(Just considering the git config settings was not good enough,
CR-LF behavior can also be influenced by .gitattributes, and even
by whether the file in the index has CR-LF. To correctly determine
CR-LF handling for check-in one needs to do a TreeWalk with at
least a FileTreeIterator and a DirCacheIterator.)
Jonathan Nieder [Tue, 26 Nov 2019 01:42:59 +0000 (17:42 -0800)]
Add missing license header to ReftableDatabase
When this class was split from DfsReftableDatabase in commit 2b1e942729617c45d2cb03b7556ab3d63253f64f (reftable: split off generic
code from DFS code, 2019-09-26), we forgot to carry over the license
header.
Change-Id: I8698001dfb69f58784df643ef0185ab2d55e52aa Reported-by: Masaya Suzuki <masayasuzuki@google.com> Signed-off-by: Jonathan Nieder <jrn@google.com>
Ivan Frade [Thu, 14 Nov 2019 21:56:27 +0000 (13:56 -0800)]
BitmappedReachabilityChecker: Use only one bitmap for the whole check
The checker is creating a new bitmap per branch leading to excessive
memory consumption. For the reachability check one bitmap with the
reachability of all branches aggregated is enough.
Build the reachability bitmap with a filter. The filter itself uses it
to emit only commits not reached before and the caller to check what
targets have been reached already.
BitmapCalculator is not required anymore.
Change-Id: Ic5c62f77fe0f188913215b7eaa51d849a9aae6a5 Signed-off-by: Ivan Frade <ifrade@google.com>
Ivan Frade [Tue, 19 Nov 2019 20:08:53 +0000 (12:08 -0800)]
checkNotAdvertisedWants: Be lazy converting Ref to RevCommit
The ref points to an ObjectId that then is translated into a RevCommit.
This translation can be costly and with the incremental reachability
check is probably not needed for most of the elements.
Delay the translation from ObjectId to RevCommit to when it is needed.
Use Streams, that have the laziness built-in, all the way from Ref to
RevCommit.
This should reduce the latency for reachability checks over big sets of
references.
Change-Id: I28693087321b2beff3eaa1f3d2e7840ab0eedc6d Signed-off-by: Ivan Frade <ifrade@google.com>
Ivan Frade [Tue, 19 Nov 2019 18:40:21 +0000 (10:40 -0800)]
ReachabilityChecker: Receive a Stream instead of a Collection
Preparatory change. Converting ObjectIds to RevCommits is potentially
expensive and in the incremental reachability check, it is probably not
required for all elements in the collection.
Pass a Stream to the reachability checker. In the follow up we make
the conversion from ObjectId to RevCommit in the stream (i.e. on
demand). This should reduce the latency of reachability checks over big
sets of references.
Change-Id: I9f310e331de5b0bf8de34143bd7dcd34316d2fba Signed-off-by: Ivan Frade <ifrade@google.com>
Ivan Frade [Tue, 12 Nov 2019 22:33:29 +0000 (14:33 -0800)]
UploadPack: Prioritize references for non-advertised wants checks
UploadPack needs to check if object ids that weren't advertised before
are reachable from the references visible to the user. In the
bitmap-based reachability check, this is done incrementally: checking
against one reference, if anything remaining adding a second and so on.
It is more efficient to check first more common references (e.g. refs/heads/*)
Sort the references for the reachability checker. This should solve the
connectivity earlier and require less bitmap creation and less memory.
Change-Id: I48ac10d71e29fab2d346479802401eaea4aacb5c Signed-off-by: Ivan Frade <ifrade@google.com>
Han-Wen Nienhuys [Mon, 12 Aug 2019 15:33:51 +0000 (17:33 +0200)]
file: implement FileReftableDatabase
Reftable is a binary, block-based storage format for the ref-database.
It provides several advantages over the traditional packed + loose
storage format:
* O(1) write performance, even for deletions and transactions.
* atomic updates to the ref database.
* O(log N) lookup and prefix scans
* free from restrictions imposed by the file system: it is
case-sensitive even on case-insensitive file systems, and has
no inherent limitations for directory/file conflicts
* prefix compression reduces space usage for repetitive ref names,
such as gerrit's refs/changes/xx/xxxxx format.
FileReftableDatabase is based on FileReftableStack, which does
compactions inline. This is simple, and has good median performance,
but every so often it will rewrite the entire ref database.
For testing, a FileReftableTest (mirroring RefUpdateTest) is added to
check for Reftable specific behavior. This must be done separately, as
reflogs have different semantics.
Add a reftable flavor of BatchRefUpdateTest.
Add a FileReftableStackTest to exercise compaction.
Add FileRepository#convertToReftable so existing testdata can be
reused.
Michael Keppler [Sat, 16 Nov 2019 15:49:38 +0000 (16:49 +0100)]
Fix Maven missing version warning
[WARNING] Some problems were encountered while building the effective
model for org.eclipse.jgit:org.eclipse.jgit.benchmarks:jar:5.6.0-SNAPSHOT
[WARNING] 'build.plugins.plugin.version' for
org.apache.maven.plugins:maven-shade-plugin is missing. @ line 131,
column 15
Change-Id: Id25916c460bc4a10d0f797779d24f53292df2b77 Signed-off-by: Michael Keppler <Michael.Keppler@gmx.de>
Michael Keppler [Sat, 16 Nov 2019 15:23:34 +0000 (16:23 +0100)]
Use Java 8 source and target in Maven
Building the jgit aggregator with Maven leads to a compiler level 1.6
for me, thereby failing the build. Multiple profiles specify
source/target level separately, but there seems no global setting.
Adding the global properties makes the jgit aggregator compile on my
system.
Change-Id: Ia5613cc0fcf6085265a9e48f940e5a7d3f998608 Signed-off-by: Michael Keppler <Michael.Keppler@gmx.de>
David Pursehouse [Thu, 14 Nov 2019 00:40:14 +0000 (16:40 -0800)]
ReceivePack: Reduce visibility of methods
Several methods were protected when they were defined in BaseReceivePack
which has since been squashed into ReceivePack. Those methods no longer
need to be protected, and can now be private instead.
Change-Id: Ic6bb5e66f353fcbdc7919847423234af398c88b4 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Thomas Wolf [Mon, 28 Oct 2019 17:11:57 +0000 (18:11 +0100)]
DiffFormatter: support core.quotePath = false
core.quotePath = false means that "bytes higher than 0x80 are not
considered "unusal" anymore"[1], i.e., they are not escaped. In
essence this preserves non-ASCII characters in path names in output.
Note that control characters and other special characters in the
ASCII range will still be escaped.
Add a new QuotedString.GIT_PATH_MINIMAL singleton implementing this.
Change the normal GIT_PATH algorithm to use bytes instead of characters
so it can be re-used. Provide a setter in DiffFormatter for the quoting
style so that an application can override the default, which is the
setting from the git config (and by default "true"). Use the new
QuotedString.GIT_PATH_MINIMAL when core.quotePath == false.
Thomas Wolf [Sat, 26 Oct 2019 17:58:07 +0000 (19:58 +0200)]
IndexDiff/SubmoduleWalk: make the RepositoryBuilder configurable
Some applications using JGit use their own repository caching. In
such applications, it may be needlessly inefficient to create new
submodule repositories from a SubmoduleWalk or in an IndexDiff. It
can be much more efficient to use an already cached repository
instance.
Provide a way to configure a SubmoduleWalk with a factory to create
BaseRepositoryBuilders to use to create repositories, and use it in
IndexDiff. Provide new IndexDiff.diff() operations that take such an
additional factory as parameter.
An application that caches Repository instances (for instance EGit)
can use a factory that provides builders that don't create a new
Repository instance but that return the already cached instance, if
one is available. Note that in such a case, the application may need
to be prepared to deal with IndexDiff.diff() also _closing_ the
obtained repository; if the application expects its cached Repository
instances to remain open while being cached, it'll have to use
Repository.incrementOpen() to prevent that the repository instance
gets closed.
Bug: 550878
Change-Id: Icc1b34dfc4cebd8ed4739dd09d37744d41adf711 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
IndexDiff would apply ignore mode ALL from .gitmodules to all remaining
submodules, and would ignore other settings from .gitignore and always
apply the setting defined on the IndexDiff instead. Correct that.
In canonical git the ignore setting from .gitmodules can also be
overridden by .git/config.[1] Implement that override in SubmoduleWalk.
I1ce92869435d5eebb7d671be44561e69c6233134 merged BaseReceivePack into
ReceivePack which breaks API but is only affecting the few jgit based
servers out in the wild.
Change-Id: Iad856a2afaf3cad95d01ad81a0116cebcd9de2d9 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
projects / jgit.git / log