JGit pgm: Format blame output to match canonical git
Canonical git shows the zero-Id for lines without commit, and
uses the current time stamp for such lines.
Also consider lines without commit when calculating the field
widths.
Change-Id: If0bee9b3e7f90861ad0f387f68e52ecc4a53dfd7
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
When a conflicting file was blamed, JGit would not identify lines
coming from the merge parents. The main cause for this was that
Blame and BlameCommand simply added the first DirCacheEntry found
for a file to its queue of candidates (blobs or commits) to consider.
In case of a conflict this typically is the merge base commit, and
comparing a auto-merged contents against that base would yield
incorrect results.
Such cases have to be handled specially. The candidate to be
considered by the blame must use the working tree contents, but
at the same time behave like a merge commit/candidate with HEAD
and the MERGE_HEADs as parents. Canonical git does something very
similar, see [1].
Implement that and add tests.
I first did this for the JGit pgm Blame command. When I then tried
to do the same in BlameCommand, I noticed that the latter also
included some fancy but incomplete CR-LF handling. In order to
be able to use the new BlameGenerator.prepareHead() also in
BlameCommand this CR-LF handling was also moved into BlameGenerator
and corrected in doing so.
(Just considering the git config settings was not good enough,
CR-LF behavior can also be influenced by .gitattributes, and even
by whether the file in the index has CR-LF. To correctly determine
CR-LF handling for check-in one needs to do a TreeWalk with at
least a FileTreeIterator and a DirCacheIterator.)
[1] https://github.com/git/git/blob/v2.22.0/blame.c#L174
Bug: 434330
Change-Id: I9d763dd6ba478b0b6ebf9456049d6301f478ef7c
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.5:
Bazel: Use java_plugin and java_binary from @rules_java in jmh.bzl
Bazel: Add missing newlines at end of BUILD files
Change-Id: Ie32c3bae056fb0745892e6c20ad37d4994542d96
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* stable-5.4:
Bazel: Use java_plugin and java_binary from @rules_java in jmh.bzl
Bazel: Add missing newlines at end of BUILD files
Change-Id: I3dd2554cbad5bb224c2799b6b133fe91ab10abc3
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* stable-5.3:
Bazel: Use java_plugin and java_binary from @rules_java in jmh.bzl
Bazel: Add missing newlines at end of BUILD files
Change-Id: I21878c42fd9abf7d858b534300df0fffe4bad431
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* stable-5.2:
Bazel: Use java_plugin and java_binary from @rules_java in jmh.bzl
Bazel: Add missing newlines at end of BUILD files
Change-Id: Ie5e66f2e26c057c81101b0d110f91ea479eb362d
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* stable-5.1:
Bazel: Use java_plugin and java_binary from @rules_java in jmh.bzl
Bazel: Add missing newlines at end of BUILD files
Change-Id: Ic1dc84a6c4a1cf430d329627642583fd6e0d0eaa
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
When this class was split from DfsReftableDatabase in commit
2b1e942729 (reftable: split off generic
code from DFS code, 2019-09-26), we forgot to carry over the license
header.
Change-Id: I8698001dfb69f58784df643ef0185ab2d55e52aa
Reported-by: Masaya Suzuki <masayasuzuki@google.com>
Signed-off-by: Jonathan Nieder <jrn@google.com>
Add missing license header to UploadPackRefSortingForReachabilityTest
Forgot to add this in commit 989a927a5f
(checkNotAdvertisedWants: Be lazy convering Ref to RevCommit,
2019-11-20).
Change-Id: I5c1177ac60eabb3a71959bbad4537e076a901b7e
Signed-off-by: Jonathan Nieder <jrn@google.com>
RepositoryCache: don't require HEAD in git repositories
Reftable-enabled repositories don't have a file called HEAD. Check for
reftable/ instead.
This fixes repository creation on reftable in Gerrit.
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Change-Id: I778c2be01d96aaf135affae4b457b5fe5b483bee
Silence API error for new method in ReachabilityChecker
Following OSGi semantic versioning which tolerates breaking implementers
in a minor release.
Change-Id: I4600c5ee9cd4ae209b69870a5d1367f83678617e
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
BitmappedReachabilityChecker: Use only one bitmap for the whole check
The checker is creating a new bitmap per branch leading to excessive
memory consumption. For the reachability check one bitmap with the
reachability of all branches aggregated is enough.
Build the reachability bitmap with a filter. The filter itself uses it
to emit only commits not reached before and the caller to check what
targets have been reached already.
BitmapCalculator is not required anymore.
Change-Id: Ic5c62f77fe0f188913215b7eaa51d849a9aae6a5
Signed-off-by: Ivan Frade <ifrade@google.com>
checkNotAdvertisedWants: Be lazy converting Ref to RevCommit
The ref points to an ObjectId that then is translated into a RevCommit.
This translation can be costly and with the incremental reachability
check is probably not needed for most of the elements.
Delay the translation from ObjectId to RevCommit to when it is needed.
Use Streams, that have the laziness built-in, all the way from Ref to
RevCommit.
This should reduce the latency for reachability checks over big sets of
references.
Change-Id: I28693087321b2beff3eaa1f3d2e7840ab0eedc6d
Signed-off-by: Ivan Frade <ifrade@google.com>
ReachabilityChecker: Receive a Stream instead of a Collection
Preparatory change. Converting ObjectIds to RevCommits is potentially
expensive and in the incremental reachability check, it is probably not
required for all elements in the collection.
Pass a Stream to the reachability checker. In the follow up we make
the conversion from ObjectId to RevCommit in the stream (i.e. on
demand). This should reduce the latency of reachability checks over big
sets of references.
Change-Id: I9f310e331de5b0bf8de34143bd7dcd34316d2fba
Signed-off-by: Ivan Frade <ifrade@google.com>
UploadPack: Prioritize references for non-advertised wants checks
UploadPack needs to check if object ids that weren't advertised before
are reachable from the references visible to the user. In the
bitmap-based reachability check, this is done incrementally: checking
against one reference, if anything remaining adding a second and so on.
It is more efficient to check first more common references (e.g. refs/heads/*)
Sort the references for the reachability checker. This should solve the
connectivity earlier and require less bitmap creation and less memory.
Change-Id: I48ac10d71e29fab2d346479802401eaea4aacb5c
Signed-off-by: Ivan Frade <ifrade@google.com>
Use lambda style comparators where possible. They are easier to read.
Change-Id: I5b80cfcd90909c94286742fa83af71015532809f
Signed-off-by: Michael Keppler <Michael.Keppler@gmx.de>
Reftable is a binary, block-based storage format for the ref-database.
It provides several advantages over the traditional packed + loose
storage format:
* O(1) write performance, even for deletions and transactions.
* atomic updates to the ref database.
* O(log N) lookup and prefix scans
* free from restrictions imposed by the file system: it is
case-sensitive even on case-insensitive file systems, and has
no inherent limitations for directory/file conflicts
* prefix compression reduces space usage for repetitive ref names,
such as gerrit's refs/changes/xx/xxxxx format.
FileReftableDatabase is based on FileReftableStack, which does
compactions inline. This is simple, and has good median performance,
but every so often it will rewrite the entire ref database.
For testing, a FileReftableTest (mirroring RefUpdateTest) is added to
check for Reftable specific behavior. This must be done separately, as
reflogs have different semantics.
Add a reftable flavor of BatchRefUpdateTest.
Add a FileReftableStackTest to exercise compaction.
Add FileRepository#convertToReftable so existing testdata can be
reused.
CQ: 21007
Change-Id: I1837f268e91c6b446cb0155061727dbaccb714b8
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
[WARNING] Some problems were encountered while building the effective
model for org.eclipse.jgit:org.eclipse.jgit.benchmarks:jar:5.6.0-SNAPSHOT
[WARNING] 'build.plugins.plugin.version' for
org.apache.maven.plugins:maven-shade-plugin is missing. @ line 131,
column 15
Change-Id: Id25916c460bc4a10d0f797779d24f53292df2b77
Signed-off-by: Michael Keppler <Michael.Keppler@gmx.de>
Building the jgit aggregator with Maven leads to a compiler level 1.6
for me, thereby failing the build. Multiple profiles specify
source/target level separately, but there seems no global setting.
Adding the global properties makes the jgit aggregator compile on my
system.
Change-Id: Ia5613cc0fcf6085265a9e48f940e5a7d3f998608
Signed-off-by: Michael Keppler <Michael.Keppler@gmx.de>
Several methods were protected when they were defined in BaseReceivePack
which has since been squashed into ReceivePack. Those methods no longer
need to be protected, and can now be private instead.
Change-Id: Ic6bb5e66f353fcbdc7919847423234af398c88b4
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Prevent adding a null parent to a commit's parent array. Doing so
can cause NPEs later on.
Bug: 552160
Change-Id: Ib24b7b9b7b08e0b6f246006b4a4cade7eeb830b9
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
core.quotePath = false means that "bytes higher than 0x80 are not
considered "unusal" anymore"[1], i.e., they are not escaped. In
essence this preserves non-ASCII characters in path names in output.
Note that control characters and other special characters in the
ASCII range will still be escaped.
Add a new QuotedString.GIT_PATH_MINIMAL singleton implementing this.
Change the normal GIT_PATH algorithm to use bytes instead of characters
so it can be re-used. Provide a setter in DiffFormatter for the quoting
style so that an application can override the default, which is the
setting from the git config (and by default "true"). Use the new
QuotedString.GIT_PATH_MINIMAL when core.quotePath == false.
[1] https://git-scm.com/docs/git-config#Documentation/git-config.txt-corequotePath
Bug: 552467
Change-Id: Ifcb233e7d10676333bf42011e32d01a4e1138059
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
IndexDiff/SubmoduleWalk: make the RepositoryBuilder configurable
Some applications using JGit use their own repository caching. In
such applications, it may be needlessly inefficient to create new
submodule repositories from a SubmoduleWalk or in an IndexDiff. It
can be much more efficient to use an already cached repository
instance.
Provide a way to configure a SubmoduleWalk with a factory to create
BaseRepositoryBuilders to use to create repositories, and use it in
IndexDiff. Provide new IndexDiff.diff() operations that take such an
additional factory as parameter.
An application that caches Repository instances (for instance EGit)
can use a factory that provides builders that don't create a new
Repository instance but that return the already cached instance, if
one is available. Note that in such a case, the application may need
to be prepared to deal with IndexDiff.diff() also _closing_ the
obtained repository; if the application expects its cached Repository
instances to remain open while being cached, it'll have to use
Repository.incrementOpen() to prevent that the repository instance
gets closed.
Bug: 550878
Change-Id: Icc1b34dfc4cebd8ed4739dd09d37744d41adf711
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
IndexDiff would apply ignore mode ALL from .gitmodules to all remaining
submodules, and would ignore other settings from .gitignore and always
apply the setting defined on the IndexDiff instead. Correct that.
In canonical git the ignore setting from .gitmodules can also be
overridden by .git/config.[1] Implement that override in SubmoduleWalk.
[1] https://git-scm.com/docs/gitmodules#Documentation/gitmodules.txt-submoduleltnamegtignore
Bug: 521613
Change-Id: I9199fd447e41c7838924856dce40678370b66395
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
I1ce92869435d5eebb7d671be44561e69c6233134 merged BaseReceivePack into
ReceivePack which breaks API but is only affecting the few jgit based
servers out in the wild.
Change-Id: Iad856a2afaf3cad95d01ad81a0116cebcd9de2d9
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Move the BaseReceivePack implementation back into ReceivePack. This is a
backward-incompatible change. For example, BaseReceivePack.FirstLine no
longer exists and cannot be referenced. However, most of the code
should just work by replacing BaseReceivePack with ReceivePack.
Although this is an API change, it only affects callers using JGit as a
server, and there are very few of those in the wild.
Change-Id: I1ce92869435d5eebb7d671be44561e69c6233134
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
This will provide exponential backoff with jitter to other JGit
components too.
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Change-Id: Idd44e3bbaef6d71134ce2e3f7d405f35e7397cbd
The getPackSize method will throw IllegalStateException if it is
called when the pack size is not set. This is the case for example
when the received commands are all DELETE and there is no pack.
Add a new method hasReceivedPack that can be called prior to calling
getPackSize, to avoid causing the IllegalStateException.
See [1] for context.
[1] https://bugs.chromium.org/p/gerrit/issues/detail?id=11918
Change-Id: I56397256a05e92c8398e65c07a859cee59b46317
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Thomas Wolf
David Pursehouse
Jonathan Nieder
Matthias Sohn
Han-Wen Nienhuys
Ivan Frade
Michael Keppler
Masaya Suzuki