Fix PathSuffixFilter: can decide only on full paths
On a subtree, a PathSuffixFilter must return -1 ("indeterminate"),
not 0 ("include"), otherwise negation goes wrong: an indeterminate
result (-1) is passed on, but a decision (0/1) is inverted.
As a result a negated PathSuffixFilter would skip all folders.
Bug: 574253
Change-Id: I27fe785c0d772392a5b5efe0a7b1c9cafcb6e566
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
This is similar to change Idbc2c29bd that skipped detecting content
renames for large files. With this change, we added a new option in
RenameDetector called "skipContentRenamesForBinaryFiles", that when set,
causes binary files with any slight modification to be identified as
added/deleted. The default for this boolean is false, so preserving
current behaviour.
Change-Id: I4770b1f69c60b1037025ddd0940ba86df6047299
There are two code paths for detecting renames: one on tree diffs
(using DiffFormatter#scan) and the other on single file diffs (using
DiffFormatter#format). The latter skips binary and large files
for rename detection - check [1], but the former doesn't.
This change skips content rename detection for the tree diffs case for
large files. This is essential to avoid expensive computations while
reading the file, especially for callers who don't want to pay that
cost. Content renames are those which involve files with slightly
modified content. Exact renames will still be identified.
The default threshold for file sizes is reused from
PackConfig.DEFAULT_BIG_FILE_THRESHOLD: 50 MB.
[1] 232876421d/org.eclipse.jgit/src/org/eclipse/jgit/diff/RawText.java (386)
Change-Id: Idbc2c29bd381c6e387185204638f76fda47df41e
Signed-off-by: Youssef Elghareeb <ghareeb@google.com>
DiffFormatter: correctly deal with tracked files in ignored folders
In JGit 5.0, the FileTreeIterator was changed to skip ignored folders
by default. To catch tracked files inside ignored folders, the tree
walk needs to have a DirCacheIterator, and the FileTreeIterator has
to know about that DirCacheIterator via setDirCacheIterator(). (Or
the optimization has to be switched off explicitly via
setWalkIgnoredDirectories(true).)
Skipping ignored directories is an important optimization in some
cases, for instance in node.js/npm projects, where we'd otherwise
traverse the whole huge and deep hierarchy of the typically ignored
node_modules folder.
While all uses of WorkingTreeIterator in JGit had been adapted,
DiffFormatter was forgotten. To make it work correctly (again) also
for such cases, make it set up a WorkingTreeeIterator automatically,
and make sure the WorkingTreeSource can find such files, too. Also
pass the repository to the TreeWalks used inside the DiffFormatter
to pick up the correct attributes, filters, and line-ending settings.
Bug: 565081
Change-Id: Ie88ac81166dc396ba28b83313964c1712b6ca199
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
DiffFormatterTest: Add a test to confirm the default rename detection settings
Add a test that confirms:
- No rename detector is initialized by default
- Rename detector is initialized after calling setDetectRenames(true)
- Rename limit and rename score have the default values 400 and
60, respectively. Note that there are no constants for these values
so the test hard codes them.
Change-Id: I327e2b348a40ef67d8a184e5ab09f4e9ab573e1c
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
RawTextComparator.WS_IGNORE_CHANGE must not compare whitespace
Only the presence or absence of whitespace is significant; but not the
actual whitespace characters. Don't compare whitespace bytes.
Compare the C git implementation at [1].
[1] https://github.com/git/git/blob/0d0e1e8/xdiff/xutils.c#L173
Bug: 563570
Change-Id: I2d0522b637ba6b5c8b911b3376a9df5daa9d4c27
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Revert 4678f4b and provide another solution for bug 467631
Making gitlinks and folders match in a tree walk was the wrong
approach to fix bug 467631. The problem is that in such a conflict
the tree walk may then not descend into the folder.
Revert the changes to Paths.java and PathsTest.java from commit
4678f4b. Instead test for the problem case from bug 467631 explicitly
in IndexDiff. Add Daniel's test case from bug 545162, and add yet
another test case for DiffEntry.scan() that covers the problem
originally reported in bug 545162.
Bug: 545162
Change-Id: Ie2214c5d5ee32ac6596b621f0f1c7b86d38fa9b7
Also-by: Daniel Veihelmann <daniel.veihelmann@gmail.com>
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
With text=auto or core.autocrlf=true, git does not normalize upon
check-in if the file in the index contains already CR/LFs. The
documentation says: "When text is set to "auto", the path is
marked for automatic end-of-line conversion. If Git decides that
the content is text, its line endings are converted to LF on
checkin. When the file has been committed with CRLF, no conversion
is done."[1]
Implement the last bit as in canonical git: check the blob in the
index for CR/LFs. For very large files, we check only the first 8000
bytes, like RawText.isBinary() and AutoLFInputStream do.
In Auto(CR)LFInputStream, ensure that the buffer is filled as much as
possible for the isBinary() check.
Regarding these content checks, there are a number of inconsistencies:
* Canonical git considers files containing lone CRs as binary.
* RawText checks the first 8000 bytes.
* Auto(CR)LFInputStream checks the first 8096 (not 8192!) bytes.
None of these are changed with this commit. It appears that canonical
git will check the whole blob, not just the first 8k bytes. Also
note: the check for CR/LF text won't work with LFS (neither in JGit
nor in git) since the blob data is not run through the smudge filter.
C.f. [2].
Two tests in AddCommandTest actually tested that normalization was
done even if the file was already committed with CR/LF.These tests
had to be adapted. I find the git documentation unclear about the
case where core.autocrlf=input, but from [3] it looks as if this
non-normalization also applies in this case.
Add new tests in CommitCommandTest testing this for the case where
the index entry is for a merge conflict. In this case, canonical git
uses the "ours" version.[4] Do the same.
[1] https://git-scm.com/docs/gitattributes
[2] https://github.com/git/git/blob/3434569fc/convert.c#L225
[3] https://github.com/git/git/blob/3434569fc/convert.c#L529
[4] https://github.com/git/git/blob/f2b6aa98b/read-cache.c#L3281
Bug: 470643
Change-Id: Ie7310539fbe6c737d78b1dcc29e34735d4616b88
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
This partially reverts commit a551b646: revert the changes in
RawParseUtils.lineMap(). Forcing all blobs containing a NUL byte
as a single line causes blame to produce useless results as soon
as it hits any version containing a NUL byte.
Doing binary detection at this level also has the problem that the
user cannot control it. Not by setting the text attribute nor in any
other way.
This came up in bug 541036, where a Java source inadvertently
contained NUL bytes in strings. Even fixing this by using escapes
"\000" will not fix JGit's blame for this file because the past
versions will still contain the NUL byte.
Native git can blame that file from bug 541036 fine.
Added new tests verifying that blaming a text file containing a NUL
byte produces sensible results.
Bug: 541036
Change-Id: I8991bec88e9827cc096868c6026ea1890b6d0d32
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Introduce new RawText constructor and RawParseUtils.lineMapOrBinary
This makes binary detection exact in ResolveMerger and DiffFormatter
This has the same intention as
Id4342a199628d9406bfa04af1b023c27a47d4014 but preserves backward
compatibility of the signature of RawParseUtils.lineMap.
Change-Id: Ia24a4e716592bab3363ae24e3a46315a7511154f
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Fix DiffFormatter for diffs against working tree with autocrlf=true
The WorkingTreeSource produced an ObjectLoader that returned
inconsistent sizes: the file size in getSize(), but then a
correctly filtered smaller stream in openStream(). This resulted
either in an IOE "short read of block" or in an EOFException
depending on the resulting filtered size.
Fix this by ensuring that getSize() does return the size of the
filtered stream.
Bug: 530106
Change-Id: I7c7c85036047dc10030ed29c1d5a6c7f34f2bdff
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Use constants from StandardCharsets instead of hard-coded strings
Instead of hard-coding the charset strings "US-ASCII", "UTF-8", and
"ISO-8859-1", use the corresponding constants from StandardCharsets.
UnsupportedEncodingException is not thrown when the StandardCharset
constants are used, so remove the now redundant handling.
Because the encoding names are no longer hard-coded strings, also
remove redundant $NON-NLS warning suppressions.
Also replace existing usages of the constants with static imports.
Change-Id: I0a4510d3d992db5e277f009a41434276f95bda4e
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Suppress "Unlikely argument type for equals()" warnings in tests
This new warning was introduced in Eclipse 4.7 Oxygen [1].
The only instances of the warning are in test code that is asserting
that some class does not compare equal to Strings. As in the Gerrit
project [2] these asserts are arguably overkill, but arguably also
a reasonable test of an equals implementation. Ignore the warning in
these cases.
Note that if the project is opened in an earlier version of Eclipse,
a warning "Unsupported @SuppressWarnings" will be emitted.
[1] https://www.eclipse.org/eclipse/news/4.7/M6/
[2] https://gerrit-review.googlesource.com/#/c/gerrit/+/110339/
Change-Id: I08ea33d71e6009cf0f37e6492a475931f447256b
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Throw BinaryBlobException from RawParseUtils#lineMap.
This makes detection of binaries exact for ResolveMerger and
DiffFormatter: they will classify files as binary regardless of where
the '\0' occurs in the text.
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Change-Id: Id4342a199628d9406bfa04af1b023c27a47d4014
This method creates a RawText from a blob, but avoids reading the blob
if the start contains null bytes. This should reduce the amount of
garbage that Gerrit produces for changes with binaries.
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Change-Id: Idd202d20251f2d1653e5f1ca374fe644c2cf205f
DiffAlgorithms can return different edit locations for inserts or
deletes, if they can be "shifted" up or down repeating blocks of
lines. This causes the 3-way merge to apply both edits, resulting in
incorrectly removing or duplicating lines.
Augment an existing "tidy-up" stage in DiffAlgorithm to move all
shiftable edits (not just the last INSERT edit) to a consistent
location, and add test cases for previously incorrect merges.
Bug: 514095
Change-Id: I5fe150a2fc04e1cdb012d22609d86df16dfb0b7e
Signed-off-by: KB Sriram <kbsriram@google.com>
Enable and fix warnings about redundant specification of type arguments
Since the introduction of generic type parameter inference in Java 7,
it's not necessary to explicitly specify the type of generic parameters.
Enable the warning in Eclipse, and fix all occurrences.
Change-Id: I9158caf1beca5e4980b6240ac401f3868520aad0
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Enable and fix 'Should be tagged with @Override' warning
Set missingOverrideAnnotation=warning in Eclipse compiler preferences
which enables the warning:
The method <method> of type <type> should be tagged with @Override
since it actually overrides a superclass method
Justification for this warning is described in:
http://stackoverflow.com/a/94411/381622
Enabling this causes in excess of 1000 warnings across the entire
code-base. They are very easy to fix automatically with Eclipse's
"Quick Fix" tool.
Fix all of them except 2 which cause compilation failure when the
project is built with mvn; add TODO comments on those for further
investigation.
Change-Id: I5772061041fd361fe93137fd8b0ad356e748a29c
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Java 8 fixed the silent flush during close issue by
FilterOutputStream (base class of BufferedOutputStream)
using try-with-resources to close the stream, getting a
behavior matching what JGit's SafeBufferedOutputStream
was doing:
try {
flush();
} finally {
out.close();
}
With Java 8 as the minimum required version to run JGit
it is no longer necessary to override close() or have
this class. Deprecate the class, and use the JRE's version
of close.
Change-Id: Ic0584c140010278dbe4062df2e71be5df9a797b3
Since If13f7b406, submodule difference are shown as a hunk. The issue
was that added and removed submodule were considered as Edit.REPLACE
instead of Edit.INSERT and Edit.DELETE in the DiffFormatter result.
Change-Id: I4330c2aa3f10e29d7d6b0b2e5286e59293a06239
Signed-off-by: Hugo Arès <hugo.ares@ericsson.com>
DiffFormatterTest: Create auto-closeable instances in try-with-resource
Git, ByteArrayOutputStream, and DiffFormatter are auto-closeable and
should be managed in try-with-resource.
Change-Id: I83395116acb4b4f7cd4300fd69564355bc07e4bb
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
When attempting to determine the size of a blob that does not exist,
the RenameDetector throws a MissingObjectException.
The fix is to return a size of zero if the size is requested for a blob
id that doesn't exist.
Bug: 481577
Change-Id: I4e86276039c630617610cc51d0eefa56d7d3952f
Signed-off-by: Rüdiger Herrmann <ruediger.herrmann@gmx.de>
Current DiffFormat behavior regarding submodules (aka git links) is
incorrect. The "Subproject commit <sha1>" appears as part of the diff
header, rather than as its own hunk.
--> From JGit implementation
diff --git a/plugins/cookbook-plugin b/plugins/cookbook-plugin
index b9d3ca8..ec6ed89 160000
--- a/plugins/cookbook-plugin
+++ b/plugins/cookbook-plugin
-Subproject commit b9d3ca8a65
+Subproject commit ec6ed89c47
--> From C Git 2.5.2
diff --git a/plugins/cookbook-plugin b/plugins/cookbook-plugin
index b9d3ca8..ec6ed89 160000
--- a/plugins/cookbook-plugin
+++ b/plugins/cookbook-plugin
@@ -1 +1 @@
-Subproject commit b9d3ca8a65
+Subproject commit ec6ed89c47
The current way of processing submodules results in no hunk header and
includes the contents of the hunk as part of the headers. To fix this, we
can't just have our writeGitLinkDiffText output the hunk header. We have
to change the flow so that the raw text gets parsed as a diff. The easiest
way to do this is to fake the RawText in the FormatResult when we have a
GITLINK.
It should be noted that it seems possible for there to be a difference
between a GITLINK and a non-GITLINK, but I don't think this can happen in
practice, so I don't think we need to worry too much about it.
This patch also fixes up the test for GitLink headers, as the test was
for the old behavior. My setup has 3 other failing tests which may or
may not be the result of environmental changes. However, the same tests
fail without this commit, so I do not believe they are related.
Bug: 477759
Change-Id: If13f7b406904fad814416c93ed09ea47ef183337
Signed-off-by: Jacob Keller <jacob.keller@gmail.com>
Native Git canonicalizes line endings when detecting
renames, more specifically it replaces CRLF by LF.
See: hash_chars in diffcore-delta.c
Bug: 449545
Change-Id: Iec2aab12ae9e67074cccb7fbd4d9defe176a0130
Signed-off-by: Marc Strapetz <marc.strapetz@syntevo.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Handle diff formatting when there is nothing to compare with
DiffFormatter now suports either side being null and the log program
will output the diff for the first commit.
Bug: 395791
Change-Id: I378957b57e9ad1f7195ba416f402178453f0ebd3
When git-core renames or copies a file and the mode differs the
header shows the mode change first, then the rename or copy data:
diff --git a/COPYING b/LICENSE
old mode 100644
new mode 100755
similarity index 92%
rename from COPYING
rename to LICENSE
index d645695..54863be
--- a/COPYING
+++ b/LICENSE
@@ -56,20 +56,6 @@
JGit relies on this ordering inside of FileHeader. Parsing "new file
mode NNN" after "copy from/to" or "rename from/to" resets the change
type to be ADD, losing the COPIED or RENAMED status and old path.
This fixes a 4 year old bug in Gerrit Code Review that prevents
opening a file for review if the file was copied from another file,
modified in this change, and the mode was updated (e.g. execute
bit was added).
Change-Id: If4c9ecd61ef0ca8e3e1ea857301f7b5c948efb96
[ms: added test case]
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Fix DiffFormatter NPEs for DiffEntry without content change
DiffEntry.getOldId() returns null for a diff without an index line (e.g.
only mode changed, rename without content change).
Bug: 407743
Change-Id: I42eac87421f2a53c985af260a253338f578492bc
Enable marking entries using TreeFilters in DiffEntry
This adds a new optional TreeFilter[] argument to DiffEntry.scan. All
filters will be checked during the scan to determine if an entry should
be "marked" with regard to that filter.
After having called scan, the user can then call isMarked(int) on the
entries to find out whether they matched the TreeFilter with the passed
index.
An example use case for this is in the file diff viewer of EGit's
History view, where we'd like to highlight entries that are matching the
current filter.
See EGit change I03da4b38d1591495cb290909f0e4c6e52270e97f.
Bug: 393610
Change-Id: Icf911fe6fca131b2567514f54d66636a44561af1
Signed-off-by: Robin Stocker <robin@nibor.org>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Thomas Wolf
Youssef Elghareeb
David Pursehouse
Matthias Sohn
Han-Wen Nienhuys
KB Sriram
Shawn Pearce
David Pursehouse
Hugo Arès
Rüdiger Herrmann
Jacob Keller
Frank Wagner
Marc Strapetz
Robin Rosenberg
Robin Stocker