Fix crashes on rare combination of file names

The NameConflictTreeWalk class is used in merge for iterating over
entries in commits. The class uses a separate iterator for each
commit's tree. In rare cases it can incorrectly report the same entry
twice. As a result, duplicated entries are added to the merge result
and later jgit throws an exception when it tries to process merge
result.

The problem appears only when there is a directory-file conflict for
the last item in trees. Example from the bug:
Commit 1:
* subtree - file
* subtree-0 - file
Commit 2:
* subtree - directory
* subtree-0 - file
Here the names are ordered like this:
"subtree" file <"subtree-0" file < "subtree" directory.

The NameConflictTreeWalk handles similar cases correctly if there are
other files after subtree... in commits - this is processed in the
AbstractTreeIterator.min function. Existing code has a special
optimization for the case, when all trees are pointed to the same
entry name - it skips additional checks. However, this optimization
incorrectly skips checks if one of trees reached the end.

The fix processes a situation when some trees reached the end, while
others are still point to an entry.

bug: 535919
Change-Id: I62fde3dd89779fac282479c093400448b4ac5c86

1 files changed, 19 insertions, 4 deletions

diff --git a/org.eclipse.jgit/src/org/eclipse/jgit/treewalk/NameConflictTreeWalk.java b/org.eclipse.jgit/src/org/eclipse/jgit/treewalk/NameConflictTreeWalk.java
index 2fd945b03f..ffb41379bd 100644
--- a/org.eclipse.jgit/src/org/eclipse/jgit/treewalk/NameConflictTreeWalk.java
+++ b/org.eclipse.jgit/src/org/eclipse/jgit/treewalk/NameConflictTreeWalk.java
@@ -56,6 +56,13 @@ import org.eclipse.jgit.lib.Repository;
 public class NameConflictTreeWalk extends TreeWalk {
 	private static final int TREE_MODE = FileMode.TREE.getBits();
 
+	/**
+	 * True if all {@link #trees} point to entries with equal names.
+	 *
+	 * If at least one tree iterator point to a different name or
+	 * reached end of the tree, the value is false.
+	 * Note: if all iterators reached end of trees, the value is true.
+	 */
 	private boolean allTreesNamesMatchFastMinRef;
 
 	private AbstractTreeIterator dfConflict;
@@ -125,13 +132,20 @@ public class NameConflictTreeWalk extends TreeWalk {
 			return trees[trees.length - 1];
 		}
 		AbstractTreeIterator minRef = trees[i];
-		allTreesNamesMatchFastMinRef = true;
+		// if i > 0 then we already know that only some trees reached the end
+		// (but not all), so it is impossible that ALL trees points to the
+		// minRef entry.
+		// Only if i == 0 it is still possible that all trees points to the same
+		// minRef entry.
+		allTreesNamesMatchFastMinRef = i == 0;
 		boolean hasConflict = false;
 		minRef.matches = minRef;
 		while (++i < trees.length) {
 			final AbstractTreeIterator t = trees[i];
-			if (t.eof())
+			if (t.eof()) {
+				allTreesNamesMatchFastMinRef = false;
 				continue;
+			}
 
 			final int cmp = t.pathCompare(minRef);
 			if (cmp < 0) {
@@ -152,8 +166,9 @@ public class NameConflictTreeWalk extends TreeWalk {
 				// Exact name/mode match is best.
 				//
 				t.matches = minRef;
-			} else if (allTreesNamesMatchFastMinRef && isTree(t) && !isTree(minRef)
-					&& !isGitlink(minRef) && nameEqual(t, minRef)) {
+			} else if (allTreesNamesMatchFastMinRef && isTree(t)
+					&& !isTree(minRef) && !isGitlink(minRef)
+					&& nameEqual(t, minRef)) {
 				// The minimum is a file (non-tree) but the next entry
 				// of this iterator is a tree whose name matches our file.
 				// This is a classic D/F conflict and commonly occurs like