Currently there is no way to determine the precise changes done
to the working tree by a JGit command. Only the CheckoutCommand
actually provides access to the lists of modified, deleted, and
to-be-deleted files, but those lists may be inaccurate (since they
are determined up-front before the working tree is modified) if
the actual checkout then fails halfway through. Moreover, other
JGit commands that modify the working tree do not offer any way to
figure out which files were changed.

This poses problems for EGit, which may need to refresh parts of the
Eclipse workspace when JGit has done java.io file operations.

Provide the foundations for better file change tracking: the working
tree is modified exclusively in DirCacheCheckout. Make it emit a new
type of RepositoryEvent that lists all files that were modified or
deleted, even if the checkout failed halfway through. We update the
'updated' and 'removed' lists determined up-front in case of file
system problems to reflect the actual state of changes made.

EGit thus can register a listener for these events and then knows
exactly which parts of the Eclipse workspace may need to be refreshed.

Two commands manage checking out individual DirCacheEntries themselves:
checkout specific paths, and applying a stash with untracked files.
Make those two also emit such a new WorkingTreeModifiedEvent.

Furthermore, merges may modify files, and clean, rm, and stash create
may delete files.

CQ: 13969
Bug: 500106
Change-Id: I7a100aee315791fa1201f43bbad61fbae60b35cb
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>

DirCacheCheckout is generating names for temporary files. It was not checking
the length of this filenames. It may happen that a generated filename is
longer than 255 chars which causes problems on certain platforms. Make sure
that filenames for temporary files do not exceed 255 chars.

Bug: 508823
Change-Id: I9475c04351ce3faebdc6ad40ea4faa3c326815f4

Since the introduction of generic type parameter inference in Java 7,
it's not necessary to explicitly specify the type of generic parameters.

Enable the warning in Eclipse, and fix all occurrences.

Change-Id: I9158caf1beca5e4980b6240ac401f3868520aad0
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>

JGit supports smudge filters defined in repository configuration. The
filters are implemented as external programs filtering content by
accepting the original content (as seen in git's object database) on
stdin and which emit the filtered content on stdout. This content is
then written to the file in the working tree. To run such a filter JGit
has to start an external process and pump data into/from this process.

This commit adds support for built-in smudge filters which are
implemented in Java and which are executed by jgit's main thread. When a
filter is defined in the configuration as
"jgit://builtin/<filterDriverName>/smudge" then JGit will lookup in a
static map whether a builtin filter is registered under this name. If
found such a filter is called to do the filtering.

The functionality in this commit requires that a program using JGit
explicitly calls the JGit API to register built-in implementations for
specific smudge filters. In follow-up commits configuration parameters
will be added which trigger such registrations.

Change-Id: Ia743aa0dbed795e71e5792f35ae55660e0eb3c24

This fixes the tests failed in JDK8.

FS uses java.nio API to get file attributes. The timestamps obtained
from that API are more precise than the ones from
java.io.File#lastModified() since Java8.

This difference accidentally makes JGit detect newly added files as
smudged. Use the precised timestamp to avoid this false positive.

Bug: 500058
Change-Id: I9e587583c85cb6efa7562ad6c5f26577869a2e7c
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>

Problem occurs when the checkout wants to create a file 'd/f' but
the workingtree contains a dirty file 'd'. In order to create d/f the
file 'd' would have to be deleted and since the file is dirty that
content would be lost. This should lead to a CheckoutConflictException
for d/f when failOnConflict was set to true.

This fix also changes jgit checkout semantics to be more like native
gits checkout semantics. If during a checkout jgit wants to delete a
folder but finds that the working tree contains a dirty file at this
path then JGit will now throw an exception instead of silently keeping
the dirty file. Like in this example:

	git init
	touch b
	git add b
	git commit -m addB
	mkdir a
	touch a/c
	git add a/c
	git commit -m addAC
	rm -fr a
	touch a
	git checkout HEAD~

Change-Id: I9089123179e09dd565285d50b0caa308d290cccd
Signed-off-by: Rüdiger Herrmann <ruediger.herrmann@gmx.de>
Also-by: Rüdiger Herrmann <ruediger.herrmann@gmx.de>

When performing a "reset --hard" a checkout is done. The pathes are
checked for potential checkout conflicts. But in the end for all
remaining conflicts these files are simply deleted from the working
tree. That's not the right strategy to handle checkout conflicts during
"reset --hard". Instead for every conflict we should simply checkout the
merge commit's content.

This is different from native gits behavior which reports errors when
during a "checkout --hard" a file shows up where a folder was expected.

  "warning: unable to unlink d/c.txt: Not a directory"

Why it is like that in native git was asked in
http://permalink.gmane.org/gmane.comp.version-control.git/279482. Unless
it is explained why native git why this error is reported JGit should
overwrite the files.

Bug: 474842
Change-Id: I08e23822a577aaf22120c5137eb169b6bd08447b
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

When CheckoutCommand or MergeCommand is called then not in all situation
the treewalks have been prepared to support clean/smudge filters. Fix
this

Bug: 491505
Change-Id: Iab5608049221c46d06812552ab97299e44d59e64

TreeWalk provides the new method getEolStreamType. This new method can
be used with EolStreamTypeUtil in order to create a wrapped InputStream
or OutputStream when reading / writing files. The implementation
implements support for the git configuration options core.crlf, core.eol
and the .gitattributes "text", "eol" and "binary"

CQ: 10896
Bug: 486563
Change-Id: Ie4f6367afc2a6aec1de56faf95120fff0339a358
Signed-off-by: Ivan Motsch <ivan.motsch@bsiag.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

Bug: 319233
Change-Id: I5137212f5cd3195a52f90ed5e4ce3cf194a13efd
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

The checkPath function is available as a byte[] form, in fact the
String form just converts to byte[] to run the algorithm.

Having DirCacheEntry take a byte[] -> String -> byte[] to check if
each path is valid is a huge waste of CPU time. On some systems it
can double the time required to read 38,999 files from trees to the
DirCache. This slows down any operation using a DirCache.

Expose the byte[] form and use it for DirCacheEntry creation.

Change-Id: I6db7bc793ece99ff3c356338d793c07c061aeac7

If defined in .gitattributes call smudge filter during checkout.

To support checkout where current HEAD,index do not contain attributes
we need to also consider attributes from the tree we checkout. Therefore
CanonicalTreeParser has to learn how to provide attributes.

Change-Id: I168fdb81a8e1a9f991587b3e95a36550ea845f0a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

If the checkout path is currently a non-empty directory (and was a link
or a regular file before), this directory will be removed before
performing checkout, but only if the checkout path is specified.

Bug: 474973
Change-Id: Ifc6c61592d9b54d26c66367163acdebea369145c
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>

Change-Id: I9e021ba2e0d1317bd98e86f832d55787ed6b0a63
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

- use try-with-resource where possible
- replace use of deprecated release() by close()

Change-Id: I0f139c3535679087b7fa09649166bca514750b81
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

Change-Id: I7fcf6534e6092ba87360ccd68a7dd7466c5c8911
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

This was deprecated and should only be used by DirCacheCheckout and
friends. Other classes should use SystemReader.checkPath() instead.

Change-Id: I37cf753b1f081602dee9f0f47979eff39d735f92
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

isValidPath is an older simple form of the validation performed by
checkValidPath. Use the latter as it more consistently matches
git-core's validation rules.

By running the same validation as fsck, callers creating an entry
for the DirCache are more likely to learn early they are trying
to build trees that will fail fsck.

Change-Id: Ibf5ac116097156aa05c18e231bc65c0854932eb1

To update the file length stat we need to use the length of the
temporary file since it's not yet renamed to the target file name here.
The incorrect file length stat update was introduced in
a606dc363d.

Bug: 453962
Change-Id: I715c048227553efae6f8f6b6878c0f04f2609d9c
Also-by: Konrad Kügler <swamblumat-eclipsebugs@yahoo.de>
Also-by: Christian Halstrick <christian.halstrick@sap.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

The bulk of the "is this sane" logic is inside of ObjectChecker.  The
only caller for the version in DirCacheCheckout is an obtuse usage for
the static isValidRefName() method in Repository.

Deprecate the weird single use method in DirCacheCheckout and move all
code for checking a sequence of path components into ObjectChecker,
where it makes sense alongside the existing code that checks a single
component at a time.

Reuse a single ObjectChecker for the local platform, to avoid looking
up the system properties on each path string considered.

Change-Id: Iae6e769f2bfcad05c166e70ff255f9cf9fcdc87e

When writing a symlink the stat data should only be written once
into the DirCacheEntry, based on the symlink itself and not the
possibly resolved destination observed by java.io.File.

Refactor the code to handle symlinks and early return.  This
removes the risk the blob stat info update is used against a
newly checked out symlink.

Hoist the file length stat update immediately after writing
the file, before a rename. This eliminates any race caused by another
process updating the file length after the rename and having it to
fall into the racily clean path.

Change-Id: I978ad9719c018ce1cf26947efbabaa8b9dff2217

Entries should only be written to the working tree managed by the
Repository. Simplify callers by passing only the entry and computing
the work tree location inside of the checkoutEntry method.

Change-Id: I574e41280d0407f1853fda12f4bd0d30f75d74e7

This deprecated method accidentally creates two ObjectReader
instances. Use the instance created one line above that is
correctly released in the finally block.

Change-Id: Ic57d041674611802a9384d8fa1d292e821055019

Callers should manage the ObjectReader, as this allows the JGit library to cache
context relevant information across files checked out at the same time. If the
caller only has one file to checkout, it should still explicitly manage the life
span of the ObjectReader.

Change-Id: Ib57fba6cb4b774ccff8c416ef4d32e2b390f16a9

In a situation where a certain path was ignored but a working tree file
with this path existed jgit didn't allow to checkout a branch which
didn't ignore this path but contained different content. JGit considered
this to be a checkout conflict to prevent overwriting the file in the
working tree and raised an error. This commit fixes this by ensuring
that ignored dirty working tree files don't lead to a checkout conflict.

Bug: 450169
Change-Id: I90288d314ffac73c24a9c70a5181f8243bd4679a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

As described in native gits file "git-read-tree.txt" git has in a
special mode when doing the "initial" checkout. "Initial" means that the
index is empty before the checkout. This was not handled correctly in
JGit and is fixed in this commit. Also see
https://github.com/git/git/blob/master/Documentation/git-read-tree.txt#L181

Change-Id: I9b9d1bd9ebf349cfca420c891c7b099a18d07ba4

Adds further tests where the working tree is dirty (differs from
index) and where we have staged but uncommitted changes.

Fixed the test case 9 for file/directory conflicts.

Bug: 428819
Change-Id: Ie44a288b052abe936ebb74272d0fefef3b218a7a
Signed-off-by: Axel Richard <axel.richard@obeo.fr>
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>

In two places we threw an IOException and the message was built using
JGitText.couldNotWriteFile. We specified 2 parameters, but this pattern
expects only one parameter. In both places we tried to rename a file,
that's why we wanted two parameters (src and target) for the exception
text.

I changed it to use JGitText.renameFileFailed which accepts two
parameters and fits better.

Change-Id: Ib8c2cf78f2b26ca2b97754fe91fdb20b30392415
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

Reuse the generic logic in ObjectChecker to examine paths.
This required extracting the scanner loop to check for bad
characters within the path name segment.

Change-Id: I02e964d114fb544a0c1657790d5367c3a2b09dff

It does not really matter functionally, but it looked bad,

Change-Id: Icb5b73b38fcaa19f7d1c5a0235c3e8dd125b888b

During a checkout we want to prevent to overwrite unsaved local file
content. Jgit was therefore checking whether the file to overwrite is
dirty or missing and would raise a conflict if this was the case. That
was wrong. It should only check if the file is dirty. It's ok to
"overwrite" a missing/non-existing file.

Change-Id: I63c3a94f663c87f09170fdf8b1b1bf4ed5246fc5
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>

The change includes comparing symbolic links between disk and index,
adding symbolic links to the index, creating/modifying links on
checkout. The behavior is controlled by the core.symlinks setting, just
as C Git does. When a new repository is created core.symlinks will be
set depending on the capabilities of the operating system and Java
runtime.

If core.symlinks is set to true, the assumption is that symlinks are
supported, which may result in runtime errors if this turns out not to
be the case.

Measuring the cost of jgit status on a repository with ~70000 files,
of which ~30000 are tracked reveals a penalty of about 10% for using
the Java7 (really NIO2) support module.

Bug: 354367
Change-Id: I12f0fdd9d26212324a586896ef7eb1f6ff89c39c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

Bug: 423551 
Change-Id: I3e71ef1b4a8181f46d2902c9169859f150cd6ad0
Also-By: Robin Stocker <robin@nibor.org>
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

This version does not attempt to unsmudge, unlike the first attempt
in Idafad150553df14827eccfde2e3b95760e16a8b6.

Bug: 372834
Change-Id: I9300e735cb16d6208e1df963abb1ff69f688155d
Also-by: Robin Stocker <robin@nibor.org>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: Robin Stocker <robin@nibor.org>

This reverts commit 1def0a1257.

We found this fix uncovers problems with unsmudged DirCacheEntry's. This
surfaced because egit's ui test CreatePatchActionTest failed since jgit
computes a wrong status. JGit doesn't detect modified content in now
unsmudged entries. Hence revert this change until these problems are
fixed.

Change-Id: Ia04277ce316d35fc5b0d82c93d2078b856af24bb
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>

Bug: 372834
Change-Id: Idafad150553df14827eccfde2e3b95760e16a8b6
Also-by: Robin Stocker <robin@nibor.org>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

Instead of just a generic "Invalid path: $path", add a reason for the
cases where it's not obvious what the problem is (e.g. "aux" being
reserved on Windows).

Bug: 413915
Change-Id: Ia6436bd2560e4f049c92d9aac907cb87348605e0
Signed-off-by: Robin Stocker <robin@nibor.org>

JGit was not handling certain file/folder conflicts during a checkout
correctly. This was reported by Axel Richard in
http://dev.eclipse.org/mhonarc/lists/jgit-dev/msg02358.html.
This commit fixes this problem.

Still JGit behaves intentionally different than native git.
If HEAD contains a tree, and workingtree, Index and Merge contain a file
with same content ... then JGit allows a conflict free checkout of
Merge. Native git always complains that it doesn't want to overwrite
local changes. But there is no need to update the working tree because
Index and Merge are already equal.

A shell script which shows how native git behaves can be found here.
https://gist.github.com/chalstrick/7694959#file-gistfile1-sh

Change-Id: Ifd6a68974d61cd4fa23bc575f3a40773db66cafc
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>

Checking of spaces at the end of the file name caused the
test to fail for Windows only.

Bug: 396662
Change-Id: I47bcccb0fa32ce606276c3f30d454851d115ca11
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>

In C Git 1.8.2, "git reset" now also works on an unborn branch (no HEAD
yet) if no explicit ref was specified. In that case, it is treated as a
reset to an empty tree.

This can be useful for callers because "unborn branch" no longer has to
be special-cased to "git rm --cached".

Bug: 414870
Change-Id: Ied750116f767518ae4d48823cf00752b049a8477
Signed-off-by: Robin Stocker <robin@nibor.org>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

There was a chance that jgit deletes symbolic links which point to the
folder on top of the working tree. Make sure not to touch these
resources.

Thanks to Cedric Darloy who reported this bug on
http://www.eclipse.org/forums/index.php/m/776910/#msg_776910 and to
Ondrej Vrabec who reported bug 412489.

Bug: 412489
Change-Id: I81735ba0394ef6794e9b2b8bdd8bd7e8b9c6460f
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Signed-off-by: Robin Stocker <robin@nibor.org>

parentFile becomes null when f is relative path, such as ".".

This patch avoids NullPointerException in such case.

Change-Id: I4752674b1daab6eedd7c3650c7749462810eaffd
Signed-off-by: Hiroshi Tomita <tomykaira@gmail.com>

There was a chance that jgit deletes symbolic links which point to the
folder on top of the working tree. Make sure not to touch these
resources.

Thanks to Cedric Darloy who reported this bug on
http://www.eclipse.org/forums/index.php/m/776910/#msg_776910 and to
Ondrej Vrabec who reported bug 412489.

Bug: 412489
Change-Id: I81735ba0394ef6794e9b2b8bdd8bd7e8b9c6460f
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Signed-off-by: Robin Stocker <robin@nibor.org>

parentFile becomes null when f is relative path, such as ".".

This patch avoids NullPointerException in such case.

Change-Id: I4752674b1daab6eedd7c3650c7749462810eaffd
Signed-off-by: Hiroshi Tomita <tomykaira@gmail.com>

On Windows renameTo will not overwrite a file, so it must be deleted
first. The fix for Bug 402834 did not account for that.

Bug: 403685
Change-Id: I3453342c17e064dcb50906a540172978941a10a6

This fixes two cases:
- A folder without tracked content exist both in the workdir and merged
commit, as long as there names within that folder does not conflict.
- An empty folder structure exists with the same name as a file in the
merged commit.

Bug: 402834
Change-Id: I4c5b9f11313dd1665fcbdae2d0755fdb64deb3ef

Change-Id: I282b702d9e4cb19063d8e9503604538f80e955b7
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>

A few classes such as Constanrs are marked with @SuppressWarnings, as are
toString() methods with many liternal, but otherwise $NLS-n$ is used for
string containing text that should not be translated. A few literals may
fall into the gray zone, but mostly I've tried to only tag the obvious
ones.

Change-Id: I22e50a77e2bf9e0b842a66bdf674e8fa1692f590

When a file is not in the index and neither contents nor mode differ
between "head" and "merge", the index state should be kept. If they
differ, a checkout conflict should occur. This is described in Git's
git-read-tree.txt.

JGit used to replace the index state with "merge" in both of the above
cases.

A confusing effect of this was that when one removed a file and then did
a rebase, the file silently reappeared again.

The changes to dir/file conflict handling are a consequence of this
change, as the index handling change made tests in DirCacheCheckoutTest
break. I compared these cases to C Git and the new behavior there also
matches what C Git does.

Bug: 387390
Change-Id: I5beb781f12172a68f98c67d4c8029eb51ceae62d
Signed-off-by: Robin Stocker <robin@nibor.org>

Change-Id: Id0b8aef92f10572d4f1ec198e0281162fcd7ed4e