This enables higher file timestamp resolution on filesystems like ext4,
Mac APFS (1ns) or NTFS (100ns) providing high timestamp resolution on
filesystem level.

Note:
- on some OSes Java 8,9 truncate milliseconds, see
https://bugs.openjdk.java.net/browse/JDK-8177809, fixed in Java 10
- UnixFileAttributes truncates timestamp resolution to microseconds when
converting the internal representation to FileTime exposed in the API,
see https://bugs.openjdk.java.net/browse/JDK-8181493
- WindowsFileAttributes also provides only microsecond resolution

Change-Id: I25ffff31a3c6f725fc345d4ddc2f26da3b88f6f2
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

The deleted code is not required as removed files are deleted correctly in
doCheckout() anyway.

The deleted code failed in case a non-empty directory had to be deleted.
file.delete() returned false, triggering an exception.

Bug: 479266
Change-Id: I011bb3882ff0c35b238aa3eccad7889041210277
Signed-off-by: René Scheibe <rene.scheibe@gmail.com>

Creating a folder failed in case a file with the same name already
existed.

Bug: 479266
Change-Id: Ia987660ec0968ad4081dbd5a60e80660539497e3
Signed-off-by: René Scheibe <rene.scheibe@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

Change-Id: I97f1367a2ea9f1f6146e264c27c3981b842f2a26
Signed-off-by: Carsten Hammer <carsten.hammer@t-online.de>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

CheckoutCommand had a setForce() method. But this didn't correspond
to native git's 'git checkout -f' option. Deprecate the old setForce()
method and move its implementation to a new method setForceRefUpdate()
and use it to implement the -B option in the CLI class Checkout.

Add a setForced() method and use it to fix the associated '-f' option of
the CLI Checkout class to behave like native git's 'git checkout -f'
which overwrites dirty worktree files during checkout.

This is still not fully matching native git's behavior: updating
additionally dirty index entries is not done yet.

Bug: 530771
Change-Id: I776b78eb623b6ea0aca42f681788f2e4b1667f15
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

A checkout done directly after cloning (the "initial
checkout") has a different semantic as a default
checkout. That is defined in the documentation of
"git read-tree" [1]. JGit was detecting that it is
doing an initial checkout differently from native
git: jgit used to check that the index is empty
but native git required that the index file does
not exist [2]. Teach JGit to behave like native
git.

[1] https://github.com/git/git/blob/master/Documentation/git-read-tree.txt#L187
[2] https://marc.info/?t=154150811200001&r=1&w=2

Change-Id: I1dd0f1ede7cd7ea60d28607916d0165269a9f628

Before this commit, a force checkout would fail if there
were any conflicting files. After this commit, a force
checkout will overwrite the conflicting files, as expected.

Making this work required fixing a bug in DirCacheCheckout.
Before this commit, when DirCacheCheckout had
failOnConflict=false, it would delete all conflicting files
from the working copy and just leave them missing. After
this commit, DirCacheCheckout overwrites conflicting files
with the merge tree.

This change in DirCacheCheckout causes "reset --hard" and
"revert --abort" to behave as expected (previously they
would simply delete conflicting files, now they will be
overwritten from the merge tree).

Change-Id: If7e328ee792ef6511ab7d9c26d8d77c39210ec9f
Signed-off-by: Ned Twigg <ned.twigg@diffplug.com>

On recent VMs, collection.toArray(new T[0]) is faster than
collection.toArray(new T[collection.size()]). Since it is also more
readable, it should now be the preferred way of collection to array
conversion.

https://shipilev.net/blog/2016/arrays-wisdom-ancients/

Change-Id: I80388532fb4b2b0663ee1fe8baa94f5df55c8442
Signed-off-by: Michael Keppler <Michael.Keppler@gmx.de>

ResolveMerger.checkout() and cleanUp() check out files directly and
must honor CR/LF settings and also smudge filters.

Deprecate the 3-argument version of DirCacheCheckout.checkoutEntry().
It isn't used anymore anywhere in JGit (nor in EGit).

Bug: 537410
Change-Id: I062b35401c8bd5bc99deb2f68f91089a0643504c
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>

DirCacheCheckout has a warning about non-localised string "lfs". Other
classes use org.eclipse.jgit.lfs.lib.Constants but that is not visible
to DirCacheCheckout.

Add a new constant in ConfigConstants and use that in DirCacheCheckout.

Replace existing uses of org.eclipse.jgit.lfs.lib.Constants.LFS with
the new constant, except where it is referring to the folder name.

Change-Id: I0f21b951babff9a2e579d68c4de0c62ee4bc23d4
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>

Enable LFS support for the CLI by registering the according filters.

Errors during filter creation must be propagated up the call stack, as a
failure to create a filter should be treated as fatal if the filter is
required.

Change-Id: I3833757839bdda97cd01b6c21c1613d199e2692d
Signed-off-by: Markus Duft <markus.duft@ssi-schaefer.com>

This doesn't handle the really hard thing, which is merging spurious
conflicts inside .gitmodules files.  That's OK: git.git doesn't
either.  Users can resolve the conflict themselves and then commit
the merge.

Previously, jgit would crash when attempting to merge conflicting
submodule changes.  Even if there was no conflict, after a merge which
adds submodules, the repository would have been missing empty
directories for newly-added submodules.

This patch fixes the crash, and adds the empty directories where
necessary. It ensures that the index is in a conflicted state when
submodule changes conflict.

Reported-by: Alexey Korobkov
Bug: 494551
Change-Id: I79db6798c2bdcc1159b5b2589b02da198dc906a1
Signed-off-by: David Turner <dturner@twosigma.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

The reason for the change is LFS: when using a lot of LFS files,
checkout can take quite some time on larger repositories. To avoid
"hanging" UI, provide progress reporting.

Also implement (partial) progress reporting for cherry-pick, reset,
revert which are using checkout internally.

The feature is also useful without LFS, so it is independent of it.

Change-Id: I021e764241f3c107eaf2771f6b5785245b146b42
Signed-off-by: Markus Duft <markus.duft@ssi-schaefer.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

Change-Id: Ib485eb217ac6be70519816f8cc0396931043a3d1
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

Change-Id: Ie08a6275f4656ebb46abe8748b73b0fb52a98a35
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>

and deprecate getEolStreamType().

This resolves a TODO that was apparently supposed to be done in
version 4.4.

Change-Id: I5c9861aedabdc3f99dcf47519b3959a979e6a591

Currently there is no way to determine the precise changes done
to the working tree by a JGit command. Only the CheckoutCommand
actually provides access to the lists of modified, deleted, and
to-be-deleted files, but those lists may be inaccurate (since they
are determined up-front before the working tree is modified) if
the actual checkout then fails halfway through. Moreover, other
JGit commands that modify the working tree do not offer any way to
figure out which files were changed.

This poses problems for EGit, which may need to refresh parts of the
Eclipse workspace when JGit has done java.io file operations.

Provide the foundations for better file change tracking: the working
tree is modified exclusively in DirCacheCheckout. Make it emit a new
type of RepositoryEvent that lists all files that were modified or
deleted, even if the checkout failed halfway through. We update the
'updated' and 'removed' lists determined up-front in case of file
system problems to reflect the actual state of changes made.

EGit thus can register a listener for these events and then knows
exactly which parts of the Eclipse workspace may need to be refreshed.

Two commands manage checking out individual DirCacheEntries themselves:
checkout specific paths, and applying a stash with untracked files.
Make those two also emit such a new WorkingTreeModifiedEvent.

Furthermore, merges may modify files, and clean, rm, and stash create
may delete files.

CQ: 13969
Bug: 500106
Change-Id: I7a100aee315791fa1201f43bbad61fbae60b35cb
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>

DirCacheCheckout is generating names for temporary files. It was not checking
the length of this filenames. It may happen that a generated filename is
longer than 255 chars which causes problems on certain platforms. Make sure
that filenames for temporary files do not exceed 255 chars.

Bug: 508823
Change-Id: I9475c04351ce3faebdc6ad40ea4faa3c326815f4

Since the introduction of generic type parameter inference in Java 7,
it's not necessary to explicitly specify the type of generic parameters.

Enable the warning in Eclipse, and fix all occurrences.

Change-Id: I9158caf1beca5e4980b6240ac401f3868520aad0
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>

JGit supports smudge filters defined in repository configuration. The
filters are implemented as external programs filtering content by
accepting the original content (as seen in git's object database) on
stdin and which emit the filtered content on stdout. This content is
then written to the file in the working tree. To run such a filter JGit
has to start an external process and pump data into/from this process.

This commit adds support for built-in smudge filters which are
implemented in Java and which are executed by jgit's main thread. When a
filter is defined in the configuration as
"jgit://builtin/<filterDriverName>/smudge" then JGit will lookup in a
static map whether a builtin filter is registered under this name. If
found such a filter is called to do the filtering.

The functionality in this commit requires that a program using JGit
explicitly calls the JGit API to register built-in implementations for
specific smudge filters. In follow-up commits configuration parameters
will be added which trigger such registrations.

Change-Id: Ia743aa0dbed795e71e5792f35ae55660e0eb3c24

This fixes the tests failed in JDK8.

FS uses java.nio API to get file attributes. The timestamps obtained
from that API are more precise than the ones from
java.io.File#lastModified() since Java8.

This difference accidentally makes JGit detect newly added files as
smudged. Use the precised timestamp to avoid this false positive.

Bug: 500058
Change-Id: I9e587583c85cb6efa7562ad6c5f26577869a2e7c
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>

Problem occurs when the checkout wants to create a file 'd/f' but
the workingtree contains a dirty file 'd'. In order to create d/f the
file 'd' would have to be deleted and since the file is dirty that
content would be lost. This should lead to a CheckoutConflictException
for d/f when failOnConflict was set to true.

This fix also changes jgit checkout semantics to be more like native
gits checkout semantics. If during a checkout jgit wants to delete a
folder but finds that the working tree contains a dirty file at this
path then JGit will now throw an exception instead of silently keeping
the dirty file. Like in this example:

	git init
	touch b
	git add b
	git commit -m addB
	mkdir a
	touch a/c
	git add a/c
	git commit -m addAC
	rm -fr a
	touch a
	git checkout HEAD~

Change-Id: I9089123179e09dd565285d50b0caa308d290cccd
Signed-off-by: Rüdiger Herrmann <ruediger.herrmann@gmx.de>
Also-by: Rüdiger Herrmann <ruediger.herrmann@gmx.de>

When performing a "reset --hard" a checkout is done. The pathes are
checked for potential checkout conflicts. But in the end for all
remaining conflicts these files are simply deleted from the working
tree. That's not the right strategy to handle checkout conflicts during
"reset --hard". Instead for every conflict we should simply checkout the
merge commit's content.

This is different from native gits behavior which reports errors when
during a "checkout --hard" a file shows up where a folder was expected.

  "warning: unable to unlink d/c.txt: Not a directory"

Why it is like that in native git was asked in
http://permalink.gmane.org/gmane.comp.version-control.git/279482. Unless
it is explained why native git why this error is reported JGit should
overwrite the files.

Bug: 474842
Change-Id: I08e23822a577aaf22120c5137eb169b6bd08447b
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

When CheckoutCommand or MergeCommand is called then not in all situation
the treewalks have been prepared to support clean/smudge filters. Fix
this

Bug: 491505
Change-Id: Iab5608049221c46d06812552ab97299e44d59e64

TreeWalk provides the new method getEolStreamType. This new method can
be used with EolStreamTypeUtil in order to create a wrapped InputStream
or OutputStream when reading / writing files. The implementation
implements support for the git configuration options core.crlf, core.eol
and the .gitattributes "text", "eol" and "binary"

CQ: 10896
Bug: 486563
Change-Id: Ie4f6367afc2a6aec1de56faf95120fff0339a358
Signed-off-by: Ivan Motsch <ivan.motsch@bsiag.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

Bug: 319233
Change-Id: I5137212f5cd3195a52f90ed5e4ce3cf194a13efd
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

The checkPath function is available as a byte[] form, in fact the
String form just converts to byte[] to run the algorithm.

Having DirCacheEntry take a byte[] -> String -> byte[] to check if
each path is valid is a huge waste of CPU time. On some systems it
can double the time required to read 38,999 files from trees to the
DirCache. This slows down any operation using a DirCache.

Expose the byte[] form and use it for DirCacheEntry creation.

Change-Id: I6db7bc793ece99ff3c356338d793c07c061aeac7

If defined in .gitattributes call smudge filter during checkout.

To support checkout where current HEAD,index do not contain attributes
we need to also consider attributes from the tree we checkout. Therefore
CanonicalTreeParser has to learn how to provide attributes.

Change-Id: I168fdb81a8e1a9f991587b3e95a36550ea845f0a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

If the checkout path is currently a non-empty directory (and was a link
or a regular file before), this directory will be removed before
performing checkout, but only if the checkout path is specified.

Bug: 474973
Change-Id: Ifc6c61592d9b54d26c66367163acdebea369145c
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>

Change-Id: I9e021ba2e0d1317bd98e86f832d55787ed6b0a63
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

- use try-with-resource where possible
- replace use of deprecated release() by close()

Change-Id: I0f139c3535679087b7fa09649166bca514750b81
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

Change-Id: I7fcf6534e6092ba87360ccd68a7dd7466c5c8911
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

This was deprecated and should only be used by DirCacheCheckout and
friends. Other classes should use SystemReader.checkPath() instead.

Change-Id: I37cf753b1f081602dee9f0f47979eff39d735f92
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

isValidPath is an older simple form of the validation performed by
checkValidPath. Use the latter as it more consistently matches
git-core's validation rules.

By running the same validation as fsck, callers creating an entry
for the DirCache are more likely to learn early they are trying
to build trees that will fail fsck.

Change-Id: Ibf5ac116097156aa05c18e231bc65c0854932eb1

To update the file length stat we need to use the length of the
temporary file since it's not yet renamed to the target file name here.
The incorrect file length stat update was introduced in
a606dc363d.

Bug: 453962
Change-Id: I715c048227553efae6f8f6b6878c0f04f2609d9c
Also-by: Konrad Kügler <swamblumat-eclipsebugs@yahoo.de>
Also-by: Christian Halstrick <christian.halstrick@sap.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

The bulk of the "is this sane" logic is inside of ObjectChecker.  The
only caller for the version in DirCacheCheckout is an obtuse usage for
the static isValidRefName() method in Repository.

Deprecate the weird single use method in DirCacheCheckout and move all
code for checking a sequence of path components into ObjectChecker,
where it makes sense alongside the existing code that checks a single
component at a time.

Reuse a single ObjectChecker for the local platform, to avoid looking
up the system properties on each path string considered.

Change-Id: Iae6e769f2bfcad05c166e70ff255f9cf9fcdc87e

When writing a symlink the stat data should only be written once
into the DirCacheEntry, based on the symlink itself and not the
possibly resolved destination observed by java.io.File.

Refactor the code to handle symlinks and early return.  This
removes the risk the blob stat info update is used against a
newly checked out symlink.

Hoist the file length stat update immediately after writing
the file, before a rename. This eliminates any race caused by another
process updating the file length after the rename and having it to
fall into the racily clean path.

Change-Id: I978ad9719c018ce1cf26947efbabaa8b9dff2217

Entries should only be written to the working tree managed by the
Repository. Simplify callers by passing only the entry and computing
the work tree location inside of the checkoutEntry method.

Change-Id: I574e41280d0407f1853fda12f4bd0d30f75d74e7

This deprecated method accidentally creates two ObjectReader
instances. Use the instance created one line above that is
correctly released in the finally block.

Change-Id: Ic57d041674611802a9384d8fa1d292e821055019

Callers should manage the ObjectReader, as this allows the JGit library to cache
context relevant information across files checked out at the same time. If the
caller only has one file to checkout, it should still explicitly manage the life
span of the ObjectReader.

Change-Id: Ib57fba6cb4b774ccff8c416ef4d32e2b390f16a9

In a situation where a certain path was ignored but a working tree file
with this path existed jgit didn't allow to checkout a branch which
didn't ignore this path but contained different content. JGit considered
this to be a checkout conflict to prevent overwriting the file in the
working tree and raised an error. This commit fixes this by ensuring
that ignored dirty working tree files don't lead to a checkout conflict.

Bug: 450169
Change-Id: I90288d314ffac73c24a9c70a5181f8243bd4679a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

As described in native gits file "git-read-tree.txt" git has in a
special mode when doing the "initial" checkout. "Initial" means that the
index is empty before the checkout. This was not handled correctly in
JGit and is fixed in this commit. Also see
https://github.com/git/git/blob/master/Documentation/git-read-tree.txt#L181

Change-Id: I9b9d1bd9ebf349cfca420c891c7b099a18d07ba4

Adds further tests where the working tree is dirty (differs from
index) and where we have staged but uncommitted changes.

Fixed the test case 9 for file/directory conflicts.

Bug: 428819
Change-Id: Ie44a288b052abe936ebb74272d0fefef3b218a7a
Signed-off-by: Axel Richard <axel.richard@obeo.fr>
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>

In two places we threw an IOException and the message was built using
JGitText.couldNotWriteFile. We specified 2 parameters, but this pattern
expects only one parameter. In both places we tried to rename a file,
that's why we wanted two parameters (src and target) for the exception
text.

I changed it to use JGitText.renameFileFailed which accepts two
parameters and fits better.

Change-Id: Ib8c2cf78f2b26ca2b97754fe91fdb20b30392415
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

Reuse the generic logic in ObjectChecker to examine paths.
This required extracting the scanner loop to check for bad
characters within the path name segment.

Change-Id: I02e964d114fb544a0c1657790d5367c3a2b09dff

It does not really matter functionally, but it looked bad,

Change-Id: Icb5b73b38fcaa19f7d1c5a0235c3e8dd125b888b

During a checkout we want to prevent to overwrite unsaved local file
content. Jgit was therefore checking whether the file to overwrite is
dirty or missing and would raise a conflict if this was the case. That
was wrong. It should only check if the file is dirty. It's ok to
"overwrite" a missing/non-existing file.

Change-Id: I63c3a94f663c87f09170fdf8b1b1bf4ed5246fc5
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>

The change includes comparing symbolic links between disk and index,
adding symbolic links to the index, creating/modifying links on
checkout. The behavior is controlled by the core.symlinks setting, just
as C Git does. When a new repository is created core.symlinks will be
set depending on the capabilities of the operating system and Java
runtime.

If core.symlinks is set to true, the assumption is that symlinks are
supported, which may result in runtime errors if this turns out not to
be the case.

Measuring the cost of jgit status on a repository with ~70000 files,
of which ~30000 are tracked reveals a penalty of about 10% for using
the Java7 (really NIO2) support module.

Bug: 354367
Change-Id: I12f0fdd9d26212324a586896ef7eb1f6ff89c39c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

Bug: 423551 
Change-Id: I3e71ef1b4a8181f46d2902c9169859f150cd6ad0
Also-By: Robin Stocker <robin@nibor.org>
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

This version does not attempt to unsmudge, unlike the first attempt
in Idafad150553df14827eccfde2e3b95760e16a8b6.

Bug: 372834
Change-Id: I9300e735cb16d6208e1df963abb1ff69f688155d
Also-by: Robin Stocker <robin@nibor.org>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: Robin Stocker <robin@nibor.org>