Use Instant instead of milliseconds for filesystem timestamp handling
This enables higher file timestamp resolution on filesystems like ext4,
Mac APFS (1ns) or NTFS (100ns) providing high timestamp resolution on
filesystem level.
Note:
- on some OSes Java 8,9 truncate milliseconds, see
https://bugs.openjdk.java.net/browse/JDK-8177809, fixed in Java 10
- UnixFileAttributes truncates timestamp resolution to microseconds when
converting the internal representation to FileTime exposed in the API,
see https://bugs.openjdk.java.net/browse/JDK-8181493
- WindowsFileAttributes also provides only microsecond resolution
Change-Id: I25ffff31a3c6f725fc345d4ddc2f26da3b88f6f2
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
MergeFormatter{Pass}: Use Charset instead of String
Deprecate constructors and methods taking a character set name as
a String, in favor of new variants taking a Charset.
Change-Id: I616c601daf232fa17610dba1087fd902030d46ea
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Fix ResolveMerger: rebase with autocrlf=true, direct checkout
ResolveMerger.checkout() and cleanUp() check out files directly and
must honor CR/LF settings and also smudge filters.
Deprecate the 3-argument version of DirCacheCheckout.checkoutEntry().
It isn't used anymore anywhere in JGit (nor in EGit).
Bug: 537410
Change-Id: I062b35401c8bd5bc99deb2f68f91089a0643504c
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
ResolveMerger: don't try needlessly to delete directories
Don't try to delete folders if the merger chooses THEIRS, but all of
BASE, OURS, and THEIRS contain the folder.
Add a test for rebase with auto-stash and subdirectories that
verifies this case. The needless directory deletion and reporting
such directories in getModifiedFiles() was the root cause of bug
536880.
Note even with this fix, bug 536880 will not be fixed in all cases
yet. There may still be cases where the set of modified files ends
up containing directories. This will be dealt with in EGit where
this set is used. (See https://git.eclipse.org/r/#/c/126242/ .)
Bug: 536880
Change-Id: I62b4571a1c1d4415934a6cb4270e0c8036deb2e9
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
ResolveMerger: Fix encoding with string; use bytes
This change fixes the issue [1]. Before this fix, a merge involving
the caching of consecutive yet similar filenames with Norwegian
characters [2] used to throw an IllegalStateException: Duplicate
stages not allowed. This was caused by inaccurate decoding of the
filenames, using string values assuming default encoding. In the
toString method of DirCacheEntry, used before through getPathString,
UTF-8 encoding is used, but the end result becomes default encoding,
through Object's default toString usage. The special characters in
those two consecutive (particular) filenames [2] were becoming the
very same decoded /single character, lending consecutive -but then
identical- filenames. Thus the perceived duplicate 0-staging of the
file(s).
Replace getPathString usage with getRawPath for this specific case,
or use byte array representations of cached entries instead of string.
Adding a test for this change is not possible, as there is no known
way to change the default encoding for filenames such as [2] (e.g.).
JGitTestUtil does write file contents through UTF-8, but encoding like
so does not apply to the actual file name. Hence there is no way to
create files with names properly made of special characters such as
[2]'s. And the test that is necessary for this case assumes such
Norwegian (or similar characters) filenames. Changing the default
locale programmatically in a test has no effect either. And changing
the LANG value passed to the JVM is only possible upon starting it.
[1] https://bugs.chromium.org/p/gerrit/issues/detail?id=9153
[2] <=>
(...)
"a/b/SíÒr-Norge.map",
"a/b/Sør-Norge.map",
(...)
Change-Id: Ib9f2f5297932337c9817064cc09d9f774dd168f4
Signed-off-by: Marco Miller <marco.miller@ericsson.com>
LFS: support merge/rebase/cherry-pick/diff/compare with LFS files
Respect merge=lfs and diff=lfs attributes where required to replace (in
memory) the content of LFS pointers with the actual blob content from
the LFS storage (and vice versa when staging/merging).
Does not implement general support for merge/diff attributes for any
other use case apart from LFS.
Change-Id: Ibad8875de1e0bee8fe3a1dffb1add93111534cae
Signed-off-by: Markus Duft <markus.duft@ssi-schaefer.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
This doesn't handle the really hard thing, which is merging spurious
conflicts inside .gitmodules files. That's OK: git.git doesn't
either. Users can resolve the conflict themselves and then commit
the merge.
Previously, jgit would crash when attempting to merge conflicting
submodule changes. Even if there was no conflict, after a merge which
adds submodules, the repository would have been missing empty
directories for newly-added submodules.
This patch fixes the crash, and adds the empty directories where
necessary. It ensures that the index is in a conflicted state when
submodule changes conflict.
Reported-by: Alexey Korobkov
Bug: 494551
Change-Id: I79db6798c2bdcc1159b5b2589b02da198dc906a1
Signed-off-by: David Turner <dturner@twosigma.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
This avoids having to re-read the merged file (twice even!) to
update the index.
Change-Id: Id13e0fd38906ed6f859604f86ca352761dca9ffe
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Commit fc7d407 corrected line endings for working tree files resulting
from merges when CRLF translations are to be done. However, that also
resulted in the file content being put as-is into the index, which is
wrong. The index must contain the file content with reverse CRLF
translations applied.
With core.autocrlf=true, the working tree file should have CR-LF, but
the index blob must still contain only LF.
Fix this oversight and apply the inverse translation when updating the
index, similar to what is done in AddCommand.
Bug: 499615
Change-Id: I3a33931318bdb580b2390f3450f91ea8f258a6a4
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Merges are performed using the raw text as stored in the git
repository. When we write the merge result, we must apply the
correct CRLF settings. Otherwise the line endings in the result
will be wrong.
Bug: 499615
Change-Id: I37a9b987e9404c97645d2720cd1c7c04c076a96b
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Reintroduce protected method which removal broke EMF Compare
So far we follow OSGi semantic versioning [1] which says the following:
"A change in the second (minor) part of the version signals that the
change is backward compatible with consumers of the API package but not
with the providers of that API. That is, when the API package goes from
version 1.5 to 1.6 it is no longer compatible with a provider of that
API but consumers of that API are backward compatible with that API
package."
The change Ib5fbf17bdaf727bc5d0e106ce88f2620d9f87a6f broke EMF Compare
which subclasses ResolveMerger since we added a new parameter to the
protected ResolveMerger.processEntry() method. According to the above
cited OSGi semantic versioning this is ok, implementers should expect
that they break on minor version changes of the API they implement.
This change reintroduces the old processEntry() method in order to help
avoid breakage for existing EMF Compare versions which expect breakage
also for the implementer case only for major version change (in this
case from JGit 4.x to 5.x).
[1] http://www.osgi.org/wp-content/uploads/SemanticVersioning1.pdf
See: https://dev.eclipse.org/mhonarc/lists/jgit-dev/msg03431.html
Change-Id: I48ba4308dee73925fa32d6c2fd6b5fd89632c571
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Make 'inCoreLimit' of LocalFile used in ResolveMerger configurable
This change makes it possible to configure the 'inCoreLimit' of LocalFile
used in ResolveMerger#insertMergeResult. Since LocalFile itself has some
risks, e.g. it may be left behind as garbage in case of failure. It should
be good to be able to control the size limit for using LocalFile.
Change-Id: I3dc545ade370b2bbdb7c610ed45d5dd4d39b9e8e
Signed-off-by: Changcheng Xiao <xchangcheng@google.com>
The merger is now able to react to the use of the merge attribute.
The value unset and the custom value 'binary' are handled (-merge
and merge=binary)
Since the specification of the merge attribute states that when the
attribute is unset, ours version must be kept in case of a conflict, we
don't overwrite the file but keep the local version.
Bug: 517128
Change-Id: Ib5fbf17bdaf727bc5d0e106ce88f2620d9f87a6f
Signed-off-by: Mathieu Cartaud <mathieu.cartaud@obeo.fr>
All that's really required to run a merge operation is a single
ObjectInserter, from which we can construct a RevWalk, plus a Config
that declares a diff algorithm. Provide some factory methods that don't
take Repository.
Change-Id: Ib884dce2528424b5bcbbbbfc043baec1886b9bbd
Enable and fix warnings about redundant specification of type arguments
Since the introduction of generic type parameter inference in Java 7,
it's not necessary to explicitly specify the type of generic parameters.
Enable the warning in Eclipse, and fix all occurrences.
Change-Id: I9158caf1beca5e4980b6240ac401f3868520aad0
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
This fixes the tests failed in JDK8.
FS uses java.nio API to get file attributes. The timestamps obtained
from that API are more precise than the ones from
java.io.File#lastModified() since Java8.
This difference accidentally makes JGit detect newly added files as
smudged. Use the precised timestamp to avoid this false positive.
Bug: 500058
Change-Id: I9e587583c85cb6efa7562ad6c5f26577869a2e7c
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
Fix calling of clean/smudge filters from Checkout,MergeCommands
When CheckoutCommand or MergeCommand is called then not in all situation
the treewalks have been prepared to support clean/smudge filters. Fix
this
Bug: 491505
Change-Id: Iab5608049221c46d06812552ab97299e44d59e64
Null-annotated Repository class and fixed related compiler errors
org.eclipse.jgit.lib.Repository class is an example of the API which
should be written with Java 8 java.util.Optional<T> type. Unfortunately
this API is already released and widely used. The good clients are
currently doing their best with checking return values for null and bad
clients do not know how bad their code is.
I've tried not to change any logic and to be as less intrusive as
possible. Most of the JGit code was well prepared to this, only few
classes needed some smaller fixes.
This change fixes all compiler errors in JGit and replaces possible
NPE's with either appropriate exceptions, avoiding multiple "Nullable
return" method calls or early returning from the method.
Because annotating getDirectory() and getFS() as Nullable would cause
lot of additional changes in JGit and EGit they are postponed.
Change-Id: Ie8369d2c9c5fac5ce83b3b1b9bc217d7b55502a3
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
Fix ResolveMerger when files should be replaced by folders
When during Merge for a certain path OURS & BASE contains a file and
THEIRS contains a folder there was a bug in JGit leading to unnecessary
conflicts. This commit fixes it and adds a test for this situation.
Bug: 472693
Change-Id: I71fac5a6a2ef926c01adc266c6f9b3275e870129
Also-by: Clemens Buchacher <drizzd@aon.at>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Use ANY_DIFF filter in ResolveMerger only for bare repositories
As Chris pointed out change I822721c76c64e614f87a080ced2457941f53adcd
slowed down merge since ANY_DIFF filter is much less efficient than the
manual detection of diffs done in ResolveMerger.processEntry() since it
avoids unnecessary filesystem calls using the git index. Hence only set
the ANY_DIFF filter on bare repositories which don't have a working tree
to scan.
To test performance I used the setup described in Chris' comment on
change I822721c76c64e614f87a080ced2457941f53adcd and modified
ResolveMerger.mergeTrees() to not add the working tree in order to
simulate merging in a bare repository.
At least on Mac I couldn't detect a speedup, with and without the
ANY_DIFF filter merge test takes an average 0.67sec.
Change-Id: I17b3a06f369cee009490f54ad1a2deb6c145c7cf
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
When reading back from an overflowed TemporaryBuffer the InputStream
must be closed to close the FileInputStream that is reading from
the backing file.
Change-Id: Id83d8f16f5b2c2618a9f841ec3508508455a6ae1
Use local GIT_DIR for overflow during merge conflicts
By writing the temporary overflow merge result to $GIT_DIR JGit
can ensure the same filesystem permissions apply to protect the
file contents.
If no directory is available from the repository (e.g. DfsRepository)
null will be passed and the system temporary directory will be used
instead.
Change-Id: I95532aa092676d18f1dc1e3fdbe6dcb1f91b782e
ResolveMerge only needs to visit differing TreeEntries
This should considerably speed up the treewalk on larger repositories.
Found by discussing new EGit API to support model merge in change
eda23bb556
Change-Id: I822721c76c64e614f87a080ced2457941f53adcd
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
cc: Laurent Delaigue <laurent.delaigue@obeo.fr>
The cleanUp path is trying to restore files that previously were
clean, but were overwritten in the work tree by a partial merge
attempt that has failed and needs to be aborted. Reuse the checkout
logic to write the file content and refresh the stat data.
Change-Id: I320d33b3744daf88d3155db99e957408937ddd00
Entries should only be written to the working tree managed by the
Repository. Simplify callers by passing only the entry and computing
the work tree location inside of the checkoutEntry method.
Change-Id: I574e41280d0407f1853fda12f4bd0d30f75d74e7
ResolveMerger: Use the ObjectReader to access objects
This is necessary to ensure objects accessed by the TreeWalk come from
the associated ObjectInserter when the merger is a RecursiveMerger
instance and a virtual common base was constructed but not flushed.
Change-Id: Iebe739d30fd868ebc4f61dbfb714673146a2c3ec
The base Merger class already has a single ObjectReader instance that
it handles releasing as necessary, so creating new readers is not
necessary.
Change-Id: I990ec43af7df448c7825fc1b10e62eadaa3e0c2a
Process most in-core merges without local temp files
Instead of always writing to disk use TemporaryBuffer.LocalFile to
store up to 10 MiB of merge result in RAM. Most source code will
fit into this limit, avoiding local disk IO for simple merges.
Larger files will automatically spool to a temporary file that
can be cleaned up in the finally, reducing the risk of leaving
them on disk and consuming space in /tmp.
Change-Id: Ieccbd9b354d4dd3d2bc1304857325ae7a9f34ec6
ResolveMerger: push result file creation into updateIndex()
The only caller of writeMergedFile is updateIndex, and the only
user of this path object is the code within the method. This is
a no-op change that opens the door to refactoring the way temp
files are handled for inCore merges.
Change-Id: I863a303194689a806b667e55eb958e1decf046c1
Fix API errors raised on ResolveMerger affecting API providers only
In change If45bc3d078b3d3de87b758e71d7379059d709603 a new parameter was
added to 3 protected methods of ResolveMerger. This breaks the code of
developers which have subclassed ResolveMerger. The API baseline check
in Eclipse reports this as API breakage.
Since this will break only providers but not consumers of the API this
should be allowed also in minor versions. According to OSGi semantic
versioning
http://www.osgi.org/wiki/uploads/Links/SemanticVersioning.pdf
breaking providers in a minor version update is ok.
Therefore silence these errors using API filter rules.
Bug: 440757
Change-Id: Icabbd0e1de7e877c66a5c4a2c8391473f992a1aa
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
RecursiveMerger should not fail on content-merge conflicts of parents
Previously when RecursiveMerger was trying to create a single virtual
common base for the merge it was failing when this lead to content-merge
conflicts. This is different from what native git is doing. When native
git's recursive merge algorithm creates a new common base it will merge
the multiple parents and simply take the merge result (potentially
including conflict markers) as common base. See my discussion with Shawn
here: http://www.spinics.net/lists/git/msg234959.html :
> - How should workingtree, index (stage1,2,3) look like if during
that
> merge of common ancestors a conflict occurs? Will I see in stage2
and
> stage3 really see content of X1 and X2?
Its done entirely in memory and never touches the working tree or
index. When a conflict exists in the X1-X2 merge the conflict is
preserved into the new virtual base.
There is still the possibility that the merge of parents lead to
conflicts. File/Folder conclicts, conflicts on filemodes. This commit
only fixes the situation for conflicts when merging content.
Bug: 438203
Change-Id: If45bc3d078b3d3de87b758e71d7379059d709603
This fixes a case where we have CRLF in the repo but
LF in the worktree and are in autocrlf mode.
Change-Id: I0388270c1cf0fd22dfd513bcaa404eb97268d39d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
The change includes comparing symbolic links between disk and index,
adding symbolic links to the index, creating/modifying links on
checkout. The behavior is controlled by the core.symlinks setting, just
as C Git does. When a new repository is created core.symlinks will be
set depending on the capabilities of the operating system and Java
runtime.
If core.symlinks is set to true, the assumption is that symlinks are
supported, which may result in runtime errors if this turns out not to
be the case.
Measuring the cost of jgit status on a repository with ~70000 files,
of which ~30000 are tracked reveals a penalty of about 10% for using
the Java7 (really NIO2) support module.
Bug: 354367
Change-Id: I12f0fdd9d26212324a586896ef7eb1f6ff89c39c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
This fixes two cases:
- A folder without tracked content exist both in the workdir and merged
commit, as long as there names within that folder does not conflict.
- An empty folder structure exists with the same name as a file in the
merged commit.
Bug: 402834
Change-Id: I4c5b9f11313dd1665fcbdae2d0755fdb64deb3ef
Extend ResolveMerger with RecursiveMerger to merge two tips
that have up to 200 bases.
Bug: 380314
CQ: 6854
Change-Id: I6292bb7bda55c0242a448a94956f2d6a94fddbaa
Also-by: Christian Halstrick <christian.halstrick@sap.com>
Signed-off-by: Chris Aniszczyk <zx@twitter.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
A few classes such as Constanrs are marked with @SuppressWarnings, as are
toString() methods with many liternal, but otherwise $NLS-n$ is used for
string containing text that should not be translated. A few literals may
fall into the gray zone, but mostly I've tried to only tag the obvious
ones.
Change-Id: I22e50a77e2bf9e0b842a66bdf674e8fa1692f590
ResolveMerger throws a MissingObjectException when it encounters
a submodule conflict while merging. The reason is that it treats
the submodule link as a blob and tries to read its contents.
We solve the issue by detecting before content merge whether the
path to be merged is a submodule link, and skip the content
merge if it is.
Bug: 389238
Change-Id: I9a58dfc7716b28a21f5c04cf3a865091ae8dfe7e
Signed-off-by: Tommi Siivola <tommi.siivola@eficode.com>
Again teach ResolveMerger to create more correct DirCacheEntry's
Currently, after a merge/cherry-pick/rebase, all index entries are
smudged as the ResolveMerger never sets entry lengths and/or
modification times. This change teaches it to re-set them at least for
things it did not touch. The other entries are then repaired when the
index is persisted, or entries are checked out.
The first attempt to get this in was commit
3ea694c252 which has been reverted.
Since then some fixes to ResolveMerger and a few more tests have
been added which check situations where the index is not matching
HEAD before we merge.
Change-Id: I648fda30846615b3bf688c34274c6cf4bc857832
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Also-by: Markus Duft <markus.duft@salomon.at>
Matthias Sohn
David Pursehouse
Thomas Wolf
Marco Miller
Han-Wen Nienhuys
Markus Duft
David Turner
Changcheng Xiao
Mathieu Cartaud
Dave Borowitz
Masaya Suzuki
Christian Halstrick
Andrey Loskutov
Shawn Pearce
Shawn Pearce
Robin Stocker
Laurent Goubet
Robin Rosenberg
George C. Young
Tommi Siivola