ObjectChecker: Disallow names potentially mapping to ".git" on HFS+
Mac's HFS+ folds concatentations of ".git" and ignorable Unicode
characters [1] to ".git" [2]. Hence we need to disallow all names which
could potentially be a shortname for ".git". Example: in an empty
directory create a folder ".g\U+200Cit". Now you can't create another
folder ".git".
The following characters are ignorable Unicode which are ignored on
HFS+:
unicode hex name
-------------------------------------------------
U+200C 0xe2808c ZERO WIDTH NON-JOINER
U+200D 0xe2808d ZERO WIDTH JOINER
U+200E 0xe2808e LEFT-TO-RIGHT MARK
U+200F 0xe2808f RIGHT-TO-LEFT MARK
U+202A 0xe280aa LEFT-TO-RIGHT EMBEDDING
U+202B 0xe280ab RIGHT-TO-LEFT EMBEDDING
U+202C 0xe280ac POP DIRECTIONAL FORMATTING
U+202D 0xe280ad LEFT-TO-RIGHT OVERRIDE
U+202E 0xe280ae RIGHT-TO-LEFT OVERRIDE
U+206A 0xe281aa INHIBIT SYMMETRIC SWAPPING
U+206B 0xe281ab ACTIVATE SYMMETRIC SWAPPING
U+206C 0xe281ac INHIBIT ARABIC FORM SHAPING
U+206D 0xe281ad ACTIVATE ARABIC FORM SHAPING
U+206E 0xe281ae NATIONAL DIGIT SHAPES
U+206F 0xe281af NOMINAL DIGIT SHAPES
U+FEFF 0xefbbbf ZERO WIDTH NO-BREAK SPACE
[1] http://www.unicode.org/versions/Unicode7.0.0/ch05.pdf#G40025http://www.unicode.org/reports/tr31/#Layout_and_Format_Control_Characters
[2] http://dubeiko.com/development/FileSystems/HFSPLUS/tn1150.html#UnicodeSubtleties
Change-Id: Ib6a1dd090b2649bdd8ec16387c994ed29de2860d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Windows creates shortnames for all non-8.3 files (see [1]). Hence we
need to disallow all names which could potentially be a shortname for
".git". Example: in an empty directory create a folder "GIT~1". Now you
can't create another folder ".git".
The path "GIT~1" may map to ".git" on Windows. A potential victim to
such an attack first has to initialize a git repository in order to
receive any git commits. Hence the .git folder created by init will get
the shortname "GIT~1". ".git" will only get a different shortname if the
user has created a file "GIT~1" before initialization of the git
repository.
[1] http://en.wikipedia.org/wiki/8.3_filename
Change-Id: I9978ab8f2d2951c46c1b9bbde57986d64d26b9b2
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Windows treats "foo." and "foo " as "foo". The ".git" directory is
special, as it contains metadata for a local Git repository. Disallow
variations that Windows considers to be the same.
Change-Id: I28eb48859a95a89111b4987c91de97557e3bb539
Always ignore case when forbidding .git in ObjectChecker
The component name ".GIT" inside a tree entry could confuse a
case insensitive filesystem into looking at a submodule and
not a directory entry.
Disallow any case permutations of ".git" to prevent this
confusion from entering a repository and showing up at a
later date on a case insensitive system.
Change-Id: Iaa3f768931d0d5764bf07ac5f6f3ff2b1fdda01b
If the DirCache contains a path that is known to be invalid, refuse to
read the DirCache into memory. This avoids confusing errors later if
an invalid path read from the DirCache were to be passed into a new
DirCacheEntry constructor.
Change-Id: Ic033d81e23a5fbd554cc4dff80a232504562ffa8
DirCache: Replace isValidPath with DirCacheCheckout.checkValidPath
isValidPath is an older simple form of the validation performed by
checkValidPath. Use the latter as it more consistently matches
git-core's validation rules.
By running the same validation as fsck, callers creating an entry
for the DirCache are more likely to learn early they are trying
to build trees that will fail fsck.
Change-Id: Ibf5ac116097156aa05c18e231bc65c0854932eb1
Windows does not like naming files "a.". The trailing "." may be
dropped by the filesystem, which is confusing. Even though these
tests currently do not write to disk, future tests like them might.
Replace "." with "-", which has the same sorting properties that
were desirable about ".", but does not have the same limitations.
Change-Id: Ie5b7594bf5e79828d1341883c73ddb70123d5055
Deprecate TemporaryBuffer.LocalFile without parent directory
Encourage callers to explicitly name a directory to hold any
overflow data. Call sites have more information about what is
going into the buffer and how it should be protected at the
filesystem level than just throwing content to the system wide
temporary directory.
Callers that still really don't care (or need to care) can pass
null for the File argument to have the system directory used.
Change-Id: I89009bbee49d3850d42cd82c2c462e51043acda0
Increase the in-memory buffer for the TREE extension to 5 MiB, and
overflow to $GIT_DIR instead of /tmp. Using a larger buffer reduces
the chances a repository will overflow and need to spool the extension
to disk. Using $GIT_DIR allows the TREE extension contents to have
the same file system protections as the final $GIT_DIR/index.
Wrap the entire thing in a try/finally to ensure the temp file is
deleted from disk after the block has finished using it. To avoid
dangling NFS files, LocalFile.destroy() does close the local file
before deleting it.
Change-Id: I8f871181a4689e3ebf0cdd4fd1769333cf7546c3
When writing new packs it should be allowed to specify objects as "have"
(objects which should not be included in the pack) which do not exist in
the local repository.
This works with the traditional PackWriter, but when PackWriter was
working on a repository with bitmap indexes and used
PackWriterBitmapWalker then this feature was broken. Non-existing "have"
objects lead to MissingObjectExceptions. That broke push and Gerrit
replication. When the replication target had branches unknown to the
replication source then the source repository wanted to build pack files
where "have" included branch-tips which were unknown in the source
repository.
Bug: 427107
Change-Id: I6b6598a1ec49af68aa77ea6f1f06e827982ea4ac
Also-by: Matthias Sohn <matthias.sohn@sap.com>
Set the commit message to be used for the merge commit (in case one is
created)
Bug: 442886
Change-Id: Ie5ecc13822faa366f00b3daa07f74c8441cae195
Signed-off-by: Axel Richard <axel.richard@obeo.fr>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Let ObjectWalk.markUninteresting also mark the root tree as
uninteresting
Using the ObjectWalk and marking a commit as uninteresting didn't mark
its root tree as uninteresting. This caused the "missing tree ..."
error in Gerrit under special circumstances. For example, if the
patch-set 2 changes only the commit message then the patch-set 1
and patch-set 2 share the same root-tree:
ps1 -> o o <- ps2
\ /
o root-tree
The transported pack will contain the ps2 commit but not the root-tree
object.
When using the BaseReceivePack.setCheckReferencedObjectsAreReachable
JGit will check the reachability of all referenced objects not provided
in the transported pack. Since the ps1 was advertised it will properly
be marked as uninteresting. However, the root-tree was reachable because
the ObjectWalk.markUninteresting missed to mark it as uninteresting.
JGit was then rejecting the pack with the "missing tree ..." exception.
Gerrit-issue: https://code.google.com/p/gerrit/issues/detail?id=1582
Change-Id: Iff2de8810f14ca304e6655fc8debeb8f3e20712b
Signed-off-by: Saša Živkov <sasa.zivkov@sap.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Strip "<", ">", and "\n" from name/email in UserConfig
This matches what C Git does, see "stripped" in `man git-commit-tree`.
It also fixes the bug of the user where an user.email like "<>" would
show up as "<<>>" in EGit.
Bug: 439844
Change-Id: I567a3c620e191ce9d37d318417e63cb5d4483419
Signed-off-by: Robin Stocker <robin@nibor.org>
Revert "Add a method to DfsOutputStream to read as an InputStream"
This reverts commit b646578d89.
openInputStream() is never used in JGit, nor is it used by any
known working DFS implementation. The method was added as a
utility for reading back from a DfsInserter, but the final
implementation of that feature does not requrire this method.
Change-Id: I075ad95e40af49c92b554480f8993ef5658f7684
Add a method to ObjectInserter to read back inserted objects
In the DFS implementation, flushing an inserter writes a new pack to
the storage system and is potentially very slow, but was the only way
to ensure previously-inserted objects were available. For some tasks,
like performing a series of three-way merges, the total size of all
inserted objects may be small enough to avoid flushing the in-memory
buffered data.
DfsOutputStream already provides a read method to read back from the
not-yet-flushed data, so use this to provide an ObjectReader in the
DFS case.
In the file-backed case, objects are written out loosely on the fly,
so the implementation can just return the existing WindowCursor.
Change-Id: I454fdfb88f4d215e31b7da2b2a069853b197b3dd
Instead of passing on the start point as is to CreateBranchCommand, the
resolved ObjectId was used. Given this, CreateBranchCommand did not set
up tracking.
This also fixes CreateBranchCommand with setStartPoint(null) to use HEAD
(instead of NPEing), as documented in the Javadoc.
Bug: 441153
Change-Id: I5ed82b4a4b4a32a81a7fa2854636b921bcb3d471
Signed-off-by: Robin Stocker <robin@nibor.org>
Add IndexDiff tests for merge conflict state BOTH_ADDED
JGit handled this case improperly which these tests demonstrate. Fixed
by I25915880f304090fe90584c79bddf021231227a2.
Bug: 440537
Change-Id: Ia29c1d6cf8c0ce724cc3ff5ed9e0b396949b44bf
Signed-off-by: Laurent Goubet <laurent.goubet@obeo.fr>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
IndexDiffFilter should never filter entries with stage!=0
If the IndexDiffFilter is asked whether it should include or filter out
a certain path and for that path there is a dircache entry with a stage
different from 0, then the filter should never filter out this entry.
IndexDiffFilter is an optimized version of AnyDiffFilter and there is no
case where the index contains non-0 stages but we still don't see any
diff for that path.
Change-Id: I25915880f304090fe90584c79bddf021231227a2
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Fix RevertCommand to correctly revert multiple commits at once.
The fix is to move the new head commit to the newly-created revert
commit, so that additional revert commits will use the correct head.
Change-Id: I5de3a9a2a4c276e60af732e9c507cbbdfd1a4652
Signed-off-by: Maik Schreiber <blizzy@blizzy.de>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Apparently repo allows projects overlapping, e.g. one project's path is "foo"
and another project's path is "foo/bar". This is not supported in git submodule.
At JGit repo side we'll skip all the submodules that are in subdirectories of
other submodules, and on repo side we'll make them submodules to resolve this
problem.
Change-Id: I6820c4ef400c530a36150b1228706adfcc43ef64
Signed-off-by: Yuxuan 'fishy' Wang <fishywang@google.com>
RecursiveMerger should not fail on content-merge conflicts of parents
Previously when RecursiveMerger was trying to create a single virtual
common base for the merge it was failing when this lead to content-merge
conflicts. This is different from what native git is doing. When native
git's recursive merge algorithm creates a new common base it will merge
the multiple parents and simply take the merge result (potentially
including conflict markers) as common base. See my discussion with Shawn
here: http://www.spinics.net/lists/git/msg234959.html :
> - How should workingtree, index (stage1,2,3) look like if during
that
> merge of common ancestors a conflict occurs? Will I see in stage2
and
> stage3 really see content of X1 and X2?
Its done entirely in memory and never touches the working tree or
index. When a conflict exists in the X1-X2 merge the conflict is
preserved into the new virtual base.
There is still the possibility that the merge of parents lead to
conflicts. File/Folder conclicts, conflicts on filemodes. This commit
only fixes the situation for conflicts when merging content.
Bug: 438203
Change-Id: If45bc3d078b3d3de87b758e71d7379059d709603
Fix RecursiveMerger's internal use of merge to find a merge base
When RecursiveMerger tried to determine a common base tree it was
recursively tried to merge multiple common bases. But these intermediate
merges which have just been done to determine a single common base for
the final merge already filled some important fields (toBeCheckedOut,
toBeDeleted, ...). These side effects of the intermediate merges led to
wrong results of the final merge. One symptom was that after a recursive
merge which should be succesful you could still see leftover files in
the worktree: files which existed in the (virtual) common base but which
don't exist anymore in the branches to be merged.
The solution is easy: Clear the appropriate fields after common base
determination and start the final merge with a clean state.
Change-Id: I644ea9e1cb15360f7901bc0483cdb9286308c226
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
As described in native gits file "git-read-tree.txt" git has in a
special mode when doing the "initial" checkout. "Initial" means that the
index is empty before the checkout. This was not handled correctly in
JGit and is fixed in this commit. Also see
https://github.com/git/git/blob/master/Documentation/git-read-tree.txt#L181
Change-Id: I9b9d1bd9ebf349cfca420c891c7b099a18d07ba4
Preserve merges during pull if configured to do so
Setting branch.<name>.rebase or pull.rebase to 'preserve' will preserve
merges during rebase. Also, pull.rebase is now consulted if there is no
branch-specific configuration.
Bug: 429664
Change-Id: I345fa295c7e774e0d0a8e6aba30fbfc3552e0084
Signed-off-by: Konrad Kügler <swamblumat-eclipsebugs@yahoo.de>
With --preserve-merges C Git re-does merges using the rewritten merge
parents, discarding the old merge commit. For the common use-case of
pull with rebase this is unfortunate, as it loses the merge conflict
resolution (and other fixes in the merge), which may have taken quite
some time to get right in the first place.
To overcome this we use a two-fold approach:
If any of the (non-first) merge parents of a merge were rewritten, we
also redo the merge, to include the (potential) new changes in those
commits.
If only the first parent was rewritten, i.e. we are merging a branch
that is otherwise unaffected by the rebase, we instead cherry-pick the
merge commit at hand. This is done with the --mainline 1 and --no-commit
options to apply the changes introduced by the merge. Then we set up an
appropriate MERGE_HEAD and commit the result, thus effectively forging a
merge.
Apart from the approach taken to rebase merge commits, this
implementation closely follows C Git. As a result, both Git
implementations can continue rebases of each other.
Preserving merges works for both interactive and non-interactive rebase,
but as in C Git it is easy do get undesired outcomes with interactive
rebase.
CommitCommand supports committing merges during rebase now.
Bug: 439421
Change-Id: I4cf69b9d4ec6109d130ab8e3f42fcbdac25a13b2
Signed-off-by: Konrad Kügler <swamblumat-eclipsebugs@yahoo.de>
Handle diff formatting when there is nothing to compare with
DiffFormatter now suports either side being null and the log program
will output the diff for the first commit.
Bug: 395791
Change-Id: I378957b57e9ad1f7195ba416f402178453f0ebd3
Allow retrying connecting SshSession in case of an exception
Connecting to an SshSession may fail due to different reasons. Jsch for
example often throws an com.jcraft.jsch.JschException: verify: false.[1]
The issue is still not fixed in JSch 0.1.51.
In such a case it is worth retrying to connect. The number of connection
attempts can be configured using ssh_config parameter
"ConnectionAttempts" [2].
Don't retry if the user canceled authentication.
[1] http://sourceforge.net/p/jsch/bugs/58/
[2] http://linux.die.net/man/5/ssh_config
Bug: 437656
Change-Id: I6dd2a3786b7d3f15f5a46821d8edac987a57e381
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Rename RewriteTreeFilter to TreeRevFilter and make it public
The current behavior of passing a TreeFilter to RevWalk has limited
usefulness, since the RevFilter derived from the TreeFilter is always
ANDed together with any other RevFilters. It is also tied fairly
tightly to the parent rewriting mechanism.
Make TreeRevFilter a generic RevFilter that matches modified paths
against any TreeFilter. This allows for more complex logic like
(modified this path OR authored by this person).
Leave the rewrite flag logic in this class, since it's closely tied to
the parent comparison code, but hidden behind a protected constructor.
Change-Id: Ia72ef591a99415e6f340c5f64583a49c91f1b82f
Unit test for the updating behavior in RepoCommand API.
When update the manifest against a bare repository, RepoCommand will replace
every existing content from the repository with contents populated from the
manifest. Added note for that and a unit test to make sure this behavior.
Change-Id: I1d5960e84bca5aa2a4e86f424d2ddd4197894cdc
Signed-off-by: Yuxuan 'fishy' Wang <fishywang@google.com>
Remove a bit of repetition and casting in PackFileTest
PackFileTest lives in o.e.j.internal.storage.file, so I think it's OK for
it to acknowledge the existence of FileRepository in order to avoid some
unnecessary casting, and probably nicer to avoid the repetition too.
Change-Id: I0de592a32f6178e6d6bf114848101e185b3111a1
Signed-off-by: Roberto Tyley <roberto.tyley@gmail.com>
Allow to include untracked files in stash operations.
Unstashed changes are saved in a commit which is added as an additional
parent to the stash commit.
This behaviour is fully compatible with C Git stashing of untracked
files.
Bug: 434411
Change-Id: I2af784deb0c2320bb57bc4fd472a8daad8674e7d
Signed-off-by: Andreas Hermann <a.v.hermann@gmail.com>
Fix for reflog corruption caused by multiline message
If a client passes a multiline message as argument to ReflogWriter.log()
the Reflog gets corrupted and cannot be parsed. ReflogWriter.log() is
invoked implicitly from various commands such as StashCreate, Rebase and
many more. However the message is not always filtered for line feeds.
Such an example is the StashCreateOperation of EGit which passes
unchecked user input as commit message. If a multiline comment is pasted
to the stash create dialog, the reflog gets corrupted.
ReflogWriter now replaces line endings in log message with spaces.
Bug: 435509
Change-Id: I3010cc902e13bee4d7b6696dfd11ab51062739d3
Signed-off-by: Andreas Hermann <a.v.hermann@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
The aim of this change is to place all commits of a branch on the same
lane and commits of other (side) branches on different lanes.
The algorithm treats first parents of a commit specially by placing them
on the same lane as the commit itself. When a commit is the first parent
of multiple children it could be placed on any of these children's
lanes. In this case it is placed on the longest child lane, as this is
usually the lane of the branch the commit actually was made on.
Other (non-first) parents are placed on new lanes. This creates a layout
that should make it easier to see branches and merges and follow linear
branch histories.
This differs from the previous approach, which sometimes plotted the
commits of a side branch on the same lane as the base branch commits and
further commits on the base branch appeared on a different lane.
This made the base branch appear as if it was the side branch and
the side branch appears to be the base branch.
In addition to lane assignment, also the plotting code changed to start
drawing a branch lane from the commit where it forks out. Previously it
started only when the first commit on the branch appeared.
Active lanes are continued with every commit that is processed.
Previously lanes were only continued when the next commit on the lane
was encountered. This could produce (temporarily) dangling commits if
the next commit on the lane was not processed yet.
CQ: 8299
Bug: 419359
Bug: 434945
Change-Id: Ibe547aa24b5948ae264f7d0f56a492a4ef335608
Signed-off-by: Konrad Kügler <swamblumat-eclipsebugs@yahoo.de>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Adds support for Negotiate(SPNEGO) HTTP authentication method. This method
is set to have a higher priority as Digest HTTP authentication method.
Bug: 428836
Change-Id: Ib181096d39f538df1dd7d3f36516843777bf12ae
Signed-off-by: Laurent Goujon <lgoujon@twitter.com>
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
By specifying a mainline parent, a merge is cherry picked as if this
parent was its only parent. If no mainline parent is given, cherry
picking merges is not allowed, as before.
Change-Id: I391cb73bf8f49e2df61428c17b40fae8c86a8b76
Signed-off-by: Konrad Kügler <swamblumat-eclipsebugs@yahoo.de>
Adds further tests where the working tree is dirty (differs from
index) and where we have staged but uncommitted changes.
Fixed the test case 9 for file/directory conflicts.
Bug: 428819
Change-Id: Ie44a288b052abe936ebb74272d0fefef3b218a7a
Signed-off-by: Axel Richard <axel.richard@obeo.fr>
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Archive: Add the ability to select one or more paths.
Previously, it was only possible to archive the entire repository.
This patch introduces the ability to select specific files and
directories to archive.
Archiving the entire repository remains the default behaviour.
org.eclipse.jgit.api.ArchiveCommand: Adding setPaths(String... paths)
method.
Change-Id: Iedcd40fbfd71238b0088174bbe2717fae196e047
Signed-off-by: Shaul Zorea <shaulzorea@gmail.com>
Matthias Sohn
Christian Halstrick
Shawn Pearce
Shawn Pearce
Konrad Kügler
Axel Richard
Saša Živkov
Robin Stocker
Yuxuan 'fishy' Wang
Dave Borowitz
Alexey Kuznetsov
Laurent Goubet
Maik Schreiber
Robin Rosenberg
Stefan Lay
Roberto Tyley
Andreas Hermann
Laurent Goujon
Shaul Zorea