If we hit a corrupt object or invalid pack remove the pack from the pack
list. Other IOException could be transient hence we should not remove
the pack from the list to avoid the problem reported on the Gerrit list
[1]. It looks like in the reported case the pack was removed from the
pack list causing MissingObjectExceptions which disappear when the
server is restarted.

[1] https://groups.google.com/forum/#!topic/repo-discuss/Qdmbl-YZ4NU

Change-Id: I331626110d54b190e46cddc2c40f29ddeb9613cd
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

This should help to identify the root cause of the problem discussed on
the Gerrit list [1].

[1] https://groups.google.com/forum/#!topic/repo-discuss/Qdmbl-YZ4NU

Change-Id: I871f70e4bb1227952e1544b789013583b14e2b96
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

C Git trims name and email before inserting them into the commit header
so that " A U Thor  " and "  author@example.com " becomes 
"A U Thor <author@example.com>" with a single separating space.

This changes PersonIdent#toExternalString() to trim name and email
before concatenating them.

Change-Id: Idd77b659d0db957626824f6632e2da38d7731625
Signed-off-by: Rüdiger Herrmann <ruediger.herrmann@gmx.de>

Native Git canonicalizes line endings when detecting
renames, more specifically it replaces CRLF by LF.
See: hash_chars in diffcore-delta.c

Bug: 449545
Change-Id: Iec2aab12ae9e67074cccb7fbd4d9defe176a0130
Signed-off-by: Marc Strapetz <marc.strapetz@syntevo.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

The path matcher should not fail if the rule ends with trailing slash,
target pattern does not ends with the slash and the "assumeDirectory"
flag is set.

E.g. */bin/ should also match a/bin if this pattern is threated as
directory by WorkingTreeIterator (FileMode.TREE).

The old code/tests have never tested directory rules with patterns
*without* trailing slashes but with the "assumeDirectory" flag set.
Unfortunately this is exactly what WorkingTreeIterator does... The tests
are changed to test *both* cases now (with trailing slash and without)
if the target pattern has trailing slash (represents directory).

Bug: 454672
Change-Id: I621c1644d9e94df3eb9f6f09c6de0fe51f0950a4
Also-by: Markus Duft <markus.duft@salomon.at>
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>

SystemReader used a chached ObjectChecker which was instantiated only
once. But in case of unit tests where we can change the platform
dynamically (e.g. MockSystemReader.setWindows()) this is wrong and
caused DirCacheCheckoutMaliciousPathTest.
testMaliciousAbsoluteCurDrivePathWindowsOnUnix() to fail. This change
allows 
user of SystemReader to force the creation of a new ObjectChecker. 
MockSystemReader.setWindows() and .setUnix() make use of this feature.

Change-Id: I87458d1dc63c1f5c18979f972b1c1f0d670a9ed8

Mac's HFS+ folds concatentations of ".git" and ignorable Unicode
characters [1] to ".git" [2]. Hence we need to disallow all names which
could potentially be a shortname for ".git". Example: in an empty
directory create a folder ".g\U+200Cit". Now you can't create another
folder ".git".

The following characters are ignorable Unicode which are ignored on
HFS+:

unicode   hex         name
-------------------------------------------------
U+200C    0xe2808c    ZERO WIDTH NON-JOINER
U+200D    0xe2808d    ZERO WIDTH JOINER
U+200E    0xe2808e    LEFT-TO-RIGHT MARK
U+200F    0xe2808f    RIGHT-TO-LEFT MARK
U+202A    0xe280aa    LEFT-TO-RIGHT EMBEDDING
U+202B    0xe280ab    RIGHT-TO-LEFT EMBEDDING
U+202C    0xe280ac    POP DIRECTIONAL FORMATTING
U+202D    0xe280ad    LEFT-TO-RIGHT OVERRIDE
U+202E    0xe280ae    RIGHT-TO-LEFT OVERRIDE
U+206A    0xe281aa    INHIBIT SYMMETRIC SWAPPING
U+206B    0xe281ab    ACTIVATE SYMMETRIC SWAPPING
U+206C    0xe281ac    INHIBIT ARABIC FORM SHAPING
U+206D    0xe281ad    ACTIVATE ARABIC FORM SHAPING
U+206E    0xe281ae    NATIONAL DIGIT SHAPES
U+206F    0xe281af    NOMINAL DIGIT SHAPES
U+FEFF    0xefbbbf    ZERO WIDTH NO-BREAK SPACE

[1] http://www.unicode.org/versions/Unicode7.0.0/ch05.pdf#G40025
    http://www.unicode.org/reports/tr31/#Layout_and_Format_Control_Characters
[2] http://dubeiko.com/development/FileSystems/HFSPLUS/tn1150.html#UnicodeSubtleties

Change-Id: Ib6a1dd090b2649bdd8ec16387c994ed29de2860d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

Windows creates shortnames for all non-8.3 files (see [1]). Hence we
need to disallow all names which could potentially be a shortname for
".git". Example: in an empty directory create a folder "GIT~1". Now you
can't create another folder ".git".

The path "GIT~1" may map to ".git" on Windows. A potential victim to
such an attack first has to initialize a git repository in order to
receive any git commits. Hence the .git folder created by init will get
the shortname "GIT~1". ".git" will only get a different shortname if the
user has created a file "GIT~1" before initialization of the git
repository.

[1] http://en.wikipedia.org/wiki/8.3_filename

Change-Id: I9978ab8f2d2951c46c1b9bbde57986d64d26b9b2
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

Windows treats "foo." and "foo " as "foo". The ".git" directory is
special, as it contains metadata for a local Git repository. Disallow
variations that Windows considers to be the same.

Change-Id: I28eb48859a95a89111b4987c91de97557e3bb539

The component name ".GIT" inside a tree entry could confuse a
case insensitive filesystem into looking at a submodule and
not a directory entry.

Disallow any case permutations of ".git" to prevent this
confusion from entering a repository and showing up at a
later date on a case insensitive system.

Change-Id: Iaa3f768931d0d5764bf07ac5f6f3ff2b1fdda01b

If the DirCache contains a path that is known to be invalid, refuse to
read the DirCache into memory.  This avoids confusing errors later if
an invalid path read from the DirCache were to be passed into a new
DirCacheEntry constructor.

Change-Id: Ic033d81e23a5fbd554cc4dff80a232504562ffa8

isValidPath is an older simple form of the validation performed by
checkValidPath. Use the latter as it more consistently matches
git-core's validation rules.

By running the same validation as fsck, callers creating an entry
for the DirCache are more likely to learn early they are trying
to build trees that will fail fsck.

Change-Id: Ibf5ac116097156aa05c18e231bc65c0854932eb1

When updating a submodule (e.g. during recursive clone) the repository
for the submodule should be located at <gitdir>/modules/<submodule-path>
whereas the working tree of the submodule should be located at
<working-tree>/<submodule-path> (<gitdir> and <working-tree> are
associated to the containing repository). Since CloneCommand has learned
about specifying a separate gitdir this is easy to implement in
SubmoduleUpdateCommand.

Change-Id: I9b56a3dfa50f97f6975c2bb7c97b36296f331b64

This feature is needed to support the new submodule layout where the
.git folder of the submodules is under .git/modules/<submodule>.

Change-Id: If5f13426cfd09b7677e23478e9700c8c25a6dae5

Native git's "init" command allows to specify the location of the .git
folder with the option "--separate-git-dir". This allows for example to
setup repositories with a non-standard layout. E.g. .git folder under
/repos/a.git and the worktree under /home/git/a. Both directories
contain pointers to the other side: /repos/a.git/config contains
core.worktree=/home/git/a . And /home/git/a/.git is a file containing
"gitdir: /repos/a.git". This commit adds that option to InitCommand.
This feature is needed to support the new submodule layout where the
.git folder of the submodules is under .git/modules/<submodule>.

Change-Id: I0208f643808bf8f28e2c979d6e33662607775f1f

Without explicitly closing repos we can't delete the test repositories
on windows.

Change-Id: Id5fa17bd764cbf28703c2f21639d7e969289c2d6
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

RepoCommand opend two repos without closing them or returning them to
the caller. This caused certain tests to fail on Windows.

Change-Id: Ia04924aaaad4d16f883b06404c2a85d3f801231f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

If the specified ref can't be resolved we should throw a
RefNotFoundException instead of an NPE.

Bug: 455005
Change-Id: I6ec9bf1c0f330eea5eb8277268f62663bdf58f66
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

To update the file length stat we need to use the length of the
temporary file since it's not yet renamed to the target file name here.
The incorrect file length stat update was introduced in
a606dc363d.

Bug: 453962
Change-Id: I715c048227553efae6f8f6b6878c0f04f2609d9c
Also-by: Konrad Kügler <swamblumat-eclipsebugs@yahoo.de>
Also-by: Christian Halstrick <christian.halstrick@sap.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

In case of an add/add conflict, no base stage exists. The previous
implementation would skip over the entries because the condition
expected the base stage to always exist.

Change-Id: Ie2b3685d958c09b241991b74e6177401e8a1ebc9
Signed-off-by: Robin Stocker <robin@nibor.org>

ObjectDirectory.searchPacksAgain() should always read trustFolderStat
from the config and not rely on a cached value.

Change-Id: I90edbaae3c64eea0c9894d05acde4267991575ee

receive.denyDeletes parameter refers only to branch:
http://thread.gmane.org/gmane.comp.version-control.git/99746

Bug: 412640
Change-Id: Ief3fa6abc0c9a18ba0a671ff9854432cec480c4f

The bulk of the "is this sane" logic is inside of ObjectChecker.  The
only caller for the version in DirCacheCheckout is an obtuse usage for
the static isValidRefName() method in Repository.

Deprecate the weird single use method in DirCacheCheckout and move all
code for checking a sequence of path components into ObjectChecker,
where it makes sense alongside the existing code that checks a single
component at a time.

Reuse a single ObjectChecker for the local platform, to avoid looking
up the system properties on each path string considered.

Change-Id: Iae6e769f2bfcad05c166e70ff255f9cf9fcdc87e

If the text extent height of a to be rendered plot line is odd, then the
SWTPlotRenderer cannot calculate the correct Y position for drawing the
label and draws the label with a 1 pixel offset. SWT text drawing uses
the baseline as Y coordinate. Due to the given centerline API in the
AbstractPlotRenderer the overall calculation of the baseline for SWT is
effectively (height / 2) * 2, thereby rounding all odd heights downward
to the next even number.

This change pushes the division by 2 from the caller into the
implementations of drawText. A corresponding change will be pushed in
the egit repository.

Bug: 450813
Change-Id: I66f4e71873bb8e6f936fde573bbe4c35fe23a022
Signed-off-by: Michael Keppler <michael.keppler@gmx.de>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

The change tries to make jgit behave more like native CLI git regarding
the negation rules. According to [1] "... prefix "!" which negates the
pattern; any matching file excluded by a previous pattern will become
included again." Negating the pattern should not automatically make the
file *not ignored* - other pattern rules have to be considered too.

The fix adds test cases for both bugs 448094 and 407475.

[1] https://www.kernel.org/pub/software/scm/git/docs/gitignore.html

Bug: 448094
Bug: 407475
Change-Id: I322954200dd3c683e3d8f4adc48506eb99e56ae1
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

The cleanUp path is trying to restore files that previously were
clean, but were overwritten in the work tree by a partial merge
attempt that has failed and needs to be aborted. Reuse the checkout
logic to write the file content and refresh the stat data.

Change-Id: I320d33b3744daf88d3155db99e957408937ddd00

When writing a symlink the stat data should only be written once
into the DirCacheEntry, based on the symlink itself and not the
possibly resolved destination observed by java.io.File.

Refactor the code to handle symlinks and early return.  This
removes the risk the blob stat info update is used against a
newly checked out symlink.

Hoist the file length stat update immediately after writing
the file, before a rename. This eliminates any race caused by another
process updating the file length after the rename and having it to
fall into the racily clean path.

Change-Id: I978ad9719c018ce1cf26947efbabaa8b9dff2217

Entries should only be written to the working tree managed by the
Repository. Simplify callers by passing only the entry and computing
the work tree location inside of the checkoutEntry method.

Change-Id: I574e41280d0407f1853fda12f4bd0d30f75d74e7

This deprecated method accidentally creates two ObjectReader
instances. Use the instance created one line above that is
correctly released in the finally block.

Change-Id: Ic57d041674611802a9384d8fa1d292e821055019

Explicitly pass STAGE_0 when creating a DirCacheEntry from String.
This matches the immediate next constructor that accepts the int
stage argument better, making the code easier to read.

Fix a weird line break where the comma was orphaned by itself.

Change-Id: Icf0970dd02a63877f9e41b51b982b0265e8b8887

By routing writes through SafeBufferedOutputStream the caller can be
alerted to any flush at close failures while writing or appending to
the rebase todo script.

Switch the character encoding to be done at the line granularity, as
this is sufficiently long enough that encoding overheads will not be a
bottleneck, but short enough that the amount of temporary data will
not cause memory problems for the JVM.

Change-Id: Ice5ec10a7cbadc58486d481b92940056f9ffc43a

Encourage callers to explicitly name a directory to hold any
overflow data. Call sites have more information about what is
going into the buffer and how it should be protected at the
filesystem level than just throwing content to the system wide
temporary directory.

Callers that still really don't care (or need to care) can pass
null for the File argument to have the system directory used.

Change-Id: I89009bbee49d3850d42cd82c2c462e51043acda0

File contents are processed into a single byte[] for character
conversion. The data must fit entirely in memory, so avoid any
file IO.

Change-Id: I3fe8be2e5f37d5ae953596dda1ed3fe6d4f6aebc

This applies the same filesystem permissions as the source objects.
Users may override in properties files using the tmpdir value.

Change-Id: I3ec332cf41f12eae246cfaee9fd792c52cb2908b

Increase the in-memory buffer for the TREE extension to 5 MiB, and
overflow to $GIT_DIR instead of /tmp.  Using a larger buffer reduces
the chances a repository will overflow and need to spool the extension
to disk.  Using $GIT_DIR allows the TREE extension contents to have
the same file system protections as the final $GIT_DIR/index.

Wrap the entire thing in a try/finally to ensure the temp file is
deleted from disk after the block has finished using it. To avoid
dangling NFS files, LocalFile.destroy() does close the local file
before deleting it.

Change-Id: I8f871181a4689e3ebf0cdd4fd1769333cf7546c3

Bug: 406379
Change-Id: I7f4f23cd50d46ffcde69d6abfe3ca0cc6fa86604

RecievePack already honors fsck settings for safeForWindows and
safeForMacOS.  Allow those same checks to be performed during fetch
through a caller-configurable ObjectChecker.

Default the fetch fsck options to match the current platform, as
it can be reasonably assumed the repository will be accessed here.

Change-Id: I3c0f411fad209c6bd8fb9c4acf5c55a6799a6a2a

The TreeWalk constructor doesn't throw in a meaninful way that
requires cleanup of the not-yet-created TreeWalk.

Hoist the constructor outside of the try/finally and remove the
now unnecessary != null check during the finally.

Change-Id: If5b8bb91562715df0699726648123a47426b9850

Callers should manage the ObjectReader, as this allows the JGit library to cache
context relevant information across files checked out at the same time. If the
caller only has one file to checkout, it should still explicitly manage the life
span of the ObjectReader.

Change-Id: Ib57fba6cb4b774ccff8c416ef4d32e2b390f16a9

cgit has this feature for some time. This will teach JGit to send symbolic refs,
too.

Change-Id: I7cb2ab4e6d31a838a0af92eac64535fdb66ed74a
Signed-off-by: Yuxuan 'fishy' Wang <fishywang@google.com>

To improve ignore parser performance we can avoid using java.util.regex
code on simple wildcard patterns with leading or trailing asterisk. As
those patterns represent a majority of ignore rules, the index diff
performance can be drastically increased on huge repository with lot of
ignore rules.

Bug: 450466
Change-Id: I80428441cc8d5de5468813f841d89322413eed8b
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

BaseRepositoryBuilder.findGitDir() was not searching correctly for bare
repositories. E.g. when running org.eclipse.jgit.pgm.Log and the current
directory was that of a bare git repository an error "fatal: error:
can't find git directory" was raised. With this fix RepositoryBuilder
will also check whether the given directory is the root of a bare
repository.

Bug: 450193
Change-Id: I4d4ad42e24ca397745adb0f3385caee3bcf3a186
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

JGit's ObjectDirectory implements the optimization that it remembers the
pack folders (.git/objects/pack) lastModified timestamp and doesn't
check for new packfiles in this folder if the lastModified attribute has
not changed.
In environments using NFS this can cause trouble. If multiple JGit
instances from multiple machines work on the same repository and one
instance creates a new ref and a new packfile (e.g. by doing a fetch)
then the other machines may detect the new ref but can't resolve the
referenced object because it doesn't detect that pack folder has a new
packfile. That's because NFS may cache file/folder metadata for quite a
long time and the pack folders modification time is not updated although
a new packfile is there and could be read.

The new config parameter core.trustfolderstat controls this behaviour.
The default is true and jgits behaviours is unchanged. But if this
parameter is set to false then jgit doesn't trust the pack directories
lastmodified anymore. Instead it will always iterate through the content
of that folder to detect new packfiles.

Change-Id: Ie3b4e92933286aa9916070a22422e629b3147f54
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

Change-Id: I6ae39b874a3c1fa9e1cda235ff137c9eb8dd359f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

Actually the test only allows a range from [1,255], so let's name the
variable so.

Change-Id: Iecdb8149b83389c67e3cd2f64f4a654c175475be
Signed-off-by: Stefan Beller <sbeller@google.com>

JGit style is to import exactly the classes required, and never
to use "import foo.*" as the foo package could add new classes
in the future which are conflicting/confusing with the imports
already used by a source file.

Change-Id: I5693408c777e5843ec65fff1163d5d717849fa34

The latest changes to IndexDiff just assumed that all configured
submodules are allways cloned. If a configured submodule did not exist
an exception was thrown. This is fixed by this commit.

Bug: 450567
Change-Id: Iabe3b196d998c19483082e5720038ebddaeb1890

Change-Id: Ia657e46b9abc8c8640f63f6ff579b2f8f2b01de4

Inspired by the series[1], this implements the possibility to
have atomic ref transactions.
If the database supports atomic ref update capabilities, we'll
advertise these. If the client wishes to use this feature, either
all refs will be updated or none at all.

[1] http://thread.gmane.org/gmane.comp.version-control.git/259019/focus=259024

Change-Id: I7b5d19c21f3b5557e41b9bcb5d359a65ff1a493d
Signed-off-by: Stefan Beller <sbeller@google.com>

For details about poodle see:
http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2014-3566

Bug 450170
Change-Id: Ib169e88d9d51f4d0e6caa3a498f48edc81e0991d
Signed-off-by: Dani Megert <Daniel_Megert@ch.ibm.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>