Matthias Sohn [Wed, 24 Dec 2014 00:08:58 +0000 (01:08 +0100)]
Merge branch 'stable-3.6'
* stable-3.6:
Prepare 3.6.1-SNAPSHOT builds
JGit v3.6.0.201412230720-r
[pgm] Add option --bare to clone command
[pgm] Implement clone using CloneCommand
Fix junit tests under windows when the platform is explicitly changed
Fix unit tests for windows by explicitly closing test repos
[pgm] Add option --tags for ls-remote
[pgm] Add option --heads for ls-remote
[pgm] Use LsRemoteCommand to implement ls-remote and add a test
Change-Id: I8f31e76cb7e9416919f37e02c7e51ab1d221df40 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Fix junit tests under windows when the platform is explicitly changed
SystemReader used a chached ObjectChecker which was instantiated only
once. But in case of unit tests where we can change the platform
dynamically (e.g. MockSystemReader.setWindows()) this is wrong and
caused DirCacheCheckoutMaliciousPathTest.
testMaliciousAbsoluteCurDrivePathWindowsOnUnix() to fail. This change
allows
user of SystemReader to force the creation of a new ObjectChecker.
MockSystemReader.setWindows() and .setUnix() make use of this feature.
Shawn Pearce [Fri, 19 Dec 2014 15:57:55 +0000 (07:57 -0800)]
Merge branch 'stable-3.6'
* stable-3.6: (26 commits)
JGit v3.5.3.201412180710-r
JGit v3.4.2.201412180340-r
ObjectChecker: Disallow names potentially mapping to ".git" on HFS+
ObjectChecker: Disallow Windows shortname "GIT~1"
ObjectChecker: Disallow ".git." and ".git<space>"
Always ignore case when forbidding .git in ObjectChecker
DirCache: Refuse to read files with invalid paths
DirCache: Replace isValidPath with DirCacheCheckout.checkValidPath
Replace "a." with "a-" in unit tests
Support the new repository layout for submodules
Allow explicit configuration of git directory in CloneCommand
Allow explicit configuration of git directory in InitCommand
Fix tests on windows by closing repos
RepoCommand should close opened repos
Fix LocalDiskRepositoryTestCase to create correct type of repos
Prevent NPE if ref can't be resolved when executing ReflogCommand
Fix DirCacheCheckout to set correct file length if core.autocrlf=true
CheckoutCommand: Fix checking out ours/theirs when no base stage exists
Make sure modifications to config-param trustFolderStat are detected
Apache HttpClientConnection: replace calls to deprecated LocalFile()
...
Matthias Sohn [Thu, 18 Dec 2014 23:17:54 +0000 (00:17 +0100)]
Merge branch 'stable-3.5' into stable-3.6
* stable-3.5:
JGit v3.5.3.201412180710-r
JGit v3.4.2.201412180340-r
ObjectChecker: Disallow names potentially mapping to ".git" on HFS+
ObjectChecker: Disallow Windows shortname "GIT~1"
ObjectChecker: Disallow ".git." and ".git<space>"
Always ignore case when forbidding .git in ObjectChecker
DirCache: Refuse to read files with invalid paths
DirCache: Replace isValidPath with DirCacheCheckout.checkValidPath
Replace "a." with "a-" in unit tests
Apache HttpClientConnection: replace calls to deprecated LocalFile()
Fix two nits about DirCacheEntry constructors
Detect buffering failures while writing rebase todo file
Deprecate TemporaryBuffer.LocalFile without parent directory
Switch FileHeader.extractFileLines to TemporaryBuffer.Heap
AmazonS3: Buffer pushed pack content under $GIT_DIR
DirCache: Buffer TREE extension to $GIT_DIR
Change-Id: Iee8acbaa9d4d9047b550641db1b8845d64530785 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Thu, 18 Dec 2014 14:18:07 +0000 (15:18 +0100)]
Merge branch 'stable-3.4' into stable-3.5
* stable-3.4:
JGit v3.4.2.201412180340-r
ObjectChecker: Disallow names potentially mapping to ".git" on HFS+
ObjectChecker: Disallow Windows shortname "GIT~1"
ObjectChecker: Disallow ".git." and ".git<space>"
Always ignore case when forbidding .git in ObjectChecker
DirCache: Refuse to read files with invalid paths
DirCache: Replace isValidPath with DirCacheCheckout.checkValidPath
Replace "a." with "a-" in unit tests
Apache HttpClientConnection: replace calls to deprecated LocalFile()
Fix two nits about DirCacheEntry constructors
Detect buffering failures while writing rebase todo file
Deprecate TemporaryBuffer.LocalFile without parent directory
Switch FileHeader.extractFileLines to TemporaryBuffer.Heap
AmazonS3: Buffer pushed pack content under $GIT_DIR
DirCache: Buffer TREE extension to $GIT_DIR
Change-Id: I398cf40b006a05a6537788fc6eb1f84df1ed8814 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Mon, 15 Dec 2014 13:42:04 +0000 (14:42 +0100)]
ObjectChecker: Disallow names potentially mapping to ".git" on HFS+
Mac's HFS+ folds concatentations of ".git" and ignorable Unicode
characters [1] to ".git" [2]. Hence we need to disallow all names which
could potentially be a shortname for ".git". Example: in an empty
directory create a folder ".g\U+200Cit". Now you can't create another
folder ".git".
The following characters are ignorable Unicode which are ignored on
HFS+:
unicode hex name
-------------------------------------------------
U+200C 0xe2808c ZERO WIDTH NON-JOINER
U+200D 0xe2808d ZERO WIDTH JOINER
U+200E 0xe2808e LEFT-TO-RIGHT MARK
U+200F 0xe2808f RIGHT-TO-LEFT MARK
U+202A 0xe280aa LEFT-TO-RIGHT EMBEDDING
U+202B 0xe280ab RIGHT-TO-LEFT EMBEDDING
U+202C 0xe280ac POP DIRECTIONAL FORMATTING
U+202D 0xe280ad LEFT-TO-RIGHT OVERRIDE
U+202E 0xe280ae RIGHT-TO-LEFT OVERRIDE
U+206A 0xe281aa INHIBIT SYMMETRIC SWAPPING
U+206B 0xe281ab ACTIVATE SYMMETRIC SWAPPING
U+206C 0xe281ac INHIBIT ARABIC FORM SHAPING
U+206D 0xe281ad ACTIVATE ARABIC FORM SHAPING
U+206E 0xe281ae NATIONAL DIGIT SHAPES
U+206F 0xe281af NOMINAL DIGIT SHAPES
U+FEFF 0xefbbbf ZERO WIDTH NO-BREAK SPACE
Windows creates shortnames for all non-8.3 files (see [1]). Hence we
need to disallow all names which could potentially be a shortname for
".git". Example: in an empty directory create a folder "GIT~1". Now you
can't create another folder ".git".
The path "GIT~1" may map to ".git" on Windows. A potential victim to
such an attack first has to initialize a git repository in order to
receive any git commits. Hence the .git folder created by init will get
the shortname "GIT~1". ".git" will only get a different shortname if the
user has created a file "GIT~1" before initialization of the git
repository.
[1] http://en.wikipedia.org/wiki/8.3_filename
Change-Id: I9978ab8f2d2951c46c1b9bbde57986d64d26b9b2 Signed-off-by: Christian Halstrick <christian.halstrick@sap.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Shawn Pearce [Mon, 1 Dec 2014 23:01:07 +0000 (15:01 -0800)]
ObjectChecker: Disallow ".git." and ".git<space>"
Windows treats "foo." and "foo " as "foo". The ".git" directory is
special, as it contains metadata for a local Git repository. Disallow
variations that Windows considers to be the same.
Shawn Pearce [Mon, 24 Nov 2014 18:27:49 +0000 (10:27 -0800)]
Always ignore case when forbidding .git in ObjectChecker
The component name ".GIT" inside a tree entry could confuse a
case insensitive filesystem into looking at a submodule and
not a directory entry.
Disallow any case permutations of ".git" to prevent this
confusion from entering a repository and showing up at a
later date on a case insensitive system.
Shawn Pearce [Tue, 25 Nov 2014 19:43:32 +0000 (11:43 -0800)]
DirCache: Refuse to read files with invalid paths
If the DirCache contains a path that is known to be invalid, refuse to
read the DirCache into memory. This avoids confusing errors later if
an invalid path read from the DirCache were to be passed into a new
DirCacheEntry constructor.
Shawn Pearce [Tue, 25 Nov 2014 08:47:48 +0000 (00:47 -0800)]
DirCache: Replace isValidPath with DirCacheCheckout.checkValidPath
isValidPath is an older simple form of the validation performed by
checkValidPath. Use the latter as it more consistently matches
git-core's validation rules.
By running the same validation as fsck, callers creating an entry
for the DirCache are more likely to learn early they are trying
to build trees that will fail fsck.
Shawn Pearce [Sat, 29 Nov 2014 04:37:18 +0000 (20:37 -0800)]
Replace "a." with "a-" in unit tests
Windows does not like naming files "a.". The trailing "." may be
dropped by the filesystem, which is confusing. Even though these
tests currently do not write to disk, future tests like them might.
Replace "." with "-", which has the same sorting properties that
were desirable about ".", but does not have the same limitations.
When updating a submodule (e.g. during recursive clone) the repository
for the submodule should be located at <gitdir>/modules/<submodule-path>
whereas the working tree of the submodule should be located at
<working-tree>/<submodule-path> (<gitdir> and <working-tree> are
associated to the containing repository). Since CloneCommand has learned
about specifying a separate gitdir this is easy to implement in
SubmoduleUpdateCommand.
Allow explicit configuration of git directory in InitCommand
Native git's "init" command allows to specify the location of the .git
folder with the option "--separate-git-dir". This allows for example to
setup repositories with a non-standard layout. E.g. .git folder under
/repos/a.git and the worktree under /home/git/a. Both directories
contain pointers to the other side: /repos/a.git/config contains
core.worktree=/home/git/a . And /home/git/a/.git is a file containing
"gitdir: /repos/a.git". This commit adds that option to InitCommand.
This feature is needed to support the new submodule layout where the
.git folder of the submodules is under .git/modules/<submodule>.
Shawn Pearce [Fri, 12 Dec 2014 21:53:18 +0000 (16:53 -0500)]
Revert "Extract path info from requests without decoding"
This reverts commit 19f869996f27adf59ec507e5f565d8b5619576f3.
Leaving path info encoded confuses applications like Gitiles.
Trying to fix this inside of JGit was maybe the wrong solution.
Fix LocalDiskRepositoryTestCase to create correct type of repos
In one place LocalDiskRepositoryTestCase was ignoring the specification
whether to create a bare or non-bare repository. Fix this and fix also
one test which fails now because bare repos don't write reflogs by
default.
Change-Id: I4bcf8cf97c5b46e2f3919809eaa121a8d0e47010 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Sat, 6 Dec 2014 01:59:34 +0000 (02:59 +0100)]
Fix DirCacheCheckout to set correct file length if core.autocrlf=true
To update the file length stat we need to use the length of the
temporary file since it's not yet renamed to the target file name here.
The incorrect file length stat update was introduced in a606dc363d0f6b09e4527cca6b645d3cb1ec407d.
Bug: 453962
Change-Id: I715c048227553efae6f8f6b6878c0f04f2609d9c Also-by: Konrad Kügler <swamblumat-eclipsebugs@yahoo.de> Also-by: Christian Halstrick <christian.halstrick@sap.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Robin Stocker [Sun, 30 Nov 2014 10:20:26 +0000 (21:20 +1100)]
CheckoutCommand: Fix checking out ours/theirs when no base stage exists
In case of an add/add conflict, no base stage exists. The previous
implementation would skip over the entries because the condition
expected the base stage to always exist.
Change-Id: Ie2b3685d958c09b241991b74e6177401e8a1ebc9 Signed-off-by: Robin Stocker <robin@nibor.org>
David Pletcher [Sat, 1 Nov 2014 21:34:18 +0000 (14:34 -0700)]
Extract path info from requests without decoding
Gitiles malfunctions in conjunction with jgit and guice
because of a recent Guice bug fix. Work around the problem
by parsing the URI directly, bypassing the unescaping
performed by the getPathInfo method.
This rest of this message is copied from
https://gerrit-review.googlesource.com/#/c/60820/ :
The fix for Guice issue #745[1] causes getPathInfo() within the
GuiceFilter to return decoded values, eliminating the difference
between "foo/bar" and "foo%2Fbar". This is in spec with the servlet
standard, whose javadoc for getPathInfo[2] states that the return
value be "decoded by the web container".
Work around this by extracting the path part directly from the request
URI, which is unmodified by the container. This is copying the Guice
behavior prior to the bugfix.
Shawn Pearce [Tue, 25 Nov 2014 05:43:59 +0000 (21:43 -0800)]
Move checkPath from DirCacheCheckout to ObjectChecker
The bulk of the "is this sane" logic is inside of ObjectChecker. The
only caller for the version in DirCacheCheckout is an obtuse usage for
the static isValidRefName() method in Repository.
Deprecate the weird single use method in DirCacheCheckout and move all
code for checking a sequence of path components into ObjectChecker,
where it makes sense alongside the existing code that checks a single
component at a time.
Reuse a single ObjectChecker for the local platform, to avoid looking
up the system properties on each path string considered.
Michael Keppler [Fri, 28 Nov 2014 20:21:59 +0000 (21:21 +0100)]
Use baseline instead of centerline in PlotRenderer
If the text extent height of a to be rendered plot line is odd, then the
SWTPlotRenderer cannot calculate the correct Y position for drawing the
label and draws the label with a 1 pixel offset. SWT text drawing uses
the baseline as Y coordinate. Due to the given centerline API in the
AbstractPlotRenderer the overall calculation of the baseline for SWT is
effectively (height / 2) * 2, thereby rounding all odd heights downward
to the next even number.
This change pushes the division by 2 from the caller into the
implementations of drawText. A corresponding change will be pushed in
the egit repository.
Bug: 450813
Change-Id: I66f4e71873bb8e6f936fde573bbe4c35fe23a022 Signed-off-by: Michael Keppler <michael.keppler@gmx.de> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Andrey Loskutov [Wed, 22 Oct 2014 07:31:42 +0000 (09:31 +0200)]
Consider parent rules if ignore rule is negated
The change tries to make jgit behave more like native CLI git regarding
the negation rules. According to [1] "... prefix "!" which negates the
pattern; any matching file excluded by a previous pattern will become
included again." Negating the pattern should not automatically make the
file *not ignored* - other pattern rules have to be considered too.
The fix adds test cases for both bugs 448094 and 407475.
Shawn Pearce [Wed, 26 Nov 2014 00:14:33 +0000 (16:14 -0800)]
ResolveMerger: Use checkoutEntry during abort
The cleanUp path is trying to restore files that previously were
clean, but were overwritten in the work tree by a partial merge
attempt that has failed and needs to be aborted. Reuse the checkout
logic to write the file content and refresh the stat data.
Shawn Pearce [Tue, 25 Nov 2014 23:10:41 +0000 (15:10 -0800)]
Cleanup double stat update of symlinks in DirCacheCheckout
When writing a symlink the stat data should only be written once
into the DirCacheEntry, based on the symlink itself and not the
possibly resolved destination observed by java.io.File.
Refactor the code to handle symlinks and early return. This
removes the risk the blob stat info update is used against a
newly checked out symlink.
Hoist the file length stat update immediately after writing
the file, before a rename. This eliminates any race caused by another
process updating the file length after the rename and having it to
fall into the racily clean path.
Shawn Pearce [Tue, 25 Nov 2014 21:49:22 +0000 (13:49 -0800)]
Deprecate checkoutEntry variant that accepts File
Entries should only be written to the working tree managed by the
Repository. Simplify callers by passing only the entry and computing
the work tree location inside of the checkoutEntry method.
Shawn Pearce [Mon, 24 Nov 2014 19:38:09 +0000 (11:38 -0800)]
DirCacheCheckout: create only one ObjectReader
This deprecated method accidentally creates two ObjectReader
instances. Use the instance created one line above that is
correctly released in the finally block.
Shawn Pearce [Tue, 25 Nov 2014 05:06:30 +0000 (21:06 -0800)]
Fix two nits about DirCacheEntry constructors
Explicitly pass STAGE_0 when creating a DirCacheEntry from String.
This matches the immediate next constructor that accepts the int
stage argument better, making the code easier to read.
Fix a weird line break where the comma was orphaned by itself.
Shawn Pearce [Tue, 25 Nov 2014 04:51:07 +0000 (20:51 -0800)]
Detect buffering failures while writing rebase todo file
By routing writes through SafeBufferedOutputStream the caller can be
alerted to any flush at close failures while writing or appending to
the rebase todo script.
Switch the character encoding to be done at the line granularity, as
this is sufficiently long enough that encoding overheads will not be a
bottleneck, but short enough that the amount of temporary data will
not cause memory problems for the JVM.
Shawn Pearce [Tue, 25 Nov 2014 17:53:57 +0000 (09:53 -0800)]
Deprecate TemporaryBuffer.LocalFile without parent directory
Encourage callers to explicitly name a directory to hold any
overflow data. Call sites have more information about what is
going into the buffer and how it should be protected at the
filesystem level than just throwing content to the system wide
temporary directory.
Callers that still really don't care (or need to care) can pass
null for the File argument to have the system directory used.
Shawn Pearce [Tue, 25 Nov 2014 04:05:34 +0000 (20:05 -0800)]
DirCache: Buffer TREE extension to $GIT_DIR
Increase the in-memory buffer for the TREE extension to 5 MiB, and
overflow to $GIT_DIR instead of /tmp. Using a larger buffer reduces
the chances a repository will overflow and need to spool the extension
to disk. Using $GIT_DIR allows the TREE extension contents to have
the same file system protections as the final $GIT_DIR/index.
Wrap the entire thing in a try/finally to ensure the temp file is
deleted from disk after the block has finished using it. To avoid
dangling NFS files, LocalFile.destroy() does close the local file
before deleting it.
Shawn Pearce [Mon, 24 Nov 2014 22:51:09 +0000 (14:51 -0800)]
Allow configurable ObjectCheckers in fetch
RecievePack already honors fsck settings for safeForWindows and
safeForMacOS. Allow those same checks to be performed during fetch
through a caller-configurable ObjectChecker.
Default the fetch fsck options to match the current platform, as
it can be reasonably assumed the repository will be accessed here.
Shawn Pearce [Mon, 24 Nov 2014 19:38:09 +0000 (11:38 -0800)]
Deprecate checkoutEntry without ObjectReader
Callers should manage the ObjectReader, as this allows the JGit library to cache
context relevant information across files checked out at the same time. If the
caller only has one file to checkout, it should still explicitly manage the life
span of the ObjectReader.
Matthias Sohn [Thu, 13 Nov 2014 14:46:13 +0000 (15:46 +0100)]
Include the java7 feature in org.eclipse.jgit.feature
This way we no longer need to advertise it in the release train and can
uncategorize the jgit features without making it harder for users to
find and install the java7 feature.
Bug: 451276
Change-Id: I4c7dd0e1609fc1939d8ea83c01251dec59c228a3 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Andrey Loskutov [Fri, 7 Nov 2014 15:30:44 +0000 (16:30 +0100)]
Don't use java.util.regex for two simple wildcard cases
To improve ignore parser performance we can avoid using java.util.regex
code on simple wildcard patterns with leading or trailing asterisk. As
those patterns represent a majority of ignore rules, the index diff
performance can be drastically increased on huge repository with lot of
ignore rules.
BaseRepositoryBuilder.findGitDir() was not searching correctly for bare
repositories. E.g. when running org.eclipse.jgit.pgm.Log and the current
directory was that of a bare git repository an error "fatal: error:
can't find git directory" was raised. With this fix RepositoryBuilder
will also check whether the given directory is the root of a bare
repository.
Bug: 450193
Change-Id: I4d4ad42e24ca397745adb0f3385caee3bcf3a186 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
JGit's ObjectDirectory implements the optimization that it remembers the
pack folders (.git/objects/pack) lastModified timestamp and doesn't
check for new packfiles in this folder if the lastModified attribute has
not changed.
In environments using NFS this can cause trouble. If multiple JGit
instances from multiple machines work on the same repository and one
instance creates a new ref and a new packfile (e.g. by doing a fetch)
then the other machines may detect the new ref but can't resolve the
referenced object because it doesn't detect that pack folder has a new
packfile. That's because NFS may cache file/folder metadata for quite a
long time and the pack folders modification time is not updated although
a new packfile is there and could be read.
The new config parameter core.trustfolderstat controls this behaviour.
The default is true and jgits behaviours is unchanged. But if this
parameter is set to false then jgit doesn't trust the pack directories
lastmodified anymore. Instead it will always iterate through the content
of that folder to detect new packfiles.
Change-Id: Ie3b4e92933286aa9916070a22422e629b3147f54 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Shawn Pearce [Mon, 10 Nov 2014 23:35:20 +0000 (15:35 -0800)]
Expand wildcard imports to specfic classes
JGit style is to import exactly the classes required, and never
to use "import foo.*" as the foo package could add new classes
in the future which are conflicting/confusing with the imports
already used by a source file.
Fix possible NPE in IndexDiff when not all submodules are cloned
The latest changes to IndexDiff just assumed that all configured
submodules are allways cloned. If a configured submodule did not exist
an exception was thrown. This is fixed by this commit.
Stefan Beller [Sat, 8 Nov 2014 02:51:18 +0000 (18:51 -0800)]
Implement atomic refs update, if possible by database
Inspired by the series[1], this implements the possibility to
have atomic ref transactions.
If the database supports atomic ref update capabilities, we'll
advertise these. If the client wishes to use this feature, either
all refs will be updated or none at all.
Make sure checkout doesn't report conflicts on ignored paths
In a situation where a certain path was ignored but a working tree file
with this path existed jgit didn't allow to checkout a branch which
didn't ignore this path but contained different content. JGit considered
this to be a checkout conflict to prevent overwriting the file in the
working tree and raised an error. This commit fixes this by ensuring
that ignored dirty working tree files don't lead to a checkout conflict.
Bug: 450169
Change-Id: I90288d314ffac73c24a9c70a5181f8243bd4679a Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Axel Richard [Wed, 5 Nov 2014 08:30:47 +0000 (09:30 +0100)]
Add new method IndexDiff#getPathsWithIndexMode
Get the list of paths that have the given file mode.
This helps EGit to efficiently determine which modified files are
symlinks and should be shown with a symlink icon in the staging view.
Bug: 429302
Change-Id: Id15f0c6f265667f5b8b57cc2d9f97de568371919 Signed-off-by: Axel Richard <axel.richard@obeo.fr> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
projects / jgit.git / log