Don't remove pack from pack list for problems which could be transient
If we hit a corrupt object or invalid pack remove the pack from the pack
list. Other IOException could be transient hence we should not remove
the pack from the list to avoid the problem reported on the Gerrit list
[1]. It looks like in the reported case the pack was removed from the
pack list causing MissingObjectExceptions which disappear when the
server is restarted.
[1] https://groups.google.com/forum/#!topic/repo-discuss/Qdmbl-YZ4NU
Change-Id: I331626110d54b190e46cddc2c40f29ddeb9613cd
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Log reason for ignoring pack when IOException occurred
This should help to identify the root cause of the problem discussed on
the Gerrit list [1].
[1] https://groups.google.com/forum/#!topic/repo-discuss/Qdmbl-YZ4NU
Change-Id: I871f70e4bb1227952e1544b789013583b14e2b96
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
This new git capability isn't yet in the wild in the git-core camp.
Hence we can silence this API warning.
Change-Id: I9e030959f658c58344b04ef2f9edd28060f0c964
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
[pgm] Prevent commands from writing progress to System.err
Commands which report progress used to write to System.err. This is not
desirable in cases where jgit.pgm is embedded. This change redirects
progress output to the error stream that is configured by the command.
Change-Id: I01fa5e167437e619448ac201fcb1cbf63bad96d7
Signed-off-by: Rüdiger Herrmann <ruediger.herrmann@gmx.de>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
The atomic feature is now cooking in -next in git-core. Very rarely
features are ejected from the the next branch in git-core, so I consider
it reasonable to come up with this patch now to make the 2 implementations
interoperable.
Change-Id: I806a8ae3c045ca5936f69cb903baf9b99ee39181
Signed-off-by: Stefan Beller <sbeller@google.com>
Core classes to parse and process .gitattributes files including
support for reading attributes in WorkingTreeIterator and the
dirCacheIterator.
The implementation follows the git ignore implementation. It supports
lazy reading attributes while walking the working tree.
Bug: 342372
CQ: 9078
Change-Id: I05f3ce1861fbf9896b1bcb7816ba78af35f3ad3d
Also-by: Marc Strapetz <marc.strapetz@syntevo.com>
Also-by: Gunnar Wagenknecht <gunnar@wagenknecht.org>
Also-by: Arthur Daussy <arthur.daussy@obeo.fr>
Signed-off-by: Gunnar Wagenknecht <gunnar@wagenknecht.org>
Signed-off-by: Marc Strapetz <marc.strapetz@syntevo.com>
Signed-off-by: Arthur Daussy <arthur.daussy@obeo.fr>
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
* stable-3.6:
Prepare 3.6.2-SNAPSHOT builds
JGit v3.6.1.201501031845-r
Trim author/committer name and email in commit header
Rename detection should canonicalize line endings
PathMatcher should respect "assumeDirectory" flag
Change-Id: Idd48c6d94cf1ab09abc07f70d50890b1b78e1833
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Trim author/committer name and email in commit header
C Git trims name and email before inserting them into the commit header
so that " A U Thor " and " author@example.com " becomes
"A U Thor <author@example.com>" with a single separating space.
This changes PersonIdent#toExternalString() to trim name and email
before concatenating them.
Change-Id: Idd77b659d0db957626824f6632e2da38d7731625
Signed-off-by: Rüdiger Herrmann <ruediger.herrmann@gmx.de>
Native Git canonicalizes line endings when detecting
renames, more specifically it replaces CRLF by LF.
See: hash_chars in diffcore-delta.c
Bug: 449545
Change-Id: Iec2aab12ae9e67074cccb7fbd4d9defe176a0130
Signed-off-by: Marc Strapetz <marc.strapetz@syntevo.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Add .mailmap to correct Shawn and Saša's log entries
Both myself and Saša Živkov have been using different names
and email addresses in the JGit history. Configure .mailmap
to map to consistent entries for us.
Saša Živkov's entries were taken from the Gerrit Code Review
.mailmap file.
Change-Id: I3fad123c453c3a1e2972ae0616757fd34b003165
The path matcher should not fail if the rule ends with trailing slash,
target pattern does not ends with the slash and the "assumeDirectory"
flag is set.
E.g. */bin/ should also match a/bin if this pattern is threated as
directory by WorkingTreeIterator (FileMode.TREE).
The old code/tests have never tested directory rules with patterns
*without* trailing slashes but with the "assumeDirectory" flag set.
Unfortunately this is exactly what WorkingTreeIterator does... The tests
are changed to test *both* cases now (with trailing slash and without)
if the target pattern has trailing slash (represents directory).
Bug: 454672
Change-Id: I621c1644d9e94df3eb9f6f09c6de0fe51f0950a4
Also-by: Markus Duft <markus.duft@salomon.at>
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
* stable-3.6:
Prepare 3.6.1-SNAPSHOT builds
JGit v3.6.0.201412230720-r
[pgm] Add option --bare to clone command
[pgm] Implement clone using CloneCommand
Fix junit tests under windows when the platform is explicitly changed
Fix unit tests for windows by explicitly closing test repos
[pgm] Add option --tags for ls-remote
[pgm] Add option --heads for ls-remote
[pgm] Use LsRemoteCommand to implement ls-remote and add a test
Change-Id: I8f31e76cb7e9416919f37e02c7e51ab1d221df40
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Fix junit tests under windows when the platform is explicitly changed
SystemReader used a chached ObjectChecker which was instantiated only
once. But in case of unit tests where we can change the platform
dynamically (e.g. MockSystemReader.setWindows()) this is wrong and
caused DirCacheCheckoutMaliciousPathTest.
testMaliciousAbsoluteCurDrivePathWindowsOnUnix() to fail. This change
allows
user of SystemReader to force the creation of a new ObjectChecker.
MockSystemReader.setWindows() and .setUnix() make use of this feature.
Change-Id: I87458d1dc63c1f5c18979f972b1c1f0d670a9ed8
* stable-3.6: (26 commits)
JGit v3.5.3.201412180710-r
JGit v3.4.2.201412180340-r
ObjectChecker: Disallow names potentially mapping to ".git" on HFS+
ObjectChecker: Disallow Windows shortname "GIT~1"
ObjectChecker: Disallow ".git." and ".git<space>"
Always ignore case when forbidding .git in ObjectChecker
DirCache: Refuse to read files with invalid paths
DirCache: Replace isValidPath with DirCacheCheckout.checkValidPath
Replace "a." with "a-" in unit tests
Support the new repository layout for submodules
Allow explicit configuration of git directory in CloneCommand
Allow explicit configuration of git directory in InitCommand
Fix tests on windows by closing repos
RepoCommand should close opened repos
Fix LocalDiskRepositoryTestCase to create correct type of repos
Prevent NPE if ref can't be resolved when executing ReflogCommand
Fix DirCacheCheckout to set correct file length if core.autocrlf=true
CheckoutCommand: Fix checking out ours/theirs when no base stage exists
Make sure modifications to config-param trustFolderStat are detected
Apache HttpClientConnection: replace calls to deprecated LocalFile()
...
Change-Id: Ife6f21d64a148dbb0d0d9055356b1568188806fe
* stable-3.5:
JGit v3.5.3.201412180710-r
JGit v3.4.2.201412180340-r
ObjectChecker: Disallow names potentially mapping to ".git" on HFS+
ObjectChecker: Disallow Windows shortname "GIT~1"
ObjectChecker: Disallow ".git." and ".git<space>"
Always ignore case when forbidding .git in ObjectChecker
DirCache: Refuse to read files with invalid paths
DirCache: Replace isValidPath with DirCacheCheckout.checkValidPath
Replace "a." with "a-" in unit tests
Apache HttpClientConnection: replace calls to deprecated LocalFile()
Fix two nits about DirCacheEntry constructors
Detect buffering failures while writing rebase todo file
Deprecate TemporaryBuffer.LocalFile without parent directory
Switch FileHeader.extractFileLines to TemporaryBuffer.Heap
AmazonS3: Buffer pushed pack content under $GIT_DIR
DirCache: Buffer TREE extension to $GIT_DIR
Change-Id: Iee8acbaa9d4d9047b550641db1b8845d64530785
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-3.4:
JGit v3.4.2.201412180340-r
ObjectChecker: Disallow names potentially mapping to ".git" on HFS+
ObjectChecker: Disallow Windows shortname "GIT~1"
ObjectChecker: Disallow ".git." and ".git<space>"
Always ignore case when forbidding .git in ObjectChecker
DirCache: Refuse to read files with invalid paths
DirCache: Replace isValidPath with DirCacheCheckout.checkValidPath
Replace "a." with "a-" in unit tests
Apache HttpClientConnection: replace calls to deprecated LocalFile()
Fix two nits about DirCacheEntry constructors
Detect buffering failures while writing rebase todo file
Deprecate TemporaryBuffer.LocalFile without parent directory
Switch FileHeader.extractFileLines to TemporaryBuffer.Heap
AmazonS3: Buffer pushed pack content under $GIT_DIR
DirCache: Buffer TREE extension to $GIT_DIR
Change-Id: I398cf40b006a05a6537788fc6eb1f84df1ed8814
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
ObjectChecker: Disallow names potentially mapping to ".git" on HFS+
Mac's HFS+ folds concatentations of ".git" and ignorable Unicode
characters [1] to ".git" [2]. Hence we need to disallow all names which
could potentially be a shortname for ".git". Example: in an empty
directory create a folder ".g\U+200Cit". Now you can't create another
folder ".git".
The following characters are ignorable Unicode which are ignored on
HFS+:
unicode hex name
-------------------------------------------------
U+200C 0xe2808c ZERO WIDTH NON-JOINER
U+200D 0xe2808d ZERO WIDTH JOINER
U+200E 0xe2808e LEFT-TO-RIGHT MARK
U+200F 0xe2808f RIGHT-TO-LEFT MARK
U+202A 0xe280aa LEFT-TO-RIGHT EMBEDDING
U+202B 0xe280ab RIGHT-TO-LEFT EMBEDDING
U+202C 0xe280ac POP DIRECTIONAL FORMATTING
U+202D 0xe280ad LEFT-TO-RIGHT OVERRIDE
U+202E 0xe280ae RIGHT-TO-LEFT OVERRIDE
U+206A 0xe281aa INHIBIT SYMMETRIC SWAPPING
U+206B 0xe281ab ACTIVATE SYMMETRIC SWAPPING
U+206C 0xe281ac INHIBIT ARABIC FORM SHAPING
U+206D 0xe281ad ACTIVATE ARABIC FORM SHAPING
U+206E 0xe281ae NATIONAL DIGIT SHAPES
U+206F 0xe281af NOMINAL DIGIT SHAPES
U+FEFF 0xefbbbf ZERO WIDTH NO-BREAK SPACE
[1] http://www.unicode.org/versions/Unicode7.0.0/ch05.pdf#G40025http://www.unicode.org/reports/tr31/#Layout_and_Format_Control_Characters
[2] http://dubeiko.com/development/FileSystems/HFSPLUS/tn1150.html#UnicodeSubtleties
Change-Id: Ib6a1dd090b2649bdd8ec16387c994ed29de2860d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Windows creates shortnames for all non-8.3 files (see [1]). Hence we
need to disallow all names which could potentially be a shortname for
".git". Example: in an empty directory create a folder "GIT~1". Now you
can't create another folder ".git".
The path "GIT~1" may map to ".git" on Windows. A potential victim to
such an attack first has to initialize a git repository in order to
receive any git commits. Hence the .git folder created by init will get
the shortname "GIT~1". ".git" will only get a different shortname if the
user has created a file "GIT~1" before initialization of the git
repository.
[1] http://en.wikipedia.org/wiki/8.3_filename
Change-Id: I9978ab8f2d2951c46c1b9bbde57986d64d26b9b2
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Windows treats "foo." and "foo " as "foo". The ".git" directory is
special, as it contains metadata for a local Git repository. Disallow
variations that Windows considers to be the same.
Change-Id: I28eb48859a95a89111b4987c91de97557e3bb539
Always ignore case when forbidding .git in ObjectChecker
The component name ".GIT" inside a tree entry could confuse a
case insensitive filesystem into looking at a submodule and
not a directory entry.
Disallow any case permutations of ".git" to prevent this
confusion from entering a repository and showing up at a
later date on a case insensitive system.
Change-Id: Iaa3f768931d0d5764bf07ac5f6f3ff2b1fdda01b
If the DirCache contains a path that is known to be invalid, refuse to
read the DirCache into memory. This avoids confusing errors later if
an invalid path read from the DirCache were to be passed into a new
DirCacheEntry constructor.
Change-Id: Ic033d81e23a5fbd554cc4dff80a232504562ffa8
DirCache: Replace isValidPath with DirCacheCheckout.checkValidPath
isValidPath is an older simple form of the validation performed by
checkValidPath. Use the latter as it more consistently matches
git-core's validation rules.
By running the same validation as fsck, callers creating an entry
for the DirCache are more likely to learn early they are trying
to build trees that will fail fsck.
Change-Id: Ibf5ac116097156aa05c18e231bc65c0854932eb1
Windows does not like naming files "a.". The trailing "." may be
dropped by the filesystem, which is confusing. Even though these
tests currently do not write to disk, future tests like them might.
Replace "." with "-", which has the same sorting properties that
were desirable about ".", but does not have the same limitations.
Change-Id: Ie5b7594bf5e79828d1341883c73ddb70123d5055
TODO: find a way to use option -h which is already captured by
TextBuiltin's option --help which also uses the alias -h.
Bug: 444072
Change-Id: Ie66584c2fc7fc224014a43cf928547703dd9d213
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
When updating a submodule (e.g. during recursive clone) the repository
for the submodule should be located at <gitdir>/modules/<submodule-path>
whereas the working tree of the submodule should be located at
<working-tree>/<submodule-path> (<gitdir> and <working-tree> are
associated to the containing repository). Since CloneCommand has learned
about specifying a separate gitdir this is easy to implement in
SubmoduleUpdateCommand.
Change-Id: I9b56a3dfa50f97f6975c2bb7c97b36296f331b64
Allow explicit configuration of git directory in CloneCommand
This feature is needed to support the new submodule layout where the
.git folder of the submodules is under .git/modules/<submodule>.
Change-Id: If5f13426cfd09b7677e23478e9700c8c25a6dae5
Allow explicit configuration of git directory in InitCommand
Native git's "init" command allows to specify the location of the .git
folder with the option "--separate-git-dir". This allows for example to
setup repositories with a non-standard layout. E.g. .git folder under
/repos/a.git and the worktree under /home/git/a. Both directories
contain pointers to the other side: /repos/a.git/config contains
core.worktree=/home/git/a . And /home/git/a/.git is a file containing
"gitdir: /repos/a.git". This commit adds that option to InitCommand.
This feature is needed to support the new submodule layout where the
.git folder of the submodules is under .git/modules/<submodule>.
Change-Id: I0208f643808bf8f28e2c979d6e33662607775f1f
Matthias Sohn
Frank Wagner
Rüdiger Herrmann
Stefan Beller
Christian Halstrick
Arthur Daussy
Andrey Loskutov
Marc Strapetz
Shawn Pearce
Shawn Pearce