aboutsummaryrefslogtreecommitdiffstats
path: root/org.eclipse.jgit.test/tst/org
Commit message (Collapse)AuthorAgeFilesLines
* Merge branch 'stable-5.1' into stable-5.2Matthias Sohn2019-06-053-2/+446
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.1: Prepare 5.1.9-SNAPSHOT builds JGit v5.1.8.201906050907-r Test detecting modified packfiles Enhance fsTick() to use filesystem timer resolution Add debug trace to measure time needed to open pack index Extend FileSnapshot for packfiles to also use checksum to detect changes Wait opening new packfile until it can't be racy anymore Avoid null PackConfig in GC Add FileSnapshot test testing recognition of file size changes Capture reason for result of FileSnapshot#isModified Skip FileSnapshotTest#testSimulatePackfileReplacement on Windows Tune max heap size for tests Fix FileSnapshotTest.testNewFileNoWait() to match its javadoc ObjectDirectory: fix closing of obsolete packs Include filekey file attribute when comparing FileSnapshots Measure file timestamp resolution used in FileSnapshot Fix FileSnapshot's consideration of file size Fix API problem filters Change-Id: I3ac77bfa03f7436de12ab86e1bba29afee5ccd01 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
| * Test detecting modified packfilesChristian Halstrick2019-06-051-11/+222
| | | | | | | | | | | | | | | | | | | | | | | | | | Test that JGit detects that packfiles have changed even if they are repacked multiple times in one tick of the filesystem timer. Test that this detection works also when repacking doesn't change the length or the filekey of the packfile. In this case where a modified file can't be detected by looking at file metadata JGit should still detect too fast modification by racy git checks and trigger rescanning the pack list and consequently rereading of packfile content. Change-Id: I67682cfb807c58afc6de9375224ff7489d6618fb Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
| * Extend FileSnapshot for packfiles to also use checksum to detect changesMatthias Sohn2019-06-041-0/+143
| | | | | | | | | | | | | | | | | | | | | | | | | | | | If the attributes of FileSnapshot don't detect modification of a packfile read the packfile's checksum and compare it against the checksum cached in the loaded packfile. Since reading the checksum needs less IO than reloading the complete packfile this may help to reduce the overhead to detect modficiation when a gc completes while ObjectDirectory scans for packfiles in another thread. Bug: 546891 Change-Id: I9811b497eb11b8a85ae689081dc5d949ca8c4be5 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
| * Add FileSnapshot test testing recognition of file size changesMatthias Sohn2019-05-291-0/+17
| | | | | | | | Change-Id: Ibcd76a5e6e4183ada0be1d4436ce957243f2094d Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
| * Capture reason for result of FileSnapshot#isModifiedMatthias Sohn2019-05-291-0/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This allows to verify the expected behavior in FileSnapshotTest#testSimulatePackfileReplacement and enables extending FileSnapshot for packfiles to read the packfile's checksum as another criterion to detect modifications without reading the full content. Also add another field capturing the result of the last check if lastModified was racily clean. Remove unnecessary determination of raciness in the constructor. It was determined twice in all relevant cases. Change-Id: I100a2f49d7949693d7b72daa89437e166f1dc107 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
| * Skip FileSnapshotTest#testSimulatePackfileReplacement on WindowsMatthias Sohn2019-05-291-0/+3
| | | | | | | | | | | | NTFS does not support FileKey hence ignore this test on Windows. Change-Id: I7b53a591daa5e03eb5e401b5b26d612ab68ce10d Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
| * Fix FileSnapshotTest.testNewFileNoWait() to match its javadocMatthias Sohn2019-05-271-2/+0
| | | | | | | | | | | | | | testNewFileNoWait() was identical to testNewFileWithWait() but claims it doesn't wait at all. Hence remove the waits. Change-Id: I49b8ca5cb49a43c55fe61870c18c42f32fb4b74d Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
| * Include filekey file attribute when comparing FileSnapshotsMatthias Sohn2019-05-221-0/+28
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Due to finite filesystem timestamp resolution the last modified timestamp of files cannot detect file changes which happened in the immediate past (less than one filesystem timer tick ago). Some filesystems expose unique file identifiers, e.g. inodes in Posix filesystems which are named filekeys in Java's BasicFileAttributes. Use them as another means to detect file modifications based on stat information. Running git gc on a repository yields a new packfile with the same id as a packfile which existed before the gc if these packfiles contain the same set of objects. The content of the old and the new packfile might differ if a different PackConfig was used when writing the packfile. Considering filekeys in FileSnapshot may help to detect such packfile modifications. Bug: 546891 Change-Id: I711a80328c55e1a31171d540880b8e80ec1fe095 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
| * Measure file timestamp resolution used in FileSnapshotMatthias Sohn2019-05-221-0/+37
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | FileSnapshot.notRacyClean() assumed a worst case filesystem timestamp resolution of 2.5 sec (FAT has a resolution of 2 sec). Instead measure timestamp resolution to avoid unnecessary IO caused by false positives in detecting the racy git problem caused by finite filesystem timestamp resolution [1]. Cache the measured resolution per FileStore since timestamp resolution depends on the respective filesystem type. If timestamp resolution cannot be measured or fails due to an exception fallback to the worst case FAT timestamp resolution and avoid caching this value. Add a 10% safety margin in FileSnapshot.notRacyClean(), though running FsTest.testFsTimestampResolution() 1000 times which is not using a safety margin didn't fail on Mac using APFS and Java 8, 11, 12. Measured Java file timestamp resolution: [2] [1] https://github.com/git/git/blob/master/Documentation/technical/racy-git.txt [2] https://docs.google.com/spreadsheets/d/1imy0y6WmRqBf0kjCxzxj2X7M50eIVfa7oaUIzEOHmjo Bug: 546891 Change-Id: I493f3b57b6b306285ffa7d392339d253e5966ab8 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* | Revert 4678f4b and provide another solution for bug 467631Thomas Wolf2019-04-203-19/+96
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Making gitlinks and folders match in a tree walk was the wrong approach to fix bug 467631. The problem is that in such a conflict the tree walk may then not descend into the folder. Revert the changes to Paths.java and PathsTest.java from commit 4678f4b. Instead test for the problem case from bug 467631 explicitly in IndexDiff. Add Daniel's test case from bug 545162, and add yet another test case for DiffEntry.scan() that covers the problem originally reported in bug 545162. Bug: 545162 Change-Id: Ie2214c5d5ee32ac6596b621f0f1c7b86d38fa9b7 Also-by: Daniel Veihelmann <daniel.veihelmann@gmail.com> Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* | Merge branch 'stable-5.1' into stable-5.2Matthias Sohn2019-03-211-0/+17
|\| | | | | | | | | | | | | | | * stable-5.1: Fix GC to delete empty fanout directories after repacking Change-Id: I0cf4d26bdee5ecee43e723c4176efbce777acabc Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
| * Merge branch 'stable-5.0' into stable-5.1Matthias Sohn2019-03-211-0/+17
| |\ | | | | | | | | | | | | | | | | | | | | | * stable-5.0: Fix GC to delete empty fanout directories after repacking Change-Id: I5c0e7d59f137c27e4588f20f4472d3ea450cd59c Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
| | * Merge branch 'stable-4.11' into stable-5.0Matthias Sohn2019-03-211-0/+17
| | |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-4.11: Fix GC to delete empty fanout directories after repacking Change-Id: Idce894a24e126e0fbe7bc9b6a3c64318f1a8eb75 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
| | | * Merge branch 'stable-4.10' into stable-4.11Matthias Sohn2019-03-211-0/+17
| | | |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-4.10: Fix GC to delete empty fanout directories after repacking Change-Id: I7118b9c668dcbb0f5435cc613e964c557bfebf01 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
| | | | * Merge branch 'stable-4.9' into stable-4.10Matthias Sohn2019-03-211-0/+17
| | | | |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-4.9: Fix GC to delete empty fanout directories after repacking Change-Id: Ibdbfe08eb290286fa738010bad1c604e857885cd Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
| | | | | * Fix GC to delete empty fanout directories after repackingMatthias Sohn2019-03-201-0/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The prune method did not delete empty fanout directories when loose objects moved to a new pack file but only when loose unreferenced objects were pruned. Change-Id: Ia068f4914c54d9cf9f40b75e8ea50759402b5000 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
| | * | | | Merge branch 'stable-4.11' into stable-5.0David Pursehouse2019-03-061-0/+52
| | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-4.11: Fix error log message in ObjectDirectory.handlePackError() Properly format pack checksums in PackFile.idx() Cancel gc if thread was interrupted PackFile: report correct message for checksum mismatch ObjectDirectory: Clean up logging Bazel: Stop using native.git_repository ObjectDirectory: extra logging on packfile exceptions Change-Id: If75b149e693005dd3fe06b523e6e6784bedf44c1 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
| | | * | | Merge branch 'stable-4.10' into stable-4.11David Pursehouse2019-03-061-0/+52
| | | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-4.10: Fix error log message in ObjectDirectory.handlePackError() Properly format pack checksums in PackFile.idx() Cancel gc if thread was interrupted PackFile: report correct message for checksum mismatch ObjectDirectory: Clean up logging Bazel: Stop using native.git_repository ObjectDirectory: extra logging on packfile exceptions Change-Id: I9052e318b5d920770f7c7121d36e3c58df9d5f5a Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
| | | | * | Merge branch 'stable-4.9' into stable-4.10David Pursehouse2019-03-061-0/+52
| | | | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-4.9: Fix error log message in ObjectDirectory.handlePackError() Properly format pack checksums in PackFile.idx() Cancel gc if thread was interrupted PackFile: report correct message for checksum mismatch ObjectDirectory: Clean up logging Bazel: Stop using native.git_repository ObjectDirectory: extra logging on packfile exceptions Change-Id: I0847251eb010616a705e0b91df4bdebc225fa95d Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
| | | | | * Cancel gc if thread was interruptedMatthias Sohn2019-03-061-0/+52
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | see https://groups.google.com/d/msg/repo-discuss/oDB2rl3doDc/tFEh5Xt0CAAJ Change-Id: Ia6d4631c64e065d8b9b09e0b45e7a9ea8ac3f41d Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> (cherry picked from commit 882fed0d96c533513c43ae77aaff9cc07b94012c)
* | | | | | Merge branch 'stable-5.1' into stable-5.2David Pursehouse2019-03-051-0/+52
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.1: Fix error log message in ObjectDirectory.handlePackError() Properly format pack checksums in PackFile.idx() Cancel gc if thread was interrupted PackFile: report correct message for checksum mismatch ObjectDirectory: Clean up logging Bazel: Stop using native.git_repository Change-Id: I42b72a3bba3b4c9389d91a1a35a8004836567e7c Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
| * | | | | Cancel gc if thread was interruptedMatthias Sohn2019-03-031-0/+52
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | see https://groups.google.com/d/msg/repo-discuss/oDB2rl3doDc/tFEh5Xt0CAAJ Change-Id: Ia6d4631c64e065d8b9b09e0b45e7a9ea8ac3f41d Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* | | | | | Merge branch 'stable-5.1' into stable-5.2Jonathan Nieder2018-12-261-28/+15
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.1: UploadPack: Avoid calling AdvertiseRefsHook twice Prepare 5.1.5-SNAPSHOT builds JGit v5.1.4.201812251853-r UploadPack: Filter refs used for want-ref resolution UploadPack: Defer want-ref resolution to after parsing Call AdvertiseRefsHook for protocol v2 Prepare 4.11.7-SNAPSHOT builds JGit v4.11.6.201812241910-r Prepare 4.9.9-SNAPSHOT builds JGit v4.9.8.201812241815-r UploadPack: Test filtering by AdvertiseRefsHook in stateless transports Prepare 4.7.8-SNAPSHOT builds JGit v4.7.7.201812240805-r Fix feature versions imported by feature org.eclipse.jgit.pgm Prepare 4.5.6-SNAPSHOT builds JGit v4.5.5.201812240535-r Call AdvertiseRefsHook before validating wants Change-Id: I5879df9b723a0dbf6a1eff89a34bbb269f3b773d Signed-off-by: Jonathan Nieder <jrn@google.com>
| * | | | | UploadPack: Defer want-ref resolution to after parsingJonathan Nieder2018-12-261-30/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ProtocolV2Parser explains: // TODO(ifrade): This validation should be done after the // protocol parsing. It is not a protocol problem asking for an // unexisting ref and we wouldn't need the ref database here. Do so. This way all ref database accesses are in one place, in the UploadPack class. No user-visible change intended --- this is just to make the code easier to manipulate. Change-Id: I68e87dff7b9a63ccc169bd0836e8e8baaf5d1048 Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* | | | | | Set git environment variables for hooksThomas Wolf2018-11-301-2/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Set GIT_DIR and GIT_WORK_TREE when calling hooks. Bug: 541622 Change-Id: I6153d8a6a934ec37a3a5e7319c2d0e516f539ab7 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* | | | | | Merge "Fix IndexDiffs for git links"Christian Halstrick2018-11-285-1/+116
|\ \ \ \ \ \
| * | | | | | Fix IndexDiffs for git linksThomas Wolf2018-11-265-1/+116
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | After cloning a repo with a submodule, non-recursively, JGit would encounter in its TreeWalk in IndexDiff: * first, a missing gitlink (in index & HEAD, not in working tree) * second, the untracked folder (not in index and head, in working tree) As a result, it would report the submodule as missing. Canonical git reports a clean workspace. The root cause of this is that the path of a gitlink "x" did not compare equal to the path of a tree "x" in JGit. Correct Paths.compare() to account for that. If two paths are otherwise equal, then let gitlinks match both trees and files. Matching trees solves the bug. Matching files is necessary to handle the case where the gitlink directory was replaced by a file; see the new test case IndexDiffSubmoduleTest.testSubmoduleReplacedByFile(). Comparisons of unequal paths are left untouched, so the sort order is unchanged. After the fix, another bug(?) in WorkingTreeIterator became apparent: with core.dirNoGitLinks = true, it was no longer possible to overwrite a gitlink in the index. This is now fixed in WorkingTreeIterator. Add new test cases for the bug itself and for some related cases (submodule directory deleted or replaced by a file) in IndexDiffSubmoduleTest. Add a test for missing files in IndexDiffTest, and adapt the PathsTest to test matching gitlinks. Bug: 467631 Change-Id: I0549d10d46b1858e5eec3cc15095aa9f1d5f5280 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* | | | | | | Merge branch 'stable-5.1'David Pursehouse2018-11-271-16/+39
|\ \ \ \ \ \ \ | |/ / / / / / |/| / / / / / | |/ / / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.1: Fix DescribeCommand with multiple match options Fix git-describe tie-breakers Change-Id: I64d9b0099fc0d8449ec08f0b5653572b5c2f840f Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
| * | | | | Merge branch 'stable-5.0' into stable-5.1David Pursehouse2018-11-271-16/+39
| |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.0: Fix DescribeCommand with multiple match options Fix git-describe tie-breakers Change-Id: I63e7a56bb617b5ce8774e1dc7f5efdde25e7cd97 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
| | * | | | Merge branch 'stable-4.11' into stable-5.0David Pursehouse2018-11-271-16/+39
| | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-4.11: Fix DescribeCommand with multiple match options Fix git-describe tie-breakers Change-Id: Ibb98f143ee0ce7635beb30ec404b4134a73788f6 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
| | | * | | Fix DescribeCommand with multiple match optionsMatthias Sohn2018-11-261-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | when multiple match options are given in git describe the result must not depend on the order of the match options. JGit wrongly picked the first match using the match options in the order they were defined. Fix this by concatenating the streams of matching tags for all match options and then choosing the first match on the concatenated stream sorted in tie break order. See https://git-scm.com/docs/git-describe#git-describe---matchltpatterngt Change-Id: Id01433d35fa16fb4c30526605bee041ac1d954b2 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
| | | * | | Fix git-describe tie-breakersHÃ¥vard Wall2018-11-091-15/+40
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Correct behaviour as git 1.7.1.1 is to resolve tie-breakers to choose the most recent tag. https://github.com/git/git/blob/master/Documentation/RelNotes/1.7.1.1.txt: * "git describe" did not tie-break tags that point at the same commit correctly; newer ones are preferred by paying attention to the tagger date now. Bug: 538610 Change-Id: Ib0b2a301997bb7f75935baf7005473f4de952a64 Signed-off-by: HÃ¥vard Wall <haavardw@gmail.com>
* | | | | | Add a method to get all values of HTTP header defined as listMatthias Sohn2018-11-261-0/+123
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | According to RFC 2616 [1] header field names are case insensitive. Header fields defined as a comma separated list can have multiple header fields with the same field name. Add a method to HttpConnection which retrieves all values with a given header field name with the field name compared case insensitive. [1] https://tools.ietf.org/html/rfc2616#section-4.2" Change-Id: I7f601b21cda99e84f43f866c7c7cb4cb0e3cf5c3 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* | | | | | Undo treating blobs with NULs as a single lineThomas Wolf2018-11-263-5/+79
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This partially reverts commit a551b646: revert the changes in RawParseUtils.lineMap(). Forcing all blobs containing a NUL byte as a single line causes blame to produce useless results as soon as it hits any version containing a NUL byte. Doing binary detection at this level also has the problem that the user cannot control it. Not by setting the text attribute nor in any other way. This came up in bug 541036, where a Java source inadvertently contained NUL bytes in strings. Even fixing this by using escapes "\000" will not fix JGit's blame for this file because the past versions will still contain the NUL byte. Native git can blame that file from bug 541036 fine. Added new tests verifying that blaming a text file containing a NUL byte produces sensible results. Bug: 541036 Change-Id: I8991bec88e9827cc096868c6026ea1890b6d0d32 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* | | | | | New GpgConfig for typed access to GPG settingsGunnar Wagenknecht2018-11-201-0/+147
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Change-Id: Ifece38132e0390931d33b5678bbbd4a43f9099da Signed-off-by: Gunnar Wagenknecht <gunnar@wagenknecht.org>
* | | | | | Apache MINA sshd clientThomas Wolf2018-11-132-1042/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add a new ssh client implementation based on Apach MINA sshd 2.0.0. This implementation uses JGit's own config file parser and host entry resolver. Code inspection of the Apache MINA implementation revealed a few bugs or idiosyncrasies that immediately would re-introduce bugs already fixed in the past in JGit. Apache MINA sshd is not without quirks either, and I had to configure and override more than I had expected. But at least it was all doable in clean ways. Apache MINA boasts support for Bouncy Castle, so in theory this should open the way to using more ssh key algorithms, such as ed25519. The implementation is in a separate bundle and is still not used in the core org.eclipse.jgit bundle. The tests re-use the ssh tests from the core test bundle. Bug: 520927 Change-Id: Ib35e73c35799140fe050d1ff4fb18d0d3596580e Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* | | | | | Add more ssh tests: pushing, known_host file handling, etc.Thomas Wolf2018-11-133-206/+786
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add support for git-receive-pack to the ssh git server and add two new tests for pushing. This actually uncovered an undocumented requirement in TransportSftp: the FTP rename operation assumes POSIX semantics, i.e., that the target is removed. This works as written only for servers that support and advertise the "posix-rename@openssh.com" FTP extension. Our little Apache MINA server does not advertise this extension. Fix the FtpChannel implementation for Jsch to handle this case in a meaningful way so that it can pass the new "push over sftp" test. Add more tests to test the behavior of server host key checking. Also refactor the tests generally to separate better the test framework from the actual tests. Bug: 520927 Change-Id: Ia4bb85e17ddacde7b36ee8c2d5d454bbfa66dfc3 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* | | | | | DepthGenerator: fix multi-child boundary handlingJonathan Tan2018-11-081-0/+85
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Suppose that a repository has the following commit graph: B C \ / A and it was cloned with --shallow-exclude=A. DepthGenerator does not mark C as shallow, causing an invalid repository to be produced on the client, because A is not sent. (A similar issue occurs when --shallow-since is used to exclude A but neither B nor C.) This happens whenever an excluded commit has more than one child that is to be sent to the client. Fix DepthGenerator to handle this case correctly. While we're editing DepthWalk.Commit, fix the documentation of DepthWalk.Commit#isBoundary. Change-Id: I7068abf0fe0c864d1b0e56e1616dad1aa8719411 Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
* | | | | | SshTestBase: Make inner class TestCredentialsProvider staticDavid Pursehouse2018-11-081-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Change-Id: Id5ba87b24e8a29157d49e9e5b151d65bdbcab37a Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* | | | | | Factor out a JSch-independent ssh config parserThomas Wolf2018-11-061-16/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Move the bulk of the basic parsing and host entry handling into a new class OpenSshConfigFile that has no dependencies on any concrete ssh implementation. Make the existing OpenSshConfig use the new parser. Introduce a new class SshConstants collecting all the various ssh- related string literals. Also use TreeMaps with a case-insensitive key comparator instead of converting keys to uppercase. Add a test to verify that keys are matched case-insensitively. Most of the parsing code was simply moved, except that the new parser supports looking up entries given host name, port, and user name, and can thus handle more %-substitutions correctly. This feature is not yet used and cannot be used with JSch since JSch only has a ConfigRepository.getConfig(String) interface. The split is still worth the trouble as it opens the way to using another ssh client altogether. Apache MINA sshd, for instance, resolves host entries giving host name, port, and user name. (Apache MINA has a built-in ssh config handling, but that has problems, too: its pattern matching is case-insensitive, and its merging of host entries if several match is not the same as in OpenSsh. But with this refactoring, it will be possible to plug in OpenSshConfigFile into an Apache MINA sshd client without dragging along JSch.) One test case that doesn't make sense anymore has been removed. It tested that repeatedly querying for a host entry returned the same object. That is no longer true since the caching has been moved to a deeper level. Bug: 520927 Change-Id: I6381d52b29099595e6eaf8b05c786aeeaefbf9cc Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* | | | | | Ssh tests with an Apache MINA sshd test git serverThomas Wolf2018-11-063-0/+594
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add a simple ssh git server based on Apache MINA sshd, and use it in new tests that verify ssh operations and in particular a number of bugs that had cropped up over time in JSch. The git server supports fetching only, and sftp access. The tests are all in an abstract base class; the concrete JschSshTest class only provides ssh-specific test setup. So the same tests could be run easily also with some other ssh client. Bug: 520927 Change-Id: Ide6687b717fb497a29fc83f22b07390a26dfce1d Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* | | | | | Merge changes from topic 'client-caps-v0-test'Jonathan Nieder2018-11-023-23/+313
|\ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * changes: Add test for protocol v0 parser ObjectIdMatcher: Custom matcher for sets of ObjectIds
| * | | | | | Add test for protocol v0 parserIvan Frade2018-10-241-0/+199
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ProtocolV2Parser has unit tests but protocol v0/v1 is not covered. Change-Id: I96022e8f8eb60d4da748d1042474fd1efd67e882 Signed-off-by: Ivan Frade <ifrade@google.com>
| * | | | | | ObjectIdMatcher: Custom matcher for sets of ObjectIdsIvan Frade2018-10-242-23/+114
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Parsed requests represent object ids (SHA1) in ObjectId instances but tests use strings for those ids because they are easier to define. Create a custom matcher that hides the conversion from string to ObjectId. Note that this reverses the existing code conversion (it was transforming ObjectIds into string). This produces more readable code, consistent with the other hamcrest assertions. Change-Id: I47ba1d25557d791fe74fb93c740ff7de9923cc00 Signed-off-by: Ivan Frade <ifrade@google.com>
* | | | | | | RepoCommand: Preserve executable bit in <copyfile>Ivan Frade2018-10-311-0/+118
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The copyfile entry in the manifest file copies the contents of the file but doesn't keep the executable flag. This is inconsistent with repo tool behaviour, plus is natural to expect that the copy of a executable file is executable. Transfer the executable bit when copying the file, aligning the RepoCommand with repo tool and user expectations. Change-Id: I01b24f482d5939e01d496f032388b3a5c02a912a Signed-off-by: Ivan Frade <ifrade@google.com>
* | | | | | | RepoCommand.RemoteReader: Add method to read contents and mode of fileIvan Frade2018-10-311-12/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The RepoCommand.RemoteReader interface doesn't offer access to the mode of a file. Caller can only default to mark the copied objects as regular files, losing e.g. the executable bit (if set). Add a new method readFileWithMode that returns the contents and mode of the remote file. It supersedes the readFile method, that is marked as deprecated. Now callers can set correctly the file mode of the copied file. Change-Id: I8fce01e4bc5707434c0cbc4aebbae1b6b64756f0 Signed-off-by: Ivan Frade <ifrade@google.com>
* | | | | | | RepoCommandTest: Extract method to assert file contentsIvan Frade2018-10-301-42/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Many tests verify the contents of files in a try-with-resources incantation that clutters the code. Extract that verification to an "assertContents" method, that is easier to read. Change-Id: If430eac6f5b9ae352e42b2d43867ceb6cd618fbb Signed-off-by: Ivan Frade <ifrade@google.com>
* | | | | | | gitignore/gitattributes: fix matching of \rMarc Strapetz2018-10-272-0/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Patterns should treat \r in file names as normal characters Change-Id: Ica3e0fa4a58acf5326db46bb28571fe5f20f6cd2 Signed-off-by: Marc Strapetz <marc.strapetz@syntevo.com>
* | | | | | | UploadPack: Implement deepen-not for protocol v2Jonathan Tan2018-10-231-0/+106
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This allows clients to use the --shallow-exclude parameter (producing a "deepen-not <ref>" line when communicating with the server) in their fetch commands when fetching against a JGit server using protocol v2. Note that the implementation in this commit is somewhat inefficient, as described in the TODO comment in DepthGenerator. Change-Id: I9fad3ed9276b624d8f668356ffd99a067dc67ef7 Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
* | | | | | | Throw error when deepen-since excludes all commitsJonathan Tan2018-10-231-0/+20
|/ / / / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In C Git, when a client fetches with "git fetch --shallow-since=<date> origin <ref>", and all commits reachable from <ref> are older than <date>, the server dies with a message "no commits selected for shallow requests". That is, (1) the --shallow-since filter applies to the commit pointed to by the ref itself, and (2) there is a check that at least one commit is not filtered out. (The pack-protocol.txt documentation does not describe this, but the C implementation does this.) The implementation in commit 1bb430dc21 ("UploadPack: support deepen-since in protocol v2", 2018-09-27) does neither (1) nor (2), so do both of these. Change-Id: I9946327a71627626ecce34ca2d017d2add8867fc Signed-off-by: Jonathan Tan <jonathantanmy@google.com>