Juergen Denner [Tue, 15 Jan 2019 12:22:54 +0000 (13:22 +0100)]
Reduce contention on PackFile.idx() function.
In case of concurrent pack file access, threads may wait on the idx()
function even for already open files. This happens especially with a
slow file system.
* stable-4.11:
Fix error log message in ObjectDirectory.handlePackError()
Properly format pack checksums in PackFile.idx()
Cancel gc if thread was interrupted
PackFile: report correct message for checksum mismatch
ObjectDirectory: Clean up logging
Bazel: Stop using native.git_repository
ObjectDirectory: extra logging on packfile exceptions
Change-Id: If75b149e693005dd3fe06b523e6e6784bedf44c1 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* stable-4.10:
Fix error log message in ObjectDirectory.handlePackError()
Properly format pack checksums in PackFile.idx()
Cancel gc if thread was interrupted
PackFile: report correct message for checksum mismatch
ObjectDirectory: Clean up logging
Bazel: Stop using native.git_repository
ObjectDirectory: extra logging on packfile exceptions
Change-Id: I9052e318b5d920770f7c7121d36e3c58df9d5f5a Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* stable-4.9:
Fix error log message in ObjectDirectory.handlePackError()
Properly format pack checksums in PackFile.idx()
Cancel gc if thread was interrupted
PackFile: report correct message for checksum mismatch
ObjectDirectory: Clean up logging
Bazel: Stop using native.git_repository
ObjectDirectory: extra logging on packfile exceptions
Change-Id: I0847251eb010616a705e0b91df4bdebc225fa95d Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Luca Milanesio [Sat, 23 Feb 2019 21:57:09 +0000 (21:57 +0000)]
PackFile: report correct message for checksum mismatch
When the packfile checksum does not match the expected one
report the correct checksum error instead of reporting that
the number of objects is incorrect.
Change-Id: I040f36dacc4152ae05453e7acbf8dfccceb46e0d Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
(cherry picked from commit 436c99ce5946f31f06b8704b1fd33136f39dc814)
Jonathan Nieder [Wed, 26 Dec 2018 20:40:28 +0000 (12:40 -0800)]
UploadPack: Avoid calling AdvertiseRefsHook twice
The AdvertiseRefsHook can be called twice if the following conditions
hold:
1. This AdvertiseRefsHook doesn't set this.refs.
2. getAdvertisedOrDefaultRefs is called after getFilteredRefs.
For example, this can happen when fetchV2 is called after lsRefsV2
when using a stateful bidirectional transport.
The second call does not accomplish anything useful. Guard it with
'if (!advertiseRefsHookCalled)' to avoid wasted work.
Reported-by: Jonathan Tan <jonathantanmy@google.com>
Change-Id: Ib746582e4ef645b767a5b3fb969596df99ac2ab5 Signed-off-by: Jonathan Nieder <jrn@google.com>
Masaya Suzuki [Tue, 18 Dec 2018 17:20:54 +0000 (09:20 -0800)]
Call AdvertiseRefsHook for protocol v2
AdvertiseRefsHook is used to limit the visibility of the refs in Gerrit.
If this hook is not called, then all refs are treated as visible.
In protocol v2, the hook is not called, causing the server to advertise
all refs. This bug was introduced in v5.0.0.201805221745-rc1~1^2~9
(Execute AdvertiseRefsHook only for protocol v0 and v1, 2018-05-14).
Even before then, the hook was not called in requests after the
capability advertisement, so in transports like HTTP that do not retain
state between round-trips, the server would advertise all refs in
response to an ls-refs (ls-remote) request.
Fix both cases by using getAdvertisedOrDefaultRefs to retrieve the
advertised refs in lsRefs, ensuring the hook is called in all cases that
use its result.
[jn: backported to stable-5.0; split out from a larger patch that also
fixes protocol v0; avoided filtering this.refs by ref prefix]
Change-Id: I64bce0e72d15b90baccc235c067e57b6af21b55f Signed-off-by: Masaya Suzuki <masayasuzuki@google.com> Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Tue, 25 Dec 2018 22:27:13 +0000 (23:27 +0100)]
Merge branch 'stable-4.11' into stable-5.0
* stable-4.11:
Prepare 4.11.7-SNAPSHOT builds
JGit v4.11.6.201812241910-r
Prepare 4.9.9-SNAPSHOT builds
JGit v4.9.8.201812241815-r
UploadPack: Test filtering by AdvertiseRefsHook in stateless transports
Prepare 4.7.8-SNAPSHOT builds
JGit v4.7.7.201812240805-r
Fix feature versions imported by feature org.eclipse.jgit.pgm
Prepare 4.5.6-SNAPSHOT builds
JGit v4.5.5.201812240535-r
Call AdvertiseRefsHook before validating wants
Change-Id: Ie81284ca6d580b0712c49eec610393d0c0c50203 Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Mon, 24 Dec 2018 23:49:46 +0000 (00:49 +0100)]
Merge branch 'stable-4.10' into stable-4.11
* stable-4.10:
Prepare 4.9.9-SNAPSHOT builds
JGit v4.9.8.201812241815-r
UploadPack: Test filtering by AdvertiseRefsHook in stateless transports
Prepare 4.7.8-SNAPSHOT builds
JGit v4.7.7.201812240805-r
Fix feature versions imported by feature org.eclipse.jgit.pgm
Prepare 4.5.6-SNAPSHOT builds
JGit v4.5.5.201812240535-r
Call AdvertiseRefsHook before validating wants
Change-Id: I937e9a4547fc10e4de7c887163022d1ab0322d64 Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Mon, 24 Dec 2018 23:33:44 +0000 (00:33 +0100)]
Merge branch 'stable-4.9' into stable-4.10
* stable-4.9:
Prepare 4.9.9-SNAPSHOT builds
JGit v4.9.8.201812241815-r
UploadPack: Test filtering by AdvertiseRefsHook in stateless transports
Prepare 4.7.8-SNAPSHOT builds
JGit v4.7.7.201812240805-r
Fix feature versions imported by feature org.eclipse.jgit.pgm
Prepare 4.5.6-SNAPSHOT builds
JGit v4.5.5.201812240535-r
Call AdvertiseRefsHook before validating wants
Change-Id: I2e499f34b1c481af794fa9325b0dfebaccdf3cb0 Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Masaya Suzuki [Tue, 18 Dec 2018 17:20:54 +0000 (09:20 -0800)]
UploadPack: Test filtering by AdvertiseRefsHook in stateless transports
AdvertiseRefsHook is used to limit the visibility of the refs in Gerrit.
If this hook is not called, then all refs are treated as visible,
causing the server to serve commits reachable from branches the client
should not be able to access, if asked to via a request naming a guessed
object id.
Until 3a529361a76e8267467071e0b13ebb36b97d8fb2 (Call AdvertiseRefsHook
before validating wants, 2018-12-18), UploadPack would invoke this hook
at ref advertisement time but not during negotiation and when serving a
pack file. Add a test to avoid regressing. Stateful bidirectional
transports were not affected, so the test uses HTTP.
[jn: split out when backporting the fix to stable-4.5. The test passes
as long as v4.9.0.201710071750-r~169 (fetch: Accept any SHA-1 on lhs of
refspec, 2017-06-04) is cherry picked along with it.]
Change-Id: I8c017107336adc7cb4c826985779676bf043e648 Signed-off-by: Masaya Suzuki <masayasuzuki@google.com> Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Change-Id: Ib44e314a68bca2349b45f4937257aa1298c8d74b Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Change-Id: I68a21067705b580b40840f8039001ff1e5273c15 Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Mon, 24 Dec 2018 12:25:31 +0000 (13:25 +0100)]
Merge branch 'stable-4.6' into stable-4.7
* stable-4.6:
Fix feature versions imported by feature org.eclipse.jgit.pgm
Prepare 4.5.6-SNAPSHOT builds
JGit v4.5.5.201812240535-r
Call AdvertiseRefsHook before validating wants
Change-Id: If637694f80dbd1e774d60c672fe78a6500650bb8 Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Change-Id: I0fd67ddd9c4966c20d82cdfe78b2f9d4898b4665 Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Masaya Suzuki [Tue, 18 Dec 2018 17:20:54 +0000 (09:20 -0800)]
Call AdvertiseRefsHook before validating wants
AdvertiseRefsHook is used to limit the visibility of the refs in Gerrit.
If this hook is not called, then all refs are treated as visible,
causing the server to serve commits reachable from branches the client
should not be able to access, if asked to via a request naming a guessed
object id.
This bug was introduced in v2.0.0.201206130900-r~123 (Modify refs in
UploadPack/ReceivePack using a hook interface, 2012-02-08). Stateful
bidirectional transports are not affected.
Fix it by moving the AdvertiseRefsHook call to
getAdvertisedOrDefaultRefs, ensuring the hook is called in all cases.
[jn: backported to stable-4.5 by splitting out tests and the protocol v2
specific parts]
Change-Id: I159f396216354f2eda3968d17802e166d8c8ec2d Signed-off-by: Masaya Suzuki <masayasuzuki@google.com> Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Sun, 25 Nov 2018 22:15:11 +0000 (23:15 +0100)]
Fix DescribeCommand with multiple match options
when multiple match options are given in git describe the result must
not depend on the order of the match options. JGit wrongly picked the
first match using the match options in the order they were defined. Fix
this by concatenating the streams of matching tags for all match options
and then choosing the first match on the concatenated stream sorted in
tie break order.
See https://git-scm.com/docs/git-describe#git-describe---matchltpatterngt
Change-Id: Id01433d35fa16fb4c30526605bee041ac1d954b2 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
HÃ¥vard Wall [Wed, 17 Oct 2018 13:34:51 +0000 (15:34 +0200)]
Fix git-describe tie-breakers
Correct behaviour as git 1.7.1.1 is to resolve tie-breakers to choose
the most recent tag.
https://github.com/git/git/blob/master/Documentation/RelNotes/1.7.1.1.txt:
* "git describe" did not tie-break tags that point at the same commit
correctly; newer ones are preferred by paying attention to the
tagger date now.
SpotBugs [1] is the spiritual successor of FindBugs, carrying on from
the point where it left off with support of its community.
This is a backport of [1] which originally did the replacement on the
master branch. This change updates to the current latest version, so
that we can get the benefit of its checks when pushing changes to the
stable branches.
Jonathan Nieder [Sun, 7 Oct 2018 21:55:52 +0000 (21:55 +0000)]
SubmoduleValidator: Permit missing path or url
A .gitmodules file can include a submodule without a path to configure
the URL for a submodule that is only present on other branches.
A .gitmodules file can include a submodule with no URL and no path to
reserve the name for a submodule that existed in earlier history but
is not available from any URL any more.
"git fsck" permits both of these cases. Permit them in JGit as well
(instead of throwing NullPointerException).
Change-Id: I3b442639ad79ea7a59227f96406a12e62d3573ae Reported-by: David Pursehouse <david.pursehouse@gmail.com> Signed-off-by: Jonathan Nieder <jrn@google.com>
The text "<tree, blob>" with angle brackets should not be used in javadoc
since it is interpreted as an HTML tag and then rejected since it's not a
valid HTML tag. Wrap the text in a @literal tag.
Also add a missing space.
Change-Id: Ide045e8c04a39a916f5b2e964e58c151e4555830 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Matthias Sohn [Sat, 6 Oct 2018 12:44:12 +0000 (14:44 +0200)]
Merge branch 'stable-4.11' into stable-5.0
* stable-4.11:
Prepare 4.11.5-SNAPSHOT builds
JGit v4.11.4.201810060650-r
Fix configuration of maven-javadoc-plugin
Prepare 4.9.7-SNAPSHOT builds
JGit v4.9.6.201810051924-r
Prepare 4.7.6-SNAPSHOT builds
JGit v4.7.5.201810051826-r
BaseReceivePack: Validate incoming .gitmodules files
ObjectChecker: Report .gitmodules files found in the pack
SubmoduleAddCommand: Reject submodule URIs that look like cli options
* Fix configuration of maven-javadoc-plugin for site generation
Change-Id: Ic6ff8d324867ee41f15a5b890c7eee5092e8453e Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Sat, 6 Oct 2018 00:25:17 +0000 (02:25 +0200)]
Merge branch 'stable-4.10' into stable-4.11
* stable-4.10:
Prepare 4.9.7-SNAPSHOT builds
JGit v4.9.6.201810051924-r
Prepare 4.7.6-SNAPSHOT builds
JGit v4.7.5.201810051826-r
BaseReceivePack: Validate incoming .gitmodules files
ObjectChecker: Report .gitmodules files found in the pack
SubmoduleAddCommand: Reject submodule URIs that look like cli options
Change-Id: Ibd759f5d425f714e79b3137ff8e5b0f989933de0 Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Fri, 5 Oct 2018 23:52:38 +0000 (01:52 +0200)]
Merge branch 'stable-4.9' into stable-4.10
* stable-4.9:
Prepare 4.9.7-SNAPSHOT builds
JGit v4.9.6.201810051924-r
Prepare 4.7.6-SNAPSHOT builds
JGit v4.7.5.201810051826-r
BaseReceivePack: Validate incoming .gitmodules files
ObjectChecker: Report .gitmodules files found in the pack
SubmoduleAddCommand: Reject submodule URIs that look like cli options
Change-Id: Ie59e34eb591a827d1ce8e483eec6d390a3c81702 Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Fri, 5 Oct 2018 23:16:08 +0000 (01:16 +0200)]
Merge branch 'stable-4.8' into stable-4.9
* stable-4.8:
Prepare 4.7.6-SNAPSHOT builds
JGit v4.7.5.201810051826-r
BaseReceivePack: Validate incoming .gitmodules files
ObjectChecker: Report .gitmodules files found in the pack
SubmoduleAddCommand: Reject submodule URIs that look like cli options
Change-Id: Ia7a826399d8d5b8a0eb7169b40e98a6f5c207a4c Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Fri, 5 Oct 2018 23:03:20 +0000 (01:03 +0200)]
Merge branch 'stable-4.7' into stable-4.8
* stable-4.7:
Prepare 4.7.6-SNAPSHOT builds
JGit v4.7.5.201810051826-r
BaseReceivePack: Validate incoming .gitmodules files
ObjectChecker: Report .gitmodules files found in the pack
SubmoduleAddCommand: Reject submodule URIs that look like cli options
Change-Id: Id6fabec4d0b682a7e20a46e88cbc05432efca062 Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
The main concern are submodule urls starting with '-' that could pass as
options to an unguarded tool.
Pass through the parser the ids of blobs identified as .gitmodules
files in the ObjectChecker. Load the blobs and parse/validate them
in SubmoduleValidator.
Change-Id: Ia0cc32ce020d288f995bf7bc68041fda36be1963 Signed-off-by: Ivan Frade <ifrade@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Ivan Frade [Thu, 27 Sep 2018 20:05:13 +0000 (13:05 -0700)]
ObjectChecker: Report .gitmodules files found in the pack
In order to validate .gitmodules files, we first need to find them
in the incoming pack.
Do it in the ObjectChecker stage. Check in the tree objects if they
point to a .gitmodules file and report the tree id and the .gitmodules
blob id.
This can be used later to check if the file is in the root of the
project and if the contents are good.
While we're here, make isMacHFSGit more accurate by detecting variants
of filenames that vary in case.
[jn: tweaked NTFS and HFS+ checking; added more tests]
Change-Id: I70802e7d2c1374116149de4f89836b9498f39582 Signed-off-by: Ivan Frade <ifrade@google.com> Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Ivan Frade [Mon, 24 Sep 2018 23:03:35 +0000 (16:03 -0700)]
SubmoduleAddCommand: Reject submodule URIs that look like cli options
In C git versions before 2.19.1, the submodule is fetched by running
"git clone <uri> <path>". A URI starting with "-" would be interpreted
as an option, causing security problems. See CVE-2018-17456.
Refuse to add submodules with URIs, names or paths starting with "-",
that could be confused with command line arguments.
[jn: backported to JGit 4.7.y, bringing portions of Masaya Suzuki's
dotdot check code in v5.1.0.201808281540-m3~57 (Add API to specify
the submodule name, 2018-07-12) along for the ride]
Change-Id: I2607c3acc480b75ab2b13386fe2cac435839f017 Signed-off-by: Ivan Frade <ifrade@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
David Ostrovsky [Tue, 25 Sep 2018 06:20:57 +0000 (08:20 +0200)]
ObjectDownloadListener#onWritePossible: Add comment on return statement
It is not obvious why this return statement is needed. Clarify with a
comment that otherwise endless loop may show up when recent versions
of Jetty are used.
Change-Id: I8e5d4de51869fb1179bf599bfb81bcd7d745874b Signed-off-by: David Ostrovsky <david@ostrovsky.org>