Matthias Sohn [Wed, 13 Mar 2019 00:55:10 +0000 (01:55 +0100)]
Merge branch 'stable-4.10' into stable-4.11
* stable-4.10:
Prepare 4.9.10-SNAPSHOT builds
JGit v4.9.9.201903122025-r
Prepare 4.7.9-SNAPSHOT builds
JGit v4.7.8.201903121755-r
Prepare 4.5.7-SNAPSHOT builds
JGit v4.5.6.201903121547-r
Check for packfile validity and fd before reading
Move throw of PackInvalidException outside the catch
Use FileSnapshot to get lastModified on PackFile
Include size when comparing FileSnapshot
Do not reuse packfiles when changed on filesystem
Silence API warnings for new API introduced for fixes
Change-Id: Ic66dc00455c05f7627919f682a7221ffeef1629d Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Wed, 13 Mar 2019 00:48:06 +0000 (01:48 +0100)]
Merge branch 'stable-4.9' into stable-4.10
* stable-4.9:
Prepare 4.9.10-SNAPSHOT builds
JGit v4.9.9.201903122025-r
Prepare 4.7.9-SNAPSHOT builds
JGit v4.7.8.201903121755-r
Prepare 4.5.7-SNAPSHOT builds
JGit v4.5.6.201903121547-r
Check for packfile validity and fd before reading
Move throw of PackInvalidException outside the catch
Use FileSnapshot to get lastModified on PackFile
Include size when comparing FileSnapshot
Do not reuse packfiles when changed on filesystem
Silence API warnings for new API introduced for fixes
Change-Id: Ifcd8ea6a2b59f6f8e9d896d8407cfcaef0a9375d Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Wed, 13 Mar 2019 00:17:47 +0000 (01:17 +0100)]
Merge branch 'stable-4.8' into stable-4.9
* stable-4.8:
Prepare 4.7.9-SNAPSHOT builds
JGit v4.7.8.201903121755-r
Prepare 4.5.7-SNAPSHOT builds
JGit v4.5.6.201903121547-r
Check for packfile validity and fd before reading
Move throw of PackInvalidException outside the catch
Use FileSnapshot to get lastModified on PackFile
Include size when comparing FileSnapshot
Do not reuse packfiles when changed on filesystem
Silence API warnings for new API introduced for fixes
Change-Id: I9a47153831f8eb10d3cd91b4157cf45385e5b13a Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Wed, 13 Mar 2019 00:16:28 +0000 (01:16 +0100)]
Merge branch 'stable-4.7' into stable-4.8
* stable-4.7:
Prepare 4.7.9-SNAPSHOT builds
JGit v4.7.8.201903121755-r
Prepare 4.5.7-SNAPSHOT builds
JGit v4.5.6.201903121547-r
Check for packfile validity and fd before reading
Move throw of PackInvalidException outside the catch
Use FileSnapshot to get lastModified on PackFile
Include size when comparing FileSnapshot
Do not reuse packfiles when changed on filesystem
Silence API warnings for new API introduced for fixes
Change-Id: I1136eaa1590b225ceaf16ba7552af0374253a74e Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Tue, 12 Mar 2019 21:39:53 +0000 (22:39 +0100)]
Merge branch 'stable-4.6' into stable-4.7
* stable-4.6:
Prepare 4.5.7-SNAPSHOT builds
JGit v4.5.6.201903121547-r
Check for packfile validity and fd before reading
Move throw of PackInvalidException outside the catch
Use FileSnapshot to get lastModified on PackFile
Include size when comparing FileSnapshot
Do not reuse packfiles when changed on filesystem
Silence API warnings for new API introduced for fixes
Change-Id: I3d1544d034783fe0fa1385dfe9b03ad8e9247c63 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Tue, 12 Mar 2019 20:01:55 +0000 (21:01 +0100)]
Merge branch 'stable-4.5' into stable-4.6
* stable-4.5:
Prepare 4.5.7-SNAPSHOT builds
JGit v4.5.6.201903121547-r
Check for packfile validity and fd before reading
Move throw of PackInvalidException outside the catch
Use FileSnapshot to get lastModified on PackFile
Include size when comparing FileSnapshot
Do not reuse packfiles when changed on filesystem
Silence API warnings for new API introduced for fixes
Change-Id: I029e1797447e6729de68bd89d4d69b324dbb3f5f Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Luca Milanesio [Sun, 10 Mar 2019 22:03:40 +0000 (22:03 +0000)]
Check for packfile validity and fd before reading
When reading from a packfile, make sure that is valid
and has a non-null file-descriptor.
Because of concurrency between a thread invalidating a packfile
and another trying to read it, the read() may result into a NPE
that won't be able to be automatically recovered.
Throwing a PackInvalidException would instead cause the packlist
to be refreshed and the read to eventually succeed.
Luca Milanesio [Wed, 6 Mar 2019 11:30:07 +0000 (11:30 +0000)]
Move throw of PackInvalidException outside the catch
When a packfile is invalid, throw an exception explicitly
outside any catch scope, so that is not accidentally caught
by the generic catch-all cause, which would set the packfile
as valid again.
Flagging an invalid packfile as valid again would have
dangerous consequences such as the corruption of the in-memory
packlist.
Luca Milanesio [Tue, 12 Mar 2019 07:00:01 +0000 (07:00 +0000)]
Use FileSnapshot to get lastModified on PackFile
Do not redundantly call File.lastModified() for extracting the
timestamp of the PackFile but rather use consistently the FileSnapshot
which reads all file attributes in a single bulk call.
Change-Id: I932675ae4fe56dcd3833dac249816f097303bb09 Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Luca Milanesio [Wed, 6 Mar 2019 17:51:59 +0000 (17:51 +0000)]
Include size when comparing FileSnapshot
Due to finite filesystem timestamp resolution the last modified
timestamp of files cannot detect file changes which happened in the
immediate past (less than one filesystem timer tick ago).
Read and consider file size also, so that differing file size can help
to more accurately detect file changes without reading the file content.
Use bulk read to avoid multiple stat calls to retrieve file attributes.
Change-Id: I974288fff78ac78c52245d9218b5639603f67a46 Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Luca Milanesio [Wed, 6 Mar 2019 00:31:45 +0000 (00:31 +0000)]
Do not reuse packfiles when changed on filesystem
The pack reload mechanism from the filesystem works only by name
and does not check the actual last modified date of the packfile.
This lead to concurrency issues where multiple threads were loading
and removing from each other list of packfiles when one of those
was failing the checksum.
Rely on FileSnapshot rather than directly checking lastModified
timestamp so that more checks can be performed.
Juergen Denner [Tue, 15 Jan 2019 12:22:54 +0000 (13:22 +0100)]
Reduce contention on PackFile.idx() function.
In case of concurrent pack file access, threads may wait on the idx()
function even for already open files. This happens especially with a
slow file system.
* stable-4.10:
Fix error log message in ObjectDirectory.handlePackError()
Properly format pack checksums in PackFile.idx()
Cancel gc if thread was interrupted
PackFile: report correct message for checksum mismatch
ObjectDirectory: Clean up logging
Bazel: Stop using native.git_repository
ObjectDirectory: extra logging on packfile exceptions
Change-Id: I9052e318b5d920770f7c7121d36e3c58df9d5f5a Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* stable-4.9:
Fix error log message in ObjectDirectory.handlePackError()
Properly format pack checksums in PackFile.idx()
Cancel gc if thread was interrupted
PackFile: report correct message for checksum mismatch
ObjectDirectory: Clean up logging
Bazel: Stop using native.git_repository
ObjectDirectory: extra logging on packfile exceptions
Change-Id: I0847251eb010616a705e0b91df4bdebc225fa95d Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Luca Milanesio [Sat, 23 Feb 2019 21:57:09 +0000 (21:57 +0000)]
PackFile: report correct message for checksum mismatch
When the packfile checksum does not match the expected one
report the correct checksum error instead of reporting that
the number of objects is incorrect.
Change-Id: I040f36dacc4152ae05453e7acbf8dfccceb46e0d Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
(cherry picked from commit 436c99ce5946f31f06b8704b1fd33136f39dc814)
Matthias Sohn [Mon, 24 Dec 2018 23:49:46 +0000 (00:49 +0100)]
Merge branch 'stable-4.10' into stable-4.11
* stable-4.10:
Prepare 4.9.9-SNAPSHOT builds
JGit v4.9.8.201812241815-r
UploadPack: Test filtering by AdvertiseRefsHook in stateless transports
Prepare 4.7.8-SNAPSHOT builds
JGit v4.7.7.201812240805-r
Fix feature versions imported by feature org.eclipse.jgit.pgm
Prepare 4.5.6-SNAPSHOT builds
JGit v4.5.5.201812240535-r
Call AdvertiseRefsHook before validating wants
Change-Id: I937e9a4547fc10e4de7c887163022d1ab0322d64 Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Mon, 24 Dec 2018 23:33:44 +0000 (00:33 +0100)]
Merge branch 'stable-4.9' into stable-4.10
* stable-4.9:
Prepare 4.9.9-SNAPSHOT builds
JGit v4.9.8.201812241815-r
UploadPack: Test filtering by AdvertiseRefsHook in stateless transports
Prepare 4.7.8-SNAPSHOT builds
JGit v4.7.7.201812240805-r
Fix feature versions imported by feature org.eclipse.jgit.pgm
Prepare 4.5.6-SNAPSHOT builds
JGit v4.5.5.201812240535-r
Call AdvertiseRefsHook before validating wants
Change-Id: I2e499f34b1c481af794fa9325b0dfebaccdf3cb0 Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Masaya Suzuki [Tue, 18 Dec 2018 17:20:54 +0000 (09:20 -0800)]
UploadPack: Test filtering by AdvertiseRefsHook in stateless transports
AdvertiseRefsHook is used to limit the visibility of the refs in Gerrit.
If this hook is not called, then all refs are treated as visible,
causing the server to serve commits reachable from branches the client
should not be able to access, if asked to via a request naming a guessed
object id.
Until 3a529361a76e8267467071e0b13ebb36b97d8fb2 (Call AdvertiseRefsHook
before validating wants, 2018-12-18), UploadPack would invoke this hook
at ref advertisement time but not during negotiation and when serving a
pack file. Add a test to avoid regressing. Stateful bidirectional
transports were not affected, so the test uses HTTP.
[jn: split out when backporting the fix to stable-4.5. The test passes
as long as v4.9.0.201710071750-r~169 (fetch: Accept any SHA-1 on lhs of
refspec, 2017-06-04) is cherry picked along with it.]
Change-Id: I8c017107336adc7cb4c826985779676bf043e648 Signed-off-by: Masaya Suzuki <masayasuzuki@google.com> Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Change-Id: Ib44e314a68bca2349b45f4937257aa1298c8d74b Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Change-Id: I68a21067705b580b40840f8039001ff1e5273c15 Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Mon, 24 Dec 2018 12:25:31 +0000 (13:25 +0100)]
Merge branch 'stable-4.6' into stable-4.7
* stable-4.6:
Fix feature versions imported by feature org.eclipse.jgit.pgm
Prepare 4.5.6-SNAPSHOT builds
JGit v4.5.5.201812240535-r
Call AdvertiseRefsHook before validating wants
Change-Id: If637694f80dbd1e774d60c672fe78a6500650bb8 Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Change-Id: I0fd67ddd9c4966c20d82cdfe78b2f9d4898b4665 Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Masaya Suzuki [Tue, 18 Dec 2018 17:20:54 +0000 (09:20 -0800)]
Call AdvertiseRefsHook before validating wants
AdvertiseRefsHook is used to limit the visibility of the refs in Gerrit.
If this hook is not called, then all refs are treated as visible,
causing the server to serve commits reachable from branches the client
should not be able to access, if asked to via a request naming a guessed
object id.
This bug was introduced in v2.0.0.201206130900-r~123 (Modify refs in
UploadPack/ReceivePack using a hook interface, 2012-02-08). Stateful
bidirectional transports are not affected.
Fix it by moving the AdvertiseRefsHook call to
getAdvertisedOrDefaultRefs, ensuring the hook is called in all cases.
[jn: backported to stable-4.5 by splitting out tests and the protocol v2
specific parts]
Change-Id: I159f396216354f2eda3968d17802e166d8c8ec2d Signed-off-by: Masaya Suzuki <masayasuzuki@google.com> Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Sun, 25 Nov 2018 22:15:11 +0000 (23:15 +0100)]
Fix DescribeCommand with multiple match options
when multiple match options are given in git describe the result must
not depend on the order of the match options. JGit wrongly picked the
first match using the match options in the order they were defined. Fix
this by concatenating the streams of matching tags for all match options
and then choosing the first match on the concatenated stream sorted in
tie break order.
See https://git-scm.com/docs/git-describe#git-describe---matchltpatterngt
Change-Id: Id01433d35fa16fb4c30526605bee041ac1d954b2 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
HÃ¥vard Wall [Wed, 17 Oct 2018 13:34:51 +0000 (15:34 +0200)]
Fix git-describe tie-breakers
Correct behaviour as git 1.7.1.1 is to resolve tie-breakers to choose
the most recent tag.
https://github.com/git/git/blob/master/Documentation/RelNotes/1.7.1.1.txt:
* "git describe" did not tie-break tags that point at the same commit
correctly; newer ones are preferred by paying attention to the
tagger date now.
SpotBugs [1] is the spiritual successor of FindBugs, carrying on from
the point where it left off with support of its community.
This is a backport of [1] which originally did the replacement on the
master branch. This change updates to the current latest version, so
that we can get the benefit of its checks when pushing changes to the
stable branches.
Jonathan Nieder [Sun, 7 Oct 2018 21:55:52 +0000 (21:55 +0000)]
SubmoduleValidator: Permit missing path or url
A .gitmodules file can include a submodule without a path to configure
the URL for a submodule that is only present on other branches.
A .gitmodules file can include a submodule with no URL and no path to
reserve the name for a submodule that existed in earlier history but
is not available from any URL any more.
"git fsck" permits both of these cases. Permit them in JGit as well
(instead of throwing NullPointerException).
Change-Id: I3b442639ad79ea7a59227f96406a12e62d3573ae Reported-by: David Pursehouse <david.pursehouse@gmail.com> Signed-off-by: Jonathan Nieder <jrn@google.com>
The text "<tree, blob>" with angle brackets should not be used in javadoc
since it is interpreted as an HTML tag and then rejected since it's not a
valid HTML tag. Wrap the text in a @literal tag.
Also add a missing space.
Change-Id: Ide045e8c04a39a916f5b2e964e58c151e4555830 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Matthias Sohn [Sat, 6 Oct 2018 00:25:17 +0000 (02:25 +0200)]
Merge branch 'stable-4.10' into stable-4.11
* stable-4.10:
Prepare 4.9.7-SNAPSHOT builds
JGit v4.9.6.201810051924-r
Prepare 4.7.6-SNAPSHOT builds
JGit v4.7.5.201810051826-r
BaseReceivePack: Validate incoming .gitmodules files
ObjectChecker: Report .gitmodules files found in the pack
SubmoduleAddCommand: Reject submodule URIs that look like cli options
Change-Id: Ibd759f5d425f714e79b3137ff8e5b0f989933de0 Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Fri, 5 Oct 2018 23:52:38 +0000 (01:52 +0200)]
Merge branch 'stable-4.9' into stable-4.10
* stable-4.9:
Prepare 4.9.7-SNAPSHOT builds
JGit v4.9.6.201810051924-r
Prepare 4.7.6-SNAPSHOT builds
JGit v4.7.5.201810051826-r
BaseReceivePack: Validate incoming .gitmodules files
ObjectChecker: Report .gitmodules files found in the pack
SubmoduleAddCommand: Reject submodule URIs that look like cli options
Change-Id: Ie59e34eb591a827d1ce8e483eec6d390a3c81702 Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Fri, 5 Oct 2018 23:16:08 +0000 (01:16 +0200)]
Merge branch 'stable-4.8' into stable-4.9
* stable-4.8:
Prepare 4.7.6-SNAPSHOT builds
JGit v4.7.5.201810051826-r
BaseReceivePack: Validate incoming .gitmodules files
ObjectChecker: Report .gitmodules files found in the pack
SubmoduleAddCommand: Reject submodule URIs that look like cli options
Change-Id: Ia7a826399d8d5b8a0eb7169b40e98a6f5c207a4c Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Fri, 5 Oct 2018 23:03:20 +0000 (01:03 +0200)]
Merge branch 'stable-4.7' into stable-4.8
* stable-4.7:
Prepare 4.7.6-SNAPSHOT builds
JGit v4.7.5.201810051826-r
BaseReceivePack: Validate incoming .gitmodules files
ObjectChecker: Report .gitmodules files found in the pack
SubmoduleAddCommand: Reject submodule URIs that look like cli options
Change-Id: Id6fabec4d0b682a7e20a46e88cbc05432efca062 Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
The main concern are submodule urls starting with '-' that could pass as
options to an unguarded tool.
Pass through the parser the ids of blobs identified as .gitmodules
files in the ObjectChecker. Load the blobs and parse/validate them
in SubmoduleValidator.
Change-Id: Ia0cc32ce020d288f995bf7bc68041fda36be1963 Signed-off-by: Ivan Frade <ifrade@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Ivan Frade [Thu, 27 Sep 2018 20:05:13 +0000 (13:05 -0700)]
ObjectChecker: Report .gitmodules files found in the pack
In order to validate .gitmodules files, we first need to find them
in the incoming pack.
Do it in the ObjectChecker stage. Check in the tree objects if they
point to a .gitmodules file and report the tree id and the .gitmodules
blob id.
This can be used later to check if the file is in the root of the
project and if the contents are good.
While we're here, make isMacHFSGit more accurate by detecting variants
of filenames that vary in case.
[jn: tweaked NTFS and HFS+ checking; added more tests]
Change-Id: I70802e7d2c1374116149de4f89836b9498f39582 Signed-off-by: Ivan Frade <ifrade@google.com> Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>