aboutsummaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* Use volatiles for bitmap and revIndex in Packstable-6.10Martin Fick26 hours1-6/+22
| | | | | | | | Previously idx() was memoized with a volatile to improve performance, do the same for the bitmap and reverseIndexes. Change-Id: I50515a3aaae7dd4f9d23989dcd5f79cefc9e0f07 Signed-off-by: Martin Fick <mfick@nvidia.com>
* Fix performance regression in Pack.idx()Martin Fick26 hours1-3/+11
| | | | | | | | | | | | | | | | | | A threading bug fixed with ae53d63837857d886cdf15442118597be717ff33 introduced a performance regression as it used a synchronized idx() for every call. Fix this by only requiring the synchronized when the index has yet to be, or is no longer, memoized. Use a volatile instead of a synchronized in the fast case, and avoid calling Optionally.clear() since we no longer access the field from only synchronized blocks. Instead, replace the volatile value with an Optinonally.Empty avoiding the need to replace an internal Optionally field in an unguarded manner. In some single fetch cases this speeds things up from 48s to 44s. In a 5 concurrent parallel fetch test, this speeds things up from 2min10s to 1min21s! Change-Id: I73a62a8b861b343b112699537e8c947db8159e44 Signed-off-by: Martin Fick <mfick@nvidia.com>
* Use LocalObjectToPack representation moreMartin Fick28 hours1-15/+36
| | | | | | | | | | Use LOTP to also get size in ObjectDirectory. This path is use when searching for deltas, however I cannot find an obvious testable improvement for this. I suspect that this would do best on a fetch with very few reuseable deltas. Change-Id: Ifae154332e3599f77fd9c9bdcf7e1ad3f5055398 Signed-off-by: Martin Fick <mfick@nvidia.com>
* Use representation from LocalObjectToPack if possibleMartin Fick29 hours1-0/+11
| | | | | | | | | | | | | | | | | | | | | When PackWriter selects an ObjectRepresentation, it will select a representation if it doesn't have a better one already selected, even if it clears ReuseAsIs from the LocalObjectToPack (presumably so that it always has a pointer to at least one copy of the object). Previously, the LocalObjectToPack representation was only used if it was marked as ReuseAsIs. Improve this to reuse this representation, if it is still there, even if it cannot be reused as is, since it can at least be used as the representation, when opening an object in ObjectDirectory to send it for the PackWriter as "wholeDeflated". This avoids an unnecessary search of the existing pack files, which can be expensive when there are many pack files in a repo, just to find any (the first in the directory) copy of this object. On a repo with around 580 packfiles, this change will reduce a fetch which normally takes around 44s to around 41s. Change-Id: I16ec71d0cc447166bf4b89e94abecbf0aeee7f44 Signed-off-by: Martin Fick <mfick@nvidia.com>
* Avoid conditional in LocalObjectRepresentation.wasDeltaAttemptedMartin Fick29 hours1-0/+10
| | | | | | | | | | | | | | LocalObjectRepresentation is overriden for both a delta and for a whole object already, which means in these cases, the outcome of the base wasDeltaAttempted logic is fixed as it depends on the output of getFormat() which is hardcoded in these overriden objects. Hardcode this outcome also in the overriden classes to avoid a useless conditional in a hotpath. This change takes about 1s off of a 31s fetch. Change-Id: Ic06c40504875d9868c6072d488e68248538d4f65 Signed-off-by: Martin Fick <mfick@nvidia.com>
* Merge branch 'stable-6.9' into stable-6.10Matthias Sohn9 days3-2/+28
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.9: MergeToolTest: update expected error from the tool Add dash license check for 3rd party dependencies Prepare 5.13.5-SNAPSHOT builds JGit v5.13.4.202507202350-r AmazonS3: Do not accept DOCTYPE and entities ManifestParser: Do not accept DOCTYPE and entities AdvertisedRequestValidator: fix WantNotValidException caused by race Prepare 5.13.4-SNAPSHOT builds Fix packaging build Change-Id: Id42981e23f88ffcfb565a9d3418e6a8899bad0bc
| * Merge branch 'stable-6.8' into stable-6.9stable-6.9Matthias Sohn9 days17-1388/+303
| |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.8: MergeToolTest: update expected error from the tool Add dash license check for 3rd party dependencies Prepare 5.13.5-SNAPSHOT builds JGit v5.13.4.202507202350-r AmazonS3: Do not accept DOCTYPE and entities ManifestParser: Do not accept DOCTYPE and entities AdvertisedRequestValidator: fix WantNotValidException caused by race Prepare 5.13.4-SNAPSHOT builds Fix packaging build Change-Id: I52b27f68a05dc07f8a181b680a62409fbb31f25e
| | * Merge branch 'stable-6.7' into stable-6.8stable-6.8Matthias Sohn10 days17-1358/+297
| | |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.7: MergeToolTest: update expected error from the tool Add dash license check for 3rd party dependencies Prepare 5.13.5-SNAPSHOT builds JGit v5.13.4.202507202350-r AmazonS3: Do not accept DOCTYPE and entities ManifestParser: Do not accept DOCTYPE and entities AdvertisedRequestValidator: fix WantNotValidException caused by race Prepare 5.13.4-SNAPSHOT builds Fix packaging build Change-Id: I38958568f0eee70394d03292d5dab881d585cef4
| | | * Merge branch 'stable-6.6' into stable-6.7stable-6.7Matthias Sohn10 days17-1358/+297
| | | |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.6: MergeToolTest: update expected error from the tool Add dash license check for 3rd party dependencies Prepare 5.13.5-SNAPSHOT builds JGit v5.13.4.202507202350-r AmazonS3: Do not accept DOCTYPE and entities ManifestParser: Do not accept DOCTYPE and entities AdvertisedRequestValidator: fix WantNotValidException caused by race Prepare 5.13.4-SNAPSHOT builds Fix packaging build Change-Id: I7fd139f513cc2c0b19814ac15881e80ce0438782
| | | | * Merge branch 'stable-6.5' into stable-6.6stable-6.6Matthias Sohn10 days17-1314/+289
| | | | |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.5: MergeToolTest: update expected error from the tool Add dash license check for 3rd party dependencies Prepare 5.13.5-SNAPSHOT builds JGit v5.13.4.202507202350-r AmazonS3: Do not accept DOCTYPE and entities ManifestParser: Do not accept DOCTYPE and entities AdvertisedRequestValidator: fix WantNotValidException caused by race Prepare 5.13.4-SNAPSHOT builds Fix packaging build Change-Id: I12e9aa0e8a597e5976b555146265aceabcb18335
| | | | | * Merge branch 'stable-6.4' into stable-6.5stable-6.5Matthias Sohn10 days18-445/+91
| | | | | |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Remove jetty source bundles from p2 repo since they aren't available for jetty 10.0.13. * stable-6.4: MergeToolTest: update expected error from the tool Add dash license check for 3rd party dependencies Prepare 5.13.5-SNAPSHOT builds JGit v5.13.4.202507202350-r AmazonS3: Do not accept DOCTYPE and entities ManifestParser: Do not accept DOCTYPE and entities AdvertisedRequestValidator: fix WantNotValidException caused by race Prepare 5.13.4-SNAPSHOT builds Fix packaging build Change-Id: I4e9262d883d31bbc9392b2140f9071a3093d56af
| | | | | | * Merge branch 'stable-6.3' into stable-6.4stable-6.4Matthias Sohn10 days17-442/+91
| | | | | | |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.3: MergeToolTest: update expected error from the tool Add dash license check for 3rd party dependencies Prepare 5.13.5-SNAPSHOT builds JGit v5.13.4.202507202350-r AmazonS3: Do not accept DOCTYPE and entities ManifestParser: Do not accept DOCTYPE and entities AdvertisedRequestValidator: fix WantNotValidException caused by race Prepare 5.13.4-SNAPSHOT builds Fix packaging build Change-Id: I448bad5c8333607ef5f39ceda70f004ee8a9ba31
| | | | | | | * Merge branch 'stable-6.2' into stable-6.3stable-6.3Matthias Sohn10 days17-443/+104
| | | | | | | |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.2: MergeToolTest: update expected error from the tool Add dash license check for 3rd party dependencies Prepare 5.13.5-SNAPSHOT builds JGit v5.13.4.202507202350-r AmazonS3: Do not accept DOCTYPE and entities ManifestParser: Do not accept DOCTYPE and entities AdvertisedRequestValidator: fix WantNotValidException caused by race Prepare 5.13.4-SNAPSHOT builds Fix packaging build Change-Id: I23df8552b799ed5bd4c8472d3775372d78a573ea
| | | | | | | | * Merge branch 'stable-6.1' into stable-6.2stable-6.2Matthias Sohn10 days16-442/+103
| | | | | | | | |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.1: Add dash license check for 3rd party dependencies Prepare 5.13.5-SNAPSHOT builds JGit v5.13.4.202507202350-r AmazonS3: Do not accept DOCTYPE and entities ManifestParser: Do not accept DOCTYPE and entities AdvertisedRequestValidator: fix WantNotValidException caused by race Prepare 5.13.4-SNAPSHOT builds Fix packaging build Change-Id: I7a0c22f2288c2201af04d60f9ed59730341a31d5
| | | | | | | | | * Merge branch 'stable-6.0' into stable-6.1stable-6.1Matthias Sohn10 days16-442/+103
| | | | | | | | | |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.0: Add dash license check for 3rd party dependencies Prepare 5.13.5-SNAPSHOT builds JGit v5.13.4.202507202350-r AmazonS3: Do not accept DOCTYPE and entities ManifestParser: Do not accept DOCTYPE and entities AdvertisedRequestValidator: fix WantNotValidException caused by race Prepare 5.13.4-SNAPSHOT builds Fix packaging build Change-Id: Ie3343cb83babb30ffc802967f491182c8c84d61d
| | | | | | | | | | * Add dash license check for 3rd party dependenciesstable-6.0Matthias Sohn10 days1-2/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Change-Id: I552bd4e5a43871d4772d2d4fba7232f5dbb101ec
| | | | | | | | | | * Merge branch 'stable-5.13' into stable-6.0Matthias Sohn10 days16-433/+90
| | | | | | | | | | |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.13: Prepare 5.13.5-SNAPSHOT builds JGit v5.13.4.202507202350-r AmazonS3: Do not accept DOCTYPE and entities ManifestParser: Do not accept DOCTYPE and entities AdvertisedRequestValidator: fix WantNotValidException caused by race Prepare 5.13.4-SNAPSHOT builds Fix packaging build Change-Id: Ib29309f853bcd3825058128d62c90f2739f30590
| | | | | | | | | | | * Prepare 5.13.5-SNAPSHOT buildsstable-5.13Matthias Sohn11 days88-503/+503
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Change-Id: Iafe52ac049219f3da3888811135e5729d053aa1a
| | | | | | | | | | | * JGit v5.13.4.202507202350-rv5.13.4.202507202350-rMatthias Sohn11 days88-121/+121
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Change-Id: Ib49e2cf19633d986d14ef059f92499fdb5c98086
| | | | | | | | | | | * AmazonS3: Do not accept DOCTYPE and entitiesMatthias Sohn11 days1-3/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This follows OWASP recommendations in https://cheatsheetseries.owasp.org/cheatsheets/XML_External_Entity_Prevention_Cheat_Sheet.html Change-Id: I3d47debf14d95c8189d51256b4eb2ba991279452
| | | | | | | | | | | * ManifestParser: Do not accept DOCTYPE and entitiesIvan Frade11 days2-3/+48
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | These open the door for XXE attacks [1] and manifest do not need them. [1] https://en.wikipedia.org/wiki/XML_external_entity_attack Change-Id: Ia79971e1c34afaf287584ae4a7f71baebcb48b6a
| | | | | | | | | | | * AdvertisedRequestValidator: fix WantNotValidException caused by raceMatthias Sohn11 days1-3/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fetch with protocol V2 failed under the following conditions - fetch uses bidirectional protocol (git, ssh) which uses a shortcut to determine invalid wants - not all wants are advertised - race condition: wanted ref is updated during fetch by another thread after the thread serving upload-pack determined wants and before it checks not advertised wants Fix this by calling `new ReachableCommitRequestValidator().checkWants(up, wants)` instead of throwing WantNotValidException in [1] if this race happened in the same way like it's done for unidirectional protocols (http) [2]. [1] https://github.com/eclipse-jgit/jgit/blob/stable-6.10/org.eclipse.jgit/src/org/eclipse/jgit/transport/UploadPack.java#L2002 [2] https://github.com/eclipse-jgit/jgit/blob/stable-6.10/org.eclipse.jgit/src/org/eclipse/jgit/transport/UploadPack.java#L2000 Bug: jgit-48 Change-Id: I32f28502923815dc49781aab5d810c9afbe7e7e6
| | | | | | | | | | | * Prepare 5.13.4-SNAPSHOT buildsMatthias Sohn11 days88-503/+503
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Change-Id: I0fe9792b7ac883f355d2dbfa5cac97b7b26648ae
| | | | | | | | | | | * Fix packaging buildMatthias Sohn11 days34-1678/+31
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - use target platform 4.21 - remove all older target platforms - update tycho to 4.0.13, hence maven needs to be run using Java 17 - enforce build is run using Java 17 or higher - remove pack200 plugins Change-Id: I50a52c22f4665b6323ee6d29da18e39960d008af
| | | | | | | | * | | | MergeToolTest: update expected error from the toolIvan Frade10 days1-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This test fails in jenkins becuase the error string is slightly different. It seems to come from an external tool (git mergetool?) that maybe changed that output (?). At the moment update the error string to match the output of the tool. Parsing error messages is bound to be flaky. Change-Id: I81e9bf65088b893af3a0d8e53d57052583fc2262
* | | | | | | | | | | | Fix: Close the "preserved" PackDirectoryNasser Grainawi2025-06-191-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This has been missing since the feature was first added in commit 6167641834e28f8ad322f8fde60866b339bfb7fe. It's possible we could be more aggressive and close soon after attempting to get an object from the preserved packs, but for concurrent misses that might cause thrashing. More likely it would be safe to attempt closing after successfully restoring a preserved pack. A follow up change should attempt that. Change-Id: I87d61007bcc3d03fc86bd18465ca66a2e6f697a1
* | | | | | | | | | | | Use the same ordering/locking in delete() as C gitDaniele Sassoli2025-05-291-26/+32
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Following the examples of cgit, lock packed-refs *before* checking for existance of refs in it [1] and *keep the lock* until the loose ref (if any) is removed [2]. The packed-refs lock is kept even when no packed-refs update is required [3] so that somebody else doesn't pack a reference that we are trying to delete. This fixes a concurrency issue that happens on projects with a substantial amount of refs(>~500k) where packing takes long enough for a ref deletion to be triggered half way through it. Not locking the packed-refs file before checking if the refs exists is not safe, as it opens up situations where loose refs are repacked in memory and locked on disk, but before the lock is released and packed-refs is flushed to disk, a ref is deleted. As packed-refs was NOT locked while checking wether a ref existed in it, the current content on disk was read, which was about to be overwritten and did not contain the ref about to be deleted. As the delete doesn't see the ref in the current, on-disk, version of packed refs, it skips processing altogether and moves on, correctly, deleting only the associated loose ref and leaving the packed one behind. Once the new packed-refs, containing the ref that was just deleted, was commited to disk, the ref would come back to life. Therefore, the packed-refs needs to be locked before checking if it contains a ref or not in the same way the C implementation of Git does at [1]. There are tradeoffs, though, in this decision, which will reduce the parallelism of deleting loose refs and performing the refs repacking, which happens very often in certain JGit implementations like Gerrit Code Review. Before this change, repacking of refs and removal of loose refs unrelated to the in-flight repacking was possible without involving any locking; after this change, all loose refs removals have to wait for the packing of refs to be completed, even though the repacking and the refs removals were completely unrelated and their namespaces disjoint. See more details on the test's performance results and the associated tradeoffs in the Issue jgit-152. NOTE: This delete ref locking logic was incorrect regardless of how the packing of the refs is implemented. Making decisions if the pack transaction is needed or not on an unlocked resource is racy and also flagged as bug at [1]. [1]https://github.com/git/git/blob/master/refs/packed-backend.c#L1590 [2]https://github.com/git/git/blob/master/refs/files-backend.c#L3261 [3]https://github.com/git/git/blob/master/refs/files-backend.c#L2943 Bug: jgit-152 Change-Id: I158ec837904617c5fdf667e295ae667b2f037945
* | | | | | | | | | | | Prepare 6.10.2-SNAPSHOT buildsMatthias Sohn2025-05-2289-537/+537
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Change-Id: Id893557b6929718f8528ce50b0f7bdb23057aa15
* | | | | | | | | | | | JGit v6.10.1.202505221210-rv6.10.1.202505221210-rMatthias Sohn2025-05-2289-123/+123
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Change-Id: I60dd6d671f605c7cfaed747cc0fc1772115e5f37
* | | | | | | | | | | | AmazonS3: Do not accept DOCTYPE and entitiesMatthias Sohn2025-05-221-0/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This follows OWASP recommendations in https://cheatsheetseries.owasp.org/cheatsheets/XML_External_Entity_Prevention_Cheat_Sheet.html Change-Id: I3d47debf14d95c8189d51256b4eb2ba991279452
* | | | | | | | | | | | ManifestParser: Do not accept DOCTYPE and entitiesIvan Frade2025-05-222-1/+44
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | These open the door for XXE attacks [1] and manifest do not need them. [1] https://en.wikipedia.org/wiki/XML_external_entity_attack Change-Id: Ia79971e1c34afaf287584ae4a7f71baebcb48b6a
* | | | | | | | | | | | Do not load bitmap indexes during directory scansMartin Fick2025-02-254-21/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Previously, if a bitmap index had not been loaded yet, it would get loaded during a directory scan. Loading a bitmap file can be expensive and there is no immediate need to do so during a scan. Fix this by simply setting bitmap index file names on the Packs during directory scans so that bitmaps can be lazily loaded at some later point if they are needed. This change has the side affect of no longer marking a Pack valid if it is currently invalid simply because a bitmap file has been found, as there is no valid reason to do so and this can incorrectly mark a Pack without an index, or with other issues valid. Since the initial lack of a bitmap file, or an invalid one, or the deletion of one, would not result in the Pack being marked invalid, there is no need to overturn the invalid flag when a new bitmap file is found. Change-Id: I056acc09e7ae6a0982acd81b552d524190ebb4be Signed-off-by: Martin Fick <mfick@nvidia.com>
* | | | | | | | | | | | Merge "Pack: no longer set invalid in openFail()" into stable-6.10Matthias Sohn2025-02-241-14/+16
|\ \ \ \ \ \ \ \ \ \ \ \
| * | | | | | | | | | | | Pack: no longer set invalid in openFail()Martin Fick2025-02-141-14/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The intention of the 'invalidate' argument in openFail() is to invalidate the Pack in certain situations. However, after moving doOpen() to a lock instead of using synchronized, the invalidation approach could also incorrectly mark an already invalid Pack valid, which was never the intention since previously invalid would only ever get set to false if it already was false. Fix this by never setting invalid in openFail(), instead set invalid explicitly before calling openFail when needed. This makes the intent clearer, and aligns better with all the existing comments already trying to explain the boolean (and some of them become obvious enough now that the comment is deleted or shortened). This is also likely faster than adding a conditional in openFail() to make 'invalidate' work properly. Change-Id: Ie6182103ee2994724cb5cb0b64030fedba84b637 Signed-off-by: Martin Fick <mfick@nvidia.com>
* | | | | | | | | | | | | Fix calculation of pack files and objects since bitmapAntonio Barone2025-02-242-8/+33
|/ / / / / / / / / / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fix a logic issue where pack files and objects created since the most recent bitmap were incorrectly counted, ignoring their modification time. Since pack files are processed in order from most recent to oldest, we can reliably stop counting as soon as we encounter the first bitmap. By definition, all subsequent pack files are older and should not be included in the count. This ensures accurate repository statistics and prevents overcounting. Bug: jgit-140 Change-Id: I99d85fb70bc7eb42a8d24c74a1fdb8e03334099e
* | | | | | | | | | | | RevWalk: Add an isMergedIntoAnyCommit() methodMartin Fick2025-02-013-9/+58
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | RevWalk had a bulk isMergedIntoAny() method, however it worked only for Refs. Add a similar method which works for RevCommits instead. Unfortunately isMergedIntoAny() cannot be overloaded since java does not include the generic types in Collections of method signatures, so the method name needs to be more complicated to differentiate it from the existing method. Change-Id: I4f8f3a83058a186fafe3b37726e21c5074a6b8e1 Signed-off-by: Martin Fick <mfick@nvidia.com>
* | | | | | | | | | | | Pack: separate an open/close accounting lockMartin Fick2024-12-201-21/+35
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Previously the open/close accounting code used whole Pack object synchronization for locking. Unfortunately, there are other unrelated methods which use whole Pack object synchronization also, mostly to avoid concurrent loading of these independent indices, and they do not touch or need to coordinate with the open/close accounting data. During heavy load when a new file appears after repacking the readFully() threads could uselessly block on threads reading the reverse index. These threads could have been reading from the Pack file instead of waiting for the reverse index to be read. Use a new lock to make this locking more fine grained to prevent the readFully() calling threads from getting blocked in beginWindowCache() while the reverse index or bitmaps are being loaded. Change-Id: I7ac9067ca10cd6d6be0ab25148d99da3ace7ba36 Signed-off-by: Martin Fick <mfick@nvidia.com>
* | | | | | | | | | | | http.server/BUILD: expose servlet resources as targetIvan Frade2024-12-201-1/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | JGit has moved to jakarta servlet-api, but gerrit is still in the javax version. This is blocking gerrit to update their jgit code. Gerrit can workaround this rebuilding the jar of jgit-servlet but it is complicated to pass-through the resources. Expose the resources of the servlet as a target, to help gerrit build a javax version of the jgit-servlet. Change-Id: Ifd794e4222ad442f4a538a4d38caa73d58be0f6d (cherry picked from commit 31db36bbea6d081c8714a4c7ed3f5d81112ae8b8)
* | | | | | | | | | | | WindowCache: share removal work among multiple threadsMartin Fick2024-12-191-5/+126
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Split the removal process into blocks so that it can be shared by multiple threads. This potential work sharing can provide 2 optimizations for removals: 1) It provides an opportunity for separate removal requests to be consolidated into one removal pass. 2) It can reduce removing thread latencies by sharing the removal work with other removing threads which otherwise might not have any work to do due to their removal request being consolidated. This makes the system more efficient and can actually reduce latencies as system load increases due to pack removals! The optimizations above are all achieved without blockng threads to wait for other threads to complete (although naturally there are some synchronization points), and while ensuring that no threads do more work than if they were the only thread available to perform a removal. Change-Id: Ic6809a8abf056299abde0f0c58c77aaf245a8df5 Signed-off-by: Martin Fick <mfick@nvidia.com>
* | | | | | | | | | | | Update target platform version in maven build to 4.32 (2024-06)Matthias Sohn2024-12-1931-4381/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The p2 repositories matching jgit-4.17 target platform were moved to the archive server and are no longer available on the download server. Update to the jgit-4.32 target platform which matches Eclipse platform 2024-06. Purge older target platforms to reduce our maintenance effort. Change-Id: I948400e5b8c0db2ab0d50f8fa4e0e42521302ce7
* | | | | | | | | | | | FileSnapshot: fix warningsMatthias Sohn2024-12-151-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - comment empty code block - suppress non-translatable text warning Change-Id: Id49b4a56bbe5454edfe1ea8b79ceeaf51ceac370
* | | | | | | | | | | | Optionally.Hard: avoid Optional creation on every use,Martin Fick2024-12-131-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Previously the getOptional() call created an new Optional for its element, and this appears to be somewhat expensive. During a time when a server is heavily loaded because of a poorly maintained repository with potentially 2K+ pack files, calls to Optionally.ofNullable() from within the getOptional() method appeared extensively in the Stacktrace. Reduce the getOptional() call overhead by storing an already created Optional of the element instead of the element itself. This trades the extra space of one extra reference for a potential speed gain in a hotspot. Since the current users of Optionally.Hard reference objects significantly larger than a single reference (and most users are likely to be, else why would they be using an Optionally?), this is likely a good tradeoff. Change-Id: I7adccc915480cbb97a43dcbe074cfb620888c936 Signed-off-by: Martin Fick <mfick@nvidia.com>
* | | | | | | | | | | | Pack: fix threading bug getting idxMartin Fick2024-12-131-46/+40
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When converting to Optionally, a threading bug was introduced, fix it. The Optionally class itself is not thread safe, and previously it was being called from idx() without any thread synchronization mechanism. This was because previously the variable which held the cached index was volatile. The Optionally<PackIndex> kept the volatile attribute, but that only synchronizes the reference to the Optionally, and not the element inside the Optionally. A clear() from another thread could thus be missed by the idx() call, potentially allowing the idx() call to interact poorly with the Optionally, potentially even causing a memory leak. To fix this, extend the synchronized inside the idx() method to the entire method. Then, additionally remove the now redundant Optional fetch in idx() and the no longer needed volatile. Change-Id: I6077e1aaed96c53200805b4c87a67afb49c2b373 Signed-off-by: Martin Fick <mfick@nvidia.com>
* | | | | | | | | | | | Fix potential NPE in TreeWalk#getFilterCommandDefinitionNasser Grainawi2024-12-101-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Change-Id: Ic6f0fd26153b47b4aeeec105bac431225d9bf8bf
* | | | | | | | | | | | Advertise "agent" capability when using protocol v2Antonio Barone2024-12-102-6/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The "agent" capability was not advertised by the server when executing git-upload-pack over protocol v2. This, in turn, prevented the clients from advertising its client agent too, as documented [1]: "The client may optionally send its own agent string by including the agent capability with a value Y (in the form agent=Y) in its request to the server (but it MUST NOT do so if the server did not advertise the agent capability)." When using jgit with Gerrit this had the effect of preventing the logging of the git client agent in the sshd_log. [1] https://git-scm.com/docs/protocol-v2#_agent Bug: jgit-118 Change-Id: Ifb6ea65fde020425920338f7dd9cc683fed6a4a4
* | | | | | | | | | | | FileSnapshot: silence "Stale file handle" exceptionsMartin Fick2024-12-051-3/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Sometimes a FileSystemException with "Stale file handle" can be thrown while creating a FileSnapshot, likely because the file or directory was deleted. Since NoSuchFileExceptions are already silenced, and the FileSnapshot already handles all IOExceptions, there is likely no value in seeing this info in the logs, treat these situation the same and silence them also. Change-Id: I922f4edf2d332cd704e60276f41a76df242f281c Signed-off-by: Martin Fick <mfick@nvidia.com>
* | | | | | | | | | | | Merge changes I022639c4,I1fcce5ef into stable-6.10Matthias Sohn2024-12-051-15/+20
|\ \ \ \ \ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * changes: FileSnapshot: silence "Not a Directory" exceptions FileSnapshot: refactor to share error handling
| * | | | | | | | | | | | FileSnapshot: silence "Not a Directory" exceptionsMartin Fick2024-12-051-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Sometimes a FileSystemException with "Not a Directory" can be thrown while creating a FileSnapshot, likely because the file or directory was deleted. Since NoSuchFileExceptions are already silenced, and the FileSnapshot already handles all IOExceptions, there is likely no value in seeing this info in the logs, treat these situation the same and silence them also. Change-Id: I022639c42154a0a4c71065741f023f5eb95f9b55 Signed-off-by: Martin Fick <mfick@nvidia.com>
| * | | | | | | | | | | | FileSnapshot: refactor to share error handlingMartin Fick2024-12-051-15/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It is important to keep the exception handling for getting file attributes the same in all places in this class; put that code into a common method. Change-Id: I1fcce5efd10aa562a4e0e34d3ce94bcc83850237 Signed-off-by: Martin Fick <mfick@nvidia.com>
* | | | | | | | | | | | | Merge changes I172c43ff,I2a4c5743,Icc4b8d2f into stable-6.10Matthias Sohn2024-12-053-10/+20
|\ \ \ \ \ \ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * changes: Mark Attribute#getValue as @Nullable Fix potential NPE in ResolveMerger#getAttributesContentMergeStrategy Fix NPE in DiffFormatter#getDiffDriver
generated by cgit v1.2.3 (git 2.39.1) at 2025-07-31 12:57:22 +0000