aboutsummaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* Merge branch 'stable-7.1' into stable-7.2stable-7.2Matthias Sohn29 hours3-13/+79
|\ | | | | | | | | | | | | | | | | | | | | * stable-7.1: Use volatiles for bitmap and revIndex in Pack Fix performance regression in Pack.idx() Use LocalObjectToPack representation more Use representation from LocalObjectToPack if possible Avoid conditional in LocalObjectRepresentation.wasDeltaAttempted Change-Id: I38dabec5d9c56201bc895fe5019e7447520cb260
| * Merge branch 'stable-7.0' into stable-7.1stable-7.1Matthias Sohn29 hours3-13/+79
| |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-7.0: Use volatiles for bitmap and revIndex in Pack Fix performance regression in Pack.idx() Use LocalObjectToPack representation more Use representation from LocalObjectToPack if possible Avoid conditional in LocalObjectRepresentation.wasDeltaAttempted Change-Id: I953fec9bac3843845fe641af77ab9b757b2638ad
| | * Merge branch 'stable-6.10' into stable-7.0stable-7.0Matthias Sohn29 hours3-13/+79
| | |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.10: Use volatiles for bitmap and revIndex in Pack Fix performance regression in Pack.idx() Use LocalObjectToPack representation more Use representation from LocalObjectToPack if possible Avoid conditional in LocalObjectRepresentation.wasDeltaAttempted Change-Id: If7e45204cc033506745ee394cc6119c1ad8dfb3b
| | | * Use volatiles for bitmap and revIndex in Packstable-6.10Martin Fick30 hours1-6/+22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Previously idx() was memoized with a volatile to improve performance, do the same for the bitmap and reverseIndexes. Change-Id: I50515a3aaae7dd4f9d23989dcd5f79cefc9e0f07 Signed-off-by: Martin Fick <mfick@nvidia.com>
| | | * Fix performance regression in Pack.idx()Martin Fick30 hours1-3/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A threading bug fixed with ae53d63837857d886cdf15442118597be717ff33 introduced a performance regression as it used a synchronized idx() for every call. Fix this by only requiring the synchronized when the index has yet to be, or is no longer, memoized. Use a volatile instead of a synchronized in the fast case, and avoid calling Optionally.clear() since we no longer access the field from only synchronized blocks. Instead, replace the volatile value with an Optinonally.Empty avoiding the need to replace an internal Optionally field in an unguarded manner. In some single fetch cases this speeds things up from 48s to 44s. In a 5 concurrent parallel fetch test, this speeds things up from 2min10s to 1min21s! Change-Id: I73a62a8b861b343b112699537e8c947db8159e44 Signed-off-by: Martin Fick <mfick@nvidia.com>
| | | * Use LocalObjectToPack representation moreMartin Fick32 hours1-15/+36
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Use LOTP to also get size in ObjectDirectory. This path is use when searching for deltas, however I cannot find an obvious testable improvement for this. I suspect that this would do best on a fetch with very few reuseable deltas. Change-Id: Ifae154332e3599f77fd9c9bdcf7e1ad3f5055398 Signed-off-by: Martin Fick <mfick@nvidia.com>
| | | * Use representation from LocalObjectToPack if possibleMartin Fick33 hours1-0/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When PackWriter selects an ObjectRepresentation, it will select a representation if it doesn't have a better one already selected, even if it clears ReuseAsIs from the LocalObjectToPack (presumably so that it always has a pointer to at least one copy of the object). Previously, the LocalObjectToPack representation was only used if it was marked as ReuseAsIs. Improve this to reuse this representation, if it is still there, even if it cannot be reused as is, since it can at least be used as the representation, when opening an object in ObjectDirectory to send it for the PackWriter as "wholeDeflated". This avoids an unnecessary search of the existing pack files, which can be expensive when there are many pack files in a repo, just to find any (the first in the directory) copy of this object. On a repo with around 580 packfiles, this change will reduce a fetch which normally takes around 44s to around 41s. Change-Id: I16ec71d0cc447166bf4b89e94abecbf0aeee7f44 Signed-off-by: Martin Fick <mfick@nvidia.com>
| | | * Avoid conditional in LocalObjectRepresentation.wasDeltaAttemptedMartin Fick34 hours1-0/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | LocalObjectRepresentation is overriden for both a delta and for a whole object already, which means in these cases, the outcome of the base wasDeltaAttempted logic is fixed as it depends on the output of getFormat() which is hardcoded in these overriden objects. Hardcode this outcome also in the overriden classes to avoid a useless conditional in a hotpath. This change takes about 1s off of a 31s fetch. Change-Id: Ic06c40504875d9868c6072d488e68248538d4f65 Signed-off-by: Martin Fick <mfick@nvidia.com>
* | | | Merge branch 'stable-7.1' into stable-7.2Matthias Sohn9 days2-1/+27
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-7.1: MergeToolTest: update expected error from the tool Add dash license check for 3rd party dependencies Prepare 5.13.5-SNAPSHOT builds JGit v5.13.4.202507202350-r AmazonS3: Do not accept DOCTYPE and entities ManifestParser: Do not accept DOCTYPE and entities AdvertisedRequestValidator: fix WantNotValidException caused by race Prepare 5.13.4-SNAPSHOT builds Fix packaging build Change-Id: Idfbe64561451814de41cfce21ba14281bd44d283
| * | | Merge branch 'stable-7.0' into stable-7.1Matthias Sohn9 days3-2/+28
| |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-7.0: MergeToolTest: update expected error from the tool Add dash license check for 3rd party dependencies Prepare 5.13.5-SNAPSHOT builds JGit v5.13.4.202507202350-r AmazonS3: Do not accept DOCTYPE and entities ManifestParser: Do not accept DOCTYPE and entities AdvertisedRequestValidator: fix WantNotValidException caused by race Prepare 5.13.4-SNAPSHOT builds Fix packaging build Change-Id: I73d5233916d021096381b4c8116312134761dfad
| | * | Merge branch 'stable-6.10' into stable-7.0Matthias Sohn9 days3-2/+28
| | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.10: MergeToolTest: update expected error from the tool Add dash license check for 3rd party dependencies Prepare 5.13.5-SNAPSHOT builds JGit v5.13.4.202507202350-r AmazonS3: Do not accept DOCTYPE and entities ManifestParser: Do not accept DOCTYPE and entities AdvertisedRequestValidator: fix WantNotValidException caused by race Prepare 5.13.4-SNAPSHOT builds Fix packaging build Change-Id: I81ef0aef1dd65ec614385f12953122da2ee9d459
| | | * Merge branch 'stable-6.9' into stable-6.10Matthias Sohn9 days3-2/+28
| | | |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.9: MergeToolTest: update expected error from the tool Add dash license check for 3rd party dependencies Prepare 5.13.5-SNAPSHOT builds JGit v5.13.4.202507202350-r AmazonS3: Do not accept DOCTYPE and entities ManifestParser: Do not accept DOCTYPE and entities AdvertisedRequestValidator: fix WantNotValidException caused by race Prepare 5.13.4-SNAPSHOT builds Fix packaging build Change-Id: Id42981e23f88ffcfb565a9d3418e6a8899bad0bc
| | | | * Merge branch 'stable-6.8' into stable-6.9stable-6.9Matthias Sohn9 days17-1388/+303
| | | | |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.8: MergeToolTest: update expected error from the tool Add dash license check for 3rd party dependencies Prepare 5.13.5-SNAPSHOT builds JGit v5.13.4.202507202350-r AmazonS3: Do not accept DOCTYPE and entities ManifestParser: Do not accept DOCTYPE and entities AdvertisedRequestValidator: fix WantNotValidException caused by race Prepare 5.13.4-SNAPSHOT builds Fix packaging build Change-Id: I52b27f68a05dc07f8a181b680a62409fbb31f25e
| | | | | * Merge branch 'stable-6.7' into stable-6.8stable-6.8Matthias Sohn10 days17-1358/+297
| | | | | |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.7: MergeToolTest: update expected error from the tool Add dash license check for 3rd party dependencies Prepare 5.13.5-SNAPSHOT builds JGit v5.13.4.202507202350-r AmazonS3: Do not accept DOCTYPE and entities ManifestParser: Do not accept DOCTYPE and entities AdvertisedRequestValidator: fix WantNotValidException caused by race Prepare 5.13.4-SNAPSHOT builds Fix packaging build Change-Id: I38958568f0eee70394d03292d5dab881d585cef4
| | | | | | * Merge branch 'stable-6.6' into stable-6.7stable-6.7Matthias Sohn10 days17-1358/+297
| | | | | | |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.6: MergeToolTest: update expected error from the tool Add dash license check for 3rd party dependencies Prepare 5.13.5-SNAPSHOT builds JGit v5.13.4.202507202350-r AmazonS3: Do not accept DOCTYPE and entities ManifestParser: Do not accept DOCTYPE and entities AdvertisedRequestValidator: fix WantNotValidException caused by race Prepare 5.13.4-SNAPSHOT builds Fix packaging build Change-Id: I7fd139f513cc2c0b19814ac15881e80ce0438782
| | | | | | | * Merge branch 'stable-6.5' into stable-6.6stable-6.6Matthias Sohn10 days17-1314/+289
| | | | | | | |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.5: MergeToolTest: update expected error from the tool Add dash license check for 3rd party dependencies Prepare 5.13.5-SNAPSHOT builds JGit v5.13.4.202507202350-r AmazonS3: Do not accept DOCTYPE and entities ManifestParser: Do not accept DOCTYPE and entities AdvertisedRequestValidator: fix WantNotValidException caused by race Prepare 5.13.4-SNAPSHOT builds Fix packaging build Change-Id: I12e9aa0e8a597e5976b555146265aceabcb18335
| | | | | | | | * Merge branch 'stable-6.4' into stable-6.5stable-6.5Matthias Sohn10 days18-445/+91
| | | | | | | | |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Remove jetty source bundles from p2 repo since they aren't available for jetty 10.0.13. * stable-6.4: MergeToolTest: update expected error from the tool Add dash license check for 3rd party dependencies Prepare 5.13.5-SNAPSHOT builds JGit v5.13.4.202507202350-r AmazonS3: Do not accept DOCTYPE and entities ManifestParser: Do not accept DOCTYPE and entities AdvertisedRequestValidator: fix WantNotValidException caused by race Prepare 5.13.4-SNAPSHOT builds Fix packaging build Change-Id: I4e9262d883d31bbc9392b2140f9071a3093d56af
| | | | | | | | | * Merge branch 'stable-6.3' into stable-6.4stable-6.4Matthias Sohn10 days17-442/+91
| | | | | | | | | |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.3: MergeToolTest: update expected error from the tool Add dash license check for 3rd party dependencies Prepare 5.13.5-SNAPSHOT builds JGit v5.13.4.202507202350-r AmazonS3: Do not accept DOCTYPE and entities ManifestParser: Do not accept DOCTYPE and entities AdvertisedRequestValidator: fix WantNotValidException caused by race Prepare 5.13.4-SNAPSHOT builds Fix packaging build Change-Id: I448bad5c8333607ef5f39ceda70f004ee8a9ba31
| | | | | | | | | | * Merge branch 'stable-6.2' into stable-6.3stable-6.3Matthias Sohn10 days17-443/+104
| | | | | | | | | | |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.2: MergeToolTest: update expected error from the tool Add dash license check for 3rd party dependencies Prepare 5.13.5-SNAPSHOT builds JGit v5.13.4.202507202350-r AmazonS3: Do not accept DOCTYPE and entities ManifestParser: Do not accept DOCTYPE and entities AdvertisedRequestValidator: fix WantNotValidException caused by race Prepare 5.13.4-SNAPSHOT builds Fix packaging build Change-Id: I23df8552b799ed5bd4c8472d3775372d78a573ea
| | | | | | | | | | | * Merge branch 'stable-6.1' into stable-6.2stable-6.2Matthias Sohn10 days16-442/+103
| | | | | | | | | | | |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.1: Add dash license check for 3rd party dependencies Prepare 5.13.5-SNAPSHOT builds JGit v5.13.4.202507202350-r AmazonS3: Do not accept DOCTYPE and entities ManifestParser: Do not accept DOCTYPE and entities AdvertisedRequestValidator: fix WantNotValidException caused by race Prepare 5.13.4-SNAPSHOT builds Fix packaging build Change-Id: I7a0c22f2288c2201af04d60f9ed59730341a31d5
| | | | | | | | | | | | * Merge branch 'stable-6.0' into stable-6.1stable-6.1Matthias Sohn10 days16-442/+103
| | | | | | | | | | | | |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.0: Add dash license check for 3rd party dependencies Prepare 5.13.5-SNAPSHOT builds JGit v5.13.4.202507202350-r AmazonS3: Do not accept DOCTYPE and entities ManifestParser: Do not accept DOCTYPE and entities AdvertisedRequestValidator: fix WantNotValidException caused by race Prepare 5.13.4-SNAPSHOT builds Fix packaging build Change-Id: Ie3343cb83babb30ffc802967f491182c8c84d61d
| | | | | | | | | | | | | * Add dash license check for 3rd party dependenciesstable-6.0Matthias Sohn10 days1-2/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Change-Id: I552bd4e5a43871d4772d2d4fba7232f5dbb101ec
| | | | | | | | | | | | | * Merge branch 'stable-5.13' into stable-6.0Matthias Sohn10 days16-433/+90
| | | | | | | | | | | | | |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.13: Prepare 5.13.5-SNAPSHOT builds JGit v5.13.4.202507202350-r AmazonS3: Do not accept DOCTYPE and entities ManifestParser: Do not accept DOCTYPE and entities AdvertisedRequestValidator: fix WantNotValidException caused by race Prepare 5.13.4-SNAPSHOT builds Fix packaging build Change-Id: Ib29309f853bcd3825058128d62c90f2739f30590
| | | | | | | | | | | | | | * Prepare 5.13.5-SNAPSHOT buildsstable-5.13Matthias Sohn11 days88-503/+503
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Change-Id: Iafe52ac049219f3da3888811135e5729d053aa1a
| | | | | | | | | | | | | | * JGit v5.13.4.202507202350-rv5.13.4.202507202350-rMatthias Sohn11 days88-121/+121
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Change-Id: Ib49e2cf19633d986d14ef059f92499fdb5c98086
| | | | | | | | | | | | | | * AmazonS3: Do not accept DOCTYPE and entitiesMatthias Sohn11 days1-3/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This follows OWASP recommendations in https://cheatsheetseries.owasp.org/cheatsheets/XML_External_Entity_Prevention_Cheat_Sheet.html Change-Id: I3d47debf14d95c8189d51256b4eb2ba991279452
| | | | | | | | | | | | | | * ManifestParser: Do not accept DOCTYPE and entitiesIvan Frade11 days2-3/+48
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | These open the door for XXE attacks [1] and manifest do not need them. [1] https://en.wikipedia.org/wiki/XML_external_entity_attack Change-Id: Ia79971e1c34afaf287584ae4a7f71baebcb48b6a
| | | | | | | | | | | | | | * AdvertisedRequestValidator: fix WantNotValidException caused by raceMatthias Sohn11 days1-3/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fetch with protocol V2 failed under the following conditions - fetch uses bidirectional protocol (git, ssh) which uses a shortcut to determine invalid wants - not all wants are advertised - race condition: wanted ref is updated during fetch by another thread after the thread serving upload-pack determined wants and before it checks not advertised wants Fix this by calling `new ReachableCommitRequestValidator().checkWants(up, wants)` instead of throwing WantNotValidException in [1] if this race happened in the same way like it's done for unidirectional protocols (http) [2]. [1] https://github.com/eclipse-jgit/jgit/blob/stable-6.10/org.eclipse.jgit/src/org/eclipse/jgit/transport/UploadPack.java#L2002 [2] https://github.com/eclipse-jgit/jgit/blob/stable-6.10/org.eclipse.jgit/src/org/eclipse/jgit/transport/UploadPack.java#L2000 Bug: jgit-48 Change-Id: I32f28502923815dc49781aab5d810c9afbe7e7e6
| | | | | | | | | | | | | | * Prepare 5.13.4-SNAPSHOT buildsMatthias Sohn11 days88-503/+503
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Change-Id: I0fe9792b7ac883f355d2dbfa5cac97b7b26648ae
| | | | | | | | | | | | | | * Fix packaging buildMatthias Sohn11 days34-1678/+31
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - use target platform 4.21 - remove all older target platforms - update tycho to 4.0.13, hence maven needs to be run using Java 17 - enforce build is run using Java 17 or higher - remove pack200 plugins Change-Id: I50a52c22f4665b6323ee6d29da18e39960d008af
| | | | | | | | | | | * | | | MergeToolTest: update expected error from the toolIvan Frade10 days1-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This test fails in jenkins becuase the error string is slightly different. It seems to come from an external tool (git mergetool?) that maybe changed that output (?). At the moment update the error string to match the output of the tool. Parsing error messages is bound to be flaky. Change-Id: I81e9bf65088b893af3a0d8e53d57052583fc2262
* | | | | | | | | | | | | | | Lock reftable auto-refresh to ensure consistencyAntonio Barone2025-07-021-2/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Ensure that reftable auto-refresh operations, clearing the database cache and reloading the reftable stack are executed in an exclusive critical section under lock. Previously, these steps were performed without an exclusive critical section, creating a window where concurrent threads could interfere with each other. In a race condition, one thread might clear the cache and before it had a chance of reloading the stack, another thread could repopulate the cache with stale data, keeping a reference to the open BlockSource channel to the underlying tables that are subsequently removed when the first thread reloads the stack. The above race condition resulted in attempts to access closed resources and lead to ClosedChannelException errors. As an example, consider the following scenario: * T0 - Thread-1 is executing auto-refresh and it clears the database cache * T1 - The master branch moves forward (for any reason): - A new refTable (`R_new`) file is created - An existing refTable (`R_old`) file is deleted due to auto-compaction. * T3 - Thread-2 repopulates the database cache before Thread-1 has had a chance to reload the refTable stack. * T4 - Thread-1 finally reloads the refTable stack, causing the closing of the BlockSource wrapping the removed `R_old` refTable file. * T5 - Thread-2 attempts to read from the already-closed `R_old` BlockSource and the `j.n.c.ClosedChannelException` is thrown To reproduce this problem, you can run a script created to craft this racing condition: I1e78e175cff. While such errors during concurrent execution might be expected and tolerable in isolation, the situation becomes more severe when the `RepositoryCache` is involved, as is the case with Gerrit. The `FileReftableDatabase` instance is cached within the `Repository` object. When a `BlockSource` is closed prematurely due to this race condition, the dangling reference remains in memory until the cached Repository expires, which is one hour by default. This means that, once the race occurs, the repository may be unable to perform any ref lookups for up to an hour, effectively causing a repository outage. By introducing a ReentrantLock around both operations, the refresh logic now guarantees that concurrent readers and writers maintain a consistent view of the reftable state, eliminating the race condition. Verified by executing the script provided at I1e78e175cff and no exceptions are raised anymore. Bug: jgit-130 Change-Id: I6153528a7b2695115b670bda04d4d4228c1731e1
* | | | | | | | | | | | | | | Merge branch 'stable-7.1' into stable-7.2Matthias Sohn2025-06-201-0/+1
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-7.1: Fix: Close the "preserved" PackDirectory Change-Id: I82f138f134fe09717e2e024b3c87971140f01b29
| * | | | | | | | | | | | | | Merge branch 'stable-7.0' into stable-7.1Matthias Sohn2025-06-201-0/+1
| |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-7.0: Fix: Close the "preserved" PackDirectory Change-Id: Icd3f79322f8c021e18fd5c881cd9f2a406230fa8
| | * | | | | | | | | | | | | Merge branch 'stable-6.10' into stable-7.0Matthias Sohn2025-06-201-0/+1
| | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.10: Fix: Close the "preserved" PackDirectory Change-Id: Ie0ecfd8178ef4e2eef6a29d46be5645648fe88f3
| | | * | | | | | | | | | | | Fix: Close the "preserved" PackDirectoryNasser Grainawi2025-06-191-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This has been missing since the feature was first added in commit 6167641834e28f8ad322f8fde60866b339bfb7fe. It's possible we could be more aggressive and close soon after attempting to get an object from the preserved packs, but for concurrent misses that might cause thrashing. More likely it would be safe to attempt closing after successfully restoring a preserved pack. A follow up change should attempt that. Change-Id: I87d61007bcc3d03fc86bd18465ca66a2e6f697a1
* | | | | | | | | | | | | | | Merge branch 'stable-7.1' into stable-7.2Matthias Sohn2025-06-031-26/+32
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-7.1: Use the same ordering/locking in delete() as C git Change-Id: Id52c938b041604162dca9162726bfb594e96f5d1
| * | | | | | | | | | | | | | Merge branch 'stable-7.0' into stable-7.1Matthias Sohn2025-06-031-26/+32
| |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-7.0: Use the same ordering/locking in delete() as C git Change-Id: I2c38321ee410d9ec60481d56315710beaebd393a
| | * | | | | | | | | | | | | Merge branch 'stable-6.10' into stable-7.0Matthias Sohn2025-06-031-26/+32
| | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.10: Use the same ordering/locking in delete() as C git Change-Id: I0d06e39d06315e0b9e770bdf79164779d98f9f50
| | | * | | | | | | | | | | | Use the same ordering/locking in delete() as C gitDaniele Sassoli2025-05-291-26/+32
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Following the examples of cgit, lock packed-refs *before* checking for existance of refs in it [1] and *keep the lock* until the loose ref (if any) is removed [2]. The packed-refs lock is kept even when no packed-refs update is required [3] so that somebody else doesn't pack a reference that we are trying to delete. This fixes a concurrency issue that happens on projects with a substantial amount of refs(>~500k) where packing takes long enough for a ref deletion to be triggered half way through it. Not locking the packed-refs file before checking if the refs exists is not safe, as it opens up situations where loose refs are repacked in memory and locked on disk, but before the lock is released and packed-refs is flushed to disk, a ref is deleted. As packed-refs was NOT locked while checking wether a ref existed in it, the current content on disk was read, which was about to be overwritten and did not contain the ref about to be deleted. As the delete doesn't see the ref in the current, on-disk, version of packed refs, it skips processing altogether and moves on, correctly, deleting only the associated loose ref and leaving the packed one behind. Once the new packed-refs, containing the ref that was just deleted, was commited to disk, the ref would come back to life. Therefore, the packed-refs needs to be locked before checking if it contains a ref or not in the same way the C implementation of Git does at [1]. There are tradeoffs, though, in this decision, which will reduce the parallelism of deleting loose refs and performing the refs repacking, which happens very often in certain JGit implementations like Gerrit Code Review. Before this change, repacking of refs and removal of loose refs unrelated to the in-flight repacking was possible without involving any locking; after this change, all loose refs removals have to wait for the packing of refs to be completed, even though the repacking and the refs removals were completely unrelated and their namespaces disjoint. See more details on the test's performance results and the associated tradeoffs in the Issue jgit-152. NOTE: This delete ref locking logic was incorrect regardless of how the packing of the refs is implemented. Making decisions if the pack transaction is needed or not on an unlocked resource is racy and also flagged as bug at [1]. [1]https://github.com/git/git/blob/master/refs/packed-backend.c#L1590 [2]https://github.com/git/git/blob/master/refs/files-backend.c#L3261 [3]https://github.com/git/git/blob/master/refs/files-backend.c#L2943 Bug: jgit-152 Change-Id: I158ec837904617c5fdf667e295ae667b2f037945
* | | | | | | | | | | | | | | Merge branch 'stable-7.1' into stable-7.2Matthias Sohn2025-05-220-0/+0
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-7.1: Prepare 7.1.2-SNAPSHOT builds JGit v7.1.1.202505221757-r Prepare 7.0.2-SNAPSHOT builds JGit v7.0.1.202505221510-r Prepare 6.10.2-SNAPSHOT builds JGit v6.10.1.202505221210-r AmazonS3: Do not accept DOCTYPE and entities ManifestParser: Do not accept DOCTYPE and entities Change-Id: I919d0a6890ba2fd4aed745151fda82a4de116846
| * | | | | | | | | | | | | | Prepare 7.1.2-SNAPSHOT buildsMatthias Sohn2025-05-2289-540/+540
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Change-Id: If1e74998a5d6495dafecd2e29b37dbf9018f99e6
| * | | | | | | | | | | | | | JGit v7.1.1.202505221757-rv7.1.1.202505221757-rMatthias Sohn2025-05-2289-123/+123
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Change-Id: Ie69fa5fb5497f8bbb66c68b03eb161fb61bcddf1
| * | | | | | | | | | | | | | Merge branch 'stable-7.0' into stable-7.1Matthias Sohn2025-05-223-1/+53
| |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-7.0: Prepare 7.0.2-SNAPSHOT builds JGit v7.0.1.202505221510-r Prepare 6.10.2-SNAPSHOT builds JGit v6.10.1.202505221210-r AmazonS3: Do not accept DOCTYPE and entities ManifestParser: Do not accept DOCTYPE and entities Change-Id: I4506e4bf51225000418b15bf09df3287be26242a
| | * | | | | | | | | | | | | Prepare 7.0.2-SNAPSHOT buildsMatthias Sohn2025-05-2289-534/+534
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Change-Id: I80b330e687e96385176f00757dbf4491ce662dbc
| | * | | | | | | | | | | | | JGit v7.0.1.202505221510-rv7.0.1.202505221510-rMatthias Sohn2025-05-2289-123/+123
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Change-Id: Id31e58be65f27c8f475f810b83491652be1266df
| | * | | | | | | | | | | | | Merge branch 'stable-6.10' into stable-7.0Matthias Sohn2025-05-223-1/+53
| | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.10: Prepare 6.10.2-SNAPSHOT builds JGit v6.10.1.202505221210-r AmazonS3: Do not accept DOCTYPE and entities ManifestParser: Do not accept DOCTYPE and entities Change-Id: I699d57974d9ef2428355c59194c6becbc16828b7
| | | * | | | | | | | | | | | Prepare 6.10.2-SNAPSHOT buildsMatthias Sohn2025-05-2289-537/+537
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Change-Id: Id893557b6929718f8528ce50b0f7bdb23057aa15
| | | * | | | | | | | | | | | JGit v6.10.1.202505221210-rv6.10.1.202505221210-rMatthias Sohn2025-05-2289-123/+123
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Change-Id: I60dd6d671f605c7cfaed747cc0fc1772115e5f37
| | | * | | | | | | | | | | | AmazonS3: Do not accept DOCTYPE and entitiesMatthias Sohn2025-05-221-0/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This follows OWASP recommendations in https://cheatsheetseries.owasp.org/cheatsheets/XML_External_Entity_Prevention_Cheat_Sheet.html Change-Id: I3d47debf14d95c8189d51256b4eb2ba991279452
generated by cgit v1.2.3 (git 2.39.1) at 2025-07-31 17:41:32 +0000