summaryrefslogtreecommitdiffstats
path: root/org.eclipse.jgit
Commit message (Collapse)AuthorAgeFilesLines
* Merge branch 'stable-6.7' into stable-6.8Matthias Sohn2024-01-206-11/+195
|\ | | | | | | | | | | | | | | | | * stable-6.7: Introduce a PriorityQueue sorting RevCommits by commit timestamp Remove org.eclipse.jgit.benchmark/.factorypath Update jmh to 1.37 for org.eclipse.jgit.benchmark Change-Id: I5d49a9dc7da17b83243229d4d8d3b9ee0a063e65
| * Merge branch 'stable-6.6' into stable-6.7Matthias Sohn2024-01-207-43/+207
| |\ | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.6: Introduce a PriorityQueue sorting RevCommits by commit timestamp Remove org.eclipse.jgit.benchmark/.factorypath Update jmh to 1.37 for org.eclipse.jgit.benchmark Change-Id: I76ebca527e523f124bfe81c821169c790eddccb6
| | * Introduce a PriorityQueue sorting RevCommits by commit timestampLuca Milanesio2024-01-196-15/+195
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The DateRevQueue uses a tailor-made algorithm to keep RevCommits sorted by reversed commit timestamp, which has a O(n*n/2) complexity and caused the explosion of the Git fetch times to tens of seconds. The standard Java PriorityQueue provides a O(n*log(n)) complexity and scales much better with the increase of the number of RevCommits. Introduce a new implementation DateRevPriorityQueue of the DateRevQueue based on PriorityQueue. Enable usage of the new DateRevPriorityQueue implementation by setting the system property REVWALK_USE_PRIORITY_QUEUE=true. By default the old implementation DateRevQueue is used. Benchmark results: ``` (numCommits) (usePriorityQueue) Mode Cnt Score Error Units 5 true avgt 10 39,4 ± 6,1 ns/op 5 false avgt 10 14,1 ± 2,2 ns/op 10 true avgt 10 29,7 ± 3,5 ns/op 10 false avgt 10 13,2 ± 2,0 ns/op 50 true avgt 10 50,4 ± 5,3 ns/op 50 false avgt 10 18,6 ± 0,2 ns/op 100 true avgt 10 58,3 ± 5,0 ns/op 100 false avgt 10 20,5 ± 0,8 ns/op 500 true avgt 10 51,7 ± 2,6 ns/op 500 false avgt 10 43,3 ± 0,5 ns/op 1000 true avgt 10 49,2 ± 2,4 ns/op 1000 false avgt 10 62,7 ± 0,2 ns/op 5000 true avgt 10 48,8 ± 1,5 ns/op 5000 false avgt 10 228,3 ± 0,5 ns/op 10000 true avgt 10 44,2 ± 0,9 ns/op 10000 false avgt 10 377,6 ± 2,7 ns/op 50000 true avgt 10 50,3 ± 1,6 ns/op 50000 false avgt 10 637,0 ± 111,8 ns/op 100000 true avgt 10 61,8 ± 4,4 ns/op 100000 false avgt 10 965,1 ± 268,0 ns/op 500000 true avgt 10 127,2 ± 7,9 ns/op 500000 false avgt 10 9610,2 ± 184,8 ns/op ``` Memory allocation results: ``` Number of commits loaded: 850 000 Custom implementation: 378 245 120 Bytes Priority queue implementation: 340 495 616 Bytes ``` Bug: 580137 Change-Id: I8b33df6e9ee88933098ecc81ce32bdb189715041 Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
* | | Merge branch 'stable-6.7' into stable-6.8Matthias Sohn2024-01-101-1/+1
|\| | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.7: Silence API warnings Make sure ref to prune is in packed refs Checkout: better directory handling Change-Id: Ie8c433e14d025055aa6ce674d6d1220fe1354785
| * | Merge branch 'stable-6.6' into stable-6.7Matthias Sohn2024-01-102-1/+102
| |\| | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.6: Silence API warnings Make sure ref to prune is in packed refs Checkout: better directory handling Change-Id: Id3b01c7a583d45b4e67e3f993006a5646f77a41d
| | * Silence API warningsMatthias Sohn2024-01-081-0/+101
| | | | | | | | | | | | Change-Id: I44084c45ed2c5e642021d1de4f37af769e4b317f
| | * Make sure ref to prune is in packed refsFabio Ponciroli2023-12-091-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | RefDirectory:pack might raise an NPE when deleting loose refs as final part of the RefDirectory.pack(). This is what the code does: 1) packed ref update: update the list of refs which will be persisted in packed-refs 2) persit packed-refs: flush on file the refs computed in #1 3) prune loose refs: prune loose refs that have been packed in #2 The code correctly locks the packed-refs file during phases 1 to 3. However, it makes the wrong assumption of considering the loose refs set as immutable between phases 1 and 3. The number and values of loose refs on the filesystem can mutate at any time whilst the RefDirectory.pack() is in progress. Assuming the contrary can lead to an NPE when retrieving refs from the mutable loose refs list during phase #3. Make sure that the ref is not null before dereferencing its object-id value. Bug: jgit-4 Change-Id: I2cd01f8a880f3c6561ad978a389ec2db45b6018b
* | | Prepare 6.8.1-SNAPSHOT buildsMatthias Sohn2023-11-293-59/+59
| | | | | | | | | | | | Change-Id: I56458c5345dcd9544868c948e90c9827d25c6850
* | | JGit v6.8.0.202311291450-rv6.8.0.202311291450-rMatthias Sohn2023-11-293-4/+4
| | | | | | | | | | | | | | | Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Change-Id: Iecfe3f628cfdc437ee0d63fe52653ef952c8c494
* | | Rename method parameter to fix warning about hidden fieldMatthias Sohn2023-11-291-3/+3
| | | | | | | | | | | | Change-Id: Id094ab022f82db0f3eb220052adc0aed1cde2d50
* | | Fix warnings about empty control flow statementsMatthias Sohn2023-11-291-3/+11
| | | | | | | | | | | | Change-Id: If1595ac148edf88a18efdd90c4f40d0ac719caab
* | | BitmapIndexImpl: externalize error messageMatthias Sohn2023-11-293-1/+3
| | | | | | | | | | | | Change-Id: I5189e966aecc57fe7c3e6c3f7f8e6b16917ee27f
* | | Suppress not-externalized string warningsMatthias Sohn2023-11-291-1/+1
| | | | | | | | | | | | Change-Id: I2a0a588227d451cb175e17eef20ecb6fe9e9ef80
* | | Silence API error for new method BitmapIndex#addBitmapLookupListenerMatthias Sohn2023-11-291-0/+8
| | | | | | | | | | | | | | | | | | it has an empty default implementation. Change-Id: I56b16c48992652a4ddaaf4c0435e08452c65749e
* | | Improve footer parsing to allow multiline footers.Kamil Musin2023-11-242-28/+30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | According to the https://git-scm.com/docs/git-interpret-trailers the CGit supports multiline trailers. Subsequent lines of such multiline trailers have to start with a whitespace. We also rewrite the original parsing code to make it easier to work with. The old code had pointers moving both backwards and forwards at the same time. In the rewritten code we first find the start of the last paragraph and then do all the parsing. Since all the getters of the FooterLine return String, I've considered rewriting the parsing code to operate on strings. However the original code seems to be written with the idea, that the data is only lazily copied in getters and no extra allocations should be performed during original parsing (ex. during RevWalk). The changed code keeps to this idea. Bug: Google b/312440626 Change-Id: Ie1e3b17a4a5ab767b771c95f00c283ea6c300220
* | | BitmapIndex: Add interface to track bitmaps found (or not)Ivan Frade2023-11-212-1/+67
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We want to know what objects had bitmaps in the walk of the request. We can check their position in the history and evaluate our bitmap selection algorithm. Introduce a listener interface to the BitmapIndex to report which getBitmap() calls returned a bitmap (or not) and a method to the bitmap index to set the listener. Change-Id: Iac8fcc1539ddd2dd450e8a1cf5a5b1089679c378
* | | BitmapWalker: Remove BitmapWalkListenerIvan Frade2023-11-211-82/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | We can track bitmaps queries that found a bitmap directly in the BitmapIndex. Remove the listener. Change-Id: I5ad518a58b681bf327fee3ae5c5f6e4449d3da1f
* | | Merge "Fix typo in FileUtils.isStaleFileHandle() javadoc"Matthias Sohn2023-11-171-5/+5
|\ \ \
| * | | Fix typo in FileUtils.isStaleFileHandle() javadocDariusz Luksza2023-11-171-5/+5
| | | | | | | | | | | | | | | | | | | | Signed-off-by: Dariusz Luksza <dariusz.luksza@gmail.com> Change-Id: Iac431bbcb00f0a1fa36e2ba4af5fe33fcee69e04
* | | | Fix branch ref exist checkflorian.signoret2023-11-181-3/+1
|/ / / | | | | | | | | | | | | | | | | | | | | | | | | When a tag with the same name as the branch exists, the branch creation process should work too. We should detect that the branch already exists, and allow to force create it when the force option is used. Bug: 582538 Change-Id: I3b350d03be8edcde10e97b2318343240ca896cb0
* | | Merge "Fix annotation of deprecated constant CONFIG_KEY_STREAM_FILE_TRESHOLD"Matthias Sohn2023-11-161-1/+2
|\ \ \
| * | | Fix annotation of deprecated constant CONFIG_KEY_STREAM_FILE_TRESHOLDMatthias Sohn2023-11-171-1/+2
| | | | | | | | | | | | | | | | Change-Id: I71f067199c63aa1407f543f6d40ef05b928e9f95
* | | | BitmapWalkListener: Use plain interface with noop instanceIvan Frade2023-11-161-14/+28
|/ / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In this new interface default methods are useful only to instantiate noop instances. We rather reuse the same noop instance and save the "default" to add backward compatible methods to existing interfaces. Make the methods regular interface methods and provide a noop instance. Change-Id: Ie84ff17c8e9f16837245751739ee8c99463e76ee
* | | BitmapWalkListener: Add method and rename for commitsIvan Frade2023-11-161-10/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | During the walk, the commit can be either 1. already in the walk bitmap 2. unvisited so far with bitmap in the bitmap index 3. unvisited so far without bitmap in the bitmap index Expose these three states in the interface. This makes the interface easier to explain: it reports the commits found during the walk. As it is all about commits, rename the methods to onCommit***. Change-Id: I661f303eb22d3e735b0e439f16df7ace612376d9
* | | Merge "CommitGraphWriter: Remove unnecesary progress update call"Ivan Frade2023-11-161-3/+0
|\ \ \
| * | | CommitGraphWriter: Remove unnecesary progress update callIvan Frade2023-11-101-3/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Change [1] reduced the scope of the "writing commit graph" monitoring task. This left some monitor#update() calls out of any task. When out of a task, the #update call is a noop. Delete this update calls as they are noops and misleading. The affected chunks are usually small and quick to write, so probably they don't need progress monitoring. [1] https://git.eclipse.org/r/c/jgit/jgit/+/205339 Change-Id: I74d94e6e44e58816937dc8a84e5a10b340e54e0b
* | | | Merge "PatchApplier: wrap output's TemporaryBuffer with a CountingOutputStream"Jonathan Nieder2023-11-161-2/+6
|\ \ \ \
| * | | | PatchApplier: wrap output's TemporaryBuffer with a CountingOutputStreamNitzan Gur-Furman2023-11-161-2/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The documentation for TemporaryBuffer::length says: "The length is only accurate after {@link #close()} has been invoked". However, we need to have the stream open while accessing the length. This prevents patches on large files to be applied correctly, as the result get trimmed. Bug: Google b/309500446 Change-Id: Ic1540f6d0044088f3b46f1fad5f6a28ec254b711
* | | | | BitmapWalker: announce walked objects via listener interfaceIvan Frade2023-11-151-2/+62
|/ / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We want to know what objects had a bitmap in the walk, to see where do they sit in the commit history and evaluate our bitmap selection algorithm. Add a listener interface to the bitmap walker announcing the objects walked and whether they had bitmap. Change-Id: I956fe2ad927a500710d2cbe78ecd4d26f178c266
* | | | Fix typo in constant name CONFIG_KEY_STREAM_FILE_TRESHOLDMatthias Sohn2023-11-153-6/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Since it's part of the API deprecate the wrong spelling and add the correct one with the same value. Change-Id: I0f6ea95a5e66c9e80142eb6d40eb7ec3a7aaf8e2
* | | | Simplify StringUtils#commonPrefixNasser Grainawi2023-11-141-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | By first checking for null-ness and then for the number of strings to compare we can get rid of a redundant null check. Change-Id: I0d9a088352c6c1ffea12bc2cded2c63e5293a8a7
* | | | Optimize RefDirectory.getRefsByPrefix(String...)Dariusz Luksza2023-11-122-0/+49
|/ / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently for file-based repositories JGit will go over all refs in the repository forach `ref-prefix` listed in the `ls-refs` command in git protocol v2 request. Native git, uses a different approach, where all refs are read once and then for each ref, all `ref-prefix` filter values are checked in one pass. This change implements this approach in JGit only in the `RefDirectory` backend. And makes `ref-prefix` filtering ~40% faster for repositories with packed refs. Different implementations were tested on a synthetic file repository with 10k refs in `refs/heads/` and `290k` in `refs/changes`. Before testing `git pack-refs` command was executed. All results are in seconds. Current Impl: 39.340 37.093 35.996 Nested for loops: 25.077 24.742 24.748 Nested streams: 24.827 24.890 27.525 Parallel stream + stream: 23.357 23.318 23.174 Nested parallel streams: 23.490 23.318 23.317 Stream + for loop: 23.147 23.210 23.126 Parallel stream + for loop: 23.317 23.423 22.847 The elapsed time was measured around `getRefByPrefix` call in `Uploadapack.getFilteredRefs(Collection<String>)` (around lines 952 and 954). For testing a modified version of `UploadPackTest.testV2LsRefsRefPrefix()` was used. The modifications here included: * shadowing protected `repo` variable with `FileRepository` pointing to the synthetic repo with 300k refs described above, * mimicking the git client clone request by adding `ref-prefix HEAD`, `ref-prefix refs/heads/` and `ref-prefix refs/tags/` Based on the above results, the implementation with parallel stream and stream was selected. Bug: 578550 Signed-off-by: Dariusz Luksza <dariusz.luksza@gmail.com> Change-Id: I6416846c074b611ff6ec9d351dbafcfbcaf68e66
* | | Merge changes I0d10aeac,I9aced1b7,Ied394859,Ibb61f63f,I20498746, ...Matthias Sohn2023-11-102-4/+5
|\ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * changes: Use try-with-resource to ensure UploadPack is closed Fix hiding field warning Fix warning for empty code blocks Fix boxing warnings errorprone: remove unnecessary parentheses Update mockito to 5.7.0 and bytebuddy to 1.14.9 Enable Maven reproducible builds Upgrade bazlets to the latest revision
| * | | Fix boxing warningsMatthias Sohn2023-11-091-2/+4
| | | | | | | | | | | | | | | | Change-Id: Ibb61f63f17e909ee0e6a781c9418f7c1ac2c7cd7
| * | | errorprone: remove unnecessary parenthesesMatthias Sohn2023-11-091-5/+4
| | | | | | | | | | | | | | | | | | | | | | | | See https://errorprone.info/bugpattern/UnnecessaryParentheses Change-Id: I204987469d478c9cc887ac66c9ffc84c9977c400
| * | | Enable Maven reproducible buildsMatthias Sohn2023-11-091-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - configure Maven to run build reproducibly [1] - use UTC timestamp of checked out commit as build timestamp - add git-describe, git-commit-id, git-commit-id, git-tags, git-remote-origin-url to MANIFEST.MF files - configure cyclonedx-maven-plugin to also use UTC timestamp of checked out commit - for packaging build use tycho-buildtimestamp-jgit [2] to ensure version uses the timestamp of the last commit - SBOMs are not reproducible by design [3] they should have a build timestamp matching the time when the build was executed and a serial number which is a unique UUID per build run. Hence exclude them from comparison [4]. - Use gmavenplus-plugin to format build timestamps. Maven expects build timestamp in ISO-8601 format, to replace the qualifier in versions the timestamp format must be compatible with rules for OSGi version numbers. Didn't find a way to read the properties set by the git-commit-id-maven-plugin from another plugin. Hence use JGit in a groovy script to get the commit time of the current HEAD and provide it in these two formats. TODO: packaging build (features and p2 repository) is not yet binary reproducible since that's not yet supported by Tycho [5], artefacts have reproducible version numbers but file lastModified timestamps are not yet reproducible. Test plan for Maven build: - build using mvn clean install" - verify second build is reproducible: mvn -T1 clean verify artifact:compare verification seems not to be thread-safe, hence run it with a single thread using option -T1 For packaging build (still fails due to non-reproducible file timestamps): - build using mvn -f org.eclipse.jgit.packaging/pom.xml clean install - verify second build is reproducible: mvn -T1 -f org.eclipse.jgit.packaging/pom.xml clean verify artifact:compare [1] https://maven.apache.org/guides/mini/guide-reproducible-builds.html [2] https://wiki.eclipse.org/Tycho/Reproducible_Version_Qualifiers [3] https://github.com/CycloneDX/cyclonedx-maven-plugin/issues/84 [4] https://maven.apache.org/plugins/maven-artifact-plugin/compare-mojo.html [5] https://github.com/eclipse-tycho/tycho/issues/233 Change-Id: I0202f55a1b6ae0edd922cfef638beb39d2ce9417
* | | | Merge "Revert "Optimise Git protocol v2 `ref-prefix` scanning""Ivan Frade2023-11-081-3/+5
|\ \ \ \ | |/ / / |/| | |
| * | | Revert "Optimise Git protocol v2 `ref-prefix` scanning"Ivan Frade2023-11-081-3/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This reverts commit 3937300f3eb4dd557ec2d195f21793f737d6cb4e. Reason for revert: This kills performance on the DFS side, that relies on loading the minimal amount of refs and reftables for quick prefix searches. Reverting as a safe option to keep master in good performance until we decide how to reintroduce this change. Change-Id: I7b1a3f900d9c78ce95cf0972abb50b6becfe3bb1
* | | | Merge "ComboBitset: Add Javadoc"Matthias Sohn2023-11-081-0/+8
|\ \ \ \
| * | | | ComboBitset: Add JavadocHan-Wen Nienhuys2023-11-071-0/+8
| | | | | | | | | | | | | | | | | | | | Change-Id: I799991327cadf646329eacbac40d41cb1b3391ad
* | | | | CommitGraphWriter: Add progress monitor to bloom filter computationIvan Frade2023-11-073-2/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Bloom filter computation can be an expensive process and right now it is invisible to the user. Report progress while calculating bloom filters. Log of GC with bloom filter enabled: Computing commit-graph path bloom filters: 100% (9551/9551) Computing commit-graph generation numbers: 100% (9551/9551) Writing out commit-graph: 100% (9551/9551) Change-Id: Ife65e63ac2c37d064d5f049a366cbb52c3ef6798
* | | | | CommitGraphWriter: Use ProgressMonitor from the OutputStreamIvan Frade2023-11-071-8/+8
|/ / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The same progress monitor is passed around as parameter and inside the output stream. The functions use one to start tasks and another to report progress, which is confusing. The stream needs the monitor to check cancellations so we cannot remove it from there. Make all code take the monitor from the stream. Change-Id: Id3cb9c1cb0bd47318b46ef934a9d4037341e25a7
* / / / CommitGraphWriter: Unnest generation-number progressIvan Frade2023-11-072-11/+4
|/ / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The ProgressMonitor task to track the calculation of generation numbers is nested inside the task that follows the writing of all lines in the commit-graph. ProgressMonitor doesn't support nested tasks and this confuses the counting. Move the start/end of the "writing commit graph" task to the writeCommitData section, after calculating the generation numbers. Make that task track by commits instead of by lines. Moving the start/end of the progress task to the chunk-writing functions is clearer and easier to extend. Logging of GC before: Writing out commit-graph in 3 passes: 51% ( 9807/19358) Computing commit-graph generation numbers: 100% (9551/9551) Logging of GC after: Computing commit-graph generation numbers: 100% (9551/9551) Writing out commit-graph: 100% (9551/9551) Change-Id: I87d69c06c9a3c7e75be12b6f0d1a63b5924e298a
* | | Optimise Git protocol v2 `ref-prefix` scanningDariusz Luksza2023-11-071-5/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currenty JGit will go over all refs in the repository for each `ref-prefix`. This means that refs will be read multiple times, which leads to subpar performance. Native git, uses a different approach, where all refs are read once and then for each ref, all `ref-prefix` filter values are checked in one pass. This change implements this approach in JGit. And makes `ref-prefix` filtering ~28% faster for a repository with fully packed refs and ~5% when RefTable is used instead of refdir. Different implementations were tested on a synthetic file repository with 300k refs. Different implementations were tested for unpacked and fully packed refs (results are in seconds). Unpacked refs: Current Impl: 54.838 57.234 56.138 Nested for loops: 36.094 37.025 36.502 Nested stream's: 36.154 35.989 37.262 Parallel stream + stream: 36.923 37.272 35.362 Nested parallel stream's: 35.512 38.395 36.745 Stream + for loop: 34.950 36.164 37.191 Parallel stream + for loop: 37.695 35.511 35.378 Packed refs: Current Impl: 39.713 39.954 38.653 Nested for loops: 29.891 29.753 29.377 Nested stream's: 30.340 29.637 30.412 Parallel stream + stream: 28.653 28.254 29.138 Nested parallel stream's: 29.942 28.850 31.030 Stream + for loop: 29.405 29.576 30.539 Parallel stream + for loop: 29.012 29.215 29.380 RefTable: Current Impl: 0.273 0.294 0.330 Nested for loops: 0.252 0.169 0.215 Nested stream's: 0.252 0.228 0.213 Parallel stream + stream: 0.233 0.259 0.247 Nested parallel stream's: 0.416 0.309 0.340 Stream + for loop: 0.224 0.247 0.242 Parallel stream + for loop: 0.347 0.246 0.346 The elapsed time was measured around `getRefsByPrefix` call in `UploadPack.getFilteredRefs(Collection<String>)` (around lines 952 and 954). Based on the above results, the implementation with parallel stream and stream was selected. Bug: 578550 Change-Id: Iac3a3aacf897b87b3448c1d528cdac64ad312199 Signed-off-by: Dariusz Luksza <dariusz.luksza@gmail.com>
* | | UploadPack: use want-refs as advertised set in fetch v2Patrick Hiesel2023-10-271-5/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Protocol v2 introduced refs-in-wants and ls-remote with prefixes. UploadPack already uses prefixes provided by the client during a v2 ref advertisement (ls-refs). However, when the client consequently sends another request to fetch a previously advertised ref (with want-ref lines), the server uses the whole set of advertised refs to compute reachability. In repos with many refs, this slows down the reachability checks setting up and walking through unnecessary refs. For gerrit it can also break valid requests because in gerrit "all" means "recent" and the wanted-ref could fall out of the "recent" range when reloading all refs at fetch time. Treat wanted-refs like a ref-prefix when calculating the advertised refs on v2 fetch command. Less refs means a faster setup and less walk for the reachability checks. Note that wanted-refs filters only over the refs visible to the user, so this doesn't give any extra visibility to the caller. If the request contains also "want <oid>" lines, we cannot use this optimization. Those objects could be reachable from any visible branch, not necessarily in the wanted-refs. Google-Bug: b/122888978 Change-Id: I2a4ae171d4fc5d4cb30b020cb073ad23dd5a66c4
* | | BasePackFetchConnection: Avoid full clone with useNegotiationTipRonald Bhuleskar2023-10-251-19/+45
| | | | | | | | | | | | | | | | | | | | | | | | With the useNegotiationTip flag (introduced in change 738dacb), the client sends to the server only the tips of the wanted refs for the negotiation. Some wanted refs may not exist in the client (yet) and our implementation ignores them. So when only non-existing refs are wanted, jgit doesn't send any tips and the server understands it is a full clone. In useNegotiationTip, send ALL_REFS if any of the wanted refs does not exists locally. Change-Id: Ide04c5df785b9212abcd9d3cba194515e0af166f
* | | DfsPackFile: Do not attempt to read stream if pack doesn't have itIvan Frade2023-10-171-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Other getters (e.g. bitmap or commit graph) cover the case that the pack doesn't have the corresponding extension. Do the same here to detect this early and avoid an IOException in openFile. Change-Id: I29726b7ede0f795d35543453a3e7f92cee872a78
* | | PackObjectSizeIndexLoader: Log wrong bytes on exceptionIvan Frade2023-10-173-3/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | When the exception is thrown, we don't know if it is because the stream didn't have data or had a wrong header. Log the read bytes to differentiate these cases. Change-Id: Ie7612eab39016f5ad7f1bfb2e07cab972dab796f
* | | Silence API warnings for API added in 5.13.3Matthias Sohn2023-10-171-0/+51
| | | | | | | | | | | | | | | | | | | | | | | | | | | This was added in - f103a1d5c605 "Add support for git config repack.packKeptObjects" - f5f4bf0ad97f "Do not exclude objects in locked packs from bitmap processing" Change-Id: Id6af9fe549535c4e92de9080a41ef9f72a6646dd
* | | FileBasedConfig: in-process synchronization for load() and save()Thomas Wolf2023-10-141-51/+71
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | On Windows reading and replacing a file via renaming concurrently may fail either in the reader or in the thread renaming the file. For renaming, FileUtils.rename() has a last-case fallback in which it deletes the target file before attempting the rename. If a reader reads at that moment, it will produce an empty config, and the snapshot and hash may be wrong because the concurrently running save() may set them. It's not really possible to do all this in a thread-safe manner without some synchronization. Add a read-write lock to synchronize readers and writers to avoid at least that JGit steps on its own feet. Bug: 451508 Change-Id: I7e5f0f26e02f34ba02dc925a445044d3e21389b4 Signed-off-by: Thomas Wolf <twolf@apache.org>