aboutsummaryrefslogtreecommitdiffstats
path: root/org.eclipse.jgit/src
Commit message (Collapse)AuthorAgeFilesLines
* Merge branch 'stable-5.4' into stable-5.5stable-5.5Matthias Sohn2021-12-302-3/+24
|\ | | | | | | | | | | | | * stable-5.4: Use FileSnapshot without using configs for FileBasedConfig Change-Id: I84e11bdaa9306e23212dac9d8670557a18d40107
| * Merge branch 'stable-5.3' into stable-5.4stable-5.4Matthias Sohn2021-12-302-3/+24
| |\ | | | | | | | | | | | | | | | | | | * stable-5.3: Use FileSnapshot without using configs for FileBasedConfig Change-Id: I3d8eb2fa721e1a791db47a2342acc690ced01715
| | * Merge branch 'stable-5.2' into stable-5.3stable-5.3Matthias Sohn2021-12-302-3/+24
| | |\ | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.2: Use FileSnapshot without using configs for FileBasedConfig Change-Id: Ib79c310c5b632e845ba69ce65e739ae0146103ca
| | | * Merge branch 'stable-5.1' into stable-5.2stable-5.2Matthias Sohn2021-12-302-3/+24
| | | |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.1: Use FileSnapshot without using configs for FileBasedConfig Change-Id: I17ede8876a0cf231c38cb9652c7bf51553b1e90e
| | | | * Use FileSnapshot without using configs for FileBasedConfigstable-5.1Luca Milanesio2021-12-302-3/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | FileBasedConfig should not rely on auto-detection of the file-snapshot attribute computation based on config. The check was already performed when a new FileBasedConfig is created at L158: // don't use config in this snapshot to avoid endless recursion newSnapshot = FileSnapshot.saveNoConfig(getFile()); The check was missing though when the FileBasedConfig is saved to disk and the new snapshot is obtained from the associated LockFile. This change fixes the issue by keeping a non-config based FileSnapshot also after a FileBasedConfig is saved. Bug: 577983 Change-Id: Id1e410ba687e683ff2b2643af31e1110b103b356
* | | | | Merge branch 'stable-5.4' into stable-5.5Thomas Wolf2021-12-262-13/+0
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.4: Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation" Change-Id: Ia1665dd92ccc3811a6116f41421a05aca10fc6eb Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
| * | | | Merge branch 'stable-5.3' into stable-5.4Thomas Wolf2021-12-262-13/+0
| |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.3: Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation" Change-Id: I52a57a17abe60e30e3d7615f8cb4d0c5e6aebd9b Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
| | * | | Merge branch 'stable-5.2' into stable-5.3Thomas Wolf2021-12-262-13/+0
| | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.2: Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation" Change-Id: Id37f47a5ef2e3c8329eca30c171941f7e5606a85 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
| | | * | Merge branch 'stable-5.1' into stable-5.2Thomas Wolf2021-12-262-13/+0
| | | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.1: Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation" Change-Id: I625667c2718ab31ae7df907c3dd6024a933913b8 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
| | | | * Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot ↵Thomas Wolf2021-12-262-13/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | creation" This reverts commit f829f5f838e0f9c17373ea6cb3407976a8f395ff. Using MISSING_FILEKEY as indicator for a non-existing file doesn't work on Windows. Bug: 577954 Change-Id: I92102a3d259f6cc0f367096a3213cfa794466817 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* | | | | Merge branch 'stable-5.4' into stable-5.5Matthias Sohn2021-12-043-9/+35
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.4: Remove unused import in ApacheSshTest Update maven plugins Ignore missing javadoc in test bundles storage: file: De-duplicate File.exists()+File.isFile() RefDirectory.scanRef: Re-use file existence check done in snapshot creation FileSnapshot: Lazy load file store attributes cache Update eclipse-jarsigner-plugin to 1.3.2 Fix p2 repository URLs Change-Id: I5ce55c6c152a59602fef9fb08c16dedf666444e2
| * | | | Merge branch 'stable-5.3' into stable-5.4Matthias Sohn2021-12-043-9/+35
| |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.3: Remove unused import in ApacheSshTest Update maven plugins Ignore missing javadoc in test bundles storage: file: De-duplicate File.exists()+File.isFile() RefDirectory.scanRef: Re-use file existence check done in snapshot creation FileSnapshot: Lazy load file store attributes cache Update eclipse-jarsigner-plugin to 1.3.2 Fix p2 repository URLs Change-Id: I0e80612c8ddb934466105312f9107e91d581de0b
| | * | | Merge branch 'stable-5.2' into stable-5.3Matthias Sohn2021-12-043-9/+35
| | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.2: Remove unused import in ApacheSshTest Update maven plugins Ignore missing javadoc in test bundles storage: file: De-duplicate File.exists()+File.isFile() RefDirectory.scanRef: Re-use file existence check done in snapshot creation FileSnapshot: Lazy load file store attributes cache Update eclipse-jarsigner-plugin to 1.3.2 Fix p2 repository URLs Change-Id: Id989365446cb55b9188e02e6e1f8cbc8ab71071c
| | | * | Merge branch 'stable-5.1' into stable-5.2Matthias Sohn2021-12-043-9/+35
| | | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.1: Update maven plugins Ignore missing javadoc in test bundles storage: file: De-duplicate File.exists()+File.isFile() RefDirectory.scanRef: Re-use file existence check done in snapshot creation FileSnapshot: Lazy load file store attributes cache Update eclipse-jarsigner-plugin to 1.3.2 Fix p2 repository URLs Change-Id: If3ad2cfb6c341b0ff5c2120a5a1646f6368d86e7
| | | | * storage: file: De-duplicate File.exists()+File.isFile()Nasser Grainawi2021-12-032-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | File.isFile() [1] checks if the file exists and is a normal file. [1] https://docs.oracle.com/javase/8/docs/api/java/io/File.html#isFile-- Change-Id: I0a883f2482ecc5ac58b270351b416742b568eb68 Signed-off-by: Nasser Grainawi <quic_nasserg@quicinc.com>
| | | | * RefDirectory.scanRef: Re-use file existence check done in snapshot creationKaushik Lingarkar2021-12-032-0/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Return immediately in scanRef if the loose ref was identified as missing when a snapshot was attempted for the ref. This will help performance of scanRef when the ref is packed but has a corresponding empty dir in 'refs/'. For example, consider the case where we create 50k sharded refs in a new namespace called 'new-refs' using an atomic 'BatchRefUpdate'. The refs are named like 'refs/new-refs/01/1/1', 'refs/new-refs/01/1/2', 'refs/new-refs/01/1/3' and so on. After the refs are created, the 'new-refs' namespace looks like below: $ find refs/new-refs -type f | wc -l 0 $ find refs/new-refs -type d | wc -l 5101 At this point, an 'exactRef' call on each of the 50k refs without this change takes ~2.5s, where as with this change it takes ~1.5s. Change-Id: I926bc41b9ae89a1a792b1b5ec9a17b05271c906b Signed-off-by: Kaushik Lingarkar <quic_kaushikl@quicinc.com>
| | | | * FileSnapshot: Lazy load file store attributes cacheKaushik Lingarkar2021-12-021-7/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Doing a getFileStoreAttributes call even when the file doesn't exist is unnecessary. This call is particularly slow on some filesystems. Instead, do it only when the file exists and load the appropriate cache. This update can help speed up RefDirectory.exactRef when the ref is packed, but has a corresponding empty dir for it under 'refs/'. This scenario can happen when an atomic 'BatchRefUpdate' creates new sharded refs. For example, consider the case where we create 50k sharded refs in a new namespace called 'new-refs' using an atomic 'BatchRefUpdate'. The refs are named like 'refs/new-refs/01/1/1', 'refs/new-refs/01/1/2', 'refs/new-refs/01/1/3' and so on. After the refs are created, the 'new-refs' namespace looks like below: $ find refs/new-refs -type f | wc -l 0 $ find refs/new-refs -type d | wc -l 5101 At this point, an 'exactRef' call on each of the 50k refs without this change takes ~30s, where as with this change it takes ~2.5s. Change-Id: I4a5d4c6a652dbeed1f4bc3b4f2b2f1416f7ca0e7 Signed-off-by: Kaushik Lingarkar <quic_kaushikl@quicinc.com>
* | | | | Merge branch 'stable-5.4' into stable-5.5Matthias Sohn2021-06-052-73/+16
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.4: Prepare 5.1.17-SNAPSHOT builds JGit v5.1.16.202106041830-r BatchRefUpdate: Skip saving conflicting ref names and prefixes in memory BatchRefUpdateTest: Accurately assert RefsChangedEvent(s) fired Optimize RefDirectory.isNameConflicting() Update bazlets and bazel version Change-Id: If5232b68d3e25df7b71e417cbcbb39476e925f22
| * | | | Merge branch 'stable-5.3' into stable-5.4Matthias Sohn2021-06-052-73/+16
| |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.3: Prepare 5.1.17-SNAPSHOT builds JGit v5.1.16.202106041830-r BatchRefUpdate: Skip saving conflicting ref names and prefixes in memory BatchRefUpdateTest: Accurately assert RefsChangedEvent(s) fired Optimize RefDirectory.isNameConflicting() Update bazlets and bazel version Change-Id: I1338fc79a7be6b77fb28df511dd7504fb19b6d1a
| | * | | Merge branch 'stable-5.2' into stable-5.3Matthias Sohn2021-06-052-73/+16
| | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.2: Prepare 5.1.17-SNAPSHOT builds JGit v5.1.16.202106041830-r BatchRefUpdate: Skip saving conflicting ref names and prefixes in memory BatchRefUpdateTest: Accurately assert RefsChangedEvent(s) fired Optimize RefDirectory.isNameConflicting() Update bazlets and bazel version Change-Id: I7838f7d237a3598bf55995426d7ba1de146cb6ad
| | | * | Merge branch 'stable-5.1' into stable-5.2Matthias Sohn2021-06-052-73/+16
| | | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.1: Prepare 5.1.17-SNAPSHOT builds JGit v5.1.16.202106041830-r BatchRefUpdate: Skip saving conflicting ref names and prefixes in memory BatchRefUpdateTest: Accurately assert RefsChangedEvent(s) fired Optimize RefDirectory.isNameConflicting() Update bazlets and bazel version Change-Id: If1b5a2b380cf155e66bf5d5c6d216f86c919bb37
| | | | * BatchRefUpdate: Skip saving conflicting ref names and prefixes in memoryKaushik Lingarkar2021-05-241-41/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Rather than getting all ref names and prefixes and saving them in memory to perform the check for conflicting names, rely on RefDirectory.isNameConflicting as it is no longer an expensive call after it was optimized in Ie994fc. The old optimization to save ref names and prefixes in memory was targeted towards making clones faster. With this change, the clone performance is unaffected when tests were done with repos containing many(~500k) refs. Here are few recorded elapsed times for creating 10 branches using BatchRefUpdate on NFS based repositories with varying loose refs count. As seen here, this change helps improve the BatchRefUpdate performance from O(n^2) to O(1). loose_refs_count with_change without_change 50 241 ms 310 ms 300 263 ms 1502 ms 1k 181 ms 4241 ms 2k 204 ms 6440 ms 9k 158 ms 25930 ms 20k 154 ms 60443 ms 50k 171 ms 135199 ms 110k 157 ms 329450 ms 160k 209 ms 396328 ms This update improves the Gerrit notedb migration performance as it uses BatchRefUpdate to write change meta refs similar to the test performed above. Change-Id: I853ac6c7feb4b39c3156c01876b38cbd182accfe Signed-off-by: Kaushik Lingarkar <quic_kaushikl@quicinc.com>
| | | | * Optimize RefDirectory.isNameConflicting()Kaushik Lingarkar2021-05-241-32/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Avoid having to scan over ALL loose refs to determine if the name is nested within or is a container of an existing reference. This can get really expensive if there are too many loose refs. Instead use exactRef and getRefsByPrefix which scan based on a prefix. With a simple shell script(like below) using jgit client to create 1k refs in a new repository on NFS, this change brings down the time from 12mins to 7mins. for ref in $(seq 1 1000); do jgit branch "$ref" done Here are few recorded elapsed times to create a new branch on NFS based repositories with varying loose refs count. As we see here, this change improves the name conflicting check from O(n^2) to O(1). loose_refs_count with_change without_change 50 44 ms 164 ms 300 45 ms 1193 ms 1k 38 ms 2610 ms 2k 44 ms 6003 ms 9k 46 ms 27860 ms 20k 45 ms 48591 ms 50k 51 ms 135471 ms 110k 43 ms 294252 ms 160k 52 ms 430976 ms Change-Id: Ie994fc184b8f82811bfb37b111eb9733dbe3e6e0 Signed-off-by: Kaushik Lingarkar <quic_kaushikl@quicinc.com>
* | | | | Merge branch 'stable-5.4' into stable-5.5Matthias Sohn2020-12-021-0/+1
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.4: Prepare 5.3.10-SNAPSHOT builds JGit v5.3.9.202012012026-r Prepare 5.1.16-SNAPSHOT builds JGit v5.1.15.202012011955-r Fix PackInvalidException when fetch and repack run concurrently Change-Id: Iede8686198332d6271771bef6eb00c25f1103979 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
| * | | | Merge branch 'stable-5.3' into stable-5.4Matthias Sohn2020-12-021-0/+1
| |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.3: Prepare 5.3.10-SNAPSHOT builds JGit v5.3.9.202012012026-r Prepare 5.1.16-SNAPSHOT builds JGit v5.1.15.202012011955-r Fix PackInvalidException when fetch and repack run concurrently Change-Id: I319b54ceffe095add60420c6ae83eac0ba9c14b6 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
| | * | | Merge branch 'stable-5.2' into stable-5.3Matthias Sohn2020-12-021-0/+1
| | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.2: Prepare 5.1.16-SNAPSHOT builds JGit v5.1.15.202012011955-r Fix PackInvalidException when fetch and repack run concurrently Change-Id: If1d8034b5e0cbc004a11a31b228cb5732efb390d Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
| | | * | Merge branch 'stable-5.1' into stable-5.2Matthias Sohn2020-12-021-0/+1
| | | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.1: Prepare 5.1.16-SNAPSHOT builds JGit v5.1.15.202012011955-r Fix PackInvalidException when fetch and repack run concurrently Change-Id: Id83e29e567646a3945a5b817860ea8f7c3e6e5cf Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
| | | | * Fix PackInvalidException when fetch and repack run concurrentlyPetr Hrebejk2020-12-021-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We are running several servers with jGit. We need to run repack from time to time to keep the repos performant. I.e. after push we test how many small packs are in the repo and when a threshold is reached we run the repack. After upgrading jGit version we've found that if someone does the clone at the time repack is running the clone sometimes (not always) fails because the repack removes .pack file used by the clone. Server exception and client error attached. I've tracked down the cause and it seems to be introduced between jGit 5.2 (which we upgraded from) and 5.3 and being caused by this commit: Move throw of PackInvalidException outside the catch - https://github.com/eclipse/jgit/commit/afef866a44cd65fef292c174cad445b3fb526400 The problem is that when the throw was inside of the try block the last catch block catched the exception and called openFailed(false) method. It is true that it called it with invalidate = false, which is wrong. The real problem though is that with the throw outside of the try block the openFail is not called at all and the fields activeWindows and activeCopyRawData are not set to 0. Which affects the later called tests like: if (++activeCopyRawData == 1 && activeWindows == 0). The fix for this is relatively simple keeping the throw outside of the try block and still having the invalid field set to true. I did exhaustive testing of the change running concurrent clones and pushes indefinitely and with the patch applied it never fails while without the patch it takes relatively short to get the error. See: https://www.eclipse.org/lists/jgit-dev/msg04014.html Bug: 569349 Change-Id: I9dbf8801c8d3131955ad7124f42b62095d96da54 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* | | | | Merge branch 'stable-5.4' into stable-5.5Matthias Sohn2020-11-273-7/+14
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.4: Prepare 5.3.9-SNAPSHOT builds JGit v5.3.8.202011260953-r Prepare 5.1.15-SNAPSHOT builds JGit v5.1.14.202011251942-r GC#deleteOrphans: log warning for deleted orphaned files GC#deleteOrphans: handle failure to list files in pack directory Ensure that GC#deleteOrphans respects pack lock Update API warning filters Remove unused imports Change-Id: Icc34f809a3bb019d8d640b9bdb71363e617942e2 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
| * | | | Merge branch 'stable-5.3' into stable-5.4Matthias Sohn2020-11-273-7/+14
| |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.3: Prepare 5.3.9-SNAPSHOT builds JGit v5.3.8.202011260953-r Prepare 5.1.15-SNAPSHOT builds JGit v5.1.14.202011251942-r GC#deleteOrphans: log warning for deleted orphaned files GC#deleteOrphans: handle failure to list files in pack directory Ensure that GC#deleteOrphans respects pack lock Update API warning filters Remove unused imports Change-Id: I82576ad0f61cf3ff11b54691b32666c61401ad9b Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
| | * | | Merge branch 'stable-5.2' into stable-5.3Matthias Sohn2020-11-263-7/+14
| | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.2: Prepare 5.1.15-SNAPSHOT builds JGit v5.1.14.202011251942-r GC#deleteOrphans: log warning for deleted orphaned files GC#deleteOrphans: handle failure to list files in pack directory Ensure that GC#deleteOrphans respects pack lock Update API warning filters Remove unused imports Change-Id: Id9386846a202b5ae98dd602744963f8897ddaa8c
| | | * | Merge branch 'stable-5.1' into stable-5.2Matthias Sohn2020-11-263-7/+14
| | | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.1: Prepare 5.1.15-SNAPSHOT builds JGit v5.1.14.202011251942-r GC#deleteOrphans: log warning for deleted orphaned files GC#deleteOrphans: handle failure to list files in pack directory Ensure that GC#deleteOrphans respects pack lock Update API warning filters Remove unused imports Change-Id: I91cfe2820c40d2d773cbf018cc2a6c36b062801e Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
| | | | * GC#deleteOrphans: log warning for deleted orphaned filesMatthias Sohn2020-11-262-1/+4
| | | | | | | | | | | | | | | | | | | | Change-Id: Ie245bf5c8c924dfb1f0f40b8bcdcb1e6f5815526 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
| | | | * GC#deleteOrphans: handle failure to list files in pack directoryMatthias Sohn2020-11-261-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - log an error - either there is no list or it is incomplete hence return immediately Change-Id: Ieee5378ca06304056b9ccc30c1acd5f52360052d Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
| | | | * Ensure that GC#deleteOrphans respects pack lockMatthias Sohn2020-11-261-2/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If pack or index files are guarded by a pack lock (.keep file) deleteOrphans() should not touch the respective files protected by the lock file. Otherwise it may interfere with PackInserter concurrently inserting a new pack file and its index. The problem was caused by the following race. All mentioned files are located in "objects/pack/". File endings relevant in "pack" dir: .pack .keep .idx .bitmap When ReceivePack receives a pack file it executes the following steps: ReceivePack.service(): receivePackAndCheckConnectivity(): receivePack(): receive the pack parse the pack, returns packLock (.keep file) PackInserter.flush(): write tmpPck file: "insert_<random>.pack" write tmpIdx file: "insert_<random>.idx" real pack name: "pack-<SHA1>.pack" real index name: "pack-<SHA1>.idx" atomic rename tmpPack to realPack atomic rename tmpIdx to tmpIdx execute commands unlock pack by removing .keep file trigger auto gc if enabled When PackInserter.flush() renames the temporary pack to the final "pack-xxx.pack" file the temporary pack index file "insert_xxx.idx" has no matching .pack file with the same base name for a short interval. If deleteOrphans() ran during that interval it deduced the pack index file was orphaned. Subsequently the missing pack index caused MissingObjectExceptions since objects contained in the pack couldn't be looked up anymore. Bug: https://bugs.chromium.org/p/gerrit/issues/detail?id=13544 Change-Id: I559c81e4b1d7c487f92a751bd78b987d32c98719 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
| | | | * Remove unused importsMatthias Sohn2020-07-191-2/+0
| | | | | | | | | | | | | | | | | | | | Change-Id: I7c44e3603df2dd368cb7c0ba0072413b887b6903 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* | | | | Merge branch 'stable-5.4' into stable-5.5Matthias Sohn2020-02-016-76/+428
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.4: Fix string format parameter for invalidRefAdvertisementLine WindowCache: add metric for cached bytes per repository pgm daemon: fallback to user and system config if no config specified WindowCache: add option to use strong refs to reference ByteWindows Replace usage of ArrayIndexOutOfBoundsException in treewalk Add config constants for WindowCache configuration options Change-Id: I76a62da98182f0c504b1ea8b7d37cecdf4eea7e0
| * | | | Merge branch 'stable-5.3' into stable-5.4Matthias Sohn2020-02-016-76/+428
| |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.3: Fix string format parameter for invalidRefAdvertisementLine WindowCache: add metric for cached bytes per repository pgm daemon: fallback to user and system config if no config specified WindowCache: add option to use strong refs to reference ByteWindows Replace usage of ArrayIndexOutOfBoundsException in treewalk Add config constants for WindowCache configuration options Change-Id: I12002dbfed9dff14fc6d2df9787d92eab5b1fa78
| | * | | Merge branch 'stable-5.2' into stable-5.3Matthias Sohn2020-02-016-76/+428
| | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.2: Fix string format parameter for invalidRefAdvertisementLine WindowCache: add metric for cached bytes per repository pgm daemon: fallback to user and system config if no config specified WindowCache: add option to use strong refs to reference ByteWindows Replace usage of ArrayIndexOutOfBoundsException in treewalk Add config constants for WindowCache configuration options Change-Id: I790098be00ff3f9b8278d54ae4fb7c11311816e9
| | | * | Merge branch 'stable-5.1' into stable-5.2David Pursehouse2020-02-014-55/+378
| | | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.1: Fix string format parameter for invalidRefAdvertisementLine WindowCache: add metric for cached bytes per repository pgm daemon: fallback to user and system config if no config specified WindowCache: add option to use strong refs to reference ByteWindows Change-Id: I741059a1d0d5950ab5bc16ec70352655ee926a24 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
| | | | * WindowCache: add metric for cached bytes per repositoryMatthias Sohn2020-01-282-10/+52
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Since ObjectDatabase and PackFile don't know their repository use the packfile's grand-grand-parent directory as an identifier for the repository the packfile resides in. Remove metric for a repository if the number of cached bytes for the repository drops to 0 in order to ensure the map of cached bytes per repository doesn't contain repositories which have no data cached in the WindowCache. Change-Id: I969ab8029db0a292e7585cbb36ca0baa797da20b Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
| | | | * WindowCache: add option to use strong refs to reference ByteWindowsMatthias Sohn2020-01-203-47/+328
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Java GC evicts all SoftReferences when the used heap size comes close to the maximum heap size. This means peaks in heap memory consumption can flush the complete WindowCache which was observed to have negative impact on performance of upload-pack in Gerrit. Hence add a boolean option core.packedGitUseStrongRefs to allow using strong references to reference packfile pages cached in the WindowCache. If this option is set to true Java gc can no longer flush the WindowCache to free memory if the used heap comes close to the maximum heap size. On the other hand this provides more predictable performance. Bug: 553573 Change-Id: I9de406293087ab0fa61130c8e0829775762ece8d Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
| | | * | Merge branch 'stable-5.1' into stable-5.2David Pursehouse2020-01-174-21/+50
| | | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.1: Replace usage of ArrayIndexOutOfBoundsException in treewalk Add config constants for WindowCache configuration options Change-Id: I17da3c5183eca536aa2be3972bc5df45c9d75f1b Signed-off-by: David Pursehouse <david.pursehouse@gmail.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
| | | | * Replace usage of ArrayIndexOutOfBoundsException in treewalkPatrick Hiesel2020-01-092-9/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Using exceptions during normal operations - for example with the desire of expanding an array in the failure case - can have a severe performance impact. When exceptions are instantiated, a stack trace is collected. Generating stack trace can be expensive. Compared to that, checking an array for length - even if done many times - is cheap since this is a check that can run in just a handful of CPU cycles. Change-Id: Ifaf10623f6a876c9faecfa44654c9296315adfcb Signed-off-by: Patrick Hiesel <hiesel@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
| | | | * Add config constants for WindowCache configuration optionsMatthias Sohn2020-01-092-12/+44
| | | | | | | | | | | | | | | | | | | | Change-Id: Icc5265f87ae58aa1e667ed1827075c4a30f75c32 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* | | | | Merge branch 'stable-5.4' into stable-5.5Matthias Sohn2020-01-095-34/+501
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.4: Fix API problem filters Fix unclosed resource warning in SmartOutputStream JschConfigSessionFactory: fix boxing warning SshSupport#runSshCommand: don't throw exception in finally block Don't override already managed maven-compiler-plugin version Remove unused import from CreateFileSnapshotBenchmark Remove duplicate ignore_optional_problems entry in .classpath Update maven-site-plugin used by benchmark module to 3.8.2 Add dependency to enable site generation for benchmark module Ignore warnings for generated source code in org.eclipse.jgit.benchmark Fix MBean registration Enhance WindowCache statistics Change-Id: I78902d5feecb2c09134b64ec2f3b48b2c3bab37d Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
| * | | | Merge branch 'stable-5.3' into stable-5.4Matthias Sohn2020-01-095-34/+501
| |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.3: Fix API problem filters Fix unclosed resource warning in SmartOutputStream JschConfigSessionFactory: fix boxing warning SshSupport#runSshCommand: don't throw exception in finally block Don't override already managed maven-compiler-plugin version Remove unused import from CreateFileSnapshotBenchmark Remove duplicate ignore_optional_problems entry in .classpath Update maven-site-plugin used by benchmark module to 3.8.2 Add dependency to enable site generation for benchmark module Ignore warnings for generated source code in org.eclipse.jgit.benchmark Fix MBean registration Enhance WindowCache statistics Change-Id: I1b560b36d169cfa02cc5450ad0fa0bd85f9f42d8 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
| | * | | Merge branch 'stable-5.2' into stable-5.3Matthias Sohn2020-01-095-41/+502
| | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.2: Fix API problem filters Fix unclosed resource warning in SmartOutputStream JschConfigSessionFactory: fix boxing warning SshSupport#runSshCommand: don't throw exception in finally block Don't override already managed maven-compiler-plugin version Remove unused import from CreateFileSnapshotBenchmark Remove duplicate ignore_optional_problems entry in .classpath Update maven-site-plugin used by benchmark module to 3.8.2 Add dependency to enable site generation for benchmark module Ignore warnings for generated source code in org.eclipse.jgit.benchmark Fix MBean registration Enhance WindowCache statistics Change-Id: I67a07d92718188bdf7f8a13b83e9f538ecf4b22f Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
| | | * | Merge branch 'stable-5.1' into stable-5.2Matthias Sohn2020-01-095-41/+502
| | | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.1: Fix unclosed resource warning in SmartOutputStream JschConfigSessionFactory: fix boxing warning SshSupport#runSshCommand: don't throw exception in finally block Don't override already managed maven-compiler-plugin version Remove unused import from CreateFileSnapshotBenchmark Remove duplicate ignore_optional_problems entry in .classpath Update maven-site-plugin used by benchmark module to 3.8.2 Add dependency to enable site generation for benchmark module Ignore warnings for generated source code in org.eclipse.jgit.benchmark Fix MBean registration Enhance WindowCache statistics Change-Id: Ic90aacf1ea40e13dc564d4d659e79535e86d0300 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
| | | | * JschConfigSessionFactory: fix boxing warningMatthias Sohn2020-01-061-1/+2
| | | | | | | | | | | | | | | | | | | | Change-Id: I1735033c56b444a9a7160cb7df89292a228d5b34 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
generated by cgit v1.2.3 (git 2.39.1) at 2025-04-25 19:21:32 +0000