aboutsummaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* Merge branch 'stable-6.2' into stable-6.3Matthias Sohn2023-10-1310-4/+340
|\ | | | | | | | | | | | | | | | | | | * stable-6.2: PackConfig: fix @since tags Remove unused API problem filters Add support for git config repack.packKeptObjects Do not exclude objects in locked packs from bitmap processing Change-Id: I22b89bf00dcef26b2096d25397aa9a57a745a92b
| * Merge branch 'stable-6.1' into stable-6.2Matthias Sohn2023-10-1310-4/+340
| |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.1: PackConfig: fix @since tags Remove unused API problem filters Add support for git config repack.packKeptObjects Do not exclude objects in locked packs from bitmap processing Change-Id: Ib4e4fe407dce334c7537bf278baa39db93aa2f09
| | * Merge branch 'stable-6.0' into stable-6.1Matthias Sohn2023-10-1310-4/+340
| | |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.0: PackConfig: fix @since tags Remove unused API problem filters Add support for git config repack.packKeptObjects Do not exclude objects in locked packs from bitmap processing Change-Id: I0c9c0b3c206cac03a93b30eda348177a4de35c36
| | | * Merge branch 'stable-5.13' into stable-6.0Matthias Sohn2023-10-1310-113/+296
| | | |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.13: PackConfig: fix @since tags Remove unused API problem filters Add support for git config repack.packKeptObjects Do not exclude objects in locked packs from bitmap processing Change-Id: Ifeaa4b4f0c5944d4ecd3042be429833ff72b43ed
| | | | * PackConfig: fix @since tagsMatthias Sohn2023-10-131-6/+9
| | | | | | | | | | | | | | | | | | | | Change-Id: Ia513f7cdbf3c197e8661720fc804984ff165fc5c
| | | | * Remove unused API problem filtersMatthias Sohn2023-10-132-223/+0
| | | | | | | | | | | | | | | | | | | | Change-Id: I9d5b96cf841478af8613667ef8574423630f8028
| | | | * Add support for git config repack.packKeptObjectsAntonio Barone2023-10-127-17/+191
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Change Ide3445e652 introduced the `--pack-kept-objects` option to GC for including the objects contained in the locked packfiles during the repack phase. Whilst this allowed to explicitly pass a command line argument to the jgit gc program, it did not allow the option to be read from configuration. Allow the pack kept objects option to be configured exactly as C-Git documents [1], by introducing a new `repack.packKeptObjects` configuration. `repack.packKeptObjects` defaults to `true`, when the `pack.buildBitmaps` is `true` (which is the default case), `false` otherwise. [1] https://git-scm.com/docs/git-config#Documentation/git-config.txt-repackpackKeptObjects Bug: 582292 Change-Id: Ia931667277410d71bc079d27c097a57094299840
| | | | * Do not exclude objects in locked packs from bitmap processingLuca Milanesio2023-10-127-1/+148
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Packfiles having an equivalent .keep file are associated with in-flight pushes that haven't been completed, with potentially a set of git objects not yet referenced by a ref. If the Git client is not up-to-date, it may result in pushing a packfile, generating a <packfile>.keep on the server, which may also contain existing commits due to the lack of Git protocol negotiation in the git-receive-pack. The Git protocol negotiation is the phase where the client and the server exchange the list of refs they have for trying to find a common base and minimise the amount of objects to be transferred. The repack phase in GC was previously skipping all objects that were contained in all packfiles having a <packfile>.keep file associated (aka "locked packfiles"), which did not take into consideration the fact that excluding the existing commits would have resulted in the generation of an invalid bitmap file. The code for excluding the objects in the locked packfiles was written well before the bitmap was introduced, hence could not consider a use case that did not exist at that time. However, when the bitmap was introduced, the exclusion of locked packfiles was not changed, hence creating a potential problem. The issue went unnoticed for many years because the bitmap generation was disabled when JGit noticed any locked packfiles; however, the bitmaps are enabled again since Id722e68d9f , and the the issue is now visible and is impacting the GC repack phase. Introduce the '--pack-kept-objects' option in GC for including the objects contained in the locked packfiles during the repack phase, which is not an issue because of the following: - If there are any existing commits duplicated in the packfiles they will be just considered once anyway because the repack doesn't generate duplicates in the output packfile. - If there are any new commits that do not have any ref pointing to them, they will be automatically excluded from the output repacked packfile. The same identical solution is adopted in the C implementation of git in repack.c. Because the locked packfile is not pruned, any new commits not pointed by any refs will remain in the repository and there will not be any accidental pruning or object loss as it is today before this change. As a side-effect of this change, it is now potentially possible to still have duplicate BLOBs after GC when the keep packfile contained existing objects. However, it is way better to keep the duplication until the next GC phase rather than omitting existing objects from repacking and, therefore generating an invalid bitmap and incorrect packfile. Bug: 582292 Bug: 582455 Change-Id: Ide3445e652fcf256a7912f881cb898897c99b8f8
* | | | | Merge branch 'stable-6.2' into stable-6.3Matthias Sohn2023-08-033-6/+35
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.2: Add verification in GcKeepFilesTest that bitmaps are generated Express the explicit intention of creating bitmaps in GC GC: prune all packfiles after the loosen phase Prepare 5.13.3-SNAPSHOT builds JGit v5.13.2.202306221912-r Change-Id: I589ed444b5cbfc5b073cac91323e2cc97ab98087
| * | | | Merge branch 'stable-6.1' into stable-6.2Matthias Sohn2023-08-033-6/+35
| |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.1: Add verification in GcKeepFilesTest that bitmaps are generated Express the explicit intention of creating bitmaps in GC GC: prune all packfiles after the loosen phase Prepare 5.13.3-SNAPSHOT builds JGit v5.13.2.202306221912-r Change-Id: I5b16c3b613a95b7f28c8f6ac0b20c4c593759cea
| | * | | Merge branch 'stable-6.0' into stable-6.1Matthias Sohn2023-08-033-6/+35
| | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.0: Add verification in GcKeepFilesTest that bitmaps are generated Express the explicit intention of creating bitmaps in GC GC: prune all packfiles after the loosen phase Prepare 5.13.3-SNAPSHOT builds JGit v5.13.2.202306221912-r Change-Id: Ib08037f6055dac1776e38cfb4ff8c88a50ad3e60
| | | * | Merge branch 'stable-5.13' into stable-6.0Matthias Sohn2023-08-033-6/+35
| | | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.13: Add verification in GcKeepFilesTest that bitmaps are generated Express the explicit intention of creating bitmaps in GC GC: prune all packfiles after the loosen phase Prepare 5.13.3-SNAPSHOT builds JGit v5.13.2.202306221912-r Change-Id: I1f50995d9d9c592ec0e02a04e0e409440b49f9f3
| | | | * Add verification in GcKeepFilesTest that bitmaps are generatedLuca Milanesio2023-07-051-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The packfiles with the .keep extensions are meant to prevent a packfile from being processed or removed during GC. From the point of view of the GC process then, the associated packfile should be completely transparent: - it should not included in the repacked file - it should not pruned - its objects should be left untouched, even if unreachable - the GC process, including the bitmap generation should continue as usual, as the the packfiles with .keep file did not exist Add one explicit test for making sure that the management of .keep file is also transparent to the generation of bitmaps, which are still generated if a .keep file exists. Bug: 582039 Change-Id: I14f6adc3f961c606fbc617e51ea6ed6e2ef8604f
| | | | * Express the explicit intention of creating bitmaps in GCLuca Milanesio2023-07-052-5/+28
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add an explicit flag to PackWriter for allowing the GC.repack() phase to explicitly generate bitmaps only for the heads packfile and not for the others. Previously the bitmap generation was conditioned to the presence of object ids exclusion from the PackWriter. The introduction of the bitmap generation in the PackWriter done in Icdb0cdd66 has accidentally made the .keep files not completely transparent, because their presence have disabled the generation of the bitmap index, even if the generation of bitmaps is enabled. This bug has been an accidental consequence of the intention of the bitmap generator to avoid generating bitmaps for the non-heads packfile, however the implementation done by Colby decided to use the excludeInPacks variable (see [1]) which is unfortunately also used for excluding the packfiles having an associated .keep file (see [2]). [1] https://git.eclipse.org/r/c/jgit/jgit/+/7940/18/org.eclipse.jgit/src/org/eclipse/jgit/storage/pack/PackWriter.java#1617 [2] https://git.eclipse.org/r/plugins/gitiles/jgit/jgit/+/dafcb8f6db82b899c917832768f1c240d273190c/org.eclipse.jgit/src/org/eclipse/jgit/storage/file/GC.java#506 Bug: 582039 Change-Id: Id722e68d9ff4ac24e73bf765ab11017586b6766e
| | | | * GC: prune all packfiles after the loosen phaseLuca Milanesio2023-07-051-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When loosening the objects inside the packfiles to be pruned, make sure that the packfile list is stable and prune all the files after the loosening is done. This prevents a series of exceptions previously thrown when loosening the packfiles, due to the too early pruning of the packfiles that were still in the pack list. Bug: 581532 Change-Id: I776776e2e083f1fa749d53f965bf50f919823b4f
| | | | * Prepare 5.13.3-SNAPSHOT buildsMatthias Sohn2023-06-2290-503/+561
| | | | | | | | | | | | | | | | | | | | Change-Id: I02b9388c8bc1c266bb29b4502504d137dd42142f
| | | | * JGit v5.13.2.202306221912-rv5.13.2.202306221912-rMatthias Sohn2023-06-2288-121/+121
| | | | | | | | | | | | | | | | | | | | | | | | | Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Change-Id: Id0ee779fba85a6d5557f6319969adb2c74feebcf
* | | | | Merge branch 'stable-6.2' into stable-6.3Luca Milanesio2023-06-081-16/+1
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.2: Revert "RefDirectory: Throw exception if CAS of packed ref list fails" Change-Id: I70db1bc8529eb6a66610946946da5447a578bffa Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
| * | | | Merge branch 'stable-6.1' into stable-6.2Luca Milanesio2023-06-081-16/+1
| |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.1: Revert "RefDirectory: Throw exception if CAS of packed ref list fails" Change-Id: I1a98e293ef10917b2d8ad64e88be9e82c7bcf693 Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
| | * | | Merge branch 'stable-6.0' into stable-6.1Luca Milanesio2023-06-081-16/+1
| | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.0: Revert "RefDirectory: Throw exception if CAS of packed ref list fails" Change-Id: Idc0d1f8ab4524868b7e9754799f70acc1d24f2cb Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
| | | * | Merge branch 'stable-5.13' into stable-6.0Luca Milanesio2023-06-081-16/+1
| | | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.13: Revert "RefDirectory: Throw exception if CAS of packed ref list fails" Change-Id: I883b21b00317cc6d9951a8a5f9505078ddd2a3a7 Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
| | | | * Revert "RefDirectory: Throw exception if CAS of packed ref list fails"Martin Fick2023-06-071-16/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This reverts commit 9c33f7364d41956240818ba12d8b79d5ea846162. Reason for revert: This change was based on the false claim that the packedrefs file lock is held while the CAS is being done, but it is actually released before the CAS (the in memory lock is still held, however that does not prevent external actors from updating the packedrefs files and then another thread from subsequently re-reading it and updating the in memory packedRefList). Although reverting this change can cause the CAS to fail, it should not actually matter since the failure would indicate that another thread has already updated the in memory packedRefList to either the same version this thread was trying to update it too, or to a more recent version. Either way, failing the CAS is then appropriate and should not be problematic. Although this change reverts the code in the RefDirectory class, it keeps the "improvements" to the test so that it continues to pass reliably. The reason for the quotes around the word "improvements" is because I believe the test alteration actually dramatically changes the intent of the test, and that the original intent of the test is untestable with the GC and RefDirectory classes as is. Bug: 582044 Change-Id: I3acee7527bb542996dcdfaddfb2bdb45ec444db5 Signed-off-by: Martin Fick <quic_mfick@quicinc.com> (cherry picked from commit c5617711a1b4d5d0807cc7eed702b78d114d46b3)
* | | | | Merge branch 'stable-6.2' into stable-6.3Matthias Sohn2023-04-277-16/+111
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.2: [bazel] Move ToolTestCase to src folder (6.2) GcConcurrentTest: @Ignore flaky testInterruptGc Fix CommitTemplateConfigTest Fix after_open config and Snapshotting RefDir tests to work with bazel [bazel] Skip ConfigTest#testCommitTemplatePathInHomeDirecory Demote severity of some error prone bug patterns to warnings UploadPack: Fix NPE when traversing a tag chain Change-Id: I736c7d0ed9c6e9718fa98976c3dc5a25ab8cda85
| * | | | [bazel] Move ToolTestCase to src folder (6.2)Matthias Sohn2023-04-271-0/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Bazel barks at the abstract ToolTestCase not containing any test. Move it from the tst/ source folder to the src/ source folder so that bazel knows this is a helper class which doesn't contain tests. Change-Id: Ie353c27526644ea9b47437b9bd9cd730a432ab29
| * | | | Merge branch 'stable-6.1' into stable-6.2Matthias Sohn2023-04-276-16/+111
| |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.1: GcConcurrentTest: @Ignore flaky testInterruptGc Fix CommitTemplateConfigTest Fix after_open config and Snapshotting RefDir tests to work with bazel [bazel] Skip ConfigTest#testCommitTemplatePathInHomeDirecory Demote severity of some error prone bug patterns to warnings UploadPack: Fix NPE when traversing a tag chain Change-Id: I9863cbce95d845efc891724898954b0b2f8dbf7b
| | * | | Merge branch 'stable-6.0' into stable-6.1Matthias Sohn2023-04-271-0/+2
| | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.0: GcConcurrentTest: @Ignore flaky testInterruptGc Fix CommitTemplateConfigTest Change-Id: I9c622de39200c6ee248d600736dc5db477f6bf9f
| | | * | Merge branch 'stable-5.13' into stable-6.0Matthias Sohn2023-04-271-0/+2
| | | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.13: GcConcurrentTest: @Ignore flaky testInterruptGc Fix CommitTemplateConfigTest Adapt GcConcurrentTest to changed FileRepositoryBuilder API. Change-Id: I8906662a6391ceed20ab29cfbe238daf13e1be63
| | | | * GcConcurrentTest: @Ignore flaky testInterruptGcJonathan Tan2023-04-271-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | During my development of Id7721cc5b7ea650e77c2db47042715487983cae6, I have found this test to be flaky when run by CI. As a speculative fix, mark this test as @Ignore so it won't be run. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Change-Id: Idfe04d7f1fb72a772d4c8d249ca86a9c2eec0b1a
| | | | * Fix CommitTemplateConfigTestMatthias Sohn2023-04-271-4/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The cherry-picked 61d4e313 doesn't match 5.13 APIs which changed in newer versions. Change-Id: I61ed0242472ed822028d86d3038f956f6bd5735c
| | * | | Fix after_open config and Snapshotting RefDir tests to work with bazelPrudhvi Akhil Alahari2023-04-262-0/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The changes I1db6fcf414b and I634b92877f added tests which were failing with errors [1] and [2] with "bazel test //...". This was not caught because we don't have CI running with bazel. Fix bazel build file so that these errors are no longer thrown when run with bazel. [1] error: cannot find symbol FileRepositoryBuilderTest [2] error: cannot find symbol RefDirectoryTest Bug: 581816 Signed-off-by: Prudhvi Akhil Alahari <quic_prudhvi@quicinc.com> Change-Id: I1e57111662825f5f14f373bc4f8d24cce1fec0b8
| | * | | Merge branch 'stable-6.0' into stable-6.1Matthias Sohn2023-04-263-16/+86
| | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.0: [bazel] Skip ConfigTest#testCommitTemplatePathInHomeDirecory Demote severity of some error prone bug patterns to warnings UploadPack: Fix NPE when traversing a tag chain Change-Id: I5e13d5b5414aef97e518898166bfa166c692e60f
| | | * | Merge branch 'stable-5.13' into stable-6.0Matthias Sohn2023-04-261-3/+3
| | | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.13: [bazel] Skip ConfigTest#testCommitTemplatePathInHomeDirecory Demote severity of some error prone bug patterns to warnings Change-Id: I63b9adc22e1e1d9a9c754fe070ddfe21a52446c7
| | | | * [bazel] Skip ConfigTest#testCommitTemplatePathInHomeDirecoryMatthias Sohn2023-04-263-24/+62
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Move this test to another class and skip it when running tests with bazel since the bazel test runner does not allow to create files in the home directory. FS#userHome retrieves the home directory on the first call and caches it for subsequent calls to avoid overhead in case path translation is required (currently on cygwin). This prevents that the test can mock the home directory using MockSystemReader like SshTestHarness does. Change-Id: I6a22f37f4a19eb4b4935509eae508a23e56db7aa
| | | | * Demote severity of some error prone bug patterns to warningsDavid Ostrovsky2023-04-261-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The code is not violation free, so demote the severity of these bug patterns to warning: o DefaultCharset o FutureReturnValueIgnored o UnusedException Change-Id: Ie886a4a247770a74953385f018498ac2515ed209
| | | * | UploadPack: Fix NPE when traversing a tag chainKaushik Lingarkar2023-04-212-13/+83
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Always parse RevTags including their body before getting their object to ensure that non-cached objects are handled correctly when traversing a tag chain. An NPE in UploadPack#addTagChain will occur on a depth=1 fetch of a branch containing a tag chain and the ref to one of the middle tags in the chain is deleted. Change-Id: Ifd8fe868869070b365df926fec5dcd8e64d4f521 Signed-off-by: Kaushik Lingarkar <quic_kaushikl@quicinc.com>
* | | | | Merge branch 'stable-6.2' into stable-6.3Matthias Sohn2023-04-217-6/+102
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.2: Add missing since tag for SshBasicTestBase Add missing since tag for SshTestHarness#publicKey2 Silence API errors Prevent infinite loop rescanning the pack list on PackMismatchException Remove blank in maven.config Change-Id: I8006068f16ae442a2246e043a680053f2af34e9f
| * | | | Merge branch 'stable-6.1' into stable-6.2Matthias Sohn2023-04-217-6/+102
| |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.1: Add missing since tag for SshBasicTestBase Add missing since tag for SshTestHarness#publicKey2 Silence API errors Prevent infinite loop rescanning the pack list on PackMismatchException Remove blank in maven.config Change-Id: I4c5b000b09287cc32f0e4d6a24a766ef4e17ddbe
| | * | | Merge branch 'stable-6.0' into stable-6.1Matthias Sohn2023-04-217-6/+102
| | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.0: Add missing since tag for SshBasicTestBase Add missing since tag for SshTestHarness#publicKey2 Silence API errors Prevent infinite loop rescanning the pack list on PackMismatchException Remove blank in maven.config Change-Id: Ia01c5ac5259b8820afb823d97bee247b5a5fb14a
| | | * | Merge branch 'stable-5.13' into stable-6.0Matthias Sohn2023-04-217-6/+102
| | | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.13: Add missing since tag for SshBasicTestBase Add missing since tag for SshTestHarness#publicKey2 Silence API errors Prevent infinite loop rescanning the pack list on PackMismatchException Remove blank in maven.config Change-Id: Id37bee59ca3c7947604c54b6d4e7c02628a657fe
| | | | * Merge branch 'stable-5.12' into stable-5.13Matthias Sohn2023-04-207-6/+102
| | | | |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.12: Add missing since tag for SshBasicTestBase Add missing since tag for SshTestHarness#publicKey2 Silence API errors Prevent infinite loop rescanning the pack list on PackMismatchException Remove blank in maven.config Change-Id: Ibe6652374ab5971105e62b05279f218c8c130fee
| | | | | * Merge branch 'stable-5.11' into stable-5.12stable-5.12Matthias Sohn2023-04-207-7/+89
| | | | | |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.11: Add missing since tag for SshBasicTestBase Add missing since tag for SshTestHarness#publicKey2 Silence API errors Prevent infinite loop rescanning the pack list on PackMismatchException Remove blank in maven.config Change-Id: I25bb99687b969f9915a7cbda8d1332bec778096a
| | | | | | * Add missing since tag for SshBasicTestBasestable-5.11Matthias Sohn2023-04-201-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Change-Id: Iad8ae9bb526418b279dc54a5e9d0c877c1eca475
| | | | | | * Merge branch 'stable-5.10' into stable-5.11Matthias Sohn2023-04-208-75/+87
| | | | | | |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.10: Add missing since tag for SshTestHarness#publicKey2 Silence API errors Prevent infinite loop rescanning the pack list on PackMismatchException Remove blank in maven.config Migrated "Prevent infinite loop rescanning the pack list on PackMismatchException" to refactoring done in https://git.eclipse.org/r/q/topic:restore-preserved-packs Change-Id: I0fb77bb9b498d48d5da88a93486b99bf8121e3bd
| | | | | | | * Add missing since tag for SshTestHarness#publicKey2stable-5.10Matthias Sohn2023-04-202-0/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Change-Id: Ib6e4945340d2e1761dc0e787bdbe72286cdc95bc
| | | | | | | * Silence API errorsMatthias Sohn2023-04-201-0/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Change-Id: I367c05c43df3385c33ce76bc10b2dc3bd330665c
| | | | | | | * Merge branch 'stable-5.9' into stable-5.10Matthias Sohn2023-04-204-7/+73
| | | | | | | |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.9: Prevent infinite loop rescanning the pack list on PackMismatchException Remove blank in maven.config Change-Id: I15ff2d7716ecaceb0eb87b8823d85670f5db709d
| | | | | | | | * Prevent infinite loop rescanning the pack list on PackMismatchExceptionstable-5.9Matthias Sohn2023-04-193-7/+76
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We found, when analysing an incident where Gerrit's gc runner thread got stuck, that we can end up in an infinite loop in ObjectDirectory#openPackedObject which tries to rescan the pack list and starts over trying to open a packed object in an unconfined loop if it catches a PackMismatchException. Here the relevant part of a thread dump we created while the gc runner was stuck: "WorkQueue-2[java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@350812a3[Not completed, task = java.util.concurrent.Executors$RunnableAdapter@5425d7ee]]" #72 tid=0x00007f73cee1c800 nid=0x584 runnable [0x00007f7392d57000] java.lang.Thread.State: RUNNABLE at org.eclipse.jgit.internal.storage.file.WindowCache.removeAll(WindowCache.java:716) at org.eclipse.jgit.internal.storage.file.WindowCache.purge(WindowCache.java:399) at org.eclipse.jgit.internal.storage.file.PackFile.close(PackFile.java:296) at org.eclipse.jgit.internal.storage.file.ObjectDirectory.reuseMap(ObjectDirectory.java:973) at org.eclipse.jgit.internal.storage.file.ObjectDirectory.scanPacksImpl(ObjectDirectory.java:904) at org.eclipse.jgit.internal.storage.file.ObjectDirectory.scanPacks(ObjectDirectory.java:895) - locked <0x000000050a498f60> (a java.util.concurrent.atomic.AtomicReference) at org.eclipse.jgit.internal.storage.file.ObjectDirectory.searchPacksAgain(ObjectDirectory.java:794) at org.eclipse.jgit.internal.storage.file.ObjectDirectory.openPackedObject(ObjectDirectory.java:465) at org.eclipse.jgit.internal.storage.file.ObjectDirectory.openPackedFromSelfOrAlternate(ObjectDirectory.java:417) at org.eclipse.jgit.internal.storage.file.ObjectDirectory.openObject(ObjectDirectory.java:408) at org.eclipse.jgit.internal.storage.file.WindowCursor.open(WindowCursor.java:132) at org.eclipse.jgit.lib.ObjectReader$1.open(ObjectReader.java:279) at org.eclipse.jgit.revwalk.RevWalk$2.next(RevWalk.java:1031) at org.eclipse.jgit.internal.storage.pack.PackWriter.findObjectsToPack(PackWriter.java:1911) at org.eclipse.jgit.internal.storage.pack.PackWriter.preparePack(PackWriter.java:960) at org.eclipse.jgit.internal.storage.pack.PackWriter.preparePack(PackWriter.java:876) at org.eclipse.jgit.internal.storage.file.GC.writePack(GC.java:1168) at org.eclipse.jgit.internal.storage.file.GC.repack(GC.java:852) at org.eclipse.jgit.internal.storage.file.GC.doGc(GC.java:269) at org.eclipse.jgit.internal.storage.file.GC.gc(GC.java:220) at org.eclipse.jgit.api.GarbageCollectCommand.call(GarbageCollectCommand.java:179) at com.google.gerrit.server.git.GarbageCollection.run(GarbageCollection.java:112) at com.google.gerrit.server.git.GarbageCollection.run(GarbageCollection.java:75) at com.google.gerrit.server.git.GarbageCollection.run(GarbageCollection.java:71) at com.google.gerrit.server.git.GarbageCollectionRunner.run(GarbageCollectionRunner.java:76) at com.google.gerrit.server.logging.LoggingContextAwareRunnable.run(LoggingContextAwareRunnable.java:103) at java.util.concurrent.Executors$RunnableAdapter.call(java.base@11.0.18/Executors.java:515) at java.util.concurrent.FutureTask.runAndReset(java.base@11.0.18/FutureTask.java:305) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(java.base@11.0.18/ScheduledThreadPoolExecutor.java:305) at com.google.gerrit.server.git.WorkQueue$Task.run(WorkQueue.java:612) at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.18/ThreadPoolExecutor.java:1128) at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.18/ThreadPoolExecutor.java:628) at java.lang.Thread.run(java.base@11.0.18/Thread.java:829) The code in ObjectDirectory#openPackedObject [1] apparently assumes that this is caused by a transient problem which it can resume from by retrying. We use `core.trustFolderStat = false` on this server since it uses NFS. The incident we had showed that we can enter into an infinite loop here if there is a permanent mismatch between a pack file and its corresponding pack index. I am not yet sure how this can happen. Break the infinite loop by limiting the number of attempts rescanning the pack list to 5 retries. When we exceed this threshold set the type of the PackMismatchException to permanent and rethrow it which breaks the infinite loop. Also apply the same limit in #getPackedObjectSize and #selectObjectRepresentation where we use similar retry loops. [1] https://git.eclipse.org/r/plugins/gitiles/jgit/jgit/+/011c26ff36b9e76c84fc2459e337f159c0f55a9a/org.eclipse.jgit/src/org/eclipse/jgit/internal/storage/file/ObjectDirectory.java#465 Change-Id: I20fb63bcc1fdc3a03d39b963f06a90e6f0ba73dc
| | | | | | | | * Remove blank in maven.configMatthias Sohn2023-04-191-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Maven 3.9.1 doesn't accept this whitespace. Change-Id: I0f6e3652b1e581615c370d35bc782184712ac922
* | | | | | | | | Merge branch 'stable-6.2' into stable-6.3Matthias Sohn2023-04-156-13/+138
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.2: Remove blank in maven.config DirCache: support option index.skipHash Change-Id: If0bb5f1a317ab981e6bbf5671851f124b18ab8ca
| * | | | | | | | Merge branch 'stable-6.1' into stable-6.2Matthias Sohn2023-04-156-13/+138
| |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.1: Remove blank in maven.config DirCache: support option index.skipHash Change-Id: Ief50a2ca8e5a8630627506f4d2142d62c0554615
generated by cgit v1.2.3 (git 2.39.1) at 2024-12-27 09:12:48 +0000