aboutsummaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
...
| | * | | | | | | Merge branch 'stable-6.0' into stable-6.1Matthias Sohn2023-04-156-13/+138
| | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.0: Remove blank in maven.config DirCache: support option index.skipHash Change-Id: Idf757bcab0d7a65ea63504674a681170c6db2f94
| | | * | | | | | Merge branch 'stable-5.13' into stable-6.0Matthias Sohn2023-04-156-13/+138
| | | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.13: Remove blank in maven.config DirCache: support option index.skipHash Change-Id: I0cc3033b1876c8c691c2a6876206cd71fa07d2e0
| | | | * | | | | Remove blank in maven.configMatthias Sohn2023-04-151-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Maven 3.9.1 doesn't accept this whitespace. Change-Id: I0f6e3652b1e581615c370d35bc782184712ac922
| | | | * | | | | DirCache: support option index.skipHashMatthias Sohn2023-03-285-12/+137
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Support the new option index.skipHash which was introduced in git 2.40 [1]. If it is set to true skip computing the git index checksum. This accelerates Git commands that manipulate the index, such as git add, git commit, or git status. Instead of storing the checksum, write a trailing set of bytes with value zero, indicating that the computation was skipped. Accept a skipped checksum consisting of 20 null bytes when reading the index since the option could have been set to true at the time when the index was written. [1] https://git-scm.com/docs/git-config#Documentation/git-config.txt-indexskipHash Bug: 581723 Change-Id: I28ebe44c5ca1cbcb882438665d686452a0c111b2
* | | | | | | | | Merge branch 'stable-6.2' into stable-6.3Matthias Sohn2023-03-231-2/+3
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.2: GC: Close File.lines stream Change-Id: Id93b1933a5ce1ede9eb388c9fd54a4b3749694bf
| * | | | | | | | Merge branch 'stable-6.1' into stable-6.2Matthias Sohn2023-03-231-2/+3
| |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.1: GC: Close File.lines stream Change-Id: Ia2be0b05ed860125a388b01d6c291832f08dd990
| | * | | | | | | Merge branch 'stable-6.0' into stable-6.1Matthias Sohn2023-03-231-2/+3
| | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.0: GC: Close File.lines stream Change-Id: I2f9e6da5584a40bb4b4efed0b87ae456f119d757
| | | * | | | | | Merge branch 'stable-5.13' into stable-6.0Matthias Sohn2023-03-231-2/+3
| | | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.13: GC: Close File.lines stream Change-Id: Ib473750e5a3ad3d74b0cb41f25052890f50a975c
| | | | * | | | | GC: Close File.lines streamXing Huang2023-03-231-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | From File#lines javadoc: The returned stream from File Lines encapsulates a Reader. If timely disposal of file system resources is required, the try-with-resources construct should be used to ensure that the stream's close method is invoked after the stream operations are completed. Wrap File.lines with try-with-resources. Change-Id: I82c6faa3ef1083f6c7e964f96e9540b4db18eee8 Signed-off-by: Xing Huang <xingkhuang@google.com> (cherry picked from commit 172a207945da376b6b4143305aef2af56f7c42e2)
* | | | | | | | | Merge branch 'stable-6.2' into stable-6.3Matthias Sohn2023-03-033-0/+65
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.2: Improve test coverage when core.trustPackedRefsStat set to after_open Change-Id: I02d7444b28b662eb46f3d0ba1a62d342820bed1f
| * | | | | | | | Merge branch 'stable-6.1' into stable-6.2Matthias Sohn2023-03-033-0/+65
| |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.1: Improve test coverage when core.trustPackedRefsStat set to after_open Change-Id: I63ba388ee2e174019733e63f8177792e1808a5c3
| | * | | | | | | Improve test coverage when core.trustPackedRefsStat set to after_openPrudhvi Akhil Alahari2023-03-023-0/+65
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | As of today, we don't have test coverage for RefDirectory when core.trustPackedRefsStat config is set to after_open. Thus create new test classes which set core.trustPackedRefsStat config to after_open in setup and extend RefDirectoryTest and FileRepositoryBuilderTest respectively. Change-Id: I1db6fcf414bc488106ad4c85fb934480f299c995 Signed-off-by: Prudhvi Akhil Alahari <quic_prudhvi@quicinc.com>
* | | | | | | | | Merge branch 'stable-6.2' into stable-6.3Matthias Sohn2023-02-222-10/+8
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.2: If tryLock fails to get the lock another gc has it Fix GcConcurrentTest#testInterruptGc Don't swallow IOException in GC.PidLock#lock Check if FileLock is valid before using or releasing it Change-Id: I5b6b10622b61fde3f0f10455a74ae159a0b69082
| * | | | | | | | Merge branch 'stable-6.1' into stable-6.2Matthias Sohn2023-02-222-10/+8
| |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.1: If tryLock fails to get the lock another gc has it Fix GcConcurrentTest#testInterruptGc Don't swallow IOException in GC.PidLock#lock Check if FileLock is valid before using or releasing it Change-Id: I3ffe92566cc145053bb762f612dd96bc6d542c62
| | * | | | | | | Merge branch 'stable-6.0' into stable-6.1Matthias Sohn2023-02-222-10/+8
| | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.0: If tryLock fails to get the lock another gc has it Fix GcConcurrentTest#testInterruptGc Don't swallow IOException in GC.PidLock#lock Check if FileLock is valid before using or releasing it Change-Id: Idea23e555c024557d7e39a86efe25f609400b962
| | | * | | | | | Merge branch 'stable-5.13' into stable-6.0Matthias Sohn2023-02-222-10/+8
| | | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.13: If tryLock fails to get the lock another gc has it Fix GcConcurrentTest#testInterruptGc Don't swallow IOException in GC.PidLock#lock Check if FileLock is valid before using or releasing it Change-Id: I708d0936fa86b028e4da4e7e21f332f8b48ad293
| | | | * | | | | If tryLock fails to get the lock another gc has itMatthias Sohn2023-02-221-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Change-Id: Ifd3bbcc5e0591883b774d23256949a83010ea134
| | | | * | | | | Fix GcConcurrentTest#testInterruptGcMatthias Sohn2023-02-221-5/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | With the new GC.PidLock interrupting a running GC throws a ClosedByInterruptException. Change-Id: I7ccea1ae9a43d4edfdab2fcfd1324c64cc22b38f
| | | | * | | | | Don't swallow IOException in GC.PidLock#lockMatthias Sohn2023-02-221-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This broke the test GcConcurrentTest#testInterruptGc which expects ClosedByInterruptException when the thread doing gc is interrupted. Change-Id: I89e02fc37aceeccb04c20cfc5b71cb8fa21793d6
| | | | * | | | | Check if FileLock is valid before using or releasing itMatthias Sohn2023-02-221-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Change-Id: I23ba67b61b9b03772f33a929c080c0d02b8c8652
* | | | | | | | | Merge branch 'stable-6.2' into stable-6.3Matthias Sohn2023-02-223-7/+170
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.2: Use Java 11 ProcessHandle to get pid of the current process Acquire file lock "gc.pid" before running gc Silence API errors introduced by 9424052f Change-Id: I53cf9675deac0b588048d8224216d2a7e8bd16ec
| * | | | | | | | Merge branch 'stable-6.1' into stable-6.2Matthias Sohn2023-02-223-7/+170
| |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.1: Use Java 11 ProcessHandle to get pid of the current process Acquire file lock "gc.pid" before running gc Silence API errors introduced by 9424052f Change-Id: I0562a4a224779ccf1e4cc1ff8f5a352e55ab220a
| | * | | | | | | Merge branch 'stable-6.0' into stable-6.1Matthias Sohn2023-02-223-7/+170
| | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.0: Use Java 11 ProcessHandle to get pid of the current process Acquire file lock "gc.pid" before running gc Silence API errors introduced by 9424052f Change-Id: Ib9a2419253ffcbc90874adbfdb8129fee3178210
| | | * | | | | | Use Java 11 ProcessHandle to get pid of the current processMatthias Sohn2023-02-221-11/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Change-Id: I790f218601c1d5e1b39c4101e3b2708e76b9d782
| | | * | | | | | Merge branch 'stable-5.13' into stable-6.0Matthias Sohn2023-02-223-7/+180
| | | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.13: Acquire file lock "gc.pid" before running gc Silence API errors introduced by 9424052f Change-Id: Ibb5c46cb79377d2d2cd7d4586f31c86665d2851c
| | | | * | | | | Acquire file lock "gc.pid" before running gcMatthias Sohn2023-02-213-8/+181
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Git guards gc by locking a lock file "gc.pid" before starting execution. The lock file contains the pid and hostname of the process holding the lock. Git tries to kill the process holding that lock if the lock file wasn't modified in the last 12 hours and was started from the same host. Teach JGit to acquire this lock before running gc but skip execution if another process already holds the lock. Killing the other process could be undesired if it's a long running application. If the lock file wasn't modified in the last 12 hours try to lock it and run gc if locking succeeds. Register a shutdown hook for the lock file to ensure it is cleaned up if the process is gracefully killed. Change-Id: I00b838dcbf4fb0d03863bf7a2cd86b743c6c6971
| | | | * | | | | Silence API errors introduced by 9424052fMatthias Sohn2023-02-211-0/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Change-Id: Ia9e619a8fa06648086b583c994e4b107ae06c44d
* | | | | | | | | Merge branch 'stable-6.2' into stable-6.3Matthias Sohn2023-02-206-9/+49
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.2: Fix getPackedRefs to not throw NoSuchFileException Add pack options to preserve and prune old pack files Allow to perform PackedBatchRefUpdate without locking loose refs Document option "core.sha1Implementation" introduced in 59029aec Change-Id: I765c7302ce84a6a9c28fdef29da2bfaa49477c6e
| * | | | | | | | Merge branch 'stable-6.1' into stable-6.2Matthias Sohn2023-02-166-9/+49
| |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.1: Fix getPackedRefs to not throw NoSuchFileException Add pack options to preserve and prune old pack files Allow to perform PackedBatchRefUpdate without locking loose refs Document option "core.sha1Implementation" introduced in 59029aec Change-Id: Id32683d5f506e082d39af269803bccee0280cc27
| | * | | | | | | Merge branch 'stable-6.0' into stable-6.1Matthias Sohn2023-02-165-8/+47
| | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.0: Add pack options to preserve and prune old pack files Allow to perform PackedBatchRefUpdate without locking loose refs Document option "core.sha1Implementation" introduced in 59029aec Change-Id: I876a38c2de8b7d5eaacd00e36b85599f88173221
| | | * | | | | | Merge branch 'stable-5.13' into stable-6.0Matthias Sohn2023-02-167-11/+148
| | | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.13: Add pack options to preserve and prune old pack files Allow to perform PackedBatchRefUpdate without locking loose refs Document option "core.sha1Implementation" introduced in 59029aec Change-Id: I423f410578f5bbe178832b80fef8998a5372182c
| | | | * | | | | Add pack options to preserve and prune old pack filesMatthias Sohn2023-02-114-8/+34
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add the options - pack.preserveOldPacks - pack.prunePreserved This allows to configure in git config if old packs should be preserved during gc and pruned during the next gc. The original implementation in 91132bb0 only allows to set these options using the API. Change-Id: I5b23ab4f317d12f5ccd234401419913e8263cc9a
| | | | * | | | | Allow to perform PackedBatchRefUpdate without locking loose refsSaša Živkov2023-02-032-3/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add another newBatchUpdate method in the RefDirectory where we can control if the created PackedBatchRefUpdate will lock the loose refs or not. This can be useful in cases when we run programs which have exclusive access to a Git repository and we know that locking loose refs is unnecessary and just a performance loss. Change-Id: I7d0932eb1598a3871a2281b1a049021380234df9 (cherry picked from commit cb90ed08526bd51f04e5d72e3ba3cf5bd30c11e4)
| | | | * | | | | Document option "core.sha1Implementation" introduced in 59029aecMatthias Sohn2023-02-021-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Bug: 580310 Change-Id: I10f3d6f6b5af7ab96683994c9cbd85e6c18a5084
| | * | | | | | | Fix getPackedRefs to not throw NoSuchFileExceptionPrudhvi Akhil Alahari2023-02-161-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Since Files.newInputStream is from java.nio package, it throws java.nio.file.NoSuchFileException. This was missed in the change I00da88e. Without this change, getPackedRefs fails with NoSuchFileException when there is no packed-refs file in a project. Change-Id: I93c202ddb73a0a5979af8e4d09e45f5645664b45 Signed-off-by: Prudhvi Akhil Alahari <quic_prudhvi@quicinc.com>
* | | | | | | | | Merge branch 'stable-6.2' into stable-6.3Matthias Sohn2023-02-0114-16/+300
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.2: Shortcut during git fetch for avoiding looping through all local refs FetchCommand: fix fetchSubmodules to work on a Ref to a blob Silence API warnings introduced by I466dcde6 Allow the exclusions of refs prefixes from bitmap PackWriterBitmapPreparer: do not include annotated tags in bitmap BatchingProgressMonitor: avoid int overflow when computing percentage Speedup GC listing objects referenced from reflogs FileSnapshotTest: Add more MISSING_FILE coverage Change-Id: I2ff386d9a096277360e6c7bd5535b49984620fb3
| * | | | | | | | Merge branch 'stable-6.1' into stable-6.2Matthias Sohn2023-02-0114-16/+300
| |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.1: Shortcut during git fetch for avoiding looping through all local refs FetchCommand: fix fetchSubmodules to work on a Ref to a blob Silence API warnings introduced by I466dcde6 Allow the exclusions of refs prefixes from bitmap PackWriterBitmapPreparer: do not include annotated tags in bitmap BatchingProgressMonitor: avoid int overflow when computing percentage Speedup GC listing objects referenced from reflogs FileSnapshotTest: Add more MISSING_FILE coverage Change-Id: Iff2fba026b49463016015b2fae1a42cf76ee2dbb
| | * | | | | | | Merge branch 'stable-6.0' into stable-6.1Matthias Sohn2023-02-0114-16/+300
| | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.0: Shortcut during git fetch for avoiding looping through all local refs FetchCommand: fix fetchSubmodules to work on a Ref to a blob Silence API warnings introduced by I466dcde6 Allow the exclusions of refs prefixes from bitmap PackWriterBitmapPreparer: do not include annotated tags in bitmap BatchingProgressMonitor: avoid int overflow when computing percentage Speedup GC listing objects referenced from reflogs FileSnapshotTest: Add more MISSING_FILE coverage Change-Id: Ib5055f2f3b8a313c178d6f6c7c5630285ad5a726
| | | * | | | | | Merge branch 'stable-5.13' into stable-6.0Matthias Sohn2023-02-0115-146/+260
| | | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.13: Shortcut during git fetch for avoiding looping through all local refs FetchCommand: fix fetchSubmodules to work on a Ref to a blob Silence API warnings introduced by I466dcde6 Allow the exclusions of refs prefixes from bitmap PackWriterBitmapPreparer: do not include annotated tags in bitmap BatchingProgressMonitor: avoid int overflow when computing percentage Speedup GC listing objects referenced from reflogs FileSnapshotTest: Add more MISSING_FILE coverage Change-Id: I58ad4c210a5e7e5a1ba6b22315b04211c8909950
| | | | * | | | | Shortcut during git fetch for avoiding looping through all local refsLuca Milanesio2023-02-011-5/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The FetchProcess needs to verify that all the refs received point to objects that are reachable from the local refs, which could be very expensive but is needed to avoid missing objects exceptions because of broken chains. When the local repository has a lot of refs (e.g. millions) and the client is fetching a non-commit object (e.g. refs/sequences/changes in Gerrit) the reachability check on all local refs can be very expensive compared to the time to fetch the remote ref. Example for a 2M refs repository: - fetching a single non-commit object: 50ms - checking the reachability of local refs: 30s A ref pointing to a non-commit object doesn't have any parent or successor objects, hence would never need to have a reachability check done. Skipping the askForIsComplete() altogether would save the 30s time spent in an unnecessary phase. Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com> Change-Id: I09ac66ded45cede199ba30f9e71cc1055f00941b
| | | | * | | | | FetchCommand: fix fetchSubmodules to work on a Ref to a blobMatthias Sohn2023-01-311-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | FetchCommand#fetchSubmodules assumed that FETCH_HEAD can always be parsed as a tree. This isn't true if it refers to a Ref referring to a BLOB. This is e.g. used in Gerrit for Refs like refs/sequences/changes which are used to implement sequences stored in git. Change-Id: I414f5b7d9f2184b2d7d53af1dfcd68cccb725ca4
| | | | * | | | | Silence API warnings introduced by I466dcde6Matthias Sohn2023-01-312-5/+60
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Change-Id: I510510da34d33757c2f83af8cd1e26f6206a486a
| | | | * | | | | Allow the exclusions of refs prefixes from bitmapLuca Milanesio2023-01-315-4/+82
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When running a GC.repack() against a repository with over one thousands of refs/heads and tens of millions of ObjectIds, the calculation of all bitmaps associated with all the refs would result in an unreasonable big file that would take up to several hours to compute. Test scenario: repo with 2500 heads / 10M obj Intel Xeon E5-2680 2.5GHz Before this change: 20 mins After this change and 2300 heads excluded: 10 mins (90s for bitmap) Having such a large bitmap file is also slow in the runtime processing and have negligible or even negative benefits, because the time lost in reading and decompressing the bitmap in memory would not be compensated by the time saved by using it. It is key to preserve the bitmaps for those refs that are mostly used in clone/fetch and give the ability to exlude some refs prefixes that are known to be less frequently accessed, even though they may actually be actively written. Example: Gerrit sandbox branches may even be actively used and selected automatically because its commits are very recent, however, they may bloat the bitmap, making it ineffective. A mono-repo with tens of thousands of developers may have a relatively small number of active branches where the CI/CD jobs are continuously fetching/cloning the code. However, because Gerrit allows the use of sandbox branches, the total number of refs/heads may be even tens to hundred thousands. Change-Id: I466dcde69fa008e7f7785735c977f6e150e3b644 Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
| | | | * | | | | PackWriterBitmapPreparer: do not include annotated tags in bitmapLuca Milanesio2023-01-312-0/+37
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The annotated tags should be excluded from the bitmap associated with the heads-only packfile. However, this was not happening because of the check of exclusion of the peeled object instead of the objectId to be excluded from the bitmap. Sample use-case: refs/heads/main ^ | commit1 <-- commit2 <- annotated-tag1 <- tag1 ^ | commit0 When creating a bitmap for the above commit graph, before this change all the commits are included (3 bitmaps), which is incorrect, because all commits reachable from annotated tags should not be included. The heads-only bitmap should include only commit0 and commit1 but because PackWriterBitPreparer was checking for the peeled pointer of tag1 to be excluded (commit2) which was not found in the list of tags to exclude (annotated-tag1), the commit2 was included, even if it wasn't reachable only from the head. Add an additional check for exclusion of the original objectId for allowing the exclusion of annotated tags and their pointed commits. Add one specific test associated with an annotated tag for making sure that this use-case is covered also. Example repository benchmark for measuring the improvement: # refs: 400k (2k heads, 88k tags, 310k changes) # objects: 11M (88k of them are annotate tags) # packfiles: 2.7G Before this change: GC time: 5h clone --bare time: 7 mins After this change: GC time: 20 mins clone --bare time: 3 mins Bug: 581267 Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com> Change-Id: Iff2bfc6587153001837220189a120ead9ac649dc
| | | | * | | | | BatchingProgressMonitor: avoid int overflow when computing percentageMatthias Sohn2023-01-312-3/+86
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When cloning huge repositories I observed percentage of object counts turning negative. This happened if lastWork * 100 exceeded Integer.MAX_VALUE. Change-Id: Ic5f5cf5a911a91338267aace4daba4b873ab3900
| | | | * | | | | Speedup GC listing objects referenced from reflogsMatthias Sohn2023-01-235-41/+36
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | GC needs to get a ReflogReader for all existing refs to list all objects referenced from reflogs. The existing Repository#getReflogReader method accepts the ref name and then resolves the Ref to create a ReflogReader. GC calling that for a huge number of Refs one by one is very slow. GC first gets all Refs in bulk and then calls getReflogReader for each of them. Fix this by adding another getReflogReader method to Repository which accepts a Ref directly. This speeds up running JGit gc on a mirror clone of the Gerrit repository from 15:36 min to 1:08 min. The repository used in this test had 45k refs, 275k commits and 1.2m git objects. Change-Id: I474897fdc6652923e35d461c065a29f54d9949f4
| | | | * | | | | FileSnapshotTest: Add more MISSING_FILE coverageNasser Grainawi2023-01-061-0/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add a couple tests that confirm what the docs say about isModified() and equals(MISSING_FILE) behavior. Change-Id: I6093040ba3594934c3270331405a44b2634b97c5 Signed-off-by: Nasser Grainawi <quic_nasserg@quicinc.com>
* | | | | | | | | Merge branch 'stable-6.2' into stable-6.3Matthias Sohn2023-01-135-32/+87
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.2: Cache trustFolderStat/trustPackedRefsStat value per-instance Refresh 'objects' dir and retry if a loose object is not found Change-Id: Ibc9bffab8c9ef9c39384b53c142d99878f7f3f98
| * | | | | | | | Merge branch 'stable-6.1' into stable-6.2Matthias Sohn2023-01-135-32/+87
| |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-6.1: Cache trustFolderStat/trustPackedRefsStat value per-instance Refresh 'objects' dir and retry if a loose object is not found Change-Id: I9e876f72f735f58bf02c7862a3d8e657fc46a7b9
| | * | | | | | | Cache trustFolderStat/trustPackedRefsStat value per-instanceNasser Grainawi2023-01-132-23/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Instead of re-reading the config every time the methods using these values were called, cache the config value at the time of instance construction. Caching the values improves performance for each of the method calls. These configs are set based on the filesystem storing the repository and unlikely to change while an application is running. Change-Id: I1cae26dad672dd28b766ac532a871671475652df Signed-off-by: Nasser Grainawi <quic_nasserg@quicinc.com>
generated by cgit v1.2.3 (git 2.39.1) at 2024-12-28 02:00:47 +0000