Update bitmap selection throttling to fully span active branches.
Replace the “bitmapCommitRange” parameter that was recently introduced
with two new parameters: “bitmapExcessiveBranchCount” and
“bitmapInactiveBranchAgeInDays”. If the count of branches does not
exceed “bitmapExcessiveBranchCount”, then the current algorithm is kept
for all branches.
If the branch count is excessive, then the commit time for the tip
commit for each branch is used to determine if a branch is “inactive”.
"Active" branches get full commit selection using the existing
algorithm. "Inactive" branches get fewer bitmaps near the branch tips.
Introduce a "contiguousCommitCount" parameter that always enforces that
the N most recent commits in a branch are selected for bitmaps. The
previous nextSelectionDistance() algorithm created anywhere from 1-100
contiguous bitmaps at branch tips.
For example, consider a branch with commits numbering 0-300, with 0
being the most recent commit. If the most recent 200 commits are not
merge commits and the 200th commit was the last one selected,
nextSelectionDistance() returned 100, causing commits 200-101 to be
ignored. Then a window of size 100 was evaluated, searching for merge
commits. Since no merge commits are found, the next commit (commit 0)
was selected, for a total of 1 commit in the topmost 100 commits.
If instead the 250th commit was selected, then by the same logic
commit 50 is selected. At that point nextSelectionDistance() switches to
selecting consecutive commits, so commits 0-50 in the topmost 100
commits are selected. The "contiguousCommitCount" parameter provides
more determinism by always selecting a constant number or topmost
commits.
Add an optimization to break out of the inner loop of selectCommits() if
all of the commits for the current branch have already been found.
When reusing bitmaps from an existing pack, remove unnecessary
populating and clearing of the writeBitmaps/PackBitmapIndexBuilder.
Add comments to PackWriterBitmapPreparer, rename methods and variables
for readability.
Add tests for bitmap selection with and without merge commits and with
excessive branch pruning triggered.
Note: I will follow up with an additional change that exposes the new
parameters through PackConfig.
Change-Id: I5ccbb96c8849f331c302d9f7840e05f9650c4608
Signed-off-by: Terry Parker <tparker@google.com>
vor 8 Jahren Update bitmap selection throttling to fully span active branches.
Replace the “bitmapCommitRange” parameter that was recently introduced
with two new parameters: “bitmapExcessiveBranchCount” and
“bitmapInactiveBranchAgeInDays”. If the count of branches does not
exceed “bitmapExcessiveBranchCount”, then the current algorithm is kept
for all branches.
If the branch count is excessive, then the commit time for the tip
commit for each branch is used to determine if a branch is “inactive”.
"Active" branches get full commit selection using the existing
algorithm. "Inactive" branches get fewer bitmaps near the branch tips.
Introduce a "contiguousCommitCount" parameter that always enforces that
the N most recent commits in a branch are selected for bitmaps. The
previous nextSelectionDistance() algorithm created anywhere from 1-100
contiguous bitmaps at branch tips.
For example, consider a branch with commits numbering 0-300, with 0
being the most recent commit. If the most recent 200 commits are not
merge commits and the 200th commit was the last one selected,
nextSelectionDistance() returned 100, causing commits 200-101 to be
ignored. Then a window of size 100 was evaluated, searching for merge
commits. Since no merge commits are found, the next commit (commit 0)
was selected, for a total of 1 commit in the topmost 100 commits.
If instead the 250th commit was selected, then by the same logic
commit 50 is selected. At that point nextSelectionDistance() switches to
selecting consecutive commits, so commits 0-50 in the topmost 100
commits are selected. The "contiguousCommitCount" parameter provides
more determinism by always selecting a constant number or topmost
commits.
Add an optimization to break out of the inner loop of selectCommits() if
all of the commits for the current branch have already been found.
When reusing bitmaps from an existing pack, remove unnecessary
populating and clearing of the writeBitmaps/PackBitmapIndexBuilder.
Add comments to PackWriterBitmapPreparer, rename methods and variables
for readability.
Add tests for bitmap selection with and without merge commits and with
excessive branch pruning triggered.
Note: I will follow up with an additional change that exposes the new
parameters through PackConfig.
Change-Id: I5ccbb96c8849f331c302d9f7840e05f9650c4608
Signed-off-by: Terry Parker <tparker@google.com>
vor 8 Jahren |
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126 |
- /*
- * Copyright (C) 2012, Christian Halstrick <christian.halstrick@sap.com> and others
- *
- * This program and the accompanying materials are made available under the
- * terms of the Eclipse Distribution License v. 1.0 which is available at
- * https://www.eclipse.org/org/documents/edl-v10.php.
- *
- * SPDX-License-Identifier: BSD-3-Clause
- */
-
- package org.eclipse.jgit.internal.storage.file;
-
- import java.io.IOException;
-
- import org.eclipse.jgit.internal.storage.file.GC.RepoStatistics;
- import org.eclipse.jgit.junit.LocalDiskRepositoryTestCase;
- import org.eclipse.jgit.junit.RepositoryTestCase;
- import org.eclipse.jgit.junit.TestRepository;
- import org.eclipse.jgit.junit.TestRepository.CommitBuilder;
- import org.eclipse.jgit.lib.AnyObjectId;
- import org.eclipse.jgit.revwalk.RevCommit;
- import org.eclipse.jgit.revwalk.RevWalk;
- import org.junit.After;
- import org.junit.Before;
-
- public abstract class GcTestCase extends LocalDiskRepositoryTestCase {
- protected TestRepository<FileRepository> tr;
- protected FileRepository repo;
- protected GC gc;
- protected RepoStatistics stats;
-
- @Override
- @Before
- public void setUp() throws Exception {
- super.setUp();
- repo = createWorkRepository();
- tr = new TestRepository<>(repo, new RevWalk(repo),
- mockSystemReader);
- gc = new GC(repo);
- }
-
- @Override
- @After
- public void tearDown() throws Exception {
- super.tearDown();
- }
-
- /**
- * Create a chain of commits of given depth.
- * <p>
- * Each commit contains one file named "a" containing the index of the
- * commit in the chain as its content. The created commit chain is
- * referenced from any ref.
- * <p>
- * A chain of depth = N will create 3*N objects in Gits object database. For
- * each depth level three objects are created: the commit object, the
- * top-level tree object and a blob for the content of the file "a".
- *
- * @param depth
- * the depth of the commit chain.
- * @return the commit that is the tip of the commit chain
- * @throws Exception
- */
- protected RevCommit commitChain(int depth) throws Exception {
- if (depth <= 0)
- throw new IllegalArgumentException("Chain depth must be > 0");
- CommitBuilder cb = tr.commit();
- RevCommit tip;
- do {
- --depth;
- tip = cb.add("a", "" + depth).message("" + depth).create();
- cb = cb.child();
- } while (depth > 0);
- return tip;
- }
-
- /**
- * Create a chain of commits of given depth with given number of added files
- * per commit.
- * <p>
- * Each commit contains {@code files} files as its content. The created
- * commit chain is referenced from any ref.
- * <p>
- * A chain will create {@code (2 + files) * depth} objects in Gits object
- * database. For each depth level the following objects are created: the
- * commit object, the top-level tree object and @code files} blobs for the
- * content of the file "a".
- *
- * @param depth
- * the depth of the commit chain.
- * @param width
- * number of files added per commit
- * @return the commit that is the tip of the commit chain
- * @throws Exception
- */
- protected RevCommit commitChain(int depth, int width) throws Exception {
- if (depth <= 0) {
- throw new IllegalArgumentException("Chain depth must be > 0");
- }
- if (width <= 0) {
- throw new IllegalArgumentException("Number of files per commit must be > 0");
- }
- CommitBuilder cb = tr.commit();
- RevCommit tip = null;
- do {
- --depth;
- for (int i=0; i < width; i++) {
- String id = depth + "-" + i;
- cb.add("a" + id, id).message(id);
- }
- tip = cb.create();
- cb = cb.child();
- } while (depth > 0);
- return tip;
- }
-
- protected long lastModified(AnyObjectId objectId) {
- return repo.getFS()
- .lastModifiedInstant(repo.getObjectDatabase().fileFor(objectId))
- .toEpochMilli();
- }
-
- protected static void fsTick() throws InterruptedException, IOException {
- RepositoryTestCase.fsTick(null);
- }
- }
|