]> source.dussan.org Git - jgit.git/log
jgit.git
19 months agoMerge branch 'stable-6.4' into stable-6.5 28/201428/1
Matthias Sohn [Thu, 20 Apr 2023 22:40:02 +0000 (00:40 +0200)]
Merge branch 'stable-6.4' into stable-6.5

* stable-6.4:
  Add missing since tag for SshBasicTestBase
  Add missing since tag for SshTestHarness#publicKey2
  Silence API errors
  Prevent infinite loop rescanning the pack list on
PackMismatchException
  Remove blank in maven.config

Change-Id: I89af76946014fb44bd64c20e2b01a53397768d90

19 months agoMerge branch 'stable-6.3' into stable-6.4 27/201427/1
Matthias Sohn [Thu, 20 Apr 2023 22:33:26 +0000 (00:33 +0200)]
Merge branch 'stable-6.3' into stable-6.4

* stable-6.3:
  Add missing since tag for SshBasicTestBase
  Add missing since tag for SshTestHarness#publicKey2
  Silence API errors
  Prevent infinite loop rescanning the pack list on PackMismatchException
  Remove blank in maven.config

Change-Id: I18b46be0f09535c61efabe24ab1579faa3d06ba8

19 months agoMerge branch 'stable-6.2' into stable-6.3 26/201426/1
Matthias Sohn [Thu, 20 Apr 2023 22:25:51 +0000 (00:25 +0200)]
Merge branch 'stable-6.2' into stable-6.3

* stable-6.2:
  Add missing since tag for SshBasicTestBase
  Add missing since tag for SshTestHarness#publicKey2
  Silence API errors
  Prevent infinite loop rescanning the pack list on PackMismatchException
  Remove blank in maven.config

Change-Id: I8006068f16ae442a2246e043a680053f2af34e9f

19 months agoMerge branch 'stable-6.1' into stable-6.2 25/201425/1
Matthias Sohn [Thu, 20 Apr 2023 22:19:38 +0000 (00:19 +0200)]
Merge branch 'stable-6.1' into stable-6.2

* stable-6.1:
  Add missing since tag for SshBasicTestBase
  Add missing since tag for SshTestHarness#publicKey2
  Silence API errors
  Prevent infinite loop rescanning the pack list on PackMismatchException
  Remove blank in maven.config

Change-Id: I4c5b000b09287cc32f0e4d6a24a766ef4e17ddbe

19 months agoMerge branch 'stable-6.0' into stable-6.1 24/201424/1
Matthias Sohn [Thu, 20 Apr 2023 22:11:40 +0000 (00:11 +0200)]
Merge branch 'stable-6.0' into stable-6.1

* stable-6.0:
  Add missing since tag for SshBasicTestBase
  Add missing since tag for SshTestHarness#publicKey2
  Silence API errors
  Prevent infinite loop rescanning the pack list on PackMismatchException
  Remove blank in maven.config

Change-Id: Ia01c5ac5259b8820afb823d97bee247b5a5fb14a

19 months agoMerge branch 'stable-5.13' into stable-6.0 23/201423/1
Matthias Sohn [Thu, 20 Apr 2023 14:01:33 +0000 (16:01 +0200)]
Merge branch 'stable-5.13' into stable-6.0

* stable-5.13:
  Add missing since tag for SshBasicTestBase
  Add missing since tag for SshTestHarness#publicKey2
  Silence API errors
  Prevent infinite loop rescanning the pack list on
PackMismatchException
  Remove blank in maven.config

Change-Id: Id37bee59ca3c7947604c54b6d4e7c02628a657fe

19 months agoMerge branch 'stable-5.12' into stable-5.13 16/201416/1
Matthias Sohn [Thu, 20 Apr 2023 13:40:36 +0000 (15:40 +0200)]
Merge branch 'stable-5.12' into stable-5.13

* stable-5.12:
  Add missing since tag for SshBasicTestBase
  Add missing since tag for SshTestHarness#publicKey2
  Silence API errors
  Prevent infinite loop rescanning the pack list on
PackMismatchException
  Remove blank in maven.config

Change-Id: Ibe6652374ab5971105e62b05279f218c8c130fee

19 months agoMerge branch 'stable-5.11' into stable-5.12 15/201415/1 stable-5.12
Matthias Sohn [Thu, 20 Apr 2023 13:12:01 +0000 (15:12 +0200)]
Merge branch 'stable-5.11' into stable-5.12

* stable-5.11:
  Add missing since tag for SshBasicTestBase
  Add missing since tag for SshTestHarness#publicKey2
  Silence API errors
  Prevent infinite loop rescanning the pack list on PackMismatchException
  Remove blank in maven.config

Change-Id: I25bb99687b969f9915a7cbda8d1332bec778096a

19 months agoAdd missing since tag for SshBasicTestBase 14/201414/2 stable-5.11
Matthias Sohn [Thu, 20 Apr 2023 12:46:05 +0000 (14:46 +0200)]
Add missing since tag for SshBasicTestBase

Change-Id: Iad8ae9bb526418b279dc54a5e9d0c877c1eca475

19 months agoMerge branch 'stable-5.10' into stable-5.11 13/201413/2
Matthias Sohn [Thu, 20 Apr 2023 12:42:56 +0000 (14:42 +0200)]
Merge branch 'stable-5.10' into stable-5.11

* stable-5.10:
  Add missing since tag for SshTestHarness#publicKey2
  Silence API errors
  Prevent infinite loop rescanning the pack list on
PackMismatchException
  Remove blank in maven.config

Migrated "Prevent infinite loop rescanning the pack list on
PackMismatchException" to refactoring done in
https://git.eclipse.org/r/q/topic:restore-preserved-packs

Change-Id: I0fb77bb9b498d48d5da88a93486b99bf8121e3bd

19 months agoAdd missing since tag for SshTestHarness#publicKey2 12/201412/1 stable-5.10
Matthias Sohn [Thu, 20 Apr 2023 12:35:58 +0000 (14:35 +0200)]
Add missing since tag for SshTestHarness#publicKey2

Change-Id: Ib6e4945340d2e1761dc0e787bdbe72286cdc95bc

19 months agoSilence API errors 11/201411/1
Matthias Sohn [Thu, 20 Apr 2023 12:37:46 +0000 (14:37 +0200)]
Silence API errors

Change-Id: I367c05c43df3385c33ce76bc10b2dc3bd330665c

19 months agoMerge branch 'stable-5.9' into stable-5.10 10/201410/2
Matthias Sohn [Thu, 20 Apr 2023 07:52:30 +0000 (09:52 +0200)]
Merge branch 'stable-5.9' into stable-5.10

* stable-5.9:
  Prevent infinite loop rescanning the pack list on
PackMismatchException
  Remove blank in maven.config

Change-Id: I15ff2d7716ecaceb0eb87b8823d85670f5db709d

19 months agoPrevent infinite loop rescanning the pack list on PackMismatchException 78/200978/6 stable-5.9
Matthias Sohn [Thu, 30 Mar 2023 11:43:17 +0000 (13:43 +0200)]
Prevent infinite loop rescanning the pack list on PackMismatchException

We found, when analysing an incident where Gerrit's gc runner thread got
stuck, that we can end up in an infinite loop in
ObjectDirectory#openPackedObject which tries to rescan the pack
list and starts over trying to open a packed object in an unconfined
loop if it catches a PackMismatchException.

Here the relevant part of a thread dump we created while the gc runner
was stuck:

"WorkQueue-2[java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@350812a3[Not
completed,
task = java.util.concurrent.Executors$RunnableAdapter@5425d7ee]]" #72
tid=0x00007f73cee1c800 nid=0x584
runnable  [0x00007f7392d57000]
   java.lang.Thread.State: RUNNABLE
at org.eclipse.jgit.internal.storage.file.WindowCache.removeAll(WindowCache.java:716)
at org.eclipse.jgit.internal.storage.file.WindowCache.purge(WindowCache.java:399)
at org.eclipse.jgit.internal.storage.file.PackFile.close(PackFile.java:296)
at org.eclipse.jgit.internal.storage.file.ObjectDirectory.reuseMap(ObjectDirectory.java:973)
at org.eclipse.jgit.internal.storage.file.ObjectDirectory.scanPacksImpl(ObjectDirectory.java:904)
at org.eclipse.jgit.internal.storage.file.ObjectDirectory.scanPacks(ObjectDirectory.java:895)
- locked <0x000000050a498f60> (a
java.util.concurrent.atomic.AtomicReference)
at org.eclipse.jgit.internal.storage.file.ObjectDirectory.searchPacksAgain(ObjectDirectory.java:794)
at org.eclipse.jgit.internal.storage.file.ObjectDirectory.openPackedObject(ObjectDirectory.java:465)
at org.eclipse.jgit.internal.storage.file.ObjectDirectory.openPackedFromSelfOrAlternate(ObjectDirectory.java:417)
at org.eclipse.jgit.internal.storage.file.ObjectDirectory.openObject(ObjectDirectory.java:408)
at org.eclipse.jgit.internal.storage.file.WindowCursor.open(WindowCursor.java:132)
at org.eclipse.jgit.lib.ObjectReader$1.open(ObjectReader.java:279)
at org.eclipse.jgit.revwalk.RevWalk$2.next(RevWalk.java:1031)
at org.eclipse.jgit.internal.storage.pack.PackWriter.findObjectsToPack(PackWriter.java:1911)
at org.eclipse.jgit.internal.storage.pack.PackWriter.preparePack(PackWriter.java:960)
at org.eclipse.jgit.internal.storage.pack.PackWriter.preparePack(PackWriter.java:876)
at org.eclipse.jgit.internal.storage.file.GC.writePack(GC.java:1168)
at org.eclipse.jgit.internal.storage.file.GC.repack(GC.java:852)
at org.eclipse.jgit.internal.storage.file.GC.doGc(GC.java:269)
at org.eclipse.jgit.internal.storage.file.GC.gc(GC.java:220)
at org.eclipse.jgit.api.GarbageCollectCommand.call(GarbageCollectCommand.java:179)
at com.google.gerrit.server.git.GarbageCollection.run(GarbageCollection.java:112)
at com.google.gerrit.server.git.GarbageCollection.run(GarbageCollection.java:75)
at com.google.gerrit.server.git.GarbageCollection.run(GarbageCollection.java:71)
at com.google.gerrit.server.git.GarbageCollectionRunner.run(GarbageCollectionRunner.java:76)
at com.google.gerrit.server.logging.LoggingContextAwareRunnable.run(LoggingContextAwareRunnable.java:103)
at java.util.concurrent.Executors$RunnableAdapter.call(java.base@11.0.18/Executors.java:515)
at java.util.concurrent.FutureTask.runAndReset(java.base@11.0.18/FutureTask.java:305)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(java.base@11.0.18/ScheduledThreadPoolExecutor.java:305)
at com.google.gerrit.server.git.WorkQueue$Task.run(WorkQueue.java:612)
at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.18/ThreadPoolExecutor.java:1128)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.18/ThreadPoolExecutor.java:628)
at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)

The code in ObjectDirectory#openPackedObject [1] apparently assumes that
this is caused by a transient problem which it can resume from by
retrying. We use `core.trustFolderStat = false` on this server since it
uses NFS. The incident we had showed that we can enter into an infinite
loop here if there is a permanent mismatch between a pack file and its
corresponding pack index. I am not yet sure how this can happen.

Break the infinite loop by limiting the number of attempts rescanning
the pack list to 5 retries.  When we exceed this threshold set the type
of the PackMismatchException to permanent and rethrow it which breaks
the infinite loop.

Also apply the same limit in #getPackedObjectSize
and #selectObjectRepresentation where we use similar retry loops.

[1] https://git.eclipse.org/r/plugins/gitiles/jgit/jgit/+/011c26ff36b9e76c84fc2459e337f159c0f55a9a/org.eclipse.jgit/src/org/eclipse/jgit/internal/storage/file/ObjectDirectory.java#465

Change-Id: I20fb63bcc1fdc3a03d39b963f06a90e6f0ba73dc

19 months agoRemove blank in maven.config 03/201403/1
Matthias Sohn [Fri, 14 Apr 2023 16:13:18 +0000 (18:13 +0200)]
Remove blank in maven.config

Maven 3.9.1 doesn't accept this whitespace.

Change-Id: I0f6e3652b1e581615c370d35bc782184712ac922

19 months agoMerge branch 'stable-6.4' into stable-6.5 46/201346/1
Matthias Sohn [Sat, 15 Apr 2023 19:39:03 +0000 (21:39 +0200)]
Merge branch 'stable-6.4' into stable-6.5

* stable-6.4:
  Remove blank in maven.config
  DirCache: support option index.skipHash

Change-Id: I7f822e8a751516a32afccd180cbf6afb389f3a28

19 months agoMerge branch 'stable-6.3' into stable-6.4 45/201345/1
Matthias Sohn [Sat, 15 Apr 2023 19:38:27 +0000 (21:38 +0200)]
Merge branch 'stable-6.3' into stable-6.4

* stable-6.3:
  Remove blank in maven.config
  DirCache: support option index.skipHash

Change-Id: I18cf0da3a5dcc74865c44d82e7c328329814acae

19 months agoMerge branch 'stable-6.2' into stable-6.3 44/201344/1
Matthias Sohn [Sat, 15 Apr 2023 19:35:16 +0000 (21:35 +0200)]
Merge branch 'stable-6.2' into stable-6.3

* stable-6.2:
  Remove blank in maven.config
  DirCache: support option index.skipHash

Change-Id: If0bb5f1a317ab981e6bbf5671851f124b18ab8ca

19 months agoMerge branch 'stable-6.1' into stable-6.2 43/201343/1
Matthias Sohn [Sat, 15 Apr 2023 19:34:41 +0000 (21:34 +0200)]
Merge branch 'stable-6.1' into stable-6.2

* stable-6.1:
  Remove blank in maven.config
  DirCache: support option index.skipHash

Change-Id: Ief50a2ca8e5a8630627506f4d2142d62c0554615

19 months agoMerge branch 'stable-6.0' into stable-6.1 42/201342/1
Matthias Sohn [Fri, 14 Apr 2023 22:49:59 +0000 (00:49 +0200)]
Merge branch 'stable-6.0' into stable-6.1

* stable-6.0:
  Remove blank in maven.config
  DirCache: support option index.skipHash

Change-Id: Idf757bcab0d7a65ea63504674a681170c6db2f94

19 months agoMerge branch 'stable-5.13' into stable-6.0 36/201336/1
Matthias Sohn [Fri, 14 Apr 2023 22:49:08 +0000 (00:49 +0200)]
Merge branch 'stable-5.13' into stable-6.0

* stable-5.13:
  Remove blank in maven.config
  DirCache: support option index.skipHash

Change-Id: I0cc3033b1876c8c691c2a6876206cd71fa07d2e0

19 months agoRemove blank in maven.config 35/201335/1
Matthias Sohn [Fri, 14 Apr 2023 16:13:18 +0000 (18:13 +0200)]
Remove blank in maven.config

Maven 3.9.1 doesn't accept this whitespace.

Change-Id: I0f6e3652b1e581615c370d35bc782184712ac922

19 months agoEnsure parsed RevCommitCG has derived data from commit-graph 42/201042/3
kylezhao [Mon, 3 Apr 2023 09:04:38 +0000 (17:04 +0800)]
Ensure parsed RevCommitCG has derived data from commit-graph

If a RevCommitCG was newly created and called #parseCanonical(RevWalk,
byte[]) method immediately, its flag was marked as PARSED, but no
derived data was obtained from the commit-graph. This is different from
what we expected.

Change-Id: I5d417efa3c42d211f19e6acf255f761e84d84450
Signed-off-by: kylezhao <kylezhao@tencent.com>
19 months agoDowngrade maven-site-plugin to 3.12.1 90/200990/1
Matthias Sohn [Thu, 30 Mar 2023 20:00:52 +0000 (22:00 +0200)]
Downgrade maven-site-plugin to 3.12.1

This may fix the current authentication failures happening when trying
to deploy the jgit Maven site.

Change-Id: I55d4706cd041d93194af48fac9e8bfcd067e2cac

19 months agoUse wagon-ssh-external to deploy Maven site 73/200973/2
Matthias Sohn [Thu, 30 Mar 2023 07:26:26 +0000 (09:26 +0200)]
Use wagon-ssh-external to deploy Maven site

Using wagon-ssh stopped working. Try to use wagon-ssh-external instead.

Bug: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/issues/2818
Change-Id: I6f8fa771ddf9623b2e528f23f2ebdc871372ba2f

19 months agoDirCache: support option index.skipHash 16/200916/2
Matthias Sohn [Mon, 27 Mar 2023 20:23:11 +0000 (22:23 +0200)]
DirCache: support option index.skipHash

Support the new option index.skipHash which was introduced in git 2.40
[1]. If it is set to true skip computing the git index checksum. This
accelerates Git commands that manipulate the index, such as git add, git
commit, or git status. Instead of storing the checksum, write a trailing
set of bytes with value zero, indicating that the computation was
skipped.

Accept a skipped checksum consisting of 20 null bytes when reading the
index since the option could have been set to true at the time when the
index was written.

[1] https://git-scm.com/docs/git-config#Documentation/git-config.txt-indexskipHash

Bug: 581723
Change-Id: I28ebe44c5ca1cbcb882438665d686452a0c111b2

19 months agoEnsure FileCommitGraph scans commit-graph file if it already exists 71/200871/5
kylezhao [Mon, 27 Mar 2023 06:48:31 +0000 (14:48 +0800)]
Ensure FileCommitGraph scans commit-graph file if it already exists

When commit-graph file already exists in the repository, a newly
created FileCommitGraph didn't scan CommitGraph until the file was
modified, resulting in wrong result.

Change-Id: Ic85676f2d3b6a88f3ae28d4065729926b6fb2f23
Signed-off-by: kylezhao <kylezhao@tencent.com>
20 months agoMerge branch 'stable-6.4' into stable-6.5 95/200795/1
Matthias Sohn [Thu, 23 Mar 2023 08:07:33 +0000 (09:07 +0100)]
Merge branch 'stable-6.4' into stable-6.5

* stable-6.4:
  GC: Close File.lines stream

Change-Id: I7e3a4b3671e779fd62062c4e10d224f432e39b54

20 months agoMerge branch 'stable-6.3' into stable-6.4 94/200794/1
Matthias Sohn [Thu, 23 Mar 2023 08:07:09 +0000 (09:07 +0100)]
Merge branch 'stable-6.3' into stable-6.4

* stable-6.3:
  GC: Close File.lines stream

Change-Id: I99455916d447f5dffed85e9a5c1d51b323f07a16

20 months agoMerge branch 'stable-6.2' into stable-6.3 93/200793/1
Matthias Sohn [Thu, 23 Mar 2023 08:06:43 +0000 (09:06 +0100)]
Merge branch 'stable-6.2' into stable-6.3

* stable-6.2:
  GC: Close File.lines stream

Change-Id: Id93b1933a5ce1ede9eb388c9fd54a4b3749694bf

20 months agoMerge branch 'stable-6.1' into stable-6.2 92/200792/1
Matthias Sohn [Thu, 23 Mar 2023 08:06:16 +0000 (09:06 +0100)]
Merge branch 'stable-6.1' into stable-6.2

* stable-6.1:
  GC: Close File.lines stream

Change-Id: Ia2be0b05ed860125a388b01d6c291832f08dd990

20 months agoMerge branch 'stable-6.0' into stable-6.1 91/200791/1
Matthias Sohn [Thu, 23 Mar 2023 08:05:42 +0000 (09:05 +0100)]
Merge branch 'stable-6.0' into stable-6.1

* stable-6.0:
  GC: Close File.lines stream

Change-Id: I2f9e6da5584a40bb4b4efed0b87ae456f119d757

20 months agoMerge branch 'stable-5.13' into stable-6.0 90/200790/1
Matthias Sohn [Thu, 23 Mar 2023 08:04:50 +0000 (09:04 +0100)]
Merge branch 'stable-5.13' into stable-6.0

* stable-5.13:
  GC: Close File.lines stream

Change-Id: Ib473750e5a3ad3d74b0cb41f25052890f50a975c

20 months agoGC: Close File.lines stream 89/200789/2
Xing Huang [Tue, 21 Mar 2023 22:27:49 +0000 (17:27 -0500)]
GC: Close File.lines stream

From File#lines javadoc: The returned stream from File Lines
encapsulates a Reader. If timely disposal of file system resources is
required, the try-with-resources construct should be used to ensure
that the stream's close method is
invoked after the stream operations are completed.

Wrap File.lines with try-with-resources.

Change-Id: I82c6faa3ef1083f6c7e964f96e9540b4db18eee8
Signed-off-by: Xing Huang <xingkhuang@google.com>
(cherry picked from commit 172a207945da376b6b4143305aef2af56f7c42e2)

20 months agoRerun flaky tests 3 times 88/200388/1
Matthias Sohn [Tue, 7 Mar 2023 15:37:45 +0000 (16:37 +0100)]
Rerun flaky tests 3 times

This can help to reduce time spent on rebuilds caused by flaky tests.

Change-Id: I8f7831e470fd8aa065f0b8728f62f39734d05016

20 months agoPrepare 6.5.1-SNAPSHOT builds 87/200387/1
Matthias Sohn [Tue, 7 Mar 2023 15:38:44 +0000 (16:38 +0100)]
Prepare 6.5.1-SNAPSHOT builds

Change-Id: Idd9977ac08a339906e33beb73f57f8f6885ad86f

20 months agoJGit v6.5.0.202303070854-r 81/200381/1 v6.5.0.202303070854-r
Matthias Sohn [Tue, 7 Mar 2023 13:54:32 +0000 (14:54 +0100)]
JGit v6.5.0.202303070854-r

Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Change-Id: I8da37ead0bd527bc4990ed5f8d5d4fb4f4d5cf01

20 months agoIgnore generated org.eclipse.jgit.benchmarks/dependency-reduced-pom.xml 80/200380/1
Matthias Sohn [Tue, 7 Mar 2023 13:53:57 +0000 (14:53 +0100)]
Ignore generated org.eclipse.jgit.benchmarks/dependency-reduced-pom.xml

Change-Id: I8afe0e5128a63e0df02c5350772c942912408171

20 months ago[sshd] Fix calculation of timeout in AbstractClientProxyConnector 58/200358/4
Stephan Wahlbrink [Mon, 6 Mar 2023 09:33:24 +0000 (10:33 +0100)]
[sshd] Fix calculation of timeout in AbstractClientProxyConnector

The previous implementation mixed nano seconds (elapsed) and milli
seconds (remaining) without conversion.

Change-Id: I9e1654afa47fa32c94808af3b2dd0418a372fb00

20 months agoSilence API error raised for removed BranchRebaseMode#PRESERVE 70/200370/1
Matthias Sohn [Mon, 6 Mar 2023 20:39:42 +0000 (21:39 +0100)]
Silence API error raised for removed BranchRebaseMode#PRESERVE

It was replaced by MERGES to match C git which did that in 2.34.

Change-Id: Ib6a33b4a3650345bf0f9d3726dd9e14c5797e836

20 months agoMerge branch 'stable-6.4' into stable-6.5 20/200320/1
Matthias Sohn [Fri, 3 Mar 2023 15:03:26 +0000 (16:03 +0100)]
Merge branch 'stable-6.4' into stable-6.5

* stable-6.4:
  Improve test coverage when core.trustPackedRefsStat set to after_open

Change-Id: Id0a3b571aa6bbf159b225794ddc884f79431c670

20 months agoMerge branch 'stable-6.3' into stable-6.4 19/200319/1
Matthias Sohn [Fri, 3 Mar 2023 15:02:49 +0000 (16:02 +0100)]
Merge branch 'stable-6.3' into stable-6.4

* stable-6.3:
  Improve test coverage when core.trustPackedRefsStat set to after_open

Change-Id: I236c5cbb638493cc4b23475be50d0f724d62a688

20 months agoMerge branch 'stable-6.2' into stable-6.3 18/200318/1
Matthias Sohn [Fri, 3 Mar 2023 15:02:16 +0000 (16:02 +0100)]
Merge branch 'stable-6.2' into stable-6.3

* stable-6.2:
  Improve test coverage when core.trustPackedRefsStat set to after_open

Change-Id: I02d7444b28b662eb46f3d0ba1a62d342820bed1f

20 months agoMerge branch 'stable-6.1' into stable-6.2 17/200317/1
Matthias Sohn [Fri, 3 Mar 2023 15:01:26 +0000 (16:01 +0100)]
Merge branch 'stable-6.1' into stable-6.2

* stable-6.1:
  Improve test coverage when core.trustPackedRefsStat set to after_open

Change-Id: I63ba388ee2e174019733e63f8177792e1808a5c3

20 months ago[errorprone] Suppress [Finally] warnings 99/200299/2
Matthias Sohn [Thu, 2 Mar 2023 09:43:10 +0000 (10:43 +0100)]
[errorprone] Suppress [Finally] warnings

In these cases we use Throwable#addSuppressed to ensure the exception
thrown in the catch block preceding the finally block throwing another
exception isn't lost.

Change-Id: I96e78a5c15238ab77ac90ca1901850ce19acfcd8

20 months agoUpdate Orbit to R20230302014618 for 2023-03 97/200297/2
Matthias Sohn [Thu, 2 Mar 2023 09:14:50 +0000 (10:14 +0100)]
Update Orbit to R20230302014618 for 2023-03

and update
- com.google.gson to 2.10.1.v20230109-0753

Change-Id: I9745797a278c43ec62a82b5620bb72f075a23c1f

20 months agoImprove test coverage when core.trustPackedRefsStat set to after_open 04/200304/1
Prudhvi Akhil Alahari [Thu, 2 Mar 2023 11:21:48 +0000 (16:51 +0530)]
Improve test coverage when core.trustPackedRefsStat set to after_open

As of today, we don't have test coverage for RefDirectory when
core.trustPackedRefsStat config is set to after_open. Thus create new
test classes which set core.trustPackedRefsStat config to after_open in
setup and extend RefDirectoryTest and FileRepositoryBuilderTest
respectively.

Change-Id: I1db6fcf414bc488106ad4c85fb934480f299c995
Signed-off-by: Prudhvi Akhil Alahari <quic_prudhvi@quicinc.com>
20 months agoPrepare 6.5.0-SNAPSHOT builds 73/200273/1
Matthias Sohn [Wed, 1 Mar 2023 14:30:29 +0000 (15:30 +0100)]
Prepare 6.5.0-SNAPSHOT builds

Change-Id: I313e3deed8fa00df0406b3d7b73e5b643dc25a05

20 months agoJGit v6.5.0.202302281825-rc1 54/200254/1 v6.5.0.202302281825-rc1
Matthias Sohn [Tue, 28 Feb 2023 23:23:58 +0000 (00:23 +0100)]
JGit v6.5.0.202302281825-rc1

Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Change-Id: I1eb2e87b70c2da1dc81468cdc7ecf7dbd21d4190

20 months agoMerge branch 'master' into stable-6.5 51/200251/1
Matthias Sohn [Tue, 28 Feb 2023 23:11:41 +0000 (00:11 +0100)]
Merge branch 'master' into stable-6.5

* master:
  Change config pull.rebase=preserve to pull.rebase=merges
  BatchingProgressMonitor: expose time spent per task
  PackWriter: offer to write an object-size index for the pack
  Fix formatting in GC#doGc
  PackExt: Define new extension for the object size index

Change-Id: I174db870e6c2fcb72ff535b39bb6e1c3358f11b8

20 months agoPrepare 6.5.0-SNAPSHOT builds 50/200250/1
Matthias Sohn [Tue, 28 Feb 2023 23:10:18 +0000 (00:10 +0100)]
Prepare 6.5.0-SNAPSHOT builds

Change-Id: Ie95ce8fcc2a570505d23d5a1642e33c041212260

20 months agoChange config pull.rebase=preserve to pull.rebase=merges 79/199379/6
Pavel Salamon [Thu, 12 Jan 2023 15:39:07 +0000 (16:39 +0100)]
Change config pull.rebase=preserve to pull.rebase=merges

The native git option to preserve merge commits during rebase
has been changed from pull.rebase=preserve to pull.rebase=merges.

This changeset in jgit makes the same config change. The old "preserve"
option is no longer recognized and is replaced by new option called
"merges".

This makes jgit's rebase configuration compatible with native git
versions 2.34 and newer where the old "preserve" option has been
removed.

Change-Id: Ic07ff954e258115e76465a1593ef3259f4c418a3

20 months agoBatchingProgressMonitor: expose time spent per task 82/199682/7
Matthias Sohn [Thu, 19 Jan 2023 00:46:50 +0000 (01:46 +0100)]
BatchingProgressMonitor: expose time spent per task

Display elapsed time per task if enabled via
ProgressMonitor#showDuration or if system property or environment
variable GIT_TRACE_PERFORMANCE is set to "true". If both the system
property and the environment variable are set the system property takes
precedence.

E.g. using jgit CLI:

$ GIT_TRACE_PERFORMANCE=true jgit clone https://foo.bar/foobar
Cloning into 'foobar'...
remote: Counting objects: 1 [0.002s]
remote: Finding sources: 100% (15531/15531) [0.006s]
Receiving objects:      100% (169737/169737) [13.045s]
Resolving deltas:       100% (67579/67579) [1.842s]

Change-Id: I4d624e7858b286aeddbe7d4e557589986d73659e

20 months agoPackWriter: offer to write an object-size index for the pack 89/191089/25
Ivan Frade [Tue, 28 Dec 2021 22:23:40 +0000 (14:23 -0800)]
PackWriter: offer to write an object-size index for the pack

PackWriter callers tell the writer what do the want to include in the
pack and invoke #writePack(). Afterwards, they can invoke #writeIndex()
to write the corresponding pack index.

Mirror this for the object-size index, adding a #writeObjectSizeIndex()
method.

Change-Id: Ic319975c72c239cd6488303f7d4cced797e6fe00

20 months agoFix formatting in GC#doGc 65/200165/1
Matthias Sohn [Fri, 24 Feb 2023 14:18:39 +0000 (15:18 +0100)]
Fix formatting in GC#doGc

Change-Id: Ifa3adb66d4e0404bab4036d6b165d6c4dafe921a

20 months agoPackExt: Define new extension for the object size index 85/191085/19
Ivan Frade [Tue, 4 Jan 2022 20:30:10 +0000 (12:30 -0800)]
PackExt: Define new extension for the object size index

Change-Id: I6bbaf43b4e6fb456ca0e9e0c6efcfeded0f94d6d

21 months agoJGit v6.5.0.202302221508-m3 30/200130/1 v6.5.0.202302221508-m3
Matthias Sohn [Wed, 22 Feb 2023 20:09:10 +0000 (21:09 +0100)]
JGit v6.5.0.202302221508-m3

Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Change-Id: I00bf91823e2fe6503d52ae23bc1854f0b1d66dbd

21 months agoMerge branch 'master' into stable-6.5 28/200128/1
Matthias Sohn [Wed, 22 Feb 2023 20:07:34 +0000 (21:07 +0100)]
Merge branch 'master' into stable-6.5

* master:
  If tryLock fails to get the lock another gc has it
  Fix GcConcurrentTest#testInterruptGc
  Don't swallow IOException in GC.PidLock#lock
  Check if FileLock is valid before using or releasing it

Change-Id: I0a599ec9241dd3b9b9a61122b2eda6bd647bcc22

21 months agoMerge branch 'stable-6.4' 27/200127/1
Matthias Sohn [Wed, 22 Feb 2023 20:06:41 +0000 (21:06 +0100)]
Merge branch 'stable-6.4'

* stable-6.4:
  If tryLock fails to get the lock another gc has it
  Fix GcConcurrentTest#testInterruptGc
  Don't swallow IOException in GC.PidLock#lock
  Check if FileLock is valid before using or releasing it

Change-Id: Ia2797b44a60342eb9df53f0b3d674cba92a512fc

21 months agoMerge branch 'stable-6.3' into stable-6.4 25/200125/2
Matthias Sohn [Wed, 22 Feb 2023 20:04:31 +0000 (21:04 +0100)]
Merge branch 'stable-6.3' into stable-6.4

* stable-6.3:
  If tryLock fails to get the lock another gc has it
  Fix GcConcurrentTest#testInterruptGc
  Don't swallow IOException in GC.PidLock#lock
  Check if FileLock is valid before using or releasing it

Change-Id: I5af34c92e423a651db53b4dc45ed844d5f39910d

21 months agoMerge branch 'stable-6.2' into stable-6.3 24/200124/1
Matthias Sohn [Wed, 22 Feb 2023 20:03:52 +0000 (21:03 +0100)]
Merge branch 'stable-6.2' into stable-6.3

* stable-6.2:
  If tryLock fails to get the lock another gc has it
  Fix GcConcurrentTest#testInterruptGc
  Don't swallow IOException in GC.PidLock#lock
  Check if FileLock is valid before using or releasing it

Change-Id: I5b6b10622b61fde3f0f10455a74ae159a0b69082

21 months agoMerge branch 'stable-6.1' into stable-6.2 23/200123/1
Matthias Sohn [Wed, 22 Feb 2023 20:03:22 +0000 (21:03 +0100)]
Merge branch 'stable-6.1' into stable-6.2

* stable-6.1:
  If tryLock fails to get the lock another gc has it
  Fix GcConcurrentTest#testInterruptGc
  Don't swallow IOException in GC.PidLock#lock
  Check if FileLock is valid before using or releasing it

Change-Id: I3ffe92566cc145053bb762f612dd96bc6d542c62

21 months agoMerge branch 'stable-6.0' into stable-6.1 22/200122/1
Matthias Sohn [Wed, 22 Feb 2023 20:02:47 +0000 (21:02 +0100)]
Merge branch 'stable-6.0' into stable-6.1

* stable-6.0:
  If tryLock fails to get the lock another gc has it
  Fix GcConcurrentTest#testInterruptGc
  Don't swallow IOException in GC.PidLock#lock
  Check if FileLock is valid before using or releasing it

Change-Id: Idea23e555c024557d7e39a86efe25f609400b962

21 months agoMerge branch 'stable-5.13' into stable-6.0 21/200121/1
Matthias Sohn [Wed, 22 Feb 2023 20:02:09 +0000 (21:02 +0100)]
Merge branch 'stable-5.13' into stable-6.0

* stable-5.13:
  If tryLock fails to get the lock another gc has it
  Fix GcConcurrentTest#testInterruptGc
  Don't swallow IOException in GC.PidLock#lock
  Check if FileLock is valid before using or releasing it

Change-Id: I708d0936fa86b028e4da4e7e21f332f8b48ad293

21 months agoIf tryLock fails to get the lock another gc has it 19/200119/2
Matthias Sohn [Wed, 22 Feb 2023 19:36:39 +0000 (20:36 +0100)]
If tryLock fails to get the lock another gc has it

Change-Id: Ifd3bbcc5e0591883b774d23256949a83010ea134

21 months agoFix GcConcurrentTest#testInterruptGc 18/200118/2
Matthias Sohn [Wed, 22 Feb 2023 19:35:01 +0000 (20:35 +0100)]
Fix GcConcurrentTest#testInterruptGc

With the new GC.PidLock interrupting a running GC throws a
ClosedByInterruptException.

Change-Id: I7ccea1ae9a43d4edfdab2fcfd1324c64cc22b38f

21 months agoDon't swallow IOException in GC.PidLock#lock 06/200106/1
Matthias Sohn [Wed, 22 Feb 2023 18:27:30 +0000 (19:27 +0100)]
Don't swallow IOException in GC.PidLock#lock

This broke the test GcConcurrentTest#testInterruptGc which expects
ClosedByInterruptException when the thread doing gc is interrupted.

Change-Id: I89e02fc37aceeccb04c20cfc5b71cb8fa21793d6

21 months agoCheck if FileLock is valid before using or releasing it 71/200071/2
Matthias Sohn [Wed, 22 Feb 2023 01:42:32 +0000 (02:42 +0100)]
Check if FileLock is valid before using or releasing it

Change-Id: I23ba67b61b9b03772f33a929c080c0d02b8c8652

21 months agoMerge branch 'master' into stable-6.5 70/200070/1
Matthias Sohn [Wed, 22 Feb 2023 01:18:10 +0000 (02:18 +0100)]
Merge branch 'master' into stable-6.5

* master:
  Use Java 11 ProcessHandle to get pid of the current process
  UploadPack: use allow-any-sha1-in-want configuration
  Acquire file lock "gc.pid" before running gc
  Silence API errors introduced by 9424052f

Change-Id: I71c8ee39e0f3438cbc1992c24f9f6c33ece88c07

21 months agoMerge branch 'stable-6.4' 68/200068/1
Matthias Sohn [Wed, 22 Feb 2023 00:29:32 +0000 (01:29 +0100)]
Merge branch 'stable-6.4'

* stable-6.4:
  Use Java 11 ProcessHandle to get pid of the current process
  Acquire file lock "gc.pid" before running gc
  Silence API errors introduced by 9424052f

Change-Id: Ifa4e56b6ecca9305f3f1685e45450019bfc82e22

21 months agoMerge branch 'stable-6.3' into stable-6.4 67/200067/1
Matthias Sohn [Wed, 22 Feb 2023 00:28:27 +0000 (01:28 +0100)]
Merge branch 'stable-6.3' into stable-6.4

* stable-6.3:
  Use Java 11 ProcessHandle to get pid of the current process
  Acquire file lock "gc.pid" before running gc
  Silence API errors introduced by 9424052f

Change-Id: Ic40dbab18616d8d9fe3820b9890c86652b80eb47

21 months agoMerge branch 'stable-6.2' into stable-6.3 66/200066/1
Matthias Sohn [Wed, 22 Feb 2023 00:27:50 +0000 (01:27 +0100)]
Merge branch 'stable-6.2' into stable-6.3

* stable-6.2:
  Use Java 11 ProcessHandle to get pid of the current process
  Acquire file lock "gc.pid" before running gc
  Silence API errors introduced by 9424052f

Change-Id: I53cf9675deac0b588048d8224216d2a7e8bd16ec

21 months agoMerge branch 'stable-6.1' into stable-6.2 65/200065/1
Matthias Sohn [Wed, 22 Feb 2023 00:27:16 +0000 (01:27 +0100)]
Merge branch 'stable-6.1' into stable-6.2

* stable-6.1:
  Use Java 11 ProcessHandle to get pid of the current process
  Acquire file lock "gc.pid" before running gc
  Silence API errors introduced by 9424052f

Change-Id: I0562a4a224779ccf1e4cc1ff8f5a352e55ab220a

21 months agoMerge branch 'stable-6.0' into stable-6.1 64/200064/1
Matthias Sohn [Wed, 22 Feb 2023 00:26:36 +0000 (01:26 +0100)]
Merge branch 'stable-6.0' into stable-6.1

* stable-6.0:
  Use Java 11 ProcessHandle to get pid of the current process
  Acquire file lock "gc.pid" before running gc
  Silence API errors introduced by 9424052f

Change-Id: Ib9a2419253ffcbc90874adbfdb8129fee3178210

21 months agoUse Java 11 ProcessHandle to get pid of the current process 63/200063/1
Matthias Sohn [Wed, 22 Feb 2023 00:06:06 +0000 (01:06 +0100)]
Use Java 11 ProcessHandle to get pid of the current process

Change-Id: I790f218601c1d5e1b39c4101e3b2708e76b9d782

21 months agoMerge branch 'stable-5.13' into stable-6.0 62/200062/1
Matthias Sohn [Wed, 22 Feb 2023 00:00:26 +0000 (01:00 +0100)]
Merge branch 'stable-5.13' into stable-6.0

* stable-5.13:
  Acquire file lock "gc.pid" before running gc
  Silence API errors introduced by 9424052f

Change-Id: Ibb5c46cb79377d2d2cd7d4586f31c86665d2851c

21 months agoUploadPack: use allow-any-sha1-in-want configuration 94/180794/8
kylezhao [Thu, 20 May 2021 03:29:59 +0000 (11:29 +0800)]
UploadPack: use allow-any-sha1-in-want configuration

C git 2.11 supports setting the equivalent of RequestPolicy.ANY with
uploadpack.allowAnySHA1InWant[1]. Parse this into TransportConfig and
use it from UploadPack.

Add additional tests for [2] and this change.

We can execute "git clone --filter=blob:none --no-checkout" successfully
with config uploadPack.allowFilter is true. But when we checkout, the
git will fetch other missing objects required by the checkout(this is
why we need this config).

When both uploadPack.allowFilter and uploadPack.allowAnySHA1InWant are
true, jgit will support partial clone. If you are using an extremely
large monorepo, this feature can help. It allows users to work on an
incomplete repo which reduces disk usage.

[1] https://github.com/git/git/commit/f8edeaa05d8623a9f6dad408237496c51101aad8
[2] change Id39771a6e42d8082099acde11249306828a053c0

Bug: 573390
Change-Id: I8fe75f03bf1fea7c11e0d67c8637bd05dd1f9b89
Signed-off-by: kylezhao <kylezhao@tencent.com>
21 months agoAcquire file lock "gc.pid" before running gc 48/199848/4
Matthias Sohn [Fri, 10 Feb 2023 22:39:20 +0000 (23:39 +0100)]
Acquire file lock "gc.pid" before running gc

Git guards gc by locking a lock file "gc.pid" before starting execution.
The lock file contains the pid and hostname of the process holding the
lock. Git tries to kill the process holding that lock if the lock file
wasn't modified in the last 12 hours and was started from the same host.

Teach JGit to acquire this lock before running gc but skip execution if
another process already holds the lock. Killing the other process could
be undesired if it's a long running application.

If the lock file wasn't modified in the last 12 hours try to lock it and
run gc if locking succeeds.

Register a shutdown hook for the lock file to ensure it is cleaned up if
the process is gracefully killed.

Change-Id: I00b838dcbf4fb0d03863bf7a2cd86b743c6c6971

21 months agoSilence API errors introduced by 9424052f 37/200037/1
Matthias Sohn [Mon, 20 Feb 2023 21:32:37 +0000 (22:32 +0100)]
Silence API errors introduced by 9424052f

Change-Id: Ia9e619a8fa06648086b583c994e4b107ae06c44d

21 months agoMerge branch 'master' into stable-6.5 33/200033/1
Matthias Sohn [Mon, 20 Feb 2023 21:18:22 +0000 (22:18 +0100)]
Merge branch 'master' into stable-6.5

* master:
  Externalize strings introduced in c9552aba
  Silence API error introduced by 596c445a
  PackConfig: add entry for minimum size to index
  Fix getPackedRefs to not throw NoSuchFileException
  PackObjectSizeIndex: interface and impl for the object-size index
  UInt24Array: Array of unsigned ints encoded in 3 bytes.
  PackIndex: expose the position of an object-id in the index
  Add pack options to preserve and prune old pack files
  DfsPackFile/DfsGC: Write commit graphs and expose in pack
  ObjectReader: Allow getCommitGraph to throw IOException
  Allow to perform PackedBatchRefUpdate without locking loose refs
  Document option "core.sha1Implementation" introduced in 59029aec
  UploadPack: consume delimiter in object-info command
  PatchApplier fix - init cache with provided tree
  Avoid error-prone warning
  Fix unused exception error-prone warning
  UploadPack: advertise object-info command if enabled
  Move MemRefDatabase creation in a separate method.
  DfsReaderIoStats: Add Commit Graph fields into DfsReaderIoStats

Change-Id: Ic9f91f2139432999b99c444302457b3c08911009

21 months agoExternalize strings introduced in c9552aba 31/200031/1
Matthias Sohn [Mon, 20 Feb 2023 20:40:40 +0000 (21:40 +0100)]
Externalize strings introduced in c9552aba

Change-Id: I81bb78344df61e6eb42622fcef6235d4da0ae052

21 months agoSilence API error introduced by 596c445a 30/200030/1
Matthias Sohn [Mon, 20 Feb 2023 20:31:09 +0000 (21:31 +0100)]
Silence API error introduced by 596c445a

Change-Id: I961ba2d89c11373ccb81e6450d7d951204ffca36

21 months agoMerge branch 'stable-6.4' 29/200029/2
Matthias Sohn [Mon, 20 Feb 2023 20:26:08 +0000 (21:26 +0100)]
Merge branch 'stable-6.4'

* stable-6.4:
  Fix getPackedRefs to not throw NoSuchFileException
  Add pack options to preserve and prune old pack files
  Allow to perform PackedBatchRefUpdate without locking loose refs
  Document option "core.sha1Implementation" introduced in 59029aec

Change-Id: I36051c623fcd480aa80ed32b4e89f9bdd1b798e0

21 months agoMerge branch 'stable-6.3' into stable-6.4 27/200027/1
Matthias Sohn [Mon, 20 Feb 2023 20:01:38 +0000 (21:01 +0100)]
Merge branch 'stable-6.3' into stable-6.4

* stable-6.3:
  Fix getPackedRefs to not throw NoSuchFileException
  Add pack options to preserve and prune old pack files
  Allow to perform PackedBatchRefUpdate without locking loose refs
  Document option "core.sha1Implementation" introduced in 59029aec

Change-Id: I1073098fb06eabafdb3c5e7fcf44d55b86a1b152

21 months agoMerge branch 'stable-6.2' into stable-6.3 26/200026/1
Matthias Sohn [Mon, 20 Feb 2023 19:59:14 +0000 (20:59 +0100)]
Merge branch 'stable-6.2' into stable-6.3

* stable-6.2:
  Fix getPackedRefs to not throw NoSuchFileException
  Add pack options to preserve and prune old pack files
  Allow to perform PackedBatchRefUpdate without locking loose refs
  Document option "core.sha1Implementation" introduced in 59029aec

Change-Id: I765c7302ce84a6a9c28fdef29da2bfaa49477c6e

21 months agoPackConfig: add entry for minimum size to index 84/191084/18
Ivan Frade [Tue, 4 Jan 2022 19:10:04 +0000 (11:10 -0800)]
PackConfig: add entry for minimum size to index

The object size index can have up to #(blobs-in-repo) entries, taking
a relevant amount of memory. Let operators configure the threshold size
to include objects in the size index.

The index will include objects with size *at or above* this
value (with -1 for none). This is more effective for the
filter-by-size case.

Lowering the threshold adds more objects to the index. This improves
performance at the cost of memory/storage space. For the object-size
case, more calls will use the index instead of reading IO. For the
filter-by-size case, lower threshold means better granularity (if
ObjectReader#isSmallerThan is implemented based only on the index).

Change-Id: I6ccd9334adbbc2abf95fde51dbbfc85b8230ade0

21 months agoMerge branch 'stable-6.1' into stable-6.2 25/200025/1
Matthias Sohn [Thu, 16 Feb 2023 15:59:56 +0000 (16:59 +0100)]
Merge branch 'stable-6.1' into stable-6.2

* stable-6.1:
  Fix getPackedRefs to not throw NoSuchFileException
  Add pack options to preserve and prune old pack files
  Allow to perform PackedBatchRefUpdate without locking loose refs
  Document option "core.sha1Implementation" introduced in 59029aec

Change-Id: Id32683d5f506e082d39af269803bccee0280cc27

21 months agoMerge branch 'stable-6.0' into stable-6.1 67/199967/1
Matthias Sohn [Thu, 16 Feb 2023 15:56:07 +0000 (16:56 +0100)]
Merge branch 'stable-6.0' into stable-6.1

* stable-6.0:
  Add pack options to preserve and prune old pack files
  Allow to perform PackedBatchRefUpdate without locking loose refs
  Document option "core.sha1Implementation" introduced in 59029aec

Change-Id: I876a38c2de8b7d5eaacd00e36b85599f88173221

21 months agoMerge branch 'stable-5.13' into stable-6.0 62/199962/2
Matthias Sohn [Thu, 16 Feb 2023 15:42:58 +0000 (16:42 +0100)]
Merge branch 'stable-5.13' into stable-6.0

* stable-5.13:
  Add pack options to preserve and prune old pack files
  Allow to perform PackedBatchRefUpdate without locking loose refs
  Document option "core.sha1Implementation" introduced in 59029aec

Change-Id: I423f410578f5bbe178832b80fef8998a5372182c

21 months agoFix getPackedRefs to not throw NoSuchFileException 59/199959/1
Prudhvi Akhil Alahari [Thu, 16 Feb 2023 11:11:55 +0000 (16:41 +0530)]
Fix getPackedRefs to not throw NoSuchFileException

Since Files.newInputStream is from java.nio package, it throws
java.nio.file.NoSuchFileException. This was missed in the change
I00da88e. Without this change, getPackedRefs fails with
NoSuchFileException when there is no packed-refs file in a project.

Change-Id: I93c202ddb73a0a5979af8e4d09e45f5645664b45
Signed-off-by: Prudhvi Akhil Alahari <quic_prudhvi@quicinc.com>
21 months agoPackObjectSizeIndex: interface and impl for the object-size index 83/191083/16
Ivan Frade [Wed, 15 Dec 2021 00:36:54 +0000 (16:36 -0800)]
PackObjectSizeIndex: interface and impl for the object-size index

Operations like "clone --filter=blob:limit=N" or the "object-info"
command need to read the size of the objects from the storage. An
index would provide those sizes at once rather than having to seek in
the packfile.

Introduce an interface for the Object-size index. This index returns
the inflated size of an object. Not all objects could be indexed (to
limit memory usage).

This implementation indexes only blobs (no trees, nor
commits) *above* certain size threshold (configurable). Lower
threshold adds more objects to the index, consumes more memory and
provides better performance. 0 means "all blobs" and -1 "disabled".

If we don't index everything, for the filter use case is more
efficient to index the biggest objects first: the set is small and
most objects are filtered by NOT being in the index. For the
object-size, the more objects in the index the better, regardless
their size. All together, it is more helpful to index above threshold.

Change-Id: I9ed608ac240677e199b90ca40d420bcad9231489

21 months agoUInt24Array: Array of unsigned ints encoded in 3 bytes. 92/199892/3
Ivan Frade [Fri, 10 Feb 2023 20:21:01 +0000 (12:21 -0800)]
UInt24Array: Array of unsigned ints encoded in 3 bytes.

The object size index stores positions of objects in the main
index (when ordered by sha1). These positions are per-pack and usually
a pack has <16 million objects (there are exceptions but rather
rare). It could save some memory storing these positions in three bytes
instead of four. Note that these positions are sorted and always positive.

Implement a wrapper around a byte[] to access and search "ints" while
they are stored as unsigned 3 bytes.

Change-Id: Iaa26ce8e2272e706e35fe4cdb648fb6ca7591972

21 months agoPackIndex: expose the position of an object-id in the index 67/199467/5
Ivan Frade [Tue, 17 Jan 2023 18:01:29 +0000 (10:01 -0800)]
PackIndex: expose the position of an object-id in the index

The primary index returns the offset in the pack for an
objectId. Internally it keeps the object-ids in lexicographical order,
but doesn't expose an API to find the position of an object-id in that
list. This is needed for the object-size index, that we want to store
as "position-in-idx, size".

Add a #findPosition(object-id) method to the PackIndex interface to
know where an object-id sits in the ordered list of ids in the pack.

Note that this index position is over the list of ordered object-ids,
while reverse-index position is over the list of objects in packed
order.

Change-Id: I89fa146599e347a26d3012d3477d7f5bbbda7ba4

21 months agoAdd pack options to preserve and prune old pack files 46/199846/2
Matthias Sohn [Fri, 10 Feb 2023 20:05:47 +0000 (21:05 +0100)]
Add pack options to preserve and prune old pack files

Add the options
- pack.preserveOldPacks
- pack.prunePreserved

This allows to configure in git config if old packs should be preserved
during gc and pruned during the next gc.

The original implementation in 91132bb0 only allows to set these options
using the API.

Change-Id: I5b23ab4f317d12f5ccd234401419913e8263cc9a

21 months agoDfsPackFile/DfsGC: Write commit graphs and expose in pack 54/199654/11
Xing Huang [Mon, 6 Feb 2023 20:18:59 +0000 (14:18 -0600)]
DfsPackFile/DfsGC: Write commit graphs and expose in pack

JGit knows how to read/write commit graphs but the DFS stack is not
using it yet.

The DFS garbage collector generates a commit-graph with commits
reachable from any ref. The pack is stored as extra stream in the GC
pack. DfsPackFile mimicks how other indices are loaded storing the
reference in DFS cache.

Signed-off-by: Xing Huang <xingkhuang@google.com>
Change-Id: I3f94997377986d21a56b300d8358dd27be37f5de

21 months agoObjectReader: Allow getCommitGraph to throw IOException 85/199785/4
Xing Huang [Mon, 6 Feb 2023 20:18:16 +0000 (14:18 -0600)]
ObjectReader: Allow getCommitGraph to throw IOException

ObjectReader#getCommitGraph doesn't report errors loading the
commit graph. The caller should be aware of the situation and
ultimately decide what to do.

Add IOException to ObjectReader#getCommitGraph signature. RevWalk
defaults to an empty commit-graph on IO errors.

Signed-off-by: Xing Huang <xingkhuang@google.com>
Change-Id: I38eeacff76c7f926b6dfb192d1e5916e40770024

21 months agoAllow to perform PackedBatchRefUpdate without locking loose refs 58/199758/1
Saša Živkov [Fri, 21 Oct 2022 14:32:03 +0000 (16:32 +0200)]
Allow to perform PackedBatchRefUpdate without locking loose refs

Add another newBatchUpdate method in the RefDirectory where we can
control if the created PackedBatchRefUpdate will lock the loose refs or
not.

This can be useful in cases when we run programs which have exclusive
access to a Git repository and we know that locking loose refs is
unnecessary and just a performance loss.

Change-Id: I7d0932eb1598a3871a2281b1a049021380234df9
(cherry picked from commit cb90ed08526bd51f04e5d72e3ba3cf5bd30c11e4)

21 months agoMerge "Merge branch 'stable-6.5'"
Matthias Sohn [Thu, 2 Feb 2023 20:20:14 +0000 (15:20 -0500)]
Merge "Merge branch 'stable-6.5'"

21 months agoDocument option "core.sha1Implementation" introduced in 59029aec 11/199711/2
Matthias Sohn [Wed, 1 Feb 2023 13:33:31 +0000 (14:33 +0100)]
Document option "core.sha1Implementation" introduced in 59029aec

Bug: 580310
Change-Id: I10f3d6f6b5af7ab96683994c9cbd85e6c18a5084

21 months agoMerge "UploadPack: consume delimiter in object-info command"
Han-Wen NIenhuys [Thu, 2 Feb 2023 14:09:25 +0000 (09:09 -0500)]
Merge "UploadPack: consume delimiter in object-info command"