]> source.dussan.org Git - jgit.git/log
jgit.git
17 months agoMerge branch 'stable-6.2' into stable-6.3 26/201426/1
Matthias Sohn [Thu, 20 Apr 2023 22:25:51 +0000 (00:25 +0200)]
Merge branch 'stable-6.2' into stable-6.3

* stable-6.2:
  Add missing since tag for SshBasicTestBase
  Add missing since tag for SshTestHarness#publicKey2
  Silence API errors
  Prevent infinite loop rescanning the pack list on PackMismatchException
  Remove blank in maven.config

Change-Id: I8006068f16ae442a2246e043a680053f2af34e9f

17 months agoMerge branch 'stable-6.1' into stable-6.2 25/201425/1
Matthias Sohn [Thu, 20 Apr 2023 22:19:38 +0000 (00:19 +0200)]
Merge branch 'stable-6.1' into stable-6.2

* stable-6.1:
  Add missing since tag for SshBasicTestBase
  Add missing since tag for SshTestHarness#publicKey2
  Silence API errors
  Prevent infinite loop rescanning the pack list on PackMismatchException
  Remove blank in maven.config

Change-Id: I4c5b000b09287cc32f0e4d6a24a766ef4e17ddbe

17 months agoMerge branch 'stable-6.0' into stable-6.1 24/201424/1
Matthias Sohn [Thu, 20 Apr 2023 22:11:40 +0000 (00:11 +0200)]
Merge branch 'stable-6.0' into stable-6.1

* stable-6.0:
  Add missing since tag for SshBasicTestBase
  Add missing since tag for SshTestHarness#publicKey2
  Silence API errors
  Prevent infinite loop rescanning the pack list on PackMismatchException
  Remove blank in maven.config

Change-Id: Ia01c5ac5259b8820afb823d97bee247b5a5fb14a

17 months agoMerge branch 'stable-5.13' into stable-6.0 23/201423/1
Matthias Sohn [Thu, 20 Apr 2023 14:01:33 +0000 (16:01 +0200)]
Merge branch 'stable-5.13' into stable-6.0

* stable-5.13:
  Add missing since tag for SshBasicTestBase
  Add missing since tag for SshTestHarness#publicKey2
  Silence API errors
  Prevent infinite loop rescanning the pack list on
PackMismatchException
  Remove blank in maven.config

Change-Id: Id37bee59ca3c7947604c54b6d4e7c02628a657fe

17 months agoMerge branch 'stable-5.12' into stable-5.13 16/201416/1
Matthias Sohn [Thu, 20 Apr 2023 13:40:36 +0000 (15:40 +0200)]
Merge branch 'stable-5.12' into stable-5.13

* stable-5.12:
  Add missing since tag for SshBasicTestBase
  Add missing since tag for SshTestHarness#publicKey2
  Silence API errors
  Prevent infinite loop rescanning the pack list on
PackMismatchException
  Remove blank in maven.config

Change-Id: Ibe6652374ab5971105e62b05279f218c8c130fee

17 months agoMerge branch 'stable-5.11' into stable-5.12 15/201415/1 stable-5.12
Matthias Sohn [Thu, 20 Apr 2023 13:12:01 +0000 (15:12 +0200)]
Merge branch 'stable-5.11' into stable-5.12

* stable-5.11:
  Add missing since tag for SshBasicTestBase
  Add missing since tag for SshTestHarness#publicKey2
  Silence API errors
  Prevent infinite loop rescanning the pack list on PackMismatchException
  Remove blank in maven.config

Change-Id: I25bb99687b969f9915a7cbda8d1332bec778096a

17 months agoAdd missing since tag for SshBasicTestBase 14/201414/2 stable-5.11
Matthias Sohn [Thu, 20 Apr 2023 12:46:05 +0000 (14:46 +0200)]
Add missing since tag for SshBasicTestBase

Change-Id: Iad8ae9bb526418b279dc54a5e9d0c877c1eca475

17 months agoMerge branch 'stable-5.10' into stable-5.11 13/201413/2
Matthias Sohn [Thu, 20 Apr 2023 12:42:56 +0000 (14:42 +0200)]
Merge branch 'stable-5.10' into stable-5.11

* stable-5.10:
  Add missing since tag for SshTestHarness#publicKey2
  Silence API errors
  Prevent infinite loop rescanning the pack list on
PackMismatchException
  Remove blank in maven.config

Migrated "Prevent infinite loop rescanning the pack list on
PackMismatchException" to refactoring done in
https://git.eclipse.org/r/q/topic:restore-preserved-packs

Change-Id: I0fb77bb9b498d48d5da88a93486b99bf8121e3bd

17 months agoAdd missing since tag for SshTestHarness#publicKey2 12/201412/1 stable-5.10
Matthias Sohn [Thu, 20 Apr 2023 12:35:58 +0000 (14:35 +0200)]
Add missing since tag for SshTestHarness#publicKey2

Change-Id: Ib6e4945340d2e1761dc0e787bdbe72286cdc95bc

17 months agoSilence API errors 11/201411/1
Matthias Sohn [Thu, 20 Apr 2023 12:37:46 +0000 (14:37 +0200)]
Silence API errors

Change-Id: I367c05c43df3385c33ce76bc10b2dc3bd330665c

17 months agoMerge branch 'stable-5.9' into stable-5.10 10/201410/2
Matthias Sohn [Thu, 20 Apr 2023 07:52:30 +0000 (09:52 +0200)]
Merge branch 'stable-5.9' into stable-5.10

* stable-5.9:
  Prevent infinite loop rescanning the pack list on
PackMismatchException
  Remove blank in maven.config

Change-Id: I15ff2d7716ecaceb0eb87b8823d85670f5db709d

17 months agoPrevent infinite loop rescanning the pack list on PackMismatchException 78/200978/6 stable-5.9
Matthias Sohn [Thu, 30 Mar 2023 11:43:17 +0000 (13:43 +0200)]
Prevent infinite loop rescanning the pack list on PackMismatchException

We found, when analysing an incident where Gerrit's gc runner thread got
stuck, that we can end up in an infinite loop in
ObjectDirectory#openPackedObject which tries to rescan the pack
list and starts over trying to open a packed object in an unconfined
loop if it catches a PackMismatchException.

Here the relevant part of a thread dump we created while the gc runner
was stuck:

"WorkQueue-2[java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@350812a3[Not
completed,
task = java.util.concurrent.Executors$RunnableAdapter@5425d7ee]]" #72
tid=0x00007f73cee1c800 nid=0x584
runnable  [0x00007f7392d57000]
   java.lang.Thread.State: RUNNABLE
at org.eclipse.jgit.internal.storage.file.WindowCache.removeAll(WindowCache.java:716)
at org.eclipse.jgit.internal.storage.file.WindowCache.purge(WindowCache.java:399)
at org.eclipse.jgit.internal.storage.file.PackFile.close(PackFile.java:296)
at org.eclipse.jgit.internal.storage.file.ObjectDirectory.reuseMap(ObjectDirectory.java:973)
at org.eclipse.jgit.internal.storage.file.ObjectDirectory.scanPacksImpl(ObjectDirectory.java:904)
at org.eclipse.jgit.internal.storage.file.ObjectDirectory.scanPacks(ObjectDirectory.java:895)
- locked <0x000000050a498f60> (a
java.util.concurrent.atomic.AtomicReference)
at org.eclipse.jgit.internal.storage.file.ObjectDirectory.searchPacksAgain(ObjectDirectory.java:794)
at org.eclipse.jgit.internal.storage.file.ObjectDirectory.openPackedObject(ObjectDirectory.java:465)
at org.eclipse.jgit.internal.storage.file.ObjectDirectory.openPackedFromSelfOrAlternate(ObjectDirectory.java:417)
at org.eclipse.jgit.internal.storage.file.ObjectDirectory.openObject(ObjectDirectory.java:408)
at org.eclipse.jgit.internal.storage.file.WindowCursor.open(WindowCursor.java:132)
at org.eclipse.jgit.lib.ObjectReader$1.open(ObjectReader.java:279)
at org.eclipse.jgit.revwalk.RevWalk$2.next(RevWalk.java:1031)
at org.eclipse.jgit.internal.storage.pack.PackWriter.findObjectsToPack(PackWriter.java:1911)
at org.eclipse.jgit.internal.storage.pack.PackWriter.preparePack(PackWriter.java:960)
at org.eclipse.jgit.internal.storage.pack.PackWriter.preparePack(PackWriter.java:876)
at org.eclipse.jgit.internal.storage.file.GC.writePack(GC.java:1168)
at org.eclipse.jgit.internal.storage.file.GC.repack(GC.java:852)
at org.eclipse.jgit.internal.storage.file.GC.doGc(GC.java:269)
at org.eclipse.jgit.internal.storage.file.GC.gc(GC.java:220)
at org.eclipse.jgit.api.GarbageCollectCommand.call(GarbageCollectCommand.java:179)
at com.google.gerrit.server.git.GarbageCollection.run(GarbageCollection.java:112)
at com.google.gerrit.server.git.GarbageCollection.run(GarbageCollection.java:75)
at com.google.gerrit.server.git.GarbageCollection.run(GarbageCollection.java:71)
at com.google.gerrit.server.git.GarbageCollectionRunner.run(GarbageCollectionRunner.java:76)
at com.google.gerrit.server.logging.LoggingContextAwareRunnable.run(LoggingContextAwareRunnable.java:103)
at java.util.concurrent.Executors$RunnableAdapter.call(java.base@11.0.18/Executors.java:515)
at java.util.concurrent.FutureTask.runAndReset(java.base@11.0.18/FutureTask.java:305)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(java.base@11.0.18/ScheduledThreadPoolExecutor.java:305)
at com.google.gerrit.server.git.WorkQueue$Task.run(WorkQueue.java:612)
at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.18/ThreadPoolExecutor.java:1128)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.18/ThreadPoolExecutor.java:628)
at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)

The code in ObjectDirectory#openPackedObject [1] apparently assumes that
this is caused by a transient problem which it can resume from by
retrying. We use `core.trustFolderStat = false` on this server since it
uses NFS. The incident we had showed that we can enter into an infinite
loop here if there is a permanent mismatch between a pack file and its
corresponding pack index. I am not yet sure how this can happen.

Break the infinite loop by limiting the number of attempts rescanning
the pack list to 5 retries.  When we exceed this threshold set the type
of the PackMismatchException to permanent and rethrow it which breaks
the infinite loop.

Also apply the same limit in #getPackedObjectSize
and #selectObjectRepresentation where we use similar retry loops.

[1] https://git.eclipse.org/r/plugins/gitiles/jgit/jgit/+/011c26ff36b9e76c84fc2459e337f159c0f55a9a/org.eclipse.jgit/src/org/eclipse/jgit/internal/storage/file/ObjectDirectory.java#465

Change-Id: I20fb63bcc1fdc3a03d39b963f06a90e6f0ba73dc

17 months agoRemove blank in maven.config 03/201403/1
Matthias Sohn [Fri, 14 Apr 2023 16:13:18 +0000 (18:13 +0200)]
Remove blank in maven.config

Maven 3.9.1 doesn't accept this whitespace.

Change-Id: I0f6e3652b1e581615c370d35bc782184712ac922

18 months agoMerge branch 'stable-6.2' into stable-6.3 44/201344/1
Matthias Sohn [Sat, 15 Apr 2023 19:35:16 +0000 (21:35 +0200)]
Merge branch 'stable-6.2' into stable-6.3

* stable-6.2:
  Remove blank in maven.config
  DirCache: support option index.skipHash

Change-Id: If0bb5f1a317ab981e6bbf5671851f124b18ab8ca

18 months agoMerge branch 'stable-6.1' into stable-6.2 43/201343/1
Matthias Sohn [Sat, 15 Apr 2023 19:34:41 +0000 (21:34 +0200)]
Merge branch 'stable-6.1' into stable-6.2

* stable-6.1:
  Remove blank in maven.config
  DirCache: support option index.skipHash

Change-Id: Ief50a2ca8e5a8630627506f4d2142d62c0554615

18 months agoMerge branch 'stable-6.0' into stable-6.1 42/201342/1
Matthias Sohn [Fri, 14 Apr 2023 22:49:59 +0000 (00:49 +0200)]
Merge branch 'stable-6.0' into stable-6.1

* stable-6.0:
  Remove blank in maven.config
  DirCache: support option index.skipHash

Change-Id: Idf757bcab0d7a65ea63504674a681170c6db2f94

18 months agoMerge branch 'stable-5.13' into stable-6.0 36/201336/1
Matthias Sohn [Fri, 14 Apr 2023 22:49:08 +0000 (00:49 +0200)]
Merge branch 'stable-5.13' into stable-6.0

* stable-5.13:
  Remove blank in maven.config
  DirCache: support option index.skipHash

Change-Id: I0cc3033b1876c8c691c2a6876206cd71fa07d2e0

18 months agoRemove blank in maven.config 35/201335/1
Matthias Sohn [Fri, 14 Apr 2023 16:13:18 +0000 (18:13 +0200)]
Remove blank in maven.config

Maven 3.9.1 doesn't accept this whitespace.

Change-Id: I0f6e3652b1e581615c370d35bc782184712ac922

18 months agoDirCache: support option index.skipHash 16/200916/2
Matthias Sohn [Mon, 27 Mar 2023 20:23:11 +0000 (22:23 +0200)]
DirCache: support option index.skipHash

Support the new option index.skipHash which was introduced in git 2.40
[1]. If it is set to true skip computing the git index checksum. This
accelerates Git commands that manipulate the index, such as git add, git
commit, or git status. Instead of storing the checksum, write a trailing
set of bytes with value zero, indicating that the computation was
skipped.

Accept a skipped checksum consisting of 20 null bytes when reading the
index since the option could have been set to true at the time when the
index was written.

[1] https://git-scm.com/docs/git-config#Documentation/git-config.txt-indexskipHash

Bug: 581723
Change-Id: I28ebe44c5ca1cbcb882438665d686452a0c111b2

18 months agoMerge branch 'stable-6.2' into stable-6.3 93/200793/1
Matthias Sohn [Thu, 23 Mar 2023 08:06:43 +0000 (09:06 +0100)]
Merge branch 'stable-6.2' into stable-6.3

* stable-6.2:
  GC: Close File.lines stream

Change-Id: Id93b1933a5ce1ede9eb388c9fd54a4b3749694bf

18 months agoMerge branch 'stable-6.1' into stable-6.2 92/200792/1
Matthias Sohn [Thu, 23 Mar 2023 08:06:16 +0000 (09:06 +0100)]
Merge branch 'stable-6.1' into stable-6.2

* stable-6.1:
  GC: Close File.lines stream

Change-Id: Ia2be0b05ed860125a388b01d6c291832f08dd990

18 months agoMerge branch 'stable-6.0' into stable-6.1 91/200791/1
Matthias Sohn [Thu, 23 Mar 2023 08:05:42 +0000 (09:05 +0100)]
Merge branch 'stable-6.0' into stable-6.1

* stable-6.0:
  GC: Close File.lines stream

Change-Id: I2f9e6da5584a40bb4b4efed0b87ae456f119d757

18 months agoMerge branch 'stable-5.13' into stable-6.0 90/200790/1
Matthias Sohn [Thu, 23 Mar 2023 08:04:50 +0000 (09:04 +0100)]
Merge branch 'stable-5.13' into stable-6.0

* stable-5.13:
  GC: Close File.lines stream

Change-Id: Ib473750e5a3ad3d74b0cb41f25052890f50a975c

18 months agoGC: Close File.lines stream 89/200789/2
Xing Huang [Tue, 21 Mar 2023 22:27:49 +0000 (17:27 -0500)]
GC: Close File.lines stream

From File#lines javadoc: The returned stream from File Lines
encapsulates a Reader. If timely disposal of file system resources is
required, the try-with-resources construct should be used to ensure
that the stream's close method is
invoked after the stream operations are completed.

Wrap File.lines with try-with-resources.

Change-Id: I82c6faa3ef1083f6c7e964f96e9540b4db18eee8
Signed-off-by: Xing Huang <xingkhuang@google.com>
(cherry picked from commit 172a207945da376b6b4143305aef2af56f7c42e2)

19 months agoMerge branch 'stable-6.2' into stable-6.3 18/200318/1
Matthias Sohn [Fri, 3 Mar 2023 15:02:16 +0000 (16:02 +0100)]
Merge branch 'stable-6.2' into stable-6.3

* stable-6.2:
  Improve test coverage when core.trustPackedRefsStat set to after_open

Change-Id: I02d7444b28b662eb46f3d0ba1a62d342820bed1f

19 months agoMerge branch 'stable-6.1' into stable-6.2 17/200317/1
Matthias Sohn [Fri, 3 Mar 2023 15:01:26 +0000 (16:01 +0100)]
Merge branch 'stable-6.1' into stable-6.2

* stable-6.1:
  Improve test coverage when core.trustPackedRefsStat set to after_open

Change-Id: I63ba388ee2e174019733e63f8177792e1808a5c3

19 months agoImprove test coverage when core.trustPackedRefsStat set to after_open 04/200304/1
Prudhvi Akhil Alahari [Thu, 2 Mar 2023 11:21:48 +0000 (16:51 +0530)]
Improve test coverage when core.trustPackedRefsStat set to after_open

As of today, we don't have test coverage for RefDirectory when
core.trustPackedRefsStat config is set to after_open. Thus create new
test classes which set core.trustPackedRefsStat config to after_open in
setup and extend RefDirectoryTest and FileRepositoryBuilderTest
respectively.

Change-Id: I1db6fcf414bc488106ad4c85fb934480f299c995
Signed-off-by: Prudhvi Akhil Alahari <quic_prudhvi@quicinc.com>
19 months agoMerge branch 'stable-6.2' into stable-6.3 24/200124/1
Matthias Sohn [Wed, 22 Feb 2023 20:03:52 +0000 (21:03 +0100)]
Merge branch 'stable-6.2' into stable-6.3

* stable-6.2:
  If tryLock fails to get the lock another gc has it
  Fix GcConcurrentTest#testInterruptGc
  Don't swallow IOException in GC.PidLock#lock
  Check if FileLock is valid before using or releasing it

Change-Id: I5b6b10622b61fde3f0f10455a74ae159a0b69082

19 months agoMerge branch 'stable-6.1' into stable-6.2 23/200123/1
Matthias Sohn [Wed, 22 Feb 2023 20:03:22 +0000 (21:03 +0100)]
Merge branch 'stable-6.1' into stable-6.2

* stable-6.1:
  If tryLock fails to get the lock another gc has it
  Fix GcConcurrentTest#testInterruptGc
  Don't swallow IOException in GC.PidLock#lock
  Check if FileLock is valid before using or releasing it

Change-Id: I3ffe92566cc145053bb762f612dd96bc6d542c62

19 months agoMerge branch 'stable-6.0' into stable-6.1 22/200122/1
Matthias Sohn [Wed, 22 Feb 2023 20:02:47 +0000 (21:02 +0100)]
Merge branch 'stable-6.0' into stable-6.1

* stable-6.0:
  If tryLock fails to get the lock another gc has it
  Fix GcConcurrentTest#testInterruptGc
  Don't swallow IOException in GC.PidLock#lock
  Check if FileLock is valid before using or releasing it

Change-Id: Idea23e555c024557d7e39a86efe25f609400b962

19 months agoMerge branch 'stable-5.13' into stable-6.0 21/200121/1
Matthias Sohn [Wed, 22 Feb 2023 20:02:09 +0000 (21:02 +0100)]
Merge branch 'stable-5.13' into stable-6.0

* stable-5.13:
  If tryLock fails to get the lock another gc has it
  Fix GcConcurrentTest#testInterruptGc
  Don't swallow IOException in GC.PidLock#lock
  Check if FileLock is valid before using or releasing it

Change-Id: I708d0936fa86b028e4da4e7e21f332f8b48ad293

19 months agoIf tryLock fails to get the lock another gc has it 19/200119/2
Matthias Sohn [Wed, 22 Feb 2023 19:36:39 +0000 (20:36 +0100)]
If tryLock fails to get the lock another gc has it

Change-Id: Ifd3bbcc5e0591883b774d23256949a83010ea134

19 months agoFix GcConcurrentTest#testInterruptGc 18/200118/2
Matthias Sohn [Wed, 22 Feb 2023 19:35:01 +0000 (20:35 +0100)]
Fix GcConcurrentTest#testInterruptGc

With the new GC.PidLock interrupting a running GC throws a
ClosedByInterruptException.

Change-Id: I7ccea1ae9a43d4edfdab2fcfd1324c64cc22b38f

19 months agoDon't swallow IOException in GC.PidLock#lock 06/200106/1
Matthias Sohn [Wed, 22 Feb 2023 18:27:30 +0000 (19:27 +0100)]
Don't swallow IOException in GC.PidLock#lock

This broke the test GcConcurrentTest#testInterruptGc which expects
ClosedByInterruptException when the thread doing gc is interrupted.

Change-Id: I89e02fc37aceeccb04c20cfc5b71cb8fa21793d6

19 months agoCheck if FileLock is valid before using or releasing it 71/200071/2
Matthias Sohn [Wed, 22 Feb 2023 01:42:32 +0000 (02:42 +0100)]
Check if FileLock is valid before using or releasing it

Change-Id: I23ba67b61b9b03772f33a929c080c0d02b8c8652

19 months agoMerge branch 'stable-6.2' into stable-6.3 66/200066/1
Matthias Sohn [Wed, 22 Feb 2023 00:27:50 +0000 (01:27 +0100)]
Merge branch 'stable-6.2' into stable-6.3

* stable-6.2:
  Use Java 11 ProcessHandle to get pid of the current process
  Acquire file lock "gc.pid" before running gc
  Silence API errors introduced by 9424052f

Change-Id: I53cf9675deac0b588048d8224216d2a7e8bd16ec

19 months agoMerge branch 'stable-6.1' into stable-6.2 65/200065/1
Matthias Sohn [Wed, 22 Feb 2023 00:27:16 +0000 (01:27 +0100)]
Merge branch 'stable-6.1' into stable-6.2

* stable-6.1:
  Use Java 11 ProcessHandle to get pid of the current process
  Acquire file lock "gc.pid" before running gc
  Silence API errors introduced by 9424052f

Change-Id: I0562a4a224779ccf1e4cc1ff8f5a352e55ab220a

19 months agoMerge branch 'stable-6.0' into stable-6.1 64/200064/1
Matthias Sohn [Wed, 22 Feb 2023 00:26:36 +0000 (01:26 +0100)]
Merge branch 'stable-6.0' into stable-6.1

* stable-6.0:
  Use Java 11 ProcessHandle to get pid of the current process
  Acquire file lock "gc.pid" before running gc
  Silence API errors introduced by 9424052f

Change-Id: Ib9a2419253ffcbc90874adbfdb8129fee3178210

19 months agoUse Java 11 ProcessHandle to get pid of the current process 63/200063/1
Matthias Sohn [Wed, 22 Feb 2023 00:06:06 +0000 (01:06 +0100)]
Use Java 11 ProcessHandle to get pid of the current process

Change-Id: I790f218601c1d5e1b39c4101e3b2708e76b9d782

19 months agoMerge branch 'stable-5.13' into stable-6.0 62/200062/1
Matthias Sohn [Wed, 22 Feb 2023 00:00:26 +0000 (01:00 +0100)]
Merge branch 'stable-5.13' into stable-6.0

* stable-5.13:
  Acquire file lock "gc.pid" before running gc
  Silence API errors introduced by 9424052f

Change-Id: Ibb5c46cb79377d2d2cd7d4586f31c86665d2851c

19 months agoAcquire file lock "gc.pid" before running gc 48/199848/4
Matthias Sohn [Fri, 10 Feb 2023 22:39:20 +0000 (23:39 +0100)]
Acquire file lock "gc.pid" before running gc

Git guards gc by locking a lock file "gc.pid" before starting execution.
The lock file contains the pid and hostname of the process holding the
lock. Git tries to kill the process holding that lock if the lock file
wasn't modified in the last 12 hours and was started from the same host.

Teach JGit to acquire this lock before running gc but skip execution if
another process already holds the lock. Killing the other process could
be undesired if it's a long running application.

If the lock file wasn't modified in the last 12 hours try to lock it and
run gc if locking succeeds.

Register a shutdown hook for the lock file to ensure it is cleaned up if
the process is gracefully killed.

Change-Id: I00b838dcbf4fb0d03863bf7a2cd86b743c6c6971

19 months agoSilence API errors introduced by 9424052f 37/200037/1
Matthias Sohn [Mon, 20 Feb 2023 21:32:37 +0000 (22:32 +0100)]
Silence API errors introduced by 9424052f

Change-Id: Ia9e619a8fa06648086b583c994e4b107ae06c44d

19 months agoMerge branch 'stable-6.2' into stable-6.3 26/200026/1
Matthias Sohn [Mon, 20 Feb 2023 19:59:14 +0000 (20:59 +0100)]
Merge branch 'stable-6.2' into stable-6.3

* stable-6.2:
  Fix getPackedRefs to not throw NoSuchFileException
  Add pack options to preserve and prune old pack files
  Allow to perform PackedBatchRefUpdate without locking loose refs
  Document option "core.sha1Implementation" introduced in 59029aec

Change-Id: I765c7302ce84a6a9c28fdef29da2bfaa49477c6e

19 months agoMerge branch 'stable-6.1' into stable-6.2 25/200025/1
Matthias Sohn [Thu, 16 Feb 2023 15:59:56 +0000 (16:59 +0100)]
Merge branch 'stable-6.1' into stable-6.2

* stable-6.1:
  Fix getPackedRefs to not throw NoSuchFileException
  Add pack options to preserve and prune old pack files
  Allow to perform PackedBatchRefUpdate without locking loose refs
  Document option "core.sha1Implementation" introduced in 59029aec

Change-Id: Id32683d5f506e082d39af269803bccee0280cc27

19 months agoMerge branch 'stable-6.0' into stable-6.1 67/199967/1
Matthias Sohn [Thu, 16 Feb 2023 15:56:07 +0000 (16:56 +0100)]
Merge branch 'stable-6.0' into stable-6.1

* stable-6.0:
  Add pack options to preserve and prune old pack files
  Allow to perform PackedBatchRefUpdate without locking loose refs
  Document option "core.sha1Implementation" introduced in 59029aec

Change-Id: I876a38c2de8b7d5eaacd00e36b85599f88173221

19 months agoMerge branch 'stable-5.13' into stable-6.0 62/199962/2
Matthias Sohn [Thu, 16 Feb 2023 15:42:58 +0000 (16:42 +0100)]
Merge branch 'stable-5.13' into stable-6.0

* stable-5.13:
  Add pack options to preserve and prune old pack files
  Allow to perform PackedBatchRefUpdate without locking loose refs
  Document option "core.sha1Implementation" introduced in 59029aec

Change-Id: I423f410578f5bbe178832b80fef8998a5372182c

19 months agoFix getPackedRefs to not throw NoSuchFileException 59/199959/1
Prudhvi Akhil Alahari [Thu, 16 Feb 2023 11:11:55 +0000 (16:41 +0530)]
Fix getPackedRefs to not throw NoSuchFileException

Since Files.newInputStream is from java.nio package, it throws
java.nio.file.NoSuchFileException. This was missed in the change
I00da88e. Without this change, getPackedRefs fails with
NoSuchFileException when there is no packed-refs file in a project.

Change-Id: I93c202ddb73a0a5979af8e4d09e45f5645664b45
Signed-off-by: Prudhvi Akhil Alahari <quic_prudhvi@quicinc.com>
20 months agoAdd pack options to preserve and prune old pack files 46/199846/2
Matthias Sohn [Fri, 10 Feb 2023 20:05:47 +0000 (21:05 +0100)]
Add pack options to preserve and prune old pack files

Add the options
- pack.preserveOldPacks
- pack.prunePreserved

This allows to configure in git config if old packs should be preserved
during gc and pruned during the next gc.

The original implementation in 91132bb0 only allows to set these options
using the API.

Change-Id: I5b23ab4f317d12f5ccd234401419913e8263cc9a

20 months agoAllow to perform PackedBatchRefUpdate without locking loose refs 58/199758/1
Saša Živkov [Fri, 21 Oct 2022 14:32:03 +0000 (16:32 +0200)]
Allow to perform PackedBatchRefUpdate without locking loose refs

Add another newBatchUpdate method in the RefDirectory where we can
control if the created PackedBatchRefUpdate will lock the loose refs or
not.

This can be useful in cases when we run programs which have exclusive
access to a Git repository and we know that locking loose refs is
unnecessary and just a performance loss.

Change-Id: I7d0932eb1598a3871a2281b1a049021380234df9
(cherry picked from commit cb90ed08526bd51f04e5d72e3ba3cf5bd30c11e4)

20 months agoDocument option "core.sha1Implementation" introduced in 59029aec 11/199711/2
Matthias Sohn [Wed, 1 Feb 2023 13:33:31 +0000 (14:33 +0100)]
Document option "core.sha1Implementation" introduced in 59029aec

Bug: 580310
Change-Id: I10f3d6f6b5af7ab96683994c9cbd85e6c18a5084

20 months agoMerge branch 'stable-6.2' into stable-6.3 86/199686/1
Matthias Sohn [Tue, 31 Jan 2023 23:59:32 +0000 (00:59 +0100)]
Merge branch 'stable-6.2' into stable-6.3

* stable-6.2:
  Shortcut during git fetch for avoiding looping through all local refs
  FetchCommand: fix fetchSubmodules to work on a Ref to a blob
  Silence API warnings introduced by I466dcde6
  Allow the exclusions of refs prefixes from bitmap
  PackWriterBitmapPreparer: do not include annotated tags in bitmap
  BatchingProgressMonitor: avoid int overflow when computing percentage
  Speedup GC listing objects referenced from reflogs
  FileSnapshotTest: Add more MISSING_FILE coverage

Change-Id: I2ff386d9a096277360e6c7bd5535b49984620fb3

20 months agoMerge branch 'stable-6.1' into stable-6.2 85/199685/1
Matthias Sohn [Tue, 31 Jan 2023 23:44:41 +0000 (00:44 +0100)]
Merge branch 'stable-6.1' into stable-6.2

* stable-6.1:
  Shortcut during git fetch for avoiding looping through all local refs
  FetchCommand: fix fetchSubmodules to work on a Ref to a blob
  Silence API warnings introduced by I466dcde6
  Allow the exclusions of refs prefixes from bitmap
  PackWriterBitmapPreparer: do not include annotated tags in bitmap
  BatchingProgressMonitor: avoid int overflow when computing percentage
  Speedup GC listing objects referenced from reflogs
  FileSnapshotTest: Add more MISSING_FILE coverage

Change-Id: Iff2fba026b49463016015b2fae1a42cf76ee2dbb

20 months agoMerge branch 'stable-6.0' into stable-6.1 84/199684/1
Matthias Sohn [Tue, 31 Jan 2023 23:38:52 +0000 (00:38 +0100)]
Merge branch 'stable-6.0' into stable-6.1

* stable-6.0:
  Shortcut during git fetch for avoiding looping through all local refs
  FetchCommand: fix fetchSubmodules to work on a Ref to a blob
  Silence API warnings introduced by I466dcde6
  Allow the exclusions of refs prefixes from bitmap
  PackWriterBitmapPreparer: do not include annotated tags in bitmap
  BatchingProgressMonitor: avoid int overflow when computing percentage
  Speedup GC listing objects referenced from reflogs
  FileSnapshotTest: Add more MISSING_FILE coverage

Change-Id: Ib5055f2f3b8a313c178d6f6c7c5630285ad5a726

20 months agoMerge branch 'stable-5.13' into stable-6.0 83/199683/1
Matthias Sohn [Tue, 31 Jan 2023 23:30:52 +0000 (00:30 +0100)]
Merge branch 'stable-5.13' into stable-6.0

* stable-5.13:
  Shortcut during git fetch for avoiding looping through all local refs
  FetchCommand: fix fetchSubmodules to work on a Ref to a blob
  Silence API warnings introduced by I466dcde6
  Allow the exclusions of refs prefixes from bitmap
  PackWriterBitmapPreparer: do not include annotated tags in bitmap
  BatchingProgressMonitor: avoid int overflow when computing percentage
  Speedup GC listing objects referenced from reflogs
  FileSnapshotTest: Add more MISSING_FILE coverage

Change-Id: I58ad4c210a5e7e5a1ba6b22315b04211c8909950

20 months agoShortcut during git fetch for avoiding looping through all local refs 64/193464/12
Luca Milanesio [Wed, 18 May 2022 12:31:30 +0000 (13:31 +0100)]
Shortcut during git fetch for avoiding looping through all local refs

The FetchProcess needs to verify that all the refs received point
to objects that are reachable from the local refs, which could be
very expensive but is needed to avoid missing objects exceptions
because of broken chains.

When the local repository has a lot of refs (e.g. millions) and the
client is fetching a non-commit object (e.g. refs/sequences/changes in
Gerrit) the reachability check on all local refs can be very expensive
compared to the time to fetch the remote ref.

Example for a 2M refs repository:
- fetching a single non-commit object: 50ms
- checking the reachability of local refs: 30s

A ref pointing to a non-commit object doesn't have any parent or
successor objects, hence would never need to have a reachability check
done. Skipping the askForIsComplete() altogether would save the 30s
time spent in an unnecessary phase.

Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
Change-Id: I09ac66ded45cede199ba30f9e71cc1055f00941b

20 months agoFetchCommand: fix fetchSubmodules to work on a Ref to a blob 23/196223/2
Matthias Sohn [Tue, 4 Oct 2022 13:42:25 +0000 (15:42 +0200)]
FetchCommand: fix fetchSubmodules to work on a Ref to a blob

FetchCommand#fetchSubmodules assumed that FETCH_HEAD can always be
parsed as a tree. This isn't true if it refers to a Ref referring to a
BLOB. This is e.g. used in Gerrit for Refs like refs/sequences/changes
which are used to implement sequences stored in git.

Change-Id: I414f5b7d9f2184b2d7d53af1dfcd68cccb725ca4

20 months agoSilence API warnings introduced by I466dcde6 80/199680/1
Matthias Sohn [Tue, 31 Jan 2023 22:45:07 +0000 (23:45 +0100)]
Silence API warnings introduced by I466dcde6

Change-Id: I510510da34d33757c2f83af8cd1e26f6206a486a

20 months agoAllow the exclusions of refs prefixes from bitmap 76/197776/28
Luca Milanesio [Tue, 20 Dec 2022 21:50:19 +0000 (21:50 +0000)]
Allow the exclusions of refs prefixes from bitmap

When running a GC.repack() against a repository with over one
thousands of refs/heads and tens of millions of ObjectIds,
the calculation of all bitmaps associated with all the refs
would result in an unreasonable big file that would take up to
several hours to compute.

Test scenario: repo with 2500 heads / 10M obj Intel Xeon E5-2680 2.5GHz
Before this change: 20 mins
After this change and 2300 heads excluded: 10 mins (90s for bitmap)

Having such a large bitmap file is also slow in the runtime
processing and have negligible or even negative benefits, because
the time lost in reading and decompressing the bitmap in memory
would not be compensated by the time saved by using it.

It is key to preserve the bitmaps for those refs that are mostly
used in clone/fetch and give the ability to exlude some refs
prefixes that are known to be less frequently accessed, even
though they may actually be actively written.

Example: Gerrit sandbox branches may even be actively
used and selected automatically because its commits are very
recent, however, they may bloat the bitmap, making it ineffective.

A mono-repo with tens of thousands of developers may have
a relatively small number of active branches where the
CI/CD jobs are continuously fetching/cloning the code. However,
because Gerrit allows the use of sandbox branches, the
total number of refs/heads may be even tens to hundred
thousands.

Change-Id: I466dcde69fa008e7f7785735c977f6e150e3b644
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
20 months agoPackWriterBitmapPreparer: do not include annotated tags in bitmap 51/197851/16
Luca Milanesio [Wed, 28 Dec 2022 01:09:52 +0000 (01:09 +0000)]
PackWriterBitmapPreparer: do not include annotated tags in bitmap

The annotated tags should be excluded from the bitmap associated
with the heads-only packfile. However, this was not happening
because of the check of exclusion of the peeled object instead
of the objectId to be excluded from the bitmap.

Sample use-case:

refs/heads/main
  ^
  |
 commit1 <-- commit2 <- annotated-tag1 <- tag1
  ^
  |
 commit0

When creating a bitmap for the above commit graph, before this
change all the commits are included (3 bitmaps), which is
incorrect, because all commits reachable from annotated tags
should not be included.

The heads-only bitmap should include only commit0 and commit1
but because PackWriterBitPreparer was checking for the peeled
pointer of tag1 to be excluded (commit2) which was not found in
the list of tags to exclude (annotated-tag1), the commit2 was
included, even if it wasn't reachable only from the head.

Add an additional check for exclusion of the original objectId
for allowing the exclusion of annotated tags and their pointed
commits. Add one specific test associated with an annotated tag
for making sure that this use-case is covered also.

Example repository benchmark for measuring the improvement:
# refs: 400k (2k heads, 88k tags, 310k changes)
# objects: 11M (88k of them are annotate tags)
# packfiles: 2.7G

Before this change:
GC time: 5h
clone --bare time: 7 mins

After this change:
GC time: 20 mins
clone --bare time: 3 mins

Bug: 581267
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
Change-Id: Iff2bfc6587153001837220189a120ead9ac649dc

20 months agoBatchingProgressMonitor: avoid int overflow when computing percentage 35/199435/4
Matthias Sohn [Mon, 16 Jan 2023 20:58:56 +0000 (21:58 +0100)]
BatchingProgressMonitor: avoid int overflow when computing percentage

When cloning huge repositories I observed percentage of object counts
turning negative. This happened if lastWork * 100 exceeded
Integer.MAX_VALUE.

Change-Id: Ic5f5cf5a911a91338267aace4daba4b873ab3900

20 months agoSpeedup GC listing objects referenced from reflogs 68/199468/3
Matthias Sohn [Wed, 18 Jan 2023 16:39:19 +0000 (17:39 +0100)]
Speedup GC listing objects referenced from reflogs

GC needs to get a ReflogReader for all existing refs to list all objects
referenced from reflogs. The existing Repository#getReflogReader method
accepts the ref name and then resolves the Ref to create a ReflogReader.
GC calling that for a huge number of Refs one by one is very slow. GC
first gets all Refs in bulk and then calls getReflogReader for each of
them.

Fix this by adding another getReflogReader method to Repository which
accepts a Ref directly.

This speeds up running JGit gc on a mirror clone of the Gerrit
repository from 15:36 min to 1:08 min. The repository used in this test
had 45k refs, 275k commits and 1.2m git objects.

Change-Id: I474897fdc6652923e35d461c065a29f54d9949f4

21 months agoMerge branch 'stable-6.2' into stable-6.3 03/199403/1
Matthias Sohn [Fri, 13 Jan 2023 18:32:06 +0000 (19:32 +0100)]
Merge branch 'stable-6.2' into stable-6.3

* stable-6.2:
  Cache trustFolderStat/trustPackedRefsStat value per-instance
  Refresh 'objects' dir and retry if a loose object is not found

Change-Id: Ibc9bffab8c9ef9c39384b53c142d99878f7f3f98

21 months agoMerge branch 'stable-6.1' into stable-6.2 02/199402/1
Matthias Sohn [Fri, 13 Jan 2023 18:31:18 +0000 (19:31 +0100)]
Merge branch 'stable-6.1' into stable-6.2

* stable-6.1:
  Cache trustFolderStat/trustPackedRefsStat value per-instance
  Refresh 'objects' dir and retry if a loose object is not found

Change-Id: I9e876f72f735f58bf02c7862a3d8e657fc46a7b9

21 months agoCache trustFolderStat/trustPackedRefsStat value per-instance 25/199325/3
Nasser Grainawi [Tue, 10 Jan 2023 23:15:42 +0000 (16:15 -0700)]
Cache trustFolderStat/trustPackedRefsStat value per-instance

Instead of re-reading the config every time the methods using these
values were called, cache the config value at the time of instance
construction. Caching the values improves performance for each of the
method calls. These configs are set based on the filesystem storing the
repository and unlikely to change while an application is running.

Change-Id: I1cae26dad672dd28b766ac532a871671475652df
Signed-off-by: Nasser Grainawi <quic_nasserg@quicinc.com>
21 months agoRefresh 'objects' dir and retry if a loose object is not found 14/197714/6
Kaushik Lingarkar [Thu, 12 Jan 2023 05:25:30 +0000 (21:25 -0800)]
Refresh 'objects' dir and retry if a loose object is not found

A new loose object may not be immediately visible on a NFS
client if it was created on another client. Refreshing the
'objects' dir and trying again can help work around the NFS
behavior.

Here's an E2E problem that this change can help fix. Consider
a Gerrit multi-primary setup with repositories based on NFS.
Add a new patch-set to an existing change and then immediately
fetch the new patch-set of that change. If the fetch is handled
by a Gerrit primary different that the one which created the
patch-set, then we sometimes run into a MissingObjectException
that causes the fetch to fail.

Bug: 581317
Change-Id: Iccc6676c68ef13a1e8b2ff52b3eeca790a89a13d
Signed-off-by: Kaushik Lingarkar <quic_kaushikl@quicinc.com>
21 months agoFileSnapshotTest: Add more MISSING_FILE coverage 72/199272/1
Nasser Grainawi [Fri, 6 Jan 2023 21:14:48 +0000 (14:14 -0700)]
FileSnapshotTest: Add more MISSING_FILE coverage

Add a couple tests that confirm what the docs say about isModified() and
equals(MISSING_FILE) behavior.

Change-Id: I6093040ba3594934c3270331405a44b2634b97c5
Signed-off-by: Nasser Grainawi <quic_nasserg@quicinc.com>
21 months agoMerge branch 'stable-6.2' into stable-6.3 50/199250/1
Matthias Sohn [Thu, 5 Jan 2023 14:56:42 +0000 (15:56 +0100)]
Merge branch 'stable-6.2' into stable-6.3

* stable-6.2:
  Introduce core.trustPackedRefsStat config
  Fix documentation for core.trustFolderStat

Change-Id: I48b6c095ac62dc859829d6fef45325accbb0a144

21 months agoMerge branch 'stable-6.1' into stable-6.2 49/199249/1
Matthias Sohn [Thu, 5 Jan 2023 14:53:34 +0000 (15:53 +0100)]
Merge branch 'stable-6.1' into stable-6.2

* stable-6.1:
  Introduce core.trustPackedRefsStat config
  Fix documentation for core.trustFolderStat

Change-Id: Ic78630f74c72624932a384eed52ef79ae1eff3e5

21 months agoIntroduce core.trustPackedRefsStat config 26/197526/7
Kaushik Lingarkar [Fri, 2 Dec 2022 21:21:02 +0000 (13:21 -0800)]
Introduce core.trustPackedRefsStat config

Currently, we always read packed-refs file when 'trustFolderStat'
is false. Introduce a new config 'trustPackedRefsStat' which takes
precedence over 'trustFolderStat' when reading packed refs. Possible
values for this new config are:

* always: Trust packed-refs file attributes
* after_open: Same as 'always', but refresh the file attributes of
              packed-refs before trusting it
* never: Always read the packed-refs file
* unset: Fallback to 'trustFolderStat' to determine if the file
  attributes of packed-refs can be trusted

Folks whose repositories are on NFS and have traditionally been
setting 'trustFolderStat=false' can now get some performance improvement
with 'trustPackedRefsStat=after_open' as it refreshes the file
attributes of packed-refs (at least on some NFS clients) before
considering it.

For example, consider a repository on NFS with ~500k packed-refs. Here
are some stats which illustrate the improvement with this new config
when reading packed refs on NFS:

trustFolderStat=true trustPackedRefsStat=unset: 0.2ms
trustFolderStat=false trustPackedRefsStat=unset: 155ms
trustFolderStat=false trustPackedRefsStat=after_open: 1.5ms

Change-Id: I00da88e4cceebbcf3475be0fc0011ff65767c111
Signed-off-by: Kaushik Lingarkar <quic_kaushikl@quicinc.com>
22 months agoFix documentation for core.trustFolderStat 92/197492/5
Kaushik Lingarkar [Mon, 5 Dec 2022 22:58:00 +0000 (14:58 -0800)]
Fix documentation for core.trustFolderStat

Update documentation for core.trustFolderStat to highlight that it is
also used when reading the packed-refs file.

Change-Id: I3eac377c3a7f48493abc8ae6d0889ee70a05d24d
Signed-off-by: Kaushik Lingarkar <quic_kaushikl@quicinc.com>
22 months agoMerge branch 'stable-6.2' into stable-6.3 43/197143/1
Matthias Sohn [Sun, 20 Nov 2022 19:22:24 +0000 (20:22 +0100)]
Merge branch 'stable-6.2' into stable-6.3

* stable-6.2:
  Remove unused imports
  Suppress non-externalized String warnings
  Remove unused API problem filters
  Silence API errors
  Silence API errors
  Silence API warnings

Change-Id: I71aa9f890c5eb05849ad16a00b9974da5e51171e

22 months agoRemove unused imports 42/197142/1
Matthias Sohn [Sun, 20 Nov 2022 19:18:01 +0000 (20:18 +0100)]
Remove unused imports

Change-Id: I57a3ae0a8608a517d9cc5cc27cee136f9faeb810

22 months agoSuppress non-externalized String warnings 41/197141/1
Matthias Sohn [Sun, 20 Nov 2022 19:16:28 +0000 (20:16 +0100)]
Suppress non-externalized String warnings

Change-Id: I95c61bd906bd3f7cdea9581e0ffa92ec607c6140

22 months agoRemove unused API problem filters 40/197140/1
Matthias Sohn [Sun, 20 Nov 2022 19:14:20 +0000 (20:14 +0100)]
Remove unused API problem filters

Change-Id: I890716bc052dcce2ce136040035b442247c08332

22 months agoMerge branch 'stable-6.1' into stable-6.2 39/197139/1
Matthias Sohn [Sun, 20 Nov 2022 19:10:37 +0000 (20:10 +0100)]
Merge branch 'stable-6.1' into stable-6.2

* stable-6.1:
  Silence API errors
  Silence API errors
  Silence API warnings

Change-Id: Id3efa1b32710661cde63e93c5b8832785a805415

22 months agoSilence API errors 38/197138/1
Matthias Sohn [Sun, 20 Nov 2022 19:09:02 +0000 (20:09 +0100)]
Silence API errors

Change-Id: I07c42fe9417edb0570dd475a7e935112a878a93b

22 months agoMerge branch 'stable-6.0' into stable-6.1 37/197137/1
Matthias Sohn [Sun, 20 Nov 2022 19:08:42 +0000 (20:08 +0100)]
Merge branch 'stable-6.0' into stable-6.1

* stable-6.0:
  Silence API errors
  Silence API warnings

Change-Id: I2b8336652e60dec97666582cf9331c8505729473

22 months agoSilence API errors 36/197136/1
Matthias Sohn [Sun, 20 Nov 2022 18:55:22 +0000 (19:55 +0100)]
Silence API errors

Change-Id: Ie112b2099ea2125bc85863524e56f09ba4907373

22 months agoMerge branch 'stable-5.13' into stable-6.0 35/197135/1
Matthias Sohn [Sun, 20 Nov 2022 18:54:44 +0000 (19:54 +0100)]
Merge branch 'stable-5.13' into stable-6.0

* stable-5.13:
  Silence API warnings

Change-Id: If5ab988a0e177c37b125e0b10625e506eeb2a74f

22 months agoSilence API warnings 34/197134/1
Matthias Sohn [Sun, 20 Nov 2022 18:45:54 +0000 (19:45 +0100)]
Silence API warnings

introduced by
- addition of configurable SHA1 implementation in 5.13.2
- 3-digit @since 5.9.1 annotations on GitServlet methods

Change-Id: If19853fcc5e3677e5b18e8e3fbbcd2773378dffc

23 months agoMerge branch 'stable-6.2' into stable-6.3 08/197008/1
Matthias Sohn [Wed, 16 Nov 2022 09:14:13 +0000 (10:14 +0100)]
Merge branch 'stable-6.2' into stable-6.3

* stable-6.2:
  Extract Exception -> HTTP status code mapping for reuse
  Don't handle internal git errors as an HTTP error
  Allow to perform PackedBatchRefUpdate without locking loose refs

Change-Id: I562be0802efa231023c5f10e6461339b2d7fbacf

23 months agoMerge branch 'stable-6.1' into stable-6.2 07/197007/1
Matthias Sohn [Wed, 16 Nov 2022 09:13:20 +0000 (10:13 +0100)]
Merge branch 'stable-6.1' into stable-6.2

* stable-6.1:
  Extract Exception -> HTTP status code mapping for reuse
  Don't handle internal git errors as an HTTP error
  Allow to perform PackedBatchRefUpdate without locking loose refs

Change-Id: Icb321779184d20f3871e236fda1a3acba605a6da

23 months agoMerge "Merge branch 'stable-6.0' into stable-6.1" into stable-6.1
Matthias Sohn [Wed, 16 Nov 2022 09:10:43 +0000 (04:10 -0500)]
Merge "Merge branch 'stable-6.0' into stable-6.1" into stable-6.1

23 months agoMerge branch 'stable-6.2' into stable-6.3 04/197004/1
Matthias Sohn [Wed, 16 Nov 2022 08:56:08 +0000 (09:56 +0100)]
Merge branch 'stable-6.2' into stable-6.3

* stable-6.2:
  [benchmarks] Remove profiler configuration
  Add SHA1 benchmark
  [benchmarks] Set version of maven-compiler-plugin to 3.8.1
  Fix running JMH benchmarks
  Add option to allow using JDK's SHA1 implementation
  Ignore IllegalStateException if JVM is already shutting down

Change-Id: I9c1576011c11b4ff8f453d18d9e786cee59860fa

23 months agoMerge branch 'stable-6.1' into stable-6.2 03/197003/1
Matthias Sohn [Wed, 16 Nov 2022 08:55:22 +0000 (09:55 +0100)]
Merge branch 'stable-6.1' into stable-6.2

* stable-6.1:
  [benchmarks] Remove profiler configuration
  Add SHA1 benchmark
  [benchmarks] Set version of maven-compiler-plugin to 3.8.1
  Fix running JMH benchmarks
  Add option to allow using JDK's SHA1 implementation
  Ignore IllegalStateException if JVM is already shutting down

Change-Id: Ie433c46a01a0f33848d54ecf99b30a44ca01e286

23 months agoMerge branch 'stable-6.0' into stable-6.1 02/197002/1
Matthias Sohn [Wed, 16 Nov 2022 08:54:28 +0000 (09:54 +0100)]
Merge branch 'stable-6.0' into stable-6.1

* stable-6.0:
  [benchmarks] Remove profiler configuration
  Add SHA1 benchmark
  [benchmarks] Set version of maven-compiler-plugin to 3.8.1
  Fix running JMH benchmarks
  Add option to allow using JDK's SHA1 implementation
  Ignore IllegalStateException if JVM is already shutting down

Change-Id: I176419026c3f4fdd8ebd34c61468c1ec3482ff45

23 months agoMerge branch 'stable-5.13' into stable-6.0 01/197001/1
Matthias Sohn [Tue, 15 Nov 2022 23:15:17 +0000 (00:15 +0100)]
Merge branch 'stable-5.13' into stable-6.0

* stable-5.13:
  [benchmarks] Remove profiler configuration
  Add SHA1 benchmark
  [benchmarks] Set version of maven-compiler-plugin to 3.8.1
  Fix running JMH benchmarks
  Add option to allow using JDK's SHA1 implementation
  Ignore IllegalStateException if JVM is already shutting down

Change-Id: I40105336f0b9e593a8a2c242a9557f854c274fdc

23 months ago[benchmarks] Remove profiler configuration 42/196942/7
Matthias Sohn [Mon, 14 Nov 2022 22:23:55 +0000 (23:23 +0100)]
[benchmarks] Remove profiler configuration

Profiler configuration can be added when required. It was commented out
in most benchmarks.

Change-Id: I725f98757f7d4d2ba2589658e34e2fd6fbbbedee

23 months agoAdd SHA1 benchmark 06/196906/13
Matthias Sohn [Tue, 15 Nov 2022 12:22:12 +0000 (13:22 +0100)]
Add SHA1 benchmark

Results on a Mac M1 max:

    size     SHA1Native SHA1Java    SHA1Java
                        without     with
                        collision   collision
                        detection   detection
    [kB]     [us/op]    [us/op]     [us/op]
---------------------------------------------
      1       3.662       4.200       4.707
      2       7.053       7.868       8.928
      4      13.883      15.149      17.608
      8      27.225      30.049      35.237
     16      54.014      59.655      70.867
     32     106.457     118.022     140.403
     64     212.712     237.702     281.522
   1024    3469.519    3868.883    4637.287
 131072  445011.724  501751.992  604061.308
1048576 3581702.104 4008087.854 4831023.563

The last 3 sizes (1, 128, 1024 MB) weren't committed
here to limit the total runtime.

Bug: 580310
Change-Id: I7d0382fd4aa4c4734806b12e96b671bee37d26e3

23 months ago[benchmarks] Set version of maven-compiler-plugin to 3.8.1 41/196941/6
Matthias Sohn [Mon, 14 Nov 2022 21:25:40 +0000 (22:25 +0100)]
[benchmarks] Set version of maven-compiler-plugin to 3.8.1

Change-Id: Ib14db133c76a55358ea79663ef38d9fb47a67f45

23 months agoFix running JMH benchmarks 25/196225/9
Matthias Sohn [Tue, 4 Oct 2022 13:45:01 +0000 (15:45 +0200)]
Fix running JMH benchmarks

Without this I sometimes hit the error:

$ java -jar target/benchmarks.jar
Exception in thread "main" java.lang.RuntimeException: ERROR: Unable to
find the resource: /META-INF/BenchmarkList
at org.openjdk.jmh.runner.AbstractResourceReader.getReaders(AbstractResourceReader.java:98)
at org.openjdk.jmh.runner.BenchmarkList.find(BenchmarkList.java:124)
at org.openjdk.jmh.runner.Runner.internalRun(Runner.java:253)
at org.openjdk.jmh.runner.Runner.run(Runner.java:209)
at org.openjdk.jmh.Main.main(Main.java:71)

Change-Id: Iea9431d5f332f799d55a8a2d178c79a2ef0da22b

23 months agoAdd option to allow using JDK's SHA1 implementation 05/196905/9
Matthias Sohn [Fri, 11 Nov 2022 16:54:06 +0000 (17:54 +0100)]
Add option to allow using JDK's SHA1 implementation

The change If6da9833 moved the computation of SHA1 from the JVM's
JCE to a pure Java implementation with collision detection.
The extra security for public sites comes with a cost of slower
SHA1 processing compared to the native implementation in the JDK.

When JGit is used internally and not exposed to any traffic from
external or untrusted users, the extra cost of the pure Java SHA1
implementation can be avoided, falling back to the previous
native MessageDigest implementation.

Bug: 580310
Change-Id: Ic24c0ba1cb0fb6282b8ca3025ffbffa84035565e

23 months agoFix API breakage caused by extracting WorkTreeUpdater 10/196910/2
Matthias Sohn [Mon, 14 Nov 2022 09:45:58 +0000 (10:45 +0100)]
Fix API breakage caused by extracting WorkTreeUpdater

448052dc2e made WorkTreeUpdater package visible which breaks API for
subclasses of ResolveMerger since they cannot access WorkTreeUpdater.

Fix this by moving WorkTreeUpdater into ResolveMerger as a nested class
and mark it protected so that subclasses can use it.

Bug: 581049
Change-Id: I5a2c4953f8514dc0a1b8041c8e069d28370bb2eb

23 months agoExtract Exception -> HTTP status code mapping for reuse 52/196852/3
Sven Selberg [Wed, 9 Nov 2022 17:28:45 +0000 (18:28 +0100)]
Extract Exception -> HTTP status code mapping for reuse

Extract private static method UploadPackServlet#statusCodeForThrowable
to a public static method in the UploadPackErrorHandler interface so
that implementers of this interface can reuse the default mapping.

Change-Id: Ie4a0a006b0148d5b828d610c55d19ce407aab055

23 months agoDon't handle internal git errors as an HTTP error 49/196849/4
Sven Selberg [Wed, 9 Nov 2022 15:58:17 +0000 (16:58 +0100)]
Don't handle internal git errors as an HTTP error

The fix that fixed the propagation of error-codes:
  8984e1f66 HTTP Smart: set correct HTTP status on error [1]
made some faulty assumptions.

"Wants not valid", can be an intermittent git error and the HTTP
response should be 200 and not 400 since the request isn't necessary
faulty.

[1] https://git.eclipse.org/r/c/jgit/jgit/+/192677

Bug: 579676
Change-Id: I461bc78ff6e450636811ece50d21c57a2a7f2ae3

23 months agoIgnore IllegalStateException if JVM is already shutting down 40/196540/1
Matthias Sohn [Thu, 27 Oct 2022 18:31:31 +0000 (20:31 +0200)]
Ignore IllegalStateException if JVM is already shutting down

Trying to register/unregister a shutdown hook when the JVM is already in
shutdown throws an IllegalStateException. Ignore this exception since we
can't do anything about it.

Bug: 580953
Change-Id: I8fc6fdd5585837c81ad0ebd6944430856556d90e

23 months agoAllow to perform PackedBatchRefUpdate without locking loose refs 10/196510/3
Saša Živkov [Fri, 21 Oct 2022 14:32:03 +0000 (16:32 +0200)]
Allow to perform PackedBatchRefUpdate without locking loose refs

Add another newBatchUpdate method in the RefDirectory where we can
control if the created PackedBatchRefUpdate will lock the loose refs or
not.

This can be useful in cases when we run programs which have exclusive
access to a Git repository and we know that locking loose refs is
unnecessary and just a performance loss.

Change-Id: I7d0932eb1598a3871a2281b1a049021380234df9

2 years agoRemove unused imports 91/196091/1
Matthias Sohn [Wed, 28 Sep 2022 15:58:50 +0000 (17:58 +0200)]
Remove unused imports

Change-Id: Ib6f80a5f013d30a9b7a59cb05e3ee13aa46b28c8

2 years agoPrepare 6.3.1-SNAPSHOT builds 22/195722/1
Matthias Sohn [Mon, 12 Sep 2022 08:09:10 +0000 (10:09 +0200)]
Prepare 6.3.1-SNAPSHOT builds

Change-Id: I44e159eca4131880d74d3078060e7e20f9b5ce76

2 years agoJGit v6.3.0.202209071007-r 36/195636/1 v6.3.0.202209071007-r
Matthias Sohn [Wed, 7 Sep 2022 14:07:11 +0000 (16:07 +0200)]
JGit v6.3.0.202209071007-r

Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Change-Id: Iea3fae9f6f6c5fb0a79f7684334a3e0059738c4f