Reason: the maven artifact has a broken MANIFEST.MF with a mandatory
dependency to sun.security.x509, which is an internal package in the
JDK and moreover not needed by the bundle except for one test class
that isn't in the bundle at all.
This extra dependency makes the JGit tycho packaging build fail when
Tycho 4 is used.
We must keep using the Orbit re-packaging of this artifact, which does
not have this unnecessary mandatory dependency.
Change-Id: Ica15a5ddcada09686de3055b2b3daf081e3c5ffc Signed-off-by: Thomas Wolf <twolf@apache.org>
Matthias Sohn [Thu, 18 May 2023 10:02:48 +0000 (12:02 +0200)]
Also add suppressed exception if unchecked exception occurs in finally
If a method called in a finally block throws an exception we should add
exceptions caught earlier to the exception we throw in the finally block
not regarding if it's a checked or unchecked exception.
Thomas Wolf [Mon, 3 Apr 2023 17:33:24 +0000 (19:33 +0200)]
Switch to Apache MINA sshd 2.10.0
Bump the version numbers in pom.xml and in MANIFESTs, and in the bazel
WORKSPACE file. Update the target platforms. Remove work-arounds in
org.eclipse.jgit.ssh.apache that are no longer necessary.
The release notes for Apache MINA sshd are at [1].
Anna Papitto [Tue, 2 May 2023 20:45:05 +0000 (13:45 -0700)]
PackWriter: write the PackReverseIndex file
PackWriter offers the ability to write out the pack file and its various
index files, except for the newly introduced file-based reverse index.
Now that PackReverseIndexWriter can write reverse index files,
PackWriter#writeReverseIndex will write one for a pack if the
corresponding config flag PackConfig#writeReverseIndex is on.
Change-Id: Ib75dd2bbfb9ee9366d5aacb46700d8cf8af4823a Signed-off-by: Anna Papitto <annapapitto@google.com>
Matthias Sohn [Wed, 3 May 2023 01:01:03 +0000 (03:01 +0200)]
Update Maven plugins
- com.github.siom79.japicmp:japicmp-maven-plugin to 0.17.2
- com.github.spotbugs:spotbugs-maven-plugin to 4.7.3.4
- maven-clean-plugin to 3.2.0
- maven-compiler-plugin to 3.11.0
- maven-deploy-plugin to 3.1.1
- maven-enforcer-plugin to 3.3.0
- maven-javadoc-plugin to 3.5.0
- maven-project-info-reports-plugin to 3.4.3
- maven-resources-plugin to 3.3.1
- maven-surefire-plugin to 3.0.0
- maven-surefire-report-plugin to 3.0.0
- org.codehaus.mojo:build-helper-maven-plugin to 3.3.0
- org.jacoco:jacoco-maven-plugin to 0.8.10
- org.springframework.boot:spring-boot-maven-plugin to 2.7.11
Nasser Grainawi [Tue, 2 May 2023 22:30:44 +0000 (16:30 -0600)]
Fix inProcessPackedRefsLock not shared with copies of the instance
The in process lock is intended to manage contention on locking the
packed-refs file within a single process without acquiring the file
system lock. Not sharing it across RefDirectory instances of the same
repository undermines that intent and results in more contention at the
file system level.
Matthias Sohn [Mon, 1 May 2023 09:38:36 +0000 (11:38 +0200)]
Update javaEWAH to 1.2.3 and use it directly from Maven central
This changes its BundleSymbolicName from "javaewah" (name in Orbit) to
com.googlecode.javaewah.JavaEWAH (name in upstream artefact from Maven
Central).
Thomas Wolf [Sun, 23 Apr 2023 19:34:37 +0000 (21:34 +0200)]
Support rebasing independent branches
With completely independent branches, there is no merge base. In this
case, the list of commits must include the root commit of the branch to
be rebased.
Bug: 581832
Change-Id: I0f5bdf179d5b07ff09f1a274d61c7a0b1c0011c6 Signed-off-by: Thomas Wolf <twolf@apache.org>
Thomas Wolf [Sun, 23 Apr 2023 19:31:40 +0000 (21:31 +0200)]
Support cherry-picking a root commit
Handle the case of the commit to be picked not having any parents.
Since JGit implements cherry-pick as a 3-way-merge between the commit
to be picked and the target commit, using the parent of the picked
commit as merge base, this is super simple: just don't set a base tree.
The merger will not find any merge base and will supply an empty tree
iterator for the base.
Bug: 581832
Change-Id: I88985f1b1723db5b35ce58bf228bc48d23d6fca3 Signed-off-by: Thomas Wolf <twolf@apache.org>
Thomas Wolf [Thu, 20 Apr 2023 19:13:59 +0000 (21:13 +0200)]
AddCommand: ability to switch off renormalization
JGit's AddCommand always renormalizes tracked files. C git does so only
on git add --renormalize. Especially for git add . and the JGit
equivalent git.add().addFilepattern(".").call() this can make a big
difference if there are many files, or large files.
Add a "renormalize" option to AddCommand. To maintain compatibility with
existing uses, this option is "true" by default, and the behavior of
AddCommand is as it has always been in JGit.
If set to "false", use an IndexDiffFilter (in addition to a path filter,
if any). This skips any unchanged files (that are not racily clean) from
content checks. Note that changes in CRLF settings or in filters will be
ignored for such files if renormalize == false.
Add the "--renormalize" option to the Add command in the JGit command
line program. For the command-line program, the default is as in C git:
renormalize is off by default and enabled only if the option is given.
Note that --renormalize implies --update in the command line program, as
in C git. In AddCommand, the two settings are independent.
Additionally, avoid opening input streams unnecessarily in
WorkingTreeIterator.getEntryContentLength() and fix some bogus
indentation.
Add a simple test that adds 1000 files of 10kB in 10 directories twice
and that fails if the second invocation (without any changes) with
renormalize=false is not significantly faster.
Matthias Sohn [Sun, 16 Apr 2023 22:46:03 +0000 (00:46 +0200)]
Update bouncycastle to 1.73
Review requests were created for
maven/mavencentral/org.bouncycastle/bcpkix-jdk18on/1.73
https://gitlab.eclipse.org/eclipsefdn/emo-team/iplab/-/issues/7892
maven/mavencentral/org.bouncycastle/bcprov-jdk18on/1.73
https://gitlab.eclipse.org/eclipsefdn/emo-team/iplab/-/issues/7893
maven/mavencentral/org.bouncycastle/bcutil-jdk18on/1.73
https://gitlab.eclipse.org/eclipsefdn/emo-team/iplab/-/issues/7894
Matthias Sohn [Fri, 28 Apr 2023 17:51:01 +0000 (19:51 +0200)]
Merge branch 'stable-6.5'
* stable-6.5:
[bazel] Move ToolTestCase to src folder (6.2)
GcConcurrentTest: @Ignore flaky testInterruptGc
Fix CommitTemplateConfigTest
Fix after_open config and Snapshotting RefDir tests to work with bazel
[bazel] Skip ConfigTest#testCommitTemplatePathInHomeDirecory
Demote severity of some error prone bug patterns to warnings
Parse pull.rebase=preserve as alias for pull.rebase=merges
UploadPack: Fix NPE when traversing a tag chain
Anna Papitto [Thu, 27 Apr 2023 18:01:30 +0000 (11:01 -0700)]
PackReverseIndexWriter: write out version 1 reverse index file
The reverse index for a pack is used to quickly find an object's
position in the pack's forward index based on that object's pack offset.
It is currently computed from the forward index by sorting the index
entries by the corresponding pack offset. This computation uses
bucket sort with insertion sort, which has an average runtime of
O(n log n) and worst case runtime of O(n^2); and memory usage of
3*size(int)*n because it maintains 3 int arrays, even after sorting is
completed. The computation must be performed every time that the reverse
index object is created in memory.
In contrast, Cgit persists a pack reverse index file to avoid
recomputing the reverse index ordering every time that it is needed.
Instead they write a file with format
https://git-scm.com/docs/pack-format#_pack_rev_files_have_the_format
which can later be read and parsed into an in-memory reverse index each
time it is needed.
Introduce these reverse index files to JGit. PackReverseIndexWriter
writes out a reverse index file to be read later when needed. Subclass
PackReverseIndexWriterV1 writes a file with the official version 1
format.
To avoid temporarily allocating an Integer collection while sorting and
writing out the contents, using memory 4*size(Integer)*n, use an
IntList and its #sort method, which uses quicksort.
Change-Id: I6437745777a16f723e2f1cfcce4e0d94e599dcee Signed-off-by: Anna Papitto <annapapitto@google.com>
Anna Papitto [Thu, 27 Apr 2023 18:01:29 +0000 (11:01 -0700)]
IntList: add #sort using quick sort for O(n log n) runtime.
IntList is a class for working with lists of primitive ints without
boxing them into Integers. For writing the reverse index file format,
sorting ints will be needed but IntList doesn't provide a sorting
method yet.
Add the #sort method to sort an IntList by an IntComparator, using
quicksort, which has a average runtime of O(n log n) and sorts in-place
by using O(log n) stack frames for recursive calls.
Change-Id: Id69a687c8a16d46b13b28783b194a880f3f4c437 Signed-off-by: Anna Papitto <annapapitto@google.com>
Matthias Sohn [Thu, 27 Apr 2023 00:30:20 +0000 (02:30 +0200)]
Merge branch 'stable-6.4' into stable-6.5
* stable-6.4:
[bazel] Move ToolTestCase to src folder (6.2)
GcConcurrentTest: @Ignore flaky testInterruptGc
Fix CommitTemplateConfigTest
Fix after_open config and Snapshotting RefDir tests to work with bazel
[bazel] Skip ConfigTest#testCommitTemplatePathInHomeDirecory
Demote severity of some error prone bug patterns to warnings
UploadPack: Fix NPE when traversing a tag chain
Matthias Sohn [Thu, 27 Apr 2023 00:20:10 +0000 (02:20 +0200)]
Merge branch 'stable-6.3' into stable-6.4
* stable-6.3:
[bazel] Move ToolTestCase to src folder (6.2)
GcConcurrentTest: @Ignore flaky testInterruptGc
Fix CommitTemplateConfigTest
Fix after_open config and Snapshotting RefDir tests to work with bazel
[bazel] Skip ConfigTest#testCommitTemplatePathInHomeDirecory
Demote severity of some error prone bug patterns to warnings
UploadPack: Fix NPE when traversing a tag chain
Matthias Sohn [Thu, 27 Apr 2023 00:07:23 +0000 (02:07 +0200)]
Merge branch 'stable-6.2' into stable-6.3
* stable-6.2:
[bazel] Move ToolTestCase to src folder (6.2)
GcConcurrentTest: @Ignore flaky testInterruptGc
Fix CommitTemplateConfigTest
Fix after_open config and Snapshotting RefDir tests to work with bazel
[bazel] Skip ConfigTest#testCommitTemplatePathInHomeDirecory
Demote severity of some error prone bug patterns to warnings
UploadPack: Fix NPE when traversing a tag chain
Matthias Sohn [Mon, 24 Apr 2023 16:50:10 +0000 (18:50 +0200)]
[bazel] Move ToolTestCase to src folder (6.2)
Bazel barks at the abstract ToolTestCase not containing any test. Move
it from the tst/ source folder to the src/ source folder so that bazel
knows this is a helper class which doesn't contain tests.
Matthias Sohn [Wed, 26 Apr 2023 23:48:07 +0000 (01:48 +0200)]
Merge branch 'stable-6.1' into stable-6.2
* stable-6.1:
GcConcurrentTest: @Ignore flaky testInterruptGc
Fix CommitTemplateConfigTest
Fix after_open config and Snapshotting RefDir tests to work with bazel
[bazel] Skip ConfigTest#testCommitTemplatePathInHomeDirecory
Demote severity of some error prone bug patterns to warnings
UploadPack: Fix NPE when traversing a tag chain
Jonathan Tan [Wed, 5 Apr 2023 20:44:59 +0000 (13:44 -0700)]
GcConcurrentTest: @Ignore flaky testInterruptGc
During my development of Id7721cc5b7ea650e77c2db47042715487983cae6, I
have found this test to be flaky when run by CI. As a speculative fix,
mark this test as @Ignore so it won't be run.
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Change-Id: Idfe04d7f1fb72a772d4c8d249ca86a9c2eec0b1a
Fix after_open config and Snapshotting RefDir tests to work with bazel
The changes I1db6fcf414b and I634b92877f added tests which were failing
with errors [1] and [2] with "bazel test //...". This was not caught
because we don't have CI running with bazel. Fix bazel build file so
that these errors are no longer thrown when run with bazel.
[1] error: cannot find symbol FileRepositoryBuilderTest
[2] error: cannot find symbol RefDirectoryTest
Matthias Sohn [Wed, 26 Apr 2023 19:55:16 +0000 (21:55 +0200)]
Merge branch 'stable-6.0' into stable-6.1
* stable-6.0:
[bazel] Skip ConfigTest#testCommitTemplatePathInHomeDirecory
Demote severity of some error prone bug patterns to warnings
UploadPack: Fix NPE when traversing a tag chain
Move this test to another class and skip it when running tests with
bazel since the bazel test runner does not allow to create files in the
home directory.
FS#userHome retrieves the home directory on the first call and caches it
for subsequent calls to avoid overhead in case path translation is
required (currently on cygwin). This prevents that the test can mock the
home directory using MockSystemReader like SshTestHarness does.
Always parse RevTags including their body before getting their object
to ensure that non-cached objects are handled correctly when traversing
a tag chain. An NPE in UploadPack#addTagChain will occur on a depth=1
fetch of a branch containing a tag chain and the ref to one of the
middle tags in the chain is deleted.
Matthias Sohn [Thu, 20 Apr 2023 22:52:18 +0000 (00:52 +0200)]
Merge branch 'stable-6.5'
* stable-6.5:
Add missing since tag for SshBasicTestBase
Add missing since tag for SshTestHarness#publicKey2
Silence API errors
Prevent infinite loop rescanning the pack list on PackMismatchException
Remove blank in maven.config
Matthias Sohn [Thu, 20 Apr 2023 22:40:02 +0000 (00:40 +0200)]
Merge branch 'stable-6.4' into stable-6.5
* stable-6.4:
Add missing since tag for SshBasicTestBase
Add missing since tag for SshTestHarness#publicKey2
Silence API errors
Prevent infinite loop rescanning the pack list on
PackMismatchException
Remove blank in maven.config
Matthias Sohn [Thu, 20 Apr 2023 22:33:26 +0000 (00:33 +0200)]
Merge branch 'stable-6.3' into stable-6.4
* stable-6.3:
Add missing since tag for SshBasicTestBase
Add missing since tag for SshTestHarness#publicKey2
Silence API errors
Prevent infinite loop rescanning the pack list on PackMismatchException
Remove blank in maven.config
Matthias Sohn [Thu, 20 Apr 2023 22:25:51 +0000 (00:25 +0200)]
Merge branch 'stable-6.2' into stable-6.3
* stable-6.2:
Add missing since tag for SshBasicTestBase
Add missing since tag for SshTestHarness#publicKey2
Silence API errors
Prevent infinite loop rescanning the pack list on PackMismatchException
Remove blank in maven.config
Matthias Sohn [Thu, 20 Apr 2023 22:19:38 +0000 (00:19 +0200)]
Merge branch 'stable-6.1' into stable-6.2
* stable-6.1:
Add missing since tag for SshBasicTestBase
Add missing since tag for SshTestHarness#publicKey2
Silence API errors
Prevent infinite loop rescanning the pack list on PackMismatchException
Remove blank in maven.config
Matthias Sohn [Thu, 20 Apr 2023 22:11:40 +0000 (00:11 +0200)]
Merge branch 'stable-6.0' into stable-6.1
* stable-6.0:
Add missing since tag for SshBasicTestBase
Add missing since tag for SshTestHarness#publicKey2
Silence API errors
Prevent infinite loop rescanning the pack list on PackMismatchException
Remove blank in maven.config
Matthias Sohn [Thu, 20 Apr 2023 14:01:33 +0000 (16:01 +0200)]
Merge branch 'stable-5.13' into stable-6.0
* stable-5.13:
Add missing since tag for SshBasicTestBase
Add missing since tag for SshTestHarness#publicKey2
Silence API errors
Prevent infinite loop rescanning the pack list on
PackMismatchException
Remove blank in maven.config
Matthias Sohn [Thu, 20 Apr 2023 13:40:36 +0000 (15:40 +0200)]
Merge branch 'stable-5.12' into stable-5.13
* stable-5.12:
Add missing since tag for SshBasicTestBase
Add missing since tag for SshTestHarness#publicKey2
Silence API errors
Prevent infinite loop rescanning the pack list on
PackMismatchException
Remove blank in maven.config
Matthias Sohn [Thu, 20 Apr 2023 13:12:01 +0000 (15:12 +0200)]
Merge branch 'stable-5.11' into stable-5.12
* stable-5.11:
Add missing since tag for SshBasicTestBase
Add missing since tag for SshTestHarness#publicKey2
Silence API errors
Prevent infinite loop rescanning the pack list on PackMismatchException
Remove blank in maven.config
Matthias Sohn [Thu, 20 Apr 2023 12:42:56 +0000 (14:42 +0200)]
Merge branch 'stable-5.10' into stable-5.11
* stable-5.10:
Add missing since tag for SshTestHarness#publicKey2
Silence API errors
Prevent infinite loop rescanning the pack list on
PackMismatchException
Remove blank in maven.config
Migrated "Prevent infinite loop rescanning the pack list on
PackMismatchException" to refactoring done in
https://git.eclipse.org/r/q/topic:restore-preserved-packs
Matthias Sohn [Thu, 30 Mar 2023 11:43:17 +0000 (13:43 +0200)]
Prevent infinite loop rescanning the pack list on PackMismatchException
We found, when analysing an incident where Gerrit's gc runner thread got
stuck, that we can end up in an infinite loop in
ObjectDirectory#openPackedObject which tries to rescan the pack
list and starts over trying to open a packed object in an unconfined
loop if it catches a PackMismatchException.
Here the relevant part of a thread dump we created while the gc runner
was stuck:
"WorkQueue-2[java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@350812a3[Not
completed,
task = java.util.concurrent.Executors$RunnableAdapter@5425d7ee]]" #72
tid=0x00007f73cee1c800 nid=0x584
runnable [0x00007f7392d57000]
java.lang.Thread.State: RUNNABLE
at org.eclipse.jgit.internal.storage.file.WindowCache.removeAll(WindowCache.java:716)
at org.eclipse.jgit.internal.storage.file.WindowCache.purge(WindowCache.java:399)
at org.eclipse.jgit.internal.storage.file.PackFile.close(PackFile.java:296)
at org.eclipse.jgit.internal.storage.file.ObjectDirectory.reuseMap(ObjectDirectory.java:973)
at org.eclipse.jgit.internal.storage.file.ObjectDirectory.scanPacksImpl(ObjectDirectory.java:904)
at org.eclipse.jgit.internal.storage.file.ObjectDirectory.scanPacks(ObjectDirectory.java:895)
- locked <0x000000050a498f60> (a
java.util.concurrent.atomic.AtomicReference)
at org.eclipse.jgit.internal.storage.file.ObjectDirectory.searchPacksAgain(ObjectDirectory.java:794)
at org.eclipse.jgit.internal.storage.file.ObjectDirectory.openPackedObject(ObjectDirectory.java:465)
at org.eclipse.jgit.internal.storage.file.ObjectDirectory.openPackedFromSelfOrAlternate(ObjectDirectory.java:417)
at org.eclipse.jgit.internal.storage.file.ObjectDirectory.openObject(ObjectDirectory.java:408)
at org.eclipse.jgit.internal.storage.file.WindowCursor.open(WindowCursor.java:132)
at org.eclipse.jgit.lib.ObjectReader$1.open(ObjectReader.java:279)
at org.eclipse.jgit.revwalk.RevWalk$2.next(RevWalk.java:1031)
at org.eclipse.jgit.internal.storage.pack.PackWriter.findObjectsToPack(PackWriter.java:1911)
at org.eclipse.jgit.internal.storage.pack.PackWriter.preparePack(PackWriter.java:960)
at org.eclipse.jgit.internal.storage.pack.PackWriter.preparePack(PackWriter.java:876)
at org.eclipse.jgit.internal.storage.file.GC.writePack(GC.java:1168)
at org.eclipse.jgit.internal.storage.file.GC.repack(GC.java:852)
at org.eclipse.jgit.internal.storage.file.GC.doGc(GC.java:269)
at org.eclipse.jgit.internal.storage.file.GC.gc(GC.java:220)
at org.eclipse.jgit.api.GarbageCollectCommand.call(GarbageCollectCommand.java:179)
at com.google.gerrit.server.git.GarbageCollection.run(GarbageCollection.java:112)
at com.google.gerrit.server.git.GarbageCollection.run(GarbageCollection.java:75)
at com.google.gerrit.server.git.GarbageCollection.run(GarbageCollection.java:71)
at com.google.gerrit.server.git.GarbageCollectionRunner.run(GarbageCollectionRunner.java:76)
at com.google.gerrit.server.logging.LoggingContextAwareRunnable.run(LoggingContextAwareRunnable.java:103)
at java.util.concurrent.Executors$RunnableAdapter.call(java.base@11.0.18/Executors.java:515)
at java.util.concurrent.FutureTask.runAndReset(java.base@11.0.18/FutureTask.java:305)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(java.base@11.0.18/ScheduledThreadPoolExecutor.java:305)
at com.google.gerrit.server.git.WorkQueue$Task.run(WorkQueue.java:612)
at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.18/ThreadPoolExecutor.java:1128)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.18/ThreadPoolExecutor.java:628)
at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)
The code in ObjectDirectory#openPackedObject [1] apparently assumes that
this is caused by a transient problem which it can resume from by
retrying. We use `core.trustFolderStat = false` on this server since it
uses NFS. The incident we had showed that we can enter into an infinite
loop here if there is a permanent mismatch between a pack file and its
corresponding pack index. I am not yet sure how this can happen.
Break the infinite loop by limiting the number of attempts rescanning
the pack list to 5 retries. When we exceed this threshold set the type
of the PackMismatchException to permanent and rethrow it which breaks
the infinite loop.
Also apply the same limit in #getPackedObjectSize
and #selectObjectRepresentation where we use similar retry loops.
Pat Patterson [Thu, 6 Apr 2023 15:05:56 +0000 (08:05 -0700)]
Add protocol configuration to Amazon S3 transport
Before this change, attempting to use the jgit Amazon S3 transport with an S3-compatible service that requires https (for example, Backblaze B2) results in an error:
$ jgit push b2
fatal: amazon-s3://metadaddy-jgit/repos/test/objects: error in packed-refs
This change adds a "protocol" property to the Amazon S3 transport configuration, defaulting to http, and uses that value when constructing the URL for the S3 service.
kylezhao [Tue, 9 Nov 2021 12:03:27 +0000 (20:03 +0800)]
RevWalk: use generation number to optimize getMergedInto()
A commit A can reach a commit B only if the generation number of A is
strictly larger than the generation number of B. This condition allows
significantly short-circuiting commit-graph walks.
On a copy of the Linux repository where HEAD is contained in v6.3-rc4
but no earlier tag, the command 'git tag --contains HEAD' of
ListTagCommand#call() had the following peformance improvement:
(excluded the startup time of the repo)
Ivan Frade [Tue, 11 Apr 2023 19:50:52 +0000 (12:50 -0700)]
DfsPackFile: Extract block aligment code
Loading of pack, bitmap and commit-graph copy the same code to adjust
the input stream buffering.
Extract to a common function. Besides reusing the code, the name hints
what it is doing.
This block aligment seems unnecessary as the reading is from storage
not dfs cache. The channel probably knows better. Left a TODO because
I don't know the original intention.