Anna Papitto [Tue, 20 Jun 2023 22:44:30 +0000 (15:44 -0700)]
PackReverseIndex: separate out the computed implementation
PackReverseIndex is a concrete class whose implementation is computed
from a pack's forward index. Callers which have a reverse index file may
want to use an implementation that is file-based instead.
Generalize PackReverseIndex into an interface without
implementation-specific logic and separate out the logic for the
computed implementation into a new concrete class.
Change-Id: I98d9835363c5e1c8c3c11a81b0761af3cdeaa41a Signed-off-by: Anna Papitto <annapapitto@google.com>
Thomas Wolf [Thu, 15 Jun 2023 20:08:06 +0000 (22:08 +0200)]
Default for global (user) git ignore file
C git has a default for git config core.excludesfile: "Its default
value is $XDG_CONFIG_HOME/git/ignore. If $XDG_CONFIG_HOME is either
not set or empty, $HOME/.config/git/ignore is used instead." [1]
Implement this in the WorkingTreeIterator$RootIgnoreNode.
To make this testable, mock the "user.home" directory for all JGit
tests, otherwise tests might pick up a real user's git ignore file.
Also ensure that JGit code always reads "user.home" via the
SystemReader.
Antoine Musso [Wed, 31 May 2023 15:57:28 +0000 (17:57 +0200)]
Fix all Javadoc warnings and fail on them
This fixes all the javadoc warnings, stops ignoring doclint 'missing'
category and fails the build on javadoc warnings for public and
protected classes and class members.
Since javadoc doesn't allow access specifiers when specifying doclint
configuration we cannot set `-Xdoclint:all,-missing/private`
hence there is no simple way to skip private elements from doclint.
Therefore we check javadoc using the Eclipse Java compiler
(which is used by default) and javadoc configuration in
`.settings/org.eclipse.jdt.core.prefs` files.
This allows more fine grained configuration.
We can reconsider this when javadoc starts supporting access specifiers
in the doclint configuration.
Below are detailled explanations for most modifications.
@inheritDoc
===========
doclint complains about explicits `{@inheritDoc}` when the parent does
not have any documentation. As far as I can tell, javadoc defaults to
inherit comments and should only be used when one wants to append extra
documentation from the parent. Given the parent has no documentation,
remove those usages which doclint complains about.
In some case I have moved up the documentation from the concrete class
up to the abstract class.
Remove `{@inheritDoc}` on overriden methods which don't add additional
documentation since javadoc defaults to inherit javadoc of overridden
methods.
@value to @link
===============
In PackConfig, DEFAULT_SEARCH_FOR_REUSE_TIMEOUT and similar are forged
from Integer.MAX_VALUE and are thus not considered constants (I guess
cause the value would depends on the platform). Replace it with a link
to `Integer.MAX_VALUE`.
In `StringUtils.toBoolean`, @value was used to refer to the
`stringValue` parameter. I have replaced it with `{@code stringValue}`.
{@link <url>} to <a>
====================
@link does not support being given an external URL. Replaces them with
HTML `<a>`.
@since: being invalid
=====================
org.eclipse.jgit/src/org/eclipse/jgit/util/Equality.java has an invalid
tag `@since: ` due to the extra `:`. Javadoc does not complain about it
with version 11.0.18+10 but does with 11.0.19.7. It is invalid
regardless.
invalid HTML syntax
===================
- javadoc doesn't allow <br/>, <p/> and </p> anymore, use <br> and <p>
instead
- replace <tt>code</tt> by {@code code}
- <table> tags don't allow summary attribute, specify caption as
<caption>caption</caption> to fix this
doclint visibility issue
========================
In the private abstract classes `BaseDirCacheEditor` and
`BasePackConnection` links to other methods in the abstract class are
inherited in the public subclasses but doclint gets confused and
considers them unreachable. The HTML documentation for the sub classes
shows the relative links in the sub classes, so it is all correct. It
must be a bug somewhere in javadoc.
Mute those warnings with: @SuppressWarnings("doclint:missing")
Misc
====
Replace `<` and `>` with HTML encoded entities (`< and `>`).
In `SshConstants` I went enclosing a serie of -> arrows in @literal.
Additional tags
===============
Configure maven-javad0c-plugin to allow the following additional tags
defined in https://openjdk.org/jeps/8068562:
- apiNote
- implSpec
- implNote
Missing javadoc
===============
Add missing @params and descriptions
Change-Id: I840056389aa59135cfb360da0d5e40463ce35bd0 Also-By: Matthias Sohn <matthias.sohn@sap.com>
Reason for revert: This change was based on the false claim that the
packedrefs file lock is held while the CAS is being done, but it is
actually released before the CAS (the in memory lock is still held,
however that does not prevent external actors from updating the
packedrefs files and then another thread from subsequently re-reading it
and updating the in memory packedRefList). Although reverting this
change can cause the CAS to fail, it should not actually matter since
the failure would indicate that another thread has already updated the
in memory packedRefList to either the same version this thread was
trying to update it too, or to a more recent version. Either way,
failing the CAS is then appropriate and should not be problematic.
Although this change reverts the code in the RefDirectory class, it
keeps the "improvements" to the test so that it continues to pass
reliably. The reason for the quotes around the word "improvements" is
because I believe the test alteration actually dramatically changes the
intent of the test, and that the original intent of the test is
untestable with the GC and RefDirectory classes as is.
Bug: 582044
Change-Id: I3acee7527bb542996dcdfaddfb2bdb45ec444db5 Signed-off-by: Martin Fick <quic_mfick@quicinc.com>
(cherry picked from commit c5617711a1b4d5d0807cc7eed702b78d114d46b3)
Anna Papitto [Tue, 30 May 2023 14:20:54 +0000 (16:20 +0200)]
PackReverseIndex: use static builder instead of constructor
PackReverseIndex instances are created using the constructor directly,
which limits control over the construction logic and refactoring
opportunities for the class itself. These will be needed for a
file-based implementation of the reverse index.
Use a static builder method to create a PackReverseIndex instance using
a pack's forward index.
Change-Id: I4421d907cd61d9ac932df5377e5e28a81679b63f Signed-off-by: Anna Papitto <annapapitto@google.com>
Anna Papitto [Tue, 30 May 2023 14:20:54 +0000 (16:20 +0200)]
Gc#writePack: write the reverse index file to disk
The reverse index is currently created in-memory when needed. A writer
for reverse index files was already implemented.
Make garbage collection write the reverse index file when the PackConfig
enables it. Write it during #writePack, which mirrors how the primary
index is written.
Change-Id: I50131af6622c41a7b24534aaaf2a423ab4178981 Signed-off-by: Anna Papitto <annapapitto@google.com>
Matthias Sohn [Wed, 24 May 2023 13:50:27 +0000 (15:50 +0200)]
Merge branch 'master' into stable-6.6
* master:
GraphObjectIndex: fix search in findGraphPosition
Update to Tycho 4.0.0-SNAPSHOT
PGP sign p2 artefacts
Revert 'Use net.i2p.crypto:eddsa directly from Maven Central'
Update dash license-tool-plugin to 1.0.2
Also add suppressed exception if unchecked exception occurs in finally
Candidate: use "Objects.equals" instead of "=="
Use hamcrest 2.2 directly from Maven Central
Use commons-logging directly from Maven Central
Update jna to 5.13.0
Use bytebuddy directly from Maven Central
Use jna directly from Maven Central
Use net.i2p.crypto:eddsa directly from Maven Central
Use org.tukaani:xz directly from Maven Central
Use args4j directly from Maven Central
Use gson directly from Maven Central
Remove unused $NON-NLS-1$
Remove unused API filters
Switch to Apache MINA sshd 2.10.0
[releng] API filter for PackIndex.DEFAULT_WRITE_REVERSE_INDEX
PackExt: add a #getTmpExtension method
UploadPack: Record negotiation stats on fetchV2 call
RewriteGeneratorTest: Introduce test cases for the RewriteGenerator
PackWriter: write the PackReverseIndex file
Jonathan Tan [Mon, 8 May 2023 20:55:40 +0000 (13:55 -0700)]
GraphObjectIndex: fix search in findGraphPosition
In findGraphPosition, when there is no object whose OID starts with
the first byte of the sought OID, low equals high. This violates an
invariant of the loop, and when the sought OID is lexicographically
greater than every other OID in the repository, causes an
ArrayIndexOutOfBoundsException (because we're trying to read outside the
list of OIDs).
Therefore, check the "low < high" condition at the start of the loop,
not only after the first iteration.
Change-Id: Ic8ac198c151bd161c4996b9e7cb6e6660f151733 Helped-by: Ivan Frade <ifrade@google.com> Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Matthias Sohn [Tue, 9 May 2023 07:44:03 +0000 (09:44 +0200)]
Update to Tycho 4.0.0-SNAPSHOT
We need to update to Tycho in order to force PGP signing of the
bouncycastle libraries which isn't supported by earlier Tycho versions.
For that we need to run Maven on Java 17 or higher.
In order to run tests on Java 11 add a `toolchain.xml` file into the
`~/.m2` directory providing the path to Java installations:
Reason: the maven artifact has a broken MANIFEST.MF with a mandatory
dependency to sun.security.x509, which is an internal package in the
JDK and moreover not needed by the bundle except for one test class
that isn't in the bundle at all.
This extra dependency makes the JGit tycho packaging build fail when
Tycho 4 is used.
We must keep using the Orbit re-packaging of this artifact, which does
not have this unnecessary mandatory dependency.
Change-Id: Ica15a5ddcada09686de3055b2b3daf081e3c5ffc Signed-off-by: Thomas Wolf <twolf@apache.org>
Matthias Sohn [Thu, 18 May 2023 10:02:48 +0000 (12:02 +0200)]
Also add suppressed exception if unchecked exception occurs in finally
If a method called in a finally block throws an exception we should add
exceptions caught earlier to the exception we throw in the finally block
not regarding if it's a checked or unchecked exception.
Thomas Wolf [Mon, 3 Apr 2023 17:33:24 +0000 (19:33 +0200)]
Switch to Apache MINA sshd 2.10.0
Bump the version numbers in pom.xml and in MANIFESTs, and in the bazel
WORKSPACE file. Update the target platforms. Remove work-arounds in
org.eclipse.jgit.ssh.apache that are no longer necessary.
The release notes for Apache MINA sshd are at [1].
Anna Papitto [Tue, 2 May 2023 20:45:05 +0000 (13:45 -0700)]
PackWriter: write the PackReverseIndex file
PackWriter offers the ability to write out the pack file and its various
index files, except for the newly introduced file-based reverse index.
Now that PackReverseIndexWriter can write reverse index files,
PackWriter#writeReverseIndex will write one for a pack if the
corresponding config flag PackConfig#writeReverseIndex is on.
Change-Id: Ib75dd2bbfb9ee9366d5aacb46700d8cf8af4823a Signed-off-by: Anna Papitto <annapapitto@google.com>
Matthias Sohn [Wed, 3 May 2023 01:01:03 +0000 (03:01 +0200)]
Update Maven plugins
- com.github.siom79.japicmp:japicmp-maven-plugin to 0.17.2
- com.github.spotbugs:spotbugs-maven-plugin to 4.7.3.4
- maven-clean-plugin to 3.2.0
- maven-compiler-plugin to 3.11.0
- maven-deploy-plugin to 3.1.1
- maven-enforcer-plugin to 3.3.0
- maven-javadoc-plugin to 3.5.0
- maven-project-info-reports-plugin to 3.4.3
- maven-resources-plugin to 3.3.1
- maven-surefire-plugin to 3.0.0
- maven-surefire-report-plugin to 3.0.0
- org.codehaus.mojo:build-helper-maven-plugin to 3.3.0
- org.jacoco:jacoco-maven-plugin to 0.8.10
- org.springframework.boot:spring-boot-maven-plugin to 2.7.11
Nasser Grainawi [Tue, 2 May 2023 22:30:44 +0000 (16:30 -0600)]
Fix inProcessPackedRefsLock not shared with copies of the instance
The in process lock is intended to manage contention on locking the
packed-refs file within a single process without acquiring the file
system lock. Not sharing it across RefDirectory instances of the same
repository undermines that intent and results in more contention at the
file system level.
Matthias Sohn [Mon, 1 May 2023 09:38:36 +0000 (11:38 +0200)]
Update javaEWAH to 1.2.3 and use it directly from Maven central
This changes its BundleSymbolicName from "javaewah" (name in Orbit) to
com.googlecode.javaewah.JavaEWAH (name in upstream artefact from Maven
Central).
Thomas Wolf [Sun, 23 Apr 2023 19:34:37 +0000 (21:34 +0200)]
Support rebasing independent branches
With completely independent branches, there is no merge base. In this
case, the list of commits must include the root commit of the branch to
be rebased.
Bug: 581832
Change-Id: I0f5bdf179d5b07ff09f1a274d61c7a0b1c0011c6 Signed-off-by: Thomas Wolf <twolf@apache.org>
Thomas Wolf [Sun, 23 Apr 2023 19:31:40 +0000 (21:31 +0200)]
Support cherry-picking a root commit
Handle the case of the commit to be picked not having any parents.
Since JGit implements cherry-pick as a 3-way-merge between the commit
to be picked and the target commit, using the parent of the picked
commit as merge base, this is super simple: just don't set a base tree.
The merger will not find any merge base and will supply an empty tree
iterator for the base.
Bug: 581832
Change-Id: I88985f1b1723db5b35ce58bf228bc48d23d6fca3 Signed-off-by: Thomas Wolf <twolf@apache.org>
Thomas Wolf [Thu, 20 Apr 2023 19:13:59 +0000 (21:13 +0200)]
AddCommand: ability to switch off renormalization
JGit's AddCommand always renormalizes tracked files. C git does so only
on git add --renormalize. Especially for git add . and the JGit
equivalent git.add().addFilepattern(".").call() this can make a big
difference if there are many files, or large files.
Add a "renormalize" option to AddCommand. To maintain compatibility with
existing uses, this option is "true" by default, and the behavior of
AddCommand is as it has always been in JGit.
If set to "false", use an IndexDiffFilter (in addition to a path filter,
if any). This skips any unchanged files (that are not racily clean) from
content checks. Note that changes in CRLF settings or in filters will be
ignored for such files if renormalize == false.
Add the "--renormalize" option to the Add command in the JGit command
line program. For the command-line program, the default is as in C git:
renormalize is off by default and enabled only if the option is given.
Note that --renormalize implies --update in the command line program, as
in C git. In AddCommand, the two settings are independent.
Additionally, avoid opening input streams unnecessarily in
WorkingTreeIterator.getEntryContentLength() and fix some bogus
indentation.
Add a simple test that adds 1000 files of 10kB in 10 directories twice
and that fails if the second invocation (without any changes) with
renormalize=false is not significantly faster.
Matthias Sohn [Sun, 16 Apr 2023 22:46:03 +0000 (00:46 +0200)]
Update bouncycastle to 1.73
Review requests were created for
maven/mavencentral/org.bouncycastle/bcpkix-jdk18on/1.73
https://gitlab.eclipse.org/eclipsefdn/emo-team/iplab/-/issues/7892
maven/mavencentral/org.bouncycastle/bcprov-jdk18on/1.73
https://gitlab.eclipse.org/eclipsefdn/emo-team/iplab/-/issues/7893
maven/mavencentral/org.bouncycastle/bcutil-jdk18on/1.73
https://gitlab.eclipse.org/eclipsefdn/emo-team/iplab/-/issues/7894
Matthias Sohn [Fri, 28 Apr 2023 17:51:01 +0000 (19:51 +0200)]
Merge branch 'stable-6.5'
* stable-6.5:
[bazel] Move ToolTestCase to src folder (6.2)
GcConcurrentTest: @Ignore flaky testInterruptGc
Fix CommitTemplateConfigTest
Fix after_open config and Snapshotting RefDir tests to work with bazel
[bazel] Skip ConfigTest#testCommitTemplatePathInHomeDirecory
Demote severity of some error prone bug patterns to warnings
Parse pull.rebase=preserve as alias for pull.rebase=merges
UploadPack: Fix NPE when traversing a tag chain
Anna Papitto [Thu, 27 Apr 2023 18:01:30 +0000 (11:01 -0700)]
PackReverseIndexWriter: write out version 1 reverse index file
The reverse index for a pack is used to quickly find an object's
position in the pack's forward index based on that object's pack offset.
It is currently computed from the forward index by sorting the index
entries by the corresponding pack offset. This computation uses
bucket sort with insertion sort, which has an average runtime of
O(n log n) and worst case runtime of O(n^2); and memory usage of
3*size(int)*n because it maintains 3 int arrays, even after sorting is
completed. The computation must be performed every time that the reverse
index object is created in memory.
In contrast, Cgit persists a pack reverse index file to avoid
recomputing the reverse index ordering every time that it is needed.
Instead they write a file with format
https://git-scm.com/docs/pack-format#_pack_rev_files_have_the_format
which can later be read and parsed into an in-memory reverse index each
time it is needed.
Introduce these reverse index files to JGit. PackReverseIndexWriter
writes out a reverse index file to be read later when needed. Subclass
PackReverseIndexWriterV1 writes a file with the official version 1
format.
To avoid temporarily allocating an Integer collection while sorting and
writing out the contents, using memory 4*size(Integer)*n, use an
IntList and its #sort method, which uses quicksort.
Change-Id: I6437745777a16f723e2f1cfcce4e0d94e599dcee Signed-off-by: Anna Papitto <annapapitto@google.com>
Anna Papitto [Thu, 27 Apr 2023 18:01:29 +0000 (11:01 -0700)]
IntList: add #sort using quick sort for O(n log n) runtime.
IntList is a class for working with lists of primitive ints without
boxing them into Integers. For writing the reverse index file format,
sorting ints will be needed but IntList doesn't provide a sorting
method yet.
Add the #sort method to sort an IntList by an IntComparator, using
quicksort, which has a average runtime of O(n log n) and sorts in-place
by using O(log n) stack frames for recursive calls.
Change-Id: Id69a687c8a16d46b13b28783b194a880f3f4c437 Signed-off-by: Anna Papitto <annapapitto@google.com>
Matthias Sohn [Thu, 27 Apr 2023 00:30:20 +0000 (02:30 +0200)]
Merge branch 'stable-6.4' into stable-6.5
* stable-6.4:
[bazel] Move ToolTestCase to src folder (6.2)
GcConcurrentTest: @Ignore flaky testInterruptGc
Fix CommitTemplateConfigTest
Fix after_open config and Snapshotting RefDir tests to work with bazel
[bazel] Skip ConfigTest#testCommitTemplatePathInHomeDirecory
Demote severity of some error prone bug patterns to warnings
UploadPack: Fix NPE when traversing a tag chain
Matthias Sohn [Thu, 27 Apr 2023 00:20:10 +0000 (02:20 +0200)]
Merge branch 'stable-6.3' into stable-6.4
* stable-6.3:
[bazel] Move ToolTestCase to src folder (6.2)
GcConcurrentTest: @Ignore flaky testInterruptGc
Fix CommitTemplateConfigTest
Fix after_open config and Snapshotting RefDir tests to work with bazel
[bazel] Skip ConfigTest#testCommitTemplatePathInHomeDirecory
Demote severity of some error prone bug patterns to warnings
UploadPack: Fix NPE when traversing a tag chain
Matthias Sohn [Thu, 27 Apr 2023 00:07:23 +0000 (02:07 +0200)]
Merge branch 'stable-6.2' into stable-6.3
* stable-6.2:
[bazel] Move ToolTestCase to src folder (6.2)
GcConcurrentTest: @Ignore flaky testInterruptGc
Fix CommitTemplateConfigTest
Fix after_open config and Snapshotting RefDir tests to work with bazel
[bazel] Skip ConfigTest#testCommitTemplatePathInHomeDirecory
Demote severity of some error prone bug patterns to warnings
UploadPack: Fix NPE when traversing a tag chain
Matthias Sohn [Mon, 24 Apr 2023 16:50:10 +0000 (18:50 +0200)]
[bazel] Move ToolTestCase to src folder (6.2)
Bazel barks at the abstract ToolTestCase not containing any test. Move
it from the tst/ source folder to the src/ source folder so that bazel
knows this is a helper class which doesn't contain tests.
Matthias Sohn [Wed, 26 Apr 2023 23:48:07 +0000 (01:48 +0200)]
Merge branch 'stable-6.1' into stable-6.2
* stable-6.1:
GcConcurrentTest: @Ignore flaky testInterruptGc
Fix CommitTemplateConfigTest
Fix after_open config and Snapshotting RefDir tests to work with bazel
[bazel] Skip ConfigTest#testCommitTemplatePathInHomeDirecory
Demote severity of some error prone bug patterns to warnings
UploadPack: Fix NPE when traversing a tag chain
Jonathan Tan [Wed, 5 Apr 2023 20:44:59 +0000 (13:44 -0700)]
GcConcurrentTest: @Ignore flaky testInterruptGc
During my development of Id7721cc5b7ea650e77c2db47042715487983cae6, I
have found this test to be flaky when run by CI. As a speculative fix,
mark this test as @Ignore so it won't be run.
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Change-Id: Idfe04d7f1fb72a772d4c8d249ca86a9c2eec0b1a
projects / jgit.git / log