Anna Papitto [Thu, 10 Nov 2022 22:48:22 +0000 (14:48 -0800)]
Gc#deleteOrphans: avoid dependence on PackExt alphabetical ordering
Deleting orphan files depends on .pack and .keep being reverse-sorted
to before the corresponding index files that could be orphans. The new
reverse index file extension (.rev) will break that frail dependency.
Rewrite Gc#deleteOrphans to avoid that dependence by tracking which pack
names have a .pack or .keep file and then deleting any index files that
without a corresponding one. This approach takes linear time instead of
the O(n logn) time needed for sorting.
Change-Id: If83c378ea070b8871d4b01ae008e7bf8270de763 Signed-off-by: Anna Papitto <annapapitto@google.com>
Xing Huang [Fri, 9 Dec 2022 22:15:50 +0000 (16:15 -0600)]
PackExt: Add a commit graph extension.
There is no commit graph PackExt because the non-DFS stack is not writing using PackExt mechanism. The extension is needed in DFS to determine the stream to write the commit-graph.
Add a commit graph extension that matches the one in cgit
(https://git-scm.com/docs/commit-graph#_file_layout)
in preparation for adding DFS support for reading and writing commit graphs.
Sergey [Wed, 7 Dec 2022 11:49:47 +0000 (15:49 +0400)]
BatchRefUpdate: Consistent switch branches in ref update
The expression RefUpdate ru = newUpdate(cmd) is eagerly evaluated before the switch statement.
But it is not used in some switch cases and thus is calculated uselessly.
Move expression evaluation to the switch case where it is actually used.
After such a move, several cases became identical and thus were squashed.
Sergey [Wed, 7 Dec 2022 11:49:47 +0000 (15:49 +0400)]
RefWriter#writePackedRefs: Remove a redundant "if" check
After checking the variable, the same variable was checked again inside
the "if" block, and after the first check, this variable does not
change. Remove the second unnecessary check.
Dmitrii Filippov [Mon, 27 Jun 2022 18:03:53 +0000 (20:03 +0200)]
Fix crashes on rare combination of file names
The NameConflictTreeWalk class is used in merge for iterating over
entries in commits. The class uses a separate iterator for each
commit's tree. In rare cases it can incorrectly report the same entry
twice. As a result, duplicated entries are added to the merge result
and later jgit throws an exception when it tries to process merge
result.
The problem appears only when there is a directory-file conflict for
the last item in trees. Example from the bug:
Commit 1:
* subtree - file
* subtree-0 - file
Commit 2:
* subtree - directory
* subtree-0 - file
Here the names are ordered like this:
"subtree" file <"subtree-0" file < "subtree" directory.
The NameConflictTreeWalk handles similar cases correctly if there are
other files after subtree... in commits - this is processed in the
AbstractTreeIterator.min function. Existing code has a special
optimization for the case, when all trees are pointed to the same
entry name - it skips additional checks. However, this optimization
incorrectly skips checks if one of trees reached the end.
The fix processes a situation when some trees reached the end, while
others are still point to an entry.
Matthias Sohn [Wed, 23 Nov 2022 15:54:34 +0000 (16:54 +0100)]
Merge branch 'master' into stable-6.4
* master:
[pgm] Add options --name-only, --name-status to diff, log, show
Update Orbit to R20221123021534 for 2022-12
RBE: Update toolchain with bazel-toolchains 5.1.2 release
SshTestGitServer: : ensure UploadPack is closed to fix resource leak
UploadPackTest: ensure UploadPack is closed to fix resource leak
[pgm] Ensure UploadPack is closed to fix resource leak
UploadPackServlet#doPost use try-with-resource to ensure up is closed
Fix warnings in PatchApplierTest
Fix boxing warnings in TransportTest
Silence warnings about unclosed BasePackPushConnection
Fix warning about non-externalized String
Remove unused imports
Suppress non-externalized String warnings
Remove unused API problem filters
Silence API errors
Silence API errors
Silence API warnings
Add 4.26 target platform
Use "releases" repository for 4.25 target platform
Update Apache Mina SSHD to 2.9.2
Update Orbit to S20221118032057
DfsBlockCache: Report IndexEventConsumer metrics for reverse indexes.
DfsStreamKey: Replace ForReverseIndex to separate metrics.
RawText.isBinary(): handle complete buffer correctly
PackExt: Add a reverse index extension.
David Ostrovsky [Fri, 11 Nov 2022 10:18:38 +0000 (11:18 +0100)]
RBE: Update toolchain with bazel-toolchains 5.1.2 release
Due to this platform style migration: [1] the RBE toolchain needs to be
updated to use the latest rbe_config_gen from bazel-toolchains (at least
version 5.1.2 so that it contains: [2]).
This change makes RBE build forwards compatible so that Bazel could be
updated to the upcoming major 6.0 release.
Matthias Sohn [Sun, 20 Nov 2022 19:24:14 +0000 (20:24 +0100)]
Merge branch 'stable-6.3'
* stable-6.3:
Remove unused imports
Suppress non-externalized String warnings
Remove unused API problem filters
Silence API errors
Silence API errors
Silence API warnings
Matthias Sohn [Sun, 20 Nov 2022 19:22:24 +0000 (20:22 +0100)]
Merge branch 'stable-6.2' into stable-6.3
* stable-6.2:
Remove unused imports
Suppress non-externalized String warnings
Remove unused API problem filters
Silence API errors
Silence API errors
Silence API warnings
Thomas Wolf [Mon, 25 Apr 2022 05:52:25 +0000 (07:52 +0200)]
Update Apache Mina SSHD to 2.9.2
Release notes for 2.9.2:
https://github.com/apache/mina-sshd/blob/master/docs/changes/2.9.2.md
Change-Id: I7809bcba1d45b76ab9dcc031f86beb2f69da3788 Signed-off-by: Thomas Wolf <twolf@apache.org> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Fri, 18 Nov 2022 21:09:04 +0000 (22:09 +0100)]
Update Orbit to S20221118032057
update
- com.jcraft.jsch to 0.1.55.v20221112-0806
- net.bytebuddy.byte-buddy to 1.12.18.v20221114-2102
- net.bytebuddy.byte-buddy-agent to 1.12.18.v20221114-2102
- org.apache.commons.codec to 1.14.0.v20221112-0806
- org.apache.httpcomponents.httpclient to 4.5.13.v20221112-0806
- org.slf4j.api to 1.7.30.v20221112-0806
- org.slf4j.binding.simple to 1.7.30.v20221112-0806
Anna Papitto [Fri, 11 Nov 2022 18:44:37 +0000 (10:44 -0800)]
DfsStreamKey: Replace ForReverseIndex to separate metrics.
Keys used for identifying reverse indexes in the DfsBlockCache use a
custom subclass ForReverseIndex because there was no PackExt for them.
This conflates BlockCacheMetrics for reverse indexes with those for
packs, since the key falls back onto 0 when there is no extension.
Replace the custom ForReverseIndex with a DfsStreamKey usage to bring
keys for the new REVERSE_INDEX extension in line with INDEX and BITMAP
and separate reverse index and pack BlockCacheMetrics.
Change-Id: I305e2c16d2a8cb2a824855ea92e0c9a9b188fce5 Signed-off-by: Anna Papitto <annapapitto@google.com>
Matthias Sohn [Wed, 16 Nov 2022 10:29:24 +0000 (11:29 +0100)]
Merge branch 'master' into stable-6.4
* master:
[benchmarks] Remove profiler configuration
Add SHA1 benchmark
[benchmarks] Set version of maven-compiler-plugin to 3.8.1
Fix running JMH benchmarks
Add option to allow using JDK's SHA1 implementation
Fix API breakage caused by extracting WorkTreeUpdater
Update Orbit to S20221109014815
Use replace instead of replaceAll in toCleanString
Extract Exception -> HTTP status code mapping for reuse
Don't handle internal git errors as an HTTP error
Fix the path for JSchText.properties
Fix Maven SHA1 for Bazel build
UploadPack: Receive and parse client session-id
TransferConfig: Move reading advertisesid setting into TransferConfig
FirstWant: Parse client session-id if received.
ReceivePack: Receive and parse client session-id.
Ignore IllegalStateException if JVM is already shutting down
Allow to perform PackedBatchRefUpdate without locking loose refs
Matthias Sohn [Wed, 16 Nov 2022 09:15:30 +0000 (10:15 +0100)]
Merge branch 'stable-6.3'
* stable-6.3:
[benchmarks] Remove profiler configuration
Add SHA1 benchmark
[benchmarks] Set version of maven-compiler-plugin to 3.8.1
Fix running JMH benchmarks
Add option to allow using JDK's SHA1 implementation
Fix API breakage caused by extracting WorkTreeUpdater
Extract Exception -> HTTP status code mapping for reuse
Don't handle internal git errors as an HTTP error
Ignore IllegalStateException if JVM is already shutting down
Allow to perform PackedBatchRefUpdate without locking loose refs
Matthias Sohn [Wed, 16 Nov 2022 09:14:13 +0000 (10:14 +0100)]
Merge branch 'stable-6.2' into stable-6.3
* stable-6.2:
Extract Exception -> HTTP status code mapping for reuse
Don't handle internal git errors as an HTTP error
Allow to perform PackedBatchRefUpdate without locking loose refs
Matthias Sohn [Wed, 16 Nov 2022 09:13:20 +0000 (10:13 +0100)]
Merge branch 'stable-6.1' into stable-6.2
* stable-6.1:
Extract Exception -> HTTP status code mapping for reuse
Don't handle internal git errors as an HTTP error
Allow to perform PackedBatchRefUpdate without locking loose refs
Matthias Sohn [Wed, 16 Nov 2022 08:56:08 +0000 (09:56 +0100)]
Merge branch 'stable-6.2' into stable-6.3
* stable-6.2:
[benchmarks] Remove profiler configuration
Add SHA1 benchmark
[benchmarks] Set version of maven-compiler-plugin to 3.8.1
Fix running JMH benchmarks
Add option to allow using JDK's SHA1 implementation
Ignore IllegalStateException if JVM is already shutting down
Matthias Sohn [Wed, 16 Nov 2022 08:55:22 +0000 (09:55 +0100)]
Merge branch 'stable-6.1' into stable-6.2
* stable-6.1:
[benchmarks] Remove profiler configuration
Add SHA1 benchmark
[benchmarks] Set version of maven-compiler-plugin to 3.8.1
Fix running JMH benchmarks
Add option to allow using JDK's SHA1 implementation
Ignore IllegalStateException if JVM is already shutting down
Matthias Sohn [Wed, 16 Nov 2022 08:54:28 +0000 (09:54 +0100)]
Merge branch 'stable-6.0' into stable-6.1
* stable-6.0:
[benchmarks] Remove profiler configuration
Add SHA1 benchmark
[benchmarks] Set version of maven-compiler-plugin to 3.8.1
Fix running JMH benchmarks
Add option to allow using JDK's SHA1 implementation
Ignore IllegalStateException if JVM is already shutting down
Anna Papitto [Thu, 10 Nov 2022 22:45:46 +0000 (14:45 -0800)]
PackExt: Add a reverse index extension.
There is no reverse index PackExt because the reverse index is not currently
written to a file. This prevents fine-grained performance reporting for reverse
indexes, which will be useful when introducing a reverse index file and
observing performance changes.
Add a reverse index extension that matches the one in cgit
(https://github.com/git/git/blob/9bf691b78cf906751e65d65ba0c6ffdcd9a5a12c/Documentation/gitformat-pack.txt#L302)
in preparation for adding a reverse index file while observing
performance before and after.
Change-Id: Iee53f1e01cf645a3c468892fcf97c8444f9a784a Signed-off-by: Anna Papitto <annapapitto@google.com>
Matthias Sohn [Tue, 15 Nov 2022 23:15:17 +0000 (00:15 +0100)]
Merge branch 'stable-5.13' into stable-6.0
* stable-5.13:
[benchmarks] Remove profiler configuration
Add SHA1 benchmark
[benchmarks] Set version of maven-compiler-plugin to 3.8.1
Fix running JMH benchmarks
Add option to allow using JDK's SHA1 implementation
Ignore IllegalStateException if JVM is already shutting down
Matthias Sohn [Tue, 4 Oct 2022 13:45:01 +0000 (15:45 +0200)]
Fix running JMH benchmarks
Without this I sometimes hit the error:
$ java -jar target/benchmarks.jar
Exception in thread "main" java.lang.RuntimeException: ERROR: Unable to
find the resource: /META-INF/BenchmarkList
at org.openjdk.jmh.runner.AbstractResourceReader.getReaders(AbstractResourceReader.java:98)
at org.openjdk.jmh.runner.BenchmarkList.find(BenchmarkList.java:124)
at org.openjdk.jmh.runner.Runner.internalRun(Runner.java:253)
at org.openjdk.jmh.runner.Runner.run(Runner.java:209)
at org.openjdk.jmh.Main.main(Main.java:71)
Matthias Sohn [Fri, 11 Nov 2022 16:54:06 +0000 (17:54 +0100)]
Add option to allow using JDK's SHA1 implementation
The change If6da9833 moved the computation of SHA1 from the JVM's
JCE to a pure Java implementation with collision detection.
The extra security for public sites comes with a cost of slower
SHA1 processing compared to the native implementation in the JDK.
When JGit is used internally and not exposed to any traffic from
external or untrusted users, the extra cost of the pure Java SHA1
implementation can be avoided, falling back to the previous
native MessageDigest implementation.
Matthias Sohn [Mon, 14 Nov 2022 13:34:30 +0000 (14:34 +0100)]
Update Orbit to S20221109014815
and update
- com.sun.jna to 5.12.1.v20221103-2317
- com.sun.jna.platform to 5.12.1.v20221103-2317
- org.bouncycastle.bcpg to 1.72.0.v20221013-1810
- org.bouncycastle.bcpkix to 1.72.0.v20221013-1810
- org.bouncycastle.bcprov to 1.72.0.v20221013-1810
- org.bouncycastle.bcutil to 1.72.0.v20221013-1810
- org.mockito.mockito-core to 4.8.1.v20221103-2317
- org.objenesis to 3.3.0.v20221103-2317
Eryk Szymanski [Tue, 8 Nov 2022 17:22:50 +0000 (18:22 +0100)]
Use replace instead of replaceAll in toCleanString
This is from SonarLint (rule.java:S4348)
Regex patterns should not be created needlessly:
When String::replaceAll is used, the first argument should be a real
regular expression. If it’s not the case, String::replace does exactly
the same thing as String::replaceAll without the performance drawback of
the regex.
Sven Selberg [Wed, 9 Nov 2022 17:28:45 +0000 (18:28 +0100)]
Extract Exception -> HTTP status code mapping for reuse
Extract private static method UploadPackServlet#statusCodeForThrowable
to a public static method in the UploadPackErrorHandler interface so
that implementers of this interface can reuse the default mapping.
Dmitrii Filippov [Mon, 27 Jun 2022 18:03:53 +0000 (20:03 +0200)]
Fix crashes on rare combination of file names
The NameConflictTreeWalk class is used in merge for iterating over
entries in commits. The class uses a separate iterator for each
commit's tree. In rare cases it can incorrectly report the same entry
twice. As a result, duplicated entries are added to the merge result
and later jgit throws an exception when it tries to process merge
result.
The problem appears only when there is a directory-file conflict for
the last item in trees. Example from the bug:
Commit 1:
* subtree - file
* subtree-0 - file
Commit 2:
* subtree - directory
* subtree-0 - file
Here the names are ordered like this:
"subtree" file <"subtree-0" file < "subtree" directory.
The NameConflictTreeWalk handles similar cases correctly if there are
other files after subtree... in commits - this is processed in the
AbstractTreeIterator.min function. Existing code has a special
optimization for the case, when all trees are pointed to the same
entry name - it skips additional checks. However, this optimization
incorrectly skips checks if one of trees reached the end.
The fix processes a situation when some trees reached the end, while
others are still point to an entry.
Josh Brown [Tue, 1 Nov 2022 20:51:48 +0000 (20:51 +0000)]
UploadPack: Receive and parse client session-id
Before this change JGit did not support the session-id capability
implemented by native Git in UploadPack. This change implements
advertising the capability from the server and parsing the session-id
received from the client during an UploadPack operation.
Enable the transfer.advertisesid config setting to advertise the
capability from the server. The client may send a session-id capability
in response. If received, the value from this is parsed and available
via the getClientSID method on the UploadPack object.
This change does not add the capability to send a session-id from the
JGit client.
Josh Brown [Tue, 1 Nov 2022 19:00:07 +0000 (19:00 +0000)]
TransferConfig: Move reading advertisesid setting into TransferConfig
The config setting to enable advertising the session-id capability is
currently read in the ReceivePack class. This change moves it to a
common location in the TransferConfig class so that it can be reused
in other places like UploadPack. TransferConfig is also a more logical
place for the setting as it resides in the `transfer` config section.
Set the transfer.advertisesid setting to true to send the session-id
capability to the client.
Change-Id: If68ecb5e68b59f5c452a7992d02e3688b0a86747 Signed-off-by: Josh Brown <sjoshbrown@google.com>
Josh Brown [Tue, 1 Nov 2022 18:16:44 +0000 (18:16 +0000)]
FirstWant: Parse client session-id if received.
In protocol V0 the client capabilities are appended to the first line.
Parsing session-id is currently only supported during a ReceivePack
operation. This change will parse the client session-id capability if
it has been sent by the client.
If the server sends the session-id capability to the client. The client
may respond with a session ID of its own. FirstWant.fromLine will now
parse the ID and make it available via the getClientSID method.
This change does not add support to send the session-id capability from
the server. The change is necessary to support session-id in UploadPack.
Change-Id: Id3fe44fdf9a72984ee3de9cf40cc4e71d434df4a Signed-off-by: Josh Brown <sjoshbrown@google.com>
Josh Brown [Tue, 11 Oct 2022 21:56:09 +0000 (14:56 -0700)]
ReceivePack: Receive and parse client session-id.
Before this change JGit did not support the session-id capability
implemented by native Git. This change implements advertising the
capability from the server and parsing the session-id received from
the client during a ReceivePack operation.
Enable the transfer.advertisesid config setting to advertise the
capability from the server. The client may send a session-id capability
in response. If received, the value from this is parsed and available
via the getClientSID method on the ReceivePack object. All capabilities
in the form `capability=value` are now split into key value pairs at the
first `=` character. This change replaces specific handling for the
agent capability.
This change does not add advertisement or parsing to UploadPack. This
change also does not add the ability to send a session ID from the JGit
client.
Matthias Sohn [Thu, 27 Oct 2022 18:31:31 +0000 (20:31 +0200)]
Ignore IllegalStateException if JVM is already shutting down
Trying to register/unregister a shutdown hook when the JVM is already in
shutdown throws an IllegalStateException. Ignore this exception since we
can't do anything about it.
Saša Živkov [Fri, 21 Oct 2022 14:32:03 +0000 (16:32 +0200)]
Allow to perform PackedBatchRefUpdate without locking loose refs
Add another newBatchUpdate method in the RefDirectory where we can
control if the created PackedBatchRefUpdate will lock the loose refs or
not.
This can be useful in cases when we run programs which have exclusive
access to a Git repository and we know that locking loose refs is
unnecessary and just a performance loss.
PushCommand: allow users to disable use of bitmaps for push
Reachability bitmaps are designed to speed up the "counting objects"
phase of generating a pack during a clone or fetch. They are not
optimized for Git clients sending a small topic branch via "git push".
In some cases (see [1]), using reachability bitmaps during "git push"
can cause significant performance regressions.
Add PushCommand#setUseBitmaps(boolean) to allow users to tell "git push"
not to use bitmaps.
Thomas Wolf [Thu, 20 Oct 2022 21:25:38 +0000 (23:25 +0200)]
I/O redirection for the pre-push hook
Fix and complete the implementation of calling the pre-push hook.
Add the missing error stream redirect, and add the missing setters
in Transport and in PushCommand. In Transport, delay setting up a
PrePushHook such that it happens only on a push. Previously, the
hook was set up also for fetches.
Bug: 549246
Change-Id: I64a576dfc6b139426f05d9ea6654027ab805734e Signed-off-by: Thomas Wolf <twolf@apache.org>
Ivan Frade [Thu, 16 Dec 2021 21:08:33 +0000 (13:08 -0800)]
PackParser: populate full size of the PackedObjectInfos
We need the full size of the objects to populate the object-size index
of a pack. This size is not always the one encoded in the object header
in the pack (e.g. for deltas).
Populate the full size of PackedObjectInfos in the PackParser, which is
invoked when receiving a pack e.g. in a push.
Ivan Frade [Sat, 4 Dec 2021 00:11:10 +0000 (16:11 -0800)]
ObjectReader: New #isNotLargerThan method
Partial clones filter the objects to send by size calling
ObjectReader#getObjectSize per object. This method reads the object from
storage to get the size, which can be expensive.
Offer a #isNotLargerThan method. The default implementation reads the
object, but subclasses can override it with more efficient lookups (e.g.
adding an index).
isNotLargerThan gives implementors more options to optimize than
getObjectIndex (e.g. can be implemented storing only object over certain
size).
Matthias Sohn [Wed, 12 Oct 2022 21:33:14 +0000 (23:33 +0200)]
Fail build if there are license issues with dependencies
Configure the dash license-tool-plugin to fail the build if there are
license issues with build dependencies. This plugin will generate a list
of all build dependencies and their license to target/dash/summary.
Delete the checked-in dependency list DEPENDENCIES since the build now
always generates it to target/dash/summary.
See https://github.com/eclipse/dash-licenses#example-maven-plugin
projects / jgit.git / log