Matthias Sohn [Tue, 5 Nov 2024 00:29:08 +0000 (01:29 +0100)]
Merge branch 'stable-7.1'
* stable-7.1:
Add missing @since 7.1 to UploadPack#implies
ResolveMerger: Allow setting the TreeWalk AttributesNodeProvider
Add Union merge strategy support
Nasser Grainawi [Tue, 29 Oct 2024 23:22:15 +0000 (17:22 -0600)]
ResolveMerger: Allow setting the TreeWalk AttributesNodeProvider
When a merger is created without a Repository, no
AttributesNodeProvider is created in the TreeWalk. Since mergers are
often created with a custom ObjectInserter and no repo, they skip any
lookups of attributes from any of the gitattributes files (within a
tree, in the repo info/ dir, or user/global). Since there are
potentially merge-affecting attributes in those files, callers might
want to use both a custom ObjectInserter and an AttributesNodeProvider.
Ivan Frade [Fri, 1 Nov 2024 15:58:27 +0000 (08:58 -0700)]
DfsPackCompactor: write object size index
Currently the compactor is not writing the object size index for
packs. As it is using PackWriter to generate the packs, it needs to
explicitely call the writes of each extension.
Invoke writeObjectSizeIndex in the compactor. The pack writer will
write one if the configuration says so.
Ivan Frade [Thu, 31 Oct 2024 19:17:09 +0000 (12:17 -0700)]
[errorprone] Remove deprecated security manager
Errorprone warns about this deprecated classes. The recommendation is
stop using SecurityManager all together.
The Security Manager is deprecated and subject to removal in a future
release. There is no replacement for the Security Manager. See JEP 411
[1] for discussion and alternatives.
Errorprone reports that: This method returns both mutable and
immutable collections or maps from different paths. This may be
confusing for users of the method.
Saril Sudhakaran [Tue, 29 Oct 2024 05:17:01 +0000 (00:17 -0500)]
DfsGarbageCollector: Add setter for reflog expiration time.
JGit reftable writer/compator knows how to prune the history, but the
DfsGarbageCollector doesn't expose the time limit.
Add a method to DfsGarbageCollector to set the reflog time limit.
This value is then passed to the reftable compactor. Callers usually
pass here the value from gc.reflogExpire.
The reflog block length is stored in 24 bits [1], limiting the size to
16MB. I have observed that in repositories with frequent commits,
reflogs hit that size in 6-12 months.
Ivan Frade [Tue, 9 Jul 2024 18:10:02 +0000 (11:10 -0700)]
[errorprone] util.Stats: Add summary fragment to javadoc
Errorprone complains about missing summary in these javadocs.
[MissingSummary] A summary fragment is required; consider using the
value of the @return b lock as a summary fragment instead.
* @return variance of the added values
^
(see https://google.github.io/styleguide/javaguide.html#s7.2-summary-fragment)
Did you mean '*Returns variance of the added values.'?
Ivan Frade [Tue, 29 Oct 2024 17:57:28 +0000 (10:57 -0700)]
DfsInserter: Read minBytesForObjectSizeIndex only from repo config
In general, JGit reads the configuration it needs from the repository
configuration. minBytesForObjectSizeIndex is a special case with a
setter for subclasses but that is unnecessary.
Remove the setter and read the conf from the repo. Make the property
final and read it directly from the conf (it is clearer than parsing a
whole PackConfig to read a single value).
Sam Delmerico [Mon, 7 Oct 2024 21:34:03 +0000 (14:34 -0700)]
PackWriter: make PackWriter.writeIndex() take a PackIndexWriter
Previously, the PackWriter implementation required that indexes and
their extensions be writable to an OutputStream with a fixed binary
format. To support more general index storage formats, allow
PackWriter to accept an PackIndexWriter interface which accepts only
the objects to store. This allows implementors to choose their storage
format.
The implementation will be provided by the DfsObjectDatabase. The
DfsObjectDatabase is already responsible for providing the OutputStream
that was previously used to write indexes. Having it provide a writing
interface would be a natural generalization.
This idea was previously implemented for PackBitmapIndex writing in
https://gerrithub.io/c/eclipse-jgit/jgit/+/1177722.
Laura Hamelin [Wed, 24 Jul 2024 21:01:38 +0000 (14:01 -0700)]
dfs: update getBlockCacheStats to return a List of BlockCacheStats
Make available all underlying cache table stats for the used cache table
implementation.
The existing cache table stats implementation only allows a "global"
view of the cache table statistics; it does not differentiate between
all possible underlying cache tables used.
This change allows callers to get the block cache stats broken down
per underlying table. These cache stats are intended to be used for
monitoring all cache tables independently.
Existing usages of getBlockCacheStats now make use of
AggregatedBlockCacheStats.fromStatsList to aggregate the list of
BlockCacheStats into a single BlockCacheStats instance.
Matthias Sohn [Wed, 23 Oct 2024 20:17:29 +0000 (20:17 +0000)]
Merge changes from topic "ssh-signatures"
* changes:
SSH signing: implement a SignatureVerifier
SSH signing: implement a Signer
SSH signing: don't require a session in PasswordProviderWrapper
SSH signing: make OpenSSH pattern matching public
SSH signing: prepare config
ssh: add a factory for KeyPasswordProvider
Thomas Wolf [Sat, 28 Sep 2024 14:11:45 +0000 (16:11 +0200)]
SSH signing: implement a SignatureVerifier
Signature verification needs quite a bit of infrastructure. There are
two files to read: a list of allowed signers, and a list of revoked keys
or certificates. Introduce a SigningKeyDatabase abstraction for these,
and give client code the possibility to plug in its own implementation.
Loading these files afresh for every signature to be checked would be
prohibitively expensive. Introduce a cache of SigningKeyDatabases, and
have them reload the files only when they have changed.
Include a default implementation that works with the OpenSSH allowed
signers file and with OpenSSH revocation lists. Binary KRLs are parsed
according to [1]; the test data was generated using the OpenSSH test
script[2].
Thomas Wolf [Sat, 28 Sep 2024 13:58:20 +0000 (15:58 +0200)]
SSH signing: implement a Signer
Implement a Signer and its factory, and publish the factory for the
ServiceLoader. SSH signatures can be created directly if the key is
given via a file in user.signingKey and the private key can be found.
Otherwise, signing is delegated to an SSH agent, if available.
If a certificate is used as public key, the signer verifies the
certificate (correct signature, and valid at the commit time).
Sam Delmerico [Mon, 7 Oct 2024 22:20:26 +0000 (15:20 -0700)]
PackIndexWriter: create interface to write indexes
PackWriter assumes that the primary index goes to a file in a well-known
format. This cannot accomodate implementations in other storages or
formats (e.g. in a database).
Create an interface to write the index (PackIndexWriter). This interface
will be implemented by the existing pack index writer classes
(PackIndexWriterV1 etc.).
As the "PackIndexWriter" name was used by the previous superclass of the
file writers, we rename that class to "BasePackIndexWriter".
Jacek Centkowski [Fri, 20 Sep 2024 06:47:13 +0000 (08:47 +0200)]
Add `numberOfPackFilesAfterBitmap` to RepoStatistics
Introduce a `numberOfPackFilesAfterBitmap` that contains the number of
packfiles created since the latest bitmap generation.
Notes:
* the `repo.getObjectDatabase().getPacks()` that obtains the list of
packs (in the existing `getStatistics` function) uses
`PackDirectory.scanPacks` that boils down to call to
`PackDirectory.scanPacksImpl` which is sorting packs prior returning
them therefore the `numberOfPackFilesAfterBitmap` is just all packs
before the one that has bitmap attached
* the improved version of `packAndPrune` function (one that skips
non-existent packfiles) was introduced for testing
Thomas Wolf [Sat, 28 Sep 2024 13:52:31 +0000 (15:52 +0200)]
SSH signing: make OpenSSH pattern matching public
SSH signing needs the same pattern matching algorithm as is used for
host matching in host entries in ~/.ssh/config. So make that pattern
matching available via a static method.
Change-Id: Ia26f23666f323f44ce66f769fbcd6c85965eb219 Signed-off-by: Thomas Wolf <twolf@apache.org>
Thomas Wolf [Fri, 20 Sep 2024 19:59:49 +0000 (21:59 +0200)]
ssh: add a factory for KeyPasswordProvider
Introduce a global default factory to create KeyPasswordProvider.
Previously, their creation was tied to the SSH session, but for SSH
signatures, we will need to be able to create KeyPasswordProviders
without having an SSH session.
Change-Id: If4a69c4d4c4e8de390cb1ef3b65966d0e39c24ff Signed-off-by: Thomas Wolf <twolf@apache.org>
Laura Hamelin [Wed, 16 Oct 2024 22:16:00 +0000 (15:16 -0700)]
DfsBlockCache: refactor stats implementations.
The stats interface has an implementation in the interface itself and another inside the PackExtBlockCache class. This asymmetry gets on the way to implement stats-per-table later.
Make DfsBlockCacheStats (the stats of a single table) a top-level class and create an aggregator class to combine multiple stats. This makes the stats classes mirror the table classes structure (singles tables + composite).
This change is part of a refactor to support providing detailed stats
breakdowns for cache implementations using multiple table instances
while keeping the existing "aggregated" view of cache stats.
This will write out configuration values on a line by line basis to a
given PrintWriter.
Primary usage is as a semi-formatted debug print of the configuration
values used by dfs block cache.
Luca Milanesio [Fri, 11 Oct 2024 21:40:16 +0000 (22:40 +0100)]
Test advertised capabilities with protocol V0 and allow*Sha1InWant
The advertised capabilities with protocol V0 were untested
leading to potential regressions when advertising what
SHA1 should or should not be on the list of capabilities.
Verify that allow-tip-sha1-in-want and allow-reachable-sha1-in-want
are properly advertised with the allow*Sha1InWant is set in
jgit.config.
Luca Milanesio [Mon, 7 Oct 2024 22:16:58 +0000 (23:16 +0100)]
Align request policies with CGit
CGit defines the SHA request policies using a bitmask
that represents which policy is implied by another policy.
For example, in CGit the ALLOW_TIP_SHA1 is 0x01 and ALLOW_REACHABLE_SHA1
is 0x02, which are associated to two different bit in a 3-bit value.
The ALLOW_ANY_SHA1 value is 0x07 which denotes a different policy that
implies the previous two ones, because is represented with a 3-bit
bitmask having all ones.
Associate the JGit RequestPolicy enum to the same CGit bitmask values
and use the same logic for the purpose of advertising the server
capabilities.
The JGit code becomes easier to read and associate with its counterpart
in CGit, especially during the capabilities advertising phase.
Also add a new utility method RequestPolicy.implies() which is more
readable than a direct bitmask and operator.
Matthias Sohn [Wed, 9 Oct 2024 00:04:55 +0000 (02:04 +0200)]
Update Apache sshd to 2.14.0
This fixes an 'incorrect signature' error when trying to use the keys
generated by SSHD during server init with an OpenSSH client.
This also includes a few other changes since 2.13.2:
* GH-524 Performance improvements
* GH-533 Fix multi-step authentication
* GH-582 Fix filtering in NamedFactory
* GH-587 Prevent NullPointerExceptionon closed channel in NettyIoSession
* GH-590 Better support for FIPS
* GH-597 Pass on Charset in ClientSession.executeRemoteCommand()
Matthias Sohn [Wed, 9 Oct 2024 00:04:55 +0000 (02:04 +0200)]
Update Apache sshd to 2.14.0
This fixes an 'incorrect signature' error when trying to use the keys
generated by SSHD during server init with an OpenSSH client.
This also includes a few other changes since 2.13.2:
* GH-524 Performance improvements
* GH-533 Fix multi-step authentication
* GH-582 Fix filtering in NamedFactory
* GH-587 Prevent NullPointerExceptionon closed channel in NettyIoSession
* GH-590 Better support for FIPS
* GH-597 Pass on Charset in ClientSession.executeRemoteCommand()
Kamil Musin [Tue, 8 Oct 2024 11:36:00 +0000 (13:36 +0200)]
RevolveMerger: honor ignoreConflicts also for binary files
Currently difference in binary files during merge will cause them to be
added to unmergedPaths regardless of whether ignoreConflicts is true.
This creates an issue during merging with strategy "RECURSIVE", as it
makes it impossible to create a virtual commit if there is a difference
in a binary file. Resulting in the
CONFLICTS_DURING_MERGE_BASE_CALCULATION error being thrown.
This is especially problematic, since JGit has a
rather simplistic rules for considering file binary, which easily leads
to false positives.
What we should do instead is keep OURS. This will not lead to silently
ignoring difference in the final result. It will allow creation of
virtual merge-base commit, and then the difference would be presented
again in the final merge results. In essense it only affects what's
shown as BASE in 3-way merge.
Additionally, this is correct because
- It's consistent with treatment of other unmergeable entities, for
example Gitlinks
- It's consistent with behaviour of CGit:
- https://git-scm.com/docs/gitattributes#Documentation/gitattributes.txt-binary
states on diffs in binary OURS is picked by default.
- In code: https://git.kernel.org/pub/scm/git/git.git/tree/merge-ll.c#n81
- ignoreConflicts in CGit afterwards ignores all issues with content
merging https://git.kernel.org/pub/scm/git/git.git/tree/merge-ort.c#n5201
We also adjust the behaviour when .gitattributes tell us to treat the
file as binary for the purpose of the merge.
We only change the behaviour when ignoreConlicts = true, as otherwise
the current behaviour works as intended.
Laura Hamelin [Mon, 10 Jun 2024 20:42:03 +0000 (13:42 -0700)]
DfsBlockCache: use PackExtBlockCacheTable when configured
Adds the usage of PackExtBlockCacheTable to the
DfsBlockCache, replacing the current DfsBlockCacheTable
when PackExtCacheConfigurations exists.
When no PackExtCacheConfigurations exists the current
DfsBlockCacheTable implementation will be used.
Matthias Sohn [Tue, 24 Sep 2024 08:51:22 +0000 (10:51 +0200)]
AdvertisedRequestValidator: fix WantNotValidException caused by race
Fetch with protocol V2 failed under the following conditions
- fetch uses bidirectional protocol (git, ssh) which uses a shortcut
to determine invalid wants
- not all wants are advertised
- race condition: wanted ref is updated during fetch by another thread
after the thread serving upload-pack determined wants and before it
checks not advertised wants
Fix this by calling
`new ReachableCommitRequestValidator().checkWants(up, wants)`
instead of throwing WantNotValidException in [1]
if this race happened in the same way like it's done for unidirectional
protocols (http) [2].
Ivan Frade [Tue, 24 Sep 2024 22:23:04 +0000 (15:23 -0700)]
DfsInserter: Create PackConfig from repo instead of repo.conf
PackConfig can be constructed from the repo or from a config. While
browing the code, it is easier to follow the provenance of the
configuration when using the repository constructor.
Use the PackConfig(Repository) constructor in the
DfsInserter. Internally it calls PackConfig(repo.getConfig()) so this
should be a noop.
This is meant to help diagnose LOCK_FAILURE errors, which otherwise provides
very little information in
https://eclipse.googlesource.com/jgit/jgit/+/refs/heads/master/org.eclipse.jgit/src/org/eclipse/jgit/lib/BatchRefUpdate.java#731.
PackIndex: Add protected setters to populate the MutableEntry
Implementations of the iterator out of package receive a MutableEntry but they don't have a way to set data into it.
Add setters to the MutableEntry via protected methods in the iterator. This way, only implementors of the Iterator can modify the entry (cannot be modified e.g. by callers).
Ivan Frade [Mon, 16 Sep 2024 19:44:00 +0000 (12:44 -0700)]
PackWriter: Remove constructor with only Reader as param
At construction time, PackWriter receives the PackConfig as a
parameter or reads it from the repository. The only exception is when
the constructor receives only a reader (no repo nor conf
provided?!).
Remove PackWriter(Reader) and let callers be explicit what conf to
use. This makes clearer the flow of conf in the PackWriter.
The iterator keeps the current position in the index and the MutableEntry reads data from there on-demand, but the iterator needs to know about the entry and this creates a complicated interaction.
Make MutableEntry a simple data object and let the iterator iterate and populate it before returning it. Code is clearer and implementors only needs to worry about the iterator.
This fixes also MutableEntry visibility, that was preventing subclassing from out of the package.
projects / jgit.git / log