Ivan Frade [Fri, 15 Nov 2024 21:14:47 +0000 (13:14 -0800)]
RawParseUtils test: Use java.time to create PersonIdents
The constructor with long/int for time/tz is deprecated in favor of
Instant/ZoneId.
Update first the expectations in the tests to the new constructors, so
we know we are creating the same PersonIdents than before. We update
the code later, knowing there is no regression in e.g. format or
precision.
Ivan Frade [Fri, 15 Nov 2024 18:26:21 +0000 (10:26 -0800)]
PersonIdent: Use java.time instead of older Date and milliseconds
From errorprone: Date has a bad API that leads to bugs; prefer
java.time.Instant or LocalDate.
Replace the long with milliseconds and int with minutes offset with an
Instant and a ZoneOffset. Create new constructors and deprecate
variants with Date, milliseconds and minute offsets.
When comparing instances of PersonIdent truncate the timestamp precision
to 1 second since git commit timestamps are persisted with 1 second
precision [1].
Ivan Frade [Mon, 11 Nov 2024 21:13:59 +0000 (13:13 -0800)]
GitTimeParser: A date parser using the java.time API
Replacement of GitDateParser that uses java.time classes instead of
the obsolete Date. Updating GitDateParser would have been a mess of
deprecation and methods with confusing names, so I opted for writing a
parallel class with the new types.
Some differences:
* The new DateTimeFormatter is thread-safe, so we don't need the
LocalThread cache
* No code seems to use other locale than the default, we don't need to
cache per locale either
Ivan Frade [Thu, 14 Nov 2024 00:25:42 +0000 (16:25 -0800)]
SystemReader#now: make it a concrete method
Abstract methods break subclasses (e.g. DelegateSystemReader in
gerrit). Updating jgit and gerrit is simpler if we do not add them. I
am not sure why some methods are abstract and others dont, but now()
can be a concrete method.
Make now() concrete. Implement it by default based on
getCurrentTime(), so subclasses overriding that method get the same
value.
Ivan Frade [Wed, 13 Nov 2024 19:58:06 +0000 (11:58 -0800)]
SystemReader: Give a default implementation to #getTimezoneAt()
This abstract method forces subclasses (e.g. DelegateSystemReader in
gerrit) to update their code, but there is no strong reason to make it
abstract (subclasses can override it if needed).
Make the method concrete using the current default implementation
(which is the same in the mock).
Jacek Centkowski [Mon, 11 Nov 2024 11:48:20 +0000 (12:48 +0100)]
Don't fail when trying to prune pack which is already gone
Update the TestRepository.prunePacked so that it doesn't fail if a pack
to be pruned is already gone. It is especially handy when prunePacked
function is called in `TestRepository.packAndPrune` function after repo
moves on after the GC was performed.
Matthias Sohn [Mon, 11 Nov 2024 23:06:18 +0000 (00:06 +0100)]
Merge branch 'master' into stable-7.1
* master:
errorprone: Disable javadoc checks in tests
Rename numberOfPackFilesAfterBitmap to numberOfPackFilesSinceBitmap
Replace custom encoder Constants#encodeASCII by JDK implementation
Replace custom encoder `Constants#encode` by JDK implementation
DfsGarbageCollector: #setReflogExpire with Instant instead of Date
ssh: Minor simplification in SerialRangeSet
DfsBlockCacheConfig: propagate hotmap configs to pack ext cache configs
SystemReader: Offer methods with java.time API
Add `numberOfPackFilesAfterBitmap` to RepoStatistics
Enhance CommitBuilder#parent to tolerate null parent
GPG: use BC PGP secret key parsing out of the box
[errorprone] bc: Remove unused SExprParser#parseSecretKey
Update bouncycastle to 1.79
Update bytebuddy to 1.15.10
DfsPackCompactor: write object size index
[errorprone] BaseRepositoryBuilder: Use #split(sep, limit)
[errorprone] Remove deprecated security manager
[errorprone] RefDatabase: #getConflictingNames immutable return
DfsGarbageCollector: Add setter for reflog expiration time.
[errorprone] SeparateClassloadertTestRunner: use #split(String,int)
[errorprone] HttpConnection: Add missing summary in java
[errorprone] PackWriter: Fix javadoc tag in new #writeIndex method
[errorprone] ByteArraySet: Add summary fragment to javadoc
[errorprone] util.Stats: Add summary fragment to javadoc
DfsInserter: Read minBytesForObjectSizeIndex only from repo config
PackWriter: make PackWriter.writeIndex() take a PackIndexWriter
dfs: update getBlockCacheStats to return a List of BlockCacheStats
Update mockito to 5.14.2
Update bytebuddy to 1.15.7
Remove unnecessary argument handler in MergeBase.java
Replace custom encoder Constants#encodeASCII by JDK implementation
Ivan Frade [Mon, 11 Nov 2024 19:07:21 +0000 (11:07 -0800)]
errorprone: Disable javadoc checks in tests
Errorprone finds many problems in the tests javadocs. This is noisy in
the logs, but fixing them also disturbs the project history and can
complicate merges.
Disable the javadoc checks in the tests packages. We can fix those
javadocs if some other modification happen in the file (as we fix
older coding style).
Ivan Frade [Fri, 8 Nov 2024 16:49:09 +0000 (08:49 -0800)]
DfsGarbageCollector: #setReflogExpire with Instant instead of Date
The Date API is full of major design flaws and pitfalls and should be
avoided at all costs. Prefer the java.time APIs, specifically,
java.time.Instant (for physical time) and java.time.LocalDate[Time]
(for civil time). [1]
Replace the Date with Instant in the
DfsGarbageCollector#setReflogExpire method.
Laura Hamelin [Wed, 6 Nov 2024 21:41:30 +0000 (13:41 -0800)]
DfsBlockCacheConfig: propagate hotmap configs to pack ext cache configs
CacheHotMap is currently only set on the base DfsBlockCacheConfig and is
not propagated down to PackExt specific caches.
Because CacheHotMap is set from a method call rather than from Configs,
this change sets per-PackExt CacheHotMap configs on PackExt cache
configs both when DfsBlockCacheConfig#setCacheHotMap(...) is called, and
when DfsBlockCacheConfig#configure(...) is called after setCacheHotMap.
The outer DfsBlockCacheConfig keeps the full CacheHotMap for the same
reason that the CacheHotMap config is propagated in both setCacheHotMap
and configure: the order of operations setting the configuration from
Configs and calling setCacheHotMap is not guaranteed.
Ivan Frade [Mon, 4 Nov 2024 22:21:24 +0000 (14:21 -0800)]
SystemReader: Offer methods with java.time API
Error prone explains: The Date API is full of major design flaws and
pitfalls and should be avoided at all costs. Prefer the java.time
APIs, specifically, java.time.Instant (for physical time) and
java.time.LocalDate[Time] (for civil time).
Add to SystemReader methods to get the time and timezone in the new
java.time classes (Instant/ZoneId) and mark as deprecated their old
counterparts.
The mapping of methods is:
* #getCurrentTime -> #now (returns Instant instead of int)
* #getTimezone -> #getTimeZoneAt (returns ZoneOffset intead of int)
* #getTimeZone -> #getTimeZoneId (return ZoneId instead of TimeZone)
Jacek Centkowski [Fri, 20 Sep 2024 06:47:13 +0000 (08:47 +0200)]
Add `numberOfPackFilesAfterBitmap` to RepoStatistics
Introduce a `numberOfPackFilesAfterBitmap` that contains the number of
packfiles created since the latest bitmap generation.
Notes:
* the `repo.getObjectDatabase().getPacks()` that obtains the list of
packs (in the existing `getStatistics` function) uses
`PackDirectory.scanPacks` that boils down to call
`PackDirectory.scanPacksImpl` which is sorting packs prior returning
them therefore the `numberOfPackFilesAfterBitmap` is just all packs
before the one that has bitmap attached
* the improved version of `packAndPrune` function (one that skips
non-existent packfiles) was introduced for testing
Thomas Wolf [Wed, 6 Nov 2024 18:14:47 +0000 (19:14 +0100)]
GPG: use BC PGP secret key parsing out of the box
Remove the custom S-expression parsing; BC has gotten many
improvements in 1.79 regarding PGP ed25519 keys, AES/OCB
encryption, and generally parsing key files. It now can do
all we need.
Change-Id: I392443e040cce150a9575d18795a7cb8195a3515 Signed-off-by: Thomas Wolf <twolf@apache.org>
errorprone complains about using Date in the SExprParser class. All
the usages are in a variant of the parseSecretKey method that doesn't
have any callers.
Matthias Sohn [Tue, 5 Nov 2024 00:29:08 +0000 (01:29 +0100)]
Merge branch 'stable-7.1'
* stable-7.1:
Add missing @since 7.1 to UploadPack#implies
ResolveMerger: Allow setting the TreeWalk AttributesNodeProvider
Add Union merge strategy support
Nasser Grainawi [Tue, 29 Oct 2024 23:22:15 +0000 (17:22 -0600)]
ResolveMerger: Allow setting the TreeWalk AttributesNodeProvider
When a merger is created without a Repository, no
AttributesNodeProvider is created in the TreeWalk. Since mergers are
often created with a custom ObjectInserter and no repo, they skip any
lookups of attributes from any of the gitattributes files (within a
tree, in the repo info/ dir, or user/global). Since there are
potentially merge-affecting attributes in those files, callers might
want to use both a custom ObjectInserter and an AttributesNodeProvider.
Ivan Frade [Fri, 1 Nov 2024 15:58:27 +0000 (08:58 -0700)]
DfsPackCompactor: write object size index
Currently the compactor is not writing the object size index for
packs. As it is using PackWriter to generate the packs, it needs to
explicitely call the writes of each extension.
Invoke writeObjectSizeIndex in the compactor. The pack writer will
write one if the configuration says so.
Ivan Frade [Thu, 31 Oct 2024 19:17:09 +0000 (12:17 -0700)]
[errorprone] Remove deprecated security manager
Errorprone warns about this deprecated classes. The recommendation is
stop using SecurityManager all together.
The Security Manager is deprecated and subject to removal in a future
release. There is no replacement for the Security Manager. See JEP 411
[1] for discussion and alternatives.
Errorprone reports that: This method returns both mutable and
immutable collections or maps from different paths. This may be
confusing for users of the method.
Saril Sudhakaran [Tue, 29 Oct 2024 05:17:01 +0000 (00:17 -0500)]
DfsGarbageCollector: Add setter for reflog expiration time.
JGit reftable writer/compator knows how to prune the history, but the
DfsGarbageCollector doesn't expose the time limit.
Add a method to DfsGarbageCollector to set the reflog time limit.
This value is then passed to the reftable compactor. Callers usually
pass here the value from gc.reflogExpire.
The reflog block length is stored in 24 bits [1], limiting the size to
16MB. I have observed that in repositories with frequent commits,
reflogs hit that size in 6-12 months.
Ivan Frade [Tue, 9 Jul 2024 18:10:02 +0000 (11:10 -0700)]
[errorprone] util.Stats: Add summary fragment to javadoc
Errorprone complains about missing summary in these javadocs.
[MissingSummary] A summary fragment is required; consider using the
value of the @return b lock as a summary fragment instead.
* @return variance of the added values
^
(see https://google.github.io/styleguide/javaguide.html#s7.2-summary-fragment)
Did you mean '*Returns variance of the added values.'?
Ivan Frade [Tue, 29 Oct 2024 17:57:28 +0000 (10:57 -0700)]
DfsInserter: Read minBytesForObjectSizeIndex only from repo config
In general, JGit reads the configuration it needs from the repository
configuration. minBytesForObjectSizeIndex is a special case with a
setter for subclasses but that is unnecessary.
Remove the setter and read the conf from the repo. Make the property
final and read it directly from the conf (it is clearer than parsing a
whole PackConfig to read a single value).
Sam Delmerico [Mon, 7 Oct 2024 21:34:03 +0000 (14:34 -0700)]
PackWriter: make PackWriter.writeIndex() take a PackIndexWriter
Previously, the PackWriter implementation required that indexes and
their extensions be writable to an OutputStream with a fixed binary
format. To support more general index storage formats, allow
PackWriter to accept an PackIndexWriter interface which accepts only
the objects to store. This allows implementors to choose their storage
format.
The implementation will be provided by the DfsObjectDatabase. The
DfsObjectDatabase is already responsible for providing the OutputStream
that was previously used to write indexes. Having it provide a writing
interface would be a natural generalization.
This idea was previously implemented for PackBitmapIndex writing in
https://gerrithub.io/c/eclipse-jgit/jgit/+/1177722.
Laura Hamelin [Wed, 24 Jul 2024 21:01:38 +0000 (14:01 -0700)]
dfs: update getBlockCacheStats to return a List of BlockCacheStats
Make available all underlying cache table stats for the used cache table
implementation.
The existing cache table stats implementation only allows a "global"
view of the cache table statistics; it does not differentiate between
all possible underlying cache tables used.
This change allows callers to get the block cache stats broken down
per underlying table. These cache stats are intended to be used for
monitoring all cache tables independently.
Existing usages of getBlockCacheStats now make use of
AggregatedBlockCacheStats.fromStatsList to aggregate the list of
BlockCacheStats into a single BlockCacheStats instance.
Matthias Sohn [Wed, 23 Oct 2024 20:17:29 +0000 (20:17 +0000)]
Merge changes from topic "ssh-signatures"
* changes:
SSH signing: implement a SignatureVerifier
SSH signing: implement a Signer
SSH signing: don't require a session in PasswordProviderWrapper
SSH signing: make OpenSSH pattern matching public
SSH signing: prepare config
ssh: add a factory for KeyPasswordProvider
Thomas Wolf [Sat, 28 Sep 2024 14:11:45 +0000 (16:11 +0200)]
SSH signing: implement a SignatureVerifier
Signature verification needs quite a bit of infrastructure. There are
two files to read: a list of allowed signers, and a list of revoked keys
or certificates. Introduce a SigningKeyDatabase abstraction for these,
and give client code the possibility to plug in its own implementation.
Loading these files afresh for every signature to be checked would be
prohibitively expensive. Introduce a cache of SigningKeyDatabases, and
have them reload the files only when they have changed.
Include a default implementation that works with the OpenSSH allowed
signers file and with OpenSSH revocation lists. Binary KRLs are parsed
according to [1]; the test data was generated using the OpenSSH test
script[2].
Thomas Wolf [Sat, 28 Sep 2024 13:58:20 +0000 (15:58 +0200)]
SSH signing: implement a Signer
Implement a Signer and its factory, and publish the factory for the
ServiceLoader. SSH signatures can be created directly if the key is
given via a file in user.signingKey and the private key can be found.
Otherwise, signing is delegated to an SSH agent, if available.
If a certificate is used as public key, the signer verifies the
certificate (correct signature, and valid at the commit time).
Sam Delmerico [Mon, 7 Oct 2024 22:20:26 +0000 (15:20 -0700)]
PackIndexWriter: create interface to write indexes
PackWriter assumes that the primary index goes to a file in a well-known
format. This cannot accomodate implementations in other storages or
formats (e.g. in a database).
Create an interface to write the index (PackIndexWriter). This interface
will be implemented by the existing pack index writer classes
(PackIndexWriterV1 etc.).
As the "PackIndexWriter" name was used by the previous superclass of the
file writers, we rename that class to "BasePackIndexWriter".
Jacek Centkowski [Fri, 20 Sep 2024 06:47:13 +0000 (08:47 +0200)]
Add `numberOfPackFilesAfterBitmap` to RepoStatistics
Introduce a `numberOfPackFilesAfterBitmap` that contains the number of
packfiles created since the latest bitmap generation.
Notes:
* the `repo.getObjectDatabase().getPacks()` that obtains the list of
packs (in the existing `getStatistics` function) uses
`PackDirectory.scanPacks` that boils down to call to
`PackDirectory.scanPacksImpl` which is sorting packs prior returning
them therefore the `numberOfPackFilesAfterBitmap` is just all packs
before the one that has bitmap attached
* the improved version of `packAndPrune` function (one that skips
non-existent packfiles) was introduced for testing
Thomas Wolf [Sat, 28 Sep 2024 13:52:31 +0000 (15:52 +0200)]
SSH signing: make OpenSSH pattern matching public
SSH signing needs the same pattern matching algorithm as is used for
host matching in host entries in ~/.ssh/config. So make that pattern
matching available via a static method.
Change-Id: Ia26f23666f323f44ce66f769fbcd6c85965eb219 Signed-off-by: Thomas Wolf <twolf@apache.org>
Thomas Wolf [Fri, 20 Sep 2024 19:59:49 +0000 (21:59 +0200)]
ssh: add a factory for KeyPasswordProvider
Introduce a global default factory to create KeyPasswordProvider.
Previously, their creation was tied to the SSH session, but for SSH
signatures, we will need to be able to create KeyPasswordProviders
without having an SSH session.
Change-Id: If4a69c4d4c4e8de390cb1ef3b65966d0e39c24ff Signed-off-by: Thomas Wolf <twolf@apache.org>
Laura Hamelin [Wed, 16 Oct 2024 22:16:00 +0000 (15:16 -0700)]
DfsBlockCache: refactor stats implementations.
The stats interface has an implementation in the interface itself and another inside the PackExtBlockCache class. This asymmetry gets on the way to implement stats-per-table later.
Make DfsBlockCacheStats (the stats of a single table) a top-level class and create an aggregator class to combine multiple stats. This makes the stats classes mirror the table classes structure (singles tables + composite).
This change is part of a refactor to support providing detailed stats
breakdowns for cache implementations using multiple table instances
while keeping the existing "aggregated" view of cache stats.
This will write out configuration values on a line by line basis to a
given PrintWriter.
Primary usage is as a semi-formatted debug print of the configuration
values used by dfs block cache.
Luca Milanesio [Fri, 11 Oct 2024 21:40:16 +0000 (22:40 +0100)]
Test advertised capabilities with protocol V0 and allow*Sha1InWant
The advertised capabilities with protocol V0 were untested
leading to potential regressions when advertising what
SHA1 should or should not be on the list of capabilities.
Verify that allow-tip-sha1-in-want and allow-reachable-sha1-in-want
are properly advertised with the allow*Sha1InWant is set in
jgit.config.
projects / jgit.git / log