Matthias Sohn [Thu, 21 Nov 2024 08:30:30 +0000 (09:30 +0100)]
Merge branch 'stable-7.0' into stable-7.1
* stable-7.0:
DiffDriver: fix doc for rust built-in
DiffDriver: fix formatting of javadoc
Add numberOfObjectsSinceBitmap to RepoStatistics
Support built-in diff drivers for hunk header function names
Don't fail when trying to prune pack which is already gone
Rename numberOfPackFilesAfterBitmap to numberOfPackFilesSinceBitmap
Matthias Sohn [Wed, 20 Nov 2024 23:24:53 +0000 (00:24 +0100)]
Merge branch 'stable-6.10' into stable-7.0
* stable-6.10:
DiffDriver: fix doc for rust built-in
DiffDriver: fix formatting of javadoc
Add numberOfObjectsSinceBitmap to RepoStatistics
Support built-in diff drivers for hunk header function names
Don't fail when trying to prune pack which is already gone
Rename numberOfPackFilesAfterBitmap to numberOfPackFilesSinceBitmap
Jacek Centkowski [Thu, 31 Oct 2024 17:30:02 +0000 (18:30 +0100)]
Add numberOfObjectsSinceBitmap to RepoStatistics
Introduce a numberOfObjectsSinceBitmap that contains the number of
objects stored in pack files and as loose objects created since the
latest bitmap generation.
Note that the existing
GcNumberOfPackFilesAfterBitmapStatisticsTest.java was renamed to
GcSinceBitmapStatisticsTest.java and extended to cover also this
statistic.
Support built-in diff drivers for hunk header function names
The regexes defined for each built-in driver will be used to determine
the function name for a hunk header. Each driver can specify a list of
regexes to negate and match. They can also define pattern compilation
flags if needed. These drivers only apply to text files with unified
patch type.
Following built-in drivers have been added:
- cpp
- dts
- java
- python
- rust
Support for more languages can be added as needed to match the cgit
implementation.
Jacek Centkowski [Mon, 11 Nov 2024 11:48:20 +0000 (12:48 +0100)]
Don't fail when trying to prune pack which is already gone
Update the TestRepository.prunePacked so that it doesn't fail if a pack
to be pruned is already gone.
It is especially handy when the prunePacked function is called in
`TestRepository.packAndPrune` function after the repo moves on after
GC was performed.
Matthias Sohn [Tue, 19 Nov 2024 13:58:31 +0000 (14:58 +0100)]
Merge branch 'master' into stable-7.1
* master:
RecursiveMerger: fix boxing warning
UploadPackTest: fix unclosed resource warning
Suppress non-externalized string warnings
Remove unused API problem filters
PersonIdent: Use java.time instead of older Date and milliseconds
GitTimeParser: A date parser using the java.time API
Configure JDT to not raise error on deprecated class linked in javadoc
Update Jetty to 12.0.15
PullCommandTest: assert git status in some simple tests
SystemReader: add method to get LocalDateTime
SystemReader#now: make it a concrete method
[errorprone] RawText: Add parenthesis for explicit op precedence
MockSystemReader: create the right time zone
RawText: improve performance of isCrLfText and isBinary
SystemReader: Give a default implementation to #getTimezoneAt()
[errorprone] ssh: suppress warning for arrays in records
Don't fail when trying to prune pack which is already gone
Ivan Frade [Fri, 15 Nov 2024 18:26:21 +0000 (10:26 -0800)]
PersonIdent: Use java.time instead of older Date and milliseconds
From errorprone: Date has a bad API that leads to bugs; prefer
java.time.Instant or LocalDate.
Replace the long with milliseconds and int with minutes offset with an
Instant and a ZoneOffset. Create new constructors and deprecate
variants with Date, milliseconds and minute offsets.
When comparing instances of PersonIdent truncate the timestamp precision
to 1 second since git commit timestamps are persisted with 1 second
precision [1].
Ivan Frade [Mon, 11 Nov 2024 21:13:59 +0000 (13:13 -0800)]
GitTimeParser: A date parser using the java.time API
Replacement of GitDateParser that uses java.time classes instead of
the obsolete Date. Updating GitDateParser would have been a mess of
deprecation and methods with confusing names, so I opted for writing a
parallel class with the new types.
Some differences:
* The new DateTimeFormatter is thread-safe, so we don't need the
LocalThread cache
* No code seems to use other locale than the default, we don't need to
cache per locale either
Ivan Frade [Thu, 14 Nov 2024 00:25:42 +0000 (16:25 -0800)]
SystemReader#now: make it a concrete method
Abstract methods break subclasses (e.g. DelegateSystemReader in
gerrit). Updating jgit and gerrit is simpler if we do not add them. I
am not sure why some methods are abstract and others dont, but now()
can be a concrete method.
Make now() concrete. Implement it by default based on
getCurrentTime(), so subclasses overriding that method get the same
value.
Ivan Frade [Wed, 13 Nov 2024 19:58:06 +0000 (11:58 -0800)]
SystemReader: Give a default implementation to #getTimezoneAt()
This abstract method forces subclasses (e.g. DelegateSystemReader in
gerrit) to update their code, but there is no strong reason to make it
abstract (subclasses can override it if needed).
Make the method concrete using the current default implementation
(which is the same in the mock).
Jacek Centkowski [Mon, 11 Nov 2024 11:48:20 +0000 (12:48 +0100)]
Don't fail when trying to prune pack which is already gone
Update the TestRepository.prunePacked so that it doesn't fail if a pack
to be pruned is already gone. It is especially handy when prunePacked
function is called in `TestRepository.packAndPrune` function after repo
moves on after the GC was performed.
Matthias Sohn [Mon, 11 Nov 2024 23:06:18 +0000 (00:06 +0100)]
Merge branch 'master' into stable-7.1
* master:
errorprone: Disable javadoc checks in tests
Rename numberOfPackFilesAfterBitmap to numberOfPackFilesSinceBitmap
Replace custom encoder Constants#encodeASCII by JDK implementation
Replace custom encoder `Constants#encode` by JDK implementation
DfsGarbageCollector: #setReflogExpire with Instant instead of Date
ssh: Minor simplification in SerialRangeSet
DfsBlockCacheConfig: propagate hotmap configs to pack ext cache configs
SystemReader: Offer methods with java.time API
Add `numberOfPackFilesAfterBitmap` to RepoStatistics
Enhance CommitBuilder#parent to tolerate null parent
GPG: use BC PGP secret key parsing out of the box
[errorprone] bc: Remove unused SExprParser#parseSecretKey
Update bouncycastle to 1.79
Update bytebuddy to 1.15.10
DfsPackCompactor: write object size index
[errorprone] BaseRepositoryBuilder: Use #split(sep, limit)
[errorprone] Remove deprecated security manager
[errorprone] RefDatabase: #getConflictingNames immutable return
DfsGarbageCollector: Add setter for reflog expiration time.
[errorprone] SeparateClassloadertTestRunner: use #split(String,int)
[errorprone] HttpConnection: Add missing summary in java
[errorprone] PackWriter: Fix javadoc tag in new #writeIndex method
[errorprone] ByteArraySet: Add summary fragment to javadoc
[errorprone] util.Stats: Add summary fragment to javadoc
DfsInserter: Read minBytesForObjectSizeIndex only from repo config
PackWriter: make PackWriter.writeIndex() take a PackIndexWriter
dfs: update getBlockCacheStats to return a List of BlockCacheStats
Update mockito to 5.14.2
Update bytebuddy to 1.15.7
Remove unnecessary argument handler in MergeBase.java
Replace custom encoder Constants#encodeASCII by JDK implementation
Ivan Frade [Mon, 11 Nov 2024 19:07:21 +0000 (11:07 -0800)]
errorprone: Disable javadoc checks in tests
Errorprone finds many problems in the tests javadocs. This is noisy in
the logs, but fixing them also disturbs the project history and can
complicate merges.
Disable the javadoc checks in the tests packages. We can fix those
javadocs if some other modification happen in the file (as we fix
older coding style).
Ivan Frade [Fri, 8 Nov 2024 16:49:09 +0000 (08:49 -0800)]
DfsGarbageCollector: #setReflogExpire with Instant instead of Date
The Date API is full of major design flaws and pitfalls and should be
avoided at all costs. Prefer the java.time APIs, specifically,
java.time.Instant (for physical time) and java.time.LocalDate[Time]
(for civil time). [1]
Replace the Date with Instant in the
DfsGarbageCollector#setReflogExpire method.
Laura Hamelin [Wed, 6 Nov 2024 21:41:30 +0000 (13:41 -0800)]
DfsBlockCacheConfig: propagate hotmap configs to pack ext cache configs
CacheHotMap is currently only set on the base DfsBlockCacheConfig and is
not propagated down to PackExt specific caches.
Because CacheHotMap is set from a method call rather than from Configs,
this change sets per-PackExt CacheHotMap configs on PackExt cache
configs both when DfsBlockCacheConfig#setCacheHotMap(...) is called, and
when DfsBlockCacheConfig#configure(...) is called after setCacheHotMap.
The outer DfsBlockCacheConfig keeps the full CacheHotMap for the same
reason that the CacheHotMap config is propagated in both setCacheHotMap
and configure: the order of operations setting the configuration from
Configs and calling setCacheHotMap is not guaranteed.
Ivan Frade [Mon, 4 Nov 2024 22:21:24 +0000 (14:21 -0800)]
SystemReader: Offer methods with java.time API
Error prone explains: The Date API is full of major design flaws and
pitfalls and should be avoided at all costs. Prefer the java.time
APIs, specifically, java.time.Instant (for physical time) and
java.time.LocalDate[Time] (for civil time).
Add to SystemReader methods to get the time and timezone in the new
java.time classes (Instant/ZoneId) and mark as deprecated their old
counterparts.
The mapping of methods is:
* #getCurrentTime -> #now (returns Instant instead of int)
* #getTimezone -> #getTimeZoneAt (returns ZoneOffset intead of int)
* #getTimeZone -> #getTimeZoneId (return ZoneId instead of TimeZone)
Jacek Centkowski [Fri, 20 Sep 2024 06:47:13 +0000 (08:47 +0200)]
Add `numberOfPackFilesAfterBitmap` to RepoStatistics
Introduce a `numberOfPackFilesAfterBitmap` that contains the number of
packfiles created since the latest bitmap generation.
Notes:
* the `repo.getObjectDatabase().getPacks()` that obtains the list of
packs (in the existing `getStatistics` function) uses
`PackDirectory.scanPacks` that boils down to call
`PackDirectory.scanPacksImpl` which is sorting packs prior returning
them therefore the `numberOfPackFilesAfterBitmap` is just all packs
before the one that has bitmap attached
* the improved version of `packAndPrune` function (one that skips
non-existent packfiles) was introduced for testing
Thomas Wolf [Wed, 6 Nov 2024 18:14:47 +0000 (19:14 +0100)]
GPG: use BC PGP secret key parsing out of the box
Remove the custom S-expression parsing; BC has gotten many
improvements in 1.79 regarding PGP ed25519 keys, AES/OCB
encryption, and generally parsing key files. It now can do
all we need.
Change-Id: I392443e040cce150a9575d18795a7cb8195a3515 Signed-off-by: Thomas Wolf <twolf@apache.org>
errorprone complains about using Date in the SExprParser class. All
the usages are in a variant of the parseSecretKey method that doesn't
have any callers.
Matthias Sohn [Tue, 5 Nov 2024 00:29:08 +0000 (01:29 +0100)]
Merge branch 'stable-7.1'
* stable-7.1:
Add missing @since 7.1 to UploadPack#implies
ResolveMerger: Allow setting the TreeWalk AttributesNodeProvider
Add Union merge strategy support
Nasser Grainawi [Tue, 29 Oct 2024 23:22:15 +0000 (17:22 -0600)]
ResolveMerger: Allow setting the TreeWalk AttributesNodeProvider
When a merger is created without a Repository, no
AttributesNodeProvider is created in the TreeWalk. Since mergers are
often created with a custom ObjectInserter and no repo, they skip any
lookups of attributes from any of the gitattributes files (within a
tree, in the repo info/ dir, or user/global). Since there are
potentially merge-affecting attributes in those files, callers might
want to use both a custom ObjectInserter and an AttributesNodeProvider.
Ivan Frade [Fri, 1 Nov 2024 15:58:27 +0000 (08:58 -0700)]
DfsPackCompactor: write object size index
Currently the compactor is not writing the object size index for
packs. As it is using PackWriter to generate the packs, it needs to
explicitely call the writes of each extension.
Invoke writeObjectSizeIndex in the compactor. The pack writer will
write one if the configuration says so.
Ivan Frade [Thu, 31 Oct 2024 19:17:09 +0000 (12:17 -0700)]
[errorprone] Remove deprecated security manager
Errorprone warns about this deprecated classes. The recommendation is
stop using SecurityManager all together.
The Security Manager is deprecated and subject to removal in a future
release. There is no replacement for the Security Manager. See JEP 411
[1] for discussion and alternatives.
Errorprone reports that: This method returns both mutable and
immutable collections or maps from different paths. This may be
confusing for users of the method.
Saril Sudhakaran [Tue, 29 Oct 2024 05:17:01 +0000 (00:17 -0500)]
DfsGarbageCollector: Add setter for reflog expiration time.
JGit reftable writer/compator knows how to prune the history, but the
DfsGarbageCollector doesn't expose the time limit.
Add a method to DfsGarbageCollector to set the reflog time limit.
This value is then passed to the reftable compactor. Callers usually
pass here the value from gc.reflogExpire.
The reflog block length is stored in 24 bits [1], limiting the size to
16MB. I have observed that in repositories with frequent commits,
reflogs hit that size in 6-12 months.
Ivan Frade [Tue, 9 Jul 2024 18:10:02 +0000 (11:10 -0700)]
[errorprone] util.Stats: Add summary fragment to javadoc
Errorprone complains about missing summary in these javadocs.
[MissingSummary] A summary fragment is required; consider using the
value of the @return b lock as a summary fragment instead.
* @return variance of the added values
^
(see https://google.github.io/styleguide/javaguide.html#s7.2-summary-fragment)
Did you mean '*Returns variance of the added values.'?
Ivan Frade [Tue, 29 Oct 2024 17:57:28 +0000 (10:57 -0700)]
DfsInserter: Read minBytesForObjectSizeIndex only from repo config
In general, JGit reads the configuration it needs from the repository
configuration. minBytesForObjectSizeIndex is a special case with a
setter for subclasses but that is unnecessary.
Remove the setter and read the conf from the repo. Make the property
final and read it directly from the conf (it is clearer than parsing a
whole PackConfig to read a single value).
Sam Delmerico [Mon, 7 Oct 2024 21:34:03 +0000 (14:34 -0700)]
PackWriter: make PackWriter.writeIndex() take a PackIndexWriter
Previously, the PackWriter implementation required that indexes and
their extensions be writable to an OutputStream with a fixed binary
format. To support more general index storage formats, allow
PackWriter to accept an PackIndexWriter interface which accepts only
the objects to store. This allows implementors to choose their storage
format.
The implementation will be provided by the DfsObjectDatabase. The
DfsObjectDatabase is already responsible for providing the OutputStream
that was previously used to write indexes. Having it provide a writing
interface would be a natural generalization.
This idea was previously implemented for PackBitmapIndex writing in
https://gerrithub.io/c/eclipse-jgit/jgit/+/1177722.
Laura Hamelin [Wed, 24 Jul 2024 21:01:38 +0000 (14:01 -0700)]
dfs: update getBlockCacheStats to return a List of BlockCacheStats
Make available all underlying cache table stats for the used cache table
implementation.
The existing cache table stats implementation only allows a "global"
view of the cache table statistics; it does not differentiate between
all possible underlying cache tables used.
This change allows callers to get the block cache stats broken down
per underlying table. These cache stats are intended to be used for
monitoring all cache tables independently.
Existing usages of getBlockCacheStats now make use of
AggregatedBlockCacheStats.fromStatsList to aggregate the list of
BlockCacheStats into a single BlockCacheStats instance.
Matthias Sohn [Wed, 23 Oct 2024 20:17:29 +0000 (20:17 +0000)]
Merge changes from topic "ssh-signatures"
* changes:
SSH signing: implement a SignatureVerifier
SSH signing: implement a Signer
SSH signing: don't require a session in PasswordProviderWrapper
SSH signing: make OpenSSH pattern matching public
SSH signing: prepare config
ssh: add a factory for KeyPasswordProvider
Thomas Wolf [Sat, 28 Sep 2024 14:11:45 +0000 (16:11 +0200)]
SSH signing: implement a SignatureVerifier
Signature verification needs quite a bit of infrastructure. There are
two files to read: a list of allowed signers, and a list of revoked keys
or certificates. Introduce a SigningKeyDatabase abstraction for these,
and give client code the possibility to plug in its own implementation.
Loading these files afresh for every signature to be checked would be
prohibitively expensive. Introduce a cache of SigningKeyDatabases, and
have them reload the files only when they have changed.
Include a default implementation that works with the OpenSSH allowed
signers file and with OpenSSH revocation lists. Binary KRLs are parsed
according to [1]; the test data was generated using the OpenSSH test
script[2].
Thomas Wolf [Sat, 28 Sep 2024 13:58:20 +0000 (15:58 +0200)]
SSH signing: implement a Signer
Implement a Signer and its factory, and publish the factory for the
ServiceLoader. SSH signatures can be created directly if the key is
given via a file in user.signingKey and the private key can be found.
Otherwise, signing is delegated to an SSH agent, if available.
If a certificate is used as public key, the signer verifies the
certificate (correct signature, and valid at the commit time).
projects / jgit.git / log