Sam Delmerico [Tue, 27 Aug 2024 20:27:35 +0000 (13:27 -0700)]
DfsPackFile: re-add metrics for bitmap index loads
The calculations for readBitmapIdxBytes and readBitmapIdxMicros
metrics were unintentionally removed in
https://gerrithub.io/c/eclipse-jgit/jgit/+/1177262, so this change
adds those metrics back.
Matthias Sohn [Tue, 27 Aug 2024 13:13:37 +0000 (15:13 +0200)]
Merge branch 'master' into stable-7.0
* master:
DfsReaderIoStats: getters to object size index micros/bytes
Do not set headers if response is already committed
AmazonS3: Ensure SAXParserFactory sets valid/expected input params
Signing: refactor interfaces
Add a missing license header
LockFile: Retry lock creation if parent dirs were removed
GpgConfig: Add missing @since
DfsReaderIoStats: Order fields and methods consistently
Change Ie8a9d411fc19e8b7bf86c0b4df0b02153a0e9444 broke setting
valid/expected input parameters for the XML parser. This can be fixed
by calling SaxParserFactory#setNamespaceAware, see [1]. Also see earlier
fix in [2].
Thomas Wolf [Tue, 20 Aug 2024 20:41:45 +0000 (22:41 +0200)]
Signing: refactor interfaces
This is a big API-breaking change cleaning up the signing interfaces.
Initially, these interfaces were GPG/OpenPGP-specific. When EGit added
new signers and signature verifiers that called an external GPG
executable, they were found inadequate and were extended to be able to
pass in the GpgConfig to get access to the "gpg.program" setting.
With the introduction of X.509 S/MIME signing, it was discovered that
the interfaces were still not quite adequate, and the "Gpg" prefix on
the class names were confusing.
Since 7.0 is a major version bump, I'm taking this chance to overhaul
these interfaces from ground up.
For signing, there is a new Signer interface. With it goes a
SignerFactory SPI interface, and a final Signers class managing the
currently set signers. By default, signers for the different signature
types are created from the signer factories, which are discovered via
the ServiceLoader. External code can install its own signers, overriding
the default factories.
For signature verification, exactly the same mechanism is used.
This simplifies the setup of signers and signature verifiers, and makes
it all more regular. Signer instances just get a byte[] to sign and
don't have to worry about ObjectBuilders at all. SignatureVerifier
instances also just get the data and signature as byte[] and don't have
to worry about extracting the signature from a commit or tag, or about
what kind of signature it is.
Both Signers and SignatureVerifiers always get passed the Repository
and the GpgConfig. The repository will be needed in an implementation
for SSH signatures because gpg.ssh.* configs may need to be loaded
explicitly, and some of those values need the current workspace
location.
For signature verification, there is exactly one place in core JGit in
SignatureVerifiers that extracts signatures, determines the signature
type, and then calls the right signature verifier.
Change RevTag to recognize all signature types known in git (GPG, X509,
and SSH).
Change-Id: I26d2731e7baebb38976c87b7f328b63a239760d5 Signed-off-by: Thomas Wolf <twolf@apache.org>
LockFile: Retry lock creation if parent dirs were removed
In the small window between creation of the lock file's parent dirs and
the lock file itself, the parent dirs may be cleaned by an external
process packing refs in the repository. When this scenario occurs, retry
creating the lock file (along with its parent dirs).
Matthias Sohn [Tue, 20 Aug 2024 13:51:15 +0000 (15:51 +0200)]
Merge branch 'master' into stable-7.0
* master:
Update tycho to 4.0.8
Update org.eclipse.dash:license-tool-plugin to 1.1.0
[ssh] Bump Apache MINA sshd 2.13.1 -> 2.13.2
ConfigConstants: Add missing @since 7.0
Fix "Comparison of narrow type with wide type in loop condition"
ObjectWalk: Remove duplicated word "the" in class documentation
RepoProject: read the 'dest-branch' attribute of a project
Make RepoProject#setUpstream public
RepoCommand: Add error to ManifestErrorException
RepoCommand: Copy manifest upstream into .gitmodules ref field
RepoProject: read the "upstream" attribute of a project
JGit v5.13.3.202401111512-r
Matthias Sohn [Tue, 20 Aug 2024 13:26:41 +0000 (15:26 +0200)]
Merge branch 'stable-6.10'
* stable-6.10:
Update tycho to 4.0.8
Update org.eclipse.dash:license-tool-plugin to 1.1.0
Fix "Comparison of narrow type with wide type in loop condition"
RepoProject: read the 'dest-branch' attribute of a project
Make RepoProject#setUpstream public
RepoCommand: Add error to ManifestErrorException
RepoCommand: Copy manifest upstream into .gitmodules ref field
RepoProject: read the "upstream" attribute of a project
JGit v5.13.3.202401111512-r
Matthias Sohn [Tue, 20 Aug 2024 13:21:43 +0000 (15:21 +0200)]
Merge branch 'stable-6.9' into stable-6.10
* stable-6.9:
Update tycho to 4.0.8
Update org.eclipse.dash:license-tool-plugin to 1.1.0
Fix "Comparison of narrow type with wide type in loop condition"
JGit v5.13.3.202401111512-r
Matthias Sohn [Tue, 20 Aug 2024 13:20:37 +0000 (15:20 +0200)]
Merge branch 'stable-6.8' into stable-6.9
* stable-6.8:
Update tycho to 4.0.8
Update org.eclipse.dash:license-tool-plugin to 1.1.0
Fix "Comparison of narrow type with wide type in loop condition"
JGit v5.13.3.202401111512-r
Matthias Sohn [Tue, 20 Aug 2024 12:56:04 +0000 (14:56 +0200)]
Merge branch 'stable-6.7' into stable-6.8
* stable-6.7:
Update tycho to 4.0.8
Update org.eclipse.dash:license-tool-plugin to 1.1.0
Fix "Comparison of narrow type with wide type in loop condition"
JGit v5.13.3.202401111512-r
Matthias Sohn [Tue, 20 Aug 2024 12:54:08 +0000 (14:54 +0200)]
Merge branch 'stable-6.6' into stable-6.7
* stable-6.6:
Update tycho to 4.0.8
Update org.eclipse.dash:license-tool-plugin to 1.1.0
Fix "Comparison of narrow type with wide type in loop condition"
JGit v5.13.3.202401111512-r
Matthias Sohn [Tue, 20 Aug 2024 12:28:33 +0000 (14:28 +0200)]
Merge branch 'stable-6.5' into stable-6.6
* stable-6.5:
Update org.eclipse.dash:license-tool-plugin to 1.1.0
Fix "Comparison of narrow type with wide type in loop condition"
JGit v5.13.3.202401111512-r
Matthias Sohn [Sun, 18 Aug 2024 16:35:29 +0000 (18:35 +0200)]
Merge branch 'stable-6.4' into stable-6.5
* stable-6.4:
Update org.eclipse.dash:license-tool-plugin to 1.1.0
Fix "Comparison of narrow type with wide type in loop condition"
JGit v5.13.3.202401111512-r
Matthias Sohn [Fri, 9 Aug 2024 09:53:01 +0000 (11:53 +0200)]
Fix "Comparison of narrow type with wide type in loop condition"
This issue was detected by a GitHub CodeQL security scan run on JGit
source code.
Description of the error raised by the security scan:
"In a loop condition, comparison of a value of a narrow type with a
value of a wide type may always evaluate to true if the wider value is
sufficiently large (or small). This is because the narrower value may
overflow. This can lead to an infinite loop."
Fix this by using type `long` for the local variable `done`.
Matthias Sohn [Wed, 31 Jul 2024 13:02:55 +0000 (15:02 +0200)]
Merge branch 'master' into stable-7.0
* master:
Lib: Fix ssh value for gpg.format throwing an IllegalArgumentException
DfsPackFile: Abstract the loading of pack indexes
PackExtBlockCacheTable: spread extensions over multiple dfs tables
PackObjectSizeIndex: Read all bytes and use the byte[] directly
DfsPackFile: Do not set local reverse index ref from cache callback
Add 4.33 target platform for Eclipse 2024-09
DfsBlockCacheTable: extract stats get* methods to interface
Add worktrees read support
DfsBlockCacheConfig: support configurations for dfs cache tables per extensions
ssh: Remove .orig file
DfsPackFile: Enable/disable object size index via DfsReaderOptions
Ivan Frade [Wed, 10 Apr 2024 20:52:56 +0000 (13:52 -0700)]
DfsPackFile: Abstract the loading of pack indexes
DfsPackFile assumes that the indexes are stored in file streams and
their references need to be cached in DFS. This doesn't allow us to
experiment other storage options, like key-value databases. In these
experiments not all indexes are together in the same storage.
Define an interface per index to load it, so implementors can focus on
the specifics of their index. Put them together in the IndexFactory
interface. The implementation of the IndexFactory chooses the right
combination of storages.
At the moment we do this only for primary and reverse
indexes. Following changes can do the same for other indexes.
Laura Hamelin [Fri, 7 Jun 2024 23:11:21 +0000 (16:11 -0700)]
PackExtBlockCacheTable: spread extensions over multiple dfs tables
The existing DfsBlockCache uses a single table for all
extensions (idx, ridx, ...).
This change introduces an implementation of the table
interface that can keep extensions in different cache
tables.
This selects the appropriate cache to use for a specific
PackExt or DfsStreamKey's PackExt type, allowing the
separation of entries from different pack types to help
limit churn in cache caused by entries of differing sizes.
This is especially useful in fine-tuning caches and
influencing interactions by extension type.
For example, a table holding INDEX types only will
not influence evictions of other PackExt types and
vice versa.
The PackExtBlockCacheTable allowing setting the
underlying DfsBlockCacheTables and mappinh directly,
letting users implement and use custom DfsBlockCacheTables.
Ivan Frade [Fri, 19 Jul 2024 22:44:15 +0000 (15:44 -0700)]
PackObjectSizeIndex: Read all bytes and use the byte[] directly
The parser reads N integers one by one from the stream, assuming the
InputStream does some ahead reading from storage. We see some very
slow loading of indexes and suspect that this preemptive reading is
not happening. The slow loading can be reproduced in clones, and it
produces higher latencies and locks many threads waiting for the
loading.
Read the whole array from storage in one shot to avoid many small IO
reads. Work directly on the resulting byte[], so there is no need of a
second copy to cast to int/long.
This is how other indexes, like primary or commit graph, work.
Ivan Frade [Wed, 10 Apr 2024 17:52:44 +0000 (10:52 -0700)]
DfsPackFile: Do not set local reverse index ref from cache callback
The DfsBlockCache loading callback sets the local reference to the
index in the DfsPackFile. This prevents abstracting the loading to
implement it over multiple backends.
Reorg the code so the loadReverseIndex do only the loading, the caller
sets it into DfsBlockCache and the external code sets the local
reference in DfsPackFile.
This is the same pattern we did with the PackIndex in the parent
commit.
projects / jgit.git / log