Matthias Sohn [Tue, 8 Mar 2022 16:23:57 +0000 (17:23 +0100)]
Merge branch 'stable-6.1'
* stable-6.1:
Prepare 6.1.1-SNAPSHOT builds
JGit v6.1.0.202203080745-r
[checkout] Use .gitattributes from the commit to be checked out
Don't use final for method parameters
[push] support the "matching" RefSpecs ":" and "+:"
[push] Call the pre-push hook later in the push process
IndexDiff: use tree filter also for SubmoduleWalk
Run license check with option -Ddash.projectId=technology.jgit
Exclude transitive dependencies of sshd-sftp
Update DEPENDENCIES for 6.1.0 release
Add dependency to dash-licenses
Fix typos of some keys in LfsText
Sort LfsText entries alphabetically
Support for "lfs.url" from ".lfsconfig"
Thomas Wolf [Sun, 13 Feb 2022 22:30:36 +0000 (23:30 +0100)]
[checkout] Use .gitattributes from the commit to be checked out
JGit used only one set of attributes constructed from the global and
info attributes, plus the attributes from working tree, index, and
HEAD.
These attributes must be used to determine whether the working tree is
dirty.
But for actually checking out a file, one must use the attributes from
global, info, and *the commit to be checked out*. Otherwise one may not
pick up definitions that are only in the .gitattributes of the commit
to be checked out or that are changed in that commit with respect to
the attributes currently in HEAD, the index, or the working tree.
Maintain in TreeWalk different Attributes per tree, and add operations
to determine EOL handling and smudge filters per tree.
Use the new methods in DirCacheCheckout and ResolveMerger. Note that
merging in JGit actually used the attributes from the base, not those
from ours, which looks dubious at least. It now uses those from ours,
and for checking out the ones from theirs.
The canBeContentMerged() determination was also done from the base
attributes, and is newly done from the ours attributes. Possibly this
should take into account all three attributes, and only if all three
agree the item can be content merged, a content merge should be
attempted? (What if the binary/text setting changes between base, ours,
or theirs?)
Also note that JGit attempts to perform content merges on non-binary
LFS files; there it used the filter attribute from base, too, even for
the ours and theirs versions. Newly it takes the filter attribute from
the correct tree. I'm not convinced doing content merges on potentially
huge files like LFS files is really a good idea.
Add tests in FilterCommandsTest and LfsGitTest to verify the behavior.
Open question: using index and working tree as fallback for the
attributes of ours (assuming it is HEAD) is OK. But does it also make
sense for base and theirs in merging?
Bug: 578707
Change-Id: I0bf433e9e3eb28479b6272e17c0666e175e67d08 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Sun, 20 Feb 2022 17:11:49 +0000 (18:11 +0100)]
[push] support the "matching" RefSpecs ":" and "+:"
The implementation of push.default=matching was not correct.
It used the RefSpec "refs/heads/*:refs/heads/*", which would push
_all_ local branches. But "matching" must push only those local
branches for which a remote branch with the same name already exists
at the remote.
This RefSpec can be expanded only once the advertisement from the
remote has been received.
Enhance RefSpec so that ":" and "+:" can be represented. Introduce a
special RemoteRefUpdate for such a RefSpec; it must carry through the
fetch RefSpecs to be able to fill in the remote tracking updates as
needed. Implement the expansion in PushProcess.
Bug: 353405
Change-Id: I54a4bfbb0a6a7d77b9128bf4a9c951d6586c3df4 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Sun, 20 Feb 2022 17:10:24 +0000 (18:10 +0100)]
[push] Call the pre-push hook later in the push process
Call the pre-push hook only after having received the remote
advertisement and having determined rejections, like C git does.
Also similar to C git, don't pass rejected or up-to-date updates
to the pre-push hook.
Bug: 578852
Change-Id: I51d379ea7bd8234ec815f8f4a9fa325816f476cf Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Fri, 17 Jul 2020 17:41:31 +0000 (19:41 +0200)]
IndexDiff: use tree filter also for SubmoduleWalk
The only uses of IndexDiff.setFilter() in JGit and EGit set a path
filter. Passing the filter on to the SubmoduleWalk gives the desired
result, which is consistent with command-line git.
Bug: 565251
Change-Id: I8eca1ed73eb1d237b8785f369352f72af9e0e168 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Matthias Sohn [Thu, 3 Mar 2022 23:30:36 +0000 (00:30 +0100)]
Exclude transitive dependencies of sshd-sftp
We don't need the dependencies of sshd-sftp to sshd-common and
sshd-core since they are contained in sshd-osgi.
Excluding them helps the dash IP log tool to not list them as required
dependencies of jgit.
Matthias Fromme [Mon, 3 Jan 2022 12:06:34 +0000 (13:06 +0100)]
Support for "lfs.url" from ".lfsconfig"
- New class LfsConfig to enrich repository configuration by settings
from ".lfsconfig" file respecting configuration file precedence.
- Adapted LfsConnectionFactory to use LfsConfig instead of directly
using configuration from repository to calculate url of the lfs
repository
Matthias Sohn [Wed, 2 Mar 2022 20:10:03 +0000 (21:10 +0100)]
Merge branch 'master' into stable-6.1
* master:
Describe: add support for core.abbrev config option
Add a typed config getter for integers confined to a range
Remove odd prefix of PersonIdent test class
PersonIdent: Add ctors that accept Instant in addition to Date
Remove ignored potentiallyUnclosedCloseable check
Make precedence more explicit
[pgm] Add describe --abbrev option
Cap describe abbrev option
DescribeCommand: Add support for --abbrev=0
Matthias Sohn [Mon, 21 Feb 2022 01:29:27 +0000 (02:29 +0100)]
Describe: add support for core.abbrev config option
If core.abbrev is unset or "auto" estimate abbreviation length like C
git does:
- Estimate repository's object count by only considering packed objects,
round up to next power of 2
- With the order of 2^len objects, we expect a collision at 2^(len/2).
But we also care about hex chars, not bits, and there are 4 bits per
hex. So all together we need to divide by 2; but we also want to round
odd numbers up, hence adding one before dividing.
- For small repos use at least 7 hexdigits
- If object database fails to determine object count use 7 hexdigits as
fallback
If it is set to "no" do not abbreviate object-ids.
Otherwise set it to the configured value capped to the range between 4
and length of an unabbreviated object-id.
David Ostrovsky [Wed, 29 Dec 2021 20:18:52 +0000 (21:18 +0100)]
PersonIdent: Add ctors that accept Instant in addition to Date
Error Prone is flagging Date-API as obsolete and recommends to migrate
to Instant and LocalDate. Given that more JGit users starting to migrate
to new Time API, offer ctors that accept Instant type and also add new
getter that returns when attribute as Instant type.
Fabio Ponciroli [Thu, 3 Feb 2022 08:49:13 +0000 (09:49 +0100)]
Remove SuppressWarnings since currently ignored
The following warning was raised by Eclipse:
"At least one of the problems in category
'unused' is not analysed due to a compiler option being ignored"
The org.eclipse.jdt.core.compiler.problem.unusedTypeParameter compiler
option is set to ignore, hence the warning suppression is redundant.
Thomas Wolf [Sun, 13 Feb 2022 22:24:14 +0000 (23:24 +0100)]
Simplify implementation of WorkingTreeIterator
All the filtering in WorkingTreeIterator is for check-in, i.e., clean
filtering. The implementation was in some parts too general, passing
around an OperationType. But since it's always CHECKIN_OP, that's not
actually necessary.
Change-Id: I73f8bc059e485a073e456962868f52b3a3db4fc1 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Fri, 11 Feb 2022 16:18:20 +0000 (17:18 +0100)]
Transport: load all refs only if push refspecs have wildcards
There is no need to load all refs if there are no wildcard push
refspecs. Load them lazily on the first wildcard refspec encountered
instead of loading them up-front.
Change-Id: I6d0e981f9ed4997dbdefeb7f83f37ff4f33e06a5 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Thu, 10 Feb 2022 18:04:21 +0000 (19:04 +0100)]
PushCommand: determine remote from git config if not given
Add ConfigConstants and expose branch.<name>.pushRemote in the
BranchConfig. Use the branch configuration and remote.pushDefault
if no remote is given explicitly. If nothing is configured, fall
back to "origin".
Bug: 578676
Change-Id: I6bb141ff02c8b04980ec34b26ef248b72614c3c9 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Rolf Theunissen [Sun, 10 Feb 2019 17:40:26 +0000 (18:40 +0100)]
PushCommand: consider push.default when no RefSpecs are given
When no RefSpecs are given, PushCommand until now simply fell back to
pushing the current branch to an upstream branch of the same name. This
corresponds to push.default=current. Any setting from the git config
for push.default was simply ignored.
Implement the other modes (nothing, matching, upstream, and simple),
too. Add a setter and getter for the PushDefault so that an application
can force a particular mode to be used. For backwards compatibility,
use "current" as the default setting; to figure out the value from the
git config, which defaults to "simple", call setPushDefault(null).
Bug: 351314
Change-Id: I86c5402318771e47d80b137e99947762e1150bb4 Signed-off-by: Rolf Theunissen <rolf.theunissen@gmail.com> Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
BasePackConnection::readAdvertisedRefsImpl was creating an exception by
calling `noRepository`, and then blindly calling `initCause` on it. As
`noRepository` can be overridden, it's not guaranteed to be missing a
cause.
BasePackPushConnection overrides `noRepository` and initiates a fetch,
which may throw a `NoRemoteRepositoryException` with a cause.
In this case calling `initCause` threw an `IllegalStateException`.
In order to throw the correct exception, we now return the
BasePackPushConnection exception and suppress the one thrown by
BasePackConnection
Nail Samatov [Mon, 7 Feb 2022 15:13:58 +0000 (18:13 +0300)]
Support LFS Server URL without .git suffix
According to Git LFS documentation, URLs with and without .git suffix
should be supported. By default, Git LFS will append .git/info/lfs to
the end of a Git remote URL. To build the LFS server URL it will use:
Fix the LfsConnectionFactory accordingly. Move a utility method to
add the ".git" suffix if not present yet from FileResolver to
StringUtils and use it.
Bug: 578621
Change-Id: I8d3645872d5f03bb8e82c9c73647adb3e81ce484 Signed-off-by: Nail Samatov <sanail@yandex.ru> Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Add a way for the handler to tell whether the commit should generate a
Gerrit Change-Id. Augment the ModifyResult interface, and set the flag
on the CommitCommand.
This enables users to have a Change-ID be generated when squashing or
rewording commits. A possibly already existing Change-Id will remain
unchanged.
Bug: 440211
Change-Id: I66a72e0646876d162a7011235cca969e20acf060 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Mon, 17 Jan 2022 23:37:33 +0000 (00:37 +0100)]
RebaseCommand: fix commit message in "fixup" case
JGit accumulated in MESSAGE_FIXUP commit messages of a fixup sequence,
just like it did in MESSAGE_SQUASH, and on the last step of a sequence
of fixups used that file, after stripping all comment lines, as the
commit message. That also stripped any lines from the original commit
message that happened to start with the comment character.
This is not how this is supposed to work. MESSAGE_FIXUP must contain
the original commit message of the base commit that is amended, and
the file contains the verbatim commit message for the final fixup.[1]
Change the implementation accordingly, and add new tests.
Han-Wen Nienhuys [Mon, 31 Jan 2022 11:23:55 +0000 (12:23 +0100)]
reftable: tweaks for Windows
Reload the stack _before_ trying to delete the files. This ensures we
don't trip over our own open file handles when deleting compacted
tables.
If there is another process reading the file, it may be impossible to
delete the compacted tables. In this case, ignore the failure.
For cleaning the garbage in this case, the protocol as described in
https://www.git-scm.com/docs/reftable#_windows should be implemented.
This is left for another commit.
Ivan Frade [Tue, 1 Feb 2022 22:41:44 +0000 (17:41 -0500)]
Merge changes I11366273,I256e1572
* changes:
RepoCommand: Offer to set extra files in the destination repository
RepoCommand: Move bare/regular superproject writing to their own classes
Ivan Frade [Tue, 23 Nov 2021 22:42:01 +0000 (14:42 -0800)]
RepoCommand: Offer to set extra files in the destination repository
We want to save in the destination repository what manifest created its
structure. This helps to detect and debug failures in the manifest ->
superproject translations. The src commit should be easily readable from
the superproject tip.
Offer an API to write a file in the destination repository. RepoCommand
callers (e.g. gerrit supermanifest plugin) can use this to add a
file with the repo/ref/hash of the manifest.
Alternatives considered to write the source repo/ref/hash:
* .gitattributes of the .gitmodules file. Some updates in the manifest
don't touch the .gitmodules (e.g. a linkfile change), so it can fall
out of sync.
* commit message. Caller would need to follow the commit history to
find the latest modification by repo command. This is not helpful
e.g. for build bots that want to get the value in one call.
Ivan Frade [Thu, 26 Aug 2021 20:19:01 +0000 (13:19 -0700)]
RepoCommand: Move bare/regular superproject writing to their own classes
RepoCommand parses the manifest to get a list of projects, clears up
conflicts and then writes to the superproject. The first steps are
common but the writing is completely different for bare or "regular"
(with working dir) repository.
Split writing to bare and regular repos into its own classes. This
simplifies RepoCommand class and makes clearer what happens on each side
(e.g. many options apply only to bare repos).
Thomas Wolf [Sat, 15 Jan 2022 22:11:41 +0000 (23:11 +0100)]
RebaseCommand: better commit message rewording
Respect git config commit.cleanup for rewording. Note that by default
this is CleanupMode.STRIP, whereas before this change, JGit would take
the reworded message verbatim.
Squashing was the only place in JGit where it automatically and
unconditionally removed comment lines from commit messages. In other
places it didn't do so, and client code needed to do so.
Unconditionally removing comments is problematic if the commit message
_should_ contain some line starting with a hash, which can easily occur
with the way Github, Gitlab, and other git web servers link to issues
or PRs: they all allow the short-hand "#<number>".
Introduce a new InteractiveHandler2 extension interface, which can
return the edited message _and_ a clean-up mode. This way, client code
can decide on its own how to clean the message, and if JGit shouldn't
do any further cleaning, it can return CleanupMode.VERBATIM. Or
CleanupMode.WHITESPACE. (In the case of SQUASH, it is then of course
the client's responsibility to remove the squash comment lines.)
If the old InteractiveHandler interface is used, CleanupMode.STRIP is
applied unconditionally for squashing, as before.
Bug: 578173
Change-Id: Ia0040c247884e684587dd45d6cb85f8b72a4b876 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Mon, 17 Jan 2022 19:28:25 +0000 (20:28 +0100)]
CommitCommand: commit message cleanup
Use CommitConfig.CleanupMode to implement git commit --cleanup. Add
setters for the clean-up mode, the comment character, and for the
default default clean-up mode.
Behavior of existing client code is unchanged as the default clean-up
mode is set to "verbatim". To use git config defaults, one can call
setCleanupMode(CleanupMode.DEFAULT). The default comment character
is hard-coded as '#' for now, as in other parts of JGit. Implementing
full support for core.commentChar shall be done in a separate change.
Bug: 553065
Change-Id: I470785e464a762d3f409f163f1cbdbb98dd81aaf Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Sat, 15 Jan 2022 21:51:06 +0000 (22:51 +0100)]
Provide git config commit.cleanup
Add an enumeration for the possible values, and a method to resolve the
"default" value. Give CommitConfig a static method to process a text
according to a given clean-up mode and comment character.
(The core.commentChar is not yet handled by JGit; it's hard-coded as #.)
Bug: 553065
Change-Id: If6e384522275f73b713fbc29ffcaa1753c239dea Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Nail Samatov [Wed, 15 Sep 2021 18:00:59 +0000 (21:00 +0300)]
[test] Fix closing of test repositories
Fix tests failing on Windows because Repository instance is created but
not closed on tear down.
Fix repositories closed twice, except in tests that test this behavior
explicitly.
Name the temporary directories the tests run in after the test method;
that makes it easier to figure out in which tests repositories are
closed twice if it should occur again in the future.
Bug: 550111
Change-Id: I9398b58f0f36d2c29236d2a9a8599117d9083980 Signed-off-by: Nail Samatov <sanail@yandex.ru> Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Sun, 30 Jan 2022 21:15:14 +0000 (22:15 +0100)]
ObjectWalk: close ObjectReader on close() if needed
If the walk is created via ObjectWalk(Repository), it creates a new
ObjectReader. This reader was closed only on dispose(). If such an
ObjectWalk was used in a try-with-resource statement the reader might
not get closed.
Bug: 578458
Change-Id: I1be31829dc466530f23006a53c29b657fd5fb410 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Sun, 30 Jan 2022 19:59:26 +0000 (20:59 +0100)]
[test] Fix OpenSshConfigFileTest for Windows
The tests assumed that a path like "/tmp" was an absolute path, and
also compared against strings with forward slashes. On Windows, "/tmp"
is not an absolute path and thus resolved against the current directory,
and the separator is a backslash.
Change the tests to use ~/ notation, and test paths resolved against
the (mocked) user home directory. That way, the tests are independent
of the file system used.
Bug: 550111
Change-Id: I1c31608ca83c8d8586256d1586a792e4a33cfaa4 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Thu, 30 Dec 2021 16:49:44 +0000 (17:49 +0100)]
sshd: Add README.md for SSH agents
Explain SSH agent protocols, what transports are available and how to
choose them in ~/.ssh/config. For Windows, add some information on
which commonly used SSH agents can be used.
Change-Id: I0b08a95654fd76643512606edb1ed74d9980aa85 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Wed, 29 Dec 2021 19:33:33 +0000 (20:33 +0100)]
sshd: Skip unknown keys from the SSH agent
An SSH agent might contain keys that Apache MINA sshd cannot handle.
Pageant for instance can contain ed448 keys, which are not implemented
in OpenSSH or in Apache MINA sshd.
When an agent delivers such keys, simply skip (and log) them. That way,
we can work with the remaining keys. Otherwise a single unknown key in
the agent would break pubkey authentication.
Change-Id: I3945d932c7e64b628465004cfbaf10f4dc05f3e4 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Tue, 28 Dec 2021 18:54:30 +0000 (19:54 +0100)]
sshd: support the AddKeysToAgent ssh config
Add parsing of the config. Implement the SSH agent protocol for adding
a key. In the pubkey authentication, add keys to the agent as soon as
they've been loaded successfully, before even attempting to use them
for authentication. OpenSSH does the same.
Bug: 577052
Change-Id: Id1c08d9676a74652256b22281c2f8fa0b6508fa6 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Tue, 28 Dec 2021 17:07:21 +0000 (18:07 +0100)]
sshd: handle "IdentityAgent SSH_AUTH_SOCK" in ssh config
OpenSSH has (for legacy reasons?) the option of specifying the default
environment variable directly, instead of using ${SSH_AUTH_SOCK}. Make
sure the plain variable name is not taken as a relative path name.
Bug: 577053
Change-Id: If8f550dffc43887254f71aa0b487c50fa14d0627 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Mon, 27 Dec 2021 20:39:23 +0000 (21:39 +0100)]
sshd: Connector for the Win32-OpenSSH SSH agent
Win32-OpenSSH uses a named Windows pipe for communication. Implement
a connector for this mechanism using JNA. Choose the appropriate
connector based on the setting of the 'identityAgent' parameter.
Bug: 577053
Change-Id: I205f07fb33654aa18ca5db92706e65544ce38641 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Mon, 27 Dec 2021 18:50:24 +0000 (19:50 +0100)]
sshd: handle IdentitiesOnly with an SSH agent
If an SSH agent is used but "IdentitiesOnly yes" is set, only those
keys from the agent that correspond to one of the keys explicitly given
via an IdentityFile directive are to be used.
Implement this by filtering the list of keys obtained from the agent
against the list of IdentityFiles, each entry suffixed with ".pub".
Load the public keys from these files, and ignore all other keys from
the agent. Keys without ".pub" file are also ignored.
Apache MINA sshd has no operation to load only the public key from a
private key file, so we have to rely on *.pub files.
Bug: 577053
Change-Id: I75c2c0b3ce35781c933ec2944bd6da1b94f4caf9 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Sun, 26 Dec 2021 13:36:48 +0000 (14:36 +0100)]
sshd: support IdentityAgent config
Handle the 'none' value, and change the value to select Pageant to
something that looks like an absolute UNC path name to avoid it's
handled as an relative path name.
Bug: 577053
Change-Id: I4ccf047abbc1def50e2782319e4fa7c744069401 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Nail Samatov [Fri, 21 Jan 2022 16:30:51 +0000 (19:30 +0300)]
LFS: Fix error occurring during delete branch
Fix TransportException occurring when deleting
a branch and push that change to remote repository
if BuiltinLFS is used to work with repository.
When finding LFS pointers in RemoteRepUpdate,
LfsPrePushHook fails to open ObjectReader
with new object id equal to ObjectId.zeroId().
If update is a deleting update (new object id is zero id),
we can assume that this update doesn't contain LFS Pointer
and we can skip step with extracting LFS pointer for that
RemoteRefUpdate.
Bug: 578313
Change-Id: Ic4367978338b8234d39d9af0d9674490f79fc22d Signed-off-by: Nail Samatov <sanail@yandex.ru>
DFS block cache: report index load and evict stats
Enhance cache performance monitoring for large data such as pack and
bitmap indexes. Provide details about what is loaded and evicted from
cache like total number of cache hits, time in cache before eviction.
Add a custom consumer to report loading events and eviction events when
enabled.
David Ostrovsky [Fri, 24 Dec 2021 06:43:48 +0000 (07:43 +0100)]
Bazel: Switch to using toolchain resolution for java rules
Bump Bazel version to release 5.0.0. In this new Bazel release,
--incompatible_use_toolchain_resolution_for_java_rules is flipped, that
means that the build must be adapted to toolchain resolution.
Specification toolchain resolution for java rules is here: [1]. Main
tracking Bazel issue is here: [2].
Given that new Bazel release also added support for remote JDK 17, add
experimental support for building with remote JDK 17 to produce major
byte code version 61.
projects / jgit.git / log