Saša Živkov [Fri, 3 Jun 2022 14:36:43 +0000 (16:36 +0200)]
Fix connection leak for smart http connections
SmartHttpPushConnection: close InputStream and OutputStream after
processing. Wrap IOExceptions which aren't TransportExceptions already
as a TransportException.
Also-By: Matthias Sohn <matthias.sohn@sap.com>
Change-Id: I8e11d899672fc470c390a455dc86367e92ef9076
Andre Bossert [Tue, 21 Jan 2020 09:13:43 +0000 (10:13 +0100)]
Teach JGit to handle external diff/merge tools defined in .gitattributes
Adds API that allows UI to find (and handle) diff/merge tools, specific
for the given path. The assumption is that user can specify file type
specific diff/merge tools via gitattributes.
Andre Bossert [Sun, 19 Jan 2020 19:57:23 +0000 (20:57 +0100)]
Adapt diff- and merge tool code for PGM and EGit usage
see: https://git-scm.com/docs/git-mergetool
* DiffTools and MergeTools
* store FS, gitDir and workTree for usage without
git repository (for EGit preferences)
* add getUserDefinedToolNames() and getPredefinedToolNames()
* replace getToolNames() with getAllToolNames() that combines the two
lists and put default tool name (diff.tool or merge.tool) as first
element (for EGit preferences)
* FileElement: refactoring of getFile() and friends to have midName
(LOCAL, REMOTE etc.) always added to the temp file name (also for EGit)
* FileElement: added directory attribute that is used in getFile() to
return path with workDir as parent
* DiffTool and MergeTool
* added errw.flush(), because sometimes stderr is not printed in case
of die()
* print e.getMessage() always to stderr
* Moved toolname and prompt logic into managers
* Exported internal packages required for egit.ui
Bug: 356832
Change-Id: I71e7f4dc362169a7612ca4f6546a021bc4b2b5f4 Signed-off-by: Andre Bossert <andre.bossert@siemens.com> Signed-off-by: Tim Neumann <Tim.Neumann@advantest.com>
Matthias Sohn [Wed, 1 Jun 2022 16:14:28 +0000 (18:14 +0200)]
Merge branch 'master' into stable-6.2
* master:
Fixed since tags added for new merge constants
Add filtering with help of DirCacheCheckout.getContent()
Avoid warning "no explicit project encoding"
Add mergetool merge feature (execute external tool)
Add command line support for "git mergetool"
Add config reader for user-defined mergetools
Add difftool compare feature (execute external tool)
Andre Bossert [Sun, 19 Jan 2020 19:54:17 +0000 (20:54 +0100)]
Add filtering with help of DirCacheCheckout.getContent()
see: https://git-scm.com/docs/git-mergetool
* refactoring of content (FileElement) handling
* now the temporary files are already filled with filtered content in
the calling classes (PGM), that can be used with EGit content too
TODO:
* keep the temporaries when no change detected and the user answers no
to the question if the merge was successful
Bug: 356832
Change-Id: I86a0a052d059957d4d152c1bb94c262902c377d2 Signed-off-by: Andre Bossert <andre.bossert@siemens.com>
Remove stray files (probes or lock files) created by background threads
NOTE: port back from master branch.
On process exit, it was possible that the filesystem timestamp
resolution measurement left behind .probe files or even a lock file
for the jgit.config.
Ensure the SAVE_RUNNER is shut down when the process exits (via
System.exit() or otherwise). Move lf.lock() into the try-finally
block when saving the config file.
Delete .probe files on JVM shutdown -- they are created in daemon
threads that may terminate abruptly, not executing the "finally"
clause that normally removes these files.
Michael Keppler [Thu, 26 May 2022 21:17:44 +0000 (23:17 +0200)]
Avoid warning "no explicit project encoding"
Eclipse 2022-06 raises that warning for each project without explicit
encoding. We can avoid the warning by adding explicit project settings
for the otherwise implicit encoding.
There is no functional change, neither for users of Eclipse nor for
users of other IDEs or build tools.
Signed-off-by: Michael Keppler <Michael.Keppler@gmx.de>
Change-Id: I30a6f1369ef09bd445f5730628d01772db2ee7b3
Matthias Sohn [Wed, 25 May 2022 14:47:37 +0000 (16:47 +0200)]
Merge branch 'master' into stable-6.2
* master:
RewriteGenerator: Fully buffering of input is no longer necessary
Update Orbit to S20220517184036
PackFileSnapshot: Resolve warnings by using "equals" instead of "=="
Remove stray files (probes or lock files) created by background threads
* add command line support for "git mergetool"
* add option handling for "--tool-help", "--tool=<mytool>",
"--[no-]prompt", "--[no-]gui"
* handle prompt
* add MergeTools
* add pre-defined mergetools
* print merge actions --> no execute, will be done later
Bug: 356832
Change-Id: I6e505ffc3d03f75ecf4bba452a25d25dfcf5793f Signed-off-by: Andre Bossert <andre.bossert@siemens.com>
* add CommandExecutor that handles tool execution with help of "jgit.FS"
* it handles tool execution with temporary created "command file" -->
for for all "command interpreters" and parameters with spaces etc.
* using of external bash.exe at Windows (MinGW) if shell-script is
used as difftool command. It can be enabled with parameter
"jgit.usemsys2bash=auto" that checks if command contains ".sh" or
enabled / disabled with "jgit.usemsys2bash=true|false"
* added special handling for empty files (e.g. deleted, added etc.) that
are named "/dev/null"
* added creation and deletion of temporary files needed for compare
* added own Exception class for reporting to pgm / command line / EGit
* added prompt option handling before executing difftool
* reworked trustExitCode option for specific difftool and override for
all difftools from config and command line
* tested with command line options "--[no]-trust-exit-code",
"--tool=<toolname>", "--[no]-gui", --[no]-prompt
* ContentSource
* added close() methods to close / cleanup used resources
(like ObjectReader TreeWalk etc.)
* added isWorkingTreeSource() methods to check if file can be used
from working tree instead of copy from "ObjectLoader / ObjectReader" to
temporary file (fixes "difftool <commit> <commit>")
Remove stray files (probes or lock files) created by background threads
On process exit, it was possible that the filesystem timestamp
resolution measurement left behind .probe files or even a lock file
for the jgit.config.
Ensure the SAVE_RUNNER is shut down when the process exits (via
System.exit() or otherwise). Move lf.lock() into the try-finally
block when saving the config file.
Delete .probe files on JVM shutdown -- they are created in daemon
threads that may terminate abruptly, not executing the "finally"
clause that normally removes these files.
Sven Selberg [Tue, 12 Apr 2022 09:36:58 +0000 (11:36 +0200)]
HTTP Smart: set correct HTTP status on error
Previous behavior was that status code was automatically set to 200
regardless of reported status and according to HTTP Smart protocol[1]:
If there is no repository at $GIT_URL, or the resource pointed to by
a location matching $GIT_URL does not exist, the server MUST NOT
respond with 200 OK response. A server SHOULD respond with
404 Not Found, 410 Gone, or any other suitable HTTP status code which
does not imply the resource exists as requested.
Since the jgit HTTP client isn't able to handle reading content from a
response reporting an error (calling HttpURLConnection#getInputStream
on a "failed" connection throws an exception and the internal interface
HttpConnection does not expose HttpURLConnection#getErrorStream) the
SmartClientSmartServerTest needed to be rewritten to expect the generic
response messages.
Thomas Wolf [Fri, 1 Apr 2022 14:56:05 +0000 (16:56 +0200)]
[sshd] Better user feedback on authentication failure
When authentication fails, JGit produces an exception with an error
message telling the user that it could not log in (including the host
name). The causal chain has an SshException from Apache MINA sshd with
message "No more authentication methods available".
This is not very helpful. The user was left without any indication why
authentication failed.
Include in the exception message a log of all attempted authentications.
That way, the user can see which keys were tried, in which order and
with which signature algorithms. The log also reports authentication
attempts for gssapi-with-mic or password authentication. For
keyboard-interactive Apache MINA sshd is lacking a callback interface.
The way Apache MINA sshd loads keys from files, the file names are lost
in higher layers. Add a mechanism to record on the session for each
key fingerprint the file it was loaded from, if any. That way the
exception message can refer to keys by file name, which is easier to
understand by users than the rather cryptic key fingerprints.
Bug: 571390
Change-Id: Ic4b6ce6b99f307d5e798fcc91b16b9ffd995d224 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Sun, 3 Apr 2022 19:06:57 +0000 (21:06 +0200)]
De-couple ServiceLoader calls from class loading
Use the holder pattern to de-couple the loading of super classes from
the ServiceLoader calls to set up global instances. This prevents
potential lock inversions.
Bug: 579550
Change-Id: Ie8284e4d6d680ddd4cc6a486bbefe8ed00266240 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Sun, 3 Apr 2022 18:11:29 +0000 (20:11 +0200)]
GpgSigner: prevent class lock inversion on the default signer
Don't store the default signer in a static field of the abstract
superclass GpgSigner. This many lead to a lock inversion on the class
initialization locks if there are concurrent loads of the GpgSigner
class and of one of its subclasses, and that subclass happens to be
the one the ServiceLoader wants to load.
Use the holder pattern to de-couple the loading of class GpgSigner
from the ServiceLoader call.
Bug: 579550
Change-Id: Ifac0ea0c8985a09fe0518d0dabc072fafd6db907 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
yunjieli [Mon, 28 Mar 2022 21:47:02 +0000 (14:47 -0700)]
Fetch: Introduce negative refspecs.
Implement negative refspecs in JGit fetch, following C Git. Git
supports negative refspecs in source only while this change supports
them in both source and destination.
If one branch is equal to any branch or matches any pattern in the
negative refspecs collection, the branch will not be fetched even if
it's in the toFetch collection.
With this feature, users can express more complex patterns during fetch.
Introduce a new benchmark that shows a typical use-case
of opening a cached repository and fetching one ref from
a repository with a high number of refs.
This specific benchmark is tailored to the Gerrit use-case
of reading frequently individual refs SHA1s and by ref-name
prefix from the All-Users repository.
Include the following variables for the benchmark:
- numBranches (from 100 up to 50000)
- trustFolderStat (true or false)
- useRefTable (true or false)
The benchmark needs to be evaluated on a local high-perf SSD
and on a slower NFS network disk.
Matthias Sohn [Wed, 16 Mar 2022 15:32:06 +0000 (16:32 +0100)]
[pgm tests] Relax version constraints for hamcrest
We updated hamcrest to 2.2. but there is no need to prevent that
older versions of hamcrest can be used. Hence relax the lower bound
to 1.1 like in other bundles.
Thomas Wolf [Sun, 23 Jan 2022 14:43:24 +0000 (15:43 +0100)]
Use git config core.commentChar
This concerns committing, creating merge conflict messages and creating
and editing squash messages. In a squash message, once the comment
character has been determined initially is always the first character.
Note that if core.commentChar=auto and there is a sequence of squashes,
it may be necessary to change the comment character when a new message
is added.
Bug: 579325
Change-Id: Idca19284a0240cd322e7512ea299a03658e1b2c1 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Sun, 20 Feb 2022 23:20:52 +0000 (00:20 +0100)]
[push, lfs] Tell the pre-push hook whether the push is a dry run
This is a feature that does not exist in C git: an external pre-push
hook doesn't know whether the push is run as a dry run. But for
internal hooks written in Java it is entirely possible to give a hook
this information.
In JGit with its internal LFS implementation, this enables us to not
perform LFS uploads in a dry run. This is kind of important because
EGit frequently does a dry-run before doing the actual push to give the
user a way to review what would be pushed before it actually happens.
Doing an LFS upload of potentially huge files during a dry-run is
wasteful, makes the dry run not actually a dry run, and leads to
uploading the same file twice if followed by a real push.
Use the information in the LfsPrePushHook to only do the initial call
to the LFS server, but then skipping the actual upload if the push is
a dry run. That way, a failure to contact the LFS server leads to an
error in the dry run, as it should.
Bug: 546567
Change-Id: I155430f27c4979d91096ba72fd95c3775dd3f28b Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Tue, 15 Mar 2022 21:48:48 +0000 (22:48 +0100)]
DirCacheCheckout: use a LinkedHashMap instead of HashMap
This guarantees that updates are checked out in git order, which
is important for LFS if a .lfsconfig file is used. That file comes
early in git order, and the LFS smudge filter will consider the
working tree version. To ensure that on branch switches the correct
version of that file is used, the checkout order must be stable and
should be the git order.
Change-Id: I20f6d11bf08558f9d5adfd2be71e36321460038c Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Tue, 15 Mar 2022 18:12:46 +0000 (19:12 +0100)]
Re-try reading a file when there are concurrent writes
Git and JGit are very careful to replace git files atomically when
writing. The normal mechanism for this is to write to a temporary
file and then to rename it atomically to the final destination. This
works fine on POSIX-compliant systems, but on systems where renaming
may not be atomic, exceptions may be thrown if code tries to read
the file while the rename is still ongoing. This happens in particular
on Windows, where the typical symptom is that a FileNotFoundException
with message "The process cannot access the file because it is being
used by another process" is thrown, but file.isFile() == true at the
same time.
In FileBasedConfig, a re-try was already implemented for this case.
But the same problem can also occur in other places, for instance
in RefDirectory when reading loose or packed refs. Additionally,
JGit has similar re-tries when a stale NFS file handle is detected,
but that mechanism wasn't used consistently (only for git configs
and packed refs, but not for loose refs).
Factor out the general re-try mechanism for reading into a new method
FileUtils.readWithRetry() and use that in all three places. The
re-try parameters are hardcoded: at most 5 times for stale NFS handles,
and at most 5 times with increasing backoff delays (50, 100, 200, 400,
and 800ms) for the above concurrent write case.
Bug: 579116
Change-Id: If0c2ad367446d3c0f32b509274cf8e814aca12cf Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Sat, 26 Feb 2022 11:46:50 +0000 (12:46 +0100)]
[sideband] Ensure last bit of progress channel is written
If the last sideband progress message didn't end in \r or \n, there
may still be a buffered message at the end of a fetch or push. Ensure
that message gets written, too, even if it may be only partial.
Bug: 575629
Change-Id: I38edccb5cffb89e00e468480b43c7d951fb63e8e Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
and LsRemoteCommand is called with an HTTPS GitHub repository URL, the
command should actually rewrite this to an SSH URI and use the SSH
transport.
Actually this same problem may exist everywhere Transport is used with
an URIish instead of with a remote name. However, doing this translation
in Transport.open(URIish) and in Transport.open(Repository, URIish,
String) if no remote name is given would change the behavior and might
break assumptions made in existing clients. For instance, EGit's
PushOperation assumes that the URI obtained from PushResult.getURI()
was the same as was passed in to Transport.open(Repository, URIish).
URIs obtained from a RemoteConfig have this translation applied
transparently, and in Transport we cannot know for sure whether or
not a URI has already been translated, if needed. So doing this in
Transport might also lead to translating URIs twice.
Therefore this commit does the translation in LsRemoteCommand, where
we can be sure that it won't affect other use cases. If other cases
besides LsRemoteCommand are found where such a URI translation is
missing, it'll have to be done at higher levels, possibly even in
client code directly.
Bug: 544769
Change-Id: I5df54a925d30b55d98e21f37f2851fe79649b064 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Matthias Sohn [Tue, 8 Mar 2022 16:23:57 +0000 (17:23 +0100)]
Merge branch 'stable-6.1'
* stable-6.1:
Prepare 6.1.1-SNAPSHOT builds
JGit v6.1.0.202203080745-r
[checkout] Use .gitattributes from the commit to be checked out
Don't use final for method parameters
[push] support the "matching" RefSpecs ":" and "+:"
[push] Call the pre-push hook later in the push process
IndexDiff: use tree filter also for SubmoduleWalk
Run license check with option -Ddash.projectId=technology.jgit
Exclude transitive dependencies of sshd-sftp
Update DEPENDENCIES for 6.1.0 release
Add dependency to dash-licenses
Fix typos of some keys in LfsText
Sort LfsText entries alphabetically
Support for "lfs.url" from ".lfsconfig"
Thomas Wolf [Sun, 13 Feb 2022 22:30:36 +0000 (23:30 +0100)]
[checkout] Use .gitattributes from the commit to be checked out
JGit used only one set of attributes constructed from the global and
info attributes, plus the attributes from working tree, index, and
HEAD.
These attributes must be used to determine whether the working tree is
dirty.
But for actually checking out a file, one must use the attributes from
global, info, and *the commit to be checked out*. Otherwise one may not
pick up definitions that are only in the .gitattributes of the commit
to be checked out or that are changed in that commit with respect to
the attributes currently in HEAD, the index, or the working tree.
Maintain in TreeWalk different Attributes per tree, and add operations
to determine EOL handling and smudge filters per tree.
Use the new methods in DirCacheCheckout and ResolveMerger. Note that
merging in JGit actually used the attributes from the base, not those
from ours, which looks dubious at least. It now uses those from ours,
and for checking out the ones from theirs.
The canBeContentMerged() determination was also done from the base
attributes, and is newly done from the ours attributes. Possibly this
should take into account all three attributes, and only if all three
agree the item can be content merged, a content merge should be
attempted? (What if the binary/text setting changes between base, ours,
or theirs?)
Also note that JGit attempts to perform content merges on non-binary
LFS files; there it used the filter attribute from base, too, even for
the ours and theirs versions. Newly it takes the filter attribute from
the correct tree. I'm not convinced doing content merges on potentially
huge files like LFS files is really a good idea.
Add tests in FilterCommandsTest and LfsGitTest to verify the behavior.
Open question: using index and working tree as fallback for the
attributes of ours (assuming it is HEAD) is OK. But does it also make
sense for base and theirs in merging?
Bug: 578707
Change-Id: I0bf433e9e3eb28479b6272e17c0666e175e67d08 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Sun, 20 Feb 2022 17:11:49 +0000 (18:11 +0100)]
[push] support the "matching" RefSpecs ":" and "+:"
The implementation of push.default=matching was not correct.
It used the RefSpec "refs/heads/*:refs/heads/*", which would push
_all_ local branches. But "matching" must push only those local
branches for which a remote branch with the same name already exists
at the remote.
This RefSpec can be expanded only once the advertisement from the
remote has been received.
Enhance RefSpec so that ":" and "+:" can be represented. Introduce a
special RemoteRefUpdate for such a RefSpec; it must carry through the
fetch RefSpecs to be able to fill in the remote tracking updates as
needed. Implement the expansion in PushProcess.
Bug: 353405
Change-Id: I54a4bfbb0a6a7d77b9128bf4a9c951d6586c3df4 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Sun, 20 Feb 2022 17:10:24 +0000 (18:10 +0100)]
[push] Call the pre-push hook later in the push process
Call the pre-push hook only after having received the remote
advertisement and having determined rejections, like C git does.
Also similar to C git, don't pass rejected or up-to-date updates
to the pre-push hook.
Bug: 578852
Change-Id: I51d379ea7bd8234ec815f8f4a9fa325816f476cf Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Fri, 17 Jul 2020 17:41:31 +0000 (19:41 +0200)]
IndexDiff: use tree filter also for SubmoduleWalk
The only uses of IndexDiff.setFilter() in JGit and EGit set a path
filter. Passing the filter on to the SubmoduleWalk gives the desired
result, which is consistent with command-line git.
Bug: 565251
Change-Id: I8eca1ed73eb1d237b8785f369352f72af9e0e168 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Matthias Sohn [Thu, 3 Mar 2022 23:30:36 +0000 (00:30 +0100)]
Exclude transitive dependencies of sshd-sftp
We don't need the dependencies of sshd-sftp to sshd-common and
sshd-core since they are contained in sshd-osgi.
Excluding them helps the dash IP log tool to not list them as required
dependencies of jgit.
projects / jgit.git / log