I had misunderstood how protocol V2 works. This implementation only
works if the negotiation during fetch is done in one round.
Fixing this is substantial work in BasePackFetchConnection. Basically
I think I'd have to change back negotiate to the V0 version, and have
a doFetch() that does
if protocol V2
doFetchV2()
else
doFetchV0()
with doFetchV0 the old code, and doFetchV2 completely new.
Plus there would need to be a HTTP test case requiring several
negotiation rounds.
This is a couple of days work at least, and I don't know when I will
have the time to revisit this. So although the rest of the code is
fine I prefer to back this out completely and not leave a only half
working implementation in the code for an indeterminate time.
Bug: 553083
Change-Id: Icbbbb09882b3b83f9897deac4a06d5f8dc99d84e Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Nail Samatov [Tue, 27 Oct 2020 14:37:24 +0000 (17:37 +0300)]
Close Repository to fix tests failing on Windows
Fix tests failing on Windows because Repository
instance is created but not closed on tear down.
Change-Id: I72488ba5efeec95110926b1fbd56b7d96fca0d37 Signed-off-by: Nail Samatov <sanail@yandex.ru> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Thomas Wolf [Sun, 2 Aug 2020 17:22:05 +0000 (19:22 +0200)]
Client-side protocol V2 support for fetching
Make all transports request protocol V2 when fetching. Depending on
the transport, set the GIT_PROTOCOL environment variable (file and
ssh), pass the Git-Protocol header (http), or set the hidden
"\0version=2\0" (git anon). We'll fall back to V0 if the server
doesn't reply with a version 2 answer.
A user can control which protocol the client requests via the git
config protocol.version; if not set, JGit requests protocol V2 for
fetching. Pushing always uses protocol V0 still.
In the API, there is only a new Transport.openFetch() version that
takes a collection of RefSpecs plus additional patterns to construct
the Ref prefixes for the "ls-refs" command in protocol V2. If none
are given, the server will still advertise all refs, even in protocol
V2.
BasePackConnection.readAdvertisedRefs() handles falling back to
protocol V0. It newly returns true if V0 was used and the advertised
refs were read, and false if V2 is used and an explicit "ls-refs" is
needed. (This can't be done transparently inside readAdvertisedRefs()
because a "stateless RPC" transport like TransportHttp may need to
open a new connection for writing.)
BasePackFetchConnection implements the changes needed for the protocol
V2 "fetch" command (simplified ACK handling, delimiters, section
headers).
In TransportHttp, change readSmartHeaders() to also recognize the
"version 2" packet line as a valid smart server indication.
Adapt tests, and run all the HTTP tests not only with both HTTP
connection factories (JDK and Apache HttpClient) but also with both
protocol V0 and V2. Do the same for the SSH transport tests.
Bug: 553083
Change-Id: Ice9866aa78020f5ca8f397cde84dc224bf5d41b4 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
John Dallaway [Wed, 2 Sep 2020 13:15:23 +0000 (14:15 +0100)]
Ensure .gitmodules is loaded when accessing submodule name
This problem occurred when calling SubmoduleWalk#getModuleName if the
first submodule processed has a name and a path which do not match
SubmoduleWalk#getModuleName returned the module path instead of the
module name. In order to fix this SubmoduleWalk#getModuleName needs to
ensure that the modules config is loaded.
Bug: 565776
Change-Id: I36ce1fbc64c4849f9d8e39864b825c6e28d344f8 Signed-off-by: John Dallaway <john@dallaway.org.uk> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Alexa Panfil [Mon, 12 Oct 2020 08:55:46 +0000 (08:55 +0000)]
Add new performance logging
Add new performance logging to register events of type duration.
The proposed logging is similar to the performance logging
in OS Gerrit https://gerrit-review.googlesource.com/c/gerrit/+/225628:
a global Singleton (LoggingContext in Gerrit) is
collecting the performance logs in a thread-safe events list,
and at the end of the monitored command the list of events is
retrieved and written to a log, after which it is cleared.
What this patch does:
The main component is the Singleton (PerformanceLogContext), which
is used to collect the records (PerformanceLogRecord) in one place
(ThreadLocal eventsList) from anywhere using
PerformanceLogContext.getInstance().addEvent().
Reason why this change is needed:
The current monitoring in JGit has several issues:
1. git fetch and git push events are handled separately
(PackStatistics and ReceivedPackStatistics), with no unified way
of writing or reading the statistics.
2. PostUploadHook is only invoked on the event of sending the
pack, which means that the PackStatistics is not available for
the fetch requests that did not end with sending the pack
(negotiation requests).
3. The way the logs are created is different from the performance
log approach, so the long-running operations need to be collected
from both performance log (for JGit DFS overridden operations and
Gerrit operations) and gitlog (for JGit ones).
The proposed performance logging is going to solve the above
mentioned issues: it collects all of the performance logs in one
place, thus accounting for the commands that do not result in
sending a pack. The logs are compatible with the ones on Gerrit.
Moreover, the Singleton is accessible anywhere in the call stack,
which proved to be successful in other projects like Dapper
(https://research.google/pubs/pub36356/).
Alexa Panfil [Fri, 9 Oct 2020 16:52:04 +0000 (16:52 +0000)]
Compute time differences with Duration
Reason why this change is needed:
Currently the durations of fetch events are computed by
registering time instants with System.currentTimeMillis()
and calculating the differences with simple minus operation,
but multiple sources suggest that the best practice is to use
the Java 8 Duration and Instant objects.
What this patch does:
Get time measurements with Instant.now() instead of
System.currentTimeMillis() and calculate the duration of fetch
events (Reachability checks and Negotiation) using
Duration.between().toMillis().
David Ostrovsky [Fri, 2 Oct 2020 16:49:21 +0000 (18:49 +0200)]
Fix OperatorPrecedence warning flagged by error prone
Building with Bazel is failing with this error message:
HttpSupport.java:480: error: [OperatorPrecedence] Use grouping
parenthesis to make the operator precedence explicit
if (c >= 'a' && c <= 'z' || c >= 'A' && c <= 'Z') {
^
(see https://errorprone.info/bugpattern/OperatorPrecedence)
Did you mean 'if ((c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z')) {'?
James Wynn [Mon, 31 Aug 2020 20:32:13 +0000 (15:32 -0500)]
Support "http.userAgent" and "http.extraHeader" from the git config
Validate the extra headers and log but otherwise ignore invalid
headers. An empty http.extraHeader starts the list afresh.
The http.userAgent is restricted to printable 7-bit ASCII, other
characters are replaced by '.'.
Moves a support method from the ssh.apache bundle to HttpSupport in
the main JGit bundle.
Bug:541500
Change-Id: Id2d8df12914e2cdbd936ff00dc824d8f871bd580 Signed-off-by: James Wynn <james@jameswynn.com> Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Thu, 24 Sep 2020 19:34:29 +0000 (21:34 +0200)]
Add TypedConfigGetter.getPath()
This enables EGit to override with a lenient variant that logs the
problem and continues with the default value. EGit needs this because
otherwise a corrupt core.excludesFile entry can render the whole UI
unusable (until the git config is fixed) because any use of the
WorkingTreeIterator will throw an InvalidPathException.
This is not a problem on OS X, where all characters are allowed in
file names. But on Windows some characters are forbidden... see bug
567296. The message of the InvalidPathException is not helpful since
it doesn't point to the origin of the problem. EGit can log a much
better message indicating the offending config file and entry in the
Eclipse error log, where the user can see it.
Bug: 567309
Change-Id: I4e57afa715ff3aaa52cd04b5733f69e53af5b1e0 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Reason why this change is needed:
Getting this metric will help estimate how much time is spent
on negotiation in fetch V2.
What this patch does:
Measure time spent on negotiation rounds in UploadPack.fetchV2()
and save it in an instance of PackStatistics.Accumulator.
This is the same way the statistics are already gathered for
protocol V0/V1.
Thomas Wolf [Thu, 10 Sep 2020 21:39:07 +0000 (23:39 +0200)]
IndexDiffFilter: handle path prefixes correctly
When comparing git directory paths to check whether one is a prefix
of another, one must add a slash to avoid false prefix matches when
one directory name is a prefix of another. The path "audio" is not
a prefix of the path "audio-new", but would be a prefix of a path
"audio/new".
Bug: 566799
Change-Id: I6f671ca043c7c2c6044eb05a71dc8cca8d0ee040 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Sun, 26 Jul 2020 18:37:57 +0000 (20:37 +0200)]
sshd: support the ProxyJump ssh config
This is useful to access git repositories behind a bastion server
(jump host).
Add a constant for the config; rewrite the whole connection initiation
to parse the value and (recursively) set up the chain of hops. Add
tests for a single hop and two different ways to configure a two-hop
chain.
The connection timeout applies to each hop in the chain individually.
Change-Id: Idd25af95aa2ec5367404587e4e530b0663c03665 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Yunjie Li [Wed, 25 Mar 2020 17:59:06 +0000 (10:59 -0700)]
ReceivePackStats: Add size and count of unnecessary pushed objects
Since there is no negotiation for a push, the client is probably sending
redundant objects and bytes which already exist in the server.
Add more metrics in the stats to quantify it. Duplicated size and number
to measure the size and the number of duplicated objects which should
not be pushed.
Change-Id: Iaacd4761ee9366a0a7ec4e26c508eff45c8744de Signed-off-by: Yunjie Li <yunjieli@google.com>
Matthias Sohn [Tue, 8 Sep 2020 22:12:45 +0000 (00:12 +0200)]
Merge branch 'stable-5.9' into master
* stable-5.9:
Prepare 5.9.1-SNAPSHOT builds
JGit v5.9.0.202009080501-r
[releng] Enable japicmp for the fragments added in 5.8.0
GitlinkMergeTest: fix boxing warnings
Remove unused API problem filters
Add missing since tag on BundleWriter#addObjectsAsIs
GPG: include signer's user ID in the signature
Change-Id: Iaa96f9228752540f446fc232a49f31a738fd8d30 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Sat, 5 Sep 2020 20:51:02 +0000 (22:51 +0200)]
Merge branch 'master' into stable-5.9
* master:
SshdSession: close channel gracefully
jgit: Add DfsBundleWriter
Prepare 5.10.0-SNAPSHOT builds
ResolveMerger: do not content-merge gitlinks on del/mod conflicts
ResolveMerger: Adding test cases for GITLINK deletion
ResolveMerger: choose OURS on gitlink when ignoreConflicts
ResolveMerger: improving content merge readability
ResolveMerger: extracting createGitLinksMergeResult method
ResolveMerger: Adding test cases for GITLINK merge
Back out the version change to 5.10.0-SNAPSHOT which was done on master
already.
Change-Id: I1a6b1f0b8f5773be47823d74f593d13b16a601d5 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Thomas Wolf [Wed, 17 Jun 2020 16:43:32 +0000 (18:43 +0200)]
GPG: include signer's user ID in the signature
Signing a commit with command line git and gpg 2.2.20 includes the
e-mail part of the key's user ID as a "Signer's User ID" subpacket
on the signature.
Implement this for signing via Bouncy Castle.
Bug: 564386
Change-Id: I68906b895349359596cf3451d65f2840c60df856 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Masaya Suzuki [Fri, 24 Jan 2020 00:47:40 +0000 (16:47 -0800)]
jgit: Add DfsBundleWriter
DfsBundleWriter writes out the entire repository to a Git bundle file.
It packs all objects included in the packfile by concatenating all pack
files. This makes the bundle creation fast and cheap. Useful for backing
up a repository as-is.
Terry Parker [Thu, 3 Sep 2020 22:35:24 +0000 (18:35 -0400)]
Merge changes from topic "fix_ui"
* changes:
ResolveMerger: do not content-merge gitlinks on del/mod conflicts
ResolveMerger: Adding test cases for GITLINK deletion
ResolveMerger: choose OURS on gitlink when ignoreConflicts
ResolveMerger: improving content merge readability
ResolveMerger: extracting createGitLinksMergeResult method
ResolveMerger: Adding test cases for GITLINK merge
Matthias Sohn [Sun, 30 Aug 2020 22:37:03 +0000 (00:37 +0200)]
Update target platform to R20200831200620
and update to new Orbit version of the following libraries
- org.apache.commons.codec to 1.14.0.v20200818-1422
- org.apache.httpcomponents.httpclient to 4.5.10.v20200830-2311
- org.bouncycastle.bcpg [1.65.0.v20200527-1955
- org.bouncycastle.bcpkix [1.65.0.v20200527-1955
- org.bouncycastle.bcprov [1.65.1.v20200529-1514
Fix minor issues in target platforms and rename 4.16-staging to 4.16
since that version was released already.
Change-Id: Ifea2600e445e783807a5d94ea23ac3c6550956b5 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Demetr Starshov [Wed, 12 Aug 2020 22:01:10 +0000 (15:01 -0700)]
ResolveMerger: do not content-merge gitlinks on del/mod conflicts
Previously ResolveMerger tried to make a fulltext merge entry in case
one of sides got deleted regardless of file mode. This is not
applicable for GITLINK type of entry. After this change it is
rendering appropriate merge result.
Demetr Starshov [Wed, 12 Aug 2020 21:47:15 +0000 (14:47 -0700)]
ResolveMerger: choose OURS on gitlink when ignoreConflicts
Option ignoreConflicts is used when a caller want to create a virtual
commit and use it in a future merge (recursive merge) or show it on
UI (e.g. Gerrit). According to contract in case of ignoreConflicts
ResolveMerger should populate only stage 0 for entries with merge
conflicts as if there is no conflict. Current implementation breaks
this contract for cases when gitlink revision is ambiguous.
Therefore, always select 'ours' when we merge in ignoreConflicts mode.
This will satisfy the contract contract, so recursive merge can
succeed, however it is an arbitrary decision, so it is not guaranteed
to select best GITLINK in all cases.
GITLINK merging is a special case of recursive merge because of
limitations of GITLINK type of entry. It can't contain more than 1 sha-1
so jgit can't write merge conflicts in place like it can with a blob.
Ideally we could signal the conflict with a special value (like
'0000...'), but that must be supported by all tooling (git fsck, c-git)."
Demetr Starshov [Wed, 12 Aug 2020 00:57:10 +0000 (17:57 -0700)]
ResolveMerger: Adding test cases for GITLINK merge
Add test cases which cover content-merge resolve logic.
Git clients try to agressively merge blobs by content, but GITLINK types
of entries can't be merged with each other or with blobs. This change
ensures all possible permutations which can trigger blob and GITLINK
content merge are covered.
Marc Strapetz [Wed, 12 Aug 2020 12:50:20 +0000 (14:50 +0200)]
Fix possible NegativeArraySizeException in PackIndexV1
Due to an integer overflow bug, the current "Index file is too large
for jgit" check did not work properly and subsequently a
NegativeArraySizeException was raised.
Change-Id: I2736efb28987c29e56bc946563b7fa781898a94a Signed-off-by: Marc Strapetz <marc.strapetz@syntevo.com>
Thomas Wolf [Fri, 14 Aug 2020 12:54:38 +0000 (14:54 +0200)]
FS: use binary search to determine filesystem timestamp resolution
Previous code used a minimum granularity of 1 microsecond and would
iterate 233 times on a system where the resolution is 1 second (for
instance, Java 8 on Mac APFS).
New code uses a binary search between the maximum we care about (2
seconds) and zero, with a minimum granularity of also 1 microsecond.
This takes at most 19 iterations (guaranteed). For a file system with 1
second resolution, it takes 4 iterations (1s, 0.5s, 0.8s, 0.9s). With
an up-front check at 1 microsecond and at 1 millisecond this performs
equally well as the old code on file systems with a fine resolution.
(For instance, Java 11 on Mac APFS.)
Also handle obscure cases where the file timestamp implementation may
yield bogus values (as observed on HP NonStop). If such an error case
occurs, log a warning and abort the measurement at the last good value.
Bug: 565707
Change-Id: I82a96729b50c284be7c23fbdf3d0df1bddf60e41 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Matthias Sohn [Sat, 22 Aug 2020 19:19:57 +0000 (21:19 +0200)]
Do not prematurely create directory of jgit's XDG config file
LockFile.lock() will create it anyway when the config file is created.
Bug: 565637
Change-Id: I078b89a695193fd76f130f6de7ac1cf26d2f8f0f Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Thu, 20 Aug 2020 20:43:13 +0000 (22:43 +0200)]
FS: write to JGit config in a background thread
Otherwise locking problems may ensue if the JGit config itself is
on a different file system. Since the internal is already updated,
it is not really important when exactly the value gets persisted.
By queueing up separate Runnables executed by a single thread we
avoid concurrent write access to the JGit config, and nested calls
to getFileStoreAttributes(Path) result in serialized attempts to
write.
The thread for writing the config must not be a daemon thread. If
it were, JVM shutdown might kill it anytime, which may lead to
the config not being written, or worse, a config.lock file being
left behind.
Bug: 566170
Change-Id: I07e3d4c5e029d3cec9ab5895002fc4e0c7948c40 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Thu, 20 Aug 2020 20:15:11 +0000 (22:15 +0200)]
FS: don't cache fallback if running in background
If the background job is a little late, the true result might
arrive and be cached later. So make sure we don't cache the large
fallback resolution in the per-directory cache. Otherwise we'd work
with the large fallback until the next restart.
Bug: 566170
Change-Id: I7354a6cfddfc0c05144bb0aa41c23029bd4f6af0 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Tue, 21 Jul 2020 22:51:24 +0000 (00:51 +0200)]
Keep line endings for text files committed with CR/LF on text=auto
Git never converts line endings if the version in the repository is a
text file with CR/LF and text=auto. See [1]: "When the file has been
committed with CRLF, no conversion is done."
Because the sentence just before is about converting line endings on
check-in, I had understood that in commit 60cf85a [2] to mean that no
conversion on check-in was to be done. However, as bug 565048 and a
code inspection of the C git code showed it really means no conversion
is done on check-in *or check-out*.
If the text attribute is not set but core.autocrlf = true, this is
the same as text=auto eol=crlf. C git does not convert on check-out
even on text=auto eol=lf if the index version is a text file with
CR/LF.
For check-in, one has to look at the intended target, which is done
in WorkingTreeIterator since commit 60cf85a. For check-out, it can
be done by looking at the source and can thus be done in the
AutoLFOutputStream.
Additionally, provide a constructor for AutoLFInputStream to do
the same; for cases where the equivalent of a check-out is done via
an input stream obtained from a blob. (EGit does that in its
GitBlobStorage for the Eclipse compare framework; it's more efficient
than using a TemporaryBuffer and DirCacheCheckout.getContent(), and
it avoids the need for a temporary file.)
Adapt existing tests, and add new checkout and merge tests to verify
the resulting files have the correct line endings.
EGit's GitBlobStorage will need to call the new version of
EolStreamTypeUtil.wrapInputStream().
The WindowCache is configured statically with a default
WindowCacheConfig. The default config says (for backwards
compatibility reasons) to publish the MBean. As a result,
the MBean always gets published.
By delaying the MBean registration until the first call to
getInstance() or get(PackFile, long) we can avoid the forced
registration and do it only if not re-configured in the meantime
not to publish the bean. (As is done by Egit, to avoid a very
early costly access to the user and system config during plug-in
activation.)
Bug: 563740
Change-Id: I8a941342c0833acee2107515e64299aada7e0520 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Mon, 10 Aug 2020 19:38:50 +0000 (21:38 +0200)]
DirCache: support index V4
Index format version 4 was introduced in C git in 2012. It's about
time that JGit can deal with it.
Version 4 added prefix path compression. Instead of writing the full
path for each index entry to disk, only the difference to the previous
entry's path is written: a variable-encoded int telling how many bytes
to remove from the previous entry's path to get the common prefix,
followed by the new suffix.
Also, cache entries in a version 4 index are not padded anymore.
Internally, version 3 and version 4 index entries are identical; it's
only the stored format that changes.
Implement this path compression, and make sure we write an index file
that we read previously in the same format. (Only changing from version
2 to version 3 if there are extended flags.)
Add support for the "feature.manyFiles" and the "index.version" git
configs, and honor them when writing a new index file.
Add tests, including a compatibility test that verifies that JGit can
read a version 4 index generated by C git and write an identical
version 4 index.
Bug: 565774
Change-Id: Id83241cf009e50f950eb42f8d56b834fb47da1ed Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Sat, 8 Aug 2020 09:53:43 +0000 (11:53 +0200)]
Update javadoc for RemoteSession and SshSessionFactory
The timeout on RemoteSession.exec() cannot be a timeout for the
whole command. It can only be a timeout for setting up the process;
after that it's the application's responsibility to implement some
timeout for the execution of the command, for instance by calling
Process.waitFor(int, TimeUnit) or through other means.
Sessions returned by an SshSessionFactory are already connected and
authenticated -- they must be, because RemoteSession offers no
operations for connecting or authenticating a session.
Change the implementation of SshdExecProcess.waitFor() to wait
indefinitely. The original implementation used the timeout from
RemoteSession.exec() because of that erroneous javadoc.
Change-Id: I3c7ede24ab66d4c81f72d178ce5012d383cd826e Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Thu, 6 Aug 2020 21:43:40 +0000 (23:43 +0200)]
Fix JSchProcess.waitFor() with time-out
SshSupport.runSshCommand() had a comment that wait with time-out
could not be used because JSchProcess.exitValue() threw the wrong
unchecked exception when the process was still running.
Fix this and make JSchProcess.exitValue() throw the right exception,
then wait with a time-out in SshSupport.
The Apache sshd client's SshdExecProcess has always used the correct
IllegalThreadStateException.
Add tests for SshSupport.runCommand().
Change-Id: Id30893174ae8be3b9a16119674049337b0cf4381 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Mon, 3 Aug 2020 14:22:37 +0000 (16:22 +0200)]
sshd: work around a race condition in Apache MINA sshd 2.4.0/2.5.x
When exceptions occur very early in the SSH connection setup, it's
possible that an exception gets lost. A subsequent authentication
attempt may then never be notified of the failure, and then wait
indefinitely or until its timeout expires.
This is caused by race conditions in sshd. The issue has been reported
upstream as SSHD-1050,[1] but will be fixed at the earliest in sshd
2.6.0.
Thomas Wolf [Sat, 25 Jul 2020 08:20:29 +0000 (10:20 +0200)]
sshd: store per-session data on the sshd session object
Don't store session properties on the client but in a dedicated
per-session object that is attached to the sshd session.
Also make sure that each sshd session gets its own instance of
IdentityPasswordProvider that asks for passphrases of encrypted
private keys, and also store it on the session itself.
Bug: 563380
Change-Id: Ia88bf9f91cd22b5fd32b5972d8204d60f2de56bf Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
then I would expect no blobs to be transferred over the wire. Alas,
JGit rewrites filter=blob:none to filter=blob:limit=0, so if the
repository contains an empty file then the empty blob gets
transferred.
Fix it by teaching JGit about filters based on object type to
complement the existing filters based on object size. This prepares
us for other future filters such as object:none.
In particular, this means we do not need to look up the size of the
filtered blobs, which should speed up clones. Noticed by Anna
Pologova and Terry Parker.
Change-Id: Id4b234921a190c108d8be2c87f54dcbfa811602a Signed-off-by: Jonathan Nieder <jrn@google.com>
Jonathan Nieder [Thu, 30 Jul 2020 03:40:50 +0000 (20:40 -0700)]
Add support for tree filters when fetching
Teach the FilterSpec serialization code about tree filters so they can
be communicated over the wire and understood by the server.
While we're here, harden the FilterSpec serialization code to throw
IllegalStateException if we encounter a FilterSpec that cannot be
expressed as a "filter" line. The only public API for creating a
Filterspec is to pass in a "filter" line to be parsed, so these should
not appear in practice.
Change-Id: I9664844059ffbc9c36eb829e2d860f198b9403a0 Signed-off-by: Jonathan Nieder <jrn@google.com>
Thomas Wolf [Tue, 28 Jul 2020 07:44:43 +0000 (09:44 +0200)]
sshd: use PropertyResolver in test
Improve the SshTestGitServer API for accessing the server properties.
Instead of returning the raw property map, return the proper sshd API
abstraction PropertyResolver.
This makes the interface more resilient against upstream changes.
Change-Id: Ie5b685bddc4e59f3eb6c121026d3658d57618ca4 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Fri, 19 Jun 2020 13:29:57 +0000 (15:29 +0200)]
FS_POSIX: avoid prompt to install the XCode tools on OS X
OS X ships with a default /usr/bin/git that is just a wrapper that
at run-time delegates to the selected XCode toolchain, and that
prompts the user to install the XCode command line tools if not
already installed.
This is annoying for people who don't want to do so, since they'll
be prompted on each Eclipse start. Also, since on OS X the $PATH for
applications started via the GUI is not the same as the $PATH as set
via the shell profile, just using /usr/bin/git (which will normally
be found when JGit runs inside Eclipse) may give slightly surprising
results if the user has installed a non-Apple git and changed his
$PATH in the shell such that the non-Apple git is used in the shell.
(For instance by placing /usr/local/bin earlier on the path.) Eclipse
and the shell will use different git executables, and thus different
git system configs.
Therefore, try to find git via bash --login -c 'which git' not only
if we couldn't find it on $PATH but also if we found the default git
/usr/bin/git. If that finds some other git, use that. If the bash
approach also finds /usr/bin/git, double check via xcode-select -p
that an XCode git is present. If not, assume there is no git installed,
and work without any system config.
Bug: 564372
Change-Id: Ie9d010ebd9437a491ba5d92b4ffd1860c203f8ca Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Thomas Wolf [Sat, 27 Jun 2020 13:01:01 +0000 (15:01 +0200)]
Remove dependency on JSch from SSH test framework
Use standard java.security to generate test keys, use sshd to write
public key files, and write PKCS#8 PEM files for our non-encrypted
test private keys. This is a format that both JSch and Apache MINA
sshd can read.
Change-Id: I6ec55cfd7346b672a7fb6139d51abfb06d81a394 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Matthias Sohn [Fri, 17 Jul 2020 15:26:17 +0000 (17:26 +0200)]
Use LinkedBlockingQueue for executor determining filesystem attributes
Using a fixed thread pool with unbounded LinkedBlockingQueue fixes the
RejectedExecutionException thrown if too many threads try to
concurrently determine filesystem attributes.
Comparing that to an alternative implementation using an unbounded
thread pool instead showed similar performance with the reproducer (in
range of 100-1000 threads in reproducer) on my mac:
threads time
fixed threadpool up to 5 threads with LinkedBlockingQueue of unlimited
queue size
100 1103 ms
200 1602 ms
300 2369 ms
500 4002 ms
1000 11071 ms
unbounded cached threadpool
100 1108 ms
200 1591 ms
300 2299 ms
500 4577 ms
1000 11196 ms
Bug: 564202
Change-Id: I773da7414a1dca8e548349442dca9b56643be946 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
David Ostrovsky [Sun, 21 Jun 2020 14:56:57 +0000 (16:56 +0200)]
Bazel: Add workspace status command to stamp final artifact
Include implementation version in jgit library. This version is used
by other products that depend on JGit, and built using Bazel and not
consume officially released artifact from Central or Eclipse own Maven
repository.
Most notably, in Gerrit Code Review JGit agent that was previously
reported as "unknown", is now reported as:
Thomas Wolf [Sat, 11 Jul 2020 13:29:04 +0000 (15:29 +0200)]
DiffFormatter: correctly deal with tracked files in ignored folders
In JGit 5.0, the FileTreeIterator was changed to skip ignored folders
by default. To catch tracked files inside ignored folders, the tree
walk needs to have a DirCacheIterator, and the FileTreeIterator has
to know about that DirCacheIterator via setDirCacheIterator(). (Or
the optimization has to be switched off explicitly via
setWalkIgnoredDirectories(true).)
Skipping ignored directories is an important optimization in some
cases, for instance in node.js/npm projects, where we'd otherwise
traverse the whole huge and deep hierarchy of the typically ignored
node_modules folder.
While all uses of WorkingTreeIterator in JGit had been adapted,
DiffFormatter was forgotten. To make it work correctly (again) also
for such cases, make it set up a WorkingTreeeIterator automatically,
and make sure the WorkingTreeSource can find such files, too. Also
pass the repository to the TreeWalks used inside the DiffFormatter
to pick up the correct attributes, filters, and line-ending settings.
Bug: 565081
Change-Id: Ie88ac81166dc396ba28b83313964c1712b6ca199 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
projects / jgit.git / log