Terry Parker [Sun, 11 Feb 2018 22:37:22 +0000 (14:37 -0800)]
Add a minimum negotiation feature for fetch
Android an Chrome have several repos with >300k refs. We sometimes see
negotiations of >100k rounds. This change provides a "minimal negotiation"
feature on the client side that limits how many "have" lines the client
sends. The client extracts the current SHA-1 values for the refs in its
wants set, and terminates negotiation early when all of those values have
been sent as haves. If a new branch is being fetched then that set will
be empty and the client will terminate after current default minimum
of two rounds.
This feature is gated behind a "fetch.useminimalnegotiation" configuration
flag, which defaults to false.
Change-Id: Ib12b095cac76a59da6e8f72773c4129e3b32ff2b Signed-off-by: Terry Parker <tparker@google.com>
Terry Parker [Sun, 11 Feb 2018 21:29:38 +0000 (13:29 -0800)]
Add negotiation statistics to PackStatistics
Add fetch statistics for the counts of advertised refs, wants and haves.
Also add the duration in milliseconds for the negotiation phase. For
non-bidirectional transports like HTTP, this is the time for the final
round that sends the pack back to the user.
Change-Id: I1af7ffd3cb7b62182340682e2a243691ea24ec2e Signed-off-by: Terry Parker <tparker@google.com>
Fix CleanCommand not to throw FileNotFoundExceptions
When CleanCommand is collecting the files and folders to be deleted
it may happen that the list of directories contains obsolete entries.
E.g. a folder and its parent folder may be in the list. Only the
parent folder would be sufficient.
This was a reason for hitting FileNotFoundExceptions when finally
trying to delete the files and folders. Improve CleanCommand
to ignore files to be deleted which are already gone.
Bug: 514434
Change-Id: I10caa01bfb9cec5967dfdaea50c6e4a713eeeabd Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
After packaging references, the folders containing these references are
not deleted. In a busy repository, this causes operations to slow down
as traversing the references tree becomes longer.
Delete empty reference folders after the loose references have been
packed.
To avoid deleting a folder that was just created by another concurrent
operation, only delete folders that were not modified in the last 30
seconds.
David Ostrovsky [Wed, 13 Dec 2017 22:41:45 +0000 (23:41 +0100)]
Bazel: Support building with Java 9
Bazel@HEAD supports Java 9.
The current code has one single issue with Java 9 compliance: the usage
of javax.xml.bind.DatatypeConverter class for printHexBinary() method.
This class is not available on Java 9. One alternative is to use guava
library. Something similar was done here: [1]. But unlike the case with
checkstyle library, JGit currently doesn't use guava. Instead, we add
java.xml.bind module with --add-modules compiler option.
To build (or test) with Java 9, build custom bazel version and issue:
The Files.exists method has noticeably poor performance in JDK 8 and can
slow an application significantly when used to check files that do not
actually exist. The same goes for Files.notExists, Files.isDirectory and
Files.isRegularFile [1].
David Turner [Thu, 12 Oct 2017 22:43:15 +0000 (18:43 -0400)]
Basic submodule merge handling
This doesn't handle the really hard thing, which is merging spurious
conflicts inside .gitmodules files. That's OK: git.git doesn't
either. Users can resolve the conflict themselves and then commit
the merge.
Previously, jgit would crash when attempting to merge conflicting
submodule changes. Even if there was no conflict, after a merge which
adds submodules, the repository would have been missing empty
directories for newly-added submodules.
This patch fixes the crash, and adds the empty directories where
necessary. It ensures that the index is in a conflicted state when
submodule changes conflict.
Reported-by: Alexey Korobkov
Bug: 494551
Change-Id: I79db6798c2bdcc1159b5b2589b02da198dc906a1 Signed-off-by: David Turner <dturner@twosigma.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Thomas Wolf [Fri, 2 Feb 2018 07:16:13 +0000 (08:16 +0100)]
Fix index blob for merges with CRLF translations
Commit fc7d407 corrected line endings for working tree files resulting
from merges when CRLF translations are to be done. However, that also
resulted in the file content being put as-is into the index, which is
wrong. The index must contain the file content with reverse CRLF
translations applied.
With core.autocrlf=true, the working tree file should have CR-LF, but
the index blob must still contain only LF.
Fix this oversight and apply the inverse translation when updating the
index, similar to what is done in AddCommand.
Bug: 499615
Change-Id: I3a33931318bdb580b2390f3450f91ea8f258a6a4 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
David Pursehouse [Wed, 24 Jan 2018 01:15:56 +0000 (10:15 +0900)]
Fix version of slf4j in manifest files
Consistently require 1.7.0. We ship 1.7.2 with our p2 repository but
there is no need to require 1.7.2 since it should be API compatible with
1.7.0.
Change-Id: I8467bb14316cb24daa79e89275332107d2716190 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
David Pursehouse [Wed, 24 Jan 2018 01:09:32 +0000 (10:09 +0900)]
Update gson version in org.eclipse.jgit.lfs.server manifest
Commit f3bb0e2 upgraded gson to 2.8.2 but did not update the LFS
server manifest file to require 2.8. Also accept 2.8.0 which should be
API compatible with 2.8.2 which is the version we ship with the p2
repository.
Change-Id: I734496c4826739f9bc6a790b9d2c70e549c0e559 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Thomas Wolf [Thu, 1 Feb 2018 20:58:23 +0000 (21:58 +0100)]
Honor CRLF settings when writing merge results
Merges are performed using the raw text as stored in the git
repository. When we write the merge result, we must apply the
correct CRLF settings. Otherwise the line endings in the result
will be wrong.
Bug: 499615
Change-Id: I37a9b987e9404c97645d2720cd1c7c04c076a96b Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Thomas Wolf [Sat, 27 Jan 2018 15:06:15 +0000 (16:06 +0100)]
Minor improvements in git config file inclusions
* Section and key names in git config files are case-insensitive.
* If an include directive is invalid, include the line in the
exception message.
* If inclusion of the included file fails, put the file name into
the exception message so that the user knows in which file the
problem is.
Change-Id: If920943af7ff93f5321b3d315dfec5222091256c Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Markus Duft [Tue, 5 Dec 2017 08:30:48 +0000 (09:30 +0100)]
Progress reporting for checkout
The reason for the change is LFS: when using a lot of LFS files,
checkout can take quite some time on larger repositories. To avoid
"hanging" UI, provide progress reporting.
Also implement (partial) progress reporting for cherry-pick, reset,
revert which are using checkout internally.
The feature is also useful without LFS, so it is independent of it.
Change-Id: I021e764241f3c107eaf2771f6b5785245b146b42 Signed-off-by: Markus Duft <markus.duft@ssi-schaefer.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
David Pursehouse [Mon, 15 Jan 2018 00:45:38 +0000 (09:45 +0900)]
Upgrade commons-compress to 1.15
Also update xz to 1.6 as this is the version commons-compress 1.15
wants. Since xz 1.6 is an optional dependency for commons-compress we
need to add a non-optional dependency to xz for
org.eclipse.jgit.pgm.test since one of the tests explicitly requires xz.
Related change adding commons-compress to Orbit:
https://git.eclipse.org/r/#/c/115366/
CQ: 15356
CQ: 15360
Change-Id: I0d61c71bc541cc30464a0fff93775b079dd3ba88 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Reported-by: Thomas Wolf <thomas.wolf@paranor.ch>
Change-Id: I96869f80dd11ee238911706581b224bca4fb12cd Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Zhen Chen [Wed, 17 Jan 2018 23:14:43 +0000 (15:14 -0800)]
Skip broken symbolic ref in DfsReftableDatabase
We skipped the broken symbolic reference in other implementation like
DfsRefDatabase, RefDirectory. The broken symbolic reference may cause
NPE when caller forget to have a null check against the object id before
calling parse it.
Thomas Wolf [Fri, 12 Jan 2018 11:51:22 +0000 (12:51 +0100)]
Revert handling of ssh IdentityFile to pre-4.9 behavior
Jsch caches keys (aka identities) specified in ~/.ssh/config via
IndentityFile only for the current Jsch Session. This results in
multiple password prompts for successive sessions.
Do the handling of IdentityFile exclusively in JGit, as it was before
4.9. JGit uses different Jsch instances per host and caches the
IdentityFile there, allowing it to be re-used in different sessions
for the same host.
* Add comments to explain this.
* Move the JschBugFixingConfig from OpenSshConfig to
JschConfigSessionFactory to have all these Jsch work-arounds
in one place.
* Make that config hide the IdentityFile config from Jsch to avoid
that Jsch overrides the JGit behavior.
Bug: 529173
Change-Id: Ib36c34a2921ba736adeb64de71323c2b91151613 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Dave Borowitz [Fri, 5 Jan 2018 18:02:47 +0000 (13:02 -0500)]
Ensure DirectoryStream is closed promptly
From the javadoc for Files.list:
"The returned stream encapsulates a DirectoryStream. If timely disposal
of file system resources is required, the try-with-resources construct
should be used to ensure that the stream's close method is invoked
after the stream operations are completed."
This is the only call to Files#newDirectoryStream that is not already in
a try-with-resources.
Matthias Sohn [Wed, 3 Jan 2018 00:09:57 +0000 (01:09 +0100)]
Add com.jcraft.jzlib 1.1.1
com.jcraft.jsch requires com.jcraft.jzlib to provide optional zlib
packet compression support. Add this library so that jgit can handle
packet compression.
CQ: 15292
Bug: 529129
Change-Id: I0297bd0488753547a5f5d47dbf0db508a79dd265 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Tim Hosey [Thu, 4 Jan 2018 01:58:05 +0000 (02:58 +0100)]
Fix file handle leak in FetchCommand#fetchSubmodules
The private fetchSubmodules method in the FetchCommand class creates a
Repository instance for each submodule being fetched, but never calls
closes on it.
This leads to the leaking of file handles.
Bug: 526494
Change-Id: I7070388b8b62063d9d5cd31afae3015a8388044f Signed-off-by: Tim Hosey <timhoseydev@gmail.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
David Turner [Fri, 13 Oct 2017 18:56:04 +0000 (14:56 -0400)]
Add a command to deinitialize submodules
Change-Id: Iaaefc2cbafbf083d6ab158b1c378ec69cc76d282 Signed-off-by: David Turner <dturner@twosigma.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
David Turner [Thu, 19 Oct 2017 20:58:07 +0000 (16:58 -0400)]
Use submodule name instead of path as key in config
When a submodule is moved, the "name" field remains the same, while
the "path" field changes. Git uses the "name" field in .git/config
when a submodule is initialized, so this patch makes JGit do so too.
Change-Id: I48d8e89f706447b860c0162822a8e68170aae42b Signed-off-by: David Turner <dturner@twosigma.com>
Dave Borowitz [Thu, 21 Dec 2017 17:01:04 +0000 (12:01 -0500)]
Merge changes I0f1df93b,Ifd40129b,I1b059e1a
* changes:
ConfigTest: Add some additional comment parsing tests
Config: Drop backslash in invalid escape sequences in subsections
Config: Match C git behavior more closely in escaping values
Dave Borowitz [Thu, 21 Dec 2017 14:46:55 +0000 (09:46 -0500)]
Make PackInserter public
The intent with the setCompressionLevel and checkExisting methods (which
are already public) is for callers to be able to call them, but they
can't do that if the class itself is not public.
Dave Borowitz [Wed, 20 Dec 2017 17:36:03 +0000 (12:36 -0500)]
PackInserter: Ensure objects are written at the end of the pack
When interleaving reads and writes from an unflushed pack, we forgot to
reset the file pointer back to the end of the file before writing more
new objects. This had at least two unfortunate effects:
* The pack data was potentially corrupt, since we could overwrite
previous portions of the file willy-nilly.
* The CountingOutputStream would report more bytes read than the size
of the file, which stored the wrong PackedObjectInfo, which would
cause EOFs during reading.
We already had a test in PackInserterTest which was supposed to catch
bugs like this, by interleaving reads and writes. Unfortunately, it
didn't catch the bug, since as an implementation detail we always read a
full buffer's worth of data from the file when inflating during
readback. If the size of the file was less than the offset of the object
we were reading back plus one buffer (8192 bytes), we would completely
accidentally end up back in the right place in the file.
So, add another test for this case where we read back a small object
positioned before a large object. Before the fix, this test exhibited
exactly the "Unexpected EOF" error reported at crbug.com/gerrit/7668.
Matthias Sohn [Tue, 19 Dec 2017 22:21:57 +0000 (23:21 +0100)]
Fix javadoc leftovers in org.eclipse.jgit api package
Change-Id: I2150889b5ed04e8739e2367fc9023b750b516398 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Matthias Sohn [Tue, 19 Dec 2017 22:18:38 +0000 (23:18 +0100)]
Remove unused imports
Change-Id: I35370c66e54d93d9b0aa3995e300706956ec0923 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
David Pursehouse [Tue, 19 Dec 2017 05:05:40 +0000 (14:05 +0900)]
Replace explicit calls to initCause where possible
Where the exception being thrown has a constructor that takes a
Throwable, use that instead of instantiating the exception and then
explicitly calling initCause.
Change-Id: I06a0df407ba751a7af8c1c4a46f9e2714f13dbe3 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
CorruptObjectException has a constructor that takes Throwable and
calls initCause with it. Use that instead of instantiating the
exception and explicitly calling initCause.
Change-Id: I1f2747d6c4cc5249e93401b9787eb4ceb50cb995 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
projects / jgit.git / log