Matthias Sohn [Sat, 1 Feb 2020 01:09:37 +0000 (02:09 +0100)]
Merge branch 'stable-5.6' into stable-5.7
* stable-5.6:
Fix string format parameter for invalidRefAdvertisementLine
WindowCache: add metric for cached bytes per repository
pgm daemon: fallback to user and system config if no config specified
WindowCache: add option to use strong refs to reference ByteWindows
Replace usage of ArrayIndexOutOfBoundsException in treewalk
Add config constants for WindowCache configuration options
Matthias Sohn [Sat, 1 Feb 2020 00:57:03 +0000 (01:57 +0100)]
Merge branch 'stable-5.5' into stable-5.6
* stable-5.5:
Fix string format parameter for invalidRefAdvertisementLine
WindowCache: add metric for cached bytes per repository
pgm daemon: fallback to user and system config if no config specified
WindowCache: add option to use strong refs to reference ByteWindows
Replace usage of ArrayIndexOutOfBoundsException in treewalk
Add config constants for WindowCache configuration options
Matthias Sohn [Sat, 1 Feb 2020 00:41:52 +0000 (01:41 +0100)]
Merge branch 'stable-5.4' into stable-5.5
* stable-5.4:
Fix string format parameter for invalidRefAdvertisementLine
WindowCache: add metric for cached bytes per repository
pgm daemon: fallback to user and system config if no config specified
WindowCache: add option to use strong refs to reference ByteWindows
Replace usage of ArrayIndexOutOfBoundsException in treewalk
Add config constants for WindowCache configuration options
Matthias Sohn [Sat, 1 Feb 2020 00:28:40 +0000 (01:28 +0100)]
Merge branch 'stable-5.3' into stable-5.4
* stable-5.3:
Fix string format parameter for invalidRefAdvertisementLine
WindowCache: add metric for cached bytes per repository
pgm daemon: fallback to user and system config if no config specified
WindowCache: add option to use strong refs to reference ByteWindows
Replace usage of ArrayIndexOutOfBoundsException in treewalk
Add config constants for WindowCache configuration options
Matthias Sohn [Sat, 1 Feb 2020 00:14:01 +0000 (01:14 +0100)]
Merge branch 'stable-5.2' into stable-5.3
* stable-5.2:
Fix string format parameter for invalidRefAdvertisementLine
WindowCache: add metric for cached bytes per repository
pgm daemon: fallback to user and system config if no config specified
WindowCache: add option to use strong refs to reference ByteWindows
Replace usage of ArrayIndexOutOfBoundsException in treewalk
Add config constants for WindowCache configuration options
David Pursehouse [Fri, 31 Jan 2020 00:12:08 +0000 (09:12 +0900)]
Merge branch 'stable-5.1' into stable-5.2
* stable-5.1:
Fix string format parameter for invalidRefAdvertisementLine
WindowCache: add metric for cached bytes per repository
pgm daemon: fallback to user and system config if no config specified
WindowCache: add option to use strong refs to reference ByteWindows
Change-Id: I741059a1d0d5950ab5bc16ec70352655ee926a24 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
David Pursehouse [Thu, 30 Jan 2020 23:38:24 +0000 (08:38 +0900)]
Fix string format parameter for invalidRefAdvertisementLine
The externalized error message added in f4fc640 ("BasePackConnection:
Check for expected length of ref advertisement", Dec 18, 2019) uses a
malformed string format. Since there is only one formatting argument,
it should be referenced with '{0}' rather than '{1}'.
Change-Id: Ibda864dfb0bb902fe07ae4bba73117b212046e8a Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Matthias Sohn [Fri, 3 Jan 2020 17:16:42 +0000 (18:16 +0100)]
WindowCache: add metric for cached bytes per repository
Since ObjectDatabase and PackFile don't know their repository use the
packfile's grand-grand-parent directory as an identifier for the
repository the packfile resides in.
Remove metric for a repository if the number of cached bytes for the
repository drops to 0 in order to ensure the map of cached bytes per
repository doesn't contain repositories which have no data cached in the
WindowCache.
Change-Id: I969ab8029db0a292e7585cbb36ca0baa797da20b Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Mon, 27 Jan 2020 22:00:57 +0000 (23:00 +0100)]
pgm daemon: fallback to user and system config if no config specified
If a config file is passed via option --config-file then use only the
options defined in that file. This helps to concisely configure the
daemon without side effects from global and system level git configs.
Otherwise fallback to user and system level configs.
Change-Id: I242de248f257579874ad0bfe4882a22502353b1f Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Fri, 13 Dec 2019 00:18:12 +0000 (01:18 +0100)]
WindowCache: add option to use strong refs to reference ByteWindows
Java GC evicts all SoftReferences when the used heap size comes close to
the maximum heap size. This means peaks in heap memory consumption can
flush the complete WindowCache which was observed to have negative
impact on performance of upload-pack in Gerrit.
Hence add a boolean option core.packedGitUseStrongRefs to allow using
strong references to reference packfile pages cached in the WindowCache.
If this option is set to true Java gc can no longer flush the
WindowCache to free memory if the used heap comes close to the maximum
heap size. On the other hand this provides more predictable performance.
Bug: 553573
Change-Id: I9de406293087ab0fa61130c8e0829775762ece8d Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
David Pursehouse [Wed, 15 Jan 2020 11:03:28 +0000 (20:03 +0900)]
Merge branch 'stable-5.1' into stable-5.2
* stable-5.1:
Replace usage of ArrayIndexOutOfBoundsException in treewalk
Add config constants for WindowCache configuration options
Change-Id: I17da3c5183eca536aa2be3972bc5df45c9d75f1b Signed-off-by: David Pursehouse <david.pursehouse@gmail.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Patrick Hiesel [Thu, 9 Jan 2020 09:23:12 +0000 (10:23 +0100)]
Replace usage of ArrayIndexOutOfBoundsException in treewalk
Using exceptions during normal operations - for example with the
desire of expanding an array in the failure case - can have a
severe performance impact. When exceptions are instantiated,
a stack trace is collected. Generating stack trace can be expensive.
Compared to that, checking an array for length - even if done many
times - is cheap since this is a check that can run in just a
handful of CPU cycles.
Change-Id: Ifaf10623f6a876c9faecfa44654c9296315adfcb Signed-off-by: Patrick Hiesel <hiesel@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Thu, 9 Jan 2020 17:06:05 +0000 (18:06 +0100)]
Move ConnectivityChecker to fix API leak
Protected field ReceivePack.connectivityChecker is of type
ConnectivityChecker which was located in internal package
org.eclipse.jgit.transport and was thus leaking an internal type.
In order to fix this move ConnectivityChecker to package
org.eclipse.jgit.transport.
Change-Id: Idc0af75502faf39f13a56157bbb6964d24720eb8 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Thu, 9 Jan 2020 15:50:13 +0000 (16:50 +0100)]
Merge branch 'stable-5.6'
* stable-5.6:
Fix API problem filters
Fix unclosed resource warning in SmartOutputStream
JschConfigSessionFactory: fix boxing warning
SshSupport#runSshCommand: don't throw exception in finally block
Don't override already managed maven-compiler-plugin version
Remove unused import from CreateFileSnapshotBenchmark
Remove duplicate ignore_optional_problems entry in .classpath
Update maven-site-plugin used by benchmark module to 3.8.2
Add dependency to enable site generation for benchmark module
Ignore warnings for generated source code in
org.eclipse.jgit.benchmark
Fix MBean registration
Enhance WindowCache statistics
Change-Id: I4a77c602bfffed60535d0b8fc251a552b128068d Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Thu, 9 Jan 2020 12:14:38 +0000 (13:14 +0100)]
Merge branch 'stable-5.5' into stable-5.6
* stable-5.5:
Fix API problem filters
Fix unclosed resource warning in SmartOutputStream
JschConfigSessionFactory: fix boxing warning
SshSupport#runSshCommand: don't throw exception in finally block
Don't override already managed maven-compiler-plugin version
Remove unused import from CreateFileSnapshotBenchmark
Remove duplicate ignore_optional_problems entry in .classpath
Update maven-site-plugin used by benchmark module to 3.8.2
Add dependency to enable site generation for benchmark module
Ignore warnings for generated source code in
org.eclipse.jgit.benchmark
Fix MBean registration
Enhance WindowCache statistics
Change-Id: I11f9a387ac3dc7d22a4f2e70bb8d89169b4e9afe Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Thu, 9 Jan 2020 11:32:03 +0000 (12:32 +0100)]
Merge branch 'stable-5.4' into stable-5.5
* stable-5.4:
Fix API problem filters
Fix unclosed resource warning in SmartOutputStream
JschConfigSessionFactory: fix boxing warning
SshSupport#runSshCommand: don't throw exception in finally block
Don't override already managed maven-compiler-plugin version
Remove unused import from CreateFileSnapshotBenchmark
Remove duplicate ignore_optional_problems entry in .classpath
Update maven-site-plugin used by benchmark module to 3.8.2
Add dependency to enable site generation for benchmark module
Ignore warnings for generated source code in
org.eclipse.jgit.benchmark
Fix MBean registration
Enhance WindowCache statistics
Change-Id: I78902d5feecb2c09134b64ec2f3b48b2c3bab37d Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Thu, 9 Jan 2020 10:38:28 +0000 (11:38 +0100)]
Merge branch 'stable-5.3' into stable-5.4
* stable-5.3:
Fix API problem filters
Fix unclosed resource warning in SmartOutputStream
JschConfigSessionFactory: fix boxing warning
SshSupport#runSshCommand: don't throw exception in finally block
Don't override already managed maven-compiler-plugin version
Remove unused import from CreateFileSnapshotBenchmark
Remove duplicate ignore_optional_problems entry in .classpath
Update maven-site-plugin used by benchmark module to 3.8.2
Add dependency to enable site generation for benchmark module
Ignore warnings for generated source code in
org.eclipse.jgit.benchmark
Fix MBean registration
Enhance WindowCache statistics
Change-Id: I1b560b36d169cfa02cc5450ad0fa0bd85f9f42d8 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Thu, 9 Jan 2020 01:19:03 +0000 (02:19 +0100)]
Merge branch 'stable-5.2' into stable-5.3
* stable-5.2:
Fix API problem filters
Fix unclosed resource warning in SmartOutputStream
JschConfigSessionFactory: fix boxing warning
SshSupport#runSshCommand: don't throw exception in finally block
Don't override already managed maven-compiler-plugin version
Remove unused import from CreateFileSnapshotBenchmark
Remove duplicate ignore_optional_problems entry in .classpath
Update maven-site-plugin used by benchmark module to 3.8.2
Add dependency to enable site generation for benchmark module
Ignore warnings for generated source code in
org.eclipse.jgit.benchmark
Fix MBean registration
Enhance WindowCache statistics
Change-Id: I67a07d92718188bdf7f8a13b83e9f538ecf4b22f Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Sat, 4 Jan 2020 14:44:19 +0000 (15:44 +0100)]
SshSupport#runSshCommand: don't throw exception in finally block
The CommandFailedException which was thrown in finally block is silently
discarded [1]. Refactor this method to throw the exception after the
finally block.
This fixes the warning "Null comparison always yields false: The
variable failure can only be null at this location".
Thomas Wolf [Tue, 31 Dec 2019 22:08:10 +0000 (23:08 +0100)]
TLS support on IBM JDKs
SSLContext.getInstance("TLS") by default behaves differently on IBM
JDK than on Oracle or OpenJDK.[1] On IBM JDK one gets sockets that
have only TLSv1 enabled, which makes HTTPS connections fail since most
servers refuse this old protocol version. On Oracle JDK/OpenJDK, one
gets sockets with all available protocol versions enabled.
Explicitly enable all available TLS protocol versions to make
HTTPS connections work also on IBM JDK.
Bug: 558709
Change-Id: I5ffc57a78e67a6239b9dad54840a49a8ed28930a Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Mon, 16 Dec 2019 14:22:08 +0000 (15:22 +0100)]
Ignore warnings for generated source code in org.eclipse.jgit.benchmark
The source code in the folder .apt_generated is generated by the JMH
code generator, so there's no point in raising any warnings as this
could only be fixed in the upstream code generator.
Change-Id: I882888e7bf924f9ae74182598fcb91671a5c9818 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Mon, 16 Dec 2019 10:20:12 +0000 (11:20 +0100)]
Add resource preferences for top level jgit project
Define resource preferences for the root directory of the jgit
repository imported as an Eclipse project. This allows to work on files
in the root folder inside Eclipse workspace.
Change-Id: I1885841fd612a646b0b9a31f001ced5ca112e90d Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Sat, 14 Dec 2019 21:45:09 +0000 (22:45 +0100)]
Set config "extensions" option when converting ref storage format
When converting to reftable format the option extensions.refStorage must
be set to "reftable" [1]. When converting back to refdir format this
config option needs to be removed.
Introduce constants for refStorage config options, also for the
"reftree" format.
Carsten Hammer [Tue, 21 May 2019 15:04:18 +0000 (17:04 +0200)]
Replace chain of if statements with switch
and switch over strings where possible. Sometimes if statements are
chained and form a series of comparisons against constants. Using switch
statements improves readability.
Tim Neumann [Thu, 28 Nov 2019 14:56:50 +0000 (15:56 +0100)]
Add ability to redirect stderr from git hooks
This will change the behavior in the CLI to resemble that of C-Git more
closely by printing the stderr of the hooks to the CLI stderr
independently of the exit code of the hook.
This is also useful for the corresponding EGIT-Change, which will add
the ability to show the hook output in eclipse.
With this also the stderr can be shown even if the exit code is 0.
Bug: 553471
Change-Id: Ie7bc503fe39e270e9b93dd1108b5879f02a12b4c Signed-off-by: Tim Neumann <Tim.Neumann@advantest.com>
Matthias Sohn [Sat, 30 Nov 2019 02:38:13 +0000 (03:38 +0100)]
Enhance WindowCache statistics
Add the following statistics
- cache hit count and hit ratio
- cache miss count and miss ratio
- count of successful and failed loads
- rate of failed loads
- load, eviction and request count
- average and total load time
Use LongAdder instead of AtomicLong to implement counters in order to
improve scalability.
Optionally expose these metrics via JMX, they are registered with the
platform MBean server if the config option jmx.WindowCacheStats = true
in the user or system level git config.
Bug: 553573
Change-Id: Ia2d5246ef69b9c2bd594a23934424bc5800774aa Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
John Tipper [Mon, 2 Dec 2019 23:00:19 +0000 (23:00 +0000)]
CLI: Add support for excluding paths from Git log command
Git log supports the exclude pathspec, which allows for excluding paths
from the log command. JGit only supports adding paths to the log
command. See the following StackOverflow question for details:
https://stackoverflow.com/questions/59143934/java-jgit-how-to-get-git-
commits-not-affecting-certain-directories
This commit adds an excludePath() method to the log command. It does not
yet support regex or glob wildcards.
Change-Id: I8cd59950b87850b55a15c7e2ea5470145c9aae28 Signed-off-by: John Tipper <john_tipper@hotmail.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Demetr Starshov [Tue, 8 Oct 2019 23:53:22 +0000 (16:53 -0700)]
ReceivePack: Moves connectivity check to separate class
Move all connectivity check to separate classes. Set default to be
FullConnectivityChecker i.e. checker which will check connectivity
from all advertised refs. Add ability to set other connectivity
checker which can use different approach.
Matthias Sohn [Thu, 5 Dec 2019 00:56:31 +0000 (01:56 +0100)]
Suppress API error for RefDatabase.hasFastTipsWithSha1()
This method was added in stable-5.6 after 5.6 RC1 was tagged. It's shown
as an API error when using 5.6.0 RC1 as the API baseline for master.
This suppression can be removed when 5.6 is released which will serve as
the API baseline for 5.7.
Change-Id: Ic6359fe59ed2d2a9431583b0854ec86bbac76880 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Han-Wen Nienhuys [Mon, 18 Nov 2019 00:29:16 +0000 (16:29 -0800)]
Introduce RefDatabase#hasFastTipsWithSha1
The reftable format supports fast inverse (SHA1 => ref) queries.
If the ref database does not support fast inverse queries, it may be
advantageous to build a complete SHA1 to ref map in advance for
multiple uses. To let applications decide, this function indicates
whether the inverse map is available.
Masaya Suzuki [Fri, 8 Nov 2019 18:12:20 +0000 (10:12 -0800)]
transport: Let users specify an error handler for unpacking a pack file
Currently, there's no way for a user to customize the error handler for
the exception happened during unpacking an incoming pack file. Create an
error handler class and let them specify one.
Masaya Suzuki [Fri, 8 Nov 2019 02:52:17 +0000 (18:52 -0800)]
transport: Consolidate status reporting code
BaseReceivePack#sendStatusReport anyway needs to know
CAPABILITY_REPORT_STATUS. By moving this flag to BaseReceivePack,
simplify the status reporting code.
Masaya Suzuki [Fri, 8 Nov 2019 02:22:17 +0000 (18:22 -0800)]
transport: Add ReceiveCommandErrorHandler
This gives a chance to handle an exception for a user. For example, when
an IOException is thrown while executing
`walk.parseAny(cmd.getNewId())`, it's always handled as
REJECTED_MISSING_OBJECT. However, IOException can mean a Git storage IO
error. By introducing an error handler class, a user can add a custom
error handler for these cases.
Masaya Suzuki [Fri, 8 Nov 2019 01:40:25 +0000 (17:40 -0800)]
transport: Move exception handler to the caller
Move exception handling code to the caller side. This is a preparation
for adding a custom error handler that is similar to UploadPack.
Note that `catch (Throwable t)` will not affect the exception spec since
Java will do an analysis of rethrown exceptions. See
https://docs.oracle.com/javase/8/docs/technotes/guides/language/catch-multiple.html.
Masaya Suzuki [Fri, 8 Nov 2019 22:30:29 +0000 (14:30 -0800)]
UploadPackServlet: Use uploadWithExceptionPropagation
As UploadPackErrorHandler's Javadoc says, UploadPackServlet should have
called uploadWithExceptionPropagation and let UploadPackErrorHandler to
handle the exception. Fix UploadPackServlet.
Masaya Suzuki [Fri, 8 Nov 2019 22:40:19 +0000 (14:40 -0800)]
GitSmartHttpTools: Do not use sideband when sending an error
Unlike ReceivePack, the V0/V1 UploadPack response does not support
sideband except for the packfile parts. By sending an error in a
sideband packet, the JGit client says "Expected ACK/NACK, got: ...".
Use an error packet always. The recent Git clients will understand it
better than out-of-context sideband packets.
Ivan Frade [Tue, 19 Nov 2019 06:45:19 +0000 (22:45 -0800)]
ReceivePack: Open visibility for some methods
This partially reverts Ic6bb5e66. It made private some methods in
ReceivePack that were protected before. At least two of those methods
(#init and #receivePackAndCheckConnectivity) are used by out-of-tree
subclasses for tracing and testing.
Make ReceivePack#init() and ReceivePack#receivePackAndCheckConnectivity
protected again to allow tracing and testing.
Change-Id: I7ff22c091fbfc2d94009c449d58e7c5ac9f4f759 Signed-off-by: Ivan Frade <ifrade@google.com>
Thomas Wolf [Tue, 6 Aug 2019 16:31:35 +0000 (18:31 +0200)]
Make blame work correctly on merge conflicts
When a conflicting file was blamed, JGit would not identify lines
coming from the merge parents. The main cause for this was that
Blame and BlameCommand simply added the first DirCacheEntry found
for a file to its queue of candidates (blobs or commits) to consider.
In case of a conflict this typically is the merge base commit, and
comparing a auto-merged contents against that base would yield
incorrect results.
Such cases have to be handled specially. The candidate to be
considered by the blame must use the working tree contents, but
at the same time behave like a merge commit/candidate with HEAD
and the MERGE_HEADs as parents. Canonical git does something very
similar, see [1].
Implement that and add tests.
I first did this for the JGit pgm Blame command. When I then tried
to do the same in BlameCommand, I noticed that the latter also
included some fancy but incomplete CR-LF handling. In order to
be able to use the new BlameGenerator.prepareHead() also in
BlameCommand this CR-LF handling was also moved into BlameGenerator
and corrected in doing so.
(Just considering the git config settings was not good enough,
CR-LF behavior can also be influenced by .gitattributes, and even
by whether the file in the index has CR-LF. To correctly determine
CR-LF handling for check-in one needs to do a TreeWalk with at
least a FileTreeIterator and a DirCacheIterator.)