Shawn Pearce [Thu, 14 Dec 2017 01:35:38 +0000 (17:35 -0800)]
Make Config.readIncludedConfig a noop by default
The Config class must be safe to run against untrusted input files.
Reading arbitrary local system paths using include.path is risky for
servers, including Gerrit Code Review. Return null by default to
incide the include should be ignored.
Only FileBasedConfig which originated from local disk should be trying
to read local system paths. FileBasedConfig already overrides this
method with its own implementation.
Shawn Pearce [Thu, 14 Dec 2017 01:10:51 +0000 (17:10 -0800)]
DfsBlockCache.hasBlock0: quickly check for file in cache
This can be useful for sophisticated pre-read algorithms to quickly
determine if a file is likely already in cache, especially small
reftables which may be smaller than a typical DFS block size.
Dmitry Pavlenko [Thu, 7 Dec 2017 18:36:35 +0000 (19:36 +0100)]
Fix IllegalThreadStateException if stderr closed without exiting
If some process executed by FS#readPipe lived for a while after
closing stderr, FS#GobblerThread#run failed with an
IllegalThreadStateException exception when accessing p.exitValue()
for the process which is still alive.
Add Process#waitFor calls to wait for the process completion.
CloneCommand: Refactor listing of files in deleteChildren
File.listFiles() returns null if the given File does not represent a
directory, so we can just test for null instead of making a separate
call to FS.DETECTED.isDirectory()
This also avoids a false-positive error from SpotBugs which claims
that there is a potential null-pointer exception on dereferencing the
result of Files.listFiles().
Change-Id: I18e09e391011db997470f5a09d8e38bb604c0213 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* stable-4.9:
LfsServerText: Fix formatting of invalidPathInfo message
InvalidLongObjectIdException: Fix formatting of message
FileLfsServlet: Do not pretty print json response
Change-Id: Ib59431865a45659fe9d91e2556b9d1ce49e3808a Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Pretty printing the response is useful for human readers, but most
(if not all) of the time, the response will be read by programs.
Remove it to avoid the additional overhead of the formatting and
extra bytes in the response. Adjust the test accordingly.
Note that LfsProtocolServlet already doesn't use pretty printing,
so this change makes FileLfsServlet's behavior consistent. In fact,
both classes now have duplicate Gson handling; this will be cleaned
up in a separate change.
Change-Id: I113a23403f9222f16e2c0ddf39461398b721d064 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
DfsBlockCache#creditSpace: release clockLock in finally block
Enclose the call to getStat in a `try`, and release the previously
acquired lock in the `finally`. This prevents that the lock is left
unreleased in the case of an exception being raised in getStat.
Change-Id: I17b4cd134dae887e23a1165253be0ac2d4fd452c Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Pretty printing the response is useful for human readers, but most
(if not all) of the time, the response will be read by programs.
Remove it to avoid the additional overhead of the formatting and
extra bytes in the response. Adjust the test accordingly.
Note that LfsProtocolServlet already doesn't use pretty printing,
so this change makes FileLfsServlet's behavior consistent. In fact,
both classes now have duplicate Gson handling; this will be cleaned
up in a separate change.
Change-Id: I113a23403f9222f16e2c0ddf39461398b721d064 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
This class is an inner class, but does not use its embedded reference
to the object which created it. This reference makes the instances
of the class larger, and may keep the reference to the creator object
alive longer than necessary. If possible, the class should be made
static.
Change-Id: I245e44678166176de0cfb275e22ddd159f88e0bd Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Marc Strapetz [Sat, 18 Nov 2017 16:50:30 +0000 (17:50 +0100)]
FileBasedConfig: support for relative includes
Relative include.path are now resolved against the config's parent
directory. include.path starting with ~/ are resolved against the
user's home directory
Change-Id: I91911ef404126618b1ddd3589294824a0ad919e6 Signed-off-by: Marc Strapetz <marc.strapetz@syntevo.com>
Marc Strapetz [Mon, 20 Nov 2017 17:21:22 +0000 (18:21 +0100)]
ConfigTest: fix on Windows
Change-Id: I37a2ef611aef97faf1b891a9660c1745435a915d Signed-off-by: Marc Strapetz <marc.strapetz@syntevo.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Sun, 3 Dec 2017 12:54:28 +0000 (13:54 +0100)]
Silence API warnings for reintroduced ResolveMerger#processEntry
This was silenced before but suppression was unintentionally lost in
merge commit 6858339c1e2878d5c5dc6cc9b422f9802be950ae.
This method was removed in 4.9.0 and reintroduced in 4.9.1 to avoid
breaking EMF compare versions which were built against older versions.
Matthias Sohn [Thu, 30 Nov 2017 12:37:30 +0000 (13:37 +0100)]
Merge branch 'stable-4.9'
* stable-4.9:
GC: Delete stale temporary packs and indexes
Change-Id: I49b37845ee8a465404b801a2d8de0205a2e7ba30 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Jonathan Nieder [Tue, 28 Nov 2017 16:16:56 +0000 (08:16 -0800)]
RepoCommand: Skip RemoteReader when encountering a full SHA-1
There is no point in calling back to the RemoteReader to resolve a
40-digit hex SHA-1 to itself. We already skip that call when not
ignoring remote failures; skip it when ignoring remote failures, too.
This should simplify RemoteReader implementations.
When a GC operation is interrupted, temporary packs and indexes can be
left on the pack folder. In big, busy repositories this can lead to
significant amounts of wasted disk space if this interruption is done
with a certain frequency.
Remove stale temporary packs and indexes at the end of the GC process so
they do not accumulate. To avoid interfering with a possible concurrent
JGit GC process in the same repository, only delete temporary files that
are older than one day.
Stephen Lawson [Wed, 22 Nov 2017 12:41:05 +0000 (12:41 +0000)]
Performance improvement on writing a large index
The index header consists of a 4-byte version number. The current
supported version numbers are 2 and 3. The code checks if any entries
are extended. If it finds any entries that are extended it picks version
'3', otherwise it chooses version '2'.
DirCache.java
-Changed the 'extended' check to exit early when any entry is considered
'extended' in the index.
(Of course, I maybe missing a bitwise optimization that is made in
the Java bytecode.)
Change-Id: If70db9454befe683319b974ebd3774060be9445d Signed-off-by: Stephen Lawson <slawson@ptc.com>
Matthias Sohn [Fri, 24 Nov 2017 00:18:13 +0000 (01:18 +0100)]
Merge branch 'stable-4.9'
* stable-4.9:
Ignore warning for minor version change without API change
Silence boxing warning
Prepare 4.5.5-SNAPSHOT builds
JGit v4.5.4.201711221230-r
Fix LockFile semantics when running on NFS
Honor trustFolderStats also when reading packed-refs
Prepare 4.5.4-SNAPSHOT builds
JGit v4.5.3.201708160445-r
When running on NFS there was a chance that JGits LockFile
semantic is broken because File#createNewFile() may allow
multiple clients to create the same file in parallel. This
change provides a fix which is only used when the new config
option core.supportsAtomicCreateNewFile is set to false. The
default for this option is true. This option can only be set in the
global or the system config file. The repository config file is not
taken into account in this case.
If the config option core.supportsAtomicCreateNewFile is true
then File#createNewFile() is trusted and the behaviour doesn't
change.
But if core.supportsAtomicCreateNewFile is set to false then after
successful creation of the lock file a hardlink to that lock file is
created and the attribute nlink of the lock file is checked to be 2. If
multiple clients manage to create the same lock file nlink would be
greater than 2 showing the error.
This expensive workaround is described in
https://www.time-travellers.org/shane/papers/NFS_considered_harmful.html
section III.d) "Exclusive File Creation"
Honor trustFolderStats also when reading packed-refs
Then list of packed refs was cached in RefDirectory based on mtime of
the packed-refs file. This may fail on NFS when attributes are cached.
A cached mtime of the packed-refs file could cause JGit to trust the
cached content of this file and to overlook that the file is modified.
Honor the config option trustFolderStats and always read the packed-refs
content if the option is false. By default this option is set to true
and this fix is not active.
Thomas Wolf [Sat, 11 Nov 2017 10:41:10 +0000 (11:41 +0100)]
Yet another work-around for a Jsch bug: timeouts
Jsch 0.1.54 passes on the values from ~/.ssh/config for
"ServerAliveInterval" and "ConnectTimeout" as read from
the config file to java.net.Socket.setSoTimeout(). That
method expects milliseconds, but the values in the config
file are seconds!
The missing conversion in Jsch means that the timeout is
set way too low, and if the server doesn't respond within
that very short time frame, Jsch kills the connection and
then throws an exception with a message such as "session is
down" or "timeout in waiting for rekeying process".
As a work-around, do the conversion to milliseconds in the
Jsch-facing Config interface of OpenSshConfig. That way Jsch
already gets these values as milliseconds.
Bug: 526867
Change-Id: Ibc9b93f7722fffe10f3e770dfe7fdabfb3b97e74 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
David Pursehouse [Mon, 20 Nov 2017 06:38:43 +0000 (15:38 +0900)]
Merge branch 'stable-4.9'
* stable-4.9:
Fix NPE in TransportGitSsh.ExtSession.exec()
Add missing help text for rev-parse's --verify option
Remove final modifier on args4j argument field in RevParse
Change-Id: I5ac9e2f185f2210ee76970501710b99b12e93e75 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
So far, in order to get the pack directory it was necessary to resolve
it from the object directory. This resolution is already done when
creating the object directory, so simplify the call by just adding a
getter to the pack directory.
Matthias Sohn [Mon, 13 Nov 2017 21:41:57 +0000 (22:41 +0100)]
Silence invalid @since 3.5 API warning on ResolveMerge#processEntry
This method was removed in 4.9 and reintroduced in
I48ba4308dee73925fa32d6c2fd6b5fd89632c571 as deprecated in 4.9.1 in
order to help EMF Compare to avoid breakage.
Change-Id: Ia638517178313da42ae13ebcf88ad535d9a02723 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* changes:
BitmapWalker: do not revisit objects in bitmap
Use bitmaps for non-commit reachability checks
Make PackWriterBitmapWalker public
UploadPackTest: construct commits in test method
Jonathan Tan [Thu, 2 Nov 2017 01:20:54 +0000 (18:20 -0700)]
BitmapWalker: do not revisit objects in bitmap
Currently, BitmapWalker walks through every object returned by the
internal ObjectWalk, regardless of whether that object has already
been marked in the bitmap. Set an object filter to ensure that only
bitmap-unmarked objects are walked through.
Change-Id: I22a8874b1e571df3c33643b365036d95f52fe7c7 Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Jonathan Tan [Thu, 2 Nov 2017 00:36:18 +0000 (17:36 -0700)]
Use bitmaps for non-commit reachability checks
Currently, unless RequestPolicy#ANY is used, UploadPack rejects all
non-commit "want" lines unless they were advertized. This is fine,
except when "uploadpack.allowreachablesha1inwant" is true
(corresponding to RequestPolicy#REACHABLE_COMMIT), in which case one
would expect that "want"-ing anything reachable would work.
(There is no restriction that "want" lines must only contain commits -
it is allowed for refs to directly point to trees and blobs, and
requesting for them using "want" lines works.)
This commit has been written to avoid performance regressions as much
as possible. In the usual (and currently working) case where the only
unadvertized things requested are commits, we do a standard RevWalk in
order to avoid incurring the cost of loading bitmaps. However, if
unadvertized non-commits are requested, bitmaps are used instead, and
if there are no bitmaps, a WantNotValidException is thrown (as is
currently done).
Change-Id: I68ed4abd0e477ff415c696c7544ccaa234df7f99 Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Jonathan Tan [Wed, 25 Oct 2017 21:09:17 +0000 (14:09 -0700)]
Make PackWriterBitmapWalker public
Make PackWriterBitmapWriter class public and move it to a more central
location, in preparation for its use by another class (in a subsequent
commit).
One of its inner static classes, AddUnseenToBitmapFilter, previously
package-private, is also used directly in its former package. Therefore,
AddUnseenToBitmapFilter and its sibling class have been moved to an
internal package instead.
Change-Id: I740bc4bfc4e4e3c857d1ee7d25fe45e90cd22a75 Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Thomas Wolf [Tue, 7 Nov 2017 07:19:56 +0000 (08:19 +0100)]
Work around a Jsch bug: ensure the user name is set from URI
JSch unconditionally overrides the user name given in the connection
URI by the one found in ~/.ssh/config (if that does specify one for
the used host). If the SSH config file has a different user name,
we'll end up using the wrong name, which typically results in an
authentication failure or in Eclipse/EGit asking for a password for
the wrong user.
Unfortunately there is no way to prevent or circumvent this Jsch
behavior up front; it occurs already in the Session constructor at
com.jcraft.jsch.Session() and the Session.applyConfig() method. And
while there is a Session.setUserName() that would enable us to correct
this, that latter method has package visibility only.
So resort to reflection to invoke that setUserName() method to ensure
that Jsch uses the user name from the URI, if there is one.
Bug: 526778
Change-Id: Ia327099b5210a037380b2750a7fd76ff25c41a5a Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Shawn Pearce [Thu, 9 Nov 2017 17:12:59 +0000 (09:12 -0800)]
Reject pack if delta exceeds array size limit
JGit's delta handling code requires the target to be a single byte
array. Any attempt to inflate a delta larger than fits in the 2GiB
limit will fail with some form of array index exceptions. Check for
this overflow early and abort pack parsing.
Suppress "Unlikely argument type for equals()" warnings in tests
This new warning was introduced in Eclipse 4.7 Oxygen [1].
The only instances of the warning are in test code that is asserting
that some class does not compare equal to Strings. As in the Gerrit
project [2] these asserts are arguably overkill, but arguably also
a reasonable test of an equals implementation. Ignore the warning in
these cases.
Note that if the project is opened in an earlier version of Eclipse,
a warning "Unsupported @SuppressWarnings" will be emitted.
projects / jgit.git / log