]> source.dussan.org Git - jgit.git/log
jgit.git
6 years agoMerge branch 'stable-4.6' into stable-4.7 98/122398/1
Matthias Sohn [Thu, 10 May 2018 09:59:56 +0000 (11:59 +0200)]
Merge branch 'stable-4.6' into stable-4.7

* stable-4.6:
  Retry stale file handles on .git/config file

Change-Id: If5a21d38224528edfc551b3216daca6a2582e3ac

6 years agoMerge branch 'stable-4.5' into stable-4.6 95/122395/1
Matthias Sohn [Thu, 10 May 2018 09:39:52 +0000 (11:39 +0200)]
Merge branch 'stable-4.5' into stable-4.6

* stable-4.5:
  Retry stale file handles on .git/config file

Change-Id: Ib6e6ec0846c3ef261ec1016bfa6d26d2eadc3f26

6 years agoRetry stale file handles on .git/config file 73/120973/4
Nasser Grainawi [Fri, 6 Apr 2018 20:05:37 +0000 (14:05 -0600)]
Retry stale file handles on .git/config file

On a local non-NFS filesystem the .git/config file will be orphaned if
it is replaced by a new process while the current process is reading the
old file. The current process successfully continues to read the
orphaned file until it closes the file handle.

Since NFS servers do not keep track of open files, instead of orphaning
the old .git/config file, such a replacement on an NFS filesystem will
instead cause the old file to be garbage collected (deleted).  A stale
file handle exception will be raised on NFS clients if the file is
garbage collected (deleted) on the server while it is being read.  Since
we no longer have access to the old file in these cases, the previous
code would just fail. However, in these cases, reopening the file and
rereading it will succeed (since it will open the new replacement file).
Since retrying the read is a viable strategy to deal with stale file
handles on the .git/config file, implement such a strategy.

Since it is possible that the .git/config file could be replaced again
while rereading it, loop on stale file handle exceptions, up to 5 extra
times, trying to read the .git/config file again, until we either read
the new file, or find that the file no longer exists. The limit of 5 is
arbitrary, and provides a safe upper bounds to prevent infinite loops
consuming resources in a potential unforeseen persistent error
condition.

Change-Id: I6901157b9dfdbd3013360ebe3eb40af147a8c626
Signed-off-by: Nasser Grainawi <nasser@codeaurora.org>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years agoMerge branch 'stable-4.6' into stable-4.7 26/112126/1
Matthias Sohn [Wed, 22 Nov 2017 22:06:50 +0000 (23:06 +0100)]
Merge branch 'stable-4.6' into stable-4.7

* stable-4.6:
  Silence boxing warning
  Prepare 4.5.5-SNAPSHOT builds
  JGit v4.5.4.201711221230-r
  Fix LockFile semantics when running on NFS
  Honor trustFolderStats also when reading packed-refs
  Prepare 4.5.4-SNAPSHOT builds
  JGit v4.5.3.201708160445-r

Change-Id: I8f6bc09540727c6273d22775a9f9ca382a729c9b
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years agoSilence boxing warning 17/112117/1
Matthias Sohn [Wed, 22 Nov 2017 20:34:37 +0000 (21:34 +0100)]
Silence boxing warning

Change-Id: I36c40eb91ce0c51f89b47911fa14beffcbc0a7cd
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years agoMerge branch 'stable-4.5' into stable-4.6 16/112116/1
Matthias Sohn [Wed, 22 Nov 2017 20:34:08 +0000 (21:34 +0100)]
Merge branch 'stable-4.5' into stable-4.6

* stable-4.5:
  Prepare 4.5.5-SNAPSHOT builds
  JGit v4.5.4.201711221230-r
  Fix LockFile semantics when running on NFS
  Honor trustFolderStats also when reading packed-refs
  Prepare 4.5.4-SNAPSHOT builds
  JGit v4.5.3.201708160445-r

Change-Id: Ie9c8e0d9172c8d53f075c284bf2a9677980d8dfb
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years agoPrepare 4.5.5-SNAPSHOT builds 06/112106/1
Matthias Sohn [Wed, 22 Nov 2017 18:27:22 +0000 (19:27 +0100)]
Prepare 4.5.5-SNAPSHOT builds

Change-Id: I71f946f2875716670a2d74c21a8ab38a1f53a25c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years agoJGit v4.5.4.201711221230-r 96/112096/1 v4.5.4.201711221230-r
Matthias Sohn [Wed, 22 Nov 2017 17:31:24 +0000 (18:31 +0100)]
JGit v4.5.4.201711221230-r

Change-Id: Ia1079da239c5b3fde1ba8d2acc4e465a46297b4d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years agoFix LockFile semantics when running on NFS 16/112016/9
Christian Halstrick [Tue, 14 Nov 2017 16:08:56 +0000 (17:08 +0100)]
Fix LockFile semantics when running on NFS

When running on NFS there was a chance that JGits LockFile
semantic is broken because File#createNewFile() may allow
multiple clients to create the same file in parallel. This
change provides a fix which is only used when the new config
option core.supportsAtomicCreateNewFile is set to false. The
default for this option is true. This option can only be set in the
global or the system config file. The repository config file is not
taken into account in this case.

If the config option core.supportsAtomicCreateNewFile is true
then File#createNewFile() is trusted and the behaviour doesn't
change.

But if core.supportsAtomicCreateNewFile is set to false then after
successful creation of the lock file a hardlink to that lock file is
created and the attribute nlink of the lock file is checked to be 2. If
multiple clients manage to create the same lock file nlink would be
greater than 2 showing the error.

This expensive workaround is described in
 https://www.time-travellers.org/shane/papers/NFS_considered_harmful.html
section III.d) "Exclusive File Creation"

Change-Id: I3d2cc48d8eb280d5f7039eb94da37804f903be6a

7 years agoHonor trustFolderStats also when reading packed-refs 15/112015/1
Christian Halstrick [Tue, 14 Nov 2017 16:20:02 +0000 (17:20 +0100)]
Honor trustFolderStats also when reading packed-refs

Then list of packed refs was cached in RefDirectory based on mtime of
the packed-refs file. This may fail on NFS when attributes are cached.
A cached mtime of the packed-refs file could cause JGit to trust the
cached content of this file and to overlook that the file is modified.

Honor the config option trustFolderStats and always read the packed-refs
content if the option is false. By default this option is set to true
and this fix is not active.

Change-Id: I2b65cfaa8f4aba2efbf8a5e865d3f09f927e2eec

7 years agoPrepare 4.5.4-SNAPSHOT builds 27/103727/1
Matthias Sohn [Sat, 26 Aug 2017 06:08:46 +0000 (08:08 +0200)]
Prepare 4.5.4-SNAPSHOT builds

Change-Id: Id8b902bf2bf590b41f2e246c5ecf1592e1c411f2
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years agoJGit v4.5.3.201708160445-r 47/103147/1 v4.5.3.201708160445-r
Matthias Sohn [Wed, 16 Aug 2017 08:42:27 +0000 (10:42 +0200)]
JGit v4.5.3.201708160445-r

Change-Id: I2d57144976e3683e180d3a42edc6c3bf2905e87c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years agoMerge branch 'stable-4.6' into stable-4.7 75/103075/2
Matthias Sohn [Mon, 14 Aug 2017 21:50:52 +0000 (23:50 +0200)]
Merge branch 'stable-4.6' into stable-4.7

* stable-4.6:
  Update Oxygen Orbit p2 repository to R20170516192513
  Fix exception handling for opening bitmap index files

Change-Id: I669fe48ce0034f9ea1977d38ee39099497422c1c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years agoMerge branch 'stable-4.5' into stable-4.6 74/103074/2
Matthias Sohn [Mon, 14 Aug 2017 21:37:44 +0000 (23:37 +0200)]
Merge branch 'stable-4.5' into stable-4.6

* stable-4.5:
  Fix exception handling for opening bitmap index files

Change-Id: Ifb511238e3e98b1bc9f79a990807b940a17ebaa6
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years agoUpdate Oxygen Orbit p2 repository to R20170516192513 80/103080/1
Matthias Sohn [Mon, 14 Aug 2017 21:37:21 +0000 (23:37 +0200)]
Update Oxygen Orbit p2 repository to R20170516192513

Change-Id: I7f1b733ca414d77c9df5572df02e742e0a84ba2f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years agoFix exception handling for opening bitmap index files 98/102598/3
Christian Halstrick [Mon, 7 Aug 2017 12:26:46 +0000 (14:26 +0200)]
Fix exception handling for opening bitmap index files

When creating a new PackFile instance it is specified whether this pack
has an associated bitmap index file or not. This information is cached
and the public method getBitmapIndex() will always assume a bitmap index
file must exist if the cached data tells so. But it may happen that the
packfiles are repacked during a gc in a different process causing the
packfile, bitmap-index and index file to be deleted. Since JGit still
has an open FileHandle on the packfile this file is not really deleted
and can still be accessed. But index and bitmap index file are deleted.
Fix getBitmapIndex() to invalidate the cached packfile instance if such
a situation occurs.

This problem showed up when a gerrit server was serving repositories
which where garbage collected with native git regularly. Fetch and
clone commands for certain repositories failed permanently after a
native git gc had deleted old bitmap index files.

Change-Id: I8e620bec74dd3f310ba42024f9a657062f868f0e
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years agoPrepare 4.7.2-SNAPSHOT builds 92/98892/1
Matthias Sohn [Thu, 8 Jun 2017 11:33:25 +0000 (13:33 +0200)]
Prepare 4.7.2-SNAPSHOT builds

Change-Id: I7c127bd402cd84c68d8f33a32c6aad093a2264c8
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years agoJGit v4.7.1.201706071930-r 53/98853/1 v4.7.1.201706071930-r
Matthias Sohn [Wed, 7 Jun 2017 23:19:38 +0000 (01:19 +0200)]
JGit v4.7.1.201706071930-r

Change-Id: I28cd8fbe995d76c8a00e7db6ddf826e983d89043
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years agoArchiveCommand: Create prefix entry with commit time 78/98478/3
Yasuhiro Takagi [Sat, 29 Apr 2017 11:35:10 +0000 (20:35 +0900)]
ArchiveCommand: Create prefix entry with commit time

The cgit archive command creates a prefix (root) directory entry
in the archive file. That entry's time is set to the commit time.

This patch makes jgit's behavior consistent with with cgit:

prefix: hoge/     -> creates prefix directory "hoge/" entry.
prefix: hoge////  -> creates prefix directory "hoge/" entry.
prefix: hoge/foo  -> does not create prefix directory entry, but for
                     each file/directory entry, prefix is added.

Change-Id: I2610e40ce37972c5f7456fdca6337e7fb07176e5
Signed-off-by: Yasuhiro Takagi <ytakagi@bea.hi-ho.ne.jp>
7 years agoRun auto GC in the background 74/98474/2
David Turner [Wed, 8 Feb 2017 20:07:18 +0000 (15:07 -0500)]
Run auto GC in the background

When running an automatic GC on a FileRepository, when the caller
passes a NullProgressMonitor, run the GC in a background thread. Use a
thread pool of size 1 to limit the number of background threads spawned
for background gc in the same application. In the next minor release we
can make the thread pool configurable.

In some cases, the auto GC limit is lower than the true number of
unreachable loose objects, so auto GC will run after every (e.g) fetch
operation.  This leads to the appearance of poor fetch performance.
Since these GCs will never make progress (until either the objects
become referenced, or the two week timeout expires), blocking on them
simply reduces throughput.

In the event that an auto GC would make progress, it's still OK if it
runs in the background. The progress will still happen.

This matches the behavior of regular git.

Git (and now jgit) uses the lock file for gc.log to prevent simultaneous
runs of background gc. Further, it writes errors to gc.log, and won't
run background gc if that file is present and recent. If gc.log is too
old (according to the config gc.logexpiry), it will be ignored.

Change-Id: I3870cadb4a0a6763feff252e6eaef99f4aa8d0df
Signed-off-by: David Turner <dturner@twosigma.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years agoCleanup and test trailing slash handling in ManifestParser 28/94528/3
Han-Wen Nienhuys [Tue, 28 Mar 2017 12:00:38 +0000 (14:00 +0200)]
Cleanup and test trailing slash handling in ManifestParser

This is a workaround for
https://bugs.openjdk.java.net/browse/JDK-4666701.

Change-Id: Idd04657e8d95a841d72230f8881b6b899daadbc2
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years agoManifestParser: Throw exception if remote does not have fetch attribute 23/94523/3
Han-Wen Nienhuys [Thu, 6 Apr 2017 04:44:10 +0000 (13:44 +0900)]
ManifestParser: Throw exception if remote does not have fetch attribute

In the repo manifest documentation [1] the fetch attribute is marked
as "#REQUIRED".

If the fetch attribute is not specified, this would previously result in
NullPointerException. Throw a SAXException instead.

[1] https://gerrit.googlesource.com/git-repo/+/master/docs/manifest-format.txt

Change-Id: Ib8ed8cee6074fe6bf8f9ac6fc7a1664a547d2d49
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
7 years agoMerge branch 'stable-4.6' into stable-4.7 19/94719/1
David Pursehouse [Sun, 9 Apr 2017 11:03:26 +0000 (20:03 +0900)]
Merge branch 'stable-4.6' into stable-4.7

* stable-4.6:
  Prepare 4.5.3-SNAPSHOT builds
  JGit v4.5.2.201704071617-r

Change-Id: I5d2044c59af7bc2786fb66ebf5130e412884d74b
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
7 years agoMerge branch 'stable-4.5' into stable-4.6 17/94717/1
David Pursehouse [Sun, 9 Apr 2017 08:22:30 +0000 (17:22 +0900)]
Merge branch 'stable-4.5' into stable-4.6

* stable-4.5:
  Prepare 4.5.3-SNAPSHOT builds
  JGit v4.5.2.201704071617-r

Change-Id: I4e1915d3a88e15c9ef1fb2b470b7f3b821243ff5
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
7 years agoPrepare 4.5.3-SNAPSHOT builds 05/94705/1
Matthias Sohn [Fri, 7 Apr 2017 22:31:09 +0000 (00:31 +0200)]
Prepare 4.5.3-SNAPSHOT builds

Change-Id: I69681b7a5687ca76bd0dd5d3e7ce2cff841d0e32
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years agoJGit v4.5.2.201704071617-r 00/94700/1 v4.5.2.201704071617-r
Matthias Sohn [Fri, 7 Apr 2017 20:17:27 +0000 (22:17 +0200)]
JGit v4.5.2.201704071617-r

Change-Id: I66402643d7c84c90bf5cefed4d2ec3aa68c94cfb
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years agoPushConnectionTest: Increase maxCommandBytes again 20/94520/1
David Pursehouse [Thu, 6 Apr 2017 02:23:46 +0000 (11:23 +0900)]
PushConnectionTest: Increase maxCommandBytes again

It was already increased in 61a943e, but that was still not enough to
take into account the length of snapshot versions.

Change-Id: Ib54cec97e97042fe274b87a3a1afa9bb06c8bf19
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
7 years agoPrepare 4.7.1-SNAPSHOT 06/94506/1
Matthias Sohn [Wed, 5 Apr 2017 22:16:53 +0000 (00:16 +0200)]
Prepare 4.7.1-SNAPSHOT

Change-Id: I16a45035258276217446bccc0ad1b0991383aa0c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years agoJGit v4.7.0.201704051617-r 96/94496/1 v4.7.0.201704051617-r
Matthias Sohn [Wed, 5 Apr 2017 20:17:44 +0000 (22:17 +0200)]
JGit v4.7.0.201704051617-r

Change-Id: Ic2bd6aca0b7a7e0597ffc1f7cf647b49878f9950
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years agoTarFormat: Use commit time as timestamp for entries 22/94422/1
Yasuhiro Takagi [Sun, 2 Apr 2017 03:08:53 +0000 (12:08 +0900)]
TarFormat: Use commit time as timestamp for entries

This was already done for ZipFormat in 1448ec3.

Change-Id: Ie131938a3f640c68718dc793d18dd35be5c71e7c
Signed-off-by: Yasuhiro Takagi <ytakagi@bea.hi-ho.ne.jp>
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
7 years agoPushConnectionTest#limitCommandBytes: Increase maxCommandBytes 66/94366/1
David Pursehouse [Tue, 4 Apr 2017 12:14:17 +0000 (21:14 +0900)]
PushConnectionTest#limitCommandBytes: Increase maxCommandBytes

A higher limit is required to account for proper JGit version number
being sent in the UserAgent.

The version string "4.7.0.201704031717-r" is 20 characters, however
the strings used during development are shorter:

- When running from mvn, "4.7.0.qualifier" is used; 15 characters
- When running in Eclipse, "unknown" is used; 7 characters

Change-Id: I9aca2f71389a42fedce305e9078db016869c3d1a
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
7 years agoDowngrade Jetty to 9.3.17.v20170317 in Bazel build 75/94275/1
David Pursehouse [Sun, 2 Apr 2017 23:22:26 +0000 (08:22 +0900)]
Downgrade Jetty to 9.3.17.v20170317 in Bazel build

Jetty was downgraded on stable-4.6 which doesn't have a Bazel
build.

Change-Id: Ibeb91436169e852a4241189f80e4895b54ce3cd0
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
7 years agoMerge branch 'stable-4.6' 68/94268/2
Matthias Sohn [Sun, 2 Apr 2017 11:56:32 +0000 (13:56 +0200)]
Merge branch 'stable-4.6'

* stable-4.6:
  Update Jetty to 9.3.17.v20170317
  Revert "Update Jetty to 9.4.1.v20170120 in buck build"
  Revert "Update Jetty to 9.4.1.v20170120"

Change-Id: I9193b568cce9c72da899a8f3eaaf833956438e13
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years agoUpdate Jetty to 9.3.17.v20170317 57/94257/2
Matthias Sohn [Sat, 1 Apr 2017 10:38:22 +0000 (12:38 +0200)]
Update Jetty to 9.3.17.v20170317

Update to the latest 9.3 version until we have a solution for bug 514336
we face with 9.4.x versions.

Change-Id: I6d8d476abe8677ce865a08099bb77330effc700a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years agoRevert "Update Jetty to 9.4.1.v20170120 in buck build" 56/94256/1
Matthias Sohn [Sat, 1 Apr 2017 11:03:48 +0000 (13:03 +0200)]
Revert "Update Jetty to 9.4.1.v20170120 in buck build"

This reverts commit d3b80b951c0bf995f7d210710e3d31e9d529efe2.

Change-Id: I596ba596f3e857346020e01b0a018d90a0b27edd

7 years agoRevert "Update Jetty to 9.4.1.v20170120" 93/94193/1
David Pursehouse [Fri, 31 Mar 2017 05:49:30 +0000 (14:49 +0900)]
Revert "Update Jetty to 9.4.1.v20170120"

This reverts commit e05caf91f1df86a95d3e0454000d345bb90a415e.

Change-Id: Ia4287f130034a790759b5035505a093b4616a05a

7 years agoRemove unused import from ManifestParser 87/94187/1
Matthias Sohn [Thu, 30 Mar 2017 22:38:36 +0000 (00:38 +0200)]
Remove unused import from ManifestParser

Change-Id: Ie60ef9c7bc6ce0fdf017949ebfb9a21753e70506
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years agoDowngrade jetty to 9.3.9.v20160517 85/94185/2
Matthias Sohn [Thu, 30 Mar 2017 22:34:57 +0000 (00:34 +0200)]
Downgrade jetty to 9.3.9.v20160517

This works around the problems we faced with jetty 9.4.3.v20170317.

Bug: 514336
Change-Id: I7e9dd539414624d71587c55cb436b843949aa296
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years agoRequire jetty 9.4.0 consistently 31/94131/2
Matthias Sohn [Sun, 26 Mar 2017 23:14:59 +0000 (01:14 +0200)]
Require jetty 9.4.0 consistently

Change-Id: If9de4c0e43850dec46858bd58e9692a7e099cbc3

7 years agoDocument the intended use of RepoCommand#setURI() 47/94047/3
Han-Wen Nienhuys [Wed, 29 Mar 2017 11:51:57 +0000 (13:51 +0200)]
Document the intended use of RepoCommand#setURI()

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Change-Id: I4a59dd8278b7b0026094692127b7f55e89c10bae

7 years agoNoop changes to ManifestParser 71/93971/3
Han-Wen Nienhuys [Tue, 7 Mar 2017 18:14:27 +0000 (19:14 +0100)]
Noop changes to ManifestParser

* Parse the base URL in ManifestParser construction.  This will signal
  errors earlier.

* Simplify stripping of trailing slashes.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Change-Id: I4a86f68c9d7737f71cf20352cfe26288fbd2b463

7 years agoConsistently use 'path' for the path to a subrepo in RepoCommand 56/93756/5
Han-Wen Nienhuys [Thu, 23 Mar 2017 17:20:08 +0000 (18:20 +0100)]
Consistently use 'path' for the path to a subrepo in RepoCommand

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Change-Id: I79ea7eb7b4d319e0100e3121aca5ef82eb8ad92a

7 years agoMerge branch 'stable-4.6' 73/93873/2
Matthias Sohn [Mon, 27 Mar 2017 08:52:36 +0000 (10:52 +0200)]
Merge branch 'stable-4.6'

* stable-4.6:
  Only mark packfile invalid if exception signals permanent problem
  Don't flag a packfile invalid if opening existing file failed
  Prepare 4.5.2-SNAPSHOT builds

Change-Id: Ife4efad1135d3870a5a0fb71e60b9524fb8777ab
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years agoMerge branch 'stable-4.5' into stable-4.6 62/93862/1
David Pursehouse [Mon, 27 Mar 2017 01:14:50 +0000 (10:14 +0900)]
Merge branch 'stable-4.5' into stable-4.6

* stable-4.5:
  Only mark packfile invalid if exception signals permanent problem
  Don't flag a packfile invalid if opening existing file failed
  Prepare 4.5.2-SNAPSHOT builds

Change-Id: I20b50981adc54c426666015ff04fe3bb1db9abd9
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
7 years agoOnly mark packfile invalid if exception signals permanent problem 44/93844/5
Matthias Sohn [Sat, 25 Mar 2017 01:33:06 +0000 (02:33 +0100)]
Only mark packfile invalid if exception signals permanent problem

Add NoPackSignatureException and UnsupportedPackVersionException to
explicitly mark permanent unrecoverable problems with a pack

Assume problem with a pack is permanent only if we are sure the
exception signals a non-transient problem we can't recover from:
- AccessDeniedException: we lack permissions
- CorruptObjectException: we detected corruption
- EOFException: file ended unexpectedly
- NoPackSignatureException: pack has no pack signature
- NoSuchFileException: file has gone missing
- PackMismatchException: pack no longer matches its index
- UnpackException: unpacking failed
- UnsupportedPackIndexVersionException: unsupported pack index version
- UnsupportedPackVersionException: unsupported pack version

Do not attempt to handle Errors since they are thrown for serious
problems applications should not try to recover from.

Change-Id: I2c416ce2b0e23255c4fb03a3f9a0ee237f7a484a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years agoRaise minimum version of o.e.jetty.security to 9.4.0 56/93856/1
Matthias Sohn [Sat, 25 Mar 2017 22:34:24 +0000 (23:34 +0100)]
Raise minimum version of o.e.jetty.security to 9.4.0

This is required since we now use AbstractLoginService which was
introduced in Jetty 9.4.0.

Change-Id: Iee6e4ae456123099f7a99cc53c5ae02fd43bfa46
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years agoDon't flag a packfile invalid if opening existing file failed 75/93775/4
Luca Milanesio [Fri, 24 Mar 2017 00:18:12 +0000 (00:18 +0000)]
Don't flag a packfile invalid if opening existing file failed

A packfile random file open operation may fail with a
FileNotFoundException even if the file exists, possibly
for the temporary lack of resources.

Instead of managing the FileNotFoundException as any generic
IOException it is best to rethrow the exception but prevent
the packfile for being flagged as invalid until it is actually
opened and read successfully or unsuccessfully.

Bug: 514170
Change-Id: Ie37edba2df77052bceafc0b314fd1d487544bf35
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years agoPrepare 4.5.2-SNAPSHOT builds 43/93843/1
Matthias Sohn [Sat, 25 Mar 2017 00:21:58 +0000 (01:21 +0100)]
Prepare 4.5.2-SNAPSHOT builds

Change-Id: I8485de1f3f63dc9ec445b8fb08093ca144aedc59
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years agoUpgrade error_prone_core to 2.0.19 80/93780/1
David Pursehouse [Fri, 24 Mar 2017 06:21:18 +0000 (15:21 +0900)]
Upgrade error_prone_core to 2.0.19

Change-Id: I019f6e8ce4b074697c2dfc6020b10dc003d0804d
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
7 years agoPullCommand: Add support for recursing into submodules 41/91241/9
David Pursehouse [Thu, 16 Feb 2017 05:18:04 +0000 (14:18 +0900)]
PullCommand: Add support for recursing into submodules

Add a new API method to set the recurse mode, and pass the mode into
the fetch command.

Extend the existing FetchCommandRecurseSubmodulesTest to also perform
the same tests for fetch. Rename the test class accordingly.

Change-Id: I12553af47774b4778f7011e1018bd575a7909bd0
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
7 years agobazel: Consume hamcrest through transitive dependency 68/93668/2
David Ostrovsky [Thu, 23 Mar 2017 05:44:51 +0000 (06:44 +0100)]
bazel: Consume hamcrest through transitive dependency

In I3ab958ce8 explicit dependency in lib/BUILD were defined and most
of the bazel build implementation was switched to using it. Switch
test.bzl test implementation to using explicit dependencies as well.

Change-Id: I4413d1a45addeeb2a980d07669fa034c2eebb3a4
Signed-off-by: David Ostrovsky <david@ostrovsky.org>
7 years agoExplain in error message how to recover from lock failure 52/93452/3
Matthias Sohn [Mon, 20 Mar 2017 19:48:49 +0000 (20:48 +0100)]
Explain in error message how to recover from lock failure

Bug: 483897
Change-Id: I70f8d9c82c1efe2928f072a2fb69461160f7c5f7
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years agoUpdate orbit to S20170306214312 (Oxygen M6) 56/93656/1
Matthias Sohn [Wed, 22 Mar 2017 17:35:49 +0000 (18:35 +0100)]
Update orbit to S20170306214312 (Oxygen M6)

Change-Id: I44d2483c045c126e01751fed9365d4889a30ab2e
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years agoMerge "Update Jetty to 9.4.3.v20170317"
Matthias Sohn [Wed, 22 Mar 2017 19:57:49 +0000 (15:57 -0400)]
Merge "Update Jetty to 9.4.3.v20170317"

7 years agoUpdate Jetty to 9.4.3.v20170317 34/93634/1
Matthias Sohn [Wed, 22 Mar 2017 16:48:23 +0000 (17:48 +0100)]
Update Jetty to 9.4.3.v20170317

Change-Id: I900c74bb76b6cfaee838e6f79d80c9b2840806df
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years agoMerge branch 'stable-4.6' 88/93588/1
David Pursehouse [Wed, 22 Mar 2017 01:25:46 +0000 (10:25 +0900)]
Merge branch 'stable-4.6'

* stable-4.6:
  JGit v4.5.1.201703201650-r

Change-Id: I16cac951cdafbe22afe597198fe3b86ecc2437e8
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
7 years agoMerge branch 'stable-4.5' into stable-4.6 87/93587/1
David Pursehouse [Wed, 22 Mar 2017 00:44:42 +0000 (09:44 +0900)]
Merge branch 'stable-4.5' into stable-4.6

* stable-4.5:
  JGit v4.5.1.201703201650-r

Change-Id: Ic9dda2317456308c222928cb8453cb00a231eea1
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
7 years agoRemove Buck build 98/90698/6
David Pursehouse [Thu, 9 Feb 2017 06:13:01 +0000 (15:13 +0900)]
Remove Buck build

Buck will be replaced with Bazel

Change-Id: I3cf07d7aaaa2a58bac34e16c50af5416693254ac
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years agoAdd remaining parts of the bazel build 46/93346/10
David Ostrovsky [Sat, 18 Mar 2017 15:13:26 +0000 (16:13 +0100)]
Add remaining parts of the bazel build

Add bazel build for ui and junit.http, and the test packages.

A number of different test labels are supported:

  api
  attributes
  dfs
  diff
  http
  lfs
  lfs-server
  nls
  notes
  pack
  patch
  pgm
  reftree
  revplot
  revwalk
  storage
  submodule
  symlinks
  transport
  treewalk
  util

To run all tests:

  bazel test //...

To run specific tests, using labels:

  bazel test --test_tag_filters=api,dfs,revplot,treewalk //...

Change-Id: Ic41b05a79d855212e67b1b4707e9c6b4dc9ea70d
Signed-off-by: David Ostrovsky <david@ostrovsky.org>
Signed-off-by: Jonathan Nieder <jrn@google.com>
7 years agoDelegate crypto algorithm creation to InsecureCipherFactory 74/93474/2
David Ostrovsky [Tue, 21 Mar 2017 05:49:49 +0000 (06:49 +0100)]
Delegate crypto algorithm creation to InsecureCipherFactory

This is a preparation change to Bazel build implementation. Error
Prone rejects the code with variable crypto algorithm as insecure
see: [1].

[1] http://errorprone.info/bugpattern/InsecureCryptoUsage

Change-Id: I92db70a7da454bc364597a995e8be5dccc2d6427
Signed-off-by: David Ostrovsky <david@ostrovsky.org>
7 years agoJGit v4.5.1.201703201650-r 58/93458/1 v4.5.1.201703201650-r
Matthias Sohn [Mon, 20 Mar 2017 20:44:47 +0000 (21:44 +0100)]
JGit v4.5.1.201703201650-r

Change-Id: I88de7c9f52abbc4921a82208ed74d22aa19fb3cd
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years agobazel: Mark junit targets testonly 94/93394/2
Jonathan Nieder [Mon, 20 Mar 2017 01:52:55 +0000 (18:52 -0700)]
bazel: Mark junit targets testonly

Only testonly targets (such as tests) need to use junit.

In particular this involves making the toplevel :all rule testonly.
It's not clear to me what that rule is for --- "bazel build //..."
already works to build all targets.  In any case it appears to be for
testing, so marking it as testonly shouldn't be harmful.

Change-Id: I28ff508ab8ce2ec0a0111109110aa9680d30600e

7 years agobazel: Add explicit targets for library dependencies 93/93393/4
Jonathan Nieder [Mon, 20 Mar 2017 00:41:26 +0000 (17:41 -0700)]
bazel: Add explicit targets for library dependencies

This provides a place to declare visibility restrictions and
transitive dependencies for each library.

Other targets should only declare dependencies on what they directly
use, making dependencies easier to maintain.

Trim the dependencies of org.eclipse.jgit:jgit to follow that rule.
It declares dependencies on Apache httpcomponents and the servlet
API but doesn't use them.

Tested:
* 'bazel build //...' succeeds
* applying the change https://gerrit-review.googlesource.com/90843
  to a copy of Gerrit, following the instructions there, and running
  'bazel test //...' in that copy of Gerrit still succeeds

Change-Id: I3ab958ce8b3227019cdbe4cc81e0f042e1541034

7 years agoFix test configuration to run RacyGitTests, and fix testRacyGitDetection 53/93353/4
David Pursehouse [Sun, 19 Mar 2017 01:23:29 +0000 (10:23 +0900)]
Fix test configuration to run RacyGitTests, and fix testRacyGitDetection

With the filename suffix "Tests", the module was not included in tests
when building with Maven, and without the @Test annotations the tests
didn't get executed under Eclipse or buck test.

testRacyGitDetection was failing because the index file did not exist.

Add the missing configuration, the missing annotations, and add a call
to reset() in testRacyGitDetection to force creation of the index file.

Change-Id: I29dd8f89c36fef4ab40bedce7f4a26bd9b2390e4
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
7 years agoLongObjectIdTest: Remove self comparison test 45/93345/1
David Ostrovsky [Sat, 18 Mar 2017 15:20:40 +0000 (16:20 +0100)]
LongObjectIdTest: Remove self comparison test

This fixes test flagged by error prone:

LongObjectIdTest.java:294: error: [SelfComparison] An object is
compared to itself

assertEquals(0, id1.compareTo(id1));

Change-Id: Ife339c92b9570db2471a2fc029b3bfa114044790
Signed-off-by: David Ostrovsky <david@ostrovsky.org>
7 years agoRevFlagSetTest: Fix compilation error flagged by error prone 44/93344/1
David Ostrovsky [Sat, 18 Mar 2017 10:29:26 +0000 (11:29 +0100)]
RevFlagSetTest: Fix compilation error flagged by error prone

This fixes error flagged by error prone:

Java compilation in rule '//org.eclipse.jgit.test:jgit' failed: Worker
process sent response with exit code: 1.
org.eclipse.jgit.test/tst/org/eclipse/jgit/revwalk/RevFlagSetTest.java:149:
error: [CollectionIncompatibleType] Argument '"bob"' should not be
passed to this method; its type String is not compatible with its
collection's type argument RevFlag
assertFalse(set.contains("bob"));

Change-Id: I4a971ce92fee55e28b2ab0c7b716ac20fa9c6709
Signed-off-by: David Ostrovsky <david@ostrovsky.org>
7 years agoMove SHA1 compress/recompress files to resource folder 43/93343/1
David Ostrovsky [Sat, 18 Mar 2017 09:41:29 +0000 (10:41 +0100)]
Move SHA1 compress/recompress files to resource folder

This fixes Bazel build:

in srcs attribute of java_library rule //org.eclipse.jgit:jgit:
file '//org.eclipse.jgit:src/org/eclipse/jgit/util/sha1/SHA1.recompress'
is misplaced here (expected .java, .srcjar or .properties).

Another option that was considered is to exclude the non source files.

Change-Id: I7083f27a4a49bf6681c85c7cf7b08a83c9a70c77
Signed-off-by: David Ostrovsky <david@ostrovsky.org>
7 years agoMerge "Merge branch 'stable-4.6'"
Matthias Sohn [Wed, 15 Mar 2017 23:50:03 +0000 (19:50 -0400)]
Merge "Merge branch 'stable-4.6'"

7 years agoMerge branch 'stable-4.6' 66/93166/2
Matthias Sohn [Wed, 15 Mar 2017 23:29:43 +0000 (00:29 +0100)]
Merge branch 'stable-4.6'

* stable-4.6:
  Don't remove pack when FileNotFoundException is transient

Change-Id: I82941a98385cda27c89e1e6750b7b6db4e39f414
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years agoMerge branch 'stable-4.5' into stable-4.6 64/93164/2
Matthias Sohn [Wed, 15 Mar 2017 23:12:49 +0000 (00:12 +0100)]
Merge branch 'stable-4.5' into stable-4.6

* stable-4.5:
  Don't remove pack when FileNotFoundException is transient

Change-Id: Ic17c542d78a4cad48ff1ed77dcdc853a4ef2dc06
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years agoDon't remove pack when FileNotFoundException is transient 64/92864/4
Luca Milanesio [Fri, 10 Mar 2017 00:20:23 +0000 (00:20 +0000)]
Don't remove pack when FileNotFoundException is transient

The FileNotFoundException is typically raised in three conditions:
1. file doesn't exist
2. incompatible read vs. read/write open modes
3. filesystem locking
4. temporary lack of resources (e.g. too many open files)

1. is already managed, 2. would never happen as packs are not
overwritten while with 3. and 4. it is worth logging the exception and
retrying to read the pack again.

Log transient errors using an exponential backoff strategy to avoid
flooding the logs with the same error if consecutive retries to access
the pack fail repeatedly.

Bug: 513435
Change-Id: I03c6f6891de3c343d3d517092eaa75dba282c0cd
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years agoDon't try to strip new line if the message buffer is empty 54/93154/1
Andrey Loskutov [Wed, 15 Mar 2017 19:29:21 +0000 (20:29 +0100)]
Don't try to strip new line if the message buffer is empty

Bug: 513726
Change-Id: I0e7c19f8883b93bad1b9de166f671d28f3e9c240
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
7 years agoUpgrade error_prone_core to 2.0.18 14/91514/2
David Pursehouse [Tue, 21 Feb 2017 01:38:09 +0000 (10:38 +0900)]
Upgrade error_prone_core to 2.0.18

Change-Id: I155a7ff1e2488c2184506e77d1a01fda06fb4d57
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
7 years agoFetchCommand: Fix detection of submodule recursion mode 70/92670/3
David Pursehouse [Thu, 9 Mar 2017 05:04:09 +0000 (14:04 +0900)]
FetchCommand: Fix detection of submodule recursion mode

The submodule.name.fetchRecurseSubmodules value was being read from the
configuration of the submodule, but it should be read from the config
of the parent repository.

Also, the fetch.recurseSubmodules value from the parent repository's
configuration was not being considered at all.

Fix both of these and add tests. Now the precedence of the recurse mode
is determined as follows:

 1. Value passed to the API
 2. Value configured in submodule.name.fetchRecurseSubmodules
 3. Value configured in fetch.recurseSubmodules
 4. Default to "on demand"

Change-Id: Ic23b7c40b5f39135fb3fd754c597dd4bcc94240c

7 years agoSilence API error reported in LfsProtocolServlet 27/92727/2
Matthias Sohn [Thu, 9 Mar 2017 19:41:02 +0000 (20:41 +0100)]
Silence API error reported in LfsProtocolServlet

Breaking API for API implementors (not clients) in a minor release is ok
following OSGi semantic versioning rules.

Change-Id: I14bb4a3084b237b4cd893a130e148148cd1e5df9
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years agoUpdate JGit version used to detect API changes to 4.6.0 25/92725/1
Matthias Sohn [Thu, 9 Mar 2017 19:35:04 +0000 (20:35 +0100)]
Update JGit version used to detect API changes to 4.6.0

We detect API changes during the Maven build using japicmp. Update the
reference version to the latest minor version 4.6.0.

Change-Id: I70f65f4a0680671612f21ee1afac7f7ef0152cbb
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years agoUpdate jsch version used in pom.xml to 0.1.54 85/92685/1
Matthias Sohn [Thu, 9 Mar 2017 11:39:43 +0000 (12:39 +0100)]
Update jsch version used in pom.xml to 0.1.54

We updated the version we use for packaging the p2 repository in
efbecb97 but missed to update the version we compile against in the
Maven build.

Bug: 513354
Change-Id: I49afd89b43cfd8735568a86ef273723909d694a3
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years agoLfsProtocolServlet: Pass HTTP Authorization header to getLargeFileRepository 72/91772/6
David Pursehouse [Fri, 24 Feb 2017 10:26:19 +0000 (19:26 +0900)]
LfsProtocolServlet: Pass HTTP Authorization header to getLargeFileRepository

This allows implementations to reject operations that do not
include proper authentication.

Change-Id: If301476d8fb56a0899e424be3789c7576097d185
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
7 years agoMerge branch 'stable-4.6' 74/92574/1
David Pursehouse [Tue, 7 Mar 2017 23:53:22 +0000 (08:53 +0900)]
Merge branch 'stable-4.6'

* stable-4.6:
  Prepare 4.6.2-SNAPSHOT builds
  JGit v4.6.1.201703071140-r

Change-Id: Idb3088ab6e64fb49f817b95bc0fef0a9efb77374
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
7 years agoPrepare 4.6.2-SNAPSHOT builds 53/92553/1
Matthias Sohn [Tue, 7 Mar 2017 19:49:24 +0000 (20:49 +0100)]
Prepare 4.6.2-SNAPSHOT builds

Change-Id: I8835f79145e6a989787d47322c3d8cb9baf0624a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years agoJGit v4.6.1.201703071140-r 35/92535/1 v4.6.1.201703071140-r
Matthias Sohn [Tue, 7 Mar 2017 16:40:32 +0000 (17:40 +0100)]
JGit v4.6.1.201703071140-r

Change-Id: I842dc95313e5b47b0b7ec983c4a0a91915ed4183
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years agoMerge branch 'stable-4.6' 47/92447/1
David Pursehouse [Tue, 7 Mar 2017 01:32:59 +0000 (10:32 +0900)]
Merge branch 'stable-4.6'

* stable-4.6:
  Update Jetty to 9.4.1.v20170120 in buck build
  Update Jetty to 9.4.1.v20170120
  Update build to use Tycho 1.0.0
  Update minimum JDK version in README

Change-Id: I735697c112094e883986ce13026d967291d88494
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
7 years agoUpdate Jetty to 9.4.1.v20170120 in buck build 32/92432/1
Jonathan Nieder [Sun, 26 Feb 2017 23:09:04 +0000 (15:09 -0800)]
Update Jetty to 9.4.1.v20170120 in buck build

5e8e2179b218ede7d14b69dc5149b0691b5859cf (Update Jetty to
9.4.1.v201470120, 2017-01-26) updated Jetty in the maven build.
Update the buck build to match so buck builds work again.

The buck build will go away soon, but in the meantime (until the bazel
build gets the same level of support) it is convenient as a faster way
of running tests than using maven.

The bazel build doesn't need this change since it doesn't build or run
http tests yet.

Change-Id: Ibbdaf2880e76b32fc9f6b5605a2ff29e3deffda2
(cherry picked from commit 2470f01d0f8bca4c8c754475846c83b9f99c93b8)

7 years agoUpdate Jetty to 9.4.1.v20170120 31/92431/1
Matthias Sohn [Thu, 26 Jan 2017 00:03:16 +0000 (01:03 +0100)]
Update Jetty to 9.4.1.v20170120

MappedLoginService is no longer available in Jetty 9.4 therefore base
TestLoginService on AbstractLoginService.

Apparently Jetty now uses slf4j hence adapt RecordingLogger accordingly
so we can log error messages containing slf4j style formatting anchors
"{}".

Change-Id: Ibb36aba8782882936849b6102001a88b699bb65c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
(cherry picked from commit 5e8e2179b218ede7d14b69dc5149b0691b5859cf)

7 years agoUpdate build to use Tycho 1.0.0 30/92430/1
Matthias Sohn [Wed, 15 Feb 2017 22:44:12 +0000 (23:44 +0100)]
Update build to use Tycho 1.0.0

Change-Id: I10f61962e1496f079a39e0f533506816be56e925
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
(cherry picked from commit 92d8c9bec2bd990cec937b3a6c5f4ab4011d3f73)

7 years agoUpdate minimum JDK version in README 29/92429/1
David Pursehouse [Fri, 10 Feb 2017 06:58:26 +0000 (15:58 +0900)]
Update minimum JDK version in README

Change-Id: I655d896b268e946e3492661b08add0ebac22c6f0
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
(cherry picked from commit 67da5635a47ad7680b3c60358a1fec080eef7d1e)

7 years agoFetchCommand: Add basic support for recursing into submodules 55/81855/17
David Pursehouse [Mon, 13 Feb 2017 12:37:30 +0000 (21:37 +0900)]
FetchCommand: Add basic support for recursing into submodules

Extend FetchCommand to expose a new method, setRecurseSubmodules(mode),
which allows to set the mode to ON, OFF or ON_DEMAND.

After fetching a repository, its submodules are recursively fetched:

- When the mode is YES, submodules are always fetched.

- When the mode is NO, submodules are not fetched.

- When the mode is ON_DEMAND, submodules are only fetched when the
  parent repository receives an update of the submodule and the new
  revision is not already in the submodule.

The mode is determined in the following order of precedence:

- Value specified in the API call using setRecurseSubmodules.

- Value specified in the repository's config under the key
  submodule.name.fetchRecurseSubmodules

- Defaults to ON_DEMAND if neither of the previous is set.

Extend FetchResult to recursively include results for submodules, as
a map of the submodule path to an instance of FetchResult.

Test setup is based on testCloneRepositoryWithNestedSubmodules.

Change-Id: Ibc841683763307cb76e78e142e0da5b11b1add2a
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
7 years agoRemove unnecessary @SuppressWarnings("nls") 06/92306/1
David Pursehouse [Sat, 4 Mar 2017 00:26:53 +0000 (09:26 +0900)]
Remove unnecessary @SuppressWarnings("nls")

Change-Id: Idc5f82af17ecc944b5657b02823412ea46b38413
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
7 years agoMake Repository.normalizeBranchName less strict 88/91488/7
Thomas Wolf [Thu, 23 Feb 2017 21:49:43 +0000 (22:49 +0100)]
Make Repository.normalizeBranchName less strict

This operation was added recently with the goal to provide some
way to auto-correct invalid user input, or to provide a correction
suggestion to the user -- EGit uses it now that way. But the initial
implementation was very restrictive; it removed all non-ASCII
characters and even slashes.

Understandably end users were not happy with that. Git has no such
restriction to ASCII-only; nor does JGit. Branch names should be
meaningful to the end user, and if a user-supplied branch name is
invalid for technical reasons, a "normalized" name should still
be meaningful to the user.

Rewrite to attempt a minimal fix such that the result will pass
isValidRefName.

* Replace all Unicode whitespace by underscore.
* Replace troublesome special characters by dash.
* Collapse sequences of underscores, dots, and dashes.
* Remove underscores, dots, and dashes following slashes, and
  collapse sequences of slashes.
* Strip leading and trailing sequences of slashes, dots, dashes,
  and underscores.
* Avoid the ".lock" extension.
* Avoid the Windows reserved device names.
* If input name is null return an empty String so callers don't need to
check for null.

This still allows branch names with single slashes as separators
between components, avoids some pitfalls that isValidRefName() tests
for, and leaves other character untouched and thus allows non-ASCII
branch names.

Also move the function from the bottom of the file up to where
isValidRefName is implemented.

Bug: 512508
Change-Id: Ia0576d9b2489162208c05e51c6d54e9f0c88c3a7
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years agoMerge "SHA-1: collision detection support"
Jonathan Nieder [Thu, 2 Mar 2017 18:26:42 +0000 (13:26 -0500)]
Merge "SHA-1: collision detection support"

7 years agoSHA-1: collision detection support 52/91852/8
Shawn Pearce [Sat, 25 Feb 2017 19:43:42 +0000 (11:43 -0800)]
SHA-1: collision detection support

Update SHA1 class to include a Java port of sha1dc[1]'s ubc_check,
which can detect the attack pattern used by the SHAttered[2] authors.

Given the shattered example files that have the same SHA-1, this
modified implementation can identify there is risk of collision given
only one file in the pair:

  $ jgit ...
  [main] WARN org.eclipse.jgit.util.sha1.SHA1 - SHA-1 collision 38762cf7f55934b34d179ae6a4c80cadccbb7f0a

When JGit detects probability of a collision the SHA1 class now warns
on the logger, reporting the object's SHA-1 hash, and then throws a
Sha1CollisionException to the caller.

From the paper[3] by Marc Stevens, the probability of a false positive
identification of a collision is about 14 * 2^(-160), sufficiently low
enough for any detected collision to likely be a real collision.

git-core[4] may adopt sha1dc before the system migrates to an entirely
new hash function.  This commit enables JGit to remain compatible with
that move to sha1dc, and help protect users by warning if similar
attacks as SHAttered are identified.

Performance declined about 8% (detection off), now:

  MessageDigest        238.41 MiB/s
  MessageDigest        244.52 MiB/s
  MessageDigest        244.06 MiB/s
  MessageDigest        242.58 MiB/s

  SHA1                 216.77 MiB/s (was ~240.83 MiB/s)
  SHA1                 220.98 MiB/s
  SHA1                 221.76 MiB/s
  SHA1                 221.34 MiB/s

This decline in throughput is attributed to the step loop unrolling in
compress(), which was necessary to easily fit the UbcCheck logic into
the hash function.  Using helper functions s1-s4 reduces the code
explosion, providing acceptable throughput.

With detection enabled (default):

  SHA1 detectCollision 180.12 MiB/s
  SHA1 detectCollision 181.59 MiB/s
  SHA1 detectCollision 181.64 MiB/s
  SHA1 detectCollision 182.24 MiB/s

  sha1dc (native C)   ~206.28 MiB/s
  sha1dc (native C)   ~204.47 MiB/s
  sha1dc (native C)   ~203.74 MiB/s

Average time across 100,000 calls to hash 4100 bytes (such as a commit
or tree) for the various algorithms available to JGit also shows SHA1
is slower than MessageDigest, but by an acceptable margin:

  MessageDigest        17 usec
  SHA1                 18 usec
  SHA1 detectCollision 22 usec

Time to index-pack for git.git (217982 objects, 69 MiB) has increased:

  MessageDigest   SHA1 w/ detectCollision
  -------------   -----------------------
         20.12s   25.25s
         19.87s   25.48s
         20.04s   25.26s

    avg  20.01s   25.33s    +26%

Being implemented in Java with these additional safety checks is
clearly a penalty, but throughput is still acceptable given the
increased security against object name collisions.

[1] https://github.com/cr-marcstevens/sha1collisiondetection
[2] https://shattered.it/
[3] https://marc-stevens.nl/research/papers/C13-S.pdf
[4] https://public-inbox.org/git/20170223230621.43anex65ndoqbgnf@sigill.intra.peff.net/

Change-Id: I9fe4c6d8fc5e5a661af72cd3246c9e67b1b9fee6

7 years agoSilence API error caused by changed return type of digest() 48/92048/1
Matthias Sohn [Tue, 28 Feb 2017 23:34:59 +0000 (00:34 +0100)]
Silence API error caused by changed return type of digest()

Change-Id: Ic0810ed7fea837c45cbc9a4649ca51d140bad6e6
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years agoCorrect the boolean logic for filtering paths 32/91432/6
Magnus Vigerlöf [Sat, 18 Feb 2017 18:28:39 +0000 (19:28 +0100)]
Correct the boolean logic for filtering paths

The TreeWalk filtering classes need to support the three different
meanings of the return value the path comparison generates.
A new path comparison method (isPathMatch) is created with
three distinct return values (isPathPrefix use value '0' to
encode two of these) which will makes it possible for the logical
operators (especially NOT) to aggregate a correct verdict.

A filter like: AND(Path("path"), NOT(Path("path/to/other")))
Should filter out 'path/to/other/file', but not 'path/to/my/file'.

The path-limiting feature when testing path/to/my/file, would
result to run test for the following paths:

    path
    path/to
    path/to/my
    path/to/my/file

isPathPrefix('path/to/other') will return '0' for the first two
and since there is no way for NOT to distinguish between an exact
match and a match indicating that the tested path is a 'parent',
it will incorrectly return false and thus remove everything below
'path' immediately.
isPathMatch has a distinguished value for 'parent' matches that
will be preserved through the logic operators and should not
cause an over-eager removal of paths.

The functionality of isPathPrefix is required by other parts
and is untouched.

Unit tests are included to ensure that the logical functionality
is correct and can be preserved.

Change-Id: Ice2ca9406f09f1b179569e99b86a0e5d77baa20d
Signed-off-by: Magnus Vigerlöf <magnus.vigerlof@gmail.com>
7 years agoMerge "SHA1: support reset() and reuse instances"
Jonathan Nieder [Mon, 27 Feb 2017 00:01:36 +0000 (19:01 -0500)]
Merge "SHA1: support reset() and reuse instances"

7 years agoSHA1: support reset() and reuse instances 72/91872/5
Shawn Pearce [Sun, 26 Feb 2017 19:44:51 +0000 (11:44 -0800)]
SHA1: support reset() and reuse instances

Allow SHA1 instances to be reused to compute another hash value, and
resume caching them in ObjectInserter and PackParser.  This shaves a
small amount of running time off parsing git.git's pack file:

  before   after
  ------   ------
  25.25s   25.55s
  25.48s   25.06s
  25.26s   24.94s

Almost noise (small difference), but recycling the instances reduces
some stress on the memory allocator finding two 80 word message block
arrays needed for hashing and collision detection.

Change-Id: I4af88a720e81460293bc5c5d1d3db1a831e7e228

7 years agoMerge changes from topic 'SHAttered'
Jonathan Nieder [Sun, 26 Feb 2017 23:17:02 +0000 (18:17 -0500)]
Merge changes from topic 'SHAttered'

* changes:
  Switch to pure Java SHA1 for ObjectId
  Pure Java SHA-1

7 years agoUpdate Jetty to 9.4.1.v20170120 in buck build 76/91876/2
Jonathan Nieder [Sun, 26 Feb 2017 23:09:04 +0000 (15:09 -0800)]
Update Jetty to 9.4.1.v20170120 in buck build

5e8e2179b218ede7d14b69dc5149b0691b5859cf (Update Jetty to
9.4.1.v201470120, 2017-01-26) updated Jetty in the maven build.
Update the buck build to match so buck builds work again.

The buck build will go away soon, but in the meantime (until the bazel
build gets the same level of support) it is convenient as a faster way
of running tests than using maven.

The bazel build doesn't need this change since it doesn't build or run
http tests yet.

Change-Id: Ibbdaf2880e76b32fc9f6b5605a2ff29e3deffda2

7 years agoSwitch to pure Java SHA1 for ObjectId 37/91837/4
Shawn Pearce [Sat, 25 Feb 2017 07:33:18 +0000 (23:33 -0800)]
Switch to pure Java SHA1 for ObjectId

Generate names for objects using only the pure Java SHA1
implementation, but continue using MessageDigest in tests.
This opens the possibility of changing the hashing function
to incorporate additional safety measures, such as those
used in sha1dc[1].

Since MessageDigest has higher throughput, continue using
MessageDigest for computing pack, idx and DirCache trailers.
These are less likely to be sensitive to SHAttered[2] types
of attacks, as Git uses them to detect random bit flips
during transfer, and not for content identity.

[1] https://github.com/cr-marcstevens/sha1collisiondetection
[2] https://shattered.it/

Change-Id: If6da98334201f7f20cb916e46f782c45f373784e

7 years agoPure Java SHA-1 36/91836/4
Shawn Pearce [Fri, 24 Feb 2017 22:57:20 +0000 (14:57 -0800)]
Pure Java SHA-1

This implementation is derived straight from the description written
in RFC 3174.  On Mac OS X with Java 1.8.0_91 it offers similar
throughput as MessageDigest SHA-1:

  system   239.75 MiB/s
  system   244.71 MiB/s
  system   245.00 MiB/s
  system   244.92 MiB/s

  sha1     234.08 MiB/s
  sha1     244.50 MiB/s
  sha1     242.99 MiB/s
  sha1     241.73 MiB/s

This is the fastest implementation I could come up with.  Common SHA-1
implementation tricks such as unrolling loops creates a method too
large for the JIT to effectively optimize, resulting in lower overall
hashing throughput. Using a preprocessor to perform the register
renaming of A-E also didn't help, as again the method was too large
for the JIT to effectively optimize.

Fortunately the fastest version is a naive, straight-forward
implementation very close to the description in RFC 3174.

Change-Id: I228b05c4a294ca2ad51386cf0e47978c68e1aa42

7 years agoRemove unused imported package in org.eclipse.jgit.ant.test 10/91510/1
David Pursehouse [Mon, 20 Feb 2017 23:46:10 +0000 (08:46 +0900)]
Remove unused imported package in org.eclipse.jgit.ant.test

Change-Id: Id8e31a4d7cc25c3b953c67674797dc575b65f6c5
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>