Matthias Sohn [Thu, 8 Aug 2019 13:09:46 +0000 (15:09 +0200)]
[error prone] suppress AmbiguousMethodReference in AnyObjectId
Move the implementation of the static equals() method to a new method
and suppress the error. Deprecate the old method to signal that we
intend to remove it in the next major release.
See https://errorprone.info/bugpattern/AmbiguousMethodReference
Change-Id: I5e29c97f4db3e11770be589a6ccd785e2c9ac7f2 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
David Pursehouse [Thu, 18 Jul 2019 07:21:13 +0000 (16:21 +0900)]
MergeAlgorithm: Suppress Error Prone warning about reference equality
Error Prone reports:
[ReferenceEquality] Comparison using reference equality instead of
value equality
The END_EDIT instance is used as a marker, and thus it's OK to use
a reference equality comparison. Factor the comparison to a method
and add a suppression.
Change-Id: I7d9dc1fa21f46c984787056b0b5d163e313026a6 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Thomas Wolf [Thu, 4 Jul 2019 07:53:25 +0000 (09:53 +0200)]
FS_POSIX: handle Files.getFileStore() failures
Android unconditionally throws a SecurityException;[1] getFileStore()
is not supported. Catch the exception and don't attempt the hard-
linking atomic file mechanism.
Matthias Sohn [Thu, 8 Aug 2019 11:17:48 +0000 (13:17 +0200)]
Merge branch 'stable-5.3' into stable-5.4
* stable-5.3:
Fix OpenSshConfigTest#config
FileSnapshot: fix bug with timestamp thresholding
In LockFile#waitForStatChange wait in units of file time resolution
Cache FileStoreAttributeCache per directory
Fix FileSnapshot#save(long) and FileSnapshot#save(Instant)
Persist minimal racy threshold and allow manual configuration
Measure minimum racy interval to auto-configure FileSnapshot
Reuse FileUtils to recursively delete files created by tests
Fix FileAttributeCache.toString()
Add test for racy git detection in FileSnapshot
Repeat RefDirectoryTest.testGetRef_DiscoversModifiedLoose 100 times
Fix org.eclipse.jdt.core.prefs of org.eclipse.jgit.junit
Add missing javadoc in org.eclipse.jgit.junit
Enhance RepeatRule to report number of failures at the end
Fix FileSnapshotTests for filesystem with high timestamp resolution
Retry deleting test files in FileBasedConfigTest
Measure filesystem timestamp resolution already in test setup
Refactor FileSnapshotTest to use NIO APIs
Measure stored timestamp resolution instead of time to touch file
Handle CancellationException in FileStoreAttributeCache
Fix FileSnapshot#saveNoConfig
Use Instant for smudge time in DirCache and DirCacheEntry
Use Instant instead of milliseconds for filesystem timestamp handling
Workaround SecurityException in FS#getFsTimestampResolution
Fix NPE in FS$FileStoreAttributeCache.getFsTimestampResolution
FS: ignore AccessDeniedException when measuring timestamp resolution
Add debug trace for FileSnapshot
Use FileChannel.open to touch file and set mtime to now
Persist filesystem timestamp resolution and allow manual configuration
Increase bazel timeout for long running tests
Bazel: Fix lint warning flagged by buildifier
Update bazlets to latest version
Bazel: Add missing dependencies for ArchiveCommandTest
Bazel: Remove FileTreeIteratorWithTimeControl from BUILD file
Add support for nanoseconds and microseconds for Config#getTimeUnit
Optionally measure filesystem timestamp resolution asynchronously
Delete unused FileTreeIteratorWithTimeControl
FileSnapshot#equals: consider UNKNOWN_SIZE
Timeout measuring file timestamp resolution after 2 seconds
Fix RacyGitTests#testRacyGitDetection
Change RacyGitTests to create a racy git situation in a stable way
Deprecate Constants.CHARACTER_ENCODING in favor of StandardCharsets.UTF_8
Fix non-deterministic hash of archives created by ArchiveCommand
Update Maven plugins ecj, plexus, error-prone
Update Maven plugins and cleanup Maven warnings
Make inner classes static where possible
Fix API problem filters
Change-Id: Iec3ad6ccc194582cb844310dc172c3103dae4457 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Thu, 8 Aug 2019 10:30:47 +0000 (12:30 +0200)]
Merge branch 'stable-5.2' into stable-5.3
* stable-5.2:
Fix OpenSshConfigTest#config
FileSnapshot: fix bug with timestamp thresholding
In LockFile#waitForStatChange wait in units of file time resolution
Cache FileStoreAttributeCache per directory
Fix FileSnapshot#save(long) and FileSnapshot#save(Instant)
Persist minimal racy threshold and allow manual configuration
Measure minimum racy interval to auto-configure FileSnapshot
Reuse FileUtils to recursively delete files created by tests
Fix FileAttributeCache.toString()
Add test for racy git detection in FileSnapshot
Repeat RefDirectoryTest.testGetRef_DiscoversModifiedLoose 100 times
Fix org.eclipse.jdt.core.prefs of org.eclipse.jgit.junit
Add missing javadoc in org.eclipse.jgit.junit
Enhance RepeatRule to report number of failures at the end
Fix FileSnapshotTests for filesystem with high timestamp resolution
Retry deleting test files in FileBasedConfigTest
Measure filesystem timestamp resolution already in test setup
Refactor FileSnapshotTest to use NIO APIs
Measure stored timestamp resolution instead of time to touch file
Handle CancellationException in FileStoreAttributeCache
Fix FileSnapshot#saveNoConfig
Use Instant for smudge time in DirCache and DirCacheEntry
Use Instant instead of milliseconds for filesystem timestamp handling
Workaround SecurityException in FS#getFsTimestampResolution
Fix NPE in FS$FileStoreAttributeCache.getFsTimestampResolution
FS: ignore AccessDeniedException when measuring timestamp resolution
Add debug trace for FileSnapshot
Use FileChannel.open to touch file and set mtime to now
Persist filesystem timestamp resolution and allow manual configuration
Increase bazel timeout for long running tests
Bazel: Fix lint warning flagged by buildifier
Update bazlets to latest version
Bazel: Add missing dependencies for ArchiveCommandTest
Bazel: Remove FileTreeIteratorWithTimeControl from BUILD file
Add support for nanoseconds and microseconds for Config#getTimeUnit
Optionally measure filesystem timestamp resolution asynchronously
Delete unused FileTreeIteratorWithTimeControl
FileSnapshot#equals: consider UNKNOWN_SIZE
Timeout measuring file timestamp resolution after 2 seconds
Fix RacyGitTests#testRacyGitDetection
Change RacyGitTests to create a racy git situation in a stable way
Deprecate Constants.CHARACTER_ENCODING in favor of
StandardCharsets.UTF_8
Fix non-deterministic hash of archives created by ArchiveCommand
Update Maven plugins ecj, plexus, error-prone
Update Maven plugins and cleanup Maven warnings
Make inner classes static where possible
Fix API problem filters
Change-Id: I238adfd3080a5fed9d64c3c757297da6ea893918 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Thu, 8 Aug 2019 09:18:14 +0000 (11:18 +0200)]
Merge branch 'stable-5.1' into stable-5.2
* stable-5.1:
Fix OpenSshConfigTest#config
FileSnapshot: fix bug with timestamp thresholding
In LockFile#waitForStatChange wait in units of file time resolution
Cache FileStoreAttributeCache per directory
Fix FileSnapshot#save(long) and FileSnapshot#save(Instant)
Persist minimal racy threshold and allow manual configuration
Measure minimum racy interval to auto-configure FileSnapshot
Reuse FileUtils to recursively delete files created by tests
Fix FileAttributeCache.toString()
Add test for racy git detection in FileSnapshot
Repeat RefDirectoryTest.testGetRef_DiscoversModifiedLoose 100 times
Fix org.eclipse.jdt.core.prefs of org.eclipse.jgit.junit
Add missing javadoc in org.eclipse.jgit.junit
Enhance RepeatRule to report number of failures at the end
Fix FileSnapshotTests for filesystem with high timestamp resolution
Retry deleting test files in FileBasedConfigTest
Measure filesystem timestamp resolution already in test setup
Refactor FileSnapshotTest to use NIO APIs
Measure stored timestamp resolution instead of time to touch file
Handle CancellationException in FileStoreAttributeCache
Fix FileSnapshot#saveNoConfig
Use Instant for smudge time in DirCache and DirCacheEntry
Use Instant instead of milliseconds for filesystem timestamp handling
Workaround SecurityException in FS#getFsTimestampResolution
Fix NPE in FS$FileStoreAttributeCache.getFsTimestampResolution
FS: ignore AccessDeniedException when measuring timestamp resolution
Add debug trace for FileSnapshot
Use FileChannel.open to touch file and set mtime to now
Persist filesystem timestamp resolution and allow manual configuration
Increase bazel timeout for long running tests
Bazel: Fix lint warning flagged by buildifier
Update bazlets to latest version
Bazel: Add missing dependencies for ArchiveCommandTest
Bazel: Remove FileTreeIteratorWithTimeControl from BUILD file
Add support for nanoseconds and microseconds for Config#getTimeUnit
Optionally measure filesystem timestamp resolution asynchronously
Delete unused FileTreeIteratorWithTimeControl
FileSnapshot#equals: consider UNKNOWN_SIZE
Timeout measuring file timestamp resolution after 2 seconds
Fix RacyGitTests#testRacyGitDetection
Change RacyGitTests to create a racy git situation in a stable way
Deprecate Constants.CHARACTER_ENCODING in favor of StandardCharsets.UTF_8
Fix non-deterministic hash of archives created by ArchiveCommand
Update Maven plugins ecj, plexus, error-prone
Update Maven plugins and cleanup Maven warnings
Make inner classes static where possible
Fix API problem filters
Change-Id: Ia57385b2a60f48a5317c8d723721c235d7043a84 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Thu, 8 Aug 2019 08:10:12 +0000 (10:10 +0200)]
Fix OpenSshConfigTest#config
- use FS.DETECTED instead of db.getFS() since the ssh config is
typically in a different place than the repository, the same is used in
OpenSshConfig
- reduce unnecessary repeated writes by introducing wait for one tick of
the file time resolution
Change-Id: Ifac915e97ff420ec5cf8e2f162e351f9f51b6b14 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Increase the safety factor to 2.5x for extra safety if max of measured
timestamp resolution and measured minimal racy threshold is < 100ms, use
1.25 otherwise since for large filesystem resolution values the
influence of finite resolution of the system clock should be negligible.
Before, not yet using the newly introduced minRacyThreshold measurement,
the threshold was 1.1x FS resolution, and we could issue the
following sequence of events,
In this case, the difference between create-file and read is 5ms,
which exceeded the 4ms FS resolution, even though the events together
took just 2ms of runtime.
Reproduce with:
bazel test --runs_per_test=100 \
//org.eclipse.jgit.test:org_eclipse_jgit_internal_storage_file_FileSnapshotTest
The file system timestamp resolution is 4ms in this case.
This code assumes that the kernel and the JVM use the same clock that
is synchronized with the file system clock. This seems plausible,
given the resolution of System.currentTimeMillis() and the latency for
a gettimeofday system call (typically ~1us), but it would be good to
justify this with specifications.
Also cover a source of flakiness: if the test runs under extreme load,
then we could have
start
create-file
<long delay>
read
end
which would register as an unmodified file. Avoid this by skipping the
test if end-start is too big.
[msohn]:
- downported from master to stable-5.1
- skip test if resolution is below 10ms
- adjust safety factor to 1.25 for resolutions above 100ms
Change-Id: I87d2cf035e01c44b7ba8364c410a860aa8e312ef Signed-off-by: Han-Wen Nienhuys <hanwen@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Mon, 5 Aug 2019 16:00:35 +0000 (18:00 +0200)]
In LockFile#waitForStatChange wait in units of file time resolution
Since we now measure file time resolution we can use it to replace the
hard coded wait time of 25ms. FileSnapshot#equals will return true until
the mtime of the old (o) and the new FileSnapshot (n) differ by at least
one file time resolution.
Change-Id: Icb713a80ce9eb929242ed083406bfb6650c72223 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Fri, 19 Jul 2019 15:35:27 +0000 (17:35 +0200)]
Cache FileStoreAttributeCache per directory
Cache FileStoreAttributeCache entries since looking up FileStore for a
file may be expensive on some platforms.
Implement a simple LRU cache based on ConcurrentHashMap using a simple
long counter to order access to cache entries.
Change-Id: I4881fa938ad2f17712c05da857838073a2fc4ddb Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Signed-off-by: Marc Strapetz <marc.strapetz@syntevo.com> Also-By: Marc Strapetz <marc.strapetz@syntevo.com>
Matthias Sohn [Wed, 17 Jul 2019 14:31:42 +0000 (16:31 +0200)]
Persist minimal racy threshold and allow manual configuration
To enable persisting the minimal racy threshold per FileStore add a
new config option to the user global git configuration:
- Config section is "filesystem"
- Config subsection is concatenation of
- Java vendor (system property "java.vendor")
- Java version (system property "java.version")
- FileStore's name, on Windows we use the attribute volume:vsn instead
since the name is not necessarily unique.
- separated by '|'
e.g.
"AdoptOpenJDK|1.8.0_212-b03|/dev/disk1s1"
The same prefix is used as for filesystem timestamp resolution, so
both values are stored in the same config section
- The config key for minmal racy threshold is "minRacyThreshold" as a
time value, supported time units are those supported by
DefaultTypedConfigGetter#getTimeUnit
- measure for 3 seconds to limit runtime which depends on hardware, OS
and Java version being used
If the minimal racy threshold is configured for a given FileStore the
configured value is used instead of measuring it.
When the minimal racy threshold was measured it is persisted in the user
global git configuration.
Rename FileStoreAttributeCache to FileStoreAttributes since this class
is now declared public in order to enable exposing all attributes in one
object.
Change-Id: I22195e488453aae8d011b0a8e3276fe3d99deaea Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Also-By: Marc Strapetz <marc.strapetz@syntevo.com>
Matthias Sohn [Mon, 15 Jul 2019 13:00:09 +0000 (15:00 +0200)]
Measure minimum racy interval to auto-configure FileSnapshot
By running FileSnapshotTest#detectFileModified we found that the sum of
measured filesystem timestamp resolution and measured clock resolution
may yield a too small interval after a file has been modified which we
need to consider racily clean. In our tests we didn't find this behavior
on all systems we tested on, e.g. on MacOS using APFS and Java 8 and 11
this effect was not observed.
On Linux (SLES 15, kernel 4.12.14-150.22-default) we collected the
following test results using Java 8 and 11:
In 23-98% of 10000 test runs (depending on filesystem type and Java
version) the test failed, which means the effective interval which needs
to be considered racily clean after a file was modified is larger than
the measured file timestamp resolution.
"delta" is the observed interval after a file has been modified but
FileSnapshot did not yet detect the modification:
"resolution" is the measured sum of file timestamp resolution and clock
resolution seen in Java.
Java version filesystem failures resolution min delta max delta
1.8.0_212-b04 btrfs 98.6% 1 ms 3.6 ms 6.6 ms
1.8.0_212-b04 ext4 82.6% 3 ms 1.1 ms 4.1 ms
1.8.0_212-b04 xfs 23.8% 4 ms 3.7 ms 3.9 ms
1.8.0_212-b04 zfs 23.1% 3 ms 4.8 ms 5.0 ms
11.0.3+7 btrfs 98.1% 3 us 0.7 ms 4.7 ms
11.0.3+7 ext4 98.1% 6 us 0.7 ms 4.7 ms
11.0.3+7 xfs 98.5% 7 us 0.1 ms 8.0 ms
11.0.3+7 zfs 98.4% 7 us 0.7 ms 5.2 ms
Mac OS
1.8.0_212 APFS 0% 1 s
11.0.3+7 APFS 0% 6 us
The observed delta is not distributed according to a normal gaussian
distribution but rather random in the observed range between "min delta"
and "max delta".
Run this test after measuring file timestamp resolution in
FS.FileAttributeCache to auto-configure JGit since it's unclear what
mechanism is causing this effect.
In FileSnapshot#isRacyClean use the maximum of the measured timestamp
resolution and the measured "delta" as explained above to decide if a
given FileSnapshot is to be considered racily clean. Add a 30% safety
margin to ensure we are on the safe side.
Change-Id: I1c8bb59f6486f174b7bbdc63072777ddbe06694d Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Fri, 12 Jul 2019 07:49:13 +0000 (09:49 +0200)]
Add test for racy git detection in FileSnapshot
Repeat the test 10000 times to get statistics if measured
fsTimestampResolution is working in practice to detect racy git
situations.
Add a class to compute statistics for this test. Log delta between
lastModified and time when FileSnapshot failed to detect modification.
This happens if the racy git limit determined by measuring filesystem
timestamp resolution and clock resolution is too small. If it would be
correct FileSnapshot would always detect modification or mark it
modified if time since modification is smaller than the racy git limit.
Change-Id: Iabe7af1a7211ca58480f8902d4fa4e366932fc77 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Fri, 12 Jul 2019 07:45:09 +0000 (09:45 +0200)]
Enhance RepeatRule to report number of failures at the end
In order to enable counting how frequently a test fails if repeated add
option abortOnFailure. If it is true the test aborts on the first
failure. Otherwise it runs the configured number of repetitions and, if
there was any failure, throws a RepeatException reporting how many of
the test repetitions failed.
Change-Id: Ic47de44d4a6273fddf04b9993ad989903efb40c3 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Wed, 10 Jul 2019 14:17:21 +0000 (16:17 +0200)]
Fix FileSnapshotTests for filesystem with high timestamp resolution
When filesystem timestamp resolution is very high some tests don't work
since runtime of the test setup is too long to reach a racily clean
FileSnapshot. Hence skip these tests when timestamp resolution is higher
than 10 millisecond.
Change-Id: Ie47dd10eda22037b5c1ebff6b6becce0654ea807 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Mon, 8 Jul 2019 08:20:16 +0000 (10:20 +0200)]
Measure stored timestamp resolution instead of time to touch file
Measure granularity of timestamps stored in the filesystem by setting
and then getting lastModified timestamp until the read value changed.
Increase increment exponentially to limit number of iterations starting
with 1 microsecond since Java's FileTime (up to Java 12) truncates
timestamps to 1 microsecond resolution. The chosen algorithm yields 2000
steps between 1 ms and 2.5 s.
Also measure clock resolution and add that for the total timestamp
resolution. This avoids systematic measurement errors introduced by
doing IO to touch a file.
Change-Id: I9b37138619422452373e298d9d8c7cb2c384db3f Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Tue, 2 Jul 2019 23:07:14 +0000 (01:07 +0200)]
Use Instant instead of milliseconds for filesystem timestamp handling
This enables higher file timestamp resolution on filesystems like ext4,
Mac APFS (1ns) or NTFS (100ns) providing high timestamp resolution on
filesystem level.
Note:
- on some OSes Java 8,9 truncate milliseconds, see
https://bugs.openjdk.java.net/browse/JDK-8177809, fixed in Java 10
- UnixFileAttributes truncates timestamp resolution to microseconds when
converting the internal representation to FileTime exposed in the API,
see https://bugs.openjdk.java.net/browse/JDK-8181493
- WindowsFileAttributes also provides only microsecond resolution
Change-Id: I25ffff31a3c6f725fc345d4ddc2f26da3b88f6f2 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Wed, 26 Jun 2019 07:10:31 +0000 (09:10 +0200)]
FS: ignore AccessDeniedException when measuring timestamp resolution
It seems on cygwin creating a file under a writable directory can fail
with AccessDeniedException. Log a warning in this case and fallback to
worst case timestamp resolution of 2 seconds.
Bug: 548648
Change-Id: Ic50c31ce9dc9ccadd4db5247df929418ac62d45c Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Wed, 10 Jul 2019 08:19:01 +0000 (10:19 +0200)]
Add debug trace for FileSnapshot
Checking lastModified is time critical hence debug trace is the only way
to analyze issues since debugging is impractical.
Also add configuration for buffering of log4j output to reduce runtime
impact when debug trace is on. Limit buffer to 1MiB and comment this
configuration out since we may not always want to use buffering.
Change-Id: Ib1a0537b67c8dc3fac994a77b42badd974ce6c97 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Tue, 25 Jun 2019 23:11:12 +0000 (01:11 +0200)]
Use FileChannel.open to touch file and set mtime to now
Use options
- StandardOpenOption.CREATE to create touched file if not existing
- StandardOpenOption.SYNC to enforce synch of data and meta data changes
- StandardOpenOption.WRITE
Also set mtime explicitly in FileUtils#touch to the current system time.
This should fix that the previous implementation didn't work on
- locally cached Windows network share (CSC-CACHE filesystem) mapped as
a drive
- nfsv4 mounts on Linux
and that it didn't create unborn file like Linux command "touch".
Apache common's and Guava's touch() use the same approach.
Immediately after creating the probe file used to measure timestamp
resolution touch it. This ensures we always use the local system clock
when measuring filesystem timestamp resolution. This should prevent that
clock skew could influence the measured timestamp resolution in case of
a mounted network filesystem.
Bug: 548598
Change-Id: Iaeaf5967963f582395a195aa637b8188bfadac60 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Fri, 21 Jun 2019 16:12:14 +0000 (18:12 +0200)]
Persist filesystem timestamp resolution and allow manual configuration
To enable persisting filesystem timestamp resolution per FileStore add a
new config section to the user global git configuration:
- Config section is "filesystem"
- Config subsection is concatenation of
- Java vendor (system property "java.vm.vendor")
- runtime version (system property "java.vm.version")
- FileStore's name
- separated by '|'
e.g.
"AdoptOpenJDK|1.8.0_212-b03|/dev/disk1s1"
The prefix is needed since some Java versions do not expose the full
timestamp resolution of the underlying filesystem. This may also
depend on the underlying operating system hence concrete key values
may not be portable.
- Config key for timestamp resolution is "timestampResolution" as a time
value, supported time units are those supported by
DefaultTypedConfigGetter#getTimeUnit
If timestamp resolution is already configured for a given FileStore
the configured value is used instead of measuring the resolution.
When timestamp resolution was measured it is persisted in the user
global git configuration.
If locking the git config file fails retry saving the resolution up to 5
times in order to workaround races with another thread.
In order to avoid stack overflow use the fallback filesystem timestamp
resolution when loading FileBasedConfig which creates itself a
FileSnapshot to help checking if the config changed.
Note:
- on some OSes Java 8,9 truncate to milliseconds or seconds, see
https://bugs.openjdk.java.net/browse/JDK-8177809, fixed in Java 10
- UnixFileAttributes up to Java 12 truncates timestamp resolution to
microseconds when converting the internal representation to FileTime
exposed in the API, see https://bugs.openjdk.java.net/browse/JDK-8181493
- WindowsFileAttributes also provides only microsecond resolution up to
Java 12
Hence do not attempt to manually configure a higher timestamp resolution
than supported by the Java version being used at runtime.
Bug: 546891
Bug: 548188
Change-Id: Iff91b8f9e6e5e2295e1463f87c8e95edf4abbcf8 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Mon, 10 Jun 2019 21:35:20 +0000 (23:35 +0200)]
Increase bazel timeout for long running tests
EolRepositoryTest and GcCommitSelectionTest timed out frequently when
running unit tests using bazel with the default timeout "moderate"
(300s). Increase timeout of these tests to "long" (900s).
Change-Id: I43588cf950f55b50f868d9fe9c66d22bd428a54c Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
In order to avoid blocking on the main thread during measurement
interactive applications like EGit may want to measure the filesystem
timestamp resolution asynchronously.
In order to enable measurement in the background call
FileStoreAttributeCache.setAsyncfileStoreAttrCache(true)
before the first access to cached FileStore attributes.
Bug: 548188
Change-Id: I8c9a2dbfc3f1d33441edea18b90e36b1dc0156c7 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
In commit I5485db55 ("Fix FileSnapshot's consideration of file size"),
the special casing of UNKNOWN_SIZE was forgotten.
This change, together with I493f3b57b ("Measure file timestamp
resolution used in FileSnapshot") introduced a regression that would
occasionally surface in Gerrit integration tests marked UseLocalDisk,
with the symptom that creating the Admin user in NoteDb failed with a
LOCK_FAILURE.
Matthias Sohn [Fri, 21 Jun 2019 09:00:05 +0000 (11:00 +0200)]
Timeout measuring file timestamp resolution after 2 seconds
It was reported that measuring file timestamp resolution may hang
indefinitely on nfs. Hence timeout this measurement at the known worst
filesystem timestamp resolution (FAT) of 2 seconds.
Bug: 548188
Change-Id: I17004b0aa49d5b0e76360a008af3adb911b289c0 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Tue, 18 Jun 2019 09:32:59 +0000 (11:32 +0200)]
Fix RacyGitTests#testRacyGitDetection
This test case assumed file system timestamp resolution of 1 second. On
filesystems with a finer resolution this test fails since the index
entry is only smudged if the file index entry's lastModified and the
lastModified of the git index itself are within the same filesystem
timer tick. Fix this by ensuring that these timestamps are identical
which should work for any filesystem timer resolution.
Bug: 548188
Change-Id: Id84d59e1cfeb48fa008f8f27f2f892c4f73985de Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Matthias Sohn [Sun, 16 Jun 2019 21:58:06 +0000 (23:58 +0200)]
Fix non-deterministic hash of archives created by ArchiveCommand
Archives created by the ArchiveCommand didn't produce deterministic
archive hashes. For RevCommits RevWalk.parseTree returns the root tree
instead of the RevCommit hence retrieving the commit's timestamp didn't
work. Instead use RevWalk.parseAny and extract the tree manually.
Archive entries store timestamps with 1 second resolution hence we need
to wait longer when creating the same archive twice and compare archive
hashes. Otherwise hash comparison in tests wouldn't fail without this
patch.
Bug: 548312
Change-Id: I437d515de51cf68265584d28a8446cebe6341b79 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Tue, 18 Jun 2019 14:55:02 +0000 (16:55 +0200)]
Update Maven plugins ecj, plexus, error-prone
update Maven plugins
- ecj to 3.17.0
- error_prone_core to 2.3.3
- plexus-compiler-eclipse to 2.8.5
- plexus-compiler-javac to 2.8.5
- plexus-compiler-javac-errorprone to 2.8.5
Change-Id: I51ecb44538915ed84db041510562394bce977a3e Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Tue, 18 Jun 2019 14:55:02 +0000 (16:55 +0200)]
Update Maven plugins and cleanup Maven warnings
update Maven plugins
- jacoco-maven-plugin to 0.8.4
- japicmp-maven-plugin to 0.14.1
- maven-compiler-plugin to 3.8.1
- maven-deploy-plugin to 3.0.0-M1
- maven-enforcer-plugin to 3.0.0-M2
- maven-install-plugin to 3.0.0-M1
- maven-jar-plugin to 3.1.2
- maven-javadoc-plugin to 3.1.0
- maven-jxr-plugin to 3.0.0
- maven-pmd-plugin to 3.12.0
- maven-resources-plugin to 3.1.0
- maven-shade-plugin to 3.2.1
- maven-source-plugin to 3.1.0
- maven-surefire-plugin to 3.0.0-M3
- spotbugs-maven-plugin to 3.1.12
- tycho to 1.3.0
- tycho-pack200a-plugin to 1.3.0
- tycho-pack200b-plugin to 1.3.0
Cleanup Maven warnings
- pin version of all used Maven plugins
- remove deprecated way to declare minimum Maven version
Change-Id: If23e2e2bb03e5e1e7b1eb9d4924a8faa0aa3704e Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
David Pursehouse [Tue, 25 Sep 2018 05:11:23 +0000 (14:11 +0900)]
Make inner classes static where possible
As reported by Error Prone:
An inner class should be static unless it references members of its
enclosing class. An inner class that is made non-static unnecessarily
uses more memory and does not make the intent of the class clear.
See https://errorprone.info/bugpattern/ClassCanBeStatic
Change-Id: Ib99d120532630dba63cf400cc1c61c318286fc41 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
(cherry picked from commit ee40efcea44bc0c9a28afe29a80c87636947484e)
David Pursehouse [Sun, 16 Jun 2019 07:28:02 +0000 (16:28 +0900)]
Error Prone: Increase severity of NonOverridingEquals to ERROR
Error Prone reports the warning on several classes:
[NonOverridingEquals] equals method doesn't override Object.equals;
if this is a type-specific helper for a method that does override
Object.equals, either inline it into the callers or rename it to
avoid ambiguity.
See https://errorprone.info/bugpattern/NonOverridingEquals
Most of these are in the public API, so we can't rename or inline them
without breaking the API. FileSnapshot is not part of the public API,
but clients may be using it anyway, so we also shouldn't change that.
Suppress all the warnings instead. Having the check at severity ERROR
will at least make sure we don't introduce any new occurrences.
Change-Id: I92345c11256f06b4fa03ccc13337f72af5a43591 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
David Pursehouse [Sun, 16 Jun 2019 04:54:31 +0000 (13:54 +0900)]
GitDateParser#ParseableSimpleDateFormat: Make formatStr private final
ParseableSimpleDateFormat is an enum, and enums must be immutable,
hence the member should be final. At the same time, make it private
since it does not need to be publicly visible.
Change-Id: I7e181f591038d556f1123b6e37adf8441059e99a Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
David Pursehouse [Wed, 12 Jun 2019 04:02:07 +0000 (13:02 +0900)]
Blame: Suppress ReferenceEquality warning for RevCommit instances
Reference comparison of the RevCommit objects is OK; BlameGenerator uses
a single RevWalk which caches the RevCommits, so if a given commit is
cached the RevWalk will always return the same instance.
Factor the comparison out to a method, and suppress the warning with an
explanatory comment.
Change-Id: I5a148001dba7749ac15119aed388adb12b6f51ad Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Ivan Frade [Wed, 12 Jun 2019 08:18:10 +0000 (10:18 +0200)]
NetscapeCookieFile: Javadoc fixes
Downstream analyzers reported empty fields in the javadoc. I corrected
few more details:
* Fill empty javadoc fields.
* Use <p> to separate description paragraphs.
* End description paragraphs with a period.
* Remove period at the end of field descriptions.
Change-Id: I749e4b821fc855999caddc442ac788fa514386ea Signed-off-by: Ivan Frade <ifrade@google.com>
Matthias Sohn [Tue, 11 Jun 2019 21:09:19 +0000 (23:09 +0200)]
Merge branch 'master' into stable-5.4
* master:
Handle missing "ours" stage in WorkingTreeIterator.hasCrLfInIndex()
Config: Add helper method to check for empty value
ErrorProne: Increase severity of FutureReturnValueIgnored to ERROR
FS_Win32: Add missing parentheses on if-blocks
Upgrade spotbugs to 3.1.12
Abbreviated{Long}ObjectId: Make operator precedence explicit
GC: Update TODO comments
FS_POSIX: Fix reference comparison of Boolean.FALSE
Increase bazel timeout for long running tests
Use bazelisk to switch between used bazel version
Bump minimum Bazel version to 0.26.1
Bazel: Bump skylib library version to 0.8.0
Retry loading config when locked by another process
Make pull --rebase on an unborn branch do a checkout
Warn if configured cookie file is missing
Handle escaped CR-LF in git config files
DescribeCommand: use glob match instead of path match
Fix off-by-one error in RebaseTodoFile when reading a todo file
Consistently use "!isEmpty()" to detect non-empty list
TransportHttp: Check for non-empty list with "!isEmpty()" rather than
"size() > 0"
TransportHttp: Fix comparison of size with ">= 0"
NetscapeCookieFileTest: Split HttpCookiesMatcher to own class
Bazel: Add missing dependency on mockito for TransportHttpTest
Determine hard-linking and nlink support per FileStore
Support reading and writing cookies.
Repository: Add getIdentifier() method to avoid instanceof operator
Update to Orbit R20190602212107
PacketLineIn: Deprecate the END constant
PacketLineIn: Add an iterator over strings in the input stream
Replace most usages of PacketLineIn.END with PacketLineIn.end()
PacketLineIn: Deprecate DELIM constant
Replace trivial reference comparison of PacketLineIn.{DELIM,END}
PacketLineIn: Rename isDelim to isDelimiter
ProtocolV2ParserTest: Fix typo in comment
Upgrade Bouncy Castle to 1.61
Update to Orbit R20190531194818 and rollback update to Ant 1.10.6
cli: Add the --always option to describe
DescribeCommand: Support the "always" option
cli: Add the --tags option to describe
DescribeCommand: Consistenly omit the default value
Remove excess blank line in FileUtilsTest
PacketLineIn: Add helper methods to check for END and DELIM
UploadPackTest: Rename variable to avoid hiding class member
UploadPackTest: Add missing <> operator on instantiation of ArrayList
BitmapCalculator: javadoc fixes
RevWalkUtils: add progress callback to findBranchesReachableFrom
Upgrade maven-source-plugin to 3.1.0
Upgrade maven-jar-plugin to 3.1.2
Upgrade jacoco-maven-plugin to 0.8.4
BitmapCalculator and its test: add missing license header
RevWalk: new method createReachabilityChecker()
Change-Id: I4d76c7c0dbe6411c842f3468b709f7df51789c08 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Thomas Wolf [Tue, 11 Jun 2019 11:16:18 +0000 (13:16 +0200)]
Handle missing "ours" stage in WorkingTreeIterator.hasCrLfInIndex()
In a delete-modify conflict with the deletion as "ours" there may be
no stage 2 in the index. Add appropriate null checks. Add a new test
for this case, and verify that the file gets added with a single LF
after conflict resolution with core.autocrlf=true. This matches the
behavior of canonical git for this case.
Bug: 547724
Change-Id: I1bafdb83d9b78bf85294c78325e818e72fae53bc Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
The TODO comments say "in 5.0", but 5.0 was already released without
resolving them. Remove "in 5.0" on the assumption that the mentioned
improvements still need to be done at some point.
Change-Id: I3eb429803e2266de3fc490e1f3912991c08aa1ad Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Matthias Sohn [Mon, 10 Jun 2019 21:35:20 +0000 (23:35 +0200)]
Increase bazel timeout for long running tests
EolRepositoryTest and GcCommitSelectionTest timed out frequently when
running unit tests using bazel with the default timeout "moderate"
(300s). Increase timeout of these tests to "long" (900s).
Change-Id: I43588cf950f55b50f868d9fe9c66d22bd428a54c Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Michael Keppler [Sun, 29 Jul 2018 05:53:11 +0000 (07:53 +0200)]
Retry loading config when locked by another process
When loading the config, a FileNotFoundException may occur if the file
exists but cannot be read (see [1]). This is the case on Windows with a
virus scanner checking the file. Therefore if the file exists and that
exception is thrown, retry multiple times, similar to how this was
already implemented for IOException.
Thomas Wolf [Sun, 3 Mar 2019 23:07:40 +0000 (00:07 +0100)]
Make pull --rebase on an unborn branch do a checkout
A merging pull on an unborn branch was already supported. But a
rebasing pull failed. If the user has pull.rebase = true in his
user config, the pull would try to rebase. Rebasing needs a parent
commit, though. Native git handles this case:
Thomas Wolf [Fri, 5 Apr 2019 19:33:14 +0000 (21:33 +0200)]
Handle escaped CR-LF in git config files
Canonical git treats CR-LF in config files as LF.[1][2] JGit does so,
too, except when escaped as a line continuation. Correct this and
treat the sequence \-CR-LF as a line continuation.
Bug: 545850
Change-Id: I51e7378a22c21b3baa3701163c423d04c900af5a Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Thomas Wolf [Sun, 21 Apr 2019 19:23:30 +0000 (21:23 +0200)]
Fix off-by-one error in RebaseTodoFile when reading a todo file
Commit messages of length 1 were not read. 'lineEnd' is the offset
of the last character in the line before the terminating LF or CR-LF,
and 'nextSpace' is actually the offset of the character _after_ the
next space. With a one-character commit message, nextSpace == lineEnd.
The code also assumes the commit message to be optional, but actually
failed in that case because it read beyond the line ending. Fix that,
too.
Add a test case for reading a todo file.
Bug: 546245
Change-Id: I368d63615930ea2398a6230e756442fd88870654 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
projects / jgit.git / log