summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* Prepare 5.1.16-SNAPSHOT buildsMatthias Sohn2020-12-0257-360/+360
| | | | | Change-Id: I50e59e1e73a92fa4fe366398fb8141f5e2e289c1 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* JGit v5.1.15.202012011955-rv5.1.15.202012011955-rMatthias Sohn2020-12-0257-60/+60
| | | | | Change-Id: Icb4f04a40ab366cbacbb3fdf0db1748f27277fda Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* Fix PackInvalidException when fetch and repack run concurrentlyPetr Hrebejk2020-12-021-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We are running several servers with jGit. We need to run repack from time to time to keep the repos performant. I.e. after push we test how many small packs are in the repo and when a threshold is reached we run the repack. After upgrading jGit version we've found that if someone does the clone at the time repack is running the clone sometimes (not always) fails because the repack removes .pack file used by the clone. Server exception and client error attached. I've tracked down the cause and it seems to be introduced between jGit 5.2 (which we upgraded from) and 5.3 and being caused by this commit: Move throw of PackInvalidException outside the catch - https://github.com/eclipse/jgit/commit/afef866a44cd65fef292c174cad445b3fb526400 The problem is that when the throw was inside of the try block the last catch block catched the exception and called openFailed(false) method. It is true that it called it with invalidate = false, which is wrong. The real problem though is that with the throw outside of the try block the openFail is not called at all and the fields activeWindows and activeCopyRawData are not set to 0. Which affects the later called tests like: if (++activeCopyRawData == 1 && activeWindows == 0). The fix for this is relatively simple keeping the throw outside of the try block and still having the invalid field set to true. I did exhaustive testing of the change running concurrent clones and pushes indefinitely and with the patch applied it never fails while without the patch it takes relatively short to get the error. See: https://www.eclipse.org/lists/jgit-dev/msg04014.html Bug: 569349 Change-Id: I9dbf8801c8d3131955ad7124f42b62095d96da54 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* Prepare 5.1.15-SNAPSHOT buildsMatthias Sohn2020-11-2657-360/+360
| | | | | Change-Id: I70246b66c76e865aef4e3adada3a507750ca7c63 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* JGit v5.1.14.202011251942-rv5.1.14.202011251942-rMatthias Sohn2020-11-2657-60/+60
| | | | | Change-Id: Ibe124988be39feaa029c3770777126dd87b18abc Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* GC#deleteOrphans: log warning for deleted orphaned filesMatthias Sohn2020-11-263-1/+5
| | | | Change-Id: Ie245bf5c8c924dfb1f0f40b8bcdcb1e6f5815526 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* GC#deleteOrphans: handle failure to list files in pack directoryMatthias Sohn2020-11-261-2/+3
| | | | | | | - log an error - either there is no list or it is incomplete hence return immediately Change-Id: Ieee5378ca06304056b9ccc30c1acd5f52360052d Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* Ensure that GC#deleteOrphans respects pack lockMatthias Sohn2020-11-262-4/+29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If pack or index files are guarded by a pack lock (.keep file) deleteOrphans() should not touch the respective files protected by the lock file. Otherwise it may interfere with PackInserter concurrently inserting a new pack file and its index. The problem was caused by the following race. All mentioned files are located in "objects/pack/". File endings relevant in "pack" dir: .pack .keep .idx .bitmap When ReceivePack receives a pack file it executes the following steps: ReceivePack.service(): receivePackAndCheckConnectivity(): receivePack(): receive the pack parse the pack, returns packLock (.keep file) PackInserter.flush(): write tmpPck file: "insert_<random>.pack" write tmpIdx file: "insert_<random>.idx" real pack name: "pack-<SHA1>.pack" real index name: "pack-<SHA1>.idx" atomic rename tmpPack to realPack atomic rename tmpIdx to tmpIdx execute commands unlock pack by removing .keep file trigger auto gc if enabled When PackInserter.flush() renames the temporary pack to the final "pack-xxx.pack" file the temporary pack index file "insert_xxx.idx" has no matching .pack file with the same base name for a short interval. If deleteOrphans() ran during that interval it deduced the pack index file was orphaned. Subsequently the missing pack index caused MissingObjectExceptions since objects contained in the pack couldn't be looked up anymore. Bug: https://bugs.chromium.org/p/gerrit/issues/detail?id=13544 Change-Id: I559c81e4b1d7c487f92a751bd78b987d32c98719 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* Update API warning filtersMatthias Sohn2020-07-191-0/+28
| | | | Change-Id: I0fa9c7725ee15da9a932dab0abfb2525d2401149 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* Remove unused importsMatthias Sohn2020-07-193-4/+0
| | | | Change-Id: I7c44e3603df2dd368cb7c0ba0072413b887b6903 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* Prepare 5.1.14-SNAPSHOT buildsMatthias Sohn2020-02-1157-360/+360
| | | | | Change-Id: Ic7d6f1a6c3123af1a1fa782f052b0cea3b6f28c6 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* JGit v5.1.13.202002110435-rv5.1.13.202002110435-rMatthias Sohn2020-02-1157-60/+60
| | | | | Change-Id: Iedc8699ad1a24efe7ddb47ae919c75b9d36141d4 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* Fix string format parameter for invalidRefAdvertisementLineDavid Pursehouse2020-01-311-1/+1
| | | | | | | | | | The externalized error message added in f4fc640 ("BasePackConnection: Check for expected length of ref advertisement", Dec 18, 2019) uses a malformed string format. Since there is only one formatting argument, it should be referenced with '{0}' rather than '{1}'. Change-Id: Ibda864dfb0bb902fe07ae4bba73117b212046e8a Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* WindowCache: add metric for cached bytes per repositoryMatthias Sohn2020-01-282-10/+52
| | | | | | | | | | | | | Since ObjectDatabase and PackFile don't know their repository use the packfile's grand-grand-parent directory as an identifier for the repository the packfile resides in. Remove metric for a repository if the number of cached bytes for the repository drops to 0 in order to ensure the map of cached bytes per repository doesn't contain repositories which have no data cached in the WindowCache. Change-Id: I969ab8029db0a292e7585cbb36ca0baa797da20b Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* pgm daemon: fallback to user and system config if no config specifiedMatthias Sohn2020-01-281-7/+22
| | | | | | | | | | If a config file is passed via option --config-file then use only the options defined in that file. This helps to concisely configure the daemon without side effects from global and system level git configs. Otherwise fallback to user and system level configs. Change-Id: I242de248f257579874ad0bfe4882a22502353b1f Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* WindowCache: add option to use strong refs to reference ByteWindowsMatthias Sohn2020-01-205-70/+379
| | | | | | | | | | | | | | | | | Java GC evicts all SoftReferences when the used heap size comes close to the maximum heap size. This means peaks in heap memory consumption can flush the complete WindowCache which was observed to have negative impact on performance of upload-pack in Gerrit. Hence add a boolean option core.packedGitUseStrongRefs to allow using strong references to reference packfile pages cached in the WindowCache. If this option is set to true Java gc can no longer flush the WindowCache to free memory if the used heap comes close to the maximum heap size. On the other hand this provides more predictable performance. Bug: 553573 Change-Id: I9de406293087ab0fa61130c8e0829775762ece8d Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* Replace usage of ArrayIndexOutOfBoundsException in treewalkPatrick Hiesel2020-01-092-9/+6
| | | | | | | | | | | | | | Using exceptions during normal operations - for example with the desire of expanding an array in the failure case - can have a severe performance impact. When exceptions are instantiated, a stack trace is collected. Generating stack trace can be expensive. Compared to that, checking an array for length - even if done many times - is cheap since this is a check that can run in just a handful of CPU cycles. Change-Id: Ifaf10623f6a876c9faecfa44654c9296315adfcb Signed-off-by: Patrick Hiesel <hiesel@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* Add config constants for WindowCache configuration optionsMatthias Sohn2020-01-093-12/+82
| | | | Change-Id: Icc5265f87ae58aa1e667ed1827075c4a30f75c32 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* Fix unclosed resource warning in SmartOutputStreamMatthias Sohn2020-01-061-14/+16
| | | | Change-Id: Ia4b96ae1c2cc9357802487384ee32a80ed40334b Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* JschConfigSessionFactory: fix boxing warningMatthias Sohn2020-01-061-1/+2
| | | | Change-Id: I1735033c56b444a9a7160cb7df89292a228d5b34 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* SshSupport#runSshCommand: don't throw exception in finally blockMatthias Sohn2020-01-061-6/+8
| | | | | | | | | | | | | | | The CommandFailedException which was thrown in finally block is silently discarded [1]. Refactor this method to throw the exception after the finally block. This fixes the warning "Null comparison always yields false: The variable failure can only be null at this location". [1] https://wiki.sei.cmu.edu/confluence/display/java/ERR04-J.+Do+not+complete+abruptly+from+a+finally+block https://docs.oracle.com/javase/specs/jls/se8/html/jls-14.html#jls-14.20.2 Change-Id: Idbfc303d9c9046ab9a43e0d4c6d65d325bdaf0ed Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* Don't override already managed maven-compiler-plugin versionMatthias Sohn2020-01-041-2/+0
| | | | Change-Id: Ie2cb178cf8d805aadc76a2096bcdde95a146d07c Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* Remove unused import from CreateFileSnapshotBenchmarkMatthias Sohn2020-01-041-1/+0
| | | | Change-Id: Iad0bcc01ada4252e9ab4f60d4375f98f084f6a5f Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* Remove duplicate ignore_optional_problems entry in .classpathMatthias Sohn2020-01-041-1/+0
| | | | Change-Id: I326377c90af59ecaada7f5185a638726a8e909b7 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* Update maven-site-plugin used by benchmark module to 3.8.2Matthias Sohn2019-12-181-1/+1
| | | | | | | The benchmark module currently has no parent, adjust the version used here to the one used by all the other jgit Maven modules. Change-Id: I8807a694fe23f8f131d1d22a58a3e18874d756cc Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* Add dependency to enable site generation for benchmark moduleMatthias Sohn2019-12-181-0/+27
| | | | Change-Id: Iae4524ddc730d57993e9c6d6807282e4b07d1336 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* Ignore warnings for generated source code in org.eclipse.jgit.benchmarkMatthias Sohn2019-12-171-0/+1
| | | | | | | | The source code in the folder .apt_generated is generated by the JMH code generator, so there's no point in raising any warnings as this could only be fixed in the upstream code generator. Change-Id: I882888e7bf924f9ae74182598fcb91671a5c9818 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* Fix MBean registrationMatthias Sohn2019-12-132-7/+14
| | | | Change-Id: I6f6b8641f6c3e8ab9f625594085014272305656a Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* Enhance WindowCache statisticsMatthias Sohn2019-12-086-41/+547
| | | | | | | | | | | | | | | | | | | | | Add the following statistics - cache hit count and hit ratio - cache miss count and miss ratio - count of successful and failed loads - rate of failed loads - load, eviction and request count - average and total load time Use LongAdder instead of AtomicLong to implement counters in order to improve scalability. Optionally expose these metrics via JMX, they are registered with the platform MBean server if the config option jmx.WindowCacheStats = true in the user or system level git config. Bug: 553573 Change-Id: Ia2d5246ef69b9c2bd594a23934424bc5800774aa Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* Bazel: Use java_plugin and java_binary from @rules_java in jmh.bzlDavid Pursehouse2019-11-261-3/+5
| | | | | Change-Id: I80456f74bc17a1ebb0248fce8050a4cb07bf18f1 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* Bazel: Add missing newlines at end of BUILD filesDavid Pursehouse2019-11-262-2/+2
| | | | | Change-Id: I11afc45141da988e28382898246995f6b45eb09c Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* Fix benchmark pom.xml to include distribution managment configMatthias Sohn2019-11-061-146/+164
| | | | | | | Also fix indentation. Change-Id: I6d223c493d537bb8aac23c7ddf0a602aff22a3c2 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* Run JMH benchmarks using bazelMatthias Sohn2019-11-054-0/+121
| | | | | | | | The benchmarks can be built and run using bazel by running: $ bazel run //org.eclipse.jgit.benchmarks:benchmarks Change-Id: I6679750eaa0f2be30ed9d45036e013b0ea4fcc86 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* Benchmark for creating files and FileSnapshotsMatthias Sohn2019-11-051-0/+111
| | | | Change-Id: I953fd8b6b9743b0b9cada244331253eb2121784d Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* Implement benchmark for looking up FileStore of a given PathMatthias Sohn2019-11-051-0/+97
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Results on Mac OS 10.14.6 using APFS: Result "org.eclipse.jgit.benchmarks.LookupFileStoreBenchmark.testLookupFileStore": 44583.277 ±(99.9%) 2666.096 ns/op [Average] (min, avg, max) = (43707.073, 44583.277, 45320.685), stdev = 692.377 CI (99.9%): [41917.181, 47249.373] (assumes normal distribution) Secondary result "org.eclipse.jgit.benchmarks.LookupFileStoreBenchmark.testLookupFileStore:·stack": Stack profiler: ....[Thread state distributions]........................................................ 100.0% RUNNABLE ....[Thread state: RUNNABLE]............................................................. 42.5% 42.5% sun.nio.fs.UnixNativeDispatcher.realpath0 37.9% 37.9% sun.nio.fs.UnixNativeDispatcher.stat0 11.0% 11.0% sun.nio.fs.BsdNativeDispatcher.getfsstat 7.6% 7.6% sun.nio.fs.BsdNativeDispatcher.fsstatEntry 0.7% 0.7% sun.nio.fs.BsdNativeDispatcher.endfsstat 0.1% 0.1% sun.nio.fs.UnixPath.initOffsets 0.0% 0.0% sun.nio.fs.UnixFileAttributes.get 0.0% 0.0% java.util.zip.Inflater.inflateBytes 0.0% 0.0% sun.misc.Unsafe.compareAndSwapInt 0.0% 0.0% sun.nio.fs.BsdFileStore.findMountEntry This shows that FS.attrCacheByPath should be useful to reduce overhead of looking up the FileStore of a given file. Change-Id: I7213086b42e0453f0ee149660d507dac2a4644cc Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* JMH benchmark for SimpleLruCacheMatthias Sohn2019-11-0517-0/+993
| | | | | | | | | | | | | | | | | | | | | | | | See [1] for JMH documentation and [2] how to use JMH in Eclipse. The benchmarks pom currently cannot use the JGit parent pom due to an ecj bug [3] regarding annotation processing. Hence for now do not inherit from the JGit parent pom and copy the compiler plugin configuration for javac from the parent pom. After running the Maven build the benchmark can be run using Maven: $ java -jar org.eclipse.jgit.benchmarks/target/benchmarks.jar or in Eclipse by running the main method of the SimpleLruCacheBenchmark class. [1] https://openjdk.java.net/projects/code-tools/jmh/ [2] http://alblue.bandlem.com/Page/3/index.html [3] https://eclip.se/532029 CQ: 20517 CQ: 20518 Change-Id: Idca8a9e0980f0b8a9c741c4c9e97d03c62f07c8d Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* Remove unused API problem filtersMatthias Sohn2019-11-041-28/+0
| | | | Change-Id: I0ae0fd4919b7ccc5218c6eb0fab22350bb9c9714 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* Silence API errors for new API added since 5.1.0Matthias Sohn2019-11-043-0/+27
| | | | Change-Id: I92a7c1b0319d61aeb25f58048bd07c144068c3db Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* Bazel: Bump bazel version to 1.0.0David Pursehouse2019-10-111-1/+1
| | | | | Change-Id: I607915af0556efe8b2fb640e9387029b755d1a7f Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* Prepare 5.1.13-SNAPSHOT buildsMatthias Sohn2019-10-0256-359/+359
| | | | | Change-Id: Ic1dca14924f49ad07eb5cd0570ce7ece9f319d0d Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* JGit v5.1.12.201910011832-rv5.1.12.201910011832-rMatthias Sohn2019-10-0256-59/+59
| | | | | Change-Id: I69fef9b2cdc18bbf1c8b9b290fb3d190684be13c Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* Do not rely on ArrayIndexOutOfBoundsException to detect end of inputSaša Živkov2019-09-271-4/+2
| | | | | | | | | | | | In the Config#StringReader we relied on ArrayIndexOutOfBoundsException to detect the end of the input. Creation of exception with (deep) stack trace can significantly degrade performance in case when we read thousands of config files, like in the case when Gerrit reads all external ids from the NoteDb. Use the buf.length to detect the end of the input. Change-Id: I12266f25751373a870ce3fa623cf2a95d882d521
* WorkingTreeIterator: handle different timestamp resolutionsThomas Wolf2019-09-173-18/+307
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Older JGit stored only milliseconds timestamps in the index. Newer JGit may get finer timestamps from the file system. This leads to slow index diffs when a new JGit runs against an index produced by older JGit because many timestamps will differ and JGit will then do many content checks. See [1]. Handle this migration case by only comparing milliseconds if the index entry has only millisecond precision. The inverse may also occur; also compare only milliseconds if the file timestamp has only millisecond precision. Do the same also for microsecond resolution. On Windows, NTFS may provide 100ns resolution and may be used by external programs writing the index, but Java's WindowsFileAttributes may provide only microseconds. File timestamp precision in Java depends not only on the Java APIs used by different JGit versions but may also change when running the same Java code on different VMs. And of course the resolution may vary among operating and file systems. Moreover, timestamp precision in the index depends on the program that wrote the index. Canonical git may use a different resolution, maybe even different between git versions. [1] https://www.eclipse.org/forums/index.php/t/1100344/ Change-Id: Idfd08606c883cb98787b2138f9baf0cc89a57b56 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* Fix WorkingTreeIterator.compareMetadata() for CheckStat.MINIMALMatthias Sohn2019-09-101-2/+3
| | | | | | | | | | | | | | | | | | | If CheckStat is MINIMAL or timestamps have no nanosecond part WorkingTreeIterator.compareMetaData only checks the second part of timestamps and ignores nanoseconds which may have ended up in the index by using native git. If fileLastModified.getEpochSecond() == cacheLastModified.getEpochSecond() we currently proceed comparing fileLastModified and cacheLastModified with full precision which is wrong since we determined that we detected reduced timestamp resolution. Fix this and also handle smudged index entries for CheckStat.MINIMAL. Change-Id: I6149885903ac63d79b42d234cc02aa4e19578f3c Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* Format BUILD files with buildifierDavid Pursehouse2019-09-084-5/+5
| | | | | | | | | Using buildifier from master branch and the command: $ buildifier -r -lint fix -warnings all . Change-Id: I19c8ff183081093cb73bed7221a78a91b6cba4dc Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* Bazel: Fix warning about deprecated lib.bzlDavid Pursehouse2019-09-071-1/+1
| | | | | Change-Id: Ifdbaaf9ec020a621070eea351f791a819d554012 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* Format lib/BUILD with buildifierDavid Pursehouse2019-09-071-2/+2
| | | | | Change-Id: I4f0ad64371b160b67797f49b9fde521e5dea9396 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* Bazel: Add fixes for --incompatible_load_java_rules_from_bzlDavid Pursehouse2019-09-0716-0/+28
| | | | | Change-Id: I24746335db132bf20f400cb3db400737596d4542 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* Bazel: Bump minimum supported version to 0.29.0David Ostrovsky2019-09-072-2/+2
| | | | | Change-Id: I069e56deb964624791bb0aa3ae095629adfe3bbb Signed-off-by: David Ostrovsky <david@ostrovsky.org>
* Bazel: Bump skylib library version to 0.8.0David Ostrovsky2019-09-071-3/+3
| | | | | Change-Id: I2089899c9f197b2c93b93390fdd55f6fe11e5ce4 Signed-off-by: David Ostrovsky <david@ostrovsky.org>
generated by cgit v1.2.3 (git 2.39.1) at 2025-01-10 07:13:52 +0000