| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
| |
Change-Id: I75e03ef8630d7a369e97be0f797253b968575354
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
|\
| |
| |
| |
| |
| |
| |
| |
| |
| | |
* stable-5.2:
Prepare 5.1.16-SNAPSHOT builds
JGit v5.1.15.202012011955-r
Fix PackInvalidException when fetch and repack run concurrently
Change-Id: If1d8034b5e0cbc004a11a31b228cb5732efb390d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| |\
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
* stable-5.1:
Prepare 5.1.16-SNAPSHOT builds
JGit v5.1.15.202012011955-r
Fix PackInvalidException when fetch and repack run concurrently
Change-Id: Id83e29e567646a3945a5b817860ea8f7c3e6e5cf
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | |
| | |
| | |
| | |
| | | |
Change-Id: I50e59e1e73a92fa4fe366398fb8141f5e2e289c1
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | |
| | |
| | |
| | |
| | | |
Change-Id: Icb4f04a40ab366cbacbb3fdf0db1748f27277fda
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
We are running several servers with jGit. We need to run repack from
time to time to keep the repos performant. I.e. after push we test how
many small packs are in the repo and when a threshold is reached we run
the repack.
After upgrading jGit version we've found that if someone does the clone
at the time repack is running the clone sometimes (not always) fails
because the repack removes .pack file used by the clone. Server
exception and client error attached.
I've tracked down the cause and it seems to be introduced between jGit
5.2 (which we upgraded from) and 5.3 and being caused by this commit:
Move throw of PackInvalidException outside the catch -
https://github.com/eclipse/jgit/commit/afef866a44cd65fef292c174cad445b3fb526400
The problem is that when the throw was inside of the try block the last
catch block catched the exception and called openFailed(false) method.
It is true that it called it with invalidate = false, which is wrong.
The real problem though is that with the throw outside of the try block
the openFail is not called at all and the fields activeWindows and
activeCopyRawData are not set to 0. Which affects the later called tests
like: if (++activeCopyRawData == 1 && activeWindows == 0).
The fix for this is relatively simple keeping the throw outside of the
try block and still having the invalid field set to true. I did
exhaustive testing of the change running concurrent clones and pushes
indefinitely and with the patch applied it never fails while without the
patch it takes relatively short to get the error.
See: https://www.eclipse.org/lists/jgit-dev/msg04014.html
Bug: 569349
Change-Id: I9dbf8801c8d3131955ad7124f42b62095d96da54
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | |
| | |
| | |
| | |
| | | |
Change-Id: I69f181453c79cef2b4f43fac38d9836917cdc973
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | |
| | |
| | |
| | |
| | | |
Change-Id: I7b128942ef224335f415f867c2d9d5da7498ed8b
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
|\| |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
* stable-5.2:
Prepare 5.1.15-SNAPSHOT builds
JGit v5.1.14.202011251942-r
GC#deleteOrphans: log warning for deleted orphaned files
GC#deleteOrphans: handle failure to list files in pack directory
Ensure that GC#deleteOrphans respects pack lock
Update API warning filters
Remove unused imports
Change-Id: Id9386846a202b5ae98dd602744963f8897ddaa8c
|
| |\|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
* stable-5.1:
Prepare 5.1.15-SNAPSHOT builds
JGit v5.1.14.202011251942-r
GC#deleteOrphans: log warning for deleted orphaned files
GC#deleteOrphans: handle failure to list files in pack directory
Ensure that GC#deleteOrphans respects pack lock
Update API warning filters
Remove unused imports
Change-Id: I91cfe2820c40d2d773cbf018cc2a6c36b062801e
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | |
| | |
| | |
| | |
| | | |
Change-Id: I70246b66c76e865aef4e3adada3a507750ca7c63
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | |
| | |
| | |
| | |
| | | |
Change-Id: Ibe124988be39feaa029c3770777126dd87b18abc
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | |
| | |
| | |
| | | |
Change-Id: Ie245bf5c8c924dfb1f0f40b8bcdcb1e6f5815526
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
- log an error
- either there is no list or it is incomplete hence return immediately
Change-Id: Ieee5378ca06304056b9ccc30c1acd5f52360052d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
If pack or index files are guarded by a pack lock (.keep file)
deleteOrphans() should not touch the respective files protected by the
lock file. Otherwise it may interfere with PackInserter concurrently
inserting a new pack file and its index.
The problem was caused by the following race.
All mentioned files are located in "objects/pack/".
File endings relevant in "pack" dir:
.pack
.keep
.idx
.bitmap
When ReceivePack receives a pack file it executes the following steps:
ReceivePack.service():
receivePackAndCheckConnectivity():
receivePack():
receive the pack
parse the pack, returns packLock (.keep file)
PackInserter.flush():
write tmpPck file: "insert_<random>.pack"
write tmpIdx file: "insert_<random>.idx"
real pack name: "pack-<SHA1>.pack"
real index name: "pack-<SHA1>.idx"
atomic rename tmpPack to realPack
atomic rename tmpIdx to tmpIdx
execute commands
unlock pack by removing .keep file
trigger auto gc if enabled
When PackInserter.flush() renames the temporary pack to the final
"pack-xxx.pack" file the temporary pack index file "insert_xxx.idx"
has no matching .pack file with the same base name for a short interval.
If deleteOrphans() ran during that interval it deduced the pack index
file was orphaned. Subsequently the missing pack index caused
MissingObjectExceptions since objects contained in the pack couldn't be
looked up anymore.
Bug: https://bugs.chromium.org/p/gerrit/issues/detail?id=13544
Change-Id: I559c81e4b1d7c487f92a751bd78b987d32c98719
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | |
| | |
| | |
| | | |
Change-Id: I0fa9c7725ee15da9a932dab0abfb2525d2401149
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | |
| | |
| | |
| | | |
Change-Id: I7c44e3603df2dd368cb7c0ba0072413b887b6903
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | |
| | |
| | |
| | |
| | | |
Change-Id: I64ef3b4cf30e87d058001c2c6f1458f4b0c1df00
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | |
| | |
| | |
| | |
| | | |
Change-Id: I3ba41af516b9d3b74ac580b3e170ad0e6e0663d4
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
|\| |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
* stable-5.2:
Prepare 5.1.14-SNAPSHOT builds
JGit v5.1.13.202002110435-r
Change-Id: I78cbed22c0f0f5872fa44b9a564f6d31dcace582
|
| |\|
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
* stable-5.1:
Prepare 5.1.14-SNAPSHOT builds
JGit v5.1.13.202002110435-r
Change-Id: I57fc818e719bc94f90ab1aeb34016eea74d84719
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | |
| | |
| | |
| | |
| | | |
Change-Id: Ic7d6f1a6c3123af1a1fa782f052b0cea3b6f28c6
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | |
| | |
| | |
| | |
| | | |
Change-Id: Iedc8699ad1a24efe7ddb47ae919c75b9d36141d4
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
|\| |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
* stable-5.2:
Fix string format parameter for invalidRefAdvertisementLine
WindowCache: add metric for cached bytes per repository
pgm daemon: fallback to user and system config if no config specified
WindowCache: add option to use strong refs to reference ByteWindows
Replace usage of ArrayIndexOutOfBoundsException in treewalk
Add config constants for WindowCache configuration options
Change-Id: I790098be00ff3f9b8278d54ae4fb7c11311816e9
|
| |\|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
* stable-5.1:
Fix string format parameter for invalidRefAdvertisementLine
WindowCache: add metric for cached bytes per repository
pgm daemon: fallback to user and system config if no config specified
WindowCache: add option to use strong refs to reference ByteWindows
Change-Id: I741059a1d0d5950ab5bc16ec70352655ee926a24
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
The externalized error message added in f4fc640 ("BasePackConnection:
Check for expected length of ref advertisement", Dec 18, 2019) uses a
malformed string format. Since there is only one formatting argument,
it should be referenced with '{0}' rather than '{1}'.
Change-Id: Ibda864dfb0bb902fe07ae4bba73117b212046e8a
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
Since ObjectDatabase and PackFile don't know their repository use the
packfile's grand-grand-parent directory as an identifier for the
repository the packfile resides in.
Remove metric for a repository if the number of cached bytes for the
repository drops to 0 in order to ensure the map of cached bytes per
repository doesn't contain repositories which have no data cached in the
WindowCache.
Change-Id: I969ab8029db0a292e7585cbb36ca0baa797da20b
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
If a config file is passed via option --config-file then use only the
options defined in that file. This helps to concisely configure the
daemon without side effects from global and system level git configs.
Otherwise fallback to user and system level configs.
Change-Id: I242de248f257579874ad0bfe4882a22502353b1f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
Java GC evicts all SoftReferences when the used heap size comes close to
the maximum heap size. This means peaks in heap memory consumption can
flush the complete WindowCache which was observed to have negative
impact on performance of upload-pack in Gerrit.
Hence add a boolean option core.packedGitUseStrongRefs to allow using
strong references to reference packfile pages cached in the WindowCache.
If this option is set to true Java gc can no longer flush the
WindowCache to free memory if the used heap comes close to the maximum
heap size. On the other hand this provides more predictable performance.
Bug: 553573
Change-Id: I9de406293087ab0fa61130c8e0829775762ece8d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| |\|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
* stable-5.1:
Replace usage of ArrayIndexOutOfBoundsException in treewalk
Add config constants for WindowCache configuration options
Change-Id: I17da3c5183eca536aa2be3972bc5df45c9d75f1b
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
Using exceptions during normal operations - for example with the
desire of expanding an array in the failure case - can have a
severe performance impact. When exceptions are instantiated,
a stack trace is collected. Generating stack trace can be expensive.
Compared to that, checking an array for length - even if done many
times - is cheap since this is a check that can run in just a
handful of CPU cycles.
Change-Id: Ifaf10623f6a876c9faecfa44654c9296315adfcb
Signed-off-by: Patrick Hiesel <hiesel@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | |
| | |
| | |
| | | |
Change-Id: Icc5265f87ae58aa1e667ed1827075c4a30f75c32
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
|\| |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
* stable-5.2:
Fix API problem filters
Fix unclosed resource warning in SmartOutputStream
JschConfigSessionFactory: fix boxing warning
SshSupport#runSshCommand: don't throw exception in finally block
Don't override already managed maven-compiler-plugin version
Remove unused import from CreateFileSnapshotBenchmark
Remove duplicate ignore_optional_problems entry in .classpath
Update maven-site-plugin used by benchmark module to 3.8.2
Add dependency to enable site generation for benchmark module
Ignore warnings for generated source code in
org.eclipse.jgit.benchmark
Fix MBean registration
Enhance WindowCache statistics
Change-Id: I67a07d92718188bdf7f8a13b83e9f538ecf4b22f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| |\|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
* stable-5.1:
Fix unclosed resource warning in SmartOutputStream
JschConfigSessionFactory: fix boxing warning
SshSupport#runSshCommand: don't throw exception in finally block
Don't override already managed maven-compiler-plugin version
Remove unused import from CreateFileSnapshotBenchmark
Remove duplicate ignore_optional_problems entry in .classpath
Update maven-site-plugin used by benchmark module to 3.8.2
Add dependency to enable site generation for benchmark module
Ignore warnings for generated source code in
org.eclipse.jgit.benchmark
Fix MBean registration
Enhance WindowCache statistics
Change-Id: Ic90aacf1ea40e13dc564d4d659e79535e86d0300
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | |
| | |
| | |
| | | |
Change-Id: Ia4b96ae1c2cc9357802487384ee32a80ed40334b
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | |
| | |
| | |
| | | |
Change-Id: I1735033c56b444a9a7160cb7df89292a228d5b34
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
The CommandFailedException which was thrown in finally block is silently
discarded [1]. Refactor this method to throw the exception after the
finally block.
This fixes the warning "Null comparison always yields false: The
variable failure can only be null at this location".
[1] https://wiki.sei.cmu.edu/confluence/display/java/ERR04-J.+Do+not+complete+abruptly+from+a+finally+block
https://docs.oracle.com/javase/specs/jls/se8/html/jls-14.html#jls-14.20.2
Change-Id: Idbfc303d9c9046ab9a43e0d4c6d65d325bdaf0ed
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | |
| | |
| | |
| | | |
Change-Id: Ie2cb178cf8d805aadc76a2096bcdde95a146d07c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | |
| | |
| | |
| | | |
Change-Id: Iad0bcc01ada4252e9ab4f60d4375f98f084f6a5f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | |
| | |
| | |
| | | |
Change-Id: I326377c90af59ecaada7f5185a638726a8e909b7
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
The benchmark module currently has no parent, adjust the version used
here to the one used by all the other jgit Maven modules.
Change-Id: I8807a694fe23f8f131d1d22a58a3e18874d756cc
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | |
| | |
| | |
| | | |
Change-Id: Iae4524ddc730d57993e9c6d6807282e4b07d1336
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
The source code in the folder .apt_generated is generated by the JMH
code generator, so there's no point in raising any warnings as this
could only be fixed in the upstream code generator.
Change-Id: I882888e7bf924f9ae74182598fcb91671a5c9818
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | |
| | |
| | |
| | | |
Change-Id: I6f6b8641f6c3e8ab9f625594085014272305656a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
Add the following statistics
- cache hit count and hit ratio
- cache miss count and miss ratio
- count of successful and failed loads
- rate of failed loads
- load, eviction and request count
- average and total load time
Use LongAdder instead of AtomicLong to implement counters in order to
improve scalability.
Optionally expose these metrics via JMX, they are registered with the
platform MBean server if the config option jmx.WindowCacheStats = true
in the user or system level git config.
Bug: 553573
Change-Id: Ia2d5246ef69b9c2bd594a23934424bc5800774aa
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | |
| | |
| | |
| | | |
Change-Id: Icc78570f949ad64beb58caa192c829e536aa8dad
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
|\| |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
* stable-5.2:
Bazel: Use java_plugin and java_binary from @rules_java in jmh.bzl
Bazel: Add missing newlines at end of BUILD files
Change-Id: Ie5e66f2e26c057c81101b0d110f91ea479eb362d
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
|
| |\|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
* stable-5.1:
Bazel: Use java_plugin and java_binary from @rules_java in jmh.bzl
Bazel: Add missing newlines at end of BUILD files
Change-Id: Ic1dc84a6c4a1cf430d329627642583fd6e0d0eaa
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
|
| | |
| | |
| | |
| | |
| | | |
Change-Id: I80456f74bc17a1ebb0248fce8050a4cb07bf18f1
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
|
| | |
| | |
| | |
| | |
| | | |
Change-Id: I11afc45141da988e28382898246995f6b45eb09c
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
|