| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| ... | |
| | | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
Change-Id: I526ed2a08553bc0b2678aaefaff9e0c6529baefc
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | | | | | | |\
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | | |
* changes:
BatchRefUpdate: Skip saving conflicting ref names and prefixes in memory
BatchRefUpdateTest: Accurately assert RefsChangedEvent(s) fired
Optimize RefDirectory.isNameConflicting()
|
| | | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | | |
Rather than getting all ref names and prefixes and saving them
in memory to perform the check for conflicting names, rely on
RefDirectory.isNameConflicting as it is no longer an expensive
call after it was optimized in Ie994fc.
The old optimization to save ref names and prefixes in memory
was targeted towards making clones faster. With this change,
the clone performance is unaffected when tests were done with
repos containing many(~500k) refs.
Here are few recorded elapsed times for creating 10 branches
using BatchRefUpdate on NFS based repositories with varying
loose refs count. As seen here, this change helps improve the
BatchRefUpdate performance from O(n^2) to O(1).
loose_refs_count with_change without_change
50 241 ms 310 ms
300 263 ms 1502 ms
1k 181 ms 4241 ms
2k 204 ms 6440 ms
9k 158 ms 25930 ms
20k 154 ms 60443 ms
50k 171 ms 135199 ms
110k 157 ms 329450 ms
160k 209 ms 396328 ms
This update improves the Gerrit notedb migration performance
as it uses BatchRefUpdate to write change meta refs similar to
the test performed above.
Change-Id: I853ac6c7feb4b39c3156c01876b38cbd182accfe
Signed-off-by: Kaushik Lingarkar <quic_kaushikl@quicinc.com>
|
| | | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | | |
Update tests to record the number of events fired post-setup and only
assert for events fired during BatchRefUpdate.execute. For tests which
use writeLooseRef to setup refs, create new tests which assert the
number of RefsChangedEvent(s) rather than updating the existing ones
to call RefDirectory.exactRef as it changes the code path.
Change-Id: I0187811628d179d9c7e874c9bb8a7ddb44dd9df4
Signed-off-by: Kaushik Lingarkar <quic_kaushikl@quicinc.com>
|
| | | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | | |
Avoid having to scan over ALL loose refs to determine if the
name is nested within or is a container of an existing reference.
This can get really expensive if there are too many loose refs.
Instead use exactRef and getRefsByPrefix which scan based on a
prefix.
With a simple shell script(like below) using jgit client to create
1k refs in a new repository on NFS, this change brings down the time
from 12mins to 7mins.
for ref in $(seq 1 1000); do
jgit branch "$ref"
done
Here are few recorded elapsed times to create a new branch on NFS
based repositories with varying loose refs count. As we see here,
this change improves the name conflicting check from O(n^2) to O(1).
loose_refs_count with_change without_change
50 44 ms 164 ms
300 45 ms 1193 ms
1k 38 ms 2610 ms
2k 44 ms 6003 ms
9k 46 ms 27860 ms
20k 45 ms 48591 ms
50k 51 ms 135471 ms
110k 43 ms 294252 ms
160k 52 ms 430976 ms
Change-Id: Ie994fc184b8f82811bfb37b111eb9733dbe3e6e0
Signed-off-by: Kaushik Lingarkar <quic_kaushikl@quicinc.com>
|
| | | | | | | |/
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
- bazlets need to be updated to react on Maven central no longer
supporting http protocol but only https
- update bazel to 2.0
Change-Id: I07f5f050f3b1db2014a5198a28b6bbf893434814
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
(cherry picked from commit d74daad1e064da9614c5ca2c0138c8046cfb0829)
|
| |\| | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
* stable-5.6:
Remove texts which were added by mistake in 00386272
Fix formatting which was broken in 00386272
Change-Id: I45d444b360485564744bf3dfad2c2f5a5e7fcdf6
|
| | | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
Change-Id: Iaed25dac0bc9af8f3fda6138a5f9fe553bff5d39
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
Change-Id: I10a3e2b117e790f64386a8e9e7663db8e59230d9
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| |\| | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
* stable-5.6:
LockFile: create OutputStream only when needed
Change-Id: I7c0e37d2cee0923662a7e39df5a802a84c017e4f
|
| | | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
Don't create the stream eagerly in lock(); that may cause JGit to
exceed OS or JVM limits on open file descriptors if many locks need
to be created, for instance when creating many refs. Instead create
the output stream only when one really needs to write something.
Bug: 573328
Change-Id: If9441ed40494d46f594a896d34a5c4f56f91ebf4
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
|
| | | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
In a distributed setting, one can have multiple datacenters use
reftables for serving, while the ground truth for the Ref database is
administered centrally. In this setting, replication delays combined
with compaction can cause update-index ranges to overlap.
Such a setting is used at Google, and the JGit code already handles
this correctly (modulo a bugfix that applied in change I8f8215b99a).
Remove the restriction that was applied at FileReftableDatabase.
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Change-Id: I6f9ed0fbd7fbc5220083ab808b22a909215f13a9
|
| |\| | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
* stable-5.6:
Prepare 5.3.10-SNAPSHOT builds
JGit v5.3.9.202012012026-r
Prepare 5.1.16-SNAPSHOT builds
JGit v5.1.15.202012011955-r
Fix PackInvalidException when fetch and repack run concurrently
Change-Id: If5f001c414e677bb2bac59714421f0191b23ea1f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | |\| | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
* stable-5.5:
Prepare 5.3.10-SNAPSHOT builds
JGit v5.3.9.202012012026-r
Prepare 5.1.16-SNAPSHOT builds
JGit v5.1.15.202012011955-r
Fix PackInvalidException when fetch and repack run concurrently
Change-Id: I986029816ef66fbfae1a59bd97179392320a485c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | | |\| | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
* stable-5.4:
Prepare 5.3.10-SNAPSHOT builds
JGit v5.3.9.202012012026-r
Prepare 5.1.16-SNAPSHOT builds
JGit v5.1.15.202012011955-r
Fix PackInvalidException when fetch and repack run concurrently
Change-Id: Iede8686198332d6271771bef6eb00c25f1103979
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | | | |\| | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
* stable-5.3:
Prepare 5.3.10-SNAPSHOT builds
JGit v5.3.9.202012012026-r
Prepare 5.1.16-SNAPSHOT builds
JGit v5.1.15.202012011955-r
Fix PackInvalidException when fetch and repack run concurrently
Change-Id: I319b54ceffe095add60420c6ae83eac0ba9c14b6
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
Change-Id: I6f131ad04574bd0d569ae6a59c29ea987be0efb2
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
Change-Id: I75e03ef8630d7a369e97be0f797253b968575354
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | | | | |\| |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
* stable-5.2:
Prepare 5.1.16-SNAPSHOT builds
JGit v5.1.15.202012011955-r
Fix PackInvalidException when fetch and repack run concurrently
Change-Id: If1d8034b5e0cbc004a11a31b228cb5732efb390d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | | | | | |\|
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
* stable-5.1:
Prepare 5.1.16-SNAPSHOT builds
JGit v5.1.15.202012011955-r
Fix PackInvalidException when fetch and repack run concurrently
Change-Id: Id83e29e567646a3945a5b817860ea8f7c3e6e5cf
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
Change-Id: I50e59e1e73a92fa4fe366398fb8141f5e2e289c1
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
Change-Id: Icb4f04a40ab366cbacbb3fdf0db1748f27277fda
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
We are running several servers with jGit. We need to run repack from
time to time to keep the repos performant. I.e. after push we test how
many small packs are in the repo and when a threshold is reached we run
the repack.
After upgrading jGit version we've found that if someone does the clone
at the time repack is running the clone sometimes (not always) fails
because the repack removes .pack file used by the clone. Server
exception and client error attached.
I've tracked down the cause and it seems to be introduced between jGit
5.2 (which we upgraded from) and 5.3 and being caused by this commit:
Move throw of PackInvalidException outside the catch -
https://github.com/eclipse/jgit/commit/afef866a44cd65fef292c174cad445b3fb526400
The problem is that when the throw was inside of the try block the last
catch block catched the exception and called openFailed(false) method.
It is true that it called it with invalidate = false, which is wrong.
The real problem though is that with the throw outside of the try block
the openFail is not called at all and the fields activeWindows and
activeCopyRawData are not set to 0. Which affects the later called tests
like: if (++activeCopyRawData == 1 && activeWindows == 0).
The fix for this is relatively simple keeping the throw outside of the
try block and still having the invalid field set to true. I did
exhaustive testing of the change running concurrent clones and pushes
indefinitely and with the patch applied it never fails while without the
patch it takes relatively short to get the error.
See: https://www.eclipse.org/lists/jgit-dev/msg04014.html
Bug: 569349
Change-Id: I9dbf8801c8d3131955ad7124f42b62095d96da54
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| |\| | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
* stable-5.6:
Remove unused imports
Silence API warnings
Remove erraneously merged source features
Change-Id: Iad4da36d82e8a4dad965c0de52910a4ce4e2d2da
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | |\| | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
* stable-5.5:
Remove unused imports
Silence API warnings
Remove erraneously merged source features
Change-Id: Ifb6ebe56577719c0dfc6425557130b91e510d5c7
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | | |\| | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
* stable-5.4:
Remove unused imports
Silence API warnings
Remove erraneously merged source features
Change-Id: I25eff574e4c1cd0aef7cf2c685aade2f87fb0409
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
Change-Id: Id78f4fd7d11a8908f41160a2ab9f3d81b7f771da
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
Change-Id: I5de476935e79f8f0beefc27885cbc3ffe31c0aed
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
In 5.4 source features were combined into a single feature.
Change-Id: I4055ec805bb036d0f445c8b9b0bb670495a32cc0
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| |\| | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
* stable-5.6:
Prepare 5.3.9-SNAPSHOT builds
JGit v5.3.8.202011260953-r
Prepare 5.1.15-SNAPSHOT builds
JGit v5.1.14.202011251942-r
GC#deleteOrphans: log warning for deleted orphaned files
GC#deleteOrphans: handle failure to list files in pack directory
Ensure that GC#deleteOrphans respects pack lock
Update API warning filters
Remove unused imports
Change-Id: Ie24d381f295cccfb99068c7ed5817179da29c1db
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | |\| | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
* stable-5.5:
Prepare 5.3.9-SNAPSHOT builds
JGit v5.3.8.202011260953-r
Prepare 5.1.15-SNAPSHOT builds
JGit v5.1.14.202011251942-r
GC#deleteOrphans: log warning for deleted orphaned files
GC#deleteOrphans: handle failure to list files in pack directory
Ensure that GC#deleteOrphans respects pack lock
Update API warning filters
Remove unused imports
Change-Id: I00c8ddad0059a20a978743dfb7ad1b513dc7fef6
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | | |\| | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
* stable-5.4:
Prepare 5.3.9-SNAPSHOT builds
JGit v5.3.8.202011260953-r
Prepare 5.1.15-SNAPSHOT builds
JGit v5.1.14.202011251942-r
GC#deleteOrphans: log warning for deleted orphaned files
GC#deleteOrphans: handle failure to list files in pack directory
Ensure that GC#deleteOrphans respects pack lock
Update API warning filters
Remove unused imports
Change-Id: Icc34f809a3bb019d8d640b9bdb71363e617942e2
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | | | |\| | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
* stable-5.3:
Prepare 5.3.9-SNAPSHOT builds
JGit v5.3.8.202011260953-r
Prepare 5.1.15-SNAPSHOT builds
JGit v5.1.14.202011251942-r
GC#deleteOrphans: log warning for deleted orphaned files
GC#deleteOrphans: handle failure to list files in pack directory
Ensure that GC#deleteOrphans respects pack lock
Update API warning filters
Remove unused imports
Change-Id: I82576ad0f61cf3ff11b54691b32666c61401ad9b
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
Change-Id: I69f181453c79cef2b4f43fac38d9836917cdc973
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
Change-Id: I7b128942ef224335f415f867c2d9d5da7498ed8b
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | | | | |\| |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
* stable-5.2:
Prepare 5.1.15-SNAPSHOT builds
JGit v5.1.14.202011251942-r
GC#deleteOrphans: log warning for deleted orphaned files
GC#deleteOrphans: handle failure to list files in pack directory
Ensure that GC#deleteOrphans respects pack lock
Update API warning filters
Remove unused imports
Change-Id: Id9386846a202b5ae98dd602744963f8897ddaa8c
|
| | | | | | |\|
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
* stable-5.1:
Prepare 5.1.15-SNAPSHOT builds
JGit v5.1.14.202011251942-r
GC#deleteOrphans: log warning for deleted orphaned files
GC#deleteOrphans: handle failure to list files in pack directory
Ensure that GC#deleteOrphans respects pack lock
Update API warning filters
Remove unused imports
Change-Id: I91cfe2820c40d2d773cbf018cc2a6c36b062801e
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
Change-Id: I70246b66c76e865aef4e3adada3a507750ca7c63
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
Change-Id: Ibe124988be39feaa029c3770777126dd87b18abc
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
Change-Id: Ie245bf5c8c924dfb1f0f40b8bcdcb1e6f5815526
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
- log an error
- either there is no list or it is incomplete hence return immediately
Change-Id: Ieee5378ca06304056b9ccc30c1acd5f52360052d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
If pack or index files are guarded by a pack lock (.keep file)
deleteOrphans() should not touch the respective files protected by the
lock file. Otherwise it may interfere with PackInserter concurrently
inserting a new pack file and its index.
The problem was caused by the following race.
All mentioned files are located in "objects/pack/".
File endings relevant in "pack" dir:
.pack
.keep
.idx
.bitmap
When ReceivePack receives a pack file it executes the following steps:
ReceivePack.service():
receivePackAndCheckConnectivity():
receivePack():
receive the pack
parse the pack, returns packLock (.keep file)
PackInserter.flush():
write tmpPck file: "insert_<random>.pack"
write tmpIdx file: "insert_<random>.idx"
real pack name: "pack-<SHA1>.pack"
real index name: "pack-<SHA1>.idx"
atomic rename tmpPack to realPack
atomic rename tmpIdx to tmpIdx
execute commands
unlock pack by removing .keep file
trigger auto gc if enabled
When PackInserter.flush() renames the temporary pack to the final
"pack-xxx.pack" file the temporary pack index file "insert_xxx.idx"
has no matching .pack file with the same base name for a short interval.
If deleteOrphans() ran during that interval it deduced the pack index
file was orphaned. Subsequently the missing pack index caused
MissingObjectExceptions since objects contained in the pack couldn't be
looked up anymore.
Bug: https://bugs.chromium.org/p/gerrit/issues/detail?id=13544
Change-Id: I559c81e4b1d7c487f92a751bd78b987d32c98719
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
Change-Id: I0fa9c7725ee15da9a932dab0abfb2525d2401149
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
Change-Id: I7c44e3603df2dd368cb7c0ba0072413b887b6903
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
| | | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
Change-Id: Icc569aeefdc79baee5dfb71fb34d881c561dcf52
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
|
| | | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
ObjectWritingException and FileNotFoundException are subclasses
of IOException, which is already declared. Error does not need
to be explicitly declared.
Change-Id: I879820a33e10ec3a7ef676adc9c9148d2b3c4b27
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
|
| | | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
- The code to move the file is repeated. Split it out into a
utility method.
- Remove the catch block for AtomicMoveNotSupportedException which
is redundant because it's handled in exactly the same way as the
IOException further down. The only exception we need to explicitly
handle differently in this block is NoSuchFileException.
- Improve the comments.
Change-Id: Ifc5490953ffb25ecd1c48a06289eccb3f19910c6
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
|
| | | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
On the first attempt to move the temp file, NoSuchFileException can
be raised if the destination folder does not exist. Instead of handling
this implicitly in the catch of IOException and then continuing to
create the destination folder and try again, explicitly catch it and
create the destination folder. If any other IOException occurs, treat
it as an unexpected error and return FAILURE.
Subsequently, on the second attempt to move the temp file, if ANY kind
of IOException occurs, also consider this an unexpected error and
return FAILURE.
In both catch blocks for IOException, add logging at ERROR level.
Change-Id: I9de9ee3d2b368be36e02ee1c0daf8e844f7e46c8
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
|
| | | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
If atomic move is not supported, AtomicMoveNotSupportedException will
be thrown on the first attempt to move the temp file. There is no
point attempting the move operation a second time because it will only
fail for the same reason.
Add an immediate return of FAILURE on the first occasion. Remove the
unnecessary handling of the exception in the second block.
Change-Id: I4658a8b37cfec2d7ef0217c8346e512968d0964c
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
|
| |\| | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
* stable-5.6:
Bump Bazel version to 3.1.0
Change-Id: I8a9f39370ab506d9ff5763ed2d05f0d970bbd10b
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
|
