GC#deleteOrphans: handle failure to list files in pack directory
- log an error
- either there is no list or it is incomplete hence return immediately
Change-Id: Ieee5378ca06304056b9ccc30c1acd5f52360052d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
If pack or index files are guarded by a pack lock (.keep file)
deleteOrphans() should not touch the respective files protected by the
lock file. Otherwise it may interfere with PackInserter concurrently
inserting a new pack file and its index.
The problem was caused by the following race.
All mentioned files are located in "objects/pack/".
File endings relevant in "pack" dir:
.pack
.keep
.idx
.bitmap
When ReceivePack receives a pack file it executes the following steps:
ReceivePack.service():
receivePackAndCheckConnectivity():
receivePack():
receive the pack
parse the pack, returns packLock (.keep file)
PackInserter.flush():
write tmpPck file: "insert_<random>.pack"
write tmpIdx file: "insert_<random>.idx"
real pack name: "pack-<SHA1>.pack"
real index name: "pack-<SHA1>.idx"
atomic rename tmpPack to realPack
atomic rename tmpIdx to tmpIdx
execute commands
unlock pack by removing .keep file
trigger auto gc if enabled
When PackInserter.flush() renames the temporary pack to the final
"pack-xxx.pack" file the temporary pack index file "insert_xxx.idx"
has no matching .pack file with the same base name for a short interval.
If deleteOrphans() ran during that interval it deduced the pack index
file was orphaned. Subsequently the missing pack index caused
MissingObjectExceptions since objects contained in the pack couldn't be
looked up anymore.
Bug: https://bugs.chromium.org/p/gerrit/issues/detail?id=13544
Change-Id: I559c81e4b1d7c487f92a751bd78b987d32c98719
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Just allocate the two string objects directly. The previously used
new StringBuilder(0).toString() returns the same object for both END
and DELIM when run on Java 15, which breaks the wire protocol since
then END and DELIM cannot be distinguished anymore.
Bug: 568950
Change-Id: I9d54d7bf484948c24b51a094256bd9d38b085f35
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
(cherry picked from commit 7da0f0a8f3)
Just allocate the two string objects directly. The previously used
new StringBuilder(0).toString() returns the same object for both END
and DELIM when run on Java 15, which breaks the wire protocol since
then END and DELIM cannot be distinguished anymore.
Bug: 568950
Change-Id: I9d54d7bf484948c24b51a094256bd9d38b085f35
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
(cherry picked from commit 7da0f0a8f3)
[releng] Enable japicmp for the fragments added in 5.8.0
Uncomment the japicmp configurations in the pom.xmls of the fragments
org.eclipse.jgit.gpg.bc and org.eclipse.jgit.ssh.jsch.
Change-Id: I7c884be014cb48387f97a300043b04aeb712fbba
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* master:
SshdSession: close channel gracefully
jgit: Add DfsBundleWriter
Prepare 5.10.0-SNAPSHOT builds
ResolveMerger: do not content-merge gitlinks on del/mod conflicts
ResolveMerger: Adding test cases for GITLINK deletion
ResolveMerger: choose OURS on gitlink when ignoreConflicts
ResolveMerger: improving content merge readability
ResolveMerger: extracting createGitLinksMergeResult method
ResolveMerger: Adding test cases for GITLINK merge
Back out the version change to 5.10.0-SNAPSHOT which was done on master
already.
Change-Id: I1a6b1f0b8f5773be47823d74f593d13b16a601d5
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Close the channel gracefully to give the server a chance to clean up
properly on its side.
Bug: 565854
Change-Id: Iedda5af3b97c8321f08f7ce854274cbb30e401de
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Signing a commit with command line git and gpg 2.2.20 includes the
e-mail part of the key's user ID as a "Signer's User ID" subpacket
on the signature.
Implement this for signing via Bouncy Castle.
Bug: 564386
Change-Id: I68906b895349359596cf3451d65f2840c60df856
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
DfsBundleWriter writes out the entire repository to a Git bundle file.
It packs all objects included in the packfile by concatenating all pack
files. This makes the bundle creation fast and cheap. Useful for backing
up a repository as-is.
Change-Id: Iee20e4b1ab45b2a178dde8c72093c0dd83f04805
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
* changes:
ResolveMerger: do not content-merge gitlinks on del/mod conflicts
ResolveMerger: Adding test cases for GITLINK deletion
ResolveMerger: choose OURS on gitlink when ignoreConflicts
ResolveMerger: improving content merge readability
ResolveMerger: extracting createGitLinksMergeResult method
ResolveMerger: Adding test cases for GITLINK merge
This latest release includes cherry picks, a new feature and some
important changes [1].
[1] https://github.com/bazelbuild/bazel/releases/tag/3.5.0
Change-Id: Ie6966b0b56a351f98cb20b25daf3412740e68562
Signed-off-by: Marco Miller <marco.miller@ericsson.com>
* stable-5.9:
Upgrade maven-resources-plugin to 3.2.0
Upgrade plexus-compiler version to 2.8.8
[bazel] Add missing dependency to slf4j-api
[errorprone] DirCacheEntry: make clear operator precedence
[errorprone] PackWriter#parallelDeltaSearch: avoid suppressed exception
[errorprone] Declare DirCache#version final
Add jgit-4.17-staging target platform for 2020-09
Update target platform to R20200831200620
Signed-off-by: Marco Miller <marco.miller@ericsson.com>
Change-Id: I2e2f41cf6ebbcb45b8978b519db3f1c8f7afb5f4
and update to new Orbit version of the following libraries
- org.apache.commons.codec to 1.14.0.v20200818-1422
- org.apache.httpcomponents.httpclient to 4.5.10.v20200830-2311
- org.bouncycastle.bcpg [1.65.0.v20200527-1955
- org.bouncycastle.bcpkix [1.65.0.v20200527-1955
- org.bouncycastle.bcprov [1.65.1.v20200529-1514
Fix minor issues in target platforms and rename 4.16-staging to 4.16
since that version was released already.
Change-Id: Ifea2600e445e783807a5d94ea23ac3c6550956b5
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
ResolveMerger: do not content-merge gitlinks on del/mod conflicts
Previously ResolveMerger tried to make a fulltext merge entry in case
one of sides got deleted regardless of file mode. This is not
applicable for GITLINK type of entry. After this change it is
rendering appropriate merge result.
Signed-off-by: Demetr Starshov <dstarshov@google.com>
Change-Id: Ibdb4557bf8781bdb48bcee6529e37dc80582ed7e
ResolveMerger: choose OURS on gitlink when ignoreConflicts
Option ignoreConflicts is used when a caller want to create a virtual
commit and use it in a future merge (recursive merge) or show it on
UI (e.g. Gerrit). According to contract in case of ignoreConflicts
ResolveMerger should populate only stage 0 for entries with merge
conflicts as if there is no conflict. Current implementation breaks
this contract for cases when gitlink revision is ambiguous.
Therefore, always select 'ours' when we merge in ignoreConflicts mode.
This will satisfy the contract contract, so recursive merge can
succeed, however it is an arbitrary decision, so it is not guaranteed
to select best GITLINK in all cases.
GITLINK merging is a special case of recursive merge because of
limitations of GITLINK type of entry. It can't contain more than 1 sha-1
so jgit can't write merge conflicts in place like it can with a blob.
Ideally we could signal the conflict with a special value (like
'0000...'), but that must be supported by all tooling (git fsck, c-git)."
Signed-off-by: Demetr Starshov <dstarshov@google.com>
Change-Id: Id4e9bebc8e828f7a1ef9f83259159137df477d89
ResolveMerger: Adding test cases for GITLINK merge
Add test cases which cover content-merge resolve logic.
Git clients try to agressively merge blobs by content, but GITLINK types
of entries can't be merged with each other or with blobs. This change
ensures all possible permutations which can trigger blob and GITLINK
content merge are covered.
Signed-off-by: Demetr Starshov <dstarshov@google.com>
Change-Id: I7e83a28a14d4d2f9e0ba2b1cffbf3224fb7f3fef
Fix possible NegativeArraySizeException in PackIndexV1
Due to an integer overflow bug, the current "Index file is too large
for jgit" check did not work properly and subsequently a
NegativeArraySizeException was raised.
Change-Id: I2736efb28987c29e56bc946563b7fa781898a94a
Signed-off-by: Marc Strapetz <marc.strapetz@syntevo.com>
FS: use binary search to determine filesystem timestamp resolution
Previous code used a minimum granularity of 1 microsecond and would
iterate 233 times on a system where the resolution is 1 second (for
instance, Java 8 on Mac APFS).
New code uses a binary search between the maximum we care about (2
seconds) and zero, with a minimum granularity of also 1 microsecond.
This takes at most 19 iterations (guaranteed). For a file system with 1
second resolution, it takes 4 iterations (1s, 0.5s, 0.8s, 0.9s). With
an up-front check at 1 microsecond and at 1 millisecond this performs
equally well as the old code on file systems with a fine resolution.
(For instance, Java 11 on Mac APFS.)
Also handle obscure cases where the file timestamp implementation may
yield bogus values (as observed on HP NonStop). If such an error case
occurs, log a warning and abort the measurement at the last good value.
Bug: 565707
Change-Id: I82a96729b50c284be7c23fbdf3d0df1bddf60e41
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Matthias Sohn
Thomas Wolf
Marco Miller
Masaya Suzuki
Terry Parker
Demetr Starshov
Marc Strapetz