PushCommand: allow users to disable use of bitmaps for push
Reachability bitmaps are designed to speed up the "counting objects"
phase of generating a pack during a clone or fetch. They are not
optimized for Git clients sending a small topic branch via "git push".
In some cases (see [1]), using reachability bitmaps during "git push"
can cause significant performance regressions.
Add PushCommand#setUseBitmaps(boolean) to allow users to tell "git push"
not to use bitmaps.
Thomas Wolf [Thu, 20 Oct 2022 21:25:38 +0000 (23:25 +0200)]
I/O redirection for the pre-push hook
Fix and complete the implementation of calling the pre-push hook.
Add the missing error stream redirect, and add the missing setters
in Transport and in PushCommand. In Transport, delay setting up a
PrePushHook such that it happens only on a push. Previously, the
hook was set up also for fetches.
Bug: 549246
Change-Id: I64a576dfc6b139426f05d9ea6654027ab805734e Signed-off-by: Thomas Wolf <twolf@apache.org>
Ivan Frade [Thu, 16 Dec 2021 21:08:33 +0000 (13:08 -0800)]
PackParser: populate full size of the PackedObjectInfos
We need the full size of the objects to populate the object-size index
of a pack. This size is not always the one encoded in the object header
in the pack (e.g. for deltas).
Populate the full size of PackedObjectInfos in the PackParser, which is
invoked when receiving a pack e.g. in a push.
Ivan Frade [Sat, 4 Dec 2021 00:11:10 +0000 (16:11 -0800)]
ObjectReader: New #isNotLargerThan method
Partial clones filter the objects to send by size calling
ObjectReader#getObjectSize per object. This method reads the object from
storage to get the size, which can be expensive.
Offer a #isNotLargerThan method. The default implementation reads the
object, but subclasses can override it with more efficient lookups (e.g.
adding an index).
isNotLargerThan gives implementors more options to optimize than
getObjectIndex (e.g. can be implemented storing only object over certain
size).
Matthias Sohn [Wed, 12 Oct 2022 21:33:14 +0000 (23:33 +0200)]
Fail build if there are license issues with dependencies
Configure the dash license-tool-plugin to fail the build if there are
license issues with build dependencies. This plugin will generate a list
of all build dependencies and their license to target/dash/summary.
Delete the checked-in dependency list DEPENDENCIES since the build now
always generates it to target/dash/summary.
See https://github.com/eclipse/dash-licenses#example-maven-plugin
Matthias Sohn [Mon, 12 Sep 2022 08:51:37 +0000 (10:51 +0200)]
Merge branch 'stable-6.3'
* stable-6.3:
Prepare 6.3.1-SNAPSHOT builds
JGit v6.3.0.202209071007-r
JGit v6.3.0.2022009070944-r
[merge] Fix merge conflicts with symlinks
Update DEPENDENCIES for 6.3.0
Update tycho to 2.7.5
Revert "Adds FilteredRevCommit that can overwrites its parents in the DAG."
Revert "Option to pass start RevCommit to be blamed on to the BlameGenerator."
Prepare 6.3.0-SNAPSHOT builds
JGit v6.3.0.202208161710-m3
Thomas Wolf [Mon, 15 Aug 2022 23:02:21 +0000 (01:02 +0200)]
[merge] Fix merge conflicts with symlinks
Previous code would do a content merge on symlinks, and write the merge
result to the working tree as a file. C git doesn't do this; it leaves
a symlink in the working tree unchanged, or in a delete-modify conflict
it would check out "theirs".
Moreover, previous code would write the merge result to the link target,
not to the link. This would overwrite an existing link target, or fail
if the link pointed to a directory.
In link/file conflicts or file/link conflicts, C git always puts the
file into the working tree.
Change conflict handling accordingly. Add tests for all the conflict
cases.
Bug: 580347
Change-Id: I3cffcb4bcf8e336a85186031fff23f0c4b6ee19d Signed-off-by: Thomas Wolf <twolf@apache.org>
Matthias Sohn [Tue, 6 Sep 2022 13:58:47 +0000 (15:58 +0200)]
Merge branch 'master' into stable-6.3
* master:
Move WorkTreeUpdater to merge package
WorkTreeUpdater: use DirCacheCheckout#StreamSupplier
DirCacheCheckout#getContent: also take InputStream supplier
WorkTreeUpdater: remove safeWrite option
* changes:
Move WorkTreeUpdater to merge package
WorkTreeUpdater: use DirCacheCheckout#StreamSupplier
DirCacheCheckout#getContent: also take InputStream supplier
WorkTreeUpdater: remove safeWrite option
Han-Wen Nienhuys [Tue, 30 Aug 2022 08:07:21 +0000 (10:07 +0200)]
DirCacheCheckout#getContent: also take InputStream supplier
This lets us use DirCacheCheckout for routines that want to write
files in the worktree that aren't available as a git object.
DirCacheCheckout#getContent takes a InputStream supplier rather than
InputStream: if filtering fails with IOException, the data is placed
unfiltered in the checkout. This means that the stream has to be read
again, from the start.
Use it in this way in ApplyCommand. This use is incorrect, though: the
same InputStream is returned twice, so if the read to be retried, the
stream will return 0 bytes. It doesn't really matter, because in
either case, the SHA1 will not match up, and the patch fails.
Han-Wen Nienhuys [Thu, 25 Aug 2022 17:37:35 +0000 (19:37 +0200)]
WorkTreeUpdater: remove safeWrite option
This was added in Ideaefd5178 to anticipate on writing files for
ApplyCommand, but we are keeping WorkTreeUpdater private to the merge
package for now.
Matthias Sohn [Mon, 5 Sep 2022 19:19:07 +0000 (21:19 +0200)]
Merge branch 'master' into stable-6.3
* master:
Update Orbit to R20220830213456 for 2022-09
BaseSuperprojectWriter: report invalid paths as manifest errors
ApplyCommand: fix ApplyResult#updatedFiles
WorkTreeUpdater: rename metadata maps
WorkTreeUpdater#Result: hide data members
Add javadoc on RevCommit
Option to pass start RevCommit to be blamed on to the BlameGenerator.
WorkTreeUpdater: re-format and clean-up
Adds FilteredRevCommit that can overwrites its parents in the DAG.
Ivan Frade [Tue, 23 Aug 2022 19:10:27 +0000 (12:10 -0700)]
BaseSuperprojectWriter: report invalid paths as manifest errors
An invalid path in the manifest (e.g. '.') is reported by DirCache in a
runtime exception. In server context this becomes a 500 instead of a user error.
Wrap the runtime invalid path exception into a checked ManifestErrorException that
caller can handle.
Option to pass start RevCommit to be blamed on to the BlameGenerator.
This can allow passing a FilteredRevCommit which is the filtered list of
commit graph making it easier for Blame to work on. This can
significantly improve blame performance since blame can skip expensive
RevWalk.
Thomas Wolf [Sun, 14 Aug 2022 15:47:36 +0000 (17:47 +0200)]
WorkTreeUpdater: re-format and clean-up
Reformat using the standard JGit formatter settings. Clean-ups:
* Try to improve javadoc.
* Remove blindly copy-pasted "@since 6.1" annotations.
* Get rid of private method nonNullNonBareRepo(); it's not needed.
* Simplify method nonNullRepo(), and annotate as @NonNull.
* Rename setInCoreFileSizeLimit() to getInCoreFileSizeLimit().
Change-Id: Ib1797e7cf925d87554307468330971e8ab2e05e9 Signed-off-by: Thomas Wolf <twolf@apache.org> Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Thomas Wolf [Sun, 14 Aug 2022 14:34:50 +0000 (16:34 +0200)]
DirCacheCheckout: load WorkingTreeOptions only once
Previous code loaded the WorkingTreeOptions afresh for every single
file being checked out. This checked the git config (all three files,
repo, user and system config) for having been modified every time.
These checks can be costly, for instance on Windows, or if one of the
three config files is not on a local disk, or on an otherwise slow
storage.
Improve this by loading the options and thus checking the git config
only once before the checkout.
Bug: 579715
Change-Id: I21cd5a808f9d90b5ca2d022f91f0eeb8ca26091c Signed-off-by: Thomas Wolf <twolf@apache.org>
Thomas Wolf [Sun, 14 Aug 2022 14:38:57 +0000 (16:38 +0200)]
WorkTreeUpdater: Fix unclosed streams
1. A TemporaryBuffer.LocalFile must be destroyed to ensure the
temporary file gets deleted on disk.
2. TemporaryBuffer.openInputStream() may be used only after
TemporaryBuffer.close().
3. The caller of DirCacheCheckout.getContent() is responsible for
closing the OutputStream!
Change-Id: I46abb0fba27656a1026858e5783fc60d4738a45e Signed-off-by: Thomas Wolf <twolf@apache.org>
Thomas Wolf [Wed, 20 Jul 2022 16:30:18 +0000 (18:30 +0200)]
Fix adding symlinks to the index when core.symlinks=false
With core.symlinks=false, symlinks are checked out as plain files.
When such a file is re-added to the index, and the index already
contains a symlink there, add the file as a symlink. Previous code
changed the index entry to a regular file.
Bug: 580412
Change-Id: I5497bedc3da89c8b10120b8077c56bc5b67cb791 Signed-off-by: Thomas Wolf <twolf@apache.org>
- add missing @since 6.3 for new protected field workTreeUpdater and new
class WorkTreeUpdater
- suppress API errors caused by removing/adding protected fields and
methods
We follow OSGi semantic versioning which allows breaking implementers in
minor versions which are e.g. subclassing a public class.
* changes:
Reapply "Create util class for work tree updating in both filesystem and index."
ResolveMerger: add coverage for inCore file => directory transition
Provide default shallowCommits getter and setter in ObjectDatabase
I649db9ae679ec2606cf7c530b040f8b6b93eb81a added a default implementation
for getShallowCommits and setShallowCommits to DfsObjDatabase, for the
convenience of any implementers that define subclasses. But we forgot
that some implementers inherit from ObjectDatabase directly instead.
Move the default getter and setter to the base class so that such
callers do not need source changes to unbreak their build.
This also lets us update the api_filters to reflect that this is no
longer an API-breaking change.
Add a bugfix for deletions in ResolveMergers instantiated with just an
ObjectInserter as argument.
Original change description:
Create util class for work tree updating in both filesystem and index.
This class intends to make future support in index updating easier.
This class currently extracts some logic from ResolveMerger. Logic
related to StreamSupplier was copied from ApplyCommand, which will be
integrated in a following change.
com.google.gerrit.extensions.restapi.RestApiException: Cannot rebase ps
[...]
at com.google.gerrit.server.api.changes.RevisionApiImpl.rebase(RevisionApiImpl.java:280)
at com.google.gerrit.acceptance.api.change.ChangeIT.rebaseChangeBase(ChangeIT.java:1584)
Caused by: com.google.gerrit.server.update.UpdateException: java.lang.NullPointerException: repository is required
at com.google.gerrit.server.update.BatchUpdate.executeUpdateRepo(BatchUpdate.java:588)
[...]
Caused by: java.lang.NullPointerException: repository is required
at org.eclipse.jgit.merge.Merger.nonNullRepo(Merger.java:128)
at org.eclipse.jgit.merge.ResolveMerger.addDeletion(ResolveMerger.java:380)
at org.eclipse.jgit.merge.ResolveMerger.processEntry(ResolveMerger.java:553)
at org.eclipse.jgit.merge.ResolveMerger.mergeTreeWalk(ResolveMerger.java:1224)
at org.eclipse.jgit.merge.ResolveMerger.mergeTrees(ResolveMerger.java:1174)
at org.eclipse.jgit.merge.ResolveMerger.mergeImpl(ResolveMerger.java:299)
at org.eclipse.jgit.merge.Merger.merge(Merger.java:233)
at org.eclipse.jgit.merge.Merger.merge(Merger.java:186)
at org.eclipse.jgit.merge.ThreeWayMerger.merge(ThreeWayMerger.java:96)
at com.google.gerrit.server.change.RebaseChangeOp.rebaseCommit(RebaseChangeOp.java:360)
Provide a default implementation for set/get shallowCommits on DfsObjDatabase
Jgit change https://git.eclipse.org/r/c/jgit/jgit/+/193329 adds an implementation for get/set shallow commits in ObjectDatabase. This failed gerrit's acceptance tests since there is no default implementation for them in DfsObjDatabase.
Option to pass start RevCommit to be blamed on to the BlameGenerator.
This can allow passing a FilteredRevCommit which is the filtered list of
commit graph making it easier for Blame to work on. This can
significantly improve blame performance since blame can skip expensive
RevWalk.
Ronald Bhuleskar [Wed, 15 Jun 2022 21:37:21 +0000 (14:37 -0700)]
Add the ability to override parents on RevCommit.
This makes RevCommit extensible to allow having different structure of
child-parent relationship. This change is a pre-requsite for having a
FilteredRevCommit that overrides parents from the RevCommit. That then
provides a cheaper way to walk over a subset of RevCommits instead of
an expensive way that applies filters while walking over selected
commits. Useful with Blame which works on a single file and that can be
made performant, if we know all the commits needed by the Blame
algorithm. So Blame algorithm can avoid walking over finding what
commits to blame on.
This change makes parents field on RevCommit private and exposes it
thrrough overrideable methods such as getParents, getParent at index,
getParentCount and setParents. All other files other than RevCommit are
updating the usages of accessing RevCommits parents.
Thomas Wolf [Tue, 19 Jul 2022 08:13:48 +0000 (10:13 +0200)]
Use constants for git packet protocol line identifiers
Introduce named constants for packet line headers and use them instead
of direct string literals everywhere. This not only makes the code more
readable because we don't need NON-NLS markers, it also makes it more
robust since we can use the length of these constants instead of magic
numbers.
Change-Id: Ie4b7239e0b479a68a2dc23e6e05f25061d481a31 Signed-off-by: Thomas Wolf <twolf@apache.org>
Robin Müller [Fri, 13 May 2022 11:46:13 +0000 (13:46 +0200)]
Fetch: add support for shallow
This adds support for shallow cloning. The CloneCommand and the
FetchCommand now have the new methods setDepth, setShallowSince and
addShallowExclude to tell the server that the client doesn't want to
download the complete history.
Thomas Wolf [Tue, 19 Jul 2022 10:12:26 +0000 (12:12 +0200)]
[tests] Stabilize the HTTP server's request log
The TestRequestLog added log entries only after a request had been
handled. But the response is already sent at that point, so the
following sequence was possible:
client sends request A
server thread T1 handles request A and sends back response
client receives response
client sends request B
server thread T2 handles request B (and sends back response)
server thread T2 logs B
server thread T1 logs A
Fix this by logging events before handling a request, and then filling
in the response data after the request has been handled. This should
avoid such inversions, at least for tests using a single single-threaded
client. With multiple concurrent or multi-threaded clients, all bets
about the log order would be off anyway.
Bug: 528187
Change-Id: I99a46df17231fa7c4f75a8c37e2c14dc098c4e22 Signed-off-by: Thomas Wolf <twolf@apache.org>
Dmitrii Filippov [Thu, 23 Jun 2022 13:28:10 +0000 (15:28 +0200)]
NameConflictTreeWalk: respect git order on multi-tree iteration
The NameConflictTreeWalk class is used in 3-way merge for iterating over
entries in 3 different commits. The class provides information about a
current entry and a state of the entry in commits (e.g entry is file,
entry is directory, entry is missing). In rare cases, the tree walker
can mix information about entries with different name.
The problem appears, because git uses unusual sorting order for
files. Example (this is a simplified real-life example):
Commit 1:
* gradle.properties - file
* gradle - directory (with nested files)
* gradle/file - file in gradle directory
Commit 2:
* gradle.properties - file
* no entry with the name gradle
Commit 3:
* gradle.properties - file
* gradle - file
Here the names are ordered like this:
"gradle" file <"gradle.properties" file < "gradle/file" file.
NameConflictTreeWalk iterator already have code for processing
git sorting order, however in the example above the code doesn't
work correctly. Before the fix, NameConflictTreeWalk returns:
#next()
"gradle - directory" | "gradle.properties" | "gradle - file" - which is
wrong. The expected result is
#next()
"gradle - directory | MISSED_FILE | "gradle - file"
#next()
"gradle.properties"|"gradle.properties"|"gradle.properties"
Ensure that the "matches" field of tree iterators (which contains the
current path) is kept in sync in the case above.
Create util class for work tree updating in both filesystem and index.
This class intends to make future support in index updating easier.
This class currently extracts some logic from ResolveMerger. Logic
related to StreamSupplier was copied from ApplyCommand, which will be
integrated in a following change.
Thomas Wolf [Fri, 15 Jul 2022 18:39:19 +0000 (20:39 +0200)]
Fix the handling of .git/info/exclude and core.excludesFile
The RootIgnoreNode in a WorkingTreeIterator must _not_ add the rules
from .git/info/exclude or from the file designated by git config
core.excludesFile to the list of rules read from the root .gitignore.
These really must be separate nodes in a hierarchy, otherwise the
precedence rules from [1] are violated and the outcome is not the
same as in C git.
[1] https://git-scm.com/docs/gitignore
Bug: 580381
Change-Id: I57802ba7bbbe4f183504c882b6c77a78cc3a9b99 Signed-off-by: Thomas Wolf <twolf@apache.org>
Matthias Sohn [Wed, 6 Jul 2022 15:02:58 +0000 (17:02 +0200)]
Merge branch 'stable-6.2'
* stable-6.2:
UploadPack: don't prematurely terminate timer in case of error
Do not create reflog for remote tracking branches during clone
UploadPack: do not check reachability of visible SHA1s
Add missing package import javax.management to org.eclipse.jgit
Matthias Sohn [Wed, 6 Jul 2022 15:01:03 +0000 (17:01 +0200)]
Merge branch 'stable-6.1' into stable-6.2
* stable-6.1:
UploadPack: don't prematurely terminate timer in case of error
Do not create reflog for remote tracking branches during clone
UploadPack: do not check reachability of visible SHA1s
Add missing package import javax.management to org.eclipse.jgit
projects / jgit.git / log