Han-Wen Nienhuys [Tue, 30 Aug 2022 08:07:21 +0000 (10:07 +0200)]
DirCacheCheckout#getContent: also take InputStream supplier
This lets us use DirCacheCheckout for routines that want to write
files in the worktree that aren't available as a git object.
DirCacheCheckout#getContent takes a InputStream supplier rather than
InputStream: if filtering fails with IOException, the data is placed
unfiltered in the checkout. This means that the stream has to be read
again, from the start.
Use it in this way in ApplyCommand. This use is incorrect, though: the
same InputStream is returned twice, so if the read to be retried, the
stream will return 0 bytes. It doesn't really matter, because in
either case, the SHA1 will not match up, and the patch fails.
Han-Wen Nienhuys [Thu, 25 Aug 2022 17:37:35 +0000 (19:37 +0200)]
WorkTreeUpdater: remove safeWrite option
This was added in Ideaefd5178 to anticipate on writing files for
ApplyCommand, but we are keeping WorkTreeUpdater private to the merge
package for now.
Option to pass start RevCommit to be blamed on to the BlameGenerator.
This can allow passing a FilteredRevCommit which is the filtered list of
commit graph making it easier for Blame to work on. This can
significantly improve blame performance since blame can skip expensive
RevWalk.
Thomas Wolf [Sun, 14 Aug 2022 15:47:36 +0000 (17:47 +0200)]
WorkTreeUpdater: re-format and clean-up
Reformat using the standard JGit formatter settings. Clean-ups:
* Try to improve javadoc.
* Remove blindly copy-pasted "@since 6.1" annotations.
* Get rid of private method nonNullNonBareRepo(); it's not needed.
* Simplify method nonNullRepo(), and annotate as @NonNull.
* Rename setInCoreFileSizeLimit() to getInCoreFileSizeLimit().
Change-Id: Ib1797e7cf925d87554307468330971e8ab2e05e9 Signed-off-by: Thomas Wolf <twolf@apache.org> Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Thomas Wolf [Sun, 14 Aug 2022 14:34:50 +0000 (16:34 +0200)]
DirCacheCheckout: load WorkingTreeOptions only once
Previous code loaded the WorkingTreeOptions afresh for every single
file being checked out. This checked the git config (all three files,
repo, user and system config) for having been modified every time.
These checks can be costly, for instance on Windows, or if one of the
three config files is not on a local disk, or on an otherwise slow
storage.
Improve this by loading the options and thus checking the git config
only once before the checkout.
Bug: 579715
Change-Id: I21cd5a808f9d90b5ca2d022f91f0eeb8ca26091c Signed-off-by: Thomas Wolf <twolf@apache.org>
Thomas Wolf [Sun, 14 Aug 2022 14:38:57 +0000 (16:38 +0200)]
WorkTreeUpdater: Fix unclosed streams
1. A TemporaryBuffer.LocalFile must be destroyed to ensure the
temporary file gets deleted on disk.
2. TemporaryBuffer.openInputStream() may be used only after
TemporaryBuffer.close().
3. The caller of DirCacheCheckout.getContent() is responsible for
closing the OutputStream!
Change-Id: I46abb0fba27656a1026858e5783fc60d4738a45e Signed-off-by: Thomas Wolf <twolf@apache.org>
Thomas Wolf [Wed, 20 Jul 2022 16:30:18 +0000 (18:30 +0200)]
Fix adding symlinks to the index when core.symlinks=false
With core.symlinks=false, symlinks are checked out as plain files.
When such a file is re-added to the index, and the index already
contains a symlink there, add the file as a symlink. Previous code
changed the index entry to a regular file.
Bug: 580412
Change-Id: I5497bedc3da89c8b10120b8077c56bc5b67cb791 Signed-off-by: Thomas Wolf <twolf@apache.org>
- add missing @since 6.3 for new protected field workTreeUpdater and new
class WorkTreeUpdater
- suppress API errors caused by removing/adding protected fields and
methods
We follow OSGi semantic versioning which allows breaking implementers in
minor versions which are e.g. subclassing a public class.
* changes:
Reapply "Create util class for work tree updating in both filesystem and index."
ResolveMerger: add coverage for inCore file => directory transition
Provide default shallowCommits getter and setter in ObjectDatabase
I649db9ae679ec2606cf7c530b040f8b6b93eb81a added a default implementation
for getShallowCommits and setShallowCommits to DfsObjDatabase, for the
convenience of any implementers that define subclasses. But we forgot
that some implementers inherit from ObjectDatabase directly instead.
Move the default getter and setter to the base class so that such
callers do not need source changes to unbreak their build.
This also lets us update the api_filters to reflect that this is no
longer an API-breaking change.
Add a bugfix for deletions in ResolveMergers instantiated with just an
ObjectInserter as argument.
Original change description:
Create util class for work tree updating in both filesystem and index.
This class intends to make future support in index updating easier.
This class currently extracts some logic from ResolveMerger. Logic
related to StreamSupplier was copied from ApplyCommand, which will be
integrated in a following change.
com.google.gerrit.extensions.restapi.RestApiException: Cannot rebase ps
[...]
at com.google.gerrit.server.api.changes.RevisionApiImpl.rebase(RevisionApiImpl.java:280)
at com.google.gerrit.acceptance.api.change.ChangeIT.rebaseChangeBase(ChangeIT.java:1584)
Caused by: com.google.gerrit.server.update.UpdateException: java.lang.NullPointerException: repository is required
at com.google.gerrit.server.update.BatchUpdate.executeUpdateRepo(BatchUpdate.java:588)
[...]
Caused by: java.lang.NullPointerException: repository is required
at org.eclipse.jgit.merge.Merger.nonNullRepo(Merger.java:128)
at org.eclipse.jgit.merge.ResolveMerger.addDeletion(ResolveMerger.java:380)
at org.eclipse.jgit.merge.ResolveMerger.processEntry(ResolveMerger.java:553)
at org.eclipse.jgit.merge.ResolveMerger.mergeTreeWalk(ResolveMerger.java:1224)
at org.eclipse.jgit.merge.ResolveMerger.mergeTrees(ResolveMerger.java:1174)
at org.eclipse.jgit.merge.ResolveMerger.mergeImpl(ResolveMerger.java:299)
at org.eclipse.jgit.merge.Merger.merge(Merger.java:233)
at org.eclipse.jgit.merge.Merger.merge(Merger.java:186)
at org.eclipse.jgit.merge.ThreeWayMerger.merge(ThreeWayMerger.java:96)
at com.google.gerrit.server.change.RebaseChangeOp.rebaseCommit(RebaseChangeOp.java:360)
Provide a default implementation for set/get shallowCommits on DfsObjDatabase
Jgit change https://git.eclipse.org/r/c/jgit/jgit/+/193329 adds an implementation for get/set shallow commits in ObjectDatabase. This failed gerrit's acceptance tests since there is no default implementation for them in DfsObjDatabase.
Option to pass start RevCommit to be blamed on to the BlameGenerator.
This can allow passing a FilteredRevCommit which is the filtered list of
commit graph making it easier for Blame to work on. This can
significantly improve blame performance since blame can skip expensive
RevWalk.
Ronald Bhuleskar [Wed, 15 Jun 2022 21:37:21 +0000 (14:37 -0700)]
Add the ability to override parents on RevCommit.
This makes RevCommit extensible to allow having different structure of
child-parent relationship. This change is a pre-requsite for having a
FilteredRevCommit that overrides parents from the RevCommit. That then
provides a cheaper way to walk over a subset of RevCommits instead of
an expensive way that applies filters while walking over selected
commits. Useful with Blame which works on a single file and that can be
made performant, if we know all the commits needed by the Blame
algorithm. So Blame algorithm can avoid walking over finding what
commits to blame on.
This change makes parents field on RevCommit private and exposes it
thrrough overrideable methods such as getParents, getParent at index,
getParentCount and setParents. All other files other than RevCommit are
updating the usages of accessing RevCommits parents.
Thomas Wolf [Tue, 19 Jul 2022 08:13:48 +0000 (10:13 +0200)]
Use constants for git packet protocol line identifiers
Introduce named constants for packet line headers and use them instead
of direct string literals everywhere. This not only makes the code more
readable because we don't need NON-NLS markers, it also makes it more
robust since we can use the length of these constants instead of magic
numbers.
Change-Id: Ie4b7239e0b479a68a2dc23e6e05f25061d481a31 Signed-off-by: Thomas Wolf <twolf@apache.org>
Robin Müller [Fri, 13 May 2022 11:46:13 +0000 (13:46 +0200)]
Fetch: add support for shallow
This adds support for shallow cloning. The CloneCommand and the
FetchCommand now have the new methods setDepth, setShallowSince and
addShallowExclude to tell the server that the client doesn't want to
download the complete history.
Thomas Wolf [Tue, 19 Jul 2022 10:12:26 +0000 (12:12 +0200)]
[tests] Stabilize the HTTP server's request log
The TestRequestLog added log entries only after a request had been
handled. But the response is already sent at that point, so the
following sequence was possible:
client sends request A
server thread T1 handles request A and sends back response
client receives response
client sends request B
server thread T2 handles request B (and sends back response)
server thread T2 logs B
server thread T1 logs A
Fix this by logging events before handling a request, and then filling
in the response data after the request has been handled. This should
avoid such inversions, at least for tests using a single single-threaded
client. With multiple concurrent or multi-threaded clients, all bets
about the log order would be off anyway.
Bug: 528187
Change-Id: I99a46df17231fa7c4f75a8c37e2c14dc098c4e22 Signed-off-by: Thomas Wolf <twolf@apache.org>
Dmitrii Filippov [Thu, 23 Jun 2022 13:28:10 +0000 (15:28 +0200)]
NameConflictTreeWalk: respect git order on multi-tree iteration
The NameConflictTreeWalk class is used in 3-way merge for iterating over
entries in 3 different commits. The class provides information about a
current entry and a state of the entry in commits (e.g entry is file,
entry is directory, entry is missing). In rare cases, the tree walker
can mix information about entries with different name.
The problem appears, because git uses unusual sorting order for
files. Example (this is a simplified real-life example):
Commit 1:
* gradle.properties - file
* gradle - directory (with nested files)
* gradle/file - file in gradle directory
Commit 2:
* gradle.properties - file
* no entry with the name gradle
Commit 3:
* gradle.properties - file
* gradle - file
Here the names are ordered like this:
"gradle" file <"gradle.properties" file < "gradle/file" file.
NameConflictTreeWalk iterator already have code for processing
git sorting order, however in the example above the code doesn't
work correctly. Before the fix, NameConflictTreeWalk returns:
#next()
"gradle - directory" | "gradle.properties" | "gradle - file" - which is
wrong. The expected result is
#next()
"gradle - directory | MISSED_FILE | "gradle - file"
#next()
"gradle.properties"|"gradle.properties"|"gradle.properties"
Ensure that the "matches" field of tree iterators (which contains the
current path) is kept in sync in the case above.
Create util class for work tree updating in both filesystem and index.
This class intends to make future support in index updating easier.
This class currently extracts some logic from ResolveMerger. Logic
related to StreamSupplier was copied from ApplyCommand, which will be
integrated in a following change.
Thomas Wolf [Fri, 15 Jul 2022 18:39:19 +0000 (20:39 +0200)]
Fix the handling of .git/info/exclude and core.excludesFile
The RootIgnoreNode in a WorkingTreeIterator must _not_ add the rules
from .git/info/exclude or from the file designated by git config
core.excludesFile to the list of rules read from the root .gitignore.
These really must be separate nodes in a hierarchy, otherwise the
precedence rules from [1] are violated and the outcome is not the
same as in C git.
[1] https://git-scm.com/docs/gitignore
Bug: 580381
Change-Id: I57802ba7bbbe4f183504c882b6c77a78cc3a9b99 Signed-off-by: Thomas Wolf <twolf@apache.org>
Matthias Sohn [Wed, 6 Jul 2022 15:02:58 +0000 (17:02 +0200)]
Merge branch 'stable-6.2'
* stable-6.2:
UploadPack: don't prematurely terminate timer in case of error
Do not create reflog for remote tracking branches during clone
UploadPack: do not check reachability of visible SHA1s
Add missing package import javax.management to org.eclipse.jgit
Matthias Sohn [Wed, 6 Jul 2022 15:01:03 +0000 (17:01 +0200)]
Merge branch 'stable-6.1' into stable-6.2
* stable-6.1:
UploadPack: don't prematurely terminate timer in case of error
Do not create reflog for remote tracking branches during clone
UploadPack: do not check reachability of visible SHA1s
Add missing package import javax.management to org.eclipse.jgit
Matthias Sohn [Wed, 6 Jul 2022 15:00:09 +0000 (17:00 +0200)]
Merge branch 'stable-6.0' into stable-6.1
* stable-6.0:
UploadPack: don't prematurely terminate timer in case of error
Do not create reflog for remote tracking branches during clone
UploadPack: do not check reachability of visible SHA1s
Add missing package import javax.management to org.eclipse.jgit
Matthias Sohn [Wed, 6 Jul 2022 08:38:30 +0000 (10:38 +0200)]
Merge branch 'stable-5.13' into stable-6.0
* stable-5.13:
UploadPack: don't prematurely terminate timer in case of error
Do not create reflog for remote tracking branches during clone
UploadPack: do not check reachability of visible SHA1s
Add missing package import javax.management to org.eclipse.jgit
Simeon Andreev [Wed, 15 Jun 2022 20:23:43 +0000 (22:23 +0200)]
JGit blame very slow for large merge commits that rename files
Adjusted BlameGenerator to filter rename detection with the blame path.
This reduces the running time of the blame computation significantly,
for repositories with massive commits involving renames.
The filtered rename detection is made (internally) available with:
org.eclipse.jgit.internal.diff.FilteredRenameDetector
Matthias Sohn [Wed, 29 Jun 2022 12:58:17 +0000 (14:58 +0200)]
UploadPack: don't prematurely terminate timer in case of error
In uploadWithExceptionPropagation don't prematurely terminate timer in
case of error to enable reporting it to the client. Expose a close
method so that callers can terminate it at the appropriate time.
If the timer is already terminated when trying to report it to the
client this failed with the error java.lang.IllegalStateException:
"Timer already terminated".
Do not create reflog for remote tracking branches during clone
When using JGit on a non-bare repository, the CloneCommand
it previously created local reflogs for all branches including remote
tracking ones, causing the generation of a potentially large
number of files on the local filesystem.
The creation of the remote-tracking branches (refs/remotes/*) during
clone is not an issue for the local filesystem because all of them are
stored in a single packed-refs file. However, the creation of a large
number of ref logs on a local filesystem IS an issue because it
may not be tuned or initialised in term of inodes to contain a very
large number of files.
When a user (or a CI system) performs the CloneCommand against
a potentially large repository (e.g., millions of branches), it is
interested in working or validating a single branch or tag and is
unlikely to work with all the remote-tracking branches.
The eager creation of a reflogs for all the remote-tracking branches is
not just a performance issue but may also compromise the ability to
use JGit for cloning a large repository.
The behaviour implemented in this change is also consistent with the
optimisation done in the C code-base [1].
We differentiate between clone and fetch commands using --branch
<initialBranch> option, that is only available in clone command,
and is set as HEAD per default.
Luca Milanesio [Thu, 19 May 2022 10:12:28 +0000 (11:12 +0100)]
UploadPack: do not check reachability of visible SHA1s
When JGit needs to serve a Git client requesting SHA1s
during the want phase, it needs to make a full reachability
check from the advertised refs to the ones requested to
keep all objects in the correct scope of confidentiality
allowed by the avertised refs.
The check is also performed when the SHA1 corresponds to
one of the tips of the advertised refs which is a waste of
resources.
Thomas Wolf [Thu, 23 Jun 2022 10:14:37 +0000 (12:14 +0200)]
[sshd] Correct signature for RSA keys from an SSH agent
Ensure that there is always a list of signature factories in public key
authentication. For keys loaded directly, Apache MINA sshd will use the
(always set) list from the SSH session by default, but for keys from an
SSH agent it won't and instead consider the list set locally on the
UserAuthPublicKey instance. Only that one is null by default, and then
Apache MINA sshd just uses the key type as signature type. Which for
RSA keys from an agent is the "ssh-rsa" signature, i.e., the deprecated
SHA1 signature.
Fix this by explicitly propagating the list from the session to the
UserAuthPublicKey instance if not set already.
Ronald Bhuleskar [Wed, 20 Apr 2022 17:28:30 +0000 (10:28 -0700)]
Annotate the exception with the possible failure reason when Bitmaps are not enabled.
When bitmaps are not enabled the RevWalk is expensive. AllowFilter
provides an override to continue walking. PedestrianWalk can be
expensive and fails when the clone is a partial clone using
--filter=tree:0. This can only work when bitmaps are enabled.
Having a message to the exception improves debuggability of such cases.
eric.steele [Wed, 1 Jun 2022 01:03:17 +0000 (18:03 -0700)]
AmazonS3: Add support for AWS API signature version 4
Updating the AmazonS3 class to support AWS Signature version 4 because
version 2 is no longer supported in all AWS regions. The version can be
selected with the new 'aws.api.signature.version' property (defaults to
2 for backwards compatibility). When set to '4', the user must also
specify the AWS region via the 'region' property. The 'region' property
must match the region that the 'domain' property resolves to.
Matthias Sohn [Tue, 7 Jun 2022 09:37:25 +0000 (11:37 +0200)]
Merge branch 'stable-6.2'
* stable-6.2:
Update DEPENDENCIES for 6.2.0 to be released with 2022-06
Update orbit to R20220531185310 for 2022-06
Prepare 6.2.0-SNAPSHOT builds
Fix connection leak for smart http connections
JGit v6.2.0.202206011217-rc1
Saša Živkov [Fri, 3 Jun 2022 14:36:43 +0000 (16:36 +0200)]
Fix connection leak for smart http connections
SmartHttpPushConnection: close InputStream and OutputStream after
processing. Wrap IOExceptions which aren't TransportExceptions already
as a TransportException.
Also-By: Matthias Sohn <matthias.sohn@sap.com>
Change-Id: I8e11d899672fc470c390a455dc86367e92ef9076
projects / jgit.git / log