Most callers/users of TemporaryBuffer are sizing the in-memory
portion large enough that most outputs fit into RAM. With this
assumption they don't pay close attention to the size of IOs
being written, as it "should" just be a copy from one byte array
to another.
Overflow sets up a local file handle, which is costly to write to
for small IO units. Wrap the local file in a BufferedOutputStream
to combine small writes together. Larger writes can still bypass the
buffer as BOS automatically avoids copying for larger writes.
When reading back from an overflowed TemporaryBuffer the InputStream
must be closed to close the FileInputStream that is reading from
the backing file.
Formatting merge conflicts one byte at a time is going to be very
slow when the final OutputStream is a FileOutputStream and the JVM
is making system calls for each byte output.
When outputting a range of bytes from a byte[] the bol (beginning
of line) value only depends on the value of the last byte written.
Other bytes in the array can be passed directly to the lower stream
for more efficient output.
Do not add a newline at the end if neither merged side had one
Bug: 390833
Change-Id: I29f7b79b241929877c93ac485c677487a91bb77b Signed-off-by: André de Oliveira <andre.oliveira@liferay.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Laurent Delaigue [Fri, 21 Nov 2014 10:28:19 +0000 (11:28 +0100)]
Make RepositoryState.REBASING_MERGE reachable again.
If a non interactive rebase is launched, stopping after a conflict
should set the repository state to RepositoryState.REBASING_MERGE
instead of RepositoryState.REBASING_INTERACTIVE.
Bug: 452623
Change-Id: Ie885aab6d71dabd158a718af0d14fff643c9b850 Also-by: Arthur Daussy <arthur.daussy@obeo.fr> Signed-off-by: Laurent Delaigue <laurent.delaigue@obeo.fr> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Fix RecursiveMerger in case of multiple, independent base commits
When RecursiveMerger found that there are multiple base-commits for the
commits to be merged it tries to temporarily merge the base commits. But
if these base commits have no common predecessor there was a bug in JGit
leading to a NPE. This commit fixes this by enforcing that an empty tree
is used as base when merging two unrelated base commits.
This logic was already there when merging two commits which have no
common predecessor (ThreeWayMerger.mergeBase()). But the code which was
computing a new temporary base commit in case of criss-cross merges
didn't take care to pick an empty tree when no common predecessor can be
found.
Dave Borowitz [Wed, 18 Mar 2015 17:46:41 +0000 (10:46 -0700)]
TestRepository: Add a cherryPick method
CherryPickCommand only works on a non-bare repository, as it must
modify the working tree and index in case of a merge conflict. In
tests, being able to recover from a merge conflict is less important,
as the caller should be able to control the full contents of files in
advance of the cherry-pick.
Matthias Sohn [Mon, 9 Mar 2015 22:57:48 +0000 (15:57 -0700)]
Include slf4j and log4j in jgit command line
This enables the command line to log. Include log4j configuration to log
warnings and errors to stderr.
Exclude the dependencies which log4j 1.2.15 should have marked optional.
See
http://unitstep.net/blog/2009/05/18/resolving-log4j-1215-dependency-problems-in-maven-using-exclusions/
for details
Change-Id: Ie730db4007fb7614fd7d130cd0858b1ac550066a Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Until git v1.7.10.2~29^2~1 (builtin/merge.c: reduce parents early,
2012-04-17), C git merge would make merge commits with duplicate parents
when asked to with a series of commands like the following:
Nowadays "git merge" removes redundant parents more aggressively
(whenever one parent is an ancestor of another and not just when
duplicates exist) but merges with duplicate parents are still permitted
and can be created with git fast-import or git commit-tree and history
viewers need to be able to cope with them.
CommitBuilder is an interface analagous to commit-tree, so it should
allow duplicate parents. (That said, an option to automatically remove
redundant parents would be useful.)
Reported-by: Dave Borowitz <dborowitz@google.com>
Change-Id: Ia682238397eb1de8541802210fa875fdd50f62f0 Signed-off-by: Jonathan Nieder <jrn@google.com>
Dave Borowitz [Wed, 18 Mar 2015 19:51:40 +0000 (12:51 -0700)]
TemporaryBuffer: Clear block pointer list instead of reallocating
The block pointer list may have been relatively large, so no need to
make more garbage. Instead, just clear the list and null out all the
elements.
Another possible motivation: a caller may have provided an inaccurate
estimated size, so the list might have been resized several times. If
the list is reused later for a similarly underestimated workload, this
fix will prevent additional resizing on subsequent usages.
Dave Borowitz [Wed, 18 Mar 2015 18:04:26 +0000 (11:04 -0700)]
TemporaryBuffer: Allow presizing block pointer list
Callers may wish to use TemporaryBuffer as an essentially unbounded
buffer by passing Integer.MAX_VALUE as the size. (This makes it
behave like ByteArrayOutputStream, only without requiring contiguous
memory.) Unfortunately, it was always allocating an array in the
backing block pointer list to hold enough blocks to MAX_VALUE--all
262,016 of them. It wasn't allocating the blocks themselves, but this
array was still extremely wasteful, using about 2MiB of memory on a
64-bit system.
Tweak the interface to specify an estimated size, and only allocate
the block pointer list enough entries to hold that size. It's an
ArrayList, so if that estimate was wrong, it'll grow. We assume the
cost of finding enough contiguous memory to grow that array is
acceptable.
While we're in there, fix an off-by-one error: due to integer division
we were undercounting the number of blocks needed to store n bytes of
data as (n / SZ).
Dave Borowitz [Tue, 17 Mar 2015 22:24:59 +0000 (15:24 -0700)]
Git: Don't close underlying repo if it came from from a caller
Since 27ae8bc65 Git has implemented AutoCloseable, which means Eclipse
may warn if close() is never called on a Git instance. For example,
the following would result in a resource warning:
Unfortunately, this construction was subtly broken: it would call both
git.close() and repo.close(), but git.close() would call repo.close()
again. Depending on the repository implementation, this might or might
not be ok. If it's not ok, it might not immediately cause an error, if
the reference count of repo was >2 at the time of closing.
Of course, explicitly calling git.close() followed by repo.close() in
two finally blocks has had the same double-closing problem since
forever. But the problem became worse when Git started implementing
AutoCloseable, because now Eclipse is _actively encouraging_
developers to change working code into broken code.
To work around this, keep track in Git's constructor of whether the
repository was passed in or opened at construction time, and only
close the repository if it was opened by Git.
Note that in the original example, there was not _actually_ a resource
leak, since repo was closed exactly once; git did not _need_ to be
closed in this case. But at least fixing this false-positive warning
no longer introduces a real bug.
Hugo Arès [Fri, 13 Mar 2015 17:57:38 +0000 (13:57 -0400)]
Make MyersDiff interruptible
For some specific file, MyersDiff goes into an infinite loop[1]. Since
this problem is hard to reproduce and possibly harder to fix, this
change makes the MyersDiff interruptible so the diff can be aborted at
least when such infinite loop happens.
When setting the parents of a commit with setParentIds() or
addParentId() it should be checked that we don't have duplicate parents.
An IllegalArgumentException should be thrown in this case.
Change-Id: I9fa9f31149b7732071b304bca232f037146de454 Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Kaloyan Raev [Fri, 6 Feb 2015 14:45:58 +0000 (16:45 +0200)]
CLI status should support --untracked-files
A special options handler is added to properly handle the short -u alias
of the option.
The "normal" mode is not supported by this patch, because this mode of
listing untracked files is not
supported by the org.eclipse.jgit.lib.IndexDiff class. This mode is not
necessary for my use case. It can be added later if anyone really needs
it.
The StatusTest is updated to cover all possible combinations of the
--porcelain and --untracked-files options.
Dave Borowitz [Wed, 11 Mar 2015 22:21:48 +0000 (15:21 -0700)]
TestRepository: Add a reset method to move HEAD around
This flushed out a number of bugs in the way DfsRefUpdate, or at least
the InMemoryRepository implementation, processes symrefs. These have
been fixed, to an extent, in InMemoryRepository, but other
implementations may still suffer from these bugs.
Dave Borowitz [Tue, 10 Mar 2015 22:21:06 +0000 (15:21 -0700)]
TreeWalk: Do not close reader passed explicitly to constructor
The TreeWalk(ObjectReader) constructor is explicitly to handle the case
where the caller is responsible for opening and closing the reader.
The reader should only be closed when it was created in the
TreeWalk(Repository) constructor.
Dave Borowitz [Tue, 10 Mar 2015 22:21:06 +0000 (15:21 -0700)]
RevWalk: Do not close reader passed explicitly to constructor
The RevWalk(ObjectReader) constructor is explicitly to handle the case
where the caller is responsible for opening and closing the reader.
The reader should only be closed when it was created in the
RevWalk(Repository) constructor.
Shawn Pearce [Tue, 10 Mar 2015 00:47:45 +0000 (17:47 -0700)]
Remove AutoCloseable from internal PackFile and friends
PackFile is held by the block cache and cannot be auto closed in a
try-with-resources statement. Remove the interface as JGit does
explicit management of the instances.
ObjectDatabase and RefDatabase are internal details of Repository
and are managed with the Repository. Marking them AutoCloseable
provides no value to the library or an application using the API.
Native git supports "git describe --long". This will enforce returning a
long description of a commit even if a tag is directly pointing to the
commit (in contrast to just returning the tag name as it is now). This
commit teaches JGits DescribeCommand and the describe command in the pgm
package to support "--long".
Bug: 460991
Change-Id: I65e179b79e89049c6deced3c71cb3ebb08ed0a8f Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Laurent Delaigue [Mon, 23 Feb 2015 10:18:50 +0000 (11:18 +0100)]
Refactored pre-commit hook to make it less invasive.
Hooks are now obtained via a convenient API like git commands, and
callers don't have to check for their existence.
The pre-commit hook has been updated accordingly.
Change-Id: I3383ffb10e2f3b588d7367b9139b606ec7f62758 Signed-off-by: Laurent Delaigue <laurent.delaigue@obeo.fr> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Dave Borowitz [Fri, 27 Feb 2015 23:08:50 +0000 (15:08 -0800)]
Add an in-process pack transport for use in tests
This allows for testing arbitrary sets of push/fetch hooks (e.g.
PreReceiveHook) without depending on either an external protocol (e.g.
HTTP) or the local filesystem.
Dave Borowitz [Fri, 27 Feb 2015 23:04:35 +0000 (15:04 -0800)]
Extract classes for transport within a JGit process
TransportLocal knows how to spin up a thread to allow two repositories
in the same process to communicate using the wire protocol. However,
it is still tied to local on-disk filesystems, and needs to be able to
fork processes if not using the default git-{upload,receive}-pack
implementation.
Extract out the connection classes so they can be used by other
transport implementations.
Matthias Sohn [Fri, 27 Feb 2015 00:54:12 +0000 (01:54 +0100)]
Merge branch 'stable-3.7'
* stable-3.7:
Prepare 3.7.1-SNAPSHOT builds
JGit v3.7.0.201502260915-r
Read user.name and email from environment first
Provide more details in exceptions thrown when packfile is invalid
Change-Id: I427f861c6bc94da5e3e05dbbebbf0ad15719a323 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Tue, 24 Feb 2015 22:44:56 +0000 (23:44 +0100)]
Merge branch 'stable-3.7'
* stable-3.7:
Add log4j and slf4j-log4j bridge to jgit feature
Use slf4j to log instead of printing to System.err
Use Target Platform Definition DSL to generate target platforms
Matthias Sohn [Mon, 23 Feb 2015 12:49:24 +0000 (13:49 +0100)]
Read user.name and email from environment first
According to [1] user name and email are taken first from the
environment variables:
GIT_AUTHOR_NAME, GIT_AUTHOR_EMAIL
GIT_COMMITTER_NAME, GIT_COMMITTER_EMAIL
In case (some of) these environment variables are not set, the
information is taken from the git configuration.
JGit doesn not yet support the environment variables GIT_AUTHOR_DATE and
GIT_COMMITTER_DATE.
David Ostrovsky [Sun, 15 Feb 2015 19:31:29 +0000 (20:31 +0100)]
ArchiveCommand: Allow to pass options to underlying stream
Current ArchiveCommand design doesn't allow to pass in options to
underlying stream implementations. To overcome this, client has to
implement custom format implementation (it cannot be derived from
the existing one, because the classes are marked as final), and set
the options using ThreadLocal, before the method
Matthias Sohn [Tue, 10 Feb 2015 17:23:14 +0000 (18:23 +0100)]
Provide more details in exceptions thrown when packfile is invalid
Mention packfile path in exceptions thrown when we detect that a
packfile is invalid and make excplicit that corrupt packs are removed
from the pack list.
Change-Id: I454ada5f8e69307d3f34d1c1b8f3cb87607ddf35 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Mon, 16 Feb 2015 23:12:01 +0000 (00:12 +0100)]
Use Target Platform Definition DSL to generate target platforms
The "Target Platform Definition DSL and Generator" [1] heavily
simplifies maintenance of target platforms. It allows to modularize
target platform definitions which eliminates code duplication. The
.target files understood by P2 and Tycho are generated from .tpd files
which are written in the target platform definition DSL.
In order to edit .tpd files and generate .target files install the
"Target Platform Definition DSL and Generator" 2.0 or later [2] (Note:
on Kepler you also need to add [3] to get Xtext 2.5 which is not
available by default on Kepler). This tools is needed only if you need
to change the Target Platform definition files (*.targetplatform and
*.tpd) and re-generate the *.target files. In normal development you do
not need this and can simply use the generated *.target themselves.
In addition
- update Orbit repository for 4.5 to Mars M5
- use latest released Orbit p2 repository for platform version Luna
and earlier
Matthias Sohn [Tue, 27 Jan 2015 00:43:24 +0000 (01:43 +0100)]
Move console classes to pgm bundle
Since we updated minimum Java version to Java 7 the console bundle
doesn't need to be a separate bundle anymore. Move the contained classes
to the pgm bundle which is using these classes.
Change-Id: If8e6f2d7405fdfe6f4b178673b4ccf99c67d4b64 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Wed, 4 Feb 2015 13:21:52 +0000 (14:21 +0100)]
Merge branch 'stable-3.7'
* stable-3.7:
Add option --orphan for checkout
Prepare post 3.7.0.201502031740-rc1 builds
JGit v3.7.0.201502031740-rc1
Support for the pre-commit hook
Fix FileUtils.testRelativize_mixedCase which failed on Mac OS X
Add a hook test
Introduce hook support into the FS implementations
If a pack isn't found on disk remove it from pack list
Laurent Goubet [Fri, 31 Oct 2014 13:58:07 +0000 (14:58 +0100)]
Introduce hook support into the FS implementations
This introduces the background plumbing necessary to run git hooks from
JGit. This implementation will be OS-dependent as it aims to be
compatible with existing hooks, mostly written in Shell. It is
compatible with unix systems and windows as long as an Unix emulator
such as Cygwin is in its PATH.
Change-Id: I1f82a5205138fd8032614dd5b52aef14e02238ed Signed-off-by: Laurent Goubet <laurent.goubet@obeo.fr> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Dave Borowitz [Mon, 26 Jan 2015 17:21:05 +0000 (09:21 -0800)]
InMemoryRepository: Ensure new ref targets exist in the repo
ObjectInserter recently learned to read back inserted objects before
they have been flushed. It is in general unsafe to create refs to such
objects, but it is now much more possible to do so, by passing "new
RevWalk(inserter.newReader())" into RefUpdate#execute(RevWalk).
We can't change the RefUpdate interface to remove execute(RevWalk);
nor would we necessarily want to, for performance reasons. And in any
case, RefUpdate#safeParse explicitly ignores MissingObjectExceptions.
But we can enforce object existence in InMemoryRepository, which will
allow callers using this class in their tests to ensure they are using
the RefDatabase correctly.
In case the index contains wrong tree extensions don't throw a
ArrayIndexOutOfBounds exception but revalidate the tree extension.
It happened that the git index written by Git for Windows contained valid
(means entryCount>0) tree extensions for pathes which are not existing
in the index. Native git handles this inconsistency silently but JGit
was crashing with a ArrayIndexOutOfBounds exception. Teach JGit to
better recognize such cases and revalidate such extensions.
It's hard to write a test because JGit doesn't write such extensions. It
only reads, validates and makes use of them. But the bug tells how to
create such situations.
* changes:
Document that repo returned by SubmoduleAddCommand needs to be closed
Document that Git instance returned by CloneCommand needs to be closed
projects / jgit.git / log