Matthias Sohn [Sat, 1 Jun 2013 21:39:52 +0000 (23:39 +0200)]
Fix version of dependency to package org.osgi.framework
OSGi 4.01 comes with package org.osgi.framework version 1.3 [1] which
has the BundleActivator interface needed by org.eclipse.jgit.archive.
OSGi 5.0 matches package org.osgi.framework version 1.7 [2].
Jonathan Nieder [Wed, 29 May 2013 19:33:44 +0000 (12:33 -0700)]
Move FormatActivator.start()/stop() to a separate class
This makes the functionality of registering all formats from the
org.eclipse.jgit.archive package available in contexts where
FormatActivator cannot be built because the OSGi core framework is not
readily available to build against.
Matthias Sohn [Wed, 29 May 2013 10:27:50 +0000 (12:27 +0200)]
Merge branch 'stable-3.0'
* stable-3.0:
Prepare post 3.0.0-rc2 builds
JGit v3.0.0.201305281830-rc2
Support refspecs with wildcard in middle (not only at end)
Fix multiple bugs in RawSubStringPattern used by MessageRevFilter
Handle short branch/tag name for setBranch in CloneCommand
Add missing Bundle-Localization header
Apply tree filter marks when pairing DiffEntry for renames
Improve feature names to become understandable by end users
Update kepler orbit version to R20130517111416
Fix BatchRefUpdate progress-monitoring so it doesn't count twice
Fix AnyObjectId's generic type declaration of Comparable
Fix DiffFormatter NPEs for DiffEntry without content change
Fix CommitCommand not to destroy repo
Fix the parameters to an exception
Prepare post 3.0.0 M7 builds
JGit v3.0.0.201305080800-m7
Change-Id: Ia8441c9796f01497e0d90e672c0aaf60520a0098 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Jonathan Nieder [Tue, 28 May 2013 23:51:32 +0000 (16:51 -0700)]
Maintain list of archive formats in one place
Add a static start() method to FormatActivator to allow outside
classes such as the Archive subcommand of the jgit program to use it
without a BundleContext. This way, the list of formats only has to be
maintained in one place.
While at it, build a list of registered formats at start() time, so
stop() doesn't have to repeat the same list of formats.
Robin Stocker [Mon, 27 May 2013 18:56:31 +0000 (20:56 +0200)]
Fix multiple bugs in RawSubStringPattern used by MessageRevFilter
* Match at end of input was not handled correctly.
* When more than one character matched but not all, the next character
was not considered as a match start (e.g. pattern "abab" didn't match
input "abaabab").
Bug: 409144
Change-Id: Ia44682c618bfbb927f5567c194227421d222a160 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Robin Stocker [Sat, 25 May 2013 13:56:50 +0000 (15:56 +0200)]
Handle short branch/tag name for setBranch in CloneCommand
Before, it was not clear from the documentation what kind of branch name
was accepted. Users specifying "branch" (instead of "refs/heads/branch")
got no error message and ended up with a repository without HEAD and no
checkout.
With this, CloneCommand now tries "$branch", then "refs/heads/$branch"
and then "refs/tags/$branch". C Git only does the last two, but for
compatibility we should still allow "refs/heads/branch".
Bug: 390994
Change-Id: I4be13144f2a21a6583e0942f0c7c40da32f2247a Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Jonathan Nieder [Sat, 25 May 2013 00:30:18 +0000 (17:30 -0700)]
Move ArchiveCommand into standard porcelain API
Allow use of ArchiveCommand without depending on the jgit command-line
tools.
To avoid complicating the process of installing and upgrading JGit,
this does not add a dependency by the org.eclipse.jgit bundle on
commons-compress. Instead, the caller is responsible for registering
any formats they want to use by calling ArchiveCommand.registerFormat.
This patch puts functionality that requires an archiver into a
separate org.eclipse.jgit.archive bundle for people who want it. One
can use it by calling ArchiveCommand.registerFormat directly to
register its formats or by relying on OSGi class loading to load
org.eclipse.jgit.archive.FormatActivator, which takes care of
registration automatically.
Once the appropriate formats are registered, you can make a tar or zip
from a git tree object as follows:
Jonathan Nieder [Fri, 24 May 2013 23:57:10 +0000 (16:57 -0700)]
Remove dependency by ArchiveCommand on archive formats
Provide static registerFormat and unregisterFormat methods to allow
formats to register themselves without the ArchiveCommand code being
aware of them.
Register the basic "zip" and "tar" support at bundle activation time
(and deregister them when unloading the bundle). For anyone using
this code as an OSGi plugin it should continue to just work.
The jgit program does not load org.eclipse.jgit.pgm as an OSGi bundle,
so let the Archive command register the formats it uses explicitly
with registerFormat.
Robin Stocker [Sun, 19 May 2013 13:19:35 +0000 (15:19 +0200)]
Apply tree filter marks when pairing DiffEntry for renames
When using a RenameDetector to generate new DiffEntries after using
DiffEntry.scan, the treeFilterMarks of the original entries were lost.
Now it combines the marks from src and dst.
Jonathan Nieder [Thu, 23 May 2013 23:53:57 +0000 (16:53 -0700)]
Drop dependency by ArchiveCommand.Format interface on commons-compress
Otherwise, anyone trying to implement a new format would have to
depend on commons-compress, even if using a different underlying
library to write the archive.
Shawn Pearce [Thu, 23 May 2013 20:07:03 +0000 (13:07 -0700)]
Use NullOutputStream not DisabledOutputStream in UploadPack
The stream should not throw IllegalStateException if it is off.
Flush the stream after the hook runs, in case any messages need
to be sent ahead of the pack.
Shawn Pearce [Wed, 22 May 2013 19:12:52 +0000 (12:12 -0700)]
Allow PreUploadHook.onSendPack to send messages to the client
Before transmitting to the client a hook may want to send along
a text message ahead of the pack, such as a "message of the day".
Enable this usage by mirroring the message sending API from
ReceivePack on the UploadPack instance, using the side band.
The update call is also moved to the end of the loop, as update() is
only supposed to be called after work has been done ("Denote that some
work units have been completed").
Change-Id: I1620fa75be16dc80df44745d0e123ea512762e31 Signed-off-by: Robin Stocker <robin@nibor.org>
Roberto Tyley [Thu, 16 May 2013 14:44:41 +0000 (15:44 +0100)]
Fix AnyObjectId's generic type declaration of Comparable
If you look at any implementation of Comparable in the JDK, you'll see
that the type parameter for Comparable is supposed to be the type of
the implementing class:
The current type signature of Comparable<Object> is pretty awful, at the
very least because you can not, in fact, successfully compare
AnyObjectId with any random subclass of Object. It also causes problems
with type-inference and the scala.math.Ordering trait in Scala.
In order to compile, this change *does* require removing the
AnyObjectId.ompareTo(Object) method - which actually only ever cast
to AnyObjectId in any case. Nothing in the JGit test suite requires this
method, but it might constitute a breaking API change, so it would be
best if it can be added in time for JGit 3.0.
There was a severe bug in CommitCommand which could corrupt
repos. When merging an annotated tag the JGit MergeCommand writes
correctly the ID of the tag (and not the id of the commit the tag was
pointing to) into MERGE_HEAD. Native git does the same. But
CommitCommand was reading this file and trusting blindly that it will
contain only IDs of commits. Then the CommitCommand created a
commit which has as parent a non-commit object (the tag object). That's
so corrupt that even native git gives up when you call "git log" in
such a repo.
To reproduce that with EGit simply right-click on a tag in the
Repository View and select Merge. The result was a corrupt repo!
Bug: 336291
Change-Id: I24cd5de19ce6ca7b68b4052c9e73dcc6d207b57c Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Shawn Pearce [Thu, 9 May 2013 00:34:47 +0000 (17:34 -0700)]
Fix hardcoded use of target/trash in LocalDiskRepositoryTestCase
`pwd`/target is only valid in Maven Reactor builds where Maven
has moved into the project directory and created a target for
the build output. Most other build systems do not use "target"
and may not even perform a directory change between test suites.
Rewrite LocalDiskRepositoryTestCase's temporary directory code
to use the system specified location and create new unique names.
This prevents fixes between concurrently running tests and allows
the caller to specify the root using java.io.tmpdir.
Update the surefire command lines to use target within each project as
the system temporary directory during unit testing, preventing JGit's
own test suite from writing to /tmp or somewhere like C:\tmp.
Dave Borowitz [Fri, 3 May 2013 18:22:25 +0000 (11:22 -0700)]
Remove unused repository field from RevWalk
The comment about legacy Tag and Object types no longer applies,
though prior to Idb273d5a92849b42935ac14eed73b796b80aad50 the field
was still being used by RewriteTreeFilter.
Dave Borowitz [Fri, 3 May 2013 18:18:53 +0000 (11:18 -0700)]
Require a DiffConfig when creating a FollowFilter
The various rename detection options are an inherent part of the
filter, similar to the path being followed.
This fixes a potential NPE when a RevWalk with a FollowFilter is
created without a Repository, since the old code path tried to get
the DiffConfig from the RevWalk's possibly-missing repository.
Matthias Sohn [Mon, 11 Mar 2013 23:40:34 +0000 (00:40 +0100)]
Add optional feature to install Java 7 support
Change-Id: I15cfd91857830d69abbe729be3e237fc9cd4fca0 Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Robin Rosenberg [Mon, 4 Feb 2013 00:10:26 +0000 (01:10 +0100)]
Extend the FS class for Java7
The most important difference is that in Java7 we have symbolic links
and for most operations in the work tree we want to operate on the link
itself rather than the link target, which the old File methods generally
do.
We also add support for the hidden attribute, which only makes sense
on Windows and exists, just since there are claims that Files.exists
is faster the File.exists.
A new bundle is only activated when run with a Java7 execution
environment. It is implemented as a fragment.
Tycho currently has no way to conditionally include optional features
based on the java version used to run the build, this means with this
change the jgit packaging build always needs to be run using java 7.
Change-Id: I3d6580d6fa7b22f60d7e54ab236898ed44954ffd Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Robin Stocker [Sat, 27 Apr 2013 14:14:46 +0000 (16:14 +0200)]
Only fetch tags that do not exist locally with auto-follow
This corresponds to what C Git does, quoting from the fetch man page:
This is done by first fetching from the remote using the given
<refspec>s, and if the repository has objects that are pointed by
remote tags that it does not yet have, then fetch those missing tags.
Before, JGit would also fetch tags that exist locally but point to a
different object, resulting in REJECTED results for these.
Also add some test cases to cover more cases.
Bug: 388095
Change-Id: Ib03d2d82e9c4b60179d626cfd5174be1da6388b2 Also-by: Stefan Lay <stefan.lay@sap.com>
Robin Stocker [Fri, 12 Apr 2013 12:21:02 +0000 (14:21 +0200)]
Abort before delete in FileUtils.delete EMPTY_DIRECTORIES_ONLY|RECURSIVE
Depending on the order in which items are traversed for RECURSIVE, an
empty directory may come first before detecting that there is a file and
aborting.
This fixes it by traversing files first.
Bug: 405558
Change-Id: I638b7da58e33ffeb0fee172b96f4c823943d29e9 Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Alex Blewitt [Sat, 27 Apr 2013 10:34:57 +0000 (11:34 +0100)]
Allow deletions to occur when there is no HEAD
If the HEAD is not present in a repository, then there is a
NullPointerException thrown in the delete code. Since this only
exists to verify if the deletion is not the HEAD reference, then
skip this check if the HEAD cannot be found.
Bug: 406722
Change-Id: I882497202d986096513a4d791cd07fa935a3f9e4 Signed-off-by: Alex Blewitt <alex.blewitt@gmail.com>
Jonathan Nieder [Thu, 25 Apr 2013 21:20:54 +0000 (14:20 -0700)]
archive: Release resources before returning
The only caller exits immediately after calling execute() so this
shouldn't make a difference, but it's good practice and should make it
easier to expose the functionality in a public API later.
Jonathan Nieder [Fri, 26 Apr 2013 17:16:09 +0000 (10:16 -0700)]
Remove unused logger from RecursiveMerger
JGit doesn't currently use java.util.logging.Logger. Remove this
never-used Logger introduced in ab99b78ca08a (Implement recursive
merge strategy, 2013-02-21) to make that easier to see.
Revert "Add tests for FileUtils.delete and EMPTY_DIREECTORIES_ONLY"
This reverts commit 7aa54967a26cb027fe390ad1c624ebb30f9ac6d5.
The unit test dependend upon the specific order of names that
listFiles() returned members in. The order is completely undefined
and may differ even on different versions of Linux based systems.
A proper unit test for this code would have considered both cases,
where the deletion function was able to remove an empty subdirectory,
or fail to remove a subdirectory because a file was still present
within. This is not such a test.
Robin Stocker [Thu, 21 Mar 2013 00:39:04 +0000 (01:39 +0100)]
IndexDiff: Provide stage state for conflicting entries
Adds a new method getConflictingStageStates() which returns a
Map<String, StageState> (path to stage state). StageState is an enum for
all possible stage combinations (BOTH_DELETED, ADDED_BY_US, ...).
This can be used to implement the conflict text for unmerged paths in
output of "git status" or in EGit for decorations/hints.
Bug: 403697
Change-Id: Ib461640a43111b7df4a0debe92ff69b82171329c Signed-off-by: Chris Aniszczyk <zx@twitter.com>
Matthias Sohn [Thu, 18 Apr 2013 20:35:37 +0000 (22:35 +0200)]
Do not export package org.eclipse.jgit from jgit tests
Commit 3344b93c erroneously exported the package org.eclipse.jgit.lib
from the org.eclipse.jgit.test bundle which made this a split package
since the bundle org.eclipse.jgit exports the same package.
Split packages are evil in general and most probably caused the build
cycle errors observed recently when importing the jgit projects in
Eclipse [1].
Rescale "Compressing objects" progress meter by size
Instead of counting objects processed, count number of bytes added
into the window. This should rescale the progress meter so that 30%
complete means 30% of the total uncompressed content size has been
inflated and fed into the window.
In theory the progress meter should be more accurate about its
percentage complete/remaining fraction than with objects. When
counting objects small objects move the progress meter more rapidly
than large objects, but demand a smaller amount of work than large
objects being compressed.
Instead of assuming all objects cost the same amount of time to
delta compress, aggregate the byte size of objects in the list
and partition threads with roughly equal total bytes.
Before splitting the list select the N largest paths and assign
each one to its own thread. This allows threads to get through the
worst cases in parallel before attempting smaller paths that are
more likely to be splittable.
By running the largest path buckets first on each thread the likely
slowest part of compression is done early, while progress is still
reporting a low percentage. This gives users a better impression of
how fast the phase will run. On very complex inputs the slow part
is more likely to happen first, making a user realize its time to
go grab lunch, or even run it overnight.
If the worst sections are earlier, memory overruns may show up
earlier, giving the user a chance to correct the configuration and
try again before wasting large amounts of time. It also makes it
less likely the delta compression phase reaches 92% in 30 minutes
and then crawls for 10 hours through the remaining 8%.
By excluding objects the compactor can avoid storing objects that
are already well packed in the base GC packs, or any other pack
not being replaced by the current compaction operation.
For deltas the base object is still included even if the base exists
in another exclusion set. This favors keeping deltas for recent
history, to support faster fetch operations for clients.
Matthias Sohn [Mon, 25 Mar 2013 02:39:58 +0000 (03:39 +0100)]
Make recursive merge strategy the default merge strategy
Use recursive merge as the default strategy since it can successfully
merge more cases than the resolve strategy can. This is also the default
in native Git.
Change-Id: I38fd522edb2791f15d83e99038185edb09fed8e1 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Update PackBitmapIndexRemapper to handle mappings not in the new pack.
Previously, the code assumed all commits in the old pack would also
be present in the new pack. This assumption caused an
ArrayIndexOutOfBoundsException during remapping of ids. Fix the
iterator to only return entries that may be remapped. Furthermore,
update getBitmap() to return null if commit does not exist in the
new pack.
* changes:
Always attempt delta compression when reuseDeltas is false
Avoid TemporaryBuffer.Heap on very small deltas
Correct distribution of allowed delta size along chain length
Split remaining delta work on path boundaries
Replace DeltaWindow array with circularly linked list
Micro-optimize copy instructions in DeltaEncoder
Micro-optimize DeltaWindow primary loop
Micro-optimize DeltaWindow maxMemory test to be != 0
Mark DeltaWindowEntry methods final
Always attempt delta compression when reuseDeltas is false
If reuseObjects=true but reuseDeltas=false the caller wants attempt
a delta for every object in the input list. Test for reuseDeltas
to ensure every object passes through the searchInWindow() method.
If no delta is possible for an object and it will be stored whole
(non-delta format), PackWriter may still reuse its content from any
source pack. This avoids an inflate()-deflate() cycle to recompress
the object contents.
TemporaryBuffer is great when the output size is not known, but must
be bound by a relatively large upper limit that fits in memory, e.g.
64 KiB or 20 MiB. The buffer gracefully supports growing storage by
allocating 8 KiB blocks and storing them in an ArrayList.
In a Git repository many deltas are less than 8 KiB. Typical tree
objects are well below this threshold, and their deltas must be
encoded even smaller.
For these much smaller cases avoid the 8 KiB minimum allocation used
by TemporaryBuffer. Instead allocate a very small OutputStream
writing to an array that is sized at the limit.
Correct distribution of allowed delta size along chain length
Nicolas Pitre discovered a very simple rule for selecting between two
different delta base candidates:
- if based whole object, must be <= 50% of target
- if at end of a chain, must be <= 1/depth * 50% of target
The rule penalizes deltas near the end of the chain, requiring them to
be very small in order to be kept by the packer. This favors deltas
that are based on a shorter chain, where the read-time unpack cost is
much lower. Fewer bytes need to be consulted from the source pack
file, and less copying is required in memory to rebuild the object.
Junio Hamano explained Nico's rule to me today, and this commit fixes
DeltaWindow to implement it as described.
When no base has been chosen the computation is simply the statements
denoted above. However once a base with depth of 9 has been chosen
(e.g. when pack.depth is limited to 10), a non-delta source may
create a new delta that is up to 10x larger than the already selected
base. This reflects the intent of Nico's size distribution rule no
matter what order objects are visited in the DeltaWindow.
With this patch and my other patches applied, repacking JGit with:
CGit (all) 5,711,735 bytes; real 0m13.942s user 0m47.722s [1]
JGit heads 5,718,295 bytes; real 0m11.880s user 0m38.177s [2]
rest 9,809 bytes
The improved JGit result for the head pack is only 6.4 KiB larger than
CGit's resulting pack. This patch allowed JGit to find an additional
39.7 KiB worth of space savings. JGit now also often runs 2s faster
than CGit, despite also creating bitmaps and pruning objects after the
head pack creation.
[1] time git repack -a -d -F --window=250 --depth=50
[2] time java -Xmx128m -jar jgit debug-gc
When an idle thread tries to steal work from a sibling's remaining
toSearch queue, always try to split along a path boundary. This
avoids missing delta opportunities in the current window of the
thread whose work is being taken.
The search order is reversed to walk further down the chain from
current position, avoiding the risk of splitting the list within
the path the thread is currently processing.
When selecting which thread to split from use an accurate estimate
of the size to be taken. This avoids selecting a thread that has
only one path remaining but may contain more pending entries than
another thread with several paths remaining.
As there is now a race condition where the straggling thread can
start the next path before the split can finish, the stealWork()
loop spins until it is able to acquire a split or there is only
one path remaining in the siblings.
PackWriter generally chooses the order for objects when it builds the
object lists. This ordering already depends on history information to
guide placing more recent objects first and historical objects last.
Allow PackWriter to make the basic ordering decisions, instead of
trying to override them. The old approach of sorting the list caused
DfsReader to override any ordering change PackWriter might have tried
to make when repacking a repository.
This now better matches with WindowCursor's implementation, where
PackWriter solely determines the object ordering.
Replace DeltaWindow array with circularly linked list
Typical window sizes are 10 and 250 (although others are accepted).
In either case the pointer overhead of 1 pointer in an array or
2 pointers for a double linked list is trivial. A doubly linked
list as used here for window=250 is only another 1024 bytes on a
32 bit machine, or 2048 bytes on a 64 bit machine.
The critical search loops scan through the array in either the
previous direction or the next direction until the cycle is finished,
or some other scan abort condition is reached. Loading the next
object's pointer from a field in the current object avoids the
branch required to test for wrapping around the edge of the array.
It also saves the array bounds check on each access.
When a delta is chosen the window is shuffled to hoist the currently
selected base as an earlier candidate for the next object. Moving
the window entry is easier in a double-linked list than sliding a
group of array entries.
projects / jgit.git / log