Shawn O. Pearce [Tue, 2 Nov 2010 23:29:39 +0000 (16:29 -0700)]
Add ObjectId getByte for random access
Processing git notes requires random access to part of the raw data
of each ObjectId... which isn't easy because ObjectIds are stored
with an internal representation of 5 ints. Expose random access
to the individual data bytes through new methods, avoiding the
need to convert first to a byte[20].
Change-Id: I99e64700b27fc0c95aa14ef8ad46a0e8832d4441 Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce [Tue, 2 Nov 2010 21:12:00 +0000 (14:12 -0700)]
Refactor tree entry formatting into a common class
Instead of hiding this logic inside of DirCacheTree and the legacy
Tree type, pull it into a common place where we can reuse it by
creating tree records in a buffer that can be passed directly into
the ObjectInserter. This allows us to avoid some copying, as the
inserter can be given the internal buffer of the formatter.
Because we trust these two callers to feed us records in the proper
order, without '/' in the names, and without duplicate names in the
same tree, we don't do any validation inside of the formatter itself.
To protect themselves from making ordering errors, developers should
continue to use DirCache to process edits to source code trees.
Change-Id: Idf7f10e736d4a44ccdf8afe060535d7b0554a92f Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce [Sat, 30 Oct 2010 01:35:43 +0000 (18:35 -0700)]
Fix ugly diff showing insertion of new method
When adding a new method near the end of the sequence we want to
show the full method inserted, and not tear the prior method due
to the common trailing curly brace being consumed as part of the
common end region of the sequences.
Bug: 328895
Change-Id: I233bc40445fb5452863f5fb082bc3097433a8da6 Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce [Sat, 30 Oct 2010 01:04:10 +0000 (18:04 -0700)]
Delete DiffPerformanceTest
This test isn't that useful. The better way to evaluate diff
algorithm performance is to run `jgit debug-diff-algorithms` over
real-world repositories, such as linux-2.6.git. Whenever we modify
an algorithm we should manually verify that its runtime performance
doesn't get any worse than it already is.
Change-Id: I0beed3a5a8a537c958a5a6438a1283f97fa2097a Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce [Sat, 30 Oct 2010 00:50:26 +0000 (17:50 -0700)]
Fix broken HistogramDiff
HistogramDiff failed on cases where the initial element for the LCS
was actually very common (e.g. has 20 occurrences), and the first
element of the inserted region after the LCS was also common but
had fewer occurrences (e.g. 10), while the LCS also contained a
unique element (1 occurrence).
This happens often in Java source code. The initial element for
the LCS might be the empty line ("\n"), and the inserted but common
element might be "\t/**\n", with the LCS being a large span of
lines that contains unique method declarations. Even though "/**"
occurs less often than the empty line its not a better LCS if the
LCS we already have contains a unique element.
The logic in HistogramDiff would normally have worked fine, except I
tried to optimize scanning of B by making tryLongestCommonSequence
return the end of the region when there are matching elements
found in A. This allows us to skip over the current LCS region,
as it has already been examined, but caused us to fail to identify
an element that had a lower occurrence count within the region.
The solution used here is to trade space-for-time by keeping a
table of A positions to their occurrence counts. This allows the
matching logic to always use the smallest count for this region,
even if the smallest count doesn't appear on the initial element.
The new unit test testEdit_LcsContainsUnique() verifies this new
behavior works as expected.
Bug: 328895
Change-Id: Id170783b891f645b6a8cf6f133c6682b8de40aaf Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Robin Stocker [Fri, 29 Oct 2010 12:52:52 +0000 (14:52 +0200)]
Use readFully() instead of read()
Fixes the "Method ignores results of InputStream.read()" warning.
This is the only place where read() was used instead of readFully()
and the return value was not checked. So it was either an oversight
or should be documented. This change assumes it was an oversight.
As described in Bug 328551 there was a bug that the merge algorithm
was not always reporting conflicts when the same line was deleted
and modified. This problem was introduced during commit 0c017188b4d41cc80c297e35097095026734b3d4 when reported conflicts have
been checked for common pre- and suffixes.
This was fixed here by better determining whether after stripping
off common prefixes and suffixes from a conflicting region there
is still some conflicting part left.
I also added a unit test to test this situation.
Bug: 328551
Change-Id: Iec6c9055d00e5049938484a27ab98dda2577afc4 Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Mathias Kinzler [Tue, 19 Oct 2010 06:54:28 +0000 (08:54 +0200)]
PullCommand: support upstream configuration for local branches
When creating a local branch based on another local branch, the
upstream configuration contains "." as origin and the source branch
as "merge". The PullCommand should support this by skipping the
fetch step altogether and use the base branch to merge with.
Shawn O. Pearce [Thu, 14 Oct 2010 20:06:54 +0000 (13:06 -0700)]
Call ProgressMonitor.update() from main thread
Don't permit transient worker threads to access the underlying output
stream of a ProgressMonitor, as they might get marked as the stream's
writer thread. Instead proxy update events from the workers back onto
the application's real work thread. This ensures that the stream only
sees a single thread, and its the thread that will remain alive for
the entire life cycle of the operation.
This fixes IOException("Write end dead") during local repository fetch
when threaded delta search is enabled. One of the transient delta
search threads became the designated writer for the pipe, and when it
terminated the reader end thought the writer was dead, even though the
main writer thread was still executing in PackWriter.
Bug: 326557
Change-Id: I01d1b20a3d7be1c0b480c7fb5c9773c161fe5c15 Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Prevent endless loop of events fired by RefsDirectory
RefsDirectory fires a RefsChangedEvent when it detect that one
ref changed (new, modified, deleted). But there was a potential
of wrong events beeing fired leading to a endless loop in EGit.
Problem is that when calling getRefs(ALL) we don't want to report
additional refs and by that we remove the additional refs from
the list of "refs reported upwards last time". We fire an
RefsChangedEvent because we think that the special refs are not
there anymore.
I fixed this by removing eventing for the additional refs. Another
alternative would be to always scan also for additional refs and
put them in the list of refs. But getRefs(ALL) would then remove
the additional refs again. I didn't do that for performance reasons
and also because I am not sure whether we want evnting for
additional refs.
Change-Id: Icb9398b55a8c6bbf03e38f6670feb67754ce91e0 Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Add support for special symref FETCH_HEAD and MERGE_HEAD
The RefDirectory class was not returning FETCH_HEAD and
MERGE_HEAD when trying to get all refs via getRefs(RefDatabase.ALL).
This fix adds constants for FETCH_HEAD and ORIG_HEAD and adds a
new getter getAdditionalRefs() to get these additional refs.
To be compatible with c git the getRefs(ALL) method will not return
FETCH_HEAD, MERGE_HEAD and ORIG_HEAD.
Change-Id: Ie114ca92e9d5e7d61d892f4413ade65acdc08c32 Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Matthias Sohn [Sat, 23 Oct 2010 21:56:06 +0000 (23:56 +0200)]
[findbugs] Fix illegal format specifier
For integral arguments the precision is not applicable, would cause a
runtime exception when executed, see
http://download.oracle.com/javase/1.5.0/docs/api/java/util/Formatter.html#syntax
Change-Id: I4738c64c1153a8d4ef5430e15d0fe54f0a37949f Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Chris Aniszczyk [Thu, 14 Oct 2010 16:26:46 +0000 (09:26 -0700)]
Add FindBugs and CPD to the build.
We need to use findbugs-maven-plugin:2.3.2-SNAPSHOT
since otherwise build fails with maven-3.0 [1], [2].
We should switch to the release version as soon
as this becomes available.
Shawn O. Pearce [Tue, 19 Oct 2010 22:51:44 +0000 (00:51 +0200)]
Make ObjectDirectory getPacks() work the first time
If an object hasn't been accessed yet the pack list for a repository
may not have been scanned from disk. If an application (e.g. the dumb
transport servlet support code) asks for the pack list for an
ObjectDirectory, we should load it immediately.
Change-Id: I93d7b1bca422d905948e8e83b2afa83c8894a68b Signed-off-by: Shawn O. Pearce <spearce@spearce.org> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Shawn O. Pearce [Mon, 18 Oct 2010 06:10:47 +0000 (23:10 -0700)]
Update CachedObjectDirectory when inserting objects
If an ObjectInserter is created from a CachedObjectDirectory, we need
to ensure the cache is updated whenever a new loose object is actually
added to the loose objects directory, otherwise a future read from an
ObjectReader on the CachedObjectDirectory might not be able to open
the newly created object.
We mostly had the infrastructure in place to implement this due to the
injection of unpacked large deltas, but we didn't have a way to pass
the ObjectId from ObjectDirectoryInserter to CachedObjectDirectory,
because the inserter was using the underlying ObjectDirectory and not
the CachedObjectDirectory. Redirecting to CachedObjectDirectory
ensures the cache is updated.
Change-Id: I1f7bdfacc7ad77ebdb885f655e549cc570652225 Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce [Sun, 17 Oct 2010 04:33:06 +0000 (21:33 -0700)]
Allow pgm Main to be extended
3rd party packages that use repository types other than FileRepository
may wish to extend our pgm package and implement their own resolution
scheme for repository "names" that are passed in by the --git-dir
command line option. Make that possible by allowing the package to
extend the Main class and override the lookup.
This is primarily useful when developing new storage implementations
and trying to experiment with the results, without linking all of it
into the core JGit package.
Change-Id: Id30e168da16341e5da43365688a63aa30c7b7e2c Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Implemented the initial version of a cherry-pick command.
A correct error handling is missing (what happens if the
checkout fails, the cherry-pick leads to conflicts etc).
But straightforward cherry-picks works.
Change-Id: I235c0eb3a7a2d5bdfe40400f1deed06f29d746e1 Signed-off-by: Christian Halstrick <christian.halstrick@sap.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Shawn O. Pearce [Thu, 14 Oct 2010 02:05:52 +0000 (19:05 -0700)]
Add getString utility functions to RawText
These routines can be useful when debugging, because we can add an
expression to the Eclipse "Expressions" panel to show the text that
appears on a line. Gerrit Code Review also uses these in its own
subclass of RawText in order to format patch files, so pulling it up
to be part of core JGit may help other applications too.
Change-Id: I20a6b112e3403ecfc1c2715ae75dcecc1a85b167 Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce [Thu, 14 Oct 2010 03:47:08 +0000 (20:47 -0700)]
Remove dead RawText(RawTextComparator) constructor
Since the introduction of HashedSequence we no longer need to supply
the RawTextComparator at the time of constructing a RawText. Drop the
definition from the constructor, because it doesn't make sense as part
of our public API.
Change-Id: Iaab34611d60eee4a2036830142b089b2dae81842 Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce [Thu, 14 Oct 2010 03:43:11 +0000 (20:43 -0700)]
Fix RawTextComparator reduceCommonStartEnd at empty lines
When an empty line was inserted at the beginning of the common end
part of a RawText the comparator incorrectly considered it to be
common, which meant the DiffAlgorithm would later not even have it be
part of the region it examines. This would cause JGit to skip a line
of insertion, which later confused Gerrit Code Review when it tried to
match up the pre and post RawText files for a difference that had this
type of insertion.
Define two new unit tests to check for this insertion of a blank line
condition and correct for it by removing the LF from the common region
when the condition is detected.
Change-Id: I2108570eb2929803b9a56f9fb9c400c758e7156b Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce [Mon, 11 Oct 2010 00:49:26 +0000 (17:49 -0700)]
Fix three-word command names
Command names like MakeCacheTree weren't coming up with hyphens between
every word, so they read "debug-make-cachetree" rather than the
expected "debug-make-cache-tree". On each lowercase character reset
the lastWasDash flag so the next uppercase will insert a hyphen before
the next word.
Change-Id: I539fabb339e60896165619c307dec71e3317b0d8 Signed-off-by: Shawn O. Pearce <spearce@spearce.org> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Wed, 13 Oct 2010 13:26:33 +0000 (15:26 +0200)]
Generate correct version for jgit source bundle
The maven 2 build for jgit source bundle didn't create a correct
OSGi version string, instead of
org.eclipse.jgit.source_0.10.0.<timestamp>
the generated OSGi version was
org.eclipse.jgit.source_0.10.0.SNAPSHOT
This caused trouble when trying to install it from p2 repository.
Bug: 327616
Change-Id: Ic27c763ae9a4bcbb5bd6ed9562cd98bb4da3386b Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Fix NPE when calling CreateBranch without explict startpoint
When creating a branch with CreateBranchCommand.call() without
specifying an explicit startPoint HEAD should be used as startPoint.
There was a bug leading to an NPE in such a case.
Change-Id: Ic0a5dc1f33a0987d66c09996c8012c45785500ff Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Remove AmbiguousObjectException from BranchCreateCommand.call()
We wanted to wrap all LowLevel JGit excpetions into a
JGitInternalException so that users of this high-level interface
don't have to explicitly catch all of them. This
was forgotten on BranchCreateCommand.call() and I added
it.
Change-Id: Ie140e99574fb004137c66e80fb92eb6c6d0fa5e1 Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
HistogramDiff outperforms it for any case where PatienceDiff needs to
fallback to another algorithm. Consequently it's not worth keeping
around, because we would always want a fallback enabled.
Change-Id: I39b99cb1db4b3be74a764dd3d68cd4c9ecd91481 Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce [Sun, 10 Oct 2010 20:36:30 +0000 (13:36 -0700)]
debug-diff-algorithms: Real world performance test implementations
When working on a difference algorithm's implementation, its generally
more important to care about how it behaves on real-world inputs than
it does on fake inputs created for unit test cases. Run each
implementation against a number of real-world repositories, looking at
changes between files in each commit. This gives a better picture of
how a particular algorithm performs.
This test suite run against JGit and linux-2.6 with the current
available algorithms shows HistogramDiff always out-performs
MyersDiff, and by a wide margin on the linux-2.6 sources. As
HistogramDiff has similar output properties as PatienceDiff, the
resulting edits are probably also more human-readable. These test
results show that HistogramDiff is a good choice for the default
implementation, and also show that PatienceDiff isn't worth keeping.
Shawn O. Pearce [Mon, 11 Oct 2010 01:48:15 +0000 (18:48 -0700)]
Fix corrupted large deltas
Large objects stored as deltas get unpacked by JGit into a loose
object, so they are cheaper to access later on. This unpacking was
broken because TeeInputStream copied the wrong length into the loose
object, sometimes copying too many bytes into the result. This
created a loose object that did not have the correct content, and
whose length did not match the length denoted in the object header.
Change-Id: I3ce1fd9f3dc5bd195249c7872b3bec49570424a2 Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
When passing to a fallback algorithm, we can avoid creating a new copy
of the hash codes for each sequence by passing in the hashed sequences
directly. This makes it cheaper to switch from HistogramDiff down to
MyersDiff in a single pass.
Change-Id: Ibf2e81be57c083862eeb134279aed676653bf9b5 Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce [Sun, 10 Oct 2010 20:36:39 +0000 (13:36 -0700)]
Fix empty block corner case in PatienceDiff
There is a corner case where we get an EMPTY region during recursion,
but we didn't expect to receive that. Its harmless to ignore the
region since the region is empty and has no content, so do so rather
than throwing an exception
Change-Id: I50dcec81ecba763072bb739adfab5879fb48b23a Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce [Sun, 10 Oct 2010 20:08:50 +0000 (13:08 -0700)]
Fix infinite loop in PatienceDiff
Certain inputs caused an infinite loop because the prior match data
couldn't be used as expected. Rather than incrementing the match
pointer before looking at an element, do it after, so the loop breaks
when we wrap around to the starting point.
Change-Id: Ieab28bb3485a914eeddc68aa38c256f255dd778c Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Mathias Kinzler [Fri, 8 Oct 2010 12:01:55 +0000 (14:01 +0200)]
Add "Branch" command
The need for branching becomes more pressing with pull
support: we need to make sure the upstream configuration entries
are written correctly when creating and renaming branches
(and of course are cleaned up when deleting them).
This adds support for listing, adding, deleting and renaming
branches including the more common options.
Shawn O. Pearce [Sat, 9 Oct 2010 05:41:18 +0000 (22:41 -0700)]
Cleanup RefUpdateTest
Application code, including unit tests for storage implementations,
should not extend RevCommit outside of the scope of using it for a
RevWalk. Its a lot of overhead and unlikely to work long-term.
Instead for the one test that matters, use a custom subclass of the
ObjectId type. This lets us measure exactly what we are looking for,
which is that the subclass isn't retained.
A lot of other tests were unnecessarily wrapping an object with a
RevCommit and storing that back into the RefUpdate. This is just
retesting what the earlier no-cache test was doing, and complicated
the test considerably. Drop that code and just rely on the value that
was configured by the helper method.
Change-Id: I5b31813484eaa306e9bc4de9622dd5bd4846b16d Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
In bug 323571 it is mentioned that if you call
'toURI().toURL().toString()' on a java.io.File you cannot pass
that string to jgit as an URIish. Problem is that the passed
URI looks like 'file:/C:/a/b.txt' and that we where expecting
double slashes after scheme':'. This fix adds support for this
single-slash file URLs.
Bug: 323571
Change-Id: I866a76a4fcd0c3b58e0d26a104fc4564e7ba5999 Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Mathias Kinzler [Thu, 7 Oct 2010 09:37:16 +0000 (11:37 +0200)]
Add "Pull" command
This is the minimal implementation of a "Pull" command. It does not
have any parameters besides the generic progress monitor and timeout.
It works on the currently checked-out branch and assumes that the
configuration contains the keys "branch.<branch name>.remote" and
"branch.<branch name>.merge" to determine the remote configuration
for the fetch and the remote branch name for the merge.
There where quite some bugs regarding wrong URI parsing. In order
to solve them the parsing has to be refactored. We now have
specialized regexps for 'scheme://host/...', scp URIs and local
file names. Now we can detect problems while parsing 'git://host:/abc' which
was previously not possible.
Bug: 315571
Bug: 292897
Bug: 307017
Bug: 323571
Bug: 317388
Change-Id: If72576576ebb6b9d9dc8b7e51ddd87c9909e8b62 Signed-off-by: Christian Halstrick <christian.halstrick@sap.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
The regular expression which should handle the
user/password part in an URI was potentially
processing too many chars. This led to problems
when user/pwd and port was specified
Change-Id: I87db02494c4b367283e1d00437b1c06d2c8fdd28 Signed-off-by: Christian Halstrick <christian.halstrick@sap.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Fix URIish tests to contain a hostname for git protocol
URIs for the git protocol have to have a hostname.
(see http://www.kernel.org/pub/software/scm/git/docs
/git-clone.html#_git_urls_a_id_urls_a) Some tests tested
URIs like git:/abc.git which is not allowed. Fixed this.
Change-Id: Ia3b8b681ad6592f03b090a874a6e91068a8301fe Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Introduce commented constants for the segments of an URI regex
The regular expressions used to parse URI's are constructed by
concatenating different segments to a big String. Introduce
String constants for these segements and document them.
Change-Id: If8b9dbaaf57ca333ac0b6c9610c3d3a515c540f9 Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Matthias Sohn [Thu, 7 Oct 2010 20:40:40 +0000 (23:40 +0300)]
Fix HTTP tests
Since 858b2c92 we have a HTTP authentication implementation hence
we now get different exception messages when required authentication
headers are not available. This broke the HTTP tests.
Change-Id: Ie08c1ec37e497c2a6f70a75f7c59f0805812a5cc Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Increase core.streamFileThreshold default to 50 MiB
Projects like org.eclipse.mdt contain large XML files about 6 MiB
in size. So does the Android project platform/frameworks/base.
Doing a clone of either project with JGit takes forever to checkout
the files into the working directory, because delta decompression
tends to be very expensive as we need to constantly reposition the
base stream for each copy instruction. This can be made worse by
a very bad ordering of offsets, possibly due to an XML editor that
doesn't preserve the order of elements in the file very well.
Increasing the threshold to the same limit PackWriter uses when
doing delta compression (50 MiB) permits a default configured
JGit to decompress these XML file objects using the faster
random-access arrays, rather than re-seeking through an inflate
stream, significantly reducing checkout time after a clone.
Since this new limit may be dangerously close to the JVM maximum
heap size, every allocation attempt is now wrapped in a try/catch
so that JGit can degrade by switching to the large object stream
mode when the allocation is refused. It will run slower, but the
operation will still complete.
The large stream mode will run very well for big objects that aren't
delta compressed, and is acceptable for delta compressed objects that
are using only forward referencing copy instructions. Copies using
prior offsets are still going to be horrible, and there is nothing
we can do about it except increase core.streamFileThreshold.
We might in the future want to consider changing the way the delta
generators work in JGit and native C Git to avoid prior offsets once
an object reaches a certain size, even if that causes the delta
instruction stream to be slightly larger. Unfortunately native
C Git won't want to do that until its also able to stream objects
rather than malloc them as contiguous blocks.
Change-Id: Ief7a3896afce15073e80d3691bed90c6a3897307 Signed-off-by: Shawn O. Pearce <spearce@spearce.org> Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
Robin Rosenberg [Thu, 9 Sep 2010 22:38:58 +0000 (00:38 +0200)]
Shut up findbugs/protect the shutdownHook in LocalDiskRepositoryTestcase
Singleton references should be protected from multiple threads. As far as we
know this cannot happen as JUnit is used today since we currently don't run
tests in parallel, but now this code will not prevent anyone.
Change-Id: I29109344d2e8025fa2a3ccaf7c2c16469544ce05 Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
Shawn O. Pearce [Sun, 21 Mar 2010 04:04:33 +0000 (21:04 -0700)]
Support HTTP basic and digest authentication
Natively support the HTTP basic and digest authentication methods
by setting the Authorization header without going through the JREs
java.net.Authenticator API. The Authenticator API is difficult to
work with in a multi-threaded server environment, where its using
a singleton for the entire JVM. Instead compute the Authorization
header from the URIish user and pass, if available.
Change-Id: Ibf83fea57cfb17964020d6aeb3363982be944f87 Signed-off-by: Shawn O. Pearce <spearce@spearce.org> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
projects / jgit.git / log