Robin Rosenberg [Tue, 28 Dec 2010 18:01:46 +0000 (19:01 +0100)]
Fix FileSnapShot
We cannot use SystemReader to get the time, unless we do that consistently,
which is harder to do and be sure we are really testing what we want.
Then we need to update our lastRead variable whenever we conclude that
our file is not racily clean according to lastRead. It may well be clean,
but we do not know that until we check the system clock again.
Finally add a test for this class.
Change-Id: I1894b032b9bd359d1b5325e5472d48e372599e4c Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
Shawn O. Pearce [Wed, 22 Dec 2010 22:11:18 +0000 (14:11 -0800)]
Fix ArrayIndexOutOfBoundsException in DirCacheIterator
If the 'TREE' extension contains an invalid subtree that has
been removed, DirCacheIterator still tried to access it due to
an invalid childCnt field within the parent DirCacheTree object.
This is easy for a user to do, they just need to move all files
out of a subdirectory.
For example, the input for the JUnit test case for this bug was
built using the following C Git sequence:
After the last step, the subdirectory a/b is empty, as its only
file was moved into the parent directory. Because of the earlier
`git write-tree` operation, there is a 'TREE' extension present, but
the a and a/b subdirectories have been marked invalid by the rename.
When JGit tried to iterate over the a tree, it tried to correct
childCnt to be zero as a/b no longer exists, but it failed to
update childCnt.
Change-Id: I7a0f78fc48a36b1a83252d354618f6807fca0426 Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce [Wed, 22 Dec 2010 21:39:38 +0000 (13:39 -0800)]
DirCacheIteratorTest: Use newInCore instead of disk
Avoid the dependency on the local filesystem by using only an in-core
DirCache instance. Each test case builds up the index from scratch
anyway through a DirCacheBuilder.
Change-Id: I5decf6bffc3ed35bf1d3e4ad5cc095891c80b772 Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Mathias Kinzler [Mon, 20 Dec 2010 09:35:10 +0000 (10:35 +0100)]
Checkout: fix handling if name does not refer to a local branch
The CheckoutCommand does not handle names other than local branch
names properly; it must detach HEAD if such a name is encountered (for
example a commit ID or a remote tracking branch).
Change-Id: I5d55177f4029bcc34fc2649fd564b125a2929cc4 Signed-off-by: Mathias Kinzler <mathias.kinzler@sap.com> Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
Shawn O. Pearce [Thu, 16 Dec 2010 00:39:28 +0000 (16:39 -0800)]
Remove deprecated WriteTree from tests
These tests doesn't need to use WriteTree anymore. There are
other means of creating tree objects in the repository that aren't
deprecated, so use those instead.
Change-Id: I89cd8ab54c66964a5fddc0a045f1c0f1c7c49055 Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Mathias Kinzler [Mon, 20 Dec 2010 09:21:49 +0000 (10:21 +0100)]
Checkout: expose a CheckoutResult
This is needed by callers to determine checkout conflicts and
possible files that were not deleted during the checkout so that they
can present the end user with a better Exception description and retry
to delete the undeleted files later, respectively.
Mathias Kinzler [Thu, 16 Dec 2010 07:41:36 +0000 (08:41 +0100)]
DirCacheCheckout: fix getToBeDeleted()
This wrongly returns the same as getConflicts() Signed-off-by: Mathias Kinzler <mathias.kinzler@sap.com>
Change-Id: Id37c625458fc5a9b3987f05b684620e24fdfe852
Shawn O. Pearce [Mon, 13 Dec 2010 20:23:07 +0000 (12:23 -0800)]
Extract pack directory last modified check code
Pulling the last modified checking logic out of ObjectDirectory
makes it possible to reuse this code for other files, such as
the $GIT_DIR/config or $GIT_DIR/packed-refs files.
Change-Id: If2f27a89fc3b7adde7e65ff40bbca5d55b98b772 Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce [Mon, 13 Dec 2010 22:18:42 +0000 (14:18 -0800)]
Save StoredConfig after modifications
When the Config is changed, it should be saved back to its local
file. This ensure that a future call to getConfig() won't wipe
out the edits that were just made.
Change-Id: Id46d3f85d1c9b377f63ef861b72824e1aa060eee Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce [Mon, 13 Dec 2010 22:18:05 +0000 (14:18 -0800)]
Reduce calls to Repository.getConfig
Each time getConfig() is called on FileRepository, it checks the
last modified time of both ~/.gitconfig and $GIT_DIR?config. If
$GIT_DIR/config appears to have been modified, it is read back in
from disk and the current config is wiped out.
When mutating a configuration file, this may cause in-memory edits
to disappear. To avoid that callers need to avoid calling getConfig
until after the configuration has been saved to disk.
Unfortunately the API is still horribly broken. Configuration should
be modified only while a lock is held on the configuration file, very
similar to the way a ref is updated via its locking protocol. But our
existing API is really broken for that so we'll have to defer cleaning
up the edit path for a future change.
Change-Id: I5888dd97bac20ddf60456c81ffc1eb8df04ef410 Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce [Wed, 15 Dec 2010 18:17:57 +0000 (10:17 -0800)]
Support GIT_SSH=tortoiseplink
The tortoiseplink command does not understand -batch, even though
it smells like the putty plink command that does use it. Don't add
-batch if GIT_SSH is tortoiseplink.
Change-Id: I638532a02faa2caf8c39d482094e7ff4f4ec7e78 Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce [Wed, 15 Dec 2010 18:17:01 +0000 (10:17 -0800)]
Correct plink -batch option
When GIT_SSH is set to use plink, the correct option name is "-batch"
and not "--batch". This was a typo introduced when we added support
for plink via GIT_SSH.
Change-Id: I391660e38f5d208bba11e3f2a8f25922de2af878 Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Philipp Thun [Tue, 14 Dec 2010 10:31:41 +0000 (11:31 +0100)]
Do not rely on filemode differences in case of symbolic links
When checking whether a file in the working tree has been modified -
WorkingTreeIterator.isModified() - we should not trust the filemode
in case of symbolic links, but check the timestamp and also the
content, if requested. Without this fix symlinks will always be shown
in EGit as modified files on Windows systems.
Change-Id: I367c807df5a7e85e828ddacff7fee7901441f187 Signed-off-by: Philipp Thun <philipp.thun@sap.com>
Shawn O. Pearce [Mon, 13 Dec 2010 18:57:26 +0000 (10:57 -0800)]
Remove remaining uses of FileWriter
FileWriter uses the platform default encoding, which might not
be UTF-8. JGit prefers UTF-8 everywhere for string encodings,
so make the unit tests more predictable by ensuring use of UTF-8.
Change-Id: I75bb9f962ee230b73ca3a942bffd7a8a28674ba5 Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce [Mon, 13 Dec 2010 18:21:04 +0000 (10:21 -0800)]
Simplify NoteParser use of prefix.length()
Sasa pointed out we only ever use the length here, so instead of
holding onto the AbbreviatedObjectId, lets just hold onto the length
as a primitive int.
Change-Id: I2444f59f9fe5ddcaea4a3537d3f1064736ae3215 Signed-off-by: Shawn O. Pearce <spearce@spearce.org> CC: Sasa Zivkov <zivkov@gmail.com>
Shawn O. Pearce [Mon, 13 Dec 2010 18:09:14 +0000 (10:09 -0800)]
Fix HTTP digest authentication
JGit's internal implementation of the HTTP digest authentication
method wasn't conforming to RFC 2617 (HTTP Authentication: Basic
and Digest Access Authentication), resulting in authentication
failures when connecting to a digest protected site.
The code now more accurately matches section 3.2.2 (The Authorization
Request Header) from the standards document.
Change-Id: If41b5c2cbdd59ddd6b2dea143f325e42cd58c395 Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Matthias Sohn [Fri, 10 Dec 2010 20:48:09 +0000 (21:48 +0100)]
File utilities for creating directories
The java.io.File methods for creating directories report failure by
returning false. To ease proper checking of return values provide
utility methods wrapping mkdir() and mkdirs() which throw IOException
on failure.
Also fix the tests to store test data under a trash folder and cleanup
after test.
Change-Id: I09c7f9909caf7e25feabda9d31e21ce154e7fcd5 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
Shawn O. Pearce [Wed, 8 Dec 2010 20:05:48 +0000 (12:05 -0800)]
DiffFormatter: Use IndexDiffFilter to speed up working tree
If DiffFormatter is asked to compare the index to the working tree,
it can go faster by using the cached stat information to compare
the two entries rather than relying on SHA-1 computation alone.
Change-Id: Icb21c15b8279ee8cee382e5e179e0cf8903aee4d Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Mathias Kinzler [Fri, 10 Dec 2010 08:44:51 +0000 (09:44 +0100)]
Rebase: abort on unknown/unsupported command in git-rebase-todo
This is needed to ensure interoperability with the command line: if
the git-rebase-todo file was created manually (by git rebase -i in the
command line), and any commands other than pick are used (reword,
edit, fixup, squash) JGit must abort as it does not understand these
commands yet.
The same is true if an unknown command is found (e.g. due to a typo);
this is the same behavior as shown by the command line.
roberto [Thu, 9 Dec 2010 23:40:34 +0000 (23:40 +0000)]
IndexPack: Remove blob-streaming size threshold
Always use streaming (for SHA-checksum & collision detection)
when indexing whole blobs, regardless of their size.
Positives:
* benefits of bugfix #312868 will apply to all runtimes, without
additional conf for mem-constrained JVMs (5MB huge for some)
* no byte array allocation
(re-uses readBuffer instead of allocating new full-size array)
* mildly better overall performance
(given the usual blob-does-not-need-collision-checking case)
* removes unnecessary code
Negative:
* doubles the disk IO for a blob comparision
(comparitively rare occurance)
I perf-tested a range of threshold sizes against a random selection
of packfiles I found on my harddrive, the results are here:
My interpretation of the results is that the streaming size threshold
isn't beneficial (actually seems to be very slightly detrimental) -so
we should just get rid of it. This tallies with some of the comments
Shawn & I had for the default value of streamFileThreshold in the
review for I862afd4c:
The perf-test code is here: https://gist.github.com/735402
It's a bit scruffy but basically does 10 runs (in randomised order)
for each threshold size on various packfiles, waiting a second
between each pack-indexing to allow GC to catch up. I know it's not
perfect - proper perf testing is hard to do :-)
Philipp Thun [Thu, 9 Dec 2010 19:34:53 +0000 (20:34 +0100)]
Add further unit tests for IndexDiffFilter
This change contains a bunch of unit tests for the newly introduced
IndexDiffFilter. With these tests the code coverage of
IndexDiffFilter.include() is now 100%, i.e. every special case is
tested at least once.
Change-Id: Ib248d1cd16084f9c8e099006af151814c63c5941 Signed-off-by: Philipp Thun <philipp.thun@sap.com>
Shawn O. Pearce [Thu, 9 Dec 2010 18:54:01 +0000 (10:54 -0800)]
Simplify logic in StrategySimpleTwoWayInCore
Sasa and I were reviewing this code today and Sasa pointed out we
can simplify the conflict logic, as the two cases (subtree and file)
are logically identical.
Change-Id: Ie0d40b2dd15605785eff453a846b1d20a2d021fc Signed-off-by: Shawn O. Pearce <spearce@spearce.org> Reviewed-by: Sasa Zivkov <zivkov@gmail.com>
Add a performance optimized variant of the ANY_DIFF filter
If a treewalk walks also over index and the workingtree then the
IndexDiffFilter filter can be used which works much faster then
the semantically equivalent ANY_DIFF filter. This is because this
filter can better avoid computing SHA-1 ids over the content of
working-tree files which is very costly.
This fix will significantly improve the performance of e.g.
EGit's commit dialog.
Change-Id: I2a51816f4ed9df2900c6307a54cd09f50004266f Signed-off-by: Christian Halstrick <christian.halstrick@sap.com> Signed-off-by: Philipp Thun <philipp.thun@sap.com>
Mathias Kinzler [Thu, 9 Dec 2010 15:10:21 +0000 (16:10 +0100)]
Implement rebase --continue and --skip
For --continue, the Rebase command asserts that there are no unmerged
paths in the current repository. Then it checks if a commit is needed.
If yes, the commit message and author are taken from the author_script
and message files, respectively, and a commit is performed before the
next step is applied.
For --skip, the workspace is reset to the current HEAD before applying
the next step.
Includes some tests and a refactoring that extracts Strings in the
code into constants.
Change-Id: I72d9968535727046e737ec20e23239fe79976179 Signed-off-by: Mathias Kinzler <mathias.kinzler@sap.com> Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Shawn O. Pearce [Wed, 8 Dec 2010 03:11:05 +0000 (19:11 -0800)]
IndexDiff: Remove unnecessary changesExist flag
Instead of setting a boolean when a difference record is found, return
false from diff() only if all of the collections are empty. When all
of them are empty, no difference was found.
Change-Id: I555fef37adb764ce253481751071c53ad12cf416 Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce [Wed, 8 Dec 2010 03:08:02 +0000 (19:08 -0800)]
IndexDiff: Use isModified() when comparing index-worktree
The isModified() is more efficient because it can skip over files that
are stat clean, without needing to scan them.
This is useful to efficently work on paths that were already staged
and thus differ between HEAD and the index, but not between the index
and the working tree.
Change-Id: I4418202e612f0571974e0898050d987c6c280966 Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce [Wed, 8 Dec 2010 03:05:07 +0000 (19:05 -0800)]
IndexDiff: Clean up tree-index compare for staged files
When comparing the ObjectIds for two tree entries its faster
to use the raw buffer compares over allocating ObjectIds and
then performing equals on their contents.
However, this also needs to consider the raw modes. It is possible
for a path to change modes but not ObjectId (e.g. making a file
executable), and in this case its still a staged change to report back
to the caller.
Change-Id: I1a267254c04b3273a97f63c71d1e6718cd9d2fa8 Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce [Wed, 8 Dec 2010 02:53:09 +0000 (18:53 -0800)]
IndexDiff: Fix getAssumeUnchanged()
If the caller really needs the list of files that are flagged as
assume-unchanged (aka assume-valid in the DirCache), we should give
them the complete list and not just those that we wrongly identified
as being modified during diff().
This change is necessary because diff() is slightly broken and is
discovering differences on files that it shouldn't have considered.
Change-Id: Ibe464c1a0e51c19dc287a4bc5348b7b07f4d840b Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce [Wed, 8 Dec 2010 02:45:11 +0000 (18:45 -0800)]
IndexDiff: Remove always true not-subtree check
The TreeWalk is configured to be recursive, which means subtrees are
never presented to the application. Therefore the working tree file
mode can never be a subtree/subdirectory at this point in the code.
Change-Id: Ie842ddc147957d09205c0d2ce87b25c566862fd9 Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce [Wed, 8 Dec 2010 02:42:10 +0000 (18:42 -0800)]
IndexDiff: Always use TreeWalk.getPathString()
Instead of asking the individual iterators for their path string, use
the TreeWalk's generic getPathString() method. Its just as fast
because it uses the path of the current matching iterator.
Change-Id: I9b827fbbafce1c78f09d5527cdc64fbe9022a16e Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce [Wed, 8 Dec 2010 02:13:36 +0000 (18:13 -0800)]
Clarify WorkingTreeOptions and filemode usage
To improve runtime performance, caching the WorkingTreeOptions inside
of the Config object using the Config.SectionParser API allows
the WorkingTreeOptions to be accessed more efficiently whenever a
FileTreeIterator is constructed for the Repository.
Instead of passing the filemode handling option into isModified(),
the WorkingTreeIterator should always honor whatever setting has
been configured in this repository, as defined by its own copy of
the WorkingTreeOptions. This simplifies all of the callers as they
no longer need to lookup core.filemode on their own.
A few locations were changed from always using a hardcoded "true"
on the file mode to passing what is actually configured in the
repository. This is a behavior change, but corrects what should be
considered to be bugs as the core.filemode variable wasn't always
being used.
Change-Id: Idb176736fa0dc97af372f1d652a94ecc72fb457c Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce [Tue, 7 Dec 2010 22:05:40 +0000 (14:05 -0800)]
IndexPack: Use streaming for large whole blobs
When indexing large blobs that are stored whole (non-delta form),
avoid allocating the entire blob in memory and instead stream it
through the SHA-1 checksum computation. This reduces the size
of memory required by IndexPack when processing very big blobs,
such as a 500 MiB uncompressable binary.
If the large blob already exists in the local repository, its
contents needs to be compared byte-for-byte after the entire pack
has been indexed, to ensure there isn't an unexpected SHA-1 collision
which may result in later data corruption. This compare is performed
as a streaming compare, again avoiding the large object allocation.
This change doesn't improve on memory utilization for large objects
stored as deltas. The change also doesn't improve handling for
any large commits, trees or annotated tags. There isn't much to
be done here for those objects, because they need to be passed down
to the ObjectChecker as a byte[]. Fortunately it isn't common for
these object types to be that large,
Bug: 312868
Change-Id: I862afd4cb78013ee033d4ec68c067b1774a05be8 Signed-off-by: Shawn O. Pearce <spearce@spearce.org> Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com> CC: Roberto Tyley <roberto.tyley@guardian.co.uk>
Some method parameters in WorkingTreeIterator are never used. Remove
them. Especially the removal of the FS parameter in isModified()
simplifies upcoming performance optimizations.
Change-Id: I7c449589283a4a6b6e23f2586cd784febdca8bcd Signed-off-by: Christian Halstrick <christian.halstrick@sap.com> Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
Jens Baumgart [Wed, 8 Dec 2010 15:20:28 +0000 (16:20 +0100)]
Introduce http test bundle
Introduce a http test bundle to make this functionality available for
EGit tests. A simple http server class is provided. The jetty version
was updated to a version that is also available via p2 (needed in EGit
UI tests).
Change-Id: I13bfc4c6c47e27d8f97d3e9752347d6d23e553d4 Signed-off-by: Jens Baumgart <jens.baumgart@sap.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Shawn O. Pearce [Wed, 8 Dec 2010 00:49:51 +0000 (16:49 -0800)]
Remove empty iterator from TreeWalk
Its confusing that a new TreeWalk() needs to have reset() invoked
on it before addTree(). This is a historical accident caused by
how TreeWalk was abused within ObjectWalk.
Drop the initial empty tree from the TreeWalk and thus remove a
number of pointless reset() operations from unit tests and some of
the internal JGit code.
Existing application code which is still calling reset() will simply
be incurring a few unnecessary field assignments, but they should
consider cleaning up their code in the future.
Change-Id: I434e94ffa43491019e7dff52ca420a4d2245f48b Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce [Tue, 7 Dec 2010 20:45:52 +0000 (12:45 -0800)]
Refactor IndexPack to use InputStream for inflation
By inflating with an InputStream like API, it is possible to stream
through large objects rather than allocating the entire thing as
a byte[]. This change only refactors the inflation code within
IndexPack to use a streaming interface.
Matthias Sohn [Mon, 6 Dec 2010 23:58:19 +0000 (00:58 +0100)]
[findbugs] Do not ignore exceptional return value
java.io.File.delete() reports failure as an exceptional
return value false. Fix the code which silently ignored
this exceptional return value. Also remove some duplicate
deletion helper methods.
Change-Id: I80ed20ca1f07a2bc6e779957a4ad0c713789c5be Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Mon, 6 Dec 2010 23:45:58 +0000 (00:45 +0100)]
Provide file utilities for file deletion
Provide file helper methods in a reusable utility class to
replace many local implementations. java.io.File has some
methods reporting failure by returning false. We prefer to
throw IOException on failure so that callers can't forget
checking the return value.
Change-Id: I430c77b5d2cffcf8b47584326ad4817a7291845e Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Jens Baumgart [Mon, 6 Dec 2010 12:40:07 +0000 (13:40 +0100)]
LockFile.commit: retry renaming
Currently the following can happen in LockFile.commit: deletion of the
original file succeeds but renaming fails afterwards. In this case the
original file (e.g. branch file in refs/heads) is lost.
To workaround the issue the same retry logic as for file deletion is
applied to file renaming.
Shawn O. Pearce [Sat, 4 Dec 2010 00:14:46 +0000 (16:14 -0800)]
Abstract SSH setup to support GIT_SSH
In order to honor GIT_SSH the TransportGitSsh class needs to run the
process named by the GIT_SSH environment variable and use that as the
pipes for connectivity to the remote peer. Refactor the current
transport code to support a different type of pipe connectivity, so we
can later add GIT_SSH.
Bug: 321062
Change-Id: I9d8ee1a95f1bac5013b33a4a42dcf1f98f92172f Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce [Fri, 3 Dec 2010 20:38:31 +0000 (12:38 -0800)]
Remove result id from CommitBuilder, TagBuilder
These objects don't need to be updated with the resulting ObjectId of
the formatted content, callers can get that from the ObjectInserter on
their own.
Change-Id: Idc5f097de9f7beafc5e54e597383d82daf9d7db4 Signed-off-by: Shawn O. Pearce <spearce@spearce.org> Reviewed-by: Chris Aniszczyk <caniszczyk@gmail.com>