Shawn O. Pearce [Tue, 22 May 2012 23:45:06 +0000 (16:45 -0700)]
Batch reference updates together for storage
clone, fetch and push can all update multiple references in a single
command invocation. Rather than performing sequential iteration
of each reference change inside of the application code, push this
down into the reference database where the implementation can take
advantage of the batch size and optimize itself.
For the local filesystem implementation the obvious optimization
is to write a packed-refs file when the repository is completely
empty. The initial clone, fetch or push into the destination may
have hundreds of new references. Writing all of these as loose
files is not efficient. This optimization is not implemented in
this commit and is left as an exercise for the reader to supply
in a future commit to JGit.
To make the API changes simple, define the BatchRefUpdate type and
implementation using the existing sequential behavior.
Shawn O. Pearce [Tue, 22 May 2012 23:22:41 +0000 (16:22 -0700)]
Expose ReceiveCommand.updateType to check for UPDATE_NONFASTFORWARD
When a command's type is UPDATE, JGit might not yet be sure if it
is a fast-forward or not. Expose a utility method to compute the
exact type by performing the merge base test, allowing the type
to be switched to UPDATE_NONFASTFORWARD if old ObjectId is not
contained in new ObjectId.
BaseReceivePack already does this test when validating the incoming
command list, so provide a package level backdoor to set the type
and avoid needing to redo the merge test later.
Shawn O. Pearce [Tue, 22 May 2012 22:59:53 +0000 (15:59 -0700)]
Reject non-fast-forwards earlier in BaseReceivePack
If BaseReceivePack has setAllowNonFastForwards(false) configured
(such as by receive.denynonfastforwards), automatically reject
any command that attempts a non-fast-forward update before it goes
further in processing.
This matches with other checks in validateCommands(), such as the
early failure of delete attempts when isAllowDeletes() is false.
Matthias Sohn [Mon, 21 May 2012 18:51:10 +0000 (20:51 +0200)]
Fix JGit source bundle packaging
Due to Tycho bug 368596 we didn't package the correct JGit source bundle
anymore. Instead of sources it contained the binary bundle. As a
workaround fall back to use packaging type eclipse-update-site which
doesn't seem to have this problem.
Bug: 368596
Bug: 379402
Change-Id: I3192dbd00b51e6ee6596d2301050b2a6f7028e3b Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Robin Rosenberg [Mon, 21 May 2012 17:48:40 +0000 (10:48 -0700)]
Smudge index entries on first write (too), as well when reading
That happens when the index and a new file is created within the same
second and becomes a problem if we then modify the newly created file
within the same second after adding it to the index. Without smudging
JGit will, on later reads, think the file is unchanged.
The accompanying test passed with the smuding on read.
Mikael Karlsson [Fri, 18 May 2012 20:07:50 +0000 (22:07 +0200)]
Add user friendly output from jgit commit command
Instead of printing the stack trace from the JGitInternalException
thrown from CommitMessage.call(), print just the exception
message, using the Die exception.
Dave Borowitz [Thu, 3 May 2012 23:44:53 +0000 (16:44 -0700)]
Factor a base class out of ReceivePack
We are working on a publish/subscribe based git protocol, and we want to
reuse several parts of the ReceivePack-like code for reading commands
and processing a pack. In this new implementation, the connection
management will be very different, in particular, there may be multiple
packs received on a single open connection. So, hoist out as much as we
can from ReceivePack, mostly just leaving behind the single-connection
version in that class.
Matthias Sohn [Mon, 7 May 2012 21:45:24 +0000 (23:45 +0200)]
Add missing @since tags to mark API added in 2.0
Change-Id: I0a86ce0e393dfde9bb27f0b29e036e76c856396e Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Signed-off-by: Chris Aniszczyk <zx@twitter.com>
Matthias Sohn [Mon, 16 Apr 2012 15:40:43 +0000 (17:40 +0200)]
Update tycho version to 0.14.1
Tycho 0.14.1 enforces that artifactId and feature/bundle SymbolicName
must match, hence follow recommended migration practice and change
groupId of feature projects in order not to change the feature names as
this would require a feature migration.
Also migrate the p2 repository build from the deprecated packaging type
eclipse-update-site to packaging type eclipse-repository.
Change-Id: I99fc6c2bbb20efbd6514bdd9940f3535707c21bc Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Robin Rosenberg [Mon, 23 Apr 2012 20:37:50 +0000 (22:37 +0200)]
Validate paths during DirCheckout
DirCacheCheckout and CanonicalTreeParser cooperate. CanonicalTreeParser
can detect malformed, potentially malicious tree entries and sets a
flag, while DirCacheCheckout refuses to work with such paths.
Malicious tree entries are ".", "..", ".git" (case insensitive), any
name containing '/' and (on Windows '\') and also (on Windows)
any paths ending in a combination of '.' or space or containing a ':'.
We also forbid all special names like "con" etc on Windows.
Some of the test can execute on any platform by enabling partial
platform emulation.
A new runtime exception, InvalidPathException, is introduced. For
backwards compatibility it extends InvalidArgumentException.
Change-Id: I86199105814b63d4340e5de0e471d0da6b579ead Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Robin Rosenberg [Mon, 16 Apr 2012 22:18:10 +0000 (00:18 +0200)]
Enable large file support
Allow adding files with size over 2 GB. The drawback is that the tests
for huge file support adds roughly 10 minutes of execution time.
For that reason we @Ignore the test in the standard test execution.
Shawn O. Pearce [Tue, 27 Mar 2012 19:32:16 +0000 (15:32 -0400)]
Index config section and subsection names in one pass
Instead of indexing the subsection names on each request for a given
section name, index both the section and subsection names in a single
scan through the entry list. This should improve lookup time for
reading the section names out of the configuration, especially for the
url.*.insteadof type of processing performed in RemoteConfig.
Markus Duft [Mon, 2 Apr 2012 05:27:19 +0000 (07:27 +0200)]
Don't use java.nio channel for file size determination
Java NIO has some problems (like files closing unexpectedly because the
thread was interrupted). To avoid those problems, don't use a NIO
channel to determine the size of a file, but rather ask the File itself.
We have to be prepared to handle wrong/outdated information in this case
too, as the inode of the File may change between opening and determining
file size.
Matthias Sohn [Sun, 1 Apr 2012 23:20:31 +0000 (01:20 +0200)]
Fix tests for Ant task "git-clone"
If project.init() isn't called GitCloneTaskTest fails when started from
Eclipse, according to [1] calling init() is necessary to properly
initialize the Ant project programmatically.
Always set the destination folder in order to ensure that all test
resources are created under the project's target folder and do not
pollute the project's source tree with test data.
Shawn O. Pearce [Wed, 28 Mar 2012 14:12:20 +0000 (10:12 -0400)]
Fix loading packed objects >2G
Parsing the size from a packed object header was incorrectly computing
the total inflated length when the length exceeded the range of a Java
int. The next 7 bits of size information was shifted left as an int
using a shift of 25 bits, placing the higher bits of the size into the
sign position. When this size was extended to a long to be added to
the current size accumulator the size went negative, resulting in
NegativeArraySizeException being thrown.
Fix all places where this particular pattern of code is used to read a
pack size field, or a binary delta header, as they both use the same
variable length encoding scheme.
Shawn O. Pearce [Tue, 27 Mar 2012 15:49:13 +0000 (11:49 -0400)]
Sort Config entries and use O(log N) lookup
Decrease running time for getStringList (and all other get methods) by
looking for configuration entries using binary search rather than
linear search through the configuration file.
Configuration lines are sorted by section, subsection, name in a
sorted list whenever the snapshot is rebuilt. Binary search is used to
locate an index in the middle of the values, then walk backwards to
find the first value in the range.
Given a configuration of file of 5000 distinct section/subsection/name
triplets (e.g. a Gerrit Code Review project.config configuration file
with 5000 unique access control rules), this new code is faster to
lookup each rule individually using getStringList():
old setStringList() 194 usec avg
getStringList() 196 usec avg
new setStringList() 188 usec avg
getStringList() 24 usec avg
Shawn O. Pearce [Tue, 27 Mar 2012 14:43:02 +0000 (10:43 -0400)]
Extract inner classes from Config
The Config class is getting very large. Extract two of its inner
classes into new top level types to reduce the size of Config.
Rename them slightly in the process.
Dave Borowitz [Mon, 26 Mar 2012 17:19:40 +0000 (10:19 -0700)]
Allow RepositoryResolver to throw ServiceMayNotContinueException
Implementations may want to send an error message to the user, which
doesn't really fit with any of the existing exception types.
ServiceMayNotContinueException, on the other hand, is documented as
always containing a user-visible error string, so use that.
Modify the git and HTTP transport mechanisms to properly relay this
message to the end user.
Kevin Sawicki [Wed, 21 Mar 2012 17:50:16 +0000 (10:50 -0700)]
Add command support for dropping a stashed commit
This extracts the logic for writing to the reflog from
RefDirectory into a new ReflogWriter class. This class
creates a public API for writing reflog entries similar
to ReflogReader for reading reflog entries.
The new command supports rewriting the stash's log to remove
a configured entry followed by updating the stash ref to
the value at the bottom of the newly written log.
Change-Id: Icfcbc70e838666769a742a94196eb8dc9c7efcc7 Signed-off-by: Chris Aniszczyk <zx@twitter.com>
Kevin Sawicki [Mon, 12 Mar 2012 17:00:52 +0000 (10:00 -0700)]
Compare repository format version as parsed long
This allows repositoryies with a missing repositoryformatversion
config value to be successfully opened but still throws exceptions
when the value is a non-long or greater than zero.
git-core attempts to parse this config value as a long as well
and defaults to 0 if the value is missing.
Bug: 368697
Change-Id: I4a93117afca37e591e8e0ab4d2f2eef4273f0cc9 Signed-off-by: Chris Aniszczyk <zx@twitter.com>
Shawn O. Pearce [Thu, 15 Mar 2012 14:57:56 +0000 (07:57 -0700)]
Clarify the purpose of ObjectInserter.buffer()
Recently Robin tried to increase the size of the buffer used by
ObjectInserter to fix a bug in the InputStream that handles AutoCRLF.
The purpose of this buffer is NOT to make a random InputStream work
correctly by passing it a larger buffer during read(byte[],int,int).
Clarify the Javadoc on the buffer() method to reduce the risk
someone tries to abuse it again.
While we are here, modify the method to load the field into a local
variable before returning. This should cut down 1 field load during
the common case of the buffer being already allocated.
Change-Id: Ic6898530d10fcd7e59f90397117a4a0d97e1f031 Signed-off-by: Chris Aniszczyk <zx@twitter.com>
Auto CRLF isn't special enough to be screwing around with the buffers
used for raw byte processing of the ObjectInserter API. If it needs a
buffer to process a file that is bigger than the buffer allocated by
an ObjectInserter, it needs to do its own buffer management.
Kevin Sawicki [Wed, 14 Mar 2012 00:01:42 +0000 (17:01 -0700)]
Only unstash files changed when originally stashed
Previously a DirCacheCheckout was done using a merge tree reflecting
the state of the repository when the stash was originally done.
This was wrong since unstashing after making subsequent commits
would undo changes already committed by checking out entries from
an outdated tree.
The new approach is to scan for conflicts initially using a 6-way
tree walk that contains the trees for the stashed HEAD, stashed
index, stashed working directory, current HEAD, current index, and
current working directory. Then perform a subsequent scan of the
stashed HEAD, index, and working directory trees and apply all
the stashed differences to the current index and working directory.
Bug: 372882
Change-Id: Ica65f162132c00a16964e838de66fc8b5cd0b0aa Signed-off-by: Chris Aniszczyk <zx@twitter.com>
Shawn O. Pearce [Tue, 13 Mar 2012 18:10:33 +0000 (11:10 -0700)]
Enable smart HTTP transport to place EOF at end of pack
When fetching over smart HTTP the InputStream that gets fed into
a PackParser doesn't really support EOF at the end of the pack. It
instead tries to make a new HTTP request, which fails because there
is no request body currently buffered by the client.
Make EOF work correctly on the end of an HTTP derived InputStream
for the pack by denoting no more requests are expected as the higher
level code is now consuming the pack (or side-band embedded pack).
Smart HTTP support doesn't automatically enqueue execute support onto
the end of the UnionInputStream, which allows the UnionInputStream
to correctly reflect EOF when the HTTP response is consumed.
* changes:
cleanup: Silence an unused-parameter warning
cleanup: Get rid of some unused-warnings
cleanup: Remove unused parameter in ConsoleCredentialsProvider
cleanup: Drop unused parameter on DhtPackParser
cleanup: Remove unneeded parameter to private method in RefUpdateTest
cleanup: Remove unnecessary @SuppressWarnings
Markus Duft [Mon, 12 Mar 2012 13:54:56 +0000 (14:54 +0100)]
Fix compilation error due to missing javadoc.
Another change introduced this problem. With default JGit project
settings there is a compilation problem as javadoc problems are set to
Error in the project preferences.
Kevin Sawicki [Sat, 10 Mar 2012 18:26:10 +0000 (10:26 -0800)]
Use object id provided by working tree iterator for submodules
The working tree iterator now supports providing an object id
for submodule entries and this value should be used instead
of recomputing it again in AddCommand.
Kevin Sawicki [Sat, 10 Mar 2012 18:02:58 +0000 (10:02 -0800)]
Keep submodules in index that are missing in working directory
Submodules present in the index but missing from the working
directory should not be staged for deletion when AddCommand
is called with the update flag set to true.
This mirrors the behavior of CGit. Submodules can still be
staged for deletion by running by using the RmCommand.
Markus Duft [Fri, 9 Mar 2012 12:36:53 +0000 (13:36 +0100)]
Fix rebase > continue when nothing left to commit.
If after resolving all conflicts nothing is left to commit, return
an according result, so that downstreams (EGit, ...) can behave like
cgit, and display a nice message informing the user.
Currently, EGit displays a "HEAD advanced fast forward" message, which
is absolutely not helpful at all.
This is the basic API revamping required to get that state communicated
to the outside world (EGit).
Shawn O. Pearce [Thu, 8 Mar 2012 03:51:12 +0000 (19:51 -0800)]
Use readString() to trim trailing LF from first line
The fetch-pack/upload-pack stream usually has an LF at the
end of the first "want" line. Trim this when checking to
see if side-band or side-band-64k was used.
Perform the same trim for send-pack/receive-pack, as it is
harmless in this context to ignore an LF just before doing
an error report.
* changes:
Try to send HTTP error messages over sideband
Extract the capability parsing logic in {Upload,Receive}Pack
Make capability strings in BasePack{Fetch,Push}Connection public
Fix a typo in "capabilities" in ReceivePack
Dave Borowitz [Wed, 7 Mar 2012 19:56:56 +0000 (11:56 -0800)]
Try to send HTTP error messages over sideband
When a client POSTs to /git-{upload,receive}-pack, the first line
includes their client capabilities. As soon as the C git client sends
side-band(-64k), it goes into a state where it chokes on data not sent
in a valid sideband channel.
GitSmartHttpTools.sendError() is called early in the request, likely
before a {Upload,Receive}Pack handler is assigned or, even so, before it
has read the request. In some cases we must read the first line manually
within sendError() to tell whether sideband is needed.
Tomasz Zarna [Tue, 6 Mar 2012 21:43:25 +0000 (22:43 +0100)]
Remove ambiguous CheckoutConflictException
Checkout command should throw o.e.j.api.errors.CheckoutConflictException
which is a GitAPIException not o.e.j.errors.CheckoutConflictException.
PullCommand should rethrow the API exception as a JGitInternalException.
Bug: 356922
Change-Id: I865c4905997d9834c85a97fbe7287604daf99075 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
projects / jgit.git / log