We didn't skip the correct number of bytes when we skipped over an
unrecognized but optional dircache extension.  We missed skipping
the 8 byte header that makes up the extension's name and length.

We also didn't include the skipped extension's payload as part of
our index checksum, resuting in a checksum failure when the index
was done reading.  So ensure we always scan through a skipped
section and include it in the checksum computation.

Add a test case for a currently unsupported index extension, 'ZZZZ',
to verify we can still read the DirCache object even though we
don't know what 'ZZZZ' is supposed to mean.

Bug: 301287
Change-Id: I4bdde94576fffe826d0782483fd98cab1ea628fa
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>

This test doesn't actually depend upon the large data set we have
in the RepositoryTestCase, so drop that from the dependency and
use the more simple LocalDiskRepositoryTestCase instead.

Change-Id: I0fd4affe1dd5ec86e8c3253db42df11d3b612e36
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>

When running jgit from inside Eclipse (e.g. rightclick on project
org.eclipse.jgit.pgm and select Run as->Java application) no commands
are found. This is because the commands are loaded from a resource file
/META-INF/services/org.eclipse.jgit.pgm.TextBuiltin and this file is
not anymore on the classpath.
I fixed this by modifying .classpath to contain the META-INF directory.

Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>

If the repository is empty, we have no HEAD branch, which means we
can't test to see if the HEAD is detached and should be advertised
as a .have line.

Change-Id: I6e85f836e7db057cede812d0d6c1aecbd6cbe6c5
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>

The new plugin contains the bulk of the logic to scan a Git repository,
and query IPZilla, in order to produce an XML formatted IP log for the
requested revision of any Git based project.  This plugin is suitable
for embedding into a servlet container, or into the Eclipse workbench.

The command line pgm package knows how to invoke this plugin through
the eclipse-iplog subcommand, permitting storage of the resulting
log as a local XML file.

Change-Id: If01d9d98d07096db6980292bd5f91618c55d00be
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>

The unsetSection method can be used to delete an entire configuration
block, such as a [branch ""] or [remote ""] section in a file.

Change-Id: I93390c9b2187eb1b0d51353518feaed83bed2aad
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>

Change-Id: I55b42bca435c538a8c5534c68813568a2188b55b
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

Ensure the background Jetty threads have been able to write the
request log record before the JUnit thread tries to read the set
of requests back.  This wait is necessary because the JUnit thread
may be able to continue as soon as Jetty has finished writing
the response onto the socket, and hasn't necessarily finished the
post-response logging activity.

By using a semaphore with a fixed number of resources, and using
one resource per request, but all of them when we want to read the
log, we implement a simple lock that requires there be no active
requests when we want to get the log from the JUnit thread.

Change-Id: I499e1c96418557185d0e19ba8befe892f26ce7e4
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>

Config was confusing the following two variables when writing the
file back to text format:

  [my]
    empty =
    enabled

When parsed, we say that my.empty has 1 value, null, and my.enabled
is an empty string value that in boolean context should be evaluated
as true.

Saving this configuration file back to text format was ignoring the
null value for my.empty, producing a completely different file than
what Config read:

  [my]
    empty
    enabled

Instead handle the writing differently to ensure the original format
is output.  New tests cases cover the expected behavior and return
values from accessor methods.

Change-Id: Id37379ce20cb27e3330923cf989444dd9f2bdd96
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>

We don't want to use the JRE cache when fetching content.

Change-Id: Id76f3e618967c98ed4fbc47a1a2a9e77acbe41ab
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>

If remote.name.uploadpack or .receivepack is misconfigured and points
to a non-existent command on the remote system, we should receive back
exit status 127.  Report this case specially with the command we used
so the user knows what is going.

Bug: 293703
Change-Id: I7504e7b6238d5d8e698d37db7411c4817a039d08
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>

Annotated tags created with C Git versions before the introduction
of c818566 ([PATCH] Update tags to record who made them, 2005-07-14),
do not have a "tagger" line present in the object header.  This line
did not appear in C Git until v0.99.1~9.

Ancient projects such as the Linux kernel contain such tags, for
example Linux 2.6.12 is older than when this feature first appeared
in C Git.  Linux v2.6.13-rc4 in late July 2005 is the first kernel
version tag to actually contain a tagger line.

It is therefore acceptable for the header to be missing, and for
the RevTag.getTaggerIdent() method to return null.

Since the Javadoc for getTaggerIdent() already explained that the
identity may be null, we just need to test that this is true when
the header is missing, and allow the ObjectChecker to pass anyway.

Change-Id: I34ba82e0624a0d1a7edcf62ffba72260af6f7e5d
See: http://code.google.com/p/gerrit/issues/detail?id=399
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>

Technically our project name is "JGit", not "Java Git".  In fact
there is already another project called "JavaGit" (no space) that we
don't want to become confused with.  Ensure we always call ourselves
"JGit" in user visible assets, like the bundle name.

Other Eclipse products list their provider as "Eclipse.org",
not "eclipse.org".  So list ourselves that way in all of our
plugin.properties files.

Change-Id: Ibcea1cd6dda2af757a8584099619fc23b7779a84
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>

Change-Id: I45caafbad4daac827f661d38c7f719e9b5511dd4
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>

The Jetty components are not available as part of Eclipse, but a
P2 packaged version can be found via [1] for Eclipse 3.5 and newer.

[1] http://wiki.eclipse.org/Jetty-OSGi_SDK

Change-Id: Ibd5930bb9fc9589125876ca50c52e58bd31b051c
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>

* ref-abstract:
  Optimize RefAdvertiser performance by avoiding sorting
  branch: Add -m option to rename a branch
  Replace writeSymref with RefUpdate.link
  Rewrite reference handling to be abstract and accurate
  Create new RefList and RefMap utility types

Change-Id: If43aacf5aa4013edbd0a6e84d84c4f9e94de5be0

Don't copy and sort the set of references if they are passed through
in a RefMap or a SortedMap using the key's natural sort ordering.
Either map is already in the order we want to present the items
to the client in, so copying and sorting is a waste of local CPU
and memory.

Change-Id: I49ada7c1220e0fc2a163b9752c2b77525d9c82c1
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>

Change-Id: I7cf8e43344eaf301592fba0c178e04daad930f9a
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>

By using RefUpdate for symbolic reference creation we can reuse
the logic related to updating the reflog with the event, without
needing to expose something such as the legacy ReflogWriter class
(which we no longer have).

Applications using writeSymref must update their code to use the
new pattern of changing the reference through the updateRef method:

    String refName = "refs/heads/master";
    RefUpdate u = repository.updateRef(Constants.HEAD);
    u.setRefLogMessage("checkout: moving to " + refName, false);
    switch (u.link(refName)) {
    case NEW:
    case FORCED:
    case NO_CHANGE:
        // A successful update of the reference
        break;
    default:
        // Handle the failure, e.g. for older behavior
        throw new IOException(u.getResult());
    }

Change-Id: I1093e1ec2970147978a786cfdd0a75d0aebf8010
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>

This commit actually does three major changes to the way references
are handled within JGit.  Unfortunately they were easier to do as
a single massive commit than to break them up into smaller units.

Disambiguate symbolic references:
---------------------------------

  Reporting a symbolic reference such as HEAD as though it were
  any other normal reference like refs/heads/master causes subtle
  programming errors.  We have been bitten by this error on several
  occasions, as have some downstream applications written by myself.

  Instead of reporting HEAD as a reference whose name differs from
  its "original name", report it as an actual SymbolicRef object
  that the application can test the type and examine the target of.

  With this change, Ref is now an abstract type with different
  subclasses for the different types.

  In the classical example of "HEAD" being a symbolic reference to
  branch "refs/heads/master", the Repository.getAllRefs() method
  will now return:

      Map<String, Ref> all = repository.getAllRefs();
      SymbolicRef HEAD = (SymbolicRef) all.get("HEAD");
      ObjectIdRef master = (ObjectIdRef) all.get("refs/heads/master");

      assertSame(master,               HEAD.getTarget());
      assertSame(master.getObjectId(), HEAD.getObjectId());

      assertEquals("HEAD",              HEAD.getName());
      assertEquals("refs/heads/master", master.getName());

  A nice side-effect of this change is the storage type of the
  symbolic reference is no longer ambiguous with the storge type
  of the underlying reference it targets.  In the above example,
  if master was only available in the packed-refs file, then the
  following is also true:

      assertSame(Ref.Storage.LOOSE,  HEAD.getStorage());
      assertSame(Ref.Storage.PACKED, master.getStorage());

  (Prior to this change we returned the ambiguous storage of
   LOOSE_PACKED for HEAD, which was confusing since it wasn't
   actually true on disk).

  Another nice side-effect of this change is all intermediate
  symbolic references are preserved, and are therefore visible
  to the application when they walk the target chain.  We can
  now correctly inspect chains of symbolic references.

  As a result of this change the Ref.getOrigName() method has been
  removed from the API.  Applications should identify a symbolic
  reference by testing for isSymbolic() and not by using an arcane
  string comparsion between properties.

Abstract the RefDatabase storage:
---------------------------------

  RefDatabase is now abstract, similar to ObjectDatabase, and a
  new concrete implementation called RefDirectory is used for the
  traditional on-disk storage layout.  In the future we plan to
  support additional implementations, such as a pure in-memory
  RefDatabase for unit testing purposes.

Optimize RefDirectory:
----------------------

  The implementation of the in-memory reference cache, reading, and
  update routines has been completely rewritten.  Much of the code
  was heavily borrowed or cribbed from the prior implementation,
  so copyright notices have been left intact as much as possible.

  The RefDirectory cache no longer confuses symbolic references
  with normal references.  This permits the cache to resolve the
  value of a symbolic reference as late as possible, ensuring it
  is always current, without needing to maintain reverse pointers.

  The cache is now 2 sorted RefLists, rather than 3 HashMaps.
  Using sorted lists allows the implementation to reduce the
  in-memory footprint when storing many refs.  Using specialized
  types for the elements allows the code to avoid additional map
  lookups for auxiliary stat information.

  To improve scan time during getRefs(), the lists are returned via
  a copy-on-write contract.  Most callers of getRefs() do not modify
  the returned collections, so the copy-on-write semantics improves
  access on repositories with a large number of packed references.

  Iterator traversals of the returned Map<String,Ref> are performed
  using a simple merge-join of the two cache lists, ensuring we can
  perform the entire traversal in linear time as a function of the
  number of references: O(PackedRefs + LooseRefs).

  Scans of the loose reference space to update the cache run in
  O(LooseRefs log LooseRefs) time, as the directory contents
  are sorted before being merged against the in-memory cache.
  Since the majority of stable references are kept packed, there
  typically are only a handful of reference names to be sorted,
  so the sorting cost should not be very high.

  Locking is reduced during getRefs() by taking advantage of the
  copy-on-write semantics of the improved cache data structure.
  This permits concurrent readers to pull back references without
  blocking each other.  If there is contention updating the cache
  during a scan, one or more updates are simply skipped and will
  get picked up again in a future scan.

  Writing to the $GIT_DIR/packed-refs during reference delete is
  now fully atomic.  The file is locked, reparsed fresh, and written
  back out if a change is necessary.  This avoids all race conditions
  with concurrent external updates of the packed-refs file.

  The RefLogWriter class has been fully folded into RefDirectory
  and is therefore deleted.  Maintaining the reference's log is
  the responsiblity of the database implementation, and not all
  implementations will use java.io for access.

  Future work still remains to be done to abstract the ReflogReader
  class away from local disk IO.

Change-Id: I26b9287c45a4b2d2be35ba2849daa316f5eec85d
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>

Translate the version qualifier using maven-antrun-plugin since we want
manifest-first and currently cannot rely on Tycho for the JGit build.

Introduce property for Eclipse p2 repository to enable builds against
other Eclipse versions.

Change-Id: I62c4e77ae91fe17f56c5a5338d53828d4e225395
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

These types can be used by RefDatabase implementations to manage
the collection.

A RefList stores items sorted by their name, and is an immutable
type using copy-on-write semantics to perform modifications to
the collection.  Binary search is used to locate an existing item
by name, or to locate the proper insertion position if an item does
not exist.

A RefMap can merge up to 3 RefList collections at once during its
entry iteration, allowing items in the resolved or loose RefList
to override items by the same name in the packed RefList.

The RefMap's goal is O(log N) lookup time, and O(N) iteration time,
which is suitable for returning from a RefDatabase.  By relying on
the immutable RefList we might be able to make map construction
nearly constant, making Repository.getAllRefs() an inexpensive
operation if the caches are current.  Since modification is not
common, changes require up to O(N + log N) time to copy the internal
list and collapse or expand the list's array.  As most changes
are made to the loose collection and not the packed collection,
in practice most changes would require less than the full O(N)
time, due to a significantly smaller N in the loose list.

Almost complete test coverage is included in the corresponding
unit tests.  A handful of methods on RefMap are not tested in this
change, as writing the proper test depends on a future refactoring
of how the Ref class represents symbolic reference names.

Change-Id: Ic2095274000336556f719edd75a5c5dd6dd1d857
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>

On Windows systems, file system lookup is a slow operation, so
checking each object if it exists during indexing (after receiving
the pack) could take a siginificant time. This patch introduces
CachedObjectDirectory that pre-caches lookup results.

Bug: 300397
Change-Id: I471b93f9bb3ee173eb37cae1d75e9e4eb49985e7
Signed-off-by: Constantine Plotnikov <constantine.plotnikov@gmail.com>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>

Change-Id: I2aa1b2e2e08108deec051594ca8ac3992e271f0f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

* changes:
  Introduce a named constant for the ".git" directory extension
  Introduce a named constant for the .git directory.

Change-Id: Icfe9205994c6810fcd880054a586e9eef29df9a1
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>

Not all occurrences of ".git" are replaced by this constant, only
those where it actually refers to the directory with that name, i.e
not the ".git" directory suffix.

Asserts and comment are also excluded from replacement.

Change-Id: I65a9da89aedd53817f2ea3eaab4f9c2bed35d7ee
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>

As discussed on the jgit-dev list here:
http://dev.eclipse.org/mhonarc/lists/egit-dev/msg00654.html

- Define a separate JGit feature.
- As of now create a separate JGit update site and zip it.

Change-Id: Ie4026f15f6250c4933dccf6f31b5009b90c036bc
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

No Eclipse support for this project is provided, because the
Jetty project does not publish a complete P2 repository.

Change-Id: Ic5fe2e79bb216e36920fd4a70ec15dd6ccfd1468
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>

The dumb HTTP transport needs to download the HEAD ref and
resolve it manually if HEAD does not appear in info/refs.

Its typically for it to not be in the info/refs file.

Change-Id: Ie2a58fdfacfeee530b10edb433b8f98c85568585
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>

During fetch over http:// clients now try to take advantage of
the info/refs?service=git-upload-pack URL to determine if the
remote side will support a standard upload-pack command stream.
If so each block of 32 have lines is sent in one POST request,
prefixed by all of the 'want' lines and any previously discovered
common bases as 'have' lines.

During push over http:// clients now try to take advantage of
the info/refs?service=git-receive-pack URL to determine if the
remote side will support a standard receive-pack command stream.
If so, commands are sent along with their pack in a single HTTP
POST request.

Bug: 291002
Change-Id: I8c69b16ac15c442e1a4c3bd60b4ea1a47882b851
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>

Clients can request smart fetch support by examining the info/refs URL
with the service parameter set to the magic git-upload-pack string:

  GET /$GIT_DIR/info/refs?service=git-upload-pack HTTP/1.1

The response is formatted with the upload pack capabilities, using
the standard packet line formatter.  A special header line is put
in front of the standard upload-pack advertisement to let clients
know the service was recognized and is supported.

If the requested service is disabled an authorization status code is
returned, allowing the user agent to retry once they have obtained
credentials from a human, in case authentication is required by
the configured UploadPackFactory implementation.

Change-Id: Ib0f1a458c88b4b5509b0f882f55f83f5752bc57a
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>

Clients can request smart push support by examining the info/refs URL
with the service parameter set to the magic git-receive-pack string:

  GET /$GIT_DIR/info/refs?service=git-receive-pack HTTP/1.1

The response is formatted with the receive pack capabilities, using
the standard packet line formatter.  A special header block is put
in front of the standard receive-pack advertisement to let clients
know the service was recognized and is supported.

If the requested service is disabled an authorization status code is
returned, allowing the user agent to retry once they have obtained
credentials from a human, in case authentication is required by
the configured ReceivePackFactory implementation.

Change-Id: Ie4f6e0c7b68a68ec4b7cdd5072f91dd406210d4f
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>

This is a simple HTTP server that provides the minimum server side
support required for dumb (non-git aware) transport clients.

We produce the info/refs and objects/info/packs file on the fly
from the local repository state, but otherwise serve data as raw
files from the on-disk structure.

In the future we could better optimize the FileSender class and the
servlets that use it to take advantage of direct file to network
APIs in more advanced servlet containers like Jetty.

Our glue package borrows the idea of a micro embedded DSL from
Google Guice and uses it to configure a collection of Filters
and HttpServlets, all of which are matched against requests using
regular expressions.  If a subgroup exists in the pattern, it is
extracted and used for the path info component of the request.

Change-Id: Ia0f1a425d07d035e344ae54faf8aeb04763e7487
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>

Change-Id: Iaa331a476e28cf2880df5607de36bc9f67d041df
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>

By making this class and its methods public, and the actual writing
abstract, we can reuse this code for other formats like writing an
info/refs file for HTTP transports.

Change-Id: Id0e349c30a0f5a8c1527e0e7383b80243819d9c5
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>

If biDirectionalPipe is false UploadPack does not start out with
the advertisement but instead assumes it should read one block of
want/have lines, process that, and write the ACK/NAKs out.

This means it only is doing one read through the input followed by
one write to the output, which fits with the HTTP request processing
model, and any other type of RPC system.

Change-Id: Ia9f7c46ee556f996367180f15d2caa8572cdd59f
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>

If biDirectionalPipe is false ReceivePack does not start out with the
advertisement but instead assumes it should read the command set once,
process that, and write the status report out.  This means it only is
doing one read through the input followed by one write to the output,
which fits with the HTTP request processing model, and any other type
of RPC system... assuming that the payload for input can be a very big
entity like the command stream followed by the pack file.

Change-Id: I6f31f6537a3b7498803a8a54e10b0622105718c1
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>

Later we are going to add support for smart HTTP, which requires us to
buffer at least some of the request created by a client before we ship
it to the server.  For many requests, we can fit it completely into a
1 MiB buffer, but if it doesn't we can drop back to using the chunked
transfer encoding to send an unknown stream length.

Rather than recoding the block based memory buffer, we refactor the
local file overflow strategy into a subclass, allowing the HTTP client
code to replace this portion of the logic with its own approach to
start the chunked encoding request.

Change-Id: Iac61ea1017b14e0ad3c4425efc3d75718b71bb8e
Signed-off-by: Shawn O. Pearce <sop@google.com>

The multi_ack_detailed extension breaks out the "ACK %s continue" status
code into "ACK %s common" and "ACK %s ready" states, making it easier to
discover which objects are truely common, and which objects are simply
on a chain the server doesn't care learning about.

Change-Id: Ie8e907424cfbbba84996ca205d49eacf339f9d04
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>