Change-Id: I50407e4a33e35b718ca40503fdd436f1f9f70fba
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>

The location of the API v1 documentation has changed. Update the
links accordingly.

Change-Id: If0148a0b573c474bbe157fcb7e6674c0055fe8b4
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>

* stable-4.4:
  JGit v4.4.1.201607150455-r
  RefDirectory: remove ref lock file for following ref dir removal

Change-Id: Ifc8a782efd7f2f991e70ad2a3691a8dba66c7554
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>

Gerrit's superproject subscription feature uses RefSpecs to formalize
the ACLs of when the superproject subscription feature is allowed.

As this is a slightly different use case than describing a local/remote
pair of refs, we need to be more permissive. Specifically we want to allow:

    refs/heads/*
    refs/heads/*:refs/heads/master
    refs/heads/master:refs/heads/*

Introduce a new constructor, that allows constructing these RefSpecs.

Change-Id: I46c0bea9d876e61eb2c8d50f404b905792bc72b3
Signed-off-by: Stefan Beller <sbeller@google.com>

We had a case in Gerrits superproject subscriptions where
'refs/heads/' was configured with the intention to mean 'refs/heads/*'.

The first expression lacks the '*', which is why it is not considered
a wildcard but it was considered valid and so was not found early to be
a typo.

Refs are not allowed to end with '/' anyway, so add a check for that.

Change-Id: I3ffdd9002146382acafb4fbc310a64af4cc1b7a9
Signed-off-by: Stefan Beller <sbeller@google.com>

Example usage:
$ ./jgit push \
  --push-option "Reviewer=j.doe@example.org" \
  --push-option "<arbitrary string>" \
  origin HEAD:refs/for/master
Stefan Beller has also made an equivalent change to CGit:
http://thread.gmane.org/gmane.comp.version-control.git/299872

Change-Id: I6797e50681054dce3bd179e80b731aef5e200d77
Signed-off-by: Dan Wang <dwwang@google.com>

Change-Id: Id045f162fa584ea14da29a9df58a42c53a78dc15

* changes:
  DfsObjectDatabase: Expose PackList and move markDirty there
  Invalidate DfsObjDatabase pack list when refs are updated

What's invalidated when an object database is "dirty" is not the whole
database, but rather a specific list of packs. If there is a race
between getting the pack list and setting the volatile dirty flag
where the packs are rescanned, we don't need to mark the new pack list
as dirty.

This is a fine point that only really applies if the decision of
whether or not to mark dirty actually requires introspecting the pack
list (say, its timestamps). The general operation of "take whatever
is the current pack list and mark it dirty" may still be inherently
racy, but the cost is not so high.

Change-Id: I159e9154bd8b2d348b4e383627a503e85462dcc6

This variable has been populated and never used since it was
introduced in commit 5cf53fdacf
(Speed up clone/fetch with large number of refs, 2013-02-18).
Noted by FindBugs:
"BatchRefUpdate.java:359, UC_USELESS_OBJECT, Priority: Normal"

Change-Id: I7aacb49540aaee4a83db3d38b15633bb6c4773d0
Signed-off-by: Dan Wang <dwwang@google.com>

Change-Id: I61dbc29a962c8185fb356fe1ca30a1e673166d47
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

9aa3748 added dummy implementations for loadRoleInfo() and
loadUserInfo() to class MappedLoginService to fix compile errors in
Eclipse when using 4.6 target platform which brings Jetty 9.3 adding
these two methods. Unfortunately this causes errors when using non 4.6
target platform coming with an older Jetty version. Fix this by
extracting the anonymous subclass of MappedLoginService which allows to
suppress the unused private method errors in Eclipse.

Change-Id: I75baeea7ff4502ce9ef2b541b3c0555da5535d79
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

Currently, there is a race where a user of a DfsRepository in a single
thread may get unexpected MissingObjectExceptions trying to look up an
object that appears as the current value of a ref:

1. Thread A scans packs before scanning refs, for example by reading
   an object by SHA-1.
2. Thread B flushes an object and updates a ref to point to that
   object.
3. Thread A looks up the ref updated in (2). Since it is scanning refs
   for the first time, it sees the new object SHA-1.
4. Thread A tries to read the object it found in (3), using the cached
   pack list it got from (1). The object appears missing.

Allow implementations to work around this by marking the object
database's current pack list as "dirty." A dirty pack list means that
DfsReader will rescan packs and try again if a requested object is
missing. Implementations should mark objects as dirty any time the ref
database reads or scans refs that might be newer than a previously
cached pack list.

Change-Id: I06c722b20c859ed1475628ec6a2f6d3d6d580700

* stable-4.4:
  Log if Repository.useCnt becomes negative
  Time based eviction strategy for repository cache
  Add method to read time unit from config
  Align include.path max depth with native git
  Config load should not fail on unsupported or nonexistent include path
  Allow using JDK 7 bootclasspath when compiling JGit using Java 8
  Extract work queue to allow reusing it

Change-Id: I6aeedb1cb8b0c3068af344a719c80a03ae68fc23
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

We observe in Gerrit 2.12 that useCnt can become negative in rare cases.
Log this to help finding the bug.

Change-Id: Ie91c7f9d190a5d7cf4733d4bf84124d119ca20f7
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

When Repository.close() decrements the useCount to 0 currently the cache
immediately evicts the repository from WindowCache and RepositoryCache.
This leads to I/O overhead on busy repositories because pack files and
references are inserted and deleted from the cache frequently.

This commit defers the eviction of a repository from the caches until
last use of the repository is older than time to live. The eviction is
handled by a background task running periodically.

Add two new configuration parameters:
* core.repositoryCacheExpireAfter: cache entries are evicted if the
cache entry wasn't accessed longer than this time in milliseconds
* core.repositoryCacheCleanupDelay: defines the interval in milliseconds
for running a background task evicting expired cache entries. If set to
-1 the delay is set to min(repositoryCacheExpireAfter, 10 minutes). If
set to 0 the time based eviction is switched off and no background task
is started. If time based eviction is switched off the JVM can still
evict cache entries if heap memory is running low.

Change-Id: I4a0214ad8b4a193985dda6a0ade63b70bdb948d7
Also-by: Matthias Sohn <matthias.sohn@sap.com>
Also-by: Hugo Arès <hugo.ares@ericsson.com>
Also-by: Sasa Zivkov <sasa.zivkov@sap.com>

Time units supported:

-milliseconds (1 ms, 2 milliseconds)
-seconds (1 s, 1 sec, 1 second, 2 seconds)
-minutes (1 m, 1 min, 1 minute, 2 minutes)
-hours (1 h, 1 hr, 1 hour, 2 hours)
-days (1 d, 1 day, 2 days)
-weeks (1 w, 1 week, 2 weeks)
-months (1 mon, 1 month, 2 months)
-years (1 y, 1 year, 2 years)

This functionality is implemented in Gerrit ConfigUtil class. Add it to
JGit so it can eventually be remove from Gerrit.

Change-Id: I2d6564ff656b6ab9424a9360624061c94fd5f413
Signed-off-by: Hugo Arès <hugo.ares@ericsson.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

Eclipse Neon comes with Jetty 9.3 which is causing unimplemented
abstract method errors in test class AppServer when using the JGit or
EGit Neon target platform. Fix this by adding dummy implementations.

Change-Id: Ie49107d814a846997de95f149e91fe1ec2fbe4d8
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

These try-with-resources blocks close the underlying output stream
twice: once when closing the CountingOutputStream wrapper, then again
when closing the DfsOutputStream out.

Simplify by only closing the CountingOutputStream.

In practice this shouldn't matter because the close() method of a
Closable is required to be idempotent, but avoiding the redundant
extra close makes the code simpler to read and understand.

Change-Id: I1778c4fc8ba075a2c6cd2129528bb272cb3a1af7

Change-Id: I75ea7deca64a707cd6b5c61c3c83062f20041684
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

This assertion only defined a message but didn't assert anything.

Change-Id: I0914642b64b69dc4e3ec24acbf8052f9171613d8
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

If the client sent a well-formed enough request to see it wants to use
side-band-64k for status reporting (meaning its a modern client), but
any other command record was somehow invalid (e.g. corrupt SHA-1)
report the parsing exception using channel 3.  This allows clients to
see the failure and know the server will not be continuing.

git-core and JGit clients send all commands and then start a sideband
demux before sending the pack. By consuming all commands first we get
the client into a state where it can see and respond to the channel 3
server failure.

This behavior is useful on HTTPS connections when the client is buggy
and sent a corrupt command, but still managed to request side-band-64k
in the first line.

Change-Id: If385b91ceb9f024ccae2d1645caf15bc6b206130

The "shallow $id" parsing can also throw InvalidObjectIdException,
just like parseCommand. Move it into its own method with a proper
try-catch block to convert to the checked PackProtocolException.

Change-Id: I6839b95f0bc4fbf4d2c213331ebb9bd24ab2af65

This exception is already handled in the outer catch block.

Change-Id: Ifc04b8e138732a97b9c0ee323af923b8e52a7f8e

Instead of deferring until after command parsing, enable the
capabilities after the first pkt-line has been read from the client.
This allows the server to setup the side-band-64k channel immediately.

Change-Id: I141b7fc92e983a41d3a58da8e1464a6917422b6c

Change-Id: I328f5952f23185a7d8f81a144979b4f8a095bd60

Change-Id: I429066d2af9c5fb8bf8bc038b99ebe5a7dc09170

If the push client has requested side-band support the server can
signal a fatal error parsing the pack using the error channel (3)
and then hang up. This may cause the PackWriter to fail to write to
data onto the network socket, which throws a misleading error back
up to the application and the user.

During a write failure poll the input to see if the side band system
can parse out an error message off channel 3. This should be fast as
there will either be an error present in the buffer, or the remote will
also have hung-up on the side band channel. In the case of a hang-up
just rethrow the original IOException as its a network error.

This roughly matches what C git does; once commands are sent and the
packer is started a new thread runs in the background to decode any
possible server error during unpacking on the remote peer

Change-Id: Idb37a4a122a565ec4b59130e08c27d963ba09395

The more specific type InvalidObjectIdException is thrown by
ObjectId.fromString().  Use it here in ReceivePack as the more
generic IAE is never thrown by the body of the try-catch block.

Change-Id: I53fc13c561c7d429a50b5eb82773f1a670431c54

A RefAdvertiser writing to the network includes both the reference's
ObjectId and its peeled ObjectId in the advertised set.  In smart HTTP
negotiation requests may bypass the RefAdvertiser and quickly build
the set based on current refs; include the peeled ObjectIds to match
behavior with the normal bidirectional protocols on git:// and SSH.

Change-Id: I5371bed60da36e8d12c4ad9a5c1d91a0f0ad486b

This field was being set twice within the block.  Setting it just once
is sufficient.  writeString() does not examine the field so it is fine
to set it after the call.

Change-Id: Ib4c74df4f1304e9df3015885bf360bf0d7bc6ca2

Change-Id: I1887e7f5c8b34b38fa1d6d699bf637c93bc36e7c

Change-Id: Ibab89fddda7ccfe3390348aa129ab2e5e3b7f538

Change-Id: I2332ce2f29e105b60b62a9a6fc7d4656a0c89343
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

Some branches in WorkingTreeIterator.getIndexFileMode() have not been
covered by tests. Enhance the tests to increase test coverage.

Change-Id: I400a221048f0f6cbaa987350eaf998b0ebb50a4e

Change-Id: I14daca6c81b003123e5862b384718fe06fb3ebd0
Signed-off-by: Michael Keppler <michael.keppler@gmx.de>

Example use case is to set a different IdentityRepository, for example,
a RemoteIdentityRepository to allow SSH Agent supplied credentials.

Change-Id: Id4a4afd64464e452ffe6d6ad49036f9e283b4655
Signed-off-by: markdingram <markdingram@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

Change-Id: I4b803b34d30a86105d111be417d4a4e6bfb1057b
Signed-off-by: Markus Keller <markus_keller@ch.ibm.com>

* changes:
  Prune UNREACHABLE_GARBAGE packs when they expire
  Use try-with-resources in DfsGarbageCollector.writePack
  Fix lastModified to be consistent in DfsGarbageCollector
  Add GC_REST PackSource to better order DFS packs

DfsGarbageCollector will now enforce a maximum time to live (TTL) for
UNREACHABLE_GARBAGE packs.  The default TTL is 1 day, which should be
enough time to avoid races with other processes that are inserting
data into the repository.

Change-Id: Id719e6e2a03cfc9a0c0aef8ed71d261dda14bd0c
Signed-off-by: Mike Williams <miwilliams@google.com>

Change-Id: I52f059816e1d94b2d60d096e3013bf4095cd0fc4
Signed-off-by: Hugo Arès <hugo.ares@ericsson.com>

1f86350 added initial support for include.path. Relative path and path
with tilde are not yet supported but config load was failing if one of
those 2 unsupported options was encountered. Another problem was that
config load was failing if the include.path file did not exist.

Change the behavior to be consistent with native git. Ignore unsupported
or nonexistent include.path.

Bug: 495505
Bug: 496732
Change-Id: I7285d0e7abb6389ba6983e9c46021bea4344af68
Signed-off-by: Hugo Arès <hugo.ares@ericsson.com>

Change-Id: I9a73125581b4d760b733fd045c3436c2aaaab730

Set all packs written by the DfsGarbageCollector to use the same
starting timestamp as lastModified. This makes it easier to see
which packs came from the same DfsGarbageCollector run, as they
share the same timestamp.

Change-Id: Id633573fbc3f0f360887b4745cacf33d6fc09320

Force reads to use a search ordering of:

  INSERT / RECEIVE
  COMPACT
  GC (heads)
  GC_REST (non-heads)
  GC_TXN (refs/txn)
  UNREACHABLE_GARBAGE

This has provided decent performance for object lookups.  Starting
from an arbitrary reference may find the content in a newer pack
created by DfsObjectInserter or a ReceivePack server.  Compaction of
recent packs also contains newer content, and then most interesting
data is in the "main" GC pack.  As the GC pack is self-contained (has
no edges leading outside) readers typically do not need to go further.

Adding a new GC_REST PackSource allows the DfsGarbageCollector to
identify to the pack ordering code which pack is which, so the
non-heads are scanned second during reads.  This removes a hack that
was unique to Google's implementation that enforced this behavior by
fixing up the lastModified timestamp.

Renumber the PackSource's categories to reflect this search ordering.

Change-Id: I19fdaab8a8d6687cbe8c88488e6daa0630bf189a