Java GC evicts all SoftReferences when the used heap size comes close to
the maximum heap size. This means peaks in heap memory consumption can
flush the complete WindowCache which was observed to have negative
impact on performance of upload-pack in Gerrit.

Hence add a boolean option core.packedGitUseStrongRefs to allow using
strong references to reference packfile pages cached in the WindowCache.

If this option is set to true Java gc can no longer flush the
WindowCache to free memory if the used heap comes close to the maximum
heap size. On the other hand this provides more predictable performance.

Bug: 553573
Change-Id: I9de406293087ab0fa61130c8e0829775762ece8d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

Add the following statistics
- cache hit count and hit ratio
- cache miss count and miss ratio
- count of successful and failed loads
- rate of failed loads
- load, eviction and request count
- average and total load time

Use LongAdder instead of AtomicLong to implement counters in order to
improve scalability.

Optionally expose these metrics via JMX, they are registered with the
platform MBean server if the config option jmx.WindowCacheStats = true
in the user or system level git config.

Bug: 553573
Change-Id: Ia2d5246ef69b9c2bd594a23934424bc5800774aa
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

As reported by Error Prone:

An inner class should be static unless it references members of its
enclosing class. An inner class that is made non-static unnecessarily
uses more memory and does not make the intent of the class clear.

See https://errorprone.info/bugpattern/ClassCanBeStatic

Change-Id: Ib99d120532630dba63cf400cc1c61c318286fc41
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
(cherry picked from commit ee40efcea4)

As reported by Error Prone:

An inner class should be static unless it references members of its
enclosing class. An inner class that is made non-static unnecessarily
uses more memory and does not make the intent of the class clear.

See https://errorprone.info/bugpattern/ClassCanBeStatic

Change-Id: Ib99d120532630dba63cf400cc1c61c318286fc41
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>

Change-Id: I3b748620f067582afef20f144feebe40d0332be2
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>

Remove it from

 * package private functions.

 * try blocks

 * for loops

this was done with the following python script:

$ cat f.py
import sys
import re
import os

def replaceFinal(m):
  return m.group(1) + "(" +  m.group(2).replace('final ', '') + ")"

methodDecl = re.compile(r"^([\t ]*[a-zA-Z_ ]+)\(([^)]*)\)")

def subst(fn):
  input = open(fn)
  os.rename(fn, fn + "~")

  dest = open(fn, 'w')
  for l in input:
    l = methodDecl.sub(replaceFinal, l)
    dest.write(l)
  dest.close()


for root, dirs, files in os.walk(".", topdown=False):
  for f in files:
    if not f.endswith('.java'):
      continue

    full = os.path.join(root, f)
    print full
    subst(full)

Change-Id: If533a75a417594fc893e7c669d2c1f0f6caeb7ca
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>

Change-Id: Id924f79c8b2c720297ebc49bf9c5d4ddd6d52547
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>

When an auto-closeable resources is not opened in try-with-resource,
the warning "should be managed by try-with-resource" is emitted by
Eclipse.

Fix the ones that can be silenced simply by moving the declaration of
the variable into a try-with-resource.

In cases where we explicitly call the close() method, for example in
tests where we are testing specific behavior caused by the close(),
suppress the warning.

Leave the ones that will require more significant refcactoring to fix.
They can be done in separate commits that can be reviewed and tested
in isolation.

Change-Id: I9682cd20fb15167d3c7f9027cecdc82bc50b83c4
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>

Since the introduction of generic type parameter inference in Java 7,
it's not necessary to explicitly specify the type of generic parameters.

Enable the warning in Eclipse, and fix all occurrences.

Change-Id: I9158caf1beca5e4980b6240ac401f3868520aad0
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>

This class requires resources which are private to another
bundle. Move the class next to its resources, removing an
odd cross bundle dependency.

Change-Id: I30d5568b09ea5fb3bd3bb60b602f149c0867f49a

This breaks all existing callers once. Applications are not supposed
to build against the internal storage API unless they can accept API
churn and make necessary updates as versions change.

Change-Id: I2ab1327c202ef2003565e1b0770a583970e432e9

Change-Id: Iedb54eb9d8396bc3ae66d8754c1527fd9ca655f9

Change-Id: I83ca25fb569c0dbc36eb374d5437fcf2b65a6f68

Eclipse has some problem re-running single JUnit tests if
the tests are in Junit 3 format, but the JUnit 4 launcher
is used. This was quite unnecessary and the move was not
completed. We still have no JUnit4 test.

This completes the extermination of JUnit3. Most of the
work was global searce/replace using regular expression,
followed by numerous invocarions of quick-fix and organize
imports and verification that we had the same number of
tests before and after.

- Annotations were introduced.
- All references to JUnit3 classes removed
- Half-good replacement for getting the test name. This was
  needed to make the TestRngs work. The initialization of
  TestRngs was also made lazily since we can not longer find
  out the test name in runtime in the @Before methods.
- Renamed test classes to end with Test, with the exception
  of TestTranslateBundle, which fails from Maven
- Moved JGitTestUtil to the junit support bundle

Change-Id: Iddcd3da6ca927a7be773a9c63ebf8bb2147e2d13
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>

The class is identical, but ObjectLoader.SmallObject is part of our
public API for storage implementations to build on top of.

Change-Id: I381a3953b14870b6d3d74a9c295769ace78869dc
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>

We drop the "Object" suffix, because its pretty clear here that
we want to open an object, given that we pass in AnyObjectId as
the main parameter.  We also fix the calling convention to throw
a MissingObjectException or IncorrectObjectTypeException, so that
callers don't have to do this error checking themselves.

Change-Id: I72c43353cea8372278b032f5086d52082c1eee39
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>

We no longer need an ObjectLoader to be lazy and try to delay
the materialization of the object content.  That was done only
to support PackWriter searching for a good reuse candidate.

Instead, simplify the code base by doing the materialization
immediately when the loader asks for it, because any caller
asking for the loader is going to need the content.

Change-Id: Id867b1004529744f234ab8f9cfab3d2c52ca3bd0
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>

These were only used by PackWriter to help it filter object
representations.  Their only user disappeared when we rewrote the
object selection code path to use the new representation type.

Change-Id: I9ed676bfe4f87fcf94aa21e53bda43115912e145
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>

This move isolates all of the local file specific implementation code
into a single package, where their package-private methods and support
classes are properly hidden away from the rest of the core library.

Because of the sheer number of files impacted, I have limited this
change to only the renames and the updated imports.

Change-Id: Icca4884e1a418f83f8b617d0c4c78b73d8a4bd17
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>

Not all of our test cases really require the sample data packs,
and we are better off not using them because its hard to see exactly
what condition a test is testing when looking only at the Java code.
Clarify the dependency by only making the packs available when
there is a real need for it.

Change-Id: Id8a76ee7ee1f7efba585be4bed19a8fb5b3b3585
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>

Per CQ 3448 this is the initial contribution of the JGit project
to eclipse.org.  It is derived from the historical JGit repository
at commit 3a2dd9921c.

Signed-off-by: Shawn O. Pearce <spearce@spearce.org>