If during an garbage collection old packfiles are deleted it could
happen that on certain platforms the index file can be deleted but the
packfile can't be deleted (because someone locked the file). This led
to repositories with packfiles without corresponding index files. Those
zombie-packfiles potentially consume a lot of space on disk and it is
never tried to delete them again. Try to avoid this situation by
deleting packfiles first and don't try to delete the other files if we
can't delete the packfile. This gives us the chance to delete the
packfile during next GC.

This commit only improves the situation - there is still the chance for
orphan files during packfile deletion. We don't have an atomic delete
of multiple files .

Change-Id: I0a19ae630186f07d0cc7fe9df246fa1cedeca8f6

Change-Id: Ib6ee3a2874a7e2240aa68f4ac32d00c4d1fab5ae
Signed-off-by: Chris Aniszczyk <zx@twitter.com>

Only on Windows the rename operation which renames temporary Packfiles
(and index-files and bitmap-files) sometime fails. This happens only
when renaming a temporary Packfile to a Packfile which already exists.
Such situations occur if you run GC twice on a repo without modifying
the repo inbetween.

In such situations there was bug in GC which led to a corrupted repo
whithout any packfiles anymore. This commit fixes the problem by
introducing a utility method which renames a file and throws an
IOException if it fails. This method also takes care to repeat a
failing rename if our FS class has found out we are running on a
platform with a unreliable File.renameTo() method.

I am searching for a better solution because even with this utility
method in hand a GC on a already GC'ed repo will fail on Windows. But
at least with this fix we will not produce corrupted repos anymore.

Bug: 389305
Change-Id: Iac1ab3e0b8c419c90404f2e2f3559672eb8f6d28
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

With JGit it is possible to write reflog entries where new objectid and
old objectid is null. Such reflogs cause FileRepository GC to crash
because it doesn't expect the new objectid to be null. One case where
this happened is in Gerrit's allProjects repo. In the same way as we
expect the old objectid to be potentially null we should also ignore
null values in the new objectid column.

Change-Id: Icf666c7ef803179b84306ca8deb602369b8df16e

This breaks all existing callers once. Applications are not supposed
to build against the internal storage API unless they can accept API
churn and make necessary updates as versions change.

Change-Id: I2ab1327c202ef2003565e1b0770a583970e432e9

Change-Id: Iedb54eb9d8396bc3ae66d8754c1527fd9ca655f9

File system time stamps and System.currentTimeMillis() may not
necessarily be running on the same clock so add some slack.

Bug: 396662
Change-Id: I25204d9e3181e15368da2902447518c6ce205017

Invoke the wrapper types' valueOf via static imports.
For booleans used in asserts, add a new assert in
the JUnit utility package since out current version of JUnit
does not have the assert(boolean, boolean) method.

Change-Id: I9099bd8efbc8c133479344d51ce7dabed8958a2b

Some GC tests were sporadically failing. The reason was that they used
the setExpireAgeMillis method to define object expiration before
invoking the prune method. Depending on the CPU load during the test
run, the prune method may reach an object (which is considered
non-expired by the test) too late and actually prune it.

To make the test stable we now use the setExpire(Date expire) method and
define a time instant before which objects are considered to be expired.
This way the outcome of the prune method doesn't depend on the CPU load.

Change-Id: Ifc3323ca55ae56dbccdbc90a282ec3cf18ad7297
Signed-off-by: Sasa Zivkov <sasa.zivkov@sap.com>

Change-Id: Id5b578f7040c6c896ab9386a6b5ed62b0f495ed5
Signed-off-by: Sasa Zivkov <sasa.zivkov@sap.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

Implements a garbage collector for FileRepositories. Main ideas are
copied from the garbage collector for DFS based repos
(DfsGarbageCollector). Added functionalities are
- pruning loose objects
- handling of the index
- packing refs
- handling of reflogs (objects referenced from reflog will not be
pruned/)

These are features of a GC which are not handled in this change and
which should come with subsequent changes:
- unpacking packed objects into loose objects (to support that pruning
packed objects doesn't delete them until they are older than two weeks)
- expiration of reflogs
- support for configuration parameters (e.g. gc.pruneExpire)

Change-Id: I14ea5cb7e0fd1b5c50b994fd77f4e05bfbb9d911
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>