This is actually a fairly common occurrence; deleting the parent
directories can work only if the file deleted was the last one
in the directory.

Bug: 537872
Change-Id: I86d1d45e1e2631332025ff24af8dfd46c9725711
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>

Log as warning when an attempt to remove a directory
fails. This helps troubleshooting some bugs like the GC leaving
behind empty directories.

Change-Id: Idb94ce17f8be9668a970c7ecae31436bf434073c
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>

Where the exception being thrown has a constructor that takes a
Throwable, use that instead of instantiating the exception and then
explicitly calling initCause.

Change-Id: I06a0df407ba751a7af8c1c4a46f9e2714f13dbe3
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>

Change-Id: Ieb2f66aef2cab7e2a6d8e35c5f5047da881994dd
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

When we are cloning we have no refs at all yet, and there cannot
(or at least should not) be any other thread doing something with
refs yet.

Locking loose refs is thus not needed, since there are no loose
refs yet and nothing should be trying to create them concurrently.

Let's skip the whole loose ref locking when we are cloning a repository.
As a result, JGit will write the refs directly to the packed-refs
file, and will not create the refs/remotes/ directories nor the
lock files underneath when cloning and packed refs are used. Since
no lock files are created, any problems on case-insensitive file
systems with tag or branch names that differ only in case are avoided
during cloning.

Detect if we are cloning based on the following heuristics:
* HEAD is a dangling symref
* There is no loose ref
* There is no packed-refs file

Note, however, that there may still be problems with such tag or
branch names later on. This is primarily a five-minutes-past-twelve
stop-gap measure to resolve the referenced bug, which affects the
Oxygen.2 release.

Bug: 528497
Change-Id: I57860c29c210568165276a123b855e462b6a107a
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

Then list of packed refs was cached in RefDirectory based on mtime of
the packed-refs file. This may fail on NFS when attributes are cached.
A cached mtime of the packed-refs file could cause JGit to trust the
cached content of this file and to overlook that the file is modified.

Honor the config option trustFolderStats and always read the packed-refs
content if the option is false. By default this option is set to true
and this fix is not active.

Change-Id: I2b65cfaa8f4aba2efbf8a5e865d3f09f927e2eec

Even if a repository has core.logAllRefUpdates=true, ReflogWriter does
not create reflog files unless the refs are under a hard-coded list of
prefixes, or unless the forceWrite bit is set. Expose the forceWrite bit
on a per-update basis in RefUpdate/BatchRefUpdate/ReceiveCommand,
creating RefLogWriters as necessary.

Change-Id: Ifc851fba00f76bf56d4134f821d0576b37810f80

The ReflogWriter constructor just took a Repository and called
getDirectory() on it to figure out the reflog dirs, but not all
Repository instances use this storage format for reflogs, so it's
incorrect to attempt to use ReflogWriter when there is not a
RefDirectory directly involved. In practice, ReflogWriter was mostly
only used by the implementation of RefDirectory, so enforcing this is
mostly just shuffling around calls in the same internal package.

The one exception is StashDropCommand, which writes to a reflog lock
file directly. This was a reasonable implementation decision, because
there is no general reflog interface in JGit beyond using
(Batch)RefUpdate to write new entries to the reflog. So to implement
"git stash drop <N>", which removes an arbitrary element from the
reflog, it's fair to fall back to the RefDirectory implementation.
Creating and using a more general interface is well beyond the scope of
this change.

That said, the old behavior of writing out the reflog file even if
that's not the reflog format used by the given Repository is clearly
wrong. Fail fast in this case instead.

Change-Id: I9bd4b047bc3e28a5607fd346ec2400dde9151730

With atomic ref updates using packed refs, JGit did not fire a
RefsChangedEvent. This resulted in a user-visible regression in
EGit: the UI would not update after a "Fetch from upstream...".
Presumably it would also make Gerrit miss out on ref changes?

Strengthen the BatchRefUpdateTest by also asserting the expected
number of RefsChangedEvents, and ensure modCnt is incremented in
RefDirectory.commitPackedRefs() when refs really changed (as opposed
to some internal housekeeping operation, such as packing loose refs).

Bug: 521296
Change-Id: Ia985bda1d99f45a5f89c8020ca4845e7a66e743e
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>

In a server scenario such as Gerrit Code Review, there may be many
atomic BatchRefUpdates contending for locks on both the packed-refs file
and some subset of loose refs. We already retry lock acquisition to
improve this situation slightly, but we can do better by using an
in-process lock. This way, instead of retrying and potentially exceeding
their timeout, different threads sharing the same Repository instance
can wait on a fair lock without having to touch the disk lock. Since a
server is probably already using RepositoryCache anyway, there is a high
likelihood of reusing the Repository instance.

Change-Id: If5dd1dc58f0ce62f26131fd5965a0e21a80e8bd3

If a repo frequently uses PackedBatchRefUpdates, there is likely to be
contention on the packed-refs file, so it's not appropriate to fail
immediately the first time we fail to acquire a lock. Add some logic to
RefDirectory to support general retrying of lock acquisition.

Currently, there is a hard-coded wait starting at 100ms and backing off
exponentially to 1600ms, for about 3s of total wait. This is no worse
than the hard-coded backoff that JGit does elsewhere, e.g. in
FileUtils#delete. One can imagine a scheme that uses per-repository
configuration of backoff, and the current interface would support this
without changing any callers.

Change-Id: I4764e11270d9336882483eb698f67a78a401c251

The existing packed-refs file provides a mechanism for implementing
atomic multi-ref updates without any changes to the on-disk format or
lockfile protocol. We just need to make sure that there are no loose
refs involved in the transaction, which we can achieve by packing the
refs while holding locks on all loose refs. Full details of the
algorithm are in the PackedBatchRefUpdate javadoc.

This change does not implement reflog support, which will come in a
later change.

Change-Id: I09829544a0d4e8dbb141d28c748c3b96ef66fee1

The RefDirectory implementation of doDelete never considered whether to
delete a symref or its leaf, because the detachingSymbolicRef bit was
never exposed from RefUpdate. The behavior was thus incorrectly to
always delete the symref, never the leaf.

There was no test for this behavior. The only thing that attempted to be
a test was testDeleteHeadInBareRepo, but this test was broken for
reasons unrelated to this bug. Specifically, it set the leaf to point to
a completely nonexistent object, and then asserted that deleting HEAD
resulted in NO_CHANGE. The only reason this test ever passed is because
of a quirk of updateImpl, which treats a missing object as the same as
null. This quirk aside, the test wasn't really testing the right thing.
Turn this into a real test by writing out a real object and pointing the
leaf at that.

Also, add a test for the detachingSymbolicRef case, i.e. deleting the
symref and leaving the leaf alone.

Change-Id: Ib96d2a35b4f99eba0734725486085fc6f9d78aa5

The contents of the packedRefList AtomicReference should never differ
from what we expect prior to writing, because this segment of the code
is protected by the packed-refs lock file on disk. If it does happen,
whether due to programmer error or a rogue process not respecting the
locking protocol, it's better to let the caller know than to silently
drop the whole commit operation on the floor.

The existing concurrentOnlyOneWritesPackedRefs test is inherently
nondeterministic as written, and was already about 6% flaky as measured
by bazel:

  $ bazel test --runs_per_test=200 //org.eclipse.jgit.test:org_eclipse_jgit_internal_storage_file_GcPackRefsTest
  ...
  INFO: Elapsed time: 42.608s, Critical Path: 10.35s
  //org.eclipse.jgit.test:org_eclipse_jgit_internal_storage_file_GcPackRefsTest FAILED in 12 out of 200 in 1.6s
    Stats over 200 runs: max = 1.6s, min = 1.1s, avg = 1.3s, dev = 0.1s

This flakiness was caused by the assumption that exactly one of the 2
threads would fail, when both might actually succeed in practice due to
racing on the compare-and-swap.

For whatever reason, this change affected the interleaving behavior in
such a way that the flakiness jumped to around 50%. Making the
interleaving of the test fully deterministic is beyond the scope of this
change, but a simple tweak to the assertion is enough to make it pass
consistently 200+ times both before and after this change.

Change-Id: I5ff4dc39ee05bda88d47909acb70118f3d0c8f74

Change-Id: Id691905599b242e48f590138a96e0c86132308fd

Since the introduction of generic type parameter inference in Java 7,
it's not necessary to explicitly specify the type of generic parameters.

Enable the warning in Eclipse, and fix all occurrences.

Change-Id: I9158caf1beca5e4980b6240ac401f3868520aad0
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>

Set missingOverrideAnnotation=warning in Eclipse compiler preferences
which enables the warning:

  The method <method> of type <type> should be tagged with @Override
  since it actually overrides a superclass method

Justification for this warning is described in:

  http://stackoverflow.com/a/94411/381622

Enabling this causes in excess of 1000 warnings across the entire
code-base. They are very easy to fix automatically with Eclipse's
"Quick Fix" tool.

Fix all of them except 2 which cause compilation failure when the
project is built with mvn; add TODO comments on those for further
investigation.

Change-Id: I5772061041fd361fe93137fd8b0ad356e748a29c
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>

Cover the case where the exception is wrapped up as a
cause, e.g., PackIndex#open(File).

Change-Id: I0df5b1e9c2ff886bdd84dee3658b6a50866699d1
Signed-off-by: Hongkai Liu <hongkai.liu@ericsson.com>

This SIOOBE happens reproducibly when trying to access
a repository containing Cygwin symlinks

Change-Id: I25f103fcc723bac7bfaaeee333a86f11627a92c7
Signed-off-by: Marc Strapetz <marc.strapetz@syntevo.com>
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>

When doing a detaching operation, JGit fakes a SymbolicRef as an
ObjectIdRef. This is because RefUpdate#updateImpl dereferences the
SymbolicRef when updating it. For example, assume that HEAD is
pointing to refs/heads/master. If I try to make a detached HEAD
pointing to a commit c0ffee, RefUpdate dereferences HEAD as
refs/heads/master first and changes refs/heads/master to c0ffee. The
detach argument of RefDatabase#newUpdate avoids this dereference by
faking HEAD as ObjectIdRef.

This faking is problematic for the linking operation of
DfsRefDatabase. It does a compare-and-swap operation on every
reference change because of its distributed systems nature. If a
SymbolicRef is faked as an ObjectRef, it thinks that there is a
racing change in the reference and rejects the update. Because of
this, DFS based repositories cannot change the link target of symbolic
refs. This has not been a problem for file-based repositories because
they have a file-lock based semantics instead of the CAS based one.
The reference implementation, InMemoryRepository, is not affected
because it only compares ObjectIds.

When [1] introduced this faking code, there was no way for RefUpdate
to distinguish the detaching operation. When [2] fixed the detaching
operation, it introduced a detachingSymbolicRef flag. This commit uses
this flag to control whether it needs to dereference the symbolic refs
by calling Ref#getLeaf. The same flag is used in the reflog update
operation.

This commit does not affect any operation that succeeds currently. In
some DFS repository implementations, this fixes a ref linking
operation, which is currently failing.

[1]: 01b5392cdb
[2]: 3a86868c08

Change-Id: I118f85f0414dbfad02250944e28d74dddd59469b
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>

Before this fix, ref directory removal did not work. That was because
the ref lock file was still in the leaf directory at deletion time.
Hence no deep ref directories were ever deleted, which negatively
impacted performance under large directory structure circumstances.

This fix removes the ref lock file before attempting to delete the ref
directory (which includes it). The other deep parent directories are
therefore now successfully deleted in turn, since leaf's content
(lock file) gets removed first.

So, given a structure such as refs/any/directory[/**], this fix now
deletes all empty directories up to -and including- 'directory'. The
'any' directory (e.g.) does not get deleted even if empty, as before.

The ref lock file is still also removed in the calling block's finally
clause, just in case, as before. Such double-unlock brought by this
fix is harmless (a no-op).

A new (private) RefDirectory#delete method is introduced to support
this #pack-specific case; other RefDirectory#delete callers remain
untouched.

Change-Id: I47ba1eeb9bcf0cb93d2ed105d84fea2dac756a5a
Signed-off-by: Marco Miller <marco.miller@ericsson.com>

Before this change, jgit used to read packed-refs before scanning
loose refs. That was not a problem if gc didn't run concurrently. When
gc did run concurrently with such refs reading, that order sometimes
broke the latter. This lead to reading an older version of a ref's
tip, which meant "losing" the real tip or commit. The specific
read-Vs-gc concurrency scenario which broke reading that way follows:

1. let ref R be in packed-refs and R' be in loose
2. jgit starts reading packed-refs
3. gc also starts its business around that very time
4. jgit still has the time to read R from packed-refs
5. as gc is not done yet updating packed-refs with R'
6. jgit then starts scanning loose refs (or is about to)
7. gc quickly ends up being done moving loose R' to packed-refs
8. so gc (quickly) removes loose refs
9. -while jgit is scanning loose refs, now gone
10. so jgit assumes no loose to consider => packed-refs winning
11. so jgit wrongfully returns R (from 4.) as the tip, instead of R'.

This fix switches the order so loose refs are scanned (secured) before
taking the time to read packed-refs. This way, knowledge of the
likelier tip is guaranteed for ref reading to return the true tip
- despite concurrent gc. If there is no loose ref to scan, jgit reads
packed-refs and lands on R' (or S), which it then returns, as
expected. The gerrit issue [1] should be solved by this fix.

[1] https://code.google.com/p/gerrit/issues/detail?id=2302

Change-Id: Ibd120120a361a3a6ed565f3836afc1db706fbcdd
Signed-off-by: Marco Miller <marco.miller@ericsson.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

When repositories are opened using the RepositoryCache, they are kept in
memory and when the repository usage counter reaches 0, the
Repository.close method is called which then calls close method on its
reference and object databases.

The problem is that RefDirectory.close method was a no-op and the
reference database was kept in memory. This problem is only happening
when opening a repository using the RepositoryCache because it never
evicts repositories, it's just calling the close method.

Change-Id: Iacb961de8e8b1f5b37824bf0d1a4caf4c6f1233f
Signed-off-by: Hugo Arès <hugo.ares@ericsson.com>

Change-Id: Ibc3e2f3372e1a65732dd6d3c71cec53fb1aa15e2
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>

This change fixes all compiler errors in JGit and replaces possible
NPE's with either appropriate exceptions, avoiding multiple "Nullable
return" method calls or early returning from the method.

Change-Id: I24c8a600ec962d61d5f40abf73eac4203e115240
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>

File, FileInputStream and friends may throw FileNotFoundException even
if the file is existing e.g. when file permissions don't allow to access
the file content. In most cases this is a severe error we should not
suppress hence rethrow the FileNotFoundException in this case.

This may also fix bug 451508.

Bug: 451508
Change-Id: If4a94217fb5b7cfd4c04d881902f3e86193c7008
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

getRef() loops over its search path to find a ref:

	Ref ref = null;
	for (String prefix : SEARCH_PATH) {
		ref = readRef(prefix + needle, packed);
		if (ref != null) {
			ref = resolve(ref, 0, null, null, packed);
			break;
		}
	}
	fireRefsChanged();
	return ref;

If readRef returns null (indicating that the ref does not exist), the
loop continues so we can find the ref later in the search path.  And
resolve should never return null, so if we return null it should mean
we exhausted the entire search path and didn't find the ref.

... except that resolve can return null: it does so when it has
followed too many symrefs and concluded that there is a symref loop:

	if (MAX_SYMBOLIC_REF_DEPTH <= depth)
		return null; // claim it doesn't exist

Continue the loop instead of returning null immediately.  This makes
the behavior more consistent.

Arguably getRef should throw an exception when a symref loop is
detected.  That would be a more invasive change, so if it's a good
idea it will have to wait for another patch.

Change-Id: Icb1c7fafd4f1e34c9b43538e27ab5bbc17ad9eef
Signed-off-by: Jonathan Nieder <jrn@google.com>

When asked to read a symref pointing to a branch-yet-to-be-born (such
as HEAD in a newly initialized repository), DfsRepository and
FileRepository return different results.

FileRepository:

	exactRef("HEAD") => null

DfsRepository:

	exactRef("HEAD") => SymbolicRef[HEAD -> refs/heads/master=00000000]

getRef("HEAD") returns the same as DfsRepository's exactRef in both
backends.

The intended behavior is the DfsRepository one: exactRef() is supposed
to be like getRef(), but more exact because it doesn't need to
traverse the search path.

The discrepancy is because DfsRefDatabase implements exactRef()
directly with the intended semantics, while RefDirectory uses a
fallback implementation built on top of getRefs().  getRefs() skips
symrefs to an unborn branch.

Override the fallback implementation with a correct implementation
that is similar to getRef() to avoid this.  A followup change will fix
the fallback.

Change-Id: Ic138a5564a099ebf32248d86b93e2de9ab3c94ee
Reported-by: David Pursehouse <david.pursehouse@sonymobile.com>
Improved-by: Christian Halstrick <christian.halstrick@sap.com>
Bug: 478865

Java compiler must generate synthetic access methods for private methods
and fields of the enclosing class if they are accessed from inner
classes and vice versa.

While invisible in the code, those synthetic access methods exist in the
bytecode and seem to produce some extra execution overhead at runtime
(compared with the direct access to this fields or methods), see
https://git.eclipse.org/r/58948/.

By removing the "private" access modifier from affected methods and
fields we help compiler to avoid generation of synthetic access methods
and hope to improve execution performance.

To validate changes, one can either use javap or use Bytecode Outline
plugin in Eclipse. In both cases one should look for "synthetic
access$<number>" methods at the end of the class and inner class files
in question - there should be none.

NB: don't mix this "synthetic access$" methods up with "public synthetic
bridge" methods generated to allow generic method override return types.

Change-Id: If53ec94145bae47b74e2561305afe6098012715c
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>

On a local filesystem the packed-refs file will be orphaned if it is
replaced by another client while the current client is reading the old
one. However, since NFS servers do not keep track of open files, instead
of orphaning the old packed-refs file, such a replacement will cause the
old file to be garbage collected instead.  A stale file handle exception
will be raised on NFS servers if the file is garbage collected (deleted)
on the server while it is being read.  Since we no longer have access to
the old file in these cases, the previous code would just fail. However,
in these cases, reopening the file and rereading it will succeed (since
it will reopen the new replacement file).  So retrying the read is a
viable strategy to deal with stale file handles on the packed-refs file,
implement such a strategy.

Since it is possible that the packed-refs file could be replaced again
while rereading it (multiple consecutive updates can easily occur with
ref deletions), loop on stale file handle exceptions, up to 5 extra
times, trying to read the packed-refs file again, until we either read
the new file, or find that the file no longer exists. The limit of 5 is
arbitrary, and provides a safe upper bounds to prevent infinite loops
consuming resources in a potential unforeseen persistent error
condition.

Change-Id: I085c472bafa6e2f32f610a33ddc8368bb4ab1814
Signed-off-by: Martin Fick<mfick@codeaurora.org>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

There should be no functional change, the logic updated only to make
code simple so that compiler can understand what is going for. Removed
all @SuppressWarnings("null") annotations since they cannot be used if
"org.eclipse.jdt.core.compiler.problem.potentialNullReference" option is
set to the "error" level.

Bug: 470647
Change-Id: Ie93c249fa46e792198d362e531d5cbabaf41fdc4
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>

- use try-with-resource where possible
- replace use of deprecated release() by close()

Change-Id: I0f139c3535679087b7fa09649166bca514750b81
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

Allegedly this should improve performance, but I could not see it.

Change-Id: Id2057cb2cfcb46e94ff954483ce23f9c4a7edc5e

This breaks all existing callers once. Applications are not supposed
to build against the internal storage API unless they can accept API
churn and make necessary updates as versions change.

Change-Id: I2ab1327c202ef2003565e1b0770a583970e432e9

A few classes such as Constanrs are marked with @SuppressWarnings, as are
toString() methods with many liternal, but otherwise $NLS-n$ is used for
string containing text that should not be translated. A few literals may
fall into the gray zone, but mostly I've tried to only tag the obvious
ones.

Change-Id: I22e50a77e2bf9e0b842a66bdf674e8fa1692f590

JGit was not able to lookup refs which had the name of files which exist
in the .git folder. When JGit was looking up a ref named X it has a
fixed set of directories where it searched for files named X
(ignore packed refs for now). First directory to search for is .git. In
case of the ref named 'config' it searched there for this file, found it
(it's the .git/config file with the repo configuration in it), parsed
it, found it is an invalid ref and stopped searching. It never looked
for a file .git/refs/heads/config.

I changed JGit in a way that when it finds a file in GIT_DIR which
corresponds to a ref name and if this file doesn't contain a valid ref
then it will ignore the InvalidObjectIdException and continue searching.

Change-Id: Ic26a329fb1624a5b2b2494c78bac4bd76817c100
Bug: 381574
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Signed-off-by: Stefan Lay <stefan.lay@sap.com>

Implements a garbage collector for FileRepositories. Main ideas are
copied from the garbage collector for DFS based repos
(DfsGarbageCollector). Added functionalities are
- pruning loose objects
- handling of the index
- packing refs
- handling of reflogs (objects referenced from reflog will not be
pruned/)

These are features of a GC which are not handled in this change and
which should come with subsequent changes:
- unpacking packed objects into loose objects (to support that pruning
packed objects doesn't delete them until they are older than two weeks)
- expiration of reflogs
- support for configuration parameters (e.g. gc.pruneExpire)

Change-Id: I14ea5cb7e0fd1b5c50b994fd77f4e05bfbb9d911
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>

Previously if a packed-refs file was racily clean then there
was a 2.5 second window in which each call to getPackedRefs
would increment the mod count causing a RefsChangedEvent to be
fired since the FileSnapshot would report the file as modified.

If a RefsChangedListener called getRef/getRefs from the
onRefsChanged method then a StackOverflowError could occur
since the stack could be exhausted before the 2.5 second
window expired and the packed-refs file would no longer
report being modified.

Now a SHA-1 is computed of the packed-refs file and the
mod count is only incremented when the packed refs are
successfully set and the id of the new packed-refs file
does not match the id of the old packed-refs file.

Change-Id: I8cab6e5929479ed748812b8598c7628370e79697

This extracts the logic for writing to the reflog from
RefDirectory into a new ReflogWriter class.  This class
creates a public API for writing reflog entries similar
to ReflogReader for reading reflog entries.

The new command supports rewriting the stash's log to remove
a configured entry followed by updating the stash ref to
the value at the bottom of the newly written log.

Change-Id: Icfcbc70e838666769a742a94196eb8dc9c7efcc7
Signed-off-by: Chris Aniszczyk <zx@twitter.com>

Change-Id: I763590a45d75f00a09097ab6f89581a3bbd3c797

Bug: 309355
Change-Id: I34a8c251b89abcdb67565ca49bee02e5e2113593
Signe-off-by: Chris Aniszczyk <zx@twitter.com>

This will allows calling classes to handle lock failures
without checking against the message and will also provide
access to the file that could not be locked.

Change-Id: I95bc59e1330a7af71ae3b0485c4516299193f504

A set of ref names like ('a/b' and 'a+b') would cause the RefDirectory
to think that the set of refs have changed because it traversed the
'a' directory in the subtree before looking at 'a+b', but it then
compared with the know refs which are sorted with 'a+b' first.

Fix this by traversing the refs tree in another order. Treat a directory
as if they ends with a '/' before deciding on the order to traverse
the refs tree.

Bug: 348834
Change-Id: I23377f8df00c7252bf27dbcfba5da193c5403917
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>

RefDirectory was not using FileSnapshot correctly in all places. This
is fixed with this commit. Additionally the constructors for the
different types of refs have been changed to take a FileSnapshot
instead of a modification time.

Change-Id: Ifb6a59e87e8b058a398c38cdfb9d648f0bad4bf8
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>

Since this change may affect performance and memory consumption on every
access to a loose ref I explicitly made it a RFC to collect opinions.

Previously RefDirectory.scanRef() was not detecting an update of a
loose ref when the update didn't changed the modification time of
the backing file. RefDirectory cached loose refs and the way to detect
outdated cache entries was to compare lastmodification timestamp on the
file representing the ref. If two updates to the same ref happen faster
than the filesystem-timer granularity (for linux this is 2 seconds)
there is the possiblity that we don't detect the update.

Because of this bug EGit's PushOperationTest only works with 2 second
sleeps inside.

This change let RefDirectory use FileSnapshot to detect such situations.
FileSnapshot helps to remember when a file was last read from disk and
therefore enables to decide when to load a file from disk although
modification time has not changed.

Change-Id: I03b9a137af097ec69c4c5e2eaa512d2bdd7fe080
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>

RefDirectory did not correctly follow the contract of RefList.  The
contract says if you use add() method of RefList builder, you MUST
sort() it afterwards, and later every other method assumes that list
is properly sorted (especially the binary search in the find() and
get() methods).  Instead RefDirectory class tried to scan the refs
recursively while sorting every folder in the process before
processing and did not call sort().

For example, when scanning the contents of refs/tags project1 string
is smaller than project1-*, so it will recursively go into the folder
and add these tags first and only then will add project-* ones. This
will result in a broken list (any project1-* string is less than
project1/* one, but they all appear after them in the list), that's
why binary search will fail making loose RefList and the whole local
RefMap completely unusable.

Change-Id: Ibad90017e3b2435b1396b69a22520db4b1b022bb
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>

Add handling of CHERRY_PICK_HEAD file in .git (similar to MERGE_HEAD),
which is written in case of a conflicting cherry-pick merge.

It is used so that Repository.getRepositoryState can return the new
states CHERRY_PICKING and CHERRY_PICKING_RESOLVED. These states, as well
as CHERRY_PICK_HEAD can be used in EGit to properly show the merge tool.

Also, in case of a conflict, MERGE_MSG is written with the original
commit message and a "Conflicts" section appended. This way, the
cherry-picked message is not lost and can later be re-used in the commit
dialog.

Bug: 339092
Change-Id: I947967fdc2f1d55016c95106b104c2afcc9797a1
Signed-off-by: Robin Stocker <robin@nibor.org>
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>

When reading refs, avoid reading huge files that were put there
accidentally, but still read the top of e.g. FETCH_HEAD, which
may be longer than our limit. We're only interested in the first line
anyway.

Bug: 340880
Change-Id: I11029b9b443f22019bf80bd3dd942b48b531bc11
Signed-off-by: Carsten Pfieffer <carsten.pfeiffer@gebit.de>
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>

There should be a way to explictly refresh the refs cached in the
RefDirectory. Since commit c261b28 (use of FileSnapshot) this is
not needed anymore for storage in the filesystem. But for DHT based
storage an explicit refresh may be needed.

Change-Id: I7d30c3496c05e1fb6e9519f3af9f23c6adb93bf9
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>

Instead of tracking the length and modification time by hand, rely
on FileSnapshot to tell RefDirectory when the $GIT_DIR/packed-refs
file has been changed or should be re-read from disk.

Change-Id: I067d268dfdca1d39c72dfa536b34e6a239117cc3
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>