Smart HTTP clients may request both multi_ack_detailed and no-done in
the same request to prevent the client from needing to send a "done"
line to the server in response to a server's "ACK %s ready".
For smart HTTP, this can save 1 full HTTP RPC in the fetch exchange,
improving overall latency when incrementally updating a client that
has not diverged very far from the remote repository.
Unfortuantely this capability cannot be enabled for the traditional
bi-directional connections. multi_ack_detailed has the client sending
more "have" lines at the same time that the server is creating the
"ACK %s ready" and writing out the PACK stream, resulting in some race
conditions and/or deadlock, depending on how the pipe buffers are
implemented. For very small updates, a server might actually be able
to send "ACK %s ready", then the PACK, and disconnect before the
client even finishes sending its first batch of "have" lines. This
may cause the client to fail with a broken pipe exception. To avoid
all of these potential problems, "no-done" is restricted only to the
smart HTTP variant of the protocol.
Change-Id: Ie0d0a39320202bc096fec2e97cb58e9efd061b2d
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
RefDirectory did not correctly follow the contract of RefList. The
contract says if you use add() method of RefList builder, you MUST
sort() it afterwards, and later every other method assumes that list
is properly sorted (especially the binary search in the find() and
get() methods). Instead RefDirectory class tried to scan the refs
recursively while sorting every folder in the process before
processing and did not call sort().
For example, when scanning the contents of refs/tags project1 string
is smaller than project1-*, so it will recursively go into the folder
and add these tags first and only then will add project-* ones. This
will result in a broken list (any project1-* string is less than
project1/* one, but they all appear after them in the list), that's
why binary search will fail making loose RefList and the whole local
RefMap completely unusable.
Change-Id: Ibad90017e3b2435b1396b69a22520db4b1b022bb
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Make running static checks configurable in maven build
In order to run the static checks run:
mvn -P static-checks clean install
Change-Id: I14077498a04be986ded123ddbfc97da8f9bc3130
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Change-Id: I22fc46dff6cc5dfd975f6e82161d265781778cde
Signed-off-by: Stefan Lay <stefan.lay@sap.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
This ensures that all test data is separated from project sources and
cleaned up after the test. Previously the cloned bare test repository
was created in org.eclipse.jgit.test/ and not deleted after the test
run.
Change-Id: I55110442e365fc8fe610f1c372f72a71ee6e1412
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Add "Conflicts" section to merge message on conflict
The same as with cherry-pick, the commit message of a merge should
include a "Conflicts" section when the merge resulted in conflicts.
Change-Id: I6261dc898262322924af5ca1bef841a654b0df55
Signed-off-by: Robin Stocker <robin@nibor.org>
Add private methods which are used for reading and writing MERGE_HEAD
and CHERRY_PICK_HEAD files, as suggested in the comments on change
I947967fdc2f1d55016c95106b104c2afcc9797a1.
Change-Id: If4617a05ee57054b8b1fcba36a06a641340ecc0e
Signed-off-by: Robin Stocker <robin@nibor.org>
Add handling of CHERRY_PICK_HEAD file in .git (similar to MERGE_HEAD),
which is written in case of a conflicting cherry-pick merge.
It is used so that Repository.getRepositoryState can return the new
states CHERRY_PICKING and CHERRY_PICKING_RESOLVED. These states, as well
as CHERRY_PICK_HEAD can be used in EGit to properly show the merge tool.
Also, in case of a conflict, MERGE_MSG is written with the original
commit message and a "Conflicts" section appended. This way, the
cherry-picked message is not lost and can later be re-used in the commit
dialog.
Bug: 339092
Change-Id: I947967fdc2f1d55016c95106b104c2afcc9797a1
Signed-off-by: Robin Stocker <robin@nibor.org>
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
The timeout is also used in the FetchCommand called by the
CloneCommand.
The possibility to provide a list of branches to fetch initially is a
feature offered by EGit. To implement it here is a prerequisite for
EGit to be able to use the CloneCommand.
Change-Id: I21453de22e9ca61919a7c3386fcc526024742f5f
Signed-off-by: Stefan Lay <stefan.lay@sap.com>
When no branch was specified in the clone command, HEAD pointed to a
commit after clone. Now the clone command tries to find a branch which
points to the same commit and checks out this branch.
Bug: 339354
Change-Id: Ie3844465329f213dee4a8868dbf434ac3ce23a08
Signed-off-by: Stefan Lay <stefan.lay@sap.com>
Change I61a1b45db2d60fdcc0f87373ac6fd75ac4c4a202 fixed a possible NPE
occurring for newly created repositories - but in that case a wrong
value (false = not modified) was returned.
If a current version of the index file exists (liveFile), but there is
no snapshot, this means that there have been modifications (i.e. true
has to be returned).
Change-Id: I698f78112249f9924860fc58eb7eab7afdf87eb7
Signed-off-by: Philipp Thun <philipp.thun@sap.com>
Findbugs doesn't like using implicitly initialized field in
initializer.
Change-Id: Ic1ff9011813cc02950a71df587f31ed9f8415b49
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
When a new Git instance for an exisiting git repository should be
created there are two use-cases: either the application has already a
Repository instance in hand or the application knows where the
repository resides in the filesystem. Two methods are added to
explicitly support these use-cases: wrap(Repository db) and open(File
gitDir)
Change-Id: I2970e4aa8d4602cb1298f01e5b76bf0f96c492e5
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Support reading first SHA-1 from large FETCH_HEAD files
When reading refs, avoid reading huge files that were put there
accidentally, but still read the top of e.g. FETCH_HEAD, which
may be longer than our limit. We're only interested in the first line
anyway.
Bug: 340880
Change-Id: I11029b9b443f22019bf80bd3dd942b48b531bc11
Signed-off-by: Carsten Pfieffer <carsten.pfeiffer@gebit.de>
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
The 'Counting objects' phase of PackWriter requires good hit rates
from the DeltaBaseCache while walking trees, the deltas need to find
their bases in the cache in order to inflate in a reasonable time.
If JGit is running in a multi-threaded server, such as Gerrit Code
Review, each thread needs its own DeltaBaseCache to prevent one thread
from evicting the other thread's relevant bases. Move the cache to be
per-ObjectReader, lazily allocated when required by a PackFile.
Change-Id: If9d5ed06728e813632ae96dcfb811f4860b276e8
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Fix ReceivePack connectivity validation with alternates
If a repository has an alternate object database, the alternate has
its references advertised as ".have" lines, which permits the client
to use these as delta base candidates when generating the pack. If
setCheckReferencedObjectsAreReachable(true) is used, these additional
have lines need to be considered in addition to the advertised refs.
Change-Id: Ie39c6696f9d3ff147ef4405cd5624f6011700ce5
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
We sometimes face the problem that the file .git/index.lock
can't deleted causing JGit operations to fail. Problem is
that LockFile.unlock() simply deletes the lockfile and ignores the
return value of File.delete(). Instead use
FileUtils.delete() with retry option. This method will retry the
deletion of the file at most 10 times with sleeps inbetween.
Bug: 335959
Change-Id: I9598edea9f2304fe12e6f470301211b503434848
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
There should be a way to explictly refresh the refs cached in the
RefDirectory. Since commit c261b28 (use of FileSnapshot) this is
not needed anymore for storage in the filesystem. But for DHT based
storage an explicit refresh may be needed.
Change-Id: I7d30c3496c05e1fb6e9519f3af9f23c6adb93bf9
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Instead of tracking the length and modification time by hand, rely
on FileSnapshot to tell RefDirectory when the $GIT_DIR/packed-refs
file has been changed or should be re-read from disk.
Change-Id: I067d268dfdca1d39c72dfa536b34e6a239117cc3
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
When the client is clearly making a smart HTTP request to our smart
HTTP server, return any errors like RepositoryNotFoundException or
ServiceNotEnabledException inside of the payload as a Git level ERR
message, rather than an HTTP error code.
This prevents the C Git command line client from retrying a failed
"$URL/info/refs?service=git-upload-pack" request without the smart
service URL, only to fail again with "403 Forbidden" when the dumb
as-is service has been disabled by the server configuration, or is
unavailable because the repository is not on the local filesystem.
Change-Id: I57e8756d5026e885e0ca615979bfcd729703be6c
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
UploadPack: Add a PreUploadHook to monitor and control behavior
Embedding applications can use this hook to watch actions within
UploadPack and possibly reject them. This could be useful to prevent
clones of a large repository from this server, or to stop abusive
negotiation rounds that offer thousands of objects in a single batch.
Change-Id: Id96f1885ac4d61f22c80b6418fff54184b7348ba
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Allow application filters on smart HTTP operations
Permit applications embedding GitServlet to wrap the
info/refs?service=$name and /$name operations with a
servlet Filter.
To help applications inspect state of the operation,
expose the UploadPack or ReceivePack object into a
request attribute. This can be useful for logging,
or to implement throttling of requests like Gerrit
Code Review uses to prevent server overload.
Change-Id: Ib8773c14e2b7a650769bd578aad745e6651210cb
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
PackWriter: Fix the way delta chain cycles are prevented
Take a very simple approach to avoiding delta chains during object
reuse: objects are now always selected from the oldest pack that
contains them. This prevents cycles because a pack must not have
a cycle in the delta chain. If both objects A and B are chosen
out of the same source pack then there cannot be an A->B->A cycle.
The oldest pack is also the most likely to have the smallest deltas.
Its the biggest pack in the system and probably came from the clone
(or last GC) of this repository, where all objects were previously
considered and packed tightly together. If an object appears again
(for example due to a revert and a push into this repository) the
newer copy of won't be nearly as small as the older delta version
of it, even if the newer one is also itself a delta.
ObjectDirectory already enumerates objects during selection in this
newest->oldest order, so it already is supplying these assumptions
to PackWriter. Taking advantage of this can speed up selection by
a tiny amount by avoiding some tests, but can also help to prevent
a cycle needing to be broken on the fly during writing.
The previous cycle breaking logic wasn't fully correct either.
If a different delta base was chosen, the new delta base might not
have been written into the output pack before the current object,
forcing the use of REF_DELTA when OFS_DELTA is always smaller.
This logic has now been reworked to always re-check the delta base
and ensure it gets written before the current object.
If a cycle occurs, it gets broken the same way as before, by
disabling delta reuse and finding an alternative form of the
object, which may require inflating/deflating in whole format.
Change-Id: I9953ab8be54ceb8b588e1280d6f7edd688887747
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
If the total number of objects to look for reuse on is under 4096
this is really close to a reasonable batch size for the DHT storage
system to lookup at once. Combine all of the objects into a single
temporary list, perform reuse, and then prune the main lists if any
duplicate objects were detected from a selected CachedPack.
The intention here is to try and avoid 4 tiny sequential lookups
on the storage system when the time to wait for each of those to
finish is higher than the CPU time required to build (and later
GC) this temporary list.
Change-Id: I528daf9d2f7744dc4a6281750c2d61d8f9da9f3a
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Instead of looping over the objectsLists array, always set slot 0 to
null and explicitly work on the 4 indexes that matter. This kills
some loops and increases the length of the code slightly, but I've
always really disliked that dummy 0 slot.
Change-Id: I5ad938501c1c61f637ffdaff0d0d88e3962d8942
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
PackWriter: Speed up pruning of objects from cached packs
During object enumeration for the thin pack, very few objects come
out that are duplicated with the cached pack. Typically these are
only cases where a blob or tree was cherry-picked forward, got a
copy or rename, or was reverted... all relatively infrequent events.
Speed up pruning of the thin pack object list by combining the phase
with the object representation selection. Implementers should already
be offering to reuse the object from the cached pack if it is stored
there, at which point the implementation can perform a very fast type
of containment test using the cached pack's identity rather than yet
another index lookup. For the local disk case this is probably not a
big improvement, but it does help on the DHT implementation where the
two passes combined into one reduces latency.
Change-Id: I6a07fc75d9075bf6233e967360b6546f9e9a2b33
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
PacketLineOut is already public. Make PacketLineIn partially public
in case an application needs to use some of the pkt-line protocol.
Change-Id: I5b383eca980bd9e16a7dbdb5aed040c6586d4f46
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
It fixes 'Corrupted MAC on input' ssh issue.
CQ: 4728
Change-Id: I7de63cb3482488ac938566706edebee8a1cdad3a
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
If the clone command clones into a bare repository it should not try
to checkout HEAD in the end. For bare repos checkout is not possible.
Change-Id: I359719d421b93c9d2e962e3c0eccc2b59235c3d1
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
This merge command accepts the merge strategy as option and uses the
resolve strategy as default. It expects exactly one other
revision which is merged with current head.
Change-Id: Ia8c188b93ade4afabe6a9ccf267faf045f359a3a
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
To be more consistent with what native git does we should allow
to run the InitCommand also on existing git repos.
Change-Id: I833637842631b37dce96ed9729b3a6ed24054056
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
The snapshot field of a DirCache object for a newly created repository
can be null. This fix prevents a NPE when isModified() is called in
such a situation.
Change-Id: I61a1b45db2d60fdcc0f87373ac6fd75ac4c4a202
Signed-off-by: Philipp Thun <philipp.thun@sap.com>
This change contains a simple renaming. Instead of using the
expression 'abnormal failure', we just treat this kind of situation
as 'failure'. This is specific enough as conflicts are already handled
separately.
Change-Id: I535acdc7d022543ed0f5ac6151b09a6985f4ef38
Signed-off-by: Philipp Thun <philipp.thun@sap.com>
We used to normalize URI's since it seems simple. This however causes
inconsistencies to the user and to out tests. Just pass backslashes
through and make sure our parser can handle them.
Bug: 341062
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
Change-Id: I2c8e917a086faabcd8749160c2acc9dd05a42838
Matthias Sohn
Shawn O. Pearce
Stefan Lay
Leonard Broman
Chris Aniszczyk
Robin Stocker
Philipp Thun
Christian Halstrick
Carsten Pfeiffer
Anatol Pomozov
Christian Halstrick
Robin Rosenberg