You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

ReceivePack.java 33KB

Rewrite reference handling to be abstract and accurate This commit actually does three major changes to the way references are handled within JGit. Unfortunately they were easier to do as a single massive commit than to break them up into smaller units. Disambiguate symbolic references: --------------------------------- Reporting a symbolic reference such as HEAD as though it were any other normal reference like refs/heads/master causes subtle programming errors. We have been bitten by this error on several occasions, as have some downstream applications written by myself. Instead of reporting HEAD as a reference whose name differs from its "original name", report it as an actual SymbolicRef object that the application can test the type and examine the target of. With this change, Ref is now an abstract type with different subclasses for the different types. In the classical example of "HEAD" being a symbolic reference to branch "refs/heads/master", the Repository.getAllRefs() method will now return: Map<String, Ref> all = repository.getAllRefs(); SymbolicRef HEAD = (SymbolicRef) all.get("HEAD"); ObjectIdRef master = (ObjectIdRef) all.get("refs/heads/master"); assertSame(master, HEAD.getTarget()); assertSame(master.getObjectId(), HEAD.getObjectId()); assertEquals("HEAD", HEAD.getName()); assertEquals("refs/heads/master", master.getName()); A nice side-effect of this change is the storage type of the symbolic reference is no longer ambiguous with the storge type of the underlying reference it targets. In the above example, if master was only available in the packed-refs file, then the following is also true: assertSame(Ref.Storage.LOOSE, HEAD.getStorage()); assertSame(Ref.Storage.PACKED, master.getStorage()); (Prior to this change we returned the ambiguous storage of LOOSE_PACKED for HEAD, which was confusing since it wasn't actually true on disk). Another nice side-effect of this change is all intermediate symbolic references are preserved, and are therefore visible to the application when they walk the target chain. We can now correctly inspect chains of symbolic references. As a result of this change the Ref.getOrigName() method has been removed from the API. Applications should identify a symbolic reference by testing for isSymbolic() and not by using an arcane string comparsion between properties. Abstract the RefDatabase storage: --------------------------------- RefDatabase is now abstract, similar to ObjectDatabase, and a new concrete implementation called RefDirectory is used for the traditional on-disk storage layout. In the future we plan to support additional implementations, such as a pure in-memory RefDatabase for unit testing purposes. Optimize RefDirectory: ---------------------- The implementation of the in-memory reference cache, reading, and update routines has been completely rewritten. Much of the code was heavily borrowed or cribbed from the prior implementation, so copyright notices have been left intact as much as possible. The RefDirectory cache no longer confuses symbolic references with normal references. This permits the cache to resolve the value of a symbolic reference as late as possible, ensuring it is always current, without needing to maintain reverse pointers. The cache is now 2 sorted RefLists, rather than 3 HashMaps. Using sorted lists allows the implementation to reduce the in-memory footprint when storing many refs. Using specialized types for the elements allows the code to avoid additional map lookups for auxiliary stat information. To improve scan time during getRefs(), the lists are returned via a copy-on-write contract. Most callers of getRefs() do not modify the returned collections, so the copy-on-write semantics improves access on repositories with a large number of packed references. Iterator traversals of the returned Map<String,Ref> are performed using a simple merge-join of the two cache lists, ensuring we can perform the entire traversal in linear time as a function of the number of references: O(PackedRefs + LooseRefs). Scans of the loose reference space to update the cache run in O(LooseRefs log LooseRefs) time, as the directory contents are sorted before being merged against the in-memory cache. Since the majority of stable references are kept packed, there typically are only a handful of reference names to be sorted, so the sorting cost should not be very high. Locking is reduced during getRefs() by taking advantage of the copy-on-write semantics of the improved cache data structure. This permits concurrent readers to pull back references without blocking each other. If there is contention updating the cache during a scan, one or more updates are simply skipped and will get picked up again in a future scan. Writing to the $GIT_DIR/packed-refs during reference delete is now fully atomic. The file is locked, reparsed fresh, and written back out if a change is necessary. This avoids all race conditions with concurrent external updates of the packed-refs file. The RefLogWriter class has been fully folded into RefDirectory and is therefore deleted. Maintaining the reference's log is the responsiblity of the database implementation, and not all implementations will use java.io for access. Future work still remains to be done to abstract the ReflogReader class away from local disk IO. Change-Id: I26b9287c45a4b2d2be35ba2849daa316f5eec85d Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
14 years ago
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571572573574575576577578579580581582583584585586587588589590591592593594595596597598599600601602603604605606607608609610611612613614615616617618619620621622623624625626627628629630631632633634635636637638639640641642643644645646647648649650651652653654655656657658659660661662663664665666667668669670671672673674675676677678679680681682683684685686687688689690691692693694695696697698699700701702703704705706707708709710711712713714715716717718719720721722723724725726727728729730731732733734735736737738739740741742743744745746747748749750751752753754755756757758759760761762763764765766767768769770771772773774775776777778779780781782783784785786787788789790791792793794795796797798799800801802803804805806807808809810811812813814815816817818819820821822823824825826827828829830831832833834835836837838839840841842843844845846847848849850851852853854855856857858859860861862863864865866867868869870871872873874875876877878879880881882883884885886887888889890891892893894895896897898899900901902903904905906907908909910911912913914915916917918919920921922923924925926927928929930931932933934935936937938939940941942943944945946947948949950951952953954955956957958959960961962963964965966967968969970971972973974975976977978979980981982983984985986987988989990991992993994995996997998999100010011002100310041005100610071008100910101011101210131014101510161017101810191020102110221023102410251026102710281029103010311032103310341035103610371038103910401041104210431044104510461047104810491050105110521053105410551056105710581059106010611062106310641065106610671068106910701071107210731074107510761077107810791080108110821083108410851086108710881089109010911092109310941095109610971098109911001101110211031104110511061107110811091110111111121113111411151116111711181119112011211122112311241125112611271128112911301131
  1. /*
  2. * Copyright (C) 2008-2010, Google Inc.
  3. * and other copyright owners as documented in the project's IP log.
  4. *
  5. * This program and the accompanying materials are made available
  6. * under the terms of the Eclipse Distribution License v1.0 which
  7. * accompanies this distribution, is reproduced below, and is
  8. * available at http://www.eclipse.org/org/documents/edl-v10.php
  9. *
  10. * All rights reserved.
  11. *
  12. * Redistribution and use in source and binary forms, with or
  13. * without modification, are permitted provided that the following
  14. * conditions are met:
  15. *
  16. * - Redistributions of source code must retain the above copyright
  17. * notice, this list of conditions and the following disclaimer.
  18. *
  19. * - Redistributions in binary form must reproduce the above
  20. * copyright notice, this list of conditions and the following
  21. * disclaimer in the documentation and/or other materials provided
  22. * with the distribution.
  23. *
  24. * - Neither the name of the Eclipse Foundation, Inc. nor the
  25. * names of its contributors may be used to endorse or promote
  26. * products derived from this software without specific prior
  27. * written permission.
  28. *
  29. * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
  30. * CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
  31. * INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
  32. * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
  33. * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
  34. * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
  35. * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
  36. * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
  37. * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
  38. * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
  39. * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
  40. * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
  41. * ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  42. */
  43. package org.eclipse.jgit.transport;
  44. import static org.eclipse.jgit.transport.BasePackPushConnection.CAPABILITY_DELETE_REFS;
  45. import static org.eclipse.jgit.transport.BasePackPushConnection.CAPABILITY_OFS_DELTA;
  46. import static org.eclipse.jgit.transport.BasePackPushConnection.CAPABILITY_REPORT_STATUS;
  47. import static org.eclipse.jgit.transport.BasePackPushConnection.CAPABILITY_SIDE_BAND_64K;
  48. import static org.eclipse.jgit.transport.SideBandOutputStream.CH_DATA;
  49. import static org.eclipse.jgit.transport.SideBandOutputStream.CH_PROGRESS;
  50. import static org.eclipse.jgit.transport.SideBandOutputStream.MAX_BUF;
  51. import java.io.EOFException;
  52. import java.io.IOException;
  53. import java.io.InputStream;
  54. import java.io.OutputStream;
  55. import java.io.OutputStreamWriter;
  56. import java.io.Writer;
  57. import java.text.MessageFormat;
  58. import java.util.ArrayList;
  59. import java.util.Collections;
  60. import java.util.HashSet;
  61. import java.util.List;
  62. import java.util.Map;
  63. import java.util.Set;
  64. import org.eclipse.jgit.JGitText;
  65. import org.eclipse.jgit.errors.MissingObjectException;
  66. import org.eclipse.jgit.errors.PackProtocolException;
  67. import org.eclipse.jgit.errors.UnpackException;
  68. import org.eclipse.jgit.lib.Config;
  69. import org.eclipse.jgit.lib.Constants;
  70. import org.eclipse.jgit.lib.NullProgressMonitor;
  71. import org.eclipse.jgit.lib.ObjectId;
  72. import org.eclipse.jgit.lib.ObjectIdSubclassMap;
  73. import org.eclipse.jgit.lib.PersonIdent;
  74. import org.eclipse.jgit.lib.Ref;
  75. import org.eclipse.jgit.lib.RefUpdate;
  76. import org.eclipse.jgit.lib.Repository;
  77. import org.eclipse.jgit.lib.Config.SectionParser;
  78. import org.eclipse.jgit.revwalk.ObjectWalk;
  79. import org.eclipse.jgit.revwalk.RevBlob;
  80. import org.eclipse.jgit.revwalk.RevCommit;
  81. import org.eclipse.jgit.revwalk.RevFlag;
  82. import org.eclipse.jgit.revwalk.RevObject;
  83. import org.eclipse.jgit.revwalk.RevTree;
  84. import org.eclipse.jgit.revwalk.RevWalk;
  85. import org.eclipse.jgit.storage.file.PackLock;
  86. import org.eclipse.jgit.transport.ReceiveCommand.Result;
  87. import org.eclipse.jgit.transport.RefAdvertiser.PacketLineOutRefAdvertiser;
  88. import org.eclipse.jgit.util.io.InterruptTimer;
  89. import org.eclipse.jgit.util.io.TimeoutInputStream;
  90. import org.eclipse.jgit.util.io.TimeoutOutputStream;
  91. /**
  92. * Implements the server side of a push connection, receiving objects.
  93. */
  94. public class ReceivePack {
  95. /** Database we write the stored objects into. */
  96. private final Repository db;
  97. /** Revision traversal support over {@link #db}. */
  98. private final RevWalk walk;
  99. /**
  100. * Is the client connection a bi-directional socket or pipe?
  101. * <p>
  102. * If true, this class assumes it can perform multiple read and write cycles
  103. * with the client over the input and output streams. This matches the
  104. * functionality available with a standard TCP/IP connection, or a local
  105. * operating system or in-memory pipe.
  106. * <p>
  107. * If false, this class runs in a read everything then output results mode,
  108. * making it suitable for single round-trip systems RPCs such as HTTP.
  109. */
  110. private boolean biDirectionalPipe = true;
  111. /** Should an incoming transfer validate objects? */
  112. private boolean checkReceivedObjects;
  113. /** Should an incoming transfer permit create requests? */
  114. private boolean allowCreates;
  115. /** Should an incoming transfer permit delete requests? */
  116. private boolean allowDeletes;
  117. /** Should an incoming transfer permit non-fast-forward requests? */
  118. private boolean allowNonFastForwards;
  119. private boolean allowOfsDelta;
  120. /** Identity to record action as within the reflog. */
  121. private PersonIdent refLogIdent;
  122. /** Filter used while advertising the refs to the client. */
  123. private RefFilter refFilter;
  124. /** Hook to validate the update commands before execution. */
  125. private PreReceiveHook preReceive;
  126. /** Hook to report on the commands after execution. */
  127. private PostReceiveHook postReceive;
  128. /** Timeout in seconds to wait for client interaction. */
  129. private int timeout;
  130. /** Timer to manage {@link #timeout}. */
  131. private InterruptTimer timer;
  132. private TimeoutInputStream timeoutIn;
  133. private InputStream rawIn;
  134. private OutputStream rawOut;
  135. private PacketLineIn pckIn;
  136. private PacketLineOut pckOut;
  137. private Writer msgs;
  138. private IndexPack ip;
  139. /** The refs we advertised as existing at the start of the connection. */
  140. private Map<String, Ref> refs;
  141. /** Capabilities requested by the client. */
  142. private Set<String> enabledCapablities;
  143. /** Commands to execute, as received by the client. */
  144. private List<ReceiveCommand> commands;
  145. /** Error to display instead of advertising the references. */
  146. private StringBuilder advertiseError;
  147. /** An exception caught while unpacking and fsck'ing the objects. */
  148. private Throwable unpackError;
  149. /** If {@link BasePackPushConnection#CAPABILITY_REPORT_STATUS} is enabled. */
  150. private boolean reportStatus;
  151. /** If {@link BasePackPushConnection#CAPABILITY_SIDE_BAND_64K} is enabled. */
  152. private boolean sideBand;
  153. /** Lock around the received pack file, while updating refs. */
  154. private PackLock packLock;
  155. private boolean checkReferencedIsReachable;
  156. /**
  157. * Create a new pack receive for an open repository.
  158. *
  159. * @param into
  160. * the destination repository.
  161. */
  162. public ReceivePack(final Repository into) {
  163. db = into;
  164. walk = new RevWalk(db);
  165. final ReceiveConfig cfg = db.getConfig().get(ReceiveConfig.KEY);
  166. checkReceivedObjects = cfg.checkReceivedObjects;
  167. allowCreates = cfg.allowCreates;
  168. allowDeletes = cfg.allowDeletes;
  169. allowNonFastForwards = cfg.allowNonFastForwards;
  170. allowOfsDelta = cfg.allowOfsDelta;
  171. refFilter = RefFilter.DEFAULT;
  172. preReceive = PreReceiveHook.NULL;
  173. postReceive = PostReceiveHook.NULL;
  174. }
  175. private static class ReceiveConfig {
  176. static final SectionParser<ReceiveConfig> KEY = new SectionParser<ReceiveConfig>() {
  177. public ReceiveConfig parse(final Config cfg) {
  178. return new ReceiveConfig(cfg);
  179. }
  180. };
  181. final boolean checkReceivedObjects;
  182. final boolean allowCreates;
  183. final boolean allowDeletes;
  184. final boolean allowNonFastForwards;
  185. final boolean allowOfsDelta;
  186. ReceiveConfig(final Config config) {
  187. checkReceivedObjects = config.getBoolean("receive", "fsckobjects",
  188. false);
  189. allowCreates = true;
  190. allowDeletes = !config.getBoolean("receive", "denydeletes", false);
  191. allowNonFastForwards = !config.getBoolean("receive",
  192. "denynonfastforwards", false);
  193. allowOfsDelta = config.getBoolean("repack", "usedeltabaseoffset",
  194. true);
  195. }
  196. }
  197. /** @return the repository this receive completes into. */
  198. public final Repository getRepository() {
  199. return db;
  200. }
  201. /** @return the RevWalk instance used by this connection. */
  202. public final RevWalk getRevWalk() {
  203. return walk;
  204. }
  205. /** @return all refs which were advertised to the client. */
  206. public final Map<String, Ref> getAdvertisedRefs() {
  207. return refs;
  208. }
  209. /**
  210. * @return true if this instance will validate all referenced, but not
  211. * supplied by the client, objects are reachable from another
  212. * reference.
  213. */
  214. public boolean isCheckReferencedObjectsAreReachable() {
  215. return checkReferencedIsReachable;
  216. }
  217. /**
  218. * Validate all referenced but not supplied objects are reachable.
  219. * <p>
  220. * If enabled, this instance will verify that references to objects not
  221. * contained within the received pack are already reachable through at least
  222. * one other reference selected by the {@link #getRefFilter()} and displayed
  223. * as part of {@link #getAdvertisedRefs()}.
  224. * <p>
  225. * This feature is useful when the application doesn't trust the client to
  226. * not provide a forged SHA-1 reference to an object, in an attempt to
  227. * access parts of the DAG that they aren't allowed to see and which have
  228. * been hidden from them via the configured {@link RefFilter}.
  229. * <p>
  230. * Enabling this feature may imply at least some, if not all, of the same
  231. * functionality performed by {@link #setCheckReceivedObjects(boolean)}.
  232. * Applications are encouraged to enable both features, if desired.
  233. *
  234. * @param b
  235. * {@code true} to enable the additional check.
  236. */
  237. public void setCheckReferencedObjectsAreReachable(boolean b) {
  238. this.checkReferencedIsReachable = b;
  239. }
  240. /**
  241. * @return true if this class expects a bi-directional pipe opened between
  242. * the client and itself. The default is true.
  243. */
  244. public boolean isBiDirectionalPipe() {
  245. return biDirectionalPipe;
  246. }
  247. /**
  248. * @param twoWay
  249. * if true, this class will assume the socket is a fully
  250. * bidirectional pipe between the two peers and takes advantage
  251. * of that by first transmitting the known refs, then waiting to
  252. * read commands. If false, this class assumes it must read the
  253. * commands before writing output and does not perform the
  254. * initial advertising.
  255. */
  256. public void setBiDirectionalPipe(final boolean twoWay) {
  257. biDirectionalPipe = twoWay;
  258. }
  259. /**
  260. * @return true if this instance will verify received objects are formatted
  261. * correctly. Validating objects requires more CPU time on this side
  262. * of the connection.
  263. */
  264. public boolean isCheckReceivedObjects() {
  265. return checkReceivedObjects;
  266. }
  267. /**
  268. * @param check
  269. * true to enable checking received objects; false to assume all
  270. * received objects are valid.
  271. */
  272. public void setCheckReceivedObjects(final boolean check) {
  273. checkReceivedObjects = check;
  274. }
  275. /** @return true if the client can request refs to be created. */
  276. public boolean isAllowCreates() {
  277. return allowCreates;
  278. }
  279. /**
  280. * @param canCreate
  281. * true to permit create ref commands to be processed.
  282. */
  283. public void setAllowCreates(final boolean canCreate) {
  284. allowCreates = canCreate;
  285. }
  286. /** @return true if the client can request refs to be deleted. */
  287. public boolean isAllowDeletes() {
  288. return allowDeletes;
  289. }
  290. /**
  291. * @param canDelete
  292. * true to permit delete ref commands to be processed.
  293. */
  294. public void setAllowDeletes(final boolean canDelete) {
  295. allowDeletes = canDelete;
  296. }
  297. /**
  298. * @return true if the client can request non-fast-forward updates of a ref,
  299. * possibly making objects unreachable.
  300. */
  301. public boolean isAllowNonFastForwards() {
  302. return allowNonFastForwards;
  303. }
  304. /**
  305. * @param canRewind
  306. * true to permit the client to ask for non-fast-forward updates
  307. * of an existing ref.
  308. */
  309. public void setAllowNonFastForwards(final boolean canRewind) {
  310. allowNonFastForwards = canRewind;
  311. }
  312. /** @return identity of the user making the changes in the reflog. */
  313. public PersonIdent getRefLogIdent() {
  314. return refLogIdent;
  315. }
  316. /**
  317. * Set the identity of the user appearing in the affected reflogs.
  318. * <p>
  319. * The timestamp portion of the identity is ignored. A new identity with the
  320. * current timestamp will be created automatically when the updates occur
  321. * and the log records are written.
  322. *
  323. * @param pi
  324. * identity of the user. If null the identity will be
  325. * automatically determined based on the repository
  326. * configuration.
  327. */
  328. public void setRefLogIdent(final PersonIdent pi) {
  329. refLogIdent = pi;
  330. }
  331. /** @return the filter used while advertising the refs to the client */
  332. public RefFilter getRefFilter() {
  333. return refFilter;
  334. }
  335. /**
  336. * Set the filter used while advertising the refs to the client.
  337. * <p>
  338. * Only refs allowed by this filter will be shown to the client.
  339. * Clients may still attempt to create or update a reference hidden
  340. * by the configured {@link RefFilter}. These attempts should be
  341. * rejected by a matching {@link PreReceiveHook}.
  342. *
  343. * @param refFilter
  344. * the filter; may be null to show all refs.
  345. */
  346. public void setRefFilter(final RefFilter refFilter) {
  347. this.refFilter = refFilter != null ? refFilter : RefFilter.DEFAULT;
  348. }
  349. /** @return get the hook invoked before updates occur. */
  350. public PreReceiveHook getPreReceiveHook() {
  351. return preReceive;
  352. }
  353. /**
  354. * Set the hook which is invoked prior to commands being executed.
  355. * <p>
  356. * Only valid commands (those which have no obvious errors according to the
  357. * received input and this instance's configuration) are passed into the
  358. * hook. The hook may mark a command with a result of any value other than
  359. * {@link Result#NOT_ATTEMPTED} to block its execution.
  360. * <p>
  361. * The hook may be called with an empty command collection if the current
  362. * set is completely invalid.
  363. *
  364. * @param h
  365. * the hook instance; may be null to disable the hook.
  366. */
  367. public void setPreReceiveHook(final PreReceiveHook h) {
  368. preReceive = h != null ? h : PreReceiveHook.NULL;
  369. }
  370. /** @return get the hook invoked after updates occur. */
  371. public PostReceiveHook getPostReceiveHook() {
  372. return postReceive;
  373. }
  374. /**
  375. * Set the hook which is invoked after commands are executed.
  376. * <p>
  377. * Only successful commands (type is {@link Result#OK}) are passed into the
  378. * hook. The hook may be called with an empty command collection if the
  379. * current set all resulted in an error.
  380. *
  381. * @param h
  382. * the hook instance; may be null to disable the hook.
  383. */
  384. public void setPostReceiveHook(final PostReceiveHook h) {
  385. postReceive = h != null ? h : PostReceiveHook.NULL;
  386. }
  387. /** @return timeout (in seconds) before aborting an IO operation. */
  388. public int getTimeout() {
  389. return timeout;
  390. }
  391. /**
  392. * Set the timeout before willing to abort an IO call.
  393. *
  394. * @param seconds
  395. * number of seconds to wait (with no data transfer occurring)
  396. * before aborting an IO read or write operation with the
  397. * connected client.
  398. */
  399. public void setTimeout(final int seconds) {
  400. timeout = seconds;
  401. }
  402. /** @return all of the command received by the current request. */
  403. public List<ReceiveCommand> getAllCommands() {
  404. return Collections.unmodifiableList(commands);
  405. }
  406. /**
  407. * Send an error message to the client.
  408. * <p>
  409. * If any error messages are sent before the references are advertised to
  410. * the client, the errors will be sent instead of the advertisement and the
  411. * receive operation will be aborted. All clients should receive and display
  412. * such early stage errors.
  413. * <p>
  414. * If the reference advertisements have already been sent, messages are sent
  415. * in a side channel. If the client doesn't support receiving messages, the
  416. * message will be discarded, with no other indication to the caller or to
  417. * the client.
  418. * <p>
  419. * {@link PreReceiveHook}s should always try to use
  420. * {@link ReceiveCommand#setResult(Result, String)} with a result status of
  421. * {@link Result#REJECTED_OTHER_REASON} to indicate any reasons for
  422. * rejecting an update. Messages attached to a command are much more likely
  423. * to be returned to the client.
  424. *
  425. * @param what
  426. * string describing the problem identified by the hook. The
  427. * string must not end with an LF, and must not contain an LF.
  428. */
  429. public void sendError(final String what) {
  430. if (refs == null) {
  431. if (advertiseError == null)
  432. advertiseError = new StringBuilder();
  433. advertiseError.append(what).append('\n');
  434. } else {
  435. try {
  436. if (msgs != null)
  437. msgs.write("error: " + what + "\n");
  438. } catch (IOException e) {
  439. // Ignore write failures.
  440. }
  441. }
  442. }
  443. /**
  444. * Send a message to the client, if it supports receiving them.
  445. * <p>
  446. * If the client doesn't support receiving messages, the message will be
  447. * discarded, with no other indication to the caller or to the client.
  448. *
  449. * @param what
  450. * string describing the problem identified by the hook. The
  451. * string must not end with an LF, and must not contain an LF.
  452. */
  453. public void sendMessage(final String what) {
  454. try {
  455. if (msgs != null)
  456. msgs.write(what + "\n");
  457. } catch (IOException e) {
  458. // Ignore write failures.
  459. }
  460. }
  461. /**
  462. * Execute the receive task on the socket.
  463. *
  464. * @param input
  465. * raw input to read client commands and pack data from. Caller
  466. * must ensure the input is buffered, otherwise read performance
  467. * may suffer.
  468. * @param output
  469. * response back to the Git network client. Caller must ensure
  470. * the output is buffered, otherwise write performance may
  471. * suffer.
  472. * @param messages
  473. * secondary "notice" channel to send additional messages out
  474. * through. When run over SSH this should be tied back to the
  475. * standard error channel of the command execution. For most
  476. * other network connections this should be null.
  477. * @throws IOException
  478. */
  479. public void receive(final InputStream input, final OutputStream output,
  480. final OutputStream messages) throws IOException {
  481. try {
  482. rawIn = input;
  483. rawOut = output;
  484. if (timeout > 0) {
  485. final Thread caller = Thread.currentThread();
  486. timer = new InterruptTimer(caller.getName() + "-Timer");
  487. timeoutIn = new TimeoutInputStream(rawIn, timer);
  488. TimeoutOutputStream o = new TimeoutOutputStream(rawOut, timer);
  489. timeoutIn.setTimeout(timeout * 1000);
  490. o.setTimeout(timeout * 1000);
  491. rawIn = timeoutIn;
  492. rawOut = o;
  493. }
  494. pckIn = new PacketLineIn(rawIn);
  495. pckOut = new PacketLineOut(rawOut);
  496. if (messages != null)
  497. msgs = new OutputStreamWriter(messages, Constants.CHARSET);
  498. enabledCapablities = new HashSet<String>();
  499. commands = new ArrayList<ReceiveCommand>();
  500. service();
  501. } finally {
  502. walk.release();
  503. try {
  504. if (pckOut != null)
  505. pckOut.flush();
  506. if (msgs != null)
  507. msgs.flush();
  508. if (sideBand) {
  509. // If we are using side band, we need to send a final
  510. // flush-pkt to tell the remote peer the side band is
  511. // complete and it should stop decoding. We need to
  512. // use the original output stream as rawOut is now the
  513. // side band data channel.
  514. //
  515. new PacketLineOut(output).end();
  516. }
  517. } finally {
  518. unlockPack();
  519. timeoutIn = null;
  520. rawIn = null;
  521. rawOut = null;
  522. pckIn = null;
  523. pckOut = null;
  524. msgs = null;
  525. refs = null;
  526. enabledCapablities = null;
  527. commands = null;
  528. if (timer != null) {
  529. try {
  530. timer.terminate();
  531. } finally {
  532. timer = null;
  533. }
  534. }
  535. }
  536. }
  537. }
  538. private void service() throws IOException {
  539. if (biDirectionalPipe)
  540. sendAdvertisedRefs(new PacketLineOutRefAdvertiser(pckOut));
  541. else
  542. refs = refFilter.filter(db.getAllRefs());
  543. if (advertiseError != null)
  544. return;
  545. recvCommands();
  546. if (!commands.isEmpty()) {
  547. enableCapabilities();
  548. if (needPack()) {
  549. try {
  550. receivePack();
  551. if (needCheckConnectivity())
  552. checkConnectivity();
  553. ip = null;
  554. unpackError = null;
  555. } catch (IOException err) {
  556. unpackError = err;
  557. } catch (RuntimeException err) {
  558. unpackError = err;
  559. } catch (Error err) {
  560. unpackError = err;
  561. }
  562. }
  563. if (unpackError == null) {
  564. validateCommands();
  565. executeCommands();
  566. }
  567. unlockPack();
  568. if (reportStatus) {
  569. sendStatusReport(true, new Reporter() {
  570. void sendString(final String s) throws IOException {
  571. pckOut.writeString(s + "\n");
  572. }
  573. });
  574. pckOut.end();
  575. } else if (msgs != null) {
  576. sendStatusReport(false, new Reporter() {
  577. void sendString(final String s) throws IOException {
  578. msgs.write(s + "\n");
  579. }
  580. });
  581. }
  582. postReceive.onPostReceive(this, filterCommands(Result.OK));
  583. if (unpackError != null)
  584. throw new UnpackException(unpackError);
  585. }
  586. }
  587. private void unlockPack() {
  588. if (packLock != null) {
  589. packLock.unlock();
  590. packLock = null;
  591. }
  592. }
  593. /**
  594. * Generate an advertisement of available refs and capabilities.
  595. *
  596. * @param adv
  597. * the advertisement formatter.
  598. * @throws IOException
  599. * the formatter failed to write an advertisement.
  600. */
  601. public void sendAdvertisedRefs(final RefAdvertiser adv) throws IOException {
  602. if (advertiseError != null) {
  603. adv.writeOne("ERR " + advertiseError);
  604. return;
  605. }
  606. final RevFlag advertised = walk.newFlag("ADVERTISED");
  607. adv.init(walk, advertised);
  608. adv.advertiseCapability(CAPABILITY_SIDE_BAND_64K);
  609. adv.advertiseCapability(CAPABILITY_DELETE_REFS);
  610. adv.advertiseCapability(CAPABILITY_REPORT_STATUS);
  611. if (allowOfsDelta)
  612. adv.advertiseCapability(CAPABILITY_OFS_DELTA);
  613. refs = refFilter.filter(db.getAllRefs());
  614. final Ref head = refs.remove(Constants.HEAD);
  615. adv.send(refs);
  616. if (head != null && !head.isSymbolic())
  617. adv.advertiseHave(head.getObjectId());
  618. adv.includeAdditionalHaves(db);
  619. if (adv.isEmpty())
  620. adv.advertiseId(ObjectId.zeroId(), "capabilities^{}");
  621. adv.end();
  622. }
  623. private void recvCommands() throws IOException {
  624. for (;;) {
  625. String line;
  626. try {
  627. line = pckIn.readStringRaw();
  628. } catch (EOFException eof) {
  629. if (commands.isEmpty())
  630. return;
  631. throw eof;
  632. }
  633. if (line == PacketLineIn.END)
  634. break;
  635. if (commands.isEmpty()) {
  636. final int nul = line.indexOf('\0');
  637. if (nul >= 0) {
  638. for (String c : line.substring(nul + 1).split(" "))
  639. enabledCapablities.add(c);
  640. line = line.substring(0, nul);
  641. }
  642. }
  643. if (line.length() < 83) {
  644. final String m = JGitText.get().errorInvalidProtocolWantedOldNewRef;
  645. sendError(m);
  646. throw new PackProtocolException(m);
  647. }
  648. final ObjectId oldId = ObjectId.fromString(line.substring(0, 40));
  649. final ObjectId newId = ObjectId.fromString(line.substring(41, 81));
  650. final String name = line.substring(82);
  651. final ReceiveCommand cmd = new ReceiveCommand(oldId, newId, name);
  652. if (name.equals(Constants.HEAD)) {
  653. cmd.setResult(Result.REJECTED_CURRENT_BRANCH);
  654. } else {
  655. cmd.setRef(refs.get(cmd.getRefName()));
  656. }
  657. commands.add(cmd);
  658. }
  659. }
  660. private void enableCapabilities() {
  661. reportStatus = enabledCapablities.contains(CAPABILITY_REPORT_STATUS);
  662. sideBand = enabledCapablities.contains(CAPABILITY_SIDE_BAND_64K);
  663. if (sideBand) {
  664. OutputStream out = rawOut;
  665. rawOut = new SideBandOutputStream(CH_DATA, MAX_BUF, out);
  666. pckOut = new PacketLineOut(rawOut);
  667. msgs = new OutputStreamWriter(new SideBandOutputStream(CH_PROGRESS,
  668. MAX_BUF, out), Constants.CHARSET);
  669. }
  670. }
  671. private boolean needPack() {
  672. for (final ReceiveCommand cmd : commands) {
  673. if (cmd.getType() != ReceiveCommand.Type.DELETE)
  674. return true;
  675. }
  676. return false;
  677. }
  678. private void receivePack() throws IOException {
  679. // It might take the client a while to pack the objects it needs
  680. // to send to us. We should increase our timeout so we don't
  681. // abort while the client is computing.
  682. //
  683. if (timeoutIn != null)
  684. timeoutIn.setTimeout(10 * timeout * 1000);
  685. ip = IndexPack.create(db, rawIn);
  686. ip.setFixThin(true);
  687. ip.setNeedNewObjectIds(checkReferencedIsReachable);
  688. ip.setNeedBaseObjectIds(checkReferencedIsReachable);
  689. ip.setObjectChecking(isCheckReceivedObjects());
  690. ip.index(NullProgressMonitor.INSTANCE);
  691. String lockMsg = "jgit receive-pack";
  692. if (getRefLogIdent() != null)
  693. lockMsg += " from " + getRefLogIdent().toExternalString();
  694. packLock = ip.renameAndOpenPack(lockMsg);
  695. if (timeoutIn != null)
  696. timeoutIn.setTimeout(timeout * 1000);
  697. }
  698. private boolean needCheckConnectivity() {
  699. return isCheckReceivedObjects()
  700. || isCheckReferencedObjectsAreReachable();
  701. }
  702. private void checkConnectivity() throws IOException {
  703. ObjectIdSubclassMap<ObjectId> baseObjects = null;
  704. ObjectIdSubclassMap<ObjectId> providedObjects = null;
  705. if (checkReferencedIsReachable) {
  706. baseObjects = ip.getBaseObjectIds();
  707. providedObjects = ip.getNewObjectIds();
  708. }
  709. ip = null;
  710. final ObjectWalk ow = new ObjectWalk(db);
  711. for (final ReceiveCommand cmd : commands) {
  712. if (cmd.getResult() != Result.NOT_ATTEMPTED)
  713. continue;
  714. if (cmd.getType() == ReceiveCommand.Type.DELETE)
  715. continue;
  716. ow.markStart(ow.parseAny(cmd.getNewId()));
  717. }
  718. for (final Ref ref : refs.values()) {
  719. RevObject o = ow.parseAny(ref.getObjectId());
  720. ow.markUninteresting(o);
  721. if (checkReferencedIsReachable && !baseObjects.isEmpty()) {
  722. o = ow.peel(o);
  723. if (o instanceof RevCommit)
  724. o = ((RevCommit) o).getTree();
  725. if (o instanceof RevTree)
  726. ow.markUninteresting(o);
  727. }
  728. }
  729. if (checkReferencedIsReachable) {
  730. for (ObjectId id : baseObjects) {
  731. RevObject b = ow.lookupAny(id, Constants.OBJ_BLOB);
  732. if (!b.has(RevFlag.UNINTERESTING))
  733. throw new MissingObjectException(b, b.getType());
  734. }
  735. }
  736. RevCommit c;
  737. while ((c = ow.next()) != null) {
  738. if (checkReferencedIsReachable && !providedObjects.contains(c))
  739. throw new MissingObjectException(c, Constants.TYPE_COMMIT);
  740. }
  741. RevObject o;
  742. while ((o = ow.nextObject()) != null) {
  743. if (checkReferencedIsReachable) {
  744. if (providedObjects.contains(o))
  745. continue;
  746. else
  747. throw new MissingObjectException(o, o.getType());
  748. }
  749. if (o instanceof RevBlob && !db.hasObject(o))
  750. throw new MissingObjectException(o, Constants.TYPE_BLOB);
  751. }
  752. }
  753. private void validateCommands() {
  754. for (final ReceiveCommand cmd : commands) {
  755. final Ref ref = cmd.getRef();
  756. if (cmd.getResult() != Result.NOT_ATTEMPTED)
  757. continue;
  758. if (cmd.getType() == ReceiveCommand.Type.DELETE
  759. && !isAllowDeletes()) {
  760. // Deletes are not supported on this repository.
  761. //
  762. cmd.setResult(Result.REJECTED_NODELETE);
  763. continue;
  764. }
  765. if (cmd.getType() == ReceiveCommand.Type.CREATE) {
  766. if (!isAllowCreates()) {
  767. cmd.setResult(Result.REJECTED_NOCREATE);
  768. continue;
  769. }
  770. if (ref != null && !isAllowNonFastForwards()) {
  771. // Creation over an existing ref is certainly not going
  772. // to be a fast-forward update. We can reject it early.
  773. //
  774. cmd.setResult(Result.REJECTED_NONFASTFORWARD);
  775. continue;
  776. }
  777. if (ref != null) {
  778. // A well behaved client shouldn't have sent us a
  779. // create command for a ref we advertised to it.
  780. //
  781. cmd.setResult(Result.REJECTED_OTHER_REASON, "ref exists");
  782. continue;
  783. }
  784. }
  785. if (cmd.getType() == ReceiveCommand.Type.DELETE && ref != null
  786. && !ObjectId.zeroId().equals(cmd.getOldId())
  787. && !ref.getObjectId().equals(cmd.getOldId())) {
  788. // Delete commands can be sent with the old id matching our
  789. // advertised value, *OR* with the old id being 0{40}. Any
  790. // other requested old id is invalid.
  791. //
  792. cmd.setResult(Result.REJECTED_OTHER_REASON,
  793. JGitText.get().invalidOldIdSent);
  794. continue;
  795. }
  796. if (cmd.getType() == ReceiveCommand.Type.UPDATE) {
  797. if (ref == null) {
  798. // The ref must have been advertised in order to be updated.
  799. //
  800. cmd.setResult(Result.REJECTED_OTHER_REASON, JGitText.get().noSuchRef);
  801. continue;
  802. }
  803. if (!ref.getObjectId().equals(cmd.getOldId())) {
  804. // A properly functioning client will send the same
  805. // object id we advertised.
  806. //
  807. cmd.setResult(Result.REJECTED_OTHER_REASON,
  808. JGitText.get().invalidOldIdSent);
  809. continue;
  810. }
  811. // Is this possibly a non-fast-forward style update?
  812. //
  813. RevObject oldObj, newObj;
  814. try {
  815. oldObj = walk.parseAny(cmd.getOldId());
  816. } catch (IOException e) {
  817. cmd.setResult(Result.REJECTED_MISSING_OBJECT, cmd
  818. .getOldId().name());
  819. continue;
  820. }
  821. try {
  822. newObj = walk.parseAny(cmd.getNewId());
  823. } catch (IOException e) {
  824. cmd.setResult(Result.REJECTED_MISSING_OBJECT, cmd
  825. .getNewId().name());
  826. continue;
  827. }
  828. if (oldObj instanceof RevCommit && newObj instanceof RevCommit) {
  829. try {
  830. if (!walk.isMergedInto((RevCommit) oldObj,
  831. (RevCommit) newObj)) {
  832. cmd
  833. .setType(ReceiveCommand.Type.UPDATE_NONFASTFORWARD);
  834. }
  835. } catch (MissingObjectException e) {
  836. cmd.setResult(Result.REJECTED_MISSING_OBJECT, e
  837. .getMessage());
  838. } catch (IOException e) {
  839. cmd.setResult(Result.REJECTED_OTHER_REASON);
  840. }
  841. } else {
  842. cmd.setType(ReceiveCommand.Type.UPDATE_NONFASTFORWARD);
  843. }
  844. }
  845. if (!cmd.getRefName().startsWith(Constants.R_REFS)
  846. || !Repository.isValidRefName(cmd.getRefName())) {
  847. cmd.setResult(Result.REJECTED_OTHER_REASON, JGitText.get().funnyRefname);
  848. }
  849. }
  850. }
  851. private void executeCommands() {
  852. preReceive.onPreReceive(this, filterCommands(Result.NOT_ATTEMPTED));
  853. for (final ReceiveCommand cmd : filterCommands(Result.NOT_ATTEMPTED))
  854. execute(cmd);
  855. }
  856. private void execute(final ReceiveCommand cmd) {
  857. try {
  858. final RefUpdate ru = db.updateRef(cmd.getRefName());
  859. ru.setRefLogIdent(getRefLogIdent());
  860. switch (cmd.getType()) {
  861. case DELETE:
  862. if (!ObjectId.zeroId().equals(cmd.getOldId())) {
  863. // We can only do a CAS style delete if the client
  864. // didn't bork its delete request by sending the
  865. // wrong zero id rather than the advertised one.
  866. //
  867. ru.setExpectedOldObjectId(cmd.getOldId());
  868. }
  869. ru.setForceUpdate(true);
  870. status(cmd, ru.delete(walk));
  871. break;
  872. case CREATE:
  873. case UPDATE:
  874. case UPDATE_NONFASTFORWARD:
  875. ru.setForceUpdate(isAllowNonFastForwards());
  876. ru.setExpectedOldObjectId(cmd.getOldId());
  877. ru.setNewObjectId(cmd.getNewId());
  878. ru.setRefLogMessage("push", true);
  879. status(cmd, ru.update(walk));
  880. break;
  881. }
  882. } catch (IOException err) {
  883. cmd.setResult(Result.REJECTED_OTHER_REASON, MessageFormat.format(
  884. JGitText.get().lockError, err.getMessage()));
  885. }
  886. }
  887. private void status(final ReceiveCommand cmd, final RefUpdate.Result result) {
  888. switch (result) {
  889. case NOT_ATTEMPTED:
  890. cmd.setResult(Result.NOT_ATTEMPTED);
  891. break;
  892. case LOCK_FAILURE:
  893. case IO_FAILURE:
  894. cmd.setResult(Result.LOCK_FAILURE);
  895. break;
  896. case NO_CHANGE:
  897. case NEW:
  898. case FORCED:
  899. case FAST_FORWARD:
  900. cmd.setResult(Result.OK);
  901. break;
  902. case REJECTED:
  903. cmd.setResult(Result.REJECTED_NONFASTFORWARD);
  904. break;
  905. case REJECTED_CURRENT_BRANCH:
  906. cmd.setResult(Result.REJECTED_CURRENT_BRANCH);
  907. break;
  908. default:
  909. cmd.setResult(Result.REJECTED_OTHER_REASON, result.name());
  910. break;
  911. }
  912. }
  913. private List<ReceiveCommand> filterCommands(final Result want) {
  914. final List<ReceiveCommand> r = new ArrayList<ReceiveCommand>(commands
  915. .size());
  916. for (final ReceiveCommand cmd : commands) {
  917. if (cmd.getResult() == want)
  918. r.add(cmd);
  919. }
  920. return r;
  921. }
  922. private void sendStatusReport(final boolean forClient, final Reporter out)
  923. throws IOException {
  924. if (unpackError != null) {
  925. out.sendString("unpack error " + unpackError.getMessage());
  926. if (forClient) {
  927. for (final ReceiveCommand cmd : commands) {
  928. out.sendString("ng " + cmd.getRefName()
  929. + " n/a (unpacker error)");
  930. }
  931. }
  932. return;
  933. }
  934. if (forClient)
  935. out.sendString("unpack ok");
  936. for (final ReceiveCommand cmd : commands) {
  937. if (cmd.getResult() == Result.OK) {
  938. if (forClient)
  939. out.sendString("ok " + cmd.getRefName());
  940. continue;
  941. }
  942. final StringBuilder r = new StringBuilder();
  943. r.append("ng ");
  944. r.append(cmd.getRefName());
  945. r.append(" ");
  946. switch (cmd.getResult()) {
  947. case NOT_ATTEMPTED:
  948. r.append("server bug; ref not processed");
  949. break;
  950. case REJECTED_NOCREATE:
  951. r.append("creation prohibited");
  952. break;
  953. case REJECTED_NODELETE:
  954. r.append("deletion prohibited");
  955. break;
  956. case REJECTED_NONFASTFORWARD:
  957. r.append("non-fast forward");
  958. break;
  959. case REJECTED_CURRENT_BRANCH:
  960. r.append("branch is currently checked out");
  961. break;
  962. case REJECTED_MISSING_OBJECT:
  963. if (cmd.getMessage() == null)
  964. r.append("missing object(s)");
  965. else if (cmd.getMessage().length() == Constants.OBJECT_ID_STRING_LENGTH)
  966. r.append("object " + cmd.getMessage() + " missing");
  967. else
  968. r.append(cmd.getMessage());
  969. break;
  970. case REJECTED_OTHER_REASON:
  971. if (cmd.getMessage() == null)
  972. r.append("unspecified reason");
  973. else
  974. r.append(cmd.getMessage());
  975. break;
  976. case LOCK_FAILURE:
  977. r.append("failed to lock");
  978. break;
  979. case OK:
  980. // We shouldn't have reached this case (see 'ok' case above).
  981. continue;
  982. }
  983. out.sendString(r.toString());
  984. }
  985. }
  986. static abstract class Reporter {
  987. abstract void sendString(String s) throws IOException;
  988. }
  989. }