You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

DepthWalk.java 7.0KB

Shallow fetch: Respect "shallow" lines When fetching from a shallow clone, the client sends "have" lines to tell the server about objects it already has and "shallow" lines to tell where its local history terminates. In some circumstances, the server fails to honor the shallow lines and fails to return objects that the client needs. UploadPack passes the "have" lines to PackWriter so PackWriter can omit them from the generated pack. UploadPack processes "shallow" lines by calling RevWalk.assumeShallow() with the set of shallow commits. RevWalk creates and caches RevCommits for these shallow commits, clearing out their parents. That way, walks correctly terminate at the shallow commits instead of assuming the client has history going back behind them. UploadPack converts its RevWalk to an ObjectWalk, maintaining the cached RevCommits, and passes it to PackWriter. Unfortunately, to support shallow fetches the PackWriter does the following: if (shallowPack && !(walk instanceof DepthWalk.ObjectWalk)) walk = new DepthWalk.ObjectWalk(reader, depth); That is, when the client sends a "deepen" line (fetch --depth=<n>) and the caller has not passed in a DepthWalk.ObjectWalk, PackWriter throws away the RevWalk that was passed in and makes a new one. The cleared parent lists prepared by RevWalk.assumeShallow() are lost. Fortunately UploadPack intends to pass in a DepthWalk.ObjectWalk. It tries to create it by calling toObjectWalkWithSameObjects() on a DepthWalk.RevWalk. But it doesn't work: because DepthWalk.RevWalk does not override the standard RevWalk#toObjectWalkWithSameObjects implementation, the result is a plain ObjectWalk instead of an instance of DepthWalk.ObjectWalk. The result is that the "shallow" information is thrown away and objects reachable from the shallow commits can be omitted from the pack sent when fetching with --depth from a shallow clone. Multiple factors collude to limit the circumstances under which this bug can be observed: 1. Commits with depth != 0 don't enter DepthGenerator's pending queue. That means a "have" cannot have any effect on DepthGenerator unless it is also a "want". 2. DepthGenerator#next() doesn't call carryFlagsImpl(), so the uninteresting flag is not propagated to ancestors there even if a "have" is also a "want". 3. JGit treats a depth of 1 as "1 past the wants". Because of (2), the only place the UNINTERESTING flag can leak to a shallow commit's parents is in the carryFlags() call from markUninteresting(). carryFlags() only traverses commits that have already been parsed: commits yet to be parsed are supposed to inherit correct flags from their parent in PendingGenerator#next (which doesn't happen here --- that is (2)). So the list of commits that have already been parsed becomes relevant. When we hit the markUninteresting() call, all "want"s, "have"s, and commits to be unshallowed have been parsed. carryFlags() only affects the parsed commits. If the "want" is a direct parent of a "have", then it carryFlags() marks it as uninteresting. If the "have" was also a "shallow", then its parent pointer should have been null and the "want" shouldn't have been marked, so we see the bug. If the "want" is a more distant ancestor then (2) keeps the uninteresting state from propagating to the "want" and we don't see the bug. If the "shallow" is not also a "have" then the shallow commit isn't parsed so (2) keeps the uninteresting state from propagating to the "want so we don't see the bug. Here is a reproduction case (time flowing left to right, arrows pointing to parents). "C" must be a commit that the client reports as a "have" during negotiation. That can only happen if the server reports it as an existing branch or tag in the first round of negotiation: A <-- B <-- C <-- D First do git clone --depth 1 <repo> which yields D as a "have" and C as a "shallow" commit. Then try git fetch --depth 1 <repo> B:refs/heads/B Negotiation sets up: have D, shallow C, have C, want B. But due to this bug B is marked as uninteresting and is not sent. Change-Id: I6e14b57b2f85e52d28cdcf356df647870f475440 Signed-off-by: Terry Parker <tparker@google.com>
7 years ago
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251
  1. /*
  2. * Copyright (C) 2010, Garmin International
  3. * Copyright (C) 2010, Matt Fischer <matt.fischer@garmin.com>
  4. * and other copyright owners as documented in the project's IP log.
  5. *
  6. * This program and the accompanying materials are made available
  7. * under the terms of the Eclipse Distribution License v1.0 which
  8. * accompanies this distribution, is reproduced below, and is
  9. * available at http://www.eclipse.org/org/documents/edl-v10.php
  10. *
  11. * All rights reserved.
  12. *
  13. * Redistribution and use in source and binary forms, with or
  14. * without modification, are permitted provided that the following
  15. * conditions are met:
  16. *
  17. * - Redistributions of source code must retain the above copyright
  18. * notice, this list of conditions and the following disclaimer.
  19. *
  20. * - Redistributions in binary form must reproduce the above
  21. * copyright notice, this list of conditions and the following
  22. * disclaimer in the documentation and/or other materials provided
  23. * with the distribution.
  24. *
  25. * - Neither the name of the Eclipse Foundation, Inc. nor the
  26. * names of its contributors may be used to endorse or promote
  27. * products derived from this software without specific prior
  28. * written permission.
  29. *
  30. * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
  31. * CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
  32. * INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
  33. * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
  34. * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
  35. * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
  36. * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
  37. * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
  38. * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
  39. * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
  40. * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
  41. * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
  42. * ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  43. */
  44. package org.eclipse.jgit.revwalk;
  45. import java.io.IOException;
  46. import org.eclipse.jgit.errors.IncorrectObjectTypeException;
  47. import org.eclipse.jgit.errors.MissingObjectException;
  48. import org.eclipse.jgit.lib.AnyObjectId;
  49. import org.eclipse.jgit.lib.ObjectReader;
  50. import org.eclipse.jgit.lib.Repository;
  51. /** Interface for revision walkers that perform depth filtering. */
  52. public interface DepthWalk {
  53. /** @return Depth to filter to. */
  54. public int getDepth();
  55. /** @return flag marking commits that should become unshallow. */
  56. public RevFlag getUnshallowFlag();
  57. /** @return flag marking commits that are interesting again. */
  58. public RevFlag getReinterestingFlag();
  59. /** RevCommit with a depth (in commits) from a root. */
  60. public static class Commit extends RevCommit {
  61. /** Depth of this commit in the graph, via shortest path. */
  62. int depth;
  63. /** @return depth of this commit, as found by the shortest path. */
  64. public int getDepth() {
  65. return depth;
  66. }
  67. /**
  68. * Initialize a new commit.
  69. *
  70. * @param id
  71. * object name for the commit.
  72. */
  73. protected Commit(AnyObjectId id) {
  74. super(id);
  75. depth = -1;
  76. }
  77. }
  78. /** Subclass of RevWalk that performs depth filtering. */
  79. public class RevWalk extends org.eclipse.jgit.revwalk.RevWalk implements DepthWalk {
  80. private final int depth;
  81. private final RevFlag UNSHALLOW;
  82. private final RevFlag REINTERESTING;
  83. /**
  84. * @param repo Repository to walk
  85. * @param depth Maximum depth to return
  86. */
  87. public RevWalk(Repository repo, int depth) {
  88. super(repo);
  89. this.depth = depth;
  90. this.UNSHALLOW = newFlag("UNSHALLOW"); //$NON-NLS-1$
  91. this.REINTERESTING = newFlag("REINTERESTING"); //$NON-NLS-1$
  92. }
  93. /**
  94. * @param or ObjectReader to use
  95. * @param depth Maximum depth to return
  96. */
  97. public RevWalk(ObjectReader or, int depth) {
  98. super(or);
  99. this.depth = depth;
  100. this.UNSHALLOW = newFlag("UNSHALLOW"); //$NON-NLS-1$
  101. this.REINTERESTING = newFlag("REINTERESTING"); //$NON-NLS-1$
  102. }
  103. /**
  104. * Mark a root commit (i.e., one whose depth should be considered 0.)
  105. *
  106. * @param c
  107. * Commit to mark
  108. * @throws IOException
  109. * @throws IncorrectObjectTypeException
  110. * @throws MissingObjectException
  111. */
  112. public void markRoot(RevCommit c) throws MissingObjectException,
  113. IncorrectObjectTypeException, IOException {
  114. if (c instanceof Commit)
  115. ((Commit) c).depth = 0;
  116. super.markStart(c);
  117. }
  118. @Override
  119. protected RevCommit createCommit(AnyObjectId id) {
  120. return new Commit(id);
  121. }
  122. public int getDepth() {
  123. return depth;
  124. }
  125. public RevFlag getUnshallowFlag() {
  126. return UNSHALLOW;
  127. }
  128. public RevFlag getReinterestingFlag() {
  129. return REINTERESTING;
  130. }
  131. @Override
  132. public ObjectWalk toObjectWalkWithSameObjects() {
  133. ObjectWalk ow = new ObjectWalk(reader, depth);
  134. ow.objects = objects;
  135. ow.freeFlags = freeFlags;
  136. return ow;
  137. }
  138. }
  139. /** Subclass of ObjectWalk that performs depth filtering. */
  140. public class ObjectWalk extends org.eclipse.jgit.revwalk.ObjectWalk implements DepthWalk {
  141. private final int depth;
  142. private final RevFlag UNSHALLOW;
  143. private final RevFlag REINTERESTING;
  144. /**
  145. * @param repo Repository to walk
  146. * @param depth Maximum depth to return
  147. */
  148. public ObjectWalk(Repository repo, int depth) {
  149. super(repo);
  150. this.depth = depth;
  151. this.UNSHALLOW = newFlag("UNSHALLOW"); //$NON-NLS-1$
  152. this.REINTERESTING = newFlag("REINTERESTING"); //$NON-NLS-1$
  153. }
  154. /**
  155. * @param or Object Reader
  156. * @param depth Maximum depth to return
  157. */
  158. public ObjectWalk(ObjectReader or, int depth) {
  159. super(or);
  160. this.depth = depth;
  161. this.UNSHALLOW = newFlag("UNSHALLOW"); //$NON-NLS-1$
  162. this.REINTERESTING = newFlag("REINTERESTING"); //$NON-NLS-1$
  163. }
  164. /**
  165. * Mark a root commit (i.e., one whose depth should be considered 0.)
  166. *
  167. * @param o
  168. * Commit to mark
  169. * @throws IOException
  170. * @throws IncorrectObjectTypeException
  171. * @throws MissingObjectException
  172. */
  173. public void markRoot(RevObject o) throws MissingObjectException,
  174. IncorrectObjectTypeException, IOException {
  175. RevObject c = o;
  176. while (c instanceof RevTag) {
  177. c = ((RevTag) c).getObject();
  178. parseHeaders(c);
  179. }
  180. if (c instanceof Commit)
  181. ((Commit) c).depth = 0;
  182. super.markStart(o);
  183. }
  184. /**
  185. * Mark an element which used to be shallow in the client, but which
  186. * should now be considered a full commit. Any ancestors of this commit
  187. * should be included in the walk, even if they are the ancestor of an
  188. * uninteresting commit.
  189. *
  190. * @param c
  191. * Commit to mark
  192. * @throws MissingObjectException
  193. * @throws IncorrectObjectTypeException
  194. * @throws IOException
  195. */
  196. public void markUnshallow(RevObject c) throws MissingObjectException,
  197. IncorrectObjectTypeException, IOException {
  198. if (c instanceof RevCommit)
  199. c.add(UNSHALLOW);
  200. super.markStart(c);
  201. }
  202. @Override
  203. protected RevCommit createCommit(AnyObjectId id) {
  204. return new Commit(id);
  205. }
  206. public int getDepth() {
  207. return depth;
  208. }
  209. public RevFlag getUnshallowFlag() {
  210. return UNSHALLOW;
  211. }
  212. public RevFlag getReinterestingFlag() {
  213. return REINTERESTING;
  214. }
  215. }
  216. }