You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

DumbClientSmartServerTest.java 7.3KB

Client-side protocol V2 support for fetching Make all transports request protocol V2 when fetching. Depending on the transport, set the GIT_PROTOCOL environment variable (file and ssh), pass the Git-Protocol header (http), or set the hidden "\0version=2\0" (git anon). We'll fall back to V0 if the server doesn't reply with a version 2 answer. A user can control which protocol the client requests via the git config protocol.version; if not set, JGit requests protocol V2 for fetching. Pushing always uses protocol V0 still. In the API, there is only a new Transport.openFetch() version that takes a collection of RefSpecs plus additional patterns to construct the Ref prefixes for the "ls-refs" command in protocol V2. If none are given, the server will still advertise all refs, even in protocol V2. BasePackConnection.readAdvertisedRefs() handles falling back to protocol V0. It newly returns true if V0 was used and the advertised refs were read, and false if V2 is used and an explicit "ls-refs" is needed. (This can't be done transparently inside readAdvertisedRefs() because a "stateless RPC" transport like TransportHttp may need to open a new connection for writing.) BasePackFetchConnection implements the changes needed for the protocol V2 "fetch" command (stateless protocol, simplified ACK handling, delimiters, section headers). In TransportHttp, change readSmartHeaders() to also recognize the "version 2" packet line as a valid smart server indication. Adapt tests, and run all the HTTP tests not only with both HTTP connection factories (JDK and Apache HttpClient) but also with both protocol V0 and V2. The SSH tests are much slower and much more focused on the SSH protocol and SSH key handling. Factor out two very simple cloning and pulling tests and make those run with protocol V2. Bug: 553083 Change-Id: I357c7f5daa7efb2872f1c64ee6f6d54229031ae1 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
3 years ago
Client-side protocol V2 support for fetching Make all transports request protocol V2 when fetching. Depending on the transport, set the GIT_PROTOCOL environment variable (file and ssh), pass the Git-Protocol header (http), or set the hidden "\0version=2\0" (git anon). We'll fall back to V0 if the server doesn't reply with a version 2 answer. A user can control which protocol the client requests via the git config protocol.version; if not set, JGit requests protocol V2 for fetching. Pushing always uses protocol V0 still. In the API, there is only a new Transport.openFetch() version that takes a collection of RefSpecs plus additional patterns to construct the Ref prefixes for the "ls-refs" command in protocol V2. If none are given, the server will still advertise all refs, even in protocol V2. BasePackConnection.readAdvertisedRefs() handles falling back to protocol V0. It newly returns true if V0 was used and the advertised refs were read, and false if V2 is used and an explicit "ls-refs" is needed. (This can't be done transparently inside readAdvertisedRefs() because a "stateless RPC" transport like TransportHttp may need to open a new connection for writing.) BasePackFetchConnection implements the changes needed for the protocol V2 "fetch" command (stateless protocol, simplified ACK handling, delimiters, section headers). In TransportHttp, change readSmartHeaders() to also recognize the "version 2" packet line as a valid smart server indication. Adapt tests, and run all the HTTP tests not only with both HTTP connection factories (JDK and Apache HttpClient) but also with both protocol V0 and V2. The SSH tests are much slower and much more focused on the SSH protocol and SSH key handling. Factor out two very simple cloning and pulling tests and make those run with protocol V2. Bug: 553083 Change-Id: I357c7f5daa7efb2872f1c64ee6f6d54229031ae1 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
3 years ago
Client-side protocol V2 support for fetching Make all transports request protocol V2 when fetching. Depending on the transport, set the GIT_PROTOCOL environment variable (file and ssh), pass the Git-Protocol header (http), or set the hidden "\0version=2\0" (git anon). We'll fall back to V0 if the server doesn't reply with a version 2 answer. A user can control which protocol the client requests via the git config protocol.version; if not set, JGit requests protocol V2 for fetching. Pushing always uses protocol V0 still. In the API, there is only a new Transport.openFetch() version that takes a collection of RefSpecs plus additional patterns to construct the Ref prefixes for the "ls-refs" command in protocol V2. If none are given, the server will still advertise all refs, even in protocol V2. BasePackConnection.readAdvertisedRefs() handles falling back to protocol V0. It newly returns true if V0 was used and the advertised refs were read, and false if V2 is used and an explicit "ls-refs" is needed. (This can't be done transparently inside readAdvertisedRefs() because a "stateless RPC" transport like TransportHttp may need to open a new connection for writing.) BasePackFetchConnection implements the changes needed for the protocol V2 "fetch" command (stateless protocol, simplified ACK handling, delimiters, section headers). In TransportHttp, change readSmartHeaders() to also recognize the "version 2" packet line as a valid smart server indication. Adapt tests, and run all the HTTP tests not only with both HTTP connection factories (JDK and Apache HttpClient) but also with both protocol V0 and V2. The SSH tests are much slower and much more focused on the SSH protocol and SSH key handling. Factor out two very simple cloning and pulling tests and make those run with protocol V2. Bug: 553083 Change-Id: I357c7f5daa7efb2872f1c64ee6f6d54229031ae1 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
3 years ago
Add support to follow HTTP redirects git-core follows HTTP redirects so JGit should also provide this. Implement config setting http.followRedirects with possible values "false" (= never), "true" (= always), and "initial" (only on GET, but not on POST).[1] We must do our own redirect handling and cannot rely on the support that the underlying real connection may offer. At least the JDK's HttpURLConnection has two features that get in the way: * it does not allow cross-protocol redirects and thus fails on http->https redirects (for instance, on Github). * it translates a redirect after a POST to a GET unless the system property "http.strictPostRedirect" is set to true. We don't want to manipulate that system setting nor require it. Additionally, git has its own rules about what redirects it accepts;[2] for instance, it does not allow a redirect that adds query arguments. We handle response codes 301, 302, 303, and 307 as per RFC 2616.[3] On POST we do not handle 303, and we follow redirects only if http.followRedirects == true. Redirects are followed only a certain number of times. There are two ways to control that limit: * by default, the limit is given by the http.maxRedirects system property that is also used by the JDK. If the system property is not set, the default is 5. (This is much lower than the JDK default of 20, but I don't see the value of following so many redirects.) * this can be overwritten by a http.maxRedirects git config setting. The JGit http.* git config settings are currently all global; JGit has no support yet for URI-specific settings "http.<pattern>.name". Adding support for that is well beyond the scope of this change. Like git-core, we log every redirect attempt (LOG.info) so that users may know about the redirection having occurred. Extends the test framework to configure an AppServer with HTTPS support so that we can test cloning via HTTPS and redirections involving HTTPS. [1] https://git-scm.com/docs/git-config [2] https://kernel.googlesource.com/pub/scm/git/git/+/6628eb41db5189c0cdfdced6d8697e7c813c5f0f [3] https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html CQ: 13987 Bug: 465167 Change-Id: I86518cb76842f7d326b51f8715e3bbf8ada89859 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
9 years ago
Client-side protocol V2 support for fetching Make all transports request protocol V2 when fetching. Depending on the transport, set the GIT_PROTOCOL environment variable (file and ssh), pass the Git-Protocol header (http), or set the hidden "\0version=2\0" (git anon). We'll fall back to V0 if the server doesn't reply with a version 2 answer. A user can control which protocol the client requests via the git config protocol.version; if not set, JGit requests protocol V2 for fetching. Pushing always uses protocol V0 still. In the API, there is only a new Transport.openFetch() version that takes a collection of RefSpecs plus additional patterns to construct the Ref prefixes for the "ls-refs" command in protocol V2. If none are given, the server will still advertise all refs, even in protocol V2. BasePackConnection.readAdvertisedRefs() handles falling back to protocol V0. It newly returns true if V0 was used and the advertised refs were read, and false if V2 is used and an explicit "ls-refs" is needed. (This can't be done transparently inside readAdvertisedRefs() because a "stateless RPC" transport like TransportHttp may need to open a new connection for writing.) BasePackFetchConnection implements the changes needed for the protocol V2 "fetch" command (stateless protocol, simplified ACK handling, delimiters, section headers). In TransportHttp, change readSmartHeaders() to also recognize the "version 2" packet line as a valid smart server indication. Adapt tests, and run all the HTTP tests not only with both HTTP connection factories (JDK and Apache HttpClient) but also with both protocol V0 and V2. The SSH tests are much slower and much more focused on the SSH protocol and SSH key handling. Factor out two very simple cloning and pulling tests and make those run with protocol V2. Bug: 553083 Change-Id: I357c7f5daa7efb2872f1c64ee6f6d54229031ae1 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
3 years ago
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223
  1. /*
  2. * Copyright (C) 2010, 2017 Google Inc. and others
  3. *
  4. * This program and the accompanying materials are made available under the
  5. * terms of the Eclipse Distribution License v. 1.0 which is available at
  6. * https://www.eclipse.org/org/documents/edl-v10.php.
  7. *
  8. * SPDX-License-Identifier: BSD-3-Clause
  9. */
  10. package org.eclipse.jgit.http.test;
  11. import static org.eclipse.jgit.util.HttpSupport.HDR_ACCEPT;
  12. import static org.eclipse.jgit.util.HttpSupport.HDR_CONTENT_TYPE;
  13. import static org.eclipse.jgit.util.HttpSupport.HDR_PRAGMA;
  14. import static org.eclipse.jgit.util.HttpSupport.HDR_USER_AGENT;
  15. import static org.junit.Assert.assertEquals;
  16. import static org.junit.Assert.assertFalse;
  17. import static org.junit.Assert.assertNotNull;
  18. import static org.junit.Assert.assertNull;
  19. import static org.junit.Assert.assertTrue;
  20. import static org.junit.Assert.fail;
  21. import java.io.IOException;
  22. import java.util.List;
  23. import java.util.Map;
  24. import org.eclipse.jetty.servlet.ServletContextHandler;
  25. import org.eclipse.jetty.servlet.ServletHolder;
  26. import org.eclipse.jgit.errors.NotSupportedException;
  27. import org.eclipse.jgit.http.server.GitServlet;
  28. import org.eclipse.jgit.junit.TestRepository;
  29. import org.eclipse.jgit.junit.http.AccessEvent;
  30. import org.eclipse.jgit.lib.Constants;
  31. import org.eclipse.jgit.lib.NullProgressMonitor;
  32. import org.eclipse.jgit.lib.Ref;
  33. import org.eclipse.jgit.lib.Repository;
  34. import org.eclipse.jgit.lib.StoredConfig;
  35. import org.eclipse.jgit.revwalk.RevBlob;
  36. import org.eclipse.jgit.revwalk.RevCommit;
  37. import org.eclipse.jgit.transport.FetchConnection;
  38. import org.eclipse.jgit.transport.HttpTransport;
  39. import org.eclipse.jgit.transport.Transport;
  40. import org.eclipse.jgit.transport.TransportHttp;
  41. import org.eclipse.jgit.transport.URIish;
  42. import org.junit.Before;
  43. import org.junit.Test;
  44. public class DumbClientSmartServerTest extends AllProtocolsHttpTestCase {
  45. private Repository remoteRepository;
  46. private URIish remoteURI;
  47. private RevBlob A_txt;
  48. private RevCommit A, B;
  49. public DumbClientSmartServerTest(TestParameters params) {
  50. super(params);
  51. }
  52. @Override
  53. @Before
  54. public void setUp() throws Exception {
  55. super.setUp();
  56. final TestRepository<Repository> src = createTestRepository();
  57. final String srcName = src.getRepository().getDirectory().getName();
  58. ServletContextHandler app = server.addContext("/git");
  59. GitServlet gs = new GitServlet();
  60. gs.setRepositoryResolver(new TestRepositoryResolver(src, srcName));
  61. app.addServlet(new ServletHolder(gs), "/*");
  62. server.setUp();
  63. remoteRepository = src.getRepository();
  64. remoteURI = toURIish(app, srcName);
  65. StoredConfig cfg = remoteRepository.getConfig();
  66. cfg.setInt("protocol", null, "version", enableProtocolV2 ? 2 : 0);
  67. cfg.save();
  68. A_txt = src.blob("A");
  69. A = src.commit().add("A_txt", A_txt).create();
  70. B = src.commit().parent(A).add("A_txt", "C").add("B", "B").create();
  71. src.update(master, B);
  72. }
  73. @Test
  74. public void testListRemote() throws IOException {
  75. Repository dst = createBareRepository();
  76. assertEquals("http", remoteURI.getScheme());
  77. Map<String, Ref> map;
  78. try (Transport t = Transport.open(dst, remoteURI)) {
  79. ((TransportHttp) t).setUseSmartHttp(false);
  80. // I didn't make up these public interface names, I just
  81. // approved them for inclusion into the code base. Sorry.
  82. // --spearce
  83. //
  84. assertTrue("isa TransportHttp", t instanceof TransportHttp);
  85. assertTrue("isa HttpTransport", t instanceof HttpTransport);
  86. try (FetchConnection c = t.openFetch()) {
  87. map = c.getRefsMap();
  88. }
  89. }
  90. assertNotNull("have map of refs", map);
  91. assertEquals(2, map.size());
  92. assertNotNull("has " + master, map.get(master));
  93. assertEquals(B, map.get(master).getObjectId());
  94. assertNotNull("has " + Constants.HEAD, map.get(Constants.HEAD));
  95. assertEquals(B, map.get(Constants.HEAD).getObjectId());
  96. List<AccessEvent> requests = getRequests();
  97. assertEquals(2, requests.size());
  98. assertEquals(0, getRequests(remoteURI, "git-upload-pack").size());
  99. AccessEvent info = requests.get(0);
  100. assertEquals("GET", info.getMethod());
  101. assertEquals(join(remoteURI, "info/refs"), info.getPath());
  102. assertEquals(0, info.getParameters().size());
  103. assertNull("no service parameter", info.getParameter("service"));
  104. assertEquals("no-cache", info.getRequestHeader(HDR_PRAGMA));
  105. assertNotNull("has user-agent", info.getRequestHeader(HDR_USER_AGENT));
  106. assertTrue("is jgit agent", info.getRequestHeader(HDR_USER_AGENT)
  107. .startsWith("JGit/"));
  108. assertEquals("*/*", info.getRequestHeader(HDR_ACCEPT));
  109. assertEquals(200, info.getStatus());
  110. assertEquals("text/plain;charset=utf-8",
  111. info
  112. .getResponseHeader(HDR_CONTENT_TYPE));
  113. AccessEvent head = requests.get(1);
  114. assertEquals("GET", head.getMethod());
  115. assertEquals(join(remoteURI, "HEAD"), head.getPath());
  116. assertEquals(0, head.getParameters().size());
  117. assertEquals(200, head.getStatus());
  118. assertEquals("text/plain", head.getResponseHeader(HDR_CONTENT_TYPE));
  119. }
  120. @Test
  121. public void testInitialClone_Small() throws Exception {
  122. Repository dst = createBareRepository();
  123. assertFalse(dst.getObjectDatabase().has(A_txt));
  124. try (Transport t = Transport.open(dst, remoteURI)) {
  125. ((TransportHttp) t).setUseSmartHttp(false);
  126. t.fetch(NullProgressMonitor.INSTANCE, mirror(master));
  127. }
  128. assertTrue(dst.getObjectDatabase().has(A_txt));
  129. assertEquals(B, dst.exactRef(master).getObjectId());
  130. fsck(dst, B);
  131. List<AccessEvent> loose = getRequests(loose(remoteURI, A_txt));
  132. assertEquals(1, loose.size());
  133. assertEquals("GET", loose.get(0).getMethod());
  134. assertEquals(0, loose.get(0).getParameters().size());
  135. assertEquals(200, loose.get(0).getStatus());
  136. assertEquals("application/x-git-loose-object", loose.get(0)
  137. .getResponseHeader(HDR_CONTENT_TYPE));
  138. }
  139. @Test
  140. public void testInitialClone_Packed() throws Exception {
  141. try (TestRepository<Repository> tr = new TestRepository<>(
  142. remoteRepository)) {
  143. tr.packAndPrune();
  144. }
  145. Repository dst = createBareRepository();
  146. assertFalse(dst.getObjectDatabase().has(A_txt));
  147. try (Transport t = Transport.open(dst, remoteURI)) {
  148. ((TransportHttp) t).setUseSmartHttp(false);
  149. t.fetch(NullProgressMonitor.INSTANCE, mirror(master));
  150. }
  151. assertTrue(dst.getObjectDatabase().has(A_txt));
  152. assertEquals(B, dst.exactRef(master).getObjectId());
  153. fsck(dst, B);
  154. List<AccessEvent> req;
  155. req = getRequests(loose(remoteURI, B));
  156. assertEquals(1, req.size());
  157. assertEquals("GET", req.get(0).getMethod());
  158. assertEquals(0, req.get(0).getParameters().size());
  159. assertEquals(404, req.get(0).getStatus());
  160. req = getRequests(join(remoteURI, "objects/info/packs"));
  161. assertEquals(1, req.size());
  162. assertEquals("GET", req.get(0).getMethod());
  163. assertEquals(0, req.get(0).getParameters().size());
  164. assertEquals(200, req.get(0).getStatus());
  165. assertEquals("text/plain;charset=utf-8",
  166. req.get(0).getResponseHeader(
  167. HDR_CONTENT_TYPE));
  168. }
  169. @Test
  170. public void testPushNotSupported() throws Exception {
  171. final TestRepository src = createTestRepository();
  172. final RevCommit Q = src.commit().create();
  173. final Repository db = src.getRepository();
  174. try (Transport t = Transport.open(db, remoteURI)) {
  175. ((TransportHttp) t).setUseSmartHttp(false);
  176. try {
  177. t.push(NullProgressMonitor.INSTANCE, push(src, Q));
  178. fail("push incorrectly completed against a smart server");
  179. } catch (NotSupportedException nse) {
  180. String exp = "smart HTTP push disabled";
  181. assertEquals(exp, nse.getMessage());
  182. }
  183. }
  184. }
  185. }