You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

AppServer.java 12KB

Add support to follow HTTP redirects git-core follows HTTP redirects so JGit should also provide this. Implement config setting http.followRedirects with possible values "false" (= never), "true" (= always), and "initial" (only on GET, but not on POST).[1] We must do our own redirect handling and cannot rely on the support that the underlying real connection may offer. At least the JDK's HttpURLConnection has two features that get in the way: * it does not allow cross-protocol redirects and thus fails on http->https redirects (for instance, on Github). * it translates a redirect after a POST to a GET unless the system property "http.strictPostRedirect" is set to true. We don't want to manipulate that system setting nor require it. Additionally, git has its own rules about what redirects it accepts;[2] for instance, it does not allow a redirect that adds query arguments. We handle response codes 301, 302, 303, and 307 as per RFC 2616.[3] On POST we do not handle 303, and we follow redirects only if http.followRedirects == true. Redirects are followed only a certain number of times. There are two ways to control that limit: * by default, the limit is given by the http.maxRedirects system property that is also used by the JDK. If the system property is not set, the default is 5. (This is much lower than the JDK default of 20, but I don't see the value of following so many redirects.) * this can be overwritten by a http.maxRedirects git config setting. The JGit http.* git config settings are currently all global; JGit has no support yet for URI-specific settings "http.<pattern>.name". Adding support for that is well beyond the scope of this change. Like git-core, we log every redirect attempt (LOG.info) so that users may know about the redirection having occurred. Extends the test framework to configure an AppServer with HTTPS support so that we can test cloning via HTTPS and redirections involving HTTPS. [1] https://git-scm.com/docs/git-config [2] https://kernel.googlesource.com/pub/scm/git/git/+/6628eb41db5189c0cdfdced6d8697e7c813c5f0f [3] https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html CQ: 13987 Bug: 465167 Change-Id: I86518cb76842f7d326b51f8715e3bbf8ada89859 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
9 years ago
Add support to follow HTTP redirects git-core follows HTTP redirects so JGit should also provide this. Implement config setting http.followRedirects with possible values "false" (= never), "true" (= always), and "initial" (only on GET, but not on POST).[1] We must do our own redirect handling and cannot rely on the support that the underlying real connection may offer. At least the JDK's HttpURLConnection has two features that get in the way: * it does not allow cross-protocol redirects and thus fails on http->https redirects (for instance, on Github). * it translates a redirect after a POST to a GET unless the system property "http.strictPostRedirect" is set to true. We don't want to manipulate that system setting nor require it. Additionally, git has its own rules about what redirects it accepts;[2] for instance, it does not allow a redirect that adds query arguments. We handle response codes 301, 302, 303, and 307 as per RFC 2616.[3] On POST we do not handle 303, and we follow redirects only if http.followRedirects == true. Redirects are followed only a certain number of times. There are two ways to control that limit: * by default, the limit is given by the http.maxRedirects system property that is also used by the JDK. If the system property is not set, the default is 5. (This is much lower than the JDK default of 20, but I don't see the value of following so many redirects.) * this can be overwritten by a http.maxRedirects git config setting. The JGit http.* git config settings are currently all global; JGit has no support yet for URI-specific settings "http.<pattern>.name". Adding support for that is well beyond the scope of this change. Like git-core, we log every redirect attempt (LOG.info) so that users may know about the redirection having occurred. Extends the test framework to configure an AppServer with HTTPS support so that we can test cloning via HTTPS and redirections involving HTTPS. [1] https://git-scm.com/docs/git-config [2] https://kernel.googlesource.com/pub/scm/git/git/+/6628eb41db5189c0cdfdced6d8697e7c813c5f0f [3] https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html CQ: 13987 Bug: 465167 Change-Id: I86518cb76842f7d326b51f8715e3bbf8ada89859 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
9 years ago
Do authentication re-tries on HTTP POST There is at least one git server out there (GOGS) that does not require authentication on the initial GET for info/refs?service=git-receive-pack but that _does_ require authentication for the subsequent POST to actually do the push. This occurs on GOGS with public repositories; for private repositories it wants authentication up front. Handle this behavior by adding 401 handling to our POST request. Note that this is suboptimal; we'll re-send the push data at least twice if an authentication failure on POST occurs. It would be much better if the server required authentication up-front in the GET request. Added authentication unit tests (using BASIC auth) to the SmartClientSmartServerTest: - clone with authentication - clone with authentication but lacking CredentialsProvider - clone with authentication and wrong password - clone with authentication after redirect - clone with authentication only on POST, but not on GET Also tested manually in the wild using repositories at try.gogs.io. That server offers only BASIC auth, so the other paths (DIGEST, NEGOTIATE, fall back from DIGEST to BASIC) are untested and I have no way to test them. * public repository: GET unauthenticated, POST authenticated Also tested after clearing the credentials and then entering a wrong password: correctly asks three times during the HTTP POST for user name and password, then gives up. * private repository: authentication already on GET; then gets applied correctly initially to the POST request, which succeeds. Also fix the authentication to use the credentials for the redirected URI if redirects had occurred. We must not present the credentials for the original URI in that case. Consider a malicious redirect A->B: this would allow server B to harvest the user credentials for server A. The unit test for authentication after a redirect also tests for this. Bug: 513043 Change-Id: I97ee5058569efa1545a6c6f6edfd2b357c40592a Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years ago
Add support to follow HTTP redirects git-core follows HTTP redirects so JGit should also provide this. Implement config setting http.followRedirects with possible values "false" (= never), "true" (= always), and "initial" (only on GET, but not on POST).[1] We must do our own redirect handling and cannot rely on the support that the underlying real connection may offer. At least the JDK's HttpURLConnection has two features that get in the way: * it does not allow cross-protocol redirects and thus fails on http->https redirects (for instance, on Github). * it translates a redirect after a POST to a GET unless the system property "http.strictPostRedirect" is set to true. We don't want to manipulate that system setting nor require it. Additionally, git has its own rules about what redirects it accepts;[2] for instance, it does not allow a redirect that adds query arguments. We handle response codes 301, 302, 303, and 307 as per RFC 2616.[3] On POST we do not handle 303, and we follow redirects only if http.followRedirects == true. Redirects are followed only a certain number of times. There are two ways to control that limit: * by default, the limit is given by the http.maxRedirects system property that is also used by the JDK. If the system property is not set, the default is 5. (This is much lower than the JDK default of 20, but I don't see the value of following so many redirects.) * this can be overwritten by a http.maxRedirects git config setting. The JGit http.* git config settings are currently all global; JGit has no support yet for URI-specific settings "http.<pattern>.name". Adding support for that is well beyond the scope of this change. Like git-core, we log every redirect attempt (LOG.info) so that users may know about the redirection having occurred. Extends the test framework to configure an AppServer with HTTPS support so that we can test cloning via HTTPS and redirections involving HTTPS. [1] https://git-scm.com/docs/git-config [2] https://kernel.googlesource.com/pub/scm/git/git/+/6628eb41db5189c0cdfdced6d8697e7c813c5f0f [3] https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html CQ: 13987 Bug: 465167 Change-Id: I86518cb76842f7d326b51f8715e3bbf8ada89859 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
9 years ago
Add support to follow HTTP redirects git-core follows HTTP redirects so JGit should also provide this. Implement config setting http.followRedirects with possible values "false" (= never), "true" (= always), and "initial" (only on GET, but not on POST).[1] We must do our own redirect handling and cannot rely on the support that the underlying real connection may offer. At least the JDK's HttpURLConnection has two features that get in the way: * it does not allow cross-protocol redirects and thus fails on http->https redirects (for instance, on Github). * it translates a redirect after a POST to a GET unless the system property "http.strictPostRedirect" is set to true. We don't want to manipulate that system setting nor require it. Additionally, git has its own rules about what redirects it accepts;[2] for instance, it does not allow a redirect that adds query arguments. We handle response codes 301, 302, 303, and 307 as per RFC 2616.[3] On POST we do not handle 303, and we follow redirects only if http.followRedirects == true. Redirects are followed only a certain number of times. There are two ways to control that limit: * by default, the limit is given by the http.maxRedirects system property that is also used by the JDK. If the system property is not set, the default is 5. (This is much lower than the JDK default of 20, but I don't see the value of following so many redirects.) * this can be overwritten by a http.maxRedirects git config setting. The JGit http.* git config settings are currently all global; JGit has no support yet for URI-specific settings "http.<pattern>.name". Adding support for that is well beyond the scope of this change. Like git-core, we log every redirect attempt (LOG.info) so that users may know about the redirection having occurred. Extends the test framework to configure an AppServer with HTTPS support so that we can test cloning via HTTPS and redirections involving HTTPS. [1] https://git-scm.com/docs/git-config [2] https://kernel.googlesource.com/pub/scm/git/git/+/6628eb41db5189c0cdfdced6d8697e7c813c5f0f [3] https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html CQ: 13987 Bug: 465167 Change-Id: I86518cb76842f7d326b51f8715e3bbf8ada89859 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
9 years ago
Add support to follow HTTP redirects git-core follows HTTP redirects so JGit should also provide this. Implement config setting http.followRedirects with possible values "false" (= never), "true" (= always), and "initial" (only on GET, but not on POST).[1] We must do our own redirect handling and cannot rely on the support that the underlying real connection may offer. At least the JDK's HttpURLConnection has two features that get in the way: * it does not allow cross-protocol redirects and thus fails on http->https redirects (for instance, on Github). * it translates a redirect after a POST to a GET unless the system property "http.strictPostRedirect" is set to true. We don't want to manipulate that system setting nor require it. Additionally, git has its own rules about what redirects it accepts;[2] for instance, it does not allow a redirect that adds query arguments. We handle response codes 301, 302, 303, and 307 as per RFC 2616.[3] On POST we do not handle 303, and we follow redirects only if http.followRedirects == true. Redirects are followed only a certain number of times. There are two ways to control that limit: * by default, the limit is given by the http.maxRedirects system property that is also used by the JDK. If the system property is not set, the default is 5. (This is much lower than the JDK default of 20, but I don't see the value of following so many redirects.) * this can be overwritten by a http.maxRedirects git config setting. The JGit http.* git config settings are currently all global; JGit has no support yet for URI-specific settings "http.<pattern>.name". Adding support for that is well beyond the scope of this change. Like git-core, we log every redirect attempt (LOG.info) so that users may know about the redirection having occurred. Extends the test framework to configure an AppServer with HTTPS support so that we can test cloning via HTTPS and redirections involving HTTPS. [1] https://git-scm.com/docs/git-config [2] https://kernel.googlesource.com/pub/scm/git/git/+/6628eb41db5189c0cdfdced6d8697e7c813c5f0f [3] https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html CQ: 13987 Bug: 465167 Change-Id: I86518cb76842f7d326b51f8715e3bbf8ada89859 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
9 years ago
Add support to follow HTTP redirects git-core follows HTTP redirects so JGit should also provide this. Implement config setting http.followRedirects with possible values "false" (= never), "true" (= always), and "initial" (only on GET, but not on POST).[1] We must do our own redirect handling and cannot rely on the support that the underlying real connection may offer. At least the JDK's HttpURLConnection has two features that get in the way: * it does not allow cross-protocol redirects and thus fails on http->https redirects (for instance, on Github). * it translates a redirect after a POST to a GET unless the system property "http.strictPostRedirect" is set to true. We don't want to manipulate that system setting nor require it. Additionally, git has its own rules about what redirects it accepts;[2] for instance, it does not allow a redirect that adds query arguments. We handle response codes 301, 302, 303, and 307 as per RFC 2616.[3] On POST we do not handle 303, and we follow redirects only if http.followRedirects == true. Redirects are followed only a certain number of times. There are two ways to control that limit: * by default, the limit is given by the http.maxRedirects system property that is also used by the JDK. If the system property is not set, the default is 5. (This is much lower than the JDK default of 20, but I don't see the value of following so many redirects.) * this can be overwritten by a http.maxRedirects git config setting. The JGit http.* git config settings are currently all global; JGit has no support yet for URI-specific settings "http.<pattern>.name". Adding support for that is well beyond the scope of this change. Like git-core, we log every redirect attempt (LOG.info) so that users may know about the redirection having occurred. Extends the test framework to configure an AppServer with HTTPS support so that we can test cloning via HTTPS and redirections involving HTTPS. [1] https://git-scm.com/docs/git-config [2] https://kernel.googlesource.com/pub/scm/git/git/+/6628eb41db5189c0cdfdced6d8697e7c813c5f0f [3] https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html CQ: 13987 Bug: 465167 Change-Id: I86518cb76842f7d326b51f8715e3bbf8ada89859 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
9 years ago
Do authentication re-tries on HTTP POST There is at least one git server out there (GOGS) that does not require authentication on the initial GET for info/refs?service=git-receive-pack but that _does_ require authentication for the subsequent POST to actually do the push. This occurs on GOGS with public repositories; for private repositories it wants authentication up front. Handle this behavior by adding 401 handling to our POST request. Note that this is suboptimal; we'll re-send the push data at least twice if an authentication failure on POST occurs. It would be much better if the server required authentication up-front in the GET request. Added authentication unit tests (using BASIC auth) to the SmartClientSmartServerTest: - clone with authentication - clone with authentication but lacking CredentialsProvider - clone with authentication and wrong password - clone with authentication after redirect - clone with authentication only on POST, but not on GET Also tested manually in the wild using repositories at try.gogs.io. That server offers only BASIC auth, so the other paths (DIGEST, NEGOTIATE, fall back from DIGEST to BASIC) are untested and I have no way to test them. * public repository: GET unauthenticated, POST authenticated Also tested after clearing the credentials and then entering a wrong password: correctly asks three times during the HTTP POST for user name and password, then gives up. * private repository: authentication already on GET; then gets applied correctly initially to the POST request, which succeeds. Also fix the authentication to use the credentials for the redirected URI if redirects had occurred. We must not present the credentials for the original URI in that case. Consider a malicious redirect A->B: this would allow server B to harvest the user credentials for server A. The unit test for authentication after a redirect also tests for this. Bug: 513043 Change-Id: I97ee5058569efa1545a6c6f6edfd2b357c40592a Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years ago
Add support to follow HTTP redirects git-core follows HTTP redirects so JGit should also provide this. Implement config setting http.followRedirects with possible values "false" (= never), "true" (= always), and "initial" (only on GET, but not on POST).[1] We must do our own redirect handling and cannot rely on the support that the underlying real connection may offer. At least the JDK's HttpURLConnection has two features that get in the way: * it does not allow cross-protocol redirects and thus fails on http->https redirects (for instance, on Github). * it translates a redirect after a POST to a GET unless the system property "http.strictPostRedirect" is set to true. We don't want to manipulate that system setting nor require it. Additionally, git has its own rules about what redirects it accepts;[2] for instance, it does not allow a redirect that adds query arguments. We handle response codes 301, 302, 303, and 307 as per RFC 2616.[3] On POST we do not handle 303, and we follow redirects only if http.followRedirects == true. Redirects are followed only a certain number of times. There are two ways to control that limit: * by default, the limit is given by the http.maxRedirects system property that is also used by the JDK. If the system property is not set, the default is 5. (This is much lower than the JDK default of 20, but I don't see the value of following so many redirects.) * this can be overwritten by a http.maxRedirects git config setting. The JGit http.* git config settings are currently all global; JGit has no support yet for URI-specific settings "http.<pattern>.name". Adding support for that is well beyond the scope of this change. Like git-core, we log every redirect attempt (LOG.info) so that users may know about the redirection having occurred. Extends the test framework to configure an AppServer with HTTPS support so that we can test cloning via HTTPS and redirections involving HTTPS. [1] https://git-scm.com/docs/git-config [2] https://kernel.googlesource.com/pub/scm/git/git/+/6628eb41db5189c0cdfdced6d8697e7c813c5f0f [3] https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html CQ: 13987 Bug: 465167 Change-Id: I86518cb76842f7d326b51f8715e3bbf8ada89859 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
9 years ago
Add support to follow HTTP redirects git-core follows HTTP redirects so JGit should also provide this. Implement config setting http.followRedirects with possible values "false" (= never), "true" (= always), and "initial" (only on GET, but not on POST).[1] We must do our own redirect handling and cannot rely on the support that the underlying real connection may offer. At least the JDK's HttpURLConnection has two features that get in the way: * it does not allow cross-protocol redirects and thus fails on http->https redirects (for instance, on Github). * it translates a redirect after a POST to a GET unless the system property "http.strictPostRedirect" is set to true. We don't want to manipulate that system setting nor require it. Additionally, git has its own rules about what redirects it accepts;[2] for instance, it does not allow a redirect that adds query arguments. We handle response codes 301, 302, 303, and 307 as per RFC 2616.[3] On POST we do not handle 303, and we follow redirects only if http.followRedirects == true. Redirects are followed only a certain number of times. There are two ways to control that limit: * by default, the limit is given by the http.maxRedirects system property that is also used by the JDK. If the system property is not set, the default is 5. (This is much lower than the JDK default of 20, but I don't see the value of following so many redirects.) * this can be overwritten by a http.maxRedirects git config setting. The JGit http.* git config settings are currently all global; JGit has no support yet for URI-specific settings "http.<pattern>.name". Adding support for that is well beyond the scope of this change. Like git-core, we log every redirect attempt (LOG.info) so that users may know about the redirection having occurred. Extends the test framework to configure an AppServer with HTTPS support so that we can test cloning via HTTPS and redirections involving HTTPS. [1] https://git-scm.com/docs/git-config [2] https://kernel.googlesource.com/pub/scm/git/git/+/6628eb41db5189c0cdfdced6d8697e7c813c5f0f [3] https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html CQ: 13987 Bug: 465167 Change-Id: I86518cb76842f7d326b51f8715e3bbf8ada89859 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
9 years ago
Add support to follow HTTP redirects git-core follows HTTP redirects so JGit should also provide this. Implement config setting http.followRedirects with possible values "false" (= never), "true" (= always), and "initial" (only on GET, but not on POST).[1] We must do our own redirect handling and cannot rely on the support that the underlying real connection may offer. At least the JDK's HttpURLConnection has two features that get in the way: * it does not allow cross-protocol redirects and thus fails on http->https redirects (for instance, on Github). * it translates a redirect after a POST to a GET unless the system property "http.strictPostRedirect" is set to true. We don't want to manipulate that system setting nor require it. Additionally, git has its own rules about what redirects it accepts;[2] for instance, it does not allow a redirect that adds query arguments. We handle response codes 301, 302, 303, and 307 as per RFC 2616.[3] On POST we do not handle 303, and we follow redirects only if http.followRedirects == true. Redirects are followed only a certain number of times. There are two ways to control that limit: * by default, the limit is given by the http.maxRedirects system property that is also used by the JDK. If the system property is not set, the default is 5. (This is much lower than the JDK default of 20, but I don't see the value of following so many redirects.) * this can be overwritten by a http.maxRedirects git config setting. The JGit http.* git config settings are currently all global; JGit has no support yet for URI-specific settings "http.<pattern>.name". Adding support for that is well beyond the scope of this change. Like git-core, we log every redirect attempt (LOG.info) so that users may know about the redirection having occurred. Extends the test framework to configure an AppServer with HTTPS support so that we can test cloning via HTTPS and redirections involving HTTPS. [1] https://git-scm.com/docs/git-config [2] https://kernel.googlesource.com/pub/scm/git/git/+/6628eb41db5189c0cdfdced6d8697e7c813c5f0f [3] https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html CQ: 13987 Bug: 465167 Change-Id: I86518cb76842f7d326b51f8715e3bbf8ada89859 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
9 years ago
Add support to follow HTTP redirects git-core follows HTTP redirects so JGit should also provide this. Implement config setting http.followRedirects with possible values "false" (= never), "true" (= always), and "initial" (only on GET, but not on POST).[1] We must do our own redirect handling and cannot rely on the support that the underlying real connection may offer. At least the JDK's HttpURLConnection has two features that get in the way: * it does not allow cross-protocol redirects and thus fails on http->https redirects (for instance, on Github). * it translates a redirect after a POST to a GET unless the system property "http.strictPostRedirect" is set to true. We don't want to manipulate that system setting nor require it. Additionally, git has its own rules about what redirects it accepts;[2] for instance, it does not allow a redirect that adds query arguments. We handle response codes 301, 302, 303, and 307 as per RFC 2616.[3] On POST we do not handle 303, and we follow redirects only if http.followRedirects == true. Redirects are followed only a certain number of times. There are two ways to control that limit: * by default, the limit is given by the http.maxRedirects system property that is also used by the JDK. If the system property is not set, the default is 5. (This is much lower than the JDK default of 20, but I don't see the value of following so many redirects.) * this can be overwritten by a http.maxRedirects git config setting. The JGit http.* git config settings are currently all global; JGit has no support yet for URI-specific settings "http.<pattern>.name". Adding support for that is well beyond the scope of this change. Like git-core, we log every redirect attempt (LOG.info) so that users may know about the redirection having occurred. Extends the test framework to configure an AppServer with HTTPS support so that we can test cloning via HTTPS and redirections involving HTTPS. [1] https://git-scm.com/docs/git-config [2] https://kernel.googlesource.com/pub/scm/git/git/+/6628eb41db5189c0cdfdced6d8697e7c813c5f0f [3] https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html CQ: 13987 Bug: 465167 Change-Id: I86518cb76842f7d326b51f8715e3bbf8ada89859 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
9 years ago
Support LFS protocol and a file system based LFS storage Implement LfsProtocolServlet handling the "Git LFS v1 Batch API" protocol [1]. Add a simple file system based LFS content store and the debug-lfs-store command to simplify testing. Introduce a LargeFileRepository interface to enable additional storage implementation while reusing the same protocol implementation. At the client side we have to configure the lfs.url, specify that we use the batch API and we don't use authentication: [lfs] url = http://host:port/lfs batch = true [lfs "http://host:port/lfs"] access = none the git-lfs client appends the "objects/batch" to the lfs.url. Hard code an Authorization header in the FileLfsRepository.getAction because then git-lfs client will skip asking for credentials. It will just forward the Authorization header from the response to the download/upload request. The FileLfsServlet supports file content storage for "Large File Storage" (LFS) server as defined by the Github LFS API [2]. - upload and download of large files is probably network bound hence use an asynchronous servlet for good scalability - simple object storage in file system with 2 level fan-out - use LockFile to protect writing large objects against multiple concurrent uploads of the same object - to prevent corrupt uploads the uploaded file is rejected if its hash doesn't match id given in URL The debug-lfs-store command is used to run the LfsProtocolServlet and, optionally, the FileLfsServlet which makes it easier to setup a local test server. [1] https://github.com/github/git-lfs/blob/master/docs/api/http-v1-batch.md [2] https://github.com/github/git-lfs/tree/master/docs/api Bug: 472961 Change-Id: I7378da5575159d2195138d799704880c5c82d5f3 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Signed-off-by: Sasa Zivkov <sasa.zivkov@sap.com>
8 years ago
Support LFS protocol and a file system based LFS storage Implement LfsProtocolServlet handling the "Git LFS v1 Batch API" protocol [1]. Add a simple file system based LFS content store and the debug-lfs-store command to simplify testing. Introduce a LargeFileRepository interface to enable additional storage implementation while reusing the same protocol implementation. At the client side we have to configure the lfs.url, specify that we use the batch API and we don't use authentication: [lfs] url = http://host:port/lfs batch = true [lfs "http://host:port/lfs"] access = none the git-lfs client appends the "objects/batch" to the lfs.url. Hard code an Authorization header in the FileLfsRepository.getAction because then git-lfs client will skip asking for credentials. It will just forward the Authorization header from the response to the download/upload request. The FileLfsServlet supports file content storage for "Large File Storage" (LFS) server as defined by the Github LFS API [2]. - upload and download of large files is probably network bound hence use an asynchronous servlet for good scalability - simple object storage in file system with 2 level fan-out - use LockFile to protect writing large objects against multiple concurrent uploads of the same object - to prevent corrupt uploads the uploaded file is rejected if its hash doesn't match id given in URL The debug-lfs-store command is used to run the LfsProtocolServlet and, optionally, the FileLfsServlet which makes it easier to setup a local test server. [1] https://github.com/github/git-lfs/blob/master/docs/api/http-v1-batch.md [2] https://github.com/github/git-lfs/tree/master/docs/api Bug: 472961 Change-Id: I7378da5575159d2195138d799704880c5c82d5f3 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Signed-off-by: Sasa Zivkov <sasa.zivkov@sap.com>
8 years ago
Add support to follow HTTP redirects git-core follows HTTP redirects so JGit should also provide this. Implement config setting http.followRedirects with possible values "false" (= never), "true" (= always), and "initial" (only on GET, but not on POST).[1] We must do our own redirect handling and cannot rely on the support that the underlying real connection may offer. At least the JDK's HttpURLConnection has two features that get in the way: * it does not allow cross-protocol redirects and thus fails on http->https redirects (for instance, on Github). * it translates a redirect after a POST to a GET unless the system property "http.strictPostRedirect" is set to true. We don't want to manipulate that system setting nor require it. Additionally, git has its own rules about what redirects it accepts;[2] for instance, it does not allow a redirect that adds query arguments. We handle response codes 301, 302, 303, and 307 as per RFC 2616.[3] On POST we do not handle 303, and we follow redirects only if http.followRedirects == true. Redirects are followed only a certain number of times. There are two ways to control that limit: * by default, the limit is given by the http.maxRedirects system property that is also used by the JDK. If the system property is not set, the default is 5. (This is much lower than the JDK default of 20, but I don't see the value of following so many redirects.) * this can be overwritten by a http.maxRedirects git config setting. The JGit http.* git config settings are currently all global; JGit has no support yet for URI-specific settings "http.<pattern>.name". Adding support for that is well beyond the scope of this change. Like git-core, we log every redirect attempt (LOG.info) so that users may know about the redirection having occurred. Extends the test framework to configure an AppServer with HTTPS support so that we can test cloning via HTTPS and redirections involving HTTPS. [1] https://git-scm.com/docs/git-config [2] https://kernel.googlesource.com/pub/scm/git/git/+/6628eb41db5189c0cdfdced6d8697e7c813c5f0f [3] https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html CQ: 13987 Bug: 465167 Change-Id: I86518cb76842f7d326b51f8715e3bbf8ada89859 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
9 years ago
Support LFS protocol and a file system based LFS storage Implement LfsProtocolServlet handling the "Git LFS v1 Batch API" protocol [1]. Add a simple file system based LFS content store and the debug-lfs-store command to simplify testing. Introduce a LargeFileRepository interface to enable additional storage implementation while reusing the same protocol implementation. At the client side we have to configure the lfs.url, specify that we use the batch API and we don't use authentication: [lfs] url = http://host:port/lfs batch = true [lfs "http://host:port/lfs"] access = none the git-lfs client appends the "objects/batch" to the lfs.url. Hard code an Authorization header in the FileLfsRepository.getAction because then git-lfs client will skip asking for credentials. It will just forward the Authorization header from the response to the download/upload request. The FileLfsServlet supports file content storage for "Large File Storage" (LFS) server as defined by the Github LFS API [2]. - upload and download of large files is probably network bound hence use an asynchronous servlet for good scalability - simple object storage in file system with 2 level fan-out - use LockFile to protect writing large objects against multiple concurrent uploads of the same object - to prevent corrupt uploads the uploaded file is rejected if its hash doesn't match id given in URL The debug-lfs-store command is used to run the LfsProtocolServlet and, optionally, the FileLfsServlet which makes it easier to setup a local test server. [1] https://github.com/github/git-lfs/blob/master/docs/api/http-v1-batch.md [2] https://github.com/github/git-lfs/tree/master/docs/api Bug: 472961 Change-Id: I7378da5575159d2195138d799704880c5c82d5f3 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Signed-off-by: Sasa Zivkov <sasa.zivkov@sap.com>
8 years ago
Add support to follow HTTP redirects git-core follows HTTP redirects so JGit should also provide this. Implement config setting http.followRedirects with possible values "false" (= never), "true" (= always), and "initial" (only on GET, but not on POST).[1] We must do our own redirect handling and cannot rely on the support that the underlying real connection may offer. At least the JDK's HttpURLConnection has two features that get in the way: * it does not allow cross-protocol redirects and thus fails on http->https redirects (for instance, on Github). * it translates a redirect after a POST to a GET unless the system property "http.strictPostRedirect" is set to true. We don't want to manipulate that system setting nor require it. Additionally, git has its own rules about what redirects it accepts;[2] for instance, it does not allow a redirect that adds query arguments. We handle response codes 301, 302, 303, and 307 as per RFC 2616.[3] On POST we do not handle 303, and we follow redirects only if http.followRedirects == true. Redirects are followed only a certain number of times. There are two ways to control that limit: * by default, the limit is given by the http.maxRedirects system property that is also used by the JDK. If the system property is not set, the default is 5. (This is much lower than the JDK default of 20, but I don't see the value of following so many redirects.) * this can be overwritten by a http.maxRedirects git config setting. The JGit http.* git config settings are currently all global; JGit has no support yet for URI-specific settings "http.<pattern>.name". Adding support for that is well beyond the scope of this change. Like git-core, we log every redirect attempt (LOG.info) so that users may know about the redirection having occurred. Extends the test framework to configure an AppServer with HTTPS support so that we can test cloning via HTTPS and redirections involving HTTPS. [1] https://git-scm.com/docs/git-config [2] https://kernel.googlesource.com/pub/scm/git/git/+/6628eb41db5189c0cdfdced6d8697e7c813c5f0f [3] https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html CQ: 13987 Bug: 465167 Change-Id: I86518cb76842f7d326b51f8715e3bbf8ada89859 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
9 years ago
Add support to follow HTTP redirects git-core follows HTTP redirects so JGit should also provide this. Implement config setting http.followRedirects with possible values "false" (= never), "true" (= always), and "initial" (only on GET, but not on POST).[1] We must do our own redirect handling and cannot rely on the support that the underlying real connection may offer. At least the JDK's HttpURLConnection has two features that get in the way: * it does not allow cross-protocol redirects and thus fails on http->https redirects (for instance, on Github). * it translates a redirect after a POST to a GET unless the system property "http.strictPostRedirect" is set to true. We don't want to manipulate that system setting nor require it. Additionally, git has its own rules about what redirects it accepts;[2] for instance, it does not allow a redirect that adds query arguments. We handle response codes 301, 302, 303, and 307 as per RFC 2616.[3] On POST we do not handle 303, and we follow redirects only if http.followRedirects == true. Redirects are followed only a certain number of times. There are two ways to control that limit: * by default, the limit is given by the http.maxRedirects system property that is also used by the JDK. If the system property is not set, the default is 5. (This is much lower than the JDK default of 20, but I don't see the value of following so many redirects.) * this can be overwritten by a http.maxRedirects git config setting. The JGit http.* git config settings are currently all global; JGit has no support yet for URI-specific settings "http.<pattern>.name". Adding support for that is well beyond the scope of this change. Like git-core, we log every redirect attempt (LOG.info) so that users may know about the redirection having occurred. Extends the test framework to configure an AppServer with HTTPS support so that we can test cloning via HTTPS and redirections involving HTTPS. [1] https://git-scm.com/docs/git-config [2] https://kernel.googlesource.com/pub/scm/git/git/+/6628eb41db5189c0cdfdced6d8697e7c813c5f0f [3] https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html CQ: 13987 Bug: 465167 Change-Id: I86518cb76842f7d326b51f8715e3bbf8ada89859 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
9 years ago
Do authentication re-tries on HTTP POST There is at least one git server out there (GOGS) that does not require authentication on the initial GET for info/refs?service=git-receive-pack but that _does_ require authentication for the subsequent POST to actually do the push. This occurs on GOGS with public repositories; for private repositories it wants authentication up front. Handle this behavior by adding 401 handling to our POST request. Note that this is suboptimal; we'll re-send the push data at least twice if an authentication failure on POST occurs. It would be much better if the server required authentication up-front in the GET request. Added authentication unit tests (using BASIC auth) to the SmartClientSmartServerTest: - clone with authentication - clone with authentication but lacking CredentialsProvider - clone with authentication and wrong password - clone with authentication after redirect - clone with authentication only on POST, but not on GET Also tested manually in the wild using repositories at try.gogs.io. That server offers only BASIC auth, so the other paths (DIGEST, NEGOTIATE, fall back from DIGEST to BASIC) are untested and I have no way to test them. * public repository: GET unauthenticated, POST authenticated Also tested after clearing the credentials and then entering a wrong password: correctly asks three times during the HTTP POST for user name and password, then gives up. * private repository: authentication already on GET; then gets applied correctly initially to the POST request, which succeeds. Also fix the authentication to use the credentials for the redirected URI if redirects had occurred. We must not present the credentials for the original URI in that case. Consider a malicious redirect A->B: this would allow server B to harvest the user credentials for server A. The unit test for authentication after a redirect also tests for this. Bug: 513043 Change-Id: I97ee5058569efa1545a6c6f6edfd2b357c40592a Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years ago
Add support to follow HTTP redirects git-core follows HTTP redirects so JGit should also provide this. Implement config setting http.followRedirects with possible values "false" (= never), "true" (= always), and "initial" (only on GET, but not on POST).[1] We must do our own redirect handling and cannot rely on the support that the underlying real connection may offer. At least the JDK's HttpURLConnection has two features that get in the way: * it does not allow cross-protocol redirects and thus fails on http->https redirects (for instance, on Github). * it translates a redirect after a POST to a GET unless the system property "http.strictPostRedirect" is set to true. We don't want to manipulate that system setting nor require it. Additionally, git has its own rules about what redirects it accepts;[2] for instance, it does not allow a redirect that adds query arguments. We handle response codes 301, 302, 303, and 307 as per RFC 2616.[3] On POST we do not handle 303, and we follow redirects only if http.followRedirects == true. Redirects are followed only a certain number of times. There are two ways to control that limit: * by default, the limit is given by the http.maxRedirects system property that is also used by the JDK. If the system property is not set, the default is 5. (This is much lower than the JDK default of 20, but I don't see the value of following so many redirects.) * this can be overwritten by a http.maxRedirects git config setting. The JGit http.* git config settings are currently all global; JGit has no support yet for URI-specific settings "http.<pattern>.name". Adding support for that is well beyond the scope of this change. Like git-core, we log every redirect attempt (LOG.info) so that users may know about the redirection having occurred. Extends the test framework to configure an AppServer with HTTPS support so that we can test cloning via HTTPS and redirections involving HTTPS. [1] https://git-scm.com/docs/git-config [2] https://kernel.googlesource.com/pub/scm/git/git/+/6628eb41db5189c0cdfdced6d8697e7c813c5f0f [3] https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html CQ: 13987 Bug: 465167 Change-Id: I86518cb76842f7d326b51f8715e3bbf8ada89859 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
9 years ago
Do authentication re-tries on HTTP POST There is at least one git server out there (GOGS) that does not require authentication on the initial GET for info/refs?service=git-receive-pack but that _does_ require authentication for the subsequent POST to actually do the push. This occurs on GOGS with public repositories; for private repositories it wants authentication up front. Handle this behavior by adding 401 handling to our POST request. Note that this is suboptimal; we'll re-send the push data at least twice if an authentication failure on POST occurs. It would be much better if the server required authentication up-front in the GET request. Added authentication unit tests (using BASIC auth) to the SmartClientSmartServerTest: - clone with authentication - clone with authentication but lacking CredentialsProvider - clone with authentication and wrong password - clone with authentication after redirect - clone with authentication only on POST, but not on GET Also tested manually in the wild using repositories at try.gogs.io. That server offers only BASIC auth, so the other paths (DIGEST, NEGOTIATE, fall back from DIGEST to BASIC) are untested and I have no way to test them. * public repository: GET unauthenticated, POST authenticated Also tested after clearing the credentials and then entering a wrong password: correctly asks three times during the HTTP POST for user name and password, then gives up. * private repository: authentication already on GET; then gets applied correctly initially to the POST request, which succeeds. Also fix the authentication to use the credentials for the redirected URI if redirects had occurred. We must not present the credentials for the original URI in that case. Consider a malicious redirect A->B: this would allow server B to harvest the user credentials for server A. The unit test for authentication after a redirect also tests for this. Bug: 513043 Change-Id: I97ee5058569efa1545a6c6f6edfd2b357c40592a Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years ago
Add support to follow HTTP redirects git-core follows HTTP redirects so JGit should also provide this. Implement config setting http.followRedirects with possible values "false" (= never), "true" (= always), and "initial" (only on GET, but not on POST).[1] We must do our own redirect handling and cannot rely on the support that the underlying real connection may offer. At least the JDK's HttpURLConnection has two features that get in the way: * it does not allow cross-protocol redirects and thus fails on http->https redirects (for instance, on Github). * it translates a redirect after a POST to a GET unless the system property "http.strictPostRedirect" is set to true. We don't want to manipulate that system setting nor require it. Additionally, git has its own rules about what redirects it accepts;[2] for instance, it does not allow a redirect that adds query arguments. We handle response codes 301, 302, 303, and 307 as per RFC 2616.[3] On POST we do not handle 303, and we follow redirects only if http.followRedirects == true. Redirects are followed only a certain number of times. There are two ways to control that limit: * by default, the limit is given by the http.maxRedirects system property that is also used by the JDK. If the system property is not set, the default is 5. (This is much lower than the JDK default of 20, but I don't see the value of following so many redirects.) * this can be overwritten by a http.maxRedirects git config setting. The JGit http.* git config settings are currently all global; JGit has no support yet for URI-specific settings "http.<pattern>.name". Adding support for that is well beyond the scope of this change. Like git-core, we log every redirect attempt (LOG.info) so that users may know about the redirection having occurred. Extends the test framework to configure an AppServer with HTTPS support so that we can test cloning via HTTPS and redirections involving HTTPS. [1] https://git-scm.com/docs/git-config [2] https://kernel.googlesource.com/pub/scm/git/git/+/6628eb41db5189c0cdfdced6d8697e7c813c5f0f [3] https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html CQ: 13987 Bug: 465167 Change-Id: I86518cb76842f7d326b51f8715e3bbf8ada89859 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
9 years ago
Add support to follow HTTP redirects git-core follows HTTP redirects so JGit should also provide this. Implement config setting http.followRedirects with possible values "false" (= never), "true" (= always), and "initial" (only on GET, but not on POST).[1] We must do our own redirect handling and cannot rely on the support that the underlying real connection may offer. At least the JDK's HttpURLConnection has two features that get in the way: * it does not allow cross-protocol redirects and thus fails on http->https redirects (for instance, on Github). * it translates a redirect after a POST to a GET unless the system property "http.strictPostRedirect" is set to true. We don't want to manipulate that system setting nor require it. Additionally, git has its own rules about what redirects it accepts;[2] for instance, it does not allow a redirect that adds query arguments. We handle response codes 301, 302, 303, and 307 as per RFC 2616.[3] On POST we do not handle 303, and we follow redirects only if http.followRedirects == true. Redirects are followed only a certain number of times. There are two ways to control that limit: * by default, the limit is given by the http.maxRedirects system property that is also used by the JDK. If the system property is not set, the default is 5. (This is much lower than the JDK default of 20, but I don't see the value of following so many redirects.) * this can be overwritten by a http.maxRedirects git config setting. The JGit http.* git config settings are currently all global; JGit has no support yet for URI-specific settings "http.<pattern>.name". Adding support for that is well beyond the scope of this change. Like git-core, we log every redirect attempt (LOG.info) so that users may know about the redirection having occurred. Extends the test framework to configure an AppServer with HTTPS support so that we can test cloning via HTTPS and redirections involving HTTPS. [1] https://git-scm.com/docs/git-config [2] https://kernel.googlesource.com/pub/scm/git/git/+/6628eb41db5189c0cdfdced6d8697e7c813c5f0f [3] https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html CQ: 13987 Bug: 465167 Change-Id: I86518cb76842f7d326b51f8715e3bbf8ada89859 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
9 years ago
Add support to follow HTTP redirects git-core follows HTTP redirects so JGit should also provide this. Implement config setting http.followRedirects with possible values "false" (= never), "true" (= always), and "initial" (only on GET, but not on POST).[1] We must do our own redirect handling and cannot rely on the support that the underlying real connection may offer. At least the JDK's HttpURLConnection has two features that get in the way: * it does not allow cross-protocol redirects and thus fails on http->https redirects (for instance, on Github). * it translates a redirect after a POST to a GET unless the system property "http.strictPostRedirect" is set to true. We don't want to manipulate that system setting nor require it. Additionally, git has its own rules about what redirects it accepts;[2] for instance, it does not allow a redirect that adds query arguments. We handle response codes 301, 302, 303, and 307 as per RFC 2616.[3] On POST we do not handle 303, and we follow redirects only if http.followRedirects == true. Redirects are followed only a certain number of times. There are two ways to control that limit: * by default, the limit is given by the http.maxRedirects system property that is also used by the JDK. If the system property is not set, the default is 5. (This is much lower than the JDK default of 20, but I don't see the value of following so many redirects.) * this can be overwritten by a http.maxRedirects git config setting. The JGit http.* git config settings are currently all global; JGit has no support yet for URI-specific settings "http.<pattern>.name". Adding support for that is well beyond the scope of this change. Like git-core, we log every redirect attempt (LOG.info) so that users may know about the redirection having occurred. Extends the test framework to configure an AppServer with HTTPS support so that we can test cloning via HTTPS and redirections involving HTTPS. [1] https://git-scm.com/docs/git-config [2] https://kernel.googlesource.com/pub/scm/git/git/+/6628eb41db5189c0cdfdced6d8697e7c813c5f0f [3] https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html CQ: 13987 Bug: 465167 Change-Id: I86518cb76842f7d326b51f8715e3bbf8ada89859 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
9 years ago
Support LFS protocol and a file system based LFS storage Implement LfsProtocolServlet handling the "Git LFS v1 Batch API" protocol [1]. Add a simple file system based LFS content store and the debug-lfs-store command to simplify testing. Introduce a LargeFileRepository interface to enable additional storage implementation while reusing the same protocol implementation. At the client side we have to configure the lfs.url, specify that we use the batch API and we don't use authentication: [lfs] url = http://host:port/lfs batch = true [lfs "http://host:port/lfs"] access = none the git-lfs client appends the "objects/batch" to the lfs.url. Hard code an Authorization header in the FileLfsRepository.getAction because then git-lfs client will skip asking for credentials. It will just forward the Authorization header from the response to the download/upload request. The FileLfsServlet supports file content storage for "Large File Storage" (LFS) server as defined by the Github LFS API [2]. - upload and download of large files is probably network bound hence use an asynchronous servlet for good scalability - simple object storage in file system with 2 level fan-out - use LockFile to protect writing large objects against multiple concurrent uploads of the same object - to prevent corrupt uploads the uploaded file is rejected if its hash doesn't match id given in URL The debug-lfs-store command is used to run the LfsProtocolServlet and, optionally, the FileLfsServlet which makes it easier to setup a local test server. [1] https://github.com/github/git-lfs/blob/master/docs/api/http-v1-batch.md [2] https://github.com/github/git-lfs/tree/master/docs/api Bug: 472961 Change-Id: I7378da5575159d2195138d799704880c5c82d5f3 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Signed-off-by: Sasa Zivkov <sasa.zivkov@sap.com>
8 years ago
Add support to follow HTTP redirects git-core follows HTTP redirects so JGit should also provide this. Implement config setting http.followRedirects with possible values "false" (= never), "true" (= always), and "initial" (only on GET, but not on POST).[1] We must do our own redirect handling and cannot rely on the support that the underlying real connection may offer. At least the JDK's HttpURLConnection has two features that get in the way: * it does not allow cross-protocol redirects and thus fails on http->https redirects (for instance, on Github). * it translates a redirect after a POST to a GET unless the system property "http.strictPostRedirect" is set to true. We don't want to manipulate that system setting nor require it. Additionally, git has its own rules about what redirects it accepts;[2] for instance, it does not allow a redirect that adds query arguments. We handle response codes 301, 302, 303, and 307 as per RFC 2616.[3] On POST we do not handle 303, and we follow redirects only if http.followRedirects == true. Redirects are followed only a certain number of times. There are two ways to control that limit: * by default, the limit is given by the http.maxRedirects system property that is also used by the JDK. If the system property is not set, the default is 5. (This is much lower than the JDK default of 20, but I don't see the value of following so many redirects.) * this can be overwritten by a http.maxRedirects git config setting. The JGit http.* git config settings are currently all global; JGit has no support yet for URI-specific settings "http.<pattern>.name". Adding support for that is well beyond the scope of this change. Like git-core, we log every redirect attempt (LOG.info) so that users may know about the redirection having occurred. Extends the test framework to configure an AppServer with HTTPS support so that we can test cloning via HTTPS and redirections involving HTTPS. [1] https://git-scm.com/docs/git-config [2] https://kernel.googlesource.com/pub/scm/git/git/+/6628eb41db5189c0cdfdced6d8697e7c813c5f0f [3] https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html CQ: 13987 Bug: 465167 Change-Id: I86518cb76842f7d326b51f8715e3bbf8ada89859 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
9 years ago
Add support to follow HTTP redirects git-core follows HTTP redirects so JGit should also provide this. Implement config setting http.followRedirects with possible values "false" (= never), "true" (= always), and "initial" (only on GET, but not on POST).[1] We must do our own redirect handling and cannot rely on the support that the underlying real connection may offer. At least the JDK's HttpURLConnection has two features that get in the way: * it does not allow cross-protocol redirects and thus fails on http->https redirects (for instance, on Github). * it translates a redirect after a POST to a GET unless the system property "http.strictPostRedirect" is set to true. We don't want to manipulate that system setting nor require it. Additionally, git has its own rules about what redirects it accepts;[2] for instance, it does not allow a redirect that adds query arguments. We handle response codes 301, 302, 303, and 307 as per RFC 2616.[3] On POST we do not handle 303, and we follow redirects only if http.followRedirects == true. Redirects are followed only a certain number of times. There are two ways to control that limit: * by default, the limit is given by the http.maxRedirects system property that is also used by the JDK. If the system property is not set, the default is 5. (This is much lower than the JDK default of 20, but I don't see the value of following so many redirects.) * this can be overwritten by a http.maxRedirects git config setting. The JGit http.* git config settings are currently all global; JGit has no support yet for URI-specific settings "http.<pattern>.name". Adding support for that is well beyond the scope of this change. Like git-core, we log every redirect attempt (LOG.info) so that users may know about the redirection having occurred. Extends the test framework to configure an AppServer with HTTPS support so that we can test cloning via HTTPS and redirections involving HTTPS. [1] https://git-scm.com/docs/git-config [2] https://kernel.googlesource.com/pub/scm/git/git/+/6628eb41db5189c0cdfdced6d8697e7c813c5f0f [3] https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html CQ: 13987 Bug: 465167 Change-Id: I86518cb76842f7d326b51f8715e3bbf8ada89859 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
9 years ago
Add support to follow HTTP redirects git-core follows HTTP redirects so JGit should also provide this. Implement config setting http.followRedirects with possible values "false" (= never), "true" (= always), and "initial" (only on GET, but not on POST).[1] We must do our own redirect handling and cannot rely on the support that the underlying real connection may offer. At least the JDK's HttpURLConnection has two features that get in the way: * it does not allow cross-protocol redirects and thus fails on http->https redirects (for instance, on Github). * it translates a redirect after a POST to a GET unless the system property "http.strictPostRedirect" is set to true. We don't want to manipulate that system setting nor require it. Additionally, git has its own rules about what redirects it accepts;[2] for instance, it does not allow a redirect that adds query arguments. We handle response codes 301, 302, 303, and 307 as per RFC 2616.[3] On POST we do not handle 303, and we follow redirects only if http.followRedirects == true. Redirects are followed only a certain number of times. There are two ways to control that limit: * by default, the limit is given by the http.maxRedirects system property that is also used by the JDK. If the system property is not set, the default is 5. (This is much lower than the JDK default of 20, but I don't see the value of following so many redirects.) * this can be overwritten by a http.maxRedirects git config setting. The JGit http.* git config settings are currently all global; JGit has no support yet for URI-specific settings "http.<pattern>.name". Adding support for that is well beyond the scope of this change. Like git-core, we log every redirect attempt (LOG.info) so that users may know about the redirection having occurred. Extends the test framework to configure an AppServer with HTTPS support so that we can test cloning via HTTPS and redirections involving HTTPS. [1] https://git-scm.com/docs/git-config [2] https://kernel.googlesource.com/pub/scm/git/git/+/6628eb41db5189c0cdfdced6d8697e7c813c5f0f [3] https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html CQ: 13987 Bug: 465167 Change-Id: I86518cb76842f7d326b51f8715e3bbf8ada89859 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
9 years ago
Add support to follow HTTP redirects git-core follows HTTP redirects so JGit should also provide this. Implement config setting http.followRedirects with possible values "false" (= never), "true" (= always), and "initial" (only on GET, but not on POST).[1] We must do our own redirect handling and cannot rely on the support that the underlying real connection may offer. At least the JDK's HttpURLConnection has two features that get in the way: * it does not allow cross-protocol redirects and thus fails on http->https redirects (for instance, on Github). * it translates a redirect after a POST to a GET unless the system property "http.strictPostRedirect" is set to true. We don't want to manipulate that system setting nor require it. Additionally, git has its own rules about what redirects it accepts;[2] for instance, it does not allow a redirect that adds query arguments. We handle response codes 301, 302, 303, and 307 as per RFC 2616.[3] On POST we do not handle 303, and we follow redirects only if http.followRedirects == true. Redirects are followed only a certain number of times. There are two ways to control that limit: * by default, the limit is given by the http.maxRedirects system property that is also used by the JDK. If the system property is not set, the default is 5. (This is much lower than the JDK default of 20, but I don't see the value of following so many redirects.) * this can be overwritten by a http.maxRedirects git config setting. The JGit http.* git config settings are currently all global; JGit has no support yet for URI-specific settings "http.<pattern>.name". Adding support for that is well beyond the scope of this change. Like git-core, we log every redirect attempt (LOG.info) so that users may know about the redirection having occurred. Extends the test framework to configure an AppServer with HTTPS support so that we can test cloning via HTTPS and redirections involving HTTPS. [1] https://git-scm.com/docs/git-config [2] https://kernel.googlesource.com/pub/scm/git/git/+/6628eb41db5189c0cdfdced6d8697e7c813c5f0f [3] https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html CQ: 13987 Bug: 465167 Change-Id: I86518cb76842f7d326b51f8715e3bbf8ada89859 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
9 years ago
Add support to follow HTTP redirects git-core follows HTTP redirects so JGit should also provide this. Implement config setting http.followRedirects with possible values "false" (= never), "true" (= always), and "initial" (only on GET, but not on POST).[1] We must do our own redirect handling and cannot rely on the support that the underlying real connection may offer. At least the JDK's HttpURLConnection has two features that get in the way: * it does not allow cross-protocol redirects and thus fails on http->https redirects (for instance, on Github). * it translates a redirect after a POST to a GET unless the system property "http.strictPostRedirect" is set to true. We don't want to manipulate that system setting nor require it. Additionally, git has its own rules about what redirects it accepts;[2] for instance, it does not allow a redirect that adds query arguments. We handle response codes 301, 302, 303, and 307 as per RFC 2616.[3] On POST we do not handle 303, and we follow redirects only if http.followRedirects == true. Redirects are followed only a certain number of times. There are two ways to control that limit: * by default, the limit is given by the http.maxRedirects system property that is also used by the JDK. If the system property is not set, the default is 5. (This is much lower than the JDK default of 20, but I don't see the value of following so many redirects.) * this can be overwritten by a http.maxRedirects git config setting. The JGit http.* git config settings are currently all global; JGit has no support yet for URI-specific settings "http.<pattern>.name". Adding support for that is well beyond the scope of this change. Like git-core, we log every redirect attempt (LOG.info) so that users may know about the redirection having occurred. Extends the test framework to configure an AppServer with HTTPS support so that we can test cloning via HTTPS and redirections involving HTTPS. [1] https://git-scm.com/docs/git-config [2] https://kernel.googlesource.com/pub/scm/git/git/+/6628eb41db5189c0cdfdced6d8697e7c813c5f0f [3] https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html CQ: 13987 Bug: 465167 Change-Id: I86518cb76842f7d326b51f8715e3bbf8ada89859 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
9 years ago
Add support to follow HTTP redirects git-core follows HTTP redirects so JGit should also provide this. Implement config setting http.followRedirects with possible values "false" (= never), "true" (= always), and "initial" (only on GET, but not on POST).[1] We must do our own redirect handling and cannot rely on the support that the underlying real connection may offer. At least the JDK's HttpURLConnection has two features that get in the way: * it does not allow cross-protocol redirects and thus fails on http->https redirects (for instance, on Github). * it translates a redirect after a POST to a GET unless the system property "http.strictPostRedirect" is set to true. We don't want to manipulate that system setting nor require it. Additionally, git has its own rules about what redirects it accepts;[2] for instance, it does not allow a redirect that adds query arguments. We handle response codes 301, 302, 303, and 307 as per RFC 2616.[3] On POST we do not handle 303, and we follow redirects only if http.followRedirects == true. Redirects are followed only a certain number of times. There are two ways to control that limit: * by default, the limit is given by the http.maxRedirects system property that is also used by the JDK. If the system property is not set, the default is 5. (This is much lower than the JDK default of 20, but I don't see the value of following so many redirects.) * this can be overwritten by a http.maxRedirects git config setting. The JGit http.* git config settings are currently all global; JGit has no support yet for URI-specific settings "http.<pattern>.name". Adding support for that is well beyond the scope of this change. Like git-core, we log every redirect attempt (LOG.info) so that users may know about the redirection having occurred. Extends the test framework to configure an AppServer with HTTPS support so that we can test cloning via HTTPS and redirections involving HTTPS. [1] https://git-scm.com/docs/git-config [2] https://kernel.googlesource.com/pub/scm/git/git/+/6628eb41db5189c0cdfdced6d8697e7c813c5f0f [3] https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html CQ: 13987 Bug: 465167 Change-Id: I86518cb76842f7d326b51f8715e3bbf8ada89859 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
9 years ago
Do authentication re-tries on HTTP POST There is at least one git server out there (GOGS) that does not require authentication on the initial GET for info/refs?service=git-receive-pack but that _does_ require authentication for the subsequent POST to actually do the push. This occurs on GOGS with public repositories; for private repositories it wants authentication up front. Handle this behavior by adding 401 handling to our POST request. Note that this is suboptimal; we'll re-send the push data at least twice if an authentication failure on POST occurs. It would be much better if the server required authentication up-front in the GET request. Added authentication unit tests (using BASIC auth) to the SmartClientSmartServerTest: - clone with authentication - clone with authentication but lacking CredentialsProvider - clone with authentication and wrong password - clone with authentication after redirect - clone with authentication only on POST, but not on GET Also tested manually in the wild using repositories at try.gogs.io. That server offers only BASIC auth, so the other paths (DIGEST, NEGOTIATE, fall back from DIGEST to BASIC) are untested and I have no way to test them. * public repository: GET unauthenticated, POST authenticated Also tested after clearing the credentials and then entering a wrong password: correctly asks three times during the HTTP POST for user name and password, then gives up. * private repository: authentication already on GET; then gets applied correctly initially to the POST request, which succeeds. Also fix the authentication to use the credentials for the redirected URI if redirects had occurred. We must not present the credentials for the original URI in that case. Consider a malicious redirect A->B: this would allow server B to harvest the user credentials for server A. The unit test for authentication after a redirect also tests for this. Bug: 513043 Change-Id: I97ee5058569efa1545a6c6f6edfd2b357c40592a Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years ago
Do authentication re-tries on HTTP POST There is at least one git server out there (GOGS) that does not require authentication on the initial GET for info/refs?service=git-receive-pack but that _does_ require authentication for the subsequent POST to actually do the push. This occurs on GOGS with public repositories; for private repositories it wants authentication up front. Handle this behavior by adding 401 handling to our POST request. Note that this is suboptimal; we'll re-send the push data at least twice if an authentication failure on POST occurs. It would be much better if the server required authentication up-front in the GET request. Added authentication unit tests (using BASIC auth) to the SmartClientSmartServerTest: - clone with authentication - clone with authentication but lacking CredentialsProvider - clone with authentication and wrong password - clone with authentication after redirect - clone with authentication only on POST, but not on GET Also tested manually in the wild using repositories at try.gogs.io. That server offers only BASIC auth, so the other paths (DIGEST, NEGOTIATE, fall back from DIGEST to BASIC) are untested and I have no way to test them. * public repository: GET unauthenticated, POST authenticated Also tested after clearing the credentials and then entering a wrong password: correctly asks three times during the HTTP POST for user name and password, then gives up. * private repository: authentication already on GET; then gets applied correctly initially to the POST request, which succeeds. Also fix the authentication to use the credentials for the redirected URI if redirects had occurred. We must not present the credentials for the original URI in that case. Consider a malicious redirect A->B: this would allow server B to harvest the user credentials for server A. The unit test for authentication after a redirect also tests for this. Bug: 513043 Change-Id: I97ee5058569efa1545a6c6f6edfd2b357c40592a Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years ago
Do authentication re-tries on HTTP POST There is at least one git server out there (GOGS) that does not require authentication on the initial GET for info/refs?service=git-receive-pack but that _does_ require authentication for the subsequent POST to actually do the push. This occurs on GOGS with public repositories; for private repositories it wants authentication up front. Handle this behavior by adding 401 handling to our POST request. Note that this is suboptimal; we'll re-send the push data at least twice if an authentication failure on POST occurs. It would be much better if the server required authentication up-front in the GET request. Added authentication unit tests (using BASIC auth) to the SmartClientSmartServerTest: - clone with authentication - clone with authentication but lacking CredentialsProvider - clone with authentication and wrong password - clone with authentication after redirect - clone with authentication only on POST, but not on GET Also tested manually in the wild using repositories at try.gogs.io. That server offers only BASIC auth, so the other paths (DIGEST, NEGOTIATE, fall back from DIGEST to BASIC) are untested and I have no way to test them. * public repository: GET unauthenticated, POST authenticated Also tested after clearing the credentials and then entering a wrong password: correctly asks three times during the HTTP POST for user name and password, then gives up. * private repository: authentication already on GET; then gets applied correctly initially to the POST request, which succeeds. Also fix the authentication to use the credentials for the redirected URI if redirects had occurred. We must not present the credentials for the original URI in that case. Consider a malicious redirect A->B: this would allow server B to harvest the user credentials for server A. The unit test for authentication after a redirect also tests for this. Bug: 513043 Change-Id: I97ee5058569efa1545a6c6f6edfd2b357c40592a Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years ago
Do authentication re-tries on HTTP POST There is at least one git server out there (GOGS) that does not require authentication on the initial GET for info/refs?service=git-receive-pack but that _does_ require authentication for the subsequent POST to actually do the push. This occurs on GOGS with public repositories; for private repositories it wants authentication up front. Handle this behavior by adding 401 handling to our POST request. Note that this is suboptimal; we'll re-send the push data at least twice if an authentication failure on POST occurs. It would be much better if the server required authentication up-front in the GET request. Added authentication unit tests (using BASIC auth) to the SmartClientSmartServerTest: - clone with authentication - clone with authentication but lacking CredentialsProvider - clone with authentication and wrong password - clone with authentication after redirect - clone with authentication only on POST, but not on GET Also tested manually in the wild using repositories at try.gogs.io. That server offers only BASIC auth, so the other paths (DIGEST, NEGOTIATE, fall back from DIGEST to BASIC) are untested and I have no way to test them. * public repository: GET unauthenticated, POST authenticated Also tested after clearing the credentials and then entering a wrong password: correctly asks three times during the HTTP POST for user name and password, then gives up. * private repository: authentication already on GET; then gets applied correctly initially to the POST request, which succeeds. Also fix the authentication to use the credentials for the redirected URI if redirects had occurred. We must not present the credentials for the original URI in that case. Consider a malicious redirect A->B: this would allow server B to harvest the user credentials for server A. The unit test for authentication after a redirect also tests for this. Bug: 513043 Change-Id: I97ee5058569efa1545a6c6f6edfd2b357c40592a Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years ago
Do authentication re-tries on HTTP POST There is at least one git server out there (GOGS) that does not require authentication on the initial GET for info/refs?service=git-receive-pack but that _does_ require authentication for the subsequent POST to actually do the push. This occurs on GOGS with public repositories; for private repositories it wants authentication up front. Handle this behavior by adding 401 handling to our POST request. Note that this is suboptimal; we'll re-send the push data at least twice if an authentication failure on POST occurs. It would be much better if the server required authentication up-front in the GET request. Added authentication unit tests (using BASIC auth) to the SmartClientSmartServerTest: - clone with authentication - clone with authentication but lacking CredentialsProvider - clone with authentication and wrong password - clone with authentication after redirect - clone with authentication only on POST, but not on GET Also tested manually in the wild using repositories at try.gogs.io. That server offers only BASIC auth, so the other paths (DIGEST, NEGOTIATE, fall back from DIGEST to BASIC) are untested and I have no way to test them. * public repository: GET unauthenticated, POST authenticated Also tested after clearing the credentials and then entering a wrong password: correctly asks three times during the HTTP POST for user name and password, then gives up. * private repository: authentication already on GET; then gets applied correctly initially to the POST request, which succeeds. Also fix the authentication to use the credentials for the redirected URI if redirects had occurred. We must not present the credentials for the original URI in that case. Consider a malicious redirect A->B: this would allow server B to harvest the user credentials for server A. The unit test for authentication after a redirect also tests for this. Bug: 513043 Change-Id: I97ee5058569efa1545a6c6f6edfd2b357c40592a Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years ago
Do authentication re-tries on HTTP POST There is at least one git server out there (GOGS) that does not require authentication on the initial GET for info/refs?service=git-receive-pack but that _does_ require authentication for the subsequent POST to actually do the push. This occurs on GOGS with public repositories; for private repositories it wants authentication up front. Handle this behavior by adding 401 handling to our POST request. Note that this is suboptimal; we'll re-send the push data at least twice if an authentication failure on POST occurs. It would be much better if the server required authentication up-front in the GET request. Added authentication unit tests (using BASIC auth) to the SmartClientSmartServerTest: - clone with authentication - clone with authentication but lacking CredentialsProvider - clone with authentication and wrong password - clone with authentication after redirect - clone with authentication only on POST, but not on GET Also tested manually in the wild using repositories at try.gogs.io. That server offers only BASIC auth, so the other paths (DIGEST, NEGOTIATE, fall back from DIGEST to BASIC) are untested and I have no way to test them. * public repository: GET unauthenticated, POST authenticated Also tested after clearing the credentials and then entering a wrong password: correctly asks three times during the HTTP POST for user name and password, then gives up. * private repository: authentication already on GET; then gets applied correctly initially to the POST request, which succeeds. Also fix the authentication to use the credentials for the redirected URI if redirects had occurred. We must not present the credentials for the original URI in that case. Consider a malicious redirect A->B: this would allow server B to harvest the user credentials for server A. The unit test for authentication after a redirect also tests for this. Bug: 513043 Change-Id: I97ee5058569efa1545a6c6f6edfd2b357c40592a Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
7 years ago
Add support to follow HTTP redirects git-core follows HTTP redirects so JGit should also provide this. Implement config setting http.followRedirects with possible values "false" (= never), "true" (= always), and "initial" (only on GET, but not on POST).[1] We must do our own redirect handling and cannot rely on the support that the underlying real connection may offer. At least the JDK's HttpURLConnection has two features that get in the way: * it does not allow cross-protocol redirects and thus fails on http->https redirects (for instance, on Github). * it translates a redirect after a POST to a GET unless the system property "http.strictPostRedirect" is set to true. We don't want to manipulate that system setting nor require it. Additionally, git has its own rules about what redirects it accepts;[2] for instance, it does not allow a redirect that adds query arguments. We handle response codes 301, 302, 303, and 307 as per RFC 2616.[3] On POST we do not handle 303, and we follow redirects only if http.followRedirects == true. Redirects are followed only a certain number of times. There are two ways to control that limit: * by default, the limit is given by the http.maxRedirects system property that is also used by the JDK. If the system property is not set, the default is 5. (This is much lower than the JDK default of 20, but I don't see the value of following so many redirects.) * this can be overwritten by a http.maxRedirects git config setting. The JGit http.* git config settings are currently all global; JGit has no support yet for URI-specific settings "http.<pattern>.name". Adding support for that is well beyond the scope of this change. Like git-core, we log every redirect attempt (LOG.info) so that users may know about the redirection having occurred. Extends the test framework to configure an AppServer with HTTPS support so that we can test cloning via HTTPS and redirections involving HTTPS. [1] https://git-scm.com/docs/git-config [2] https://kernel.googlesource.com/pub/scm/git/git/+/6628eb41db5189c0cdfdced6d8697e7c813c5f0f [3] https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html CQ: 13987 Bug: 465167 Change-Id: I86518cb76842f7d326b51f8715e3bbf8ada89859 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
9 years ago
Add support to follow HTTP redirects git-core follows HTTP redirects so JGit should also provide this. Implement config setting http.followRedirects with possible values "false" (= never), "true" (= always), and "initial" (only on GET, but not on POST).[1] We must do our own redirect handling and cannot rely on the support that the underlying real connection may offer. At least the JDK's HttpURLConnection has two features that get in the way: * it does not allow cross-protocol redirects and thus fails on http->https redirects (for instance, on Github). * it translates a redirect after a POST to a GET unless the system property "http.strictPostRedirect" is set to true. We don't want to manipulate that system setting nor require it. Additionally, git has its own rules about what redirects it accepts;[2] for instance, it does not allow a redirect that adds query arguments. We handle response codes 301, 302, 303, and 307 as per RFC 2616.[3] On POST we do not handle 303, and we follow redirects only if http.followRedirects == true. Redirects are followed only a certain number of times. There are two ways to control that limit: * by default, the limit is given by the http.maxRedirects system property that is also used by the JDK. If the system property is not set, the default is 5. (This is much lower than the JDK default of 20, but I don't see the value of following so many redirects.) * this can be overwritten by a http.maxRedirects git config setting. The JGit http.* git config settings are currently all global; JGit has no support yet for URI-specific settings "http.<pattern>.name". Adding support for that is well beyond the scope of this change. Like git-core, we log every redirect attempt (LOG.info) so that users may know about the redirection having occurred. Extends the test framework to configure an AppServer with HTTPS support so that we can test cloning via HTTPS and redirections involving HTTPS. [1] https://git-scm.com/docs/git-config [2] https://kernel.googlesource.com/pub/scm/git/git/+/6628eb41db5189c0cdfdced6d8697e7c813c5f0f [3] https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html CQ: 13987 Bug: 465167 Change-Id: I86518cb76842f7d326b51f8715e3bbf8ada89859 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
9 years ago
Add support to follow HTTP redirects git-core follows HTTP redirects so JGit should also provide this. Implement config setting http.followRedirects with possible values "false" (= never), "true" (= always), and "initial" (only on GET, but not on POST).[1] We must do our own redirect handling and cannot rely on the support that the underlying real connection may offer. At least the JDK's HttpURLConnection has two features that get in the way: * it does not allow cross-protocol redirects and thus fails on http->https redirects (for instance, on Github). * it translates a redirect after a POST to a GET unless the system property "http.strictPostRedirect" is set to true. We don't want to manipulate that system setting nor require it. Additionally, git has its own rules about what redirects it accepts;[2] for instance, it does not allow a redirect that adds query arguments. We handle response codes 301, 302, 303, and 307 as per RFC 2616.[3] On POST we do not handle 303, and we follow redirects only if http.followRedirects == true. Redirects are followed only a certain number of times. There are two ways to control that limit: * by default, the limit is given by the http.maxRedirects system property that is also used by the JDK. If the system property is not set, the default is 5. (This is much lower than the JDK default of 20, but I don't see the value of following so many redirects.) * this can be overwritten by a http.maxRedirects git config setting. The JGit http.* git config settings are currently all global; JGit has no support yet for URI-specific settings "http.<pattern>.name". Adding support for that is well beyond the scope of this change. Like git-core, we log every redirect attempt (LOG.info) so that users may know about the redirection having occurred. Extends the test framework to configure an AppServer with HTTPS support so that we can test cloning via HTTPS and redirections involving HTTPS. [1] https://git-scm.com/docs/git-config [2] https://kernel.googlesource.com/pub/scm/git/git/+/6628eb41db5189c0cdfdced6d8697e7c813c5f0f [3] https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html CQ: 13987 Bug: 465167 Change-Id: I86518cb76842f7d326b51f8715e3bbf8ada89859 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
9 years ago
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445
  1. /*
  2. * Copyright (C) 2010, 2017 Google Inc. and others
  3. *
  4. * This program and the accompanying materials are made available under the
  5. * terms of the Eclipse Distribution License v. 1.0 which is available at
  6. * https://www.eclipse.org/org/documents/edl-v10.php.
  7. *
  8. * SPDX-License-Identifier: BSD-3-Clause
  9. */
  10. package org.eclipse.jgit.junit.http;
  11. import static org.junit.Assert.assertFalse;
  12. import static org.junit.Assert.assertTrue;
  13. import java.io.File;
  14. import java.io.IOException;
  15. import java.net.InetAddress;
  16. import java.net.URI;
  17. import java.net.URISyntaxException;
  18. import java.net.UnknownHostException;
  19. import java.nio.file.Files;
  20. import java.util.ArrayList;
  21. import java.util.List;
  22. import java.util.Locale;
  23. import java.util.concurrent.ConcurrentHashMap;
  24. import java.util.concurrent.ConcurrentMap;
  25. import org.eclipse.jetty.http.HttpVersion;
  26. import org.eclipse.jetty.security.AbstractLoginService;
  27. import org.eclipse.jetty.security.Authenticator;
  28. import org.eclipse.jetty.security.ConstraintMapping;
  29. import org.eclipse.jetty.security.ConstraintSecurityHandler;
  30. import org.eclipse.jetty.security.authentication.BasicAuthenticator;
  31. import org.eclipse.jetty.server.Connector;
  32. import org.eclipse.jetty.server.HttpConfiguration;
  33. import org.eclipse.jetty.server.HttpConnectionFactory;
  34. import org.eclipse.jetty.server.Server;
  35. import org.eclipse.jetty.server.ServerConnector;
  36. import org.eclipse.jetty.server.SslConnectionFactory;
  37. import org.eclipse.jetty.server.handler.ContextHandlerCollection;
  38. import org.eclipse.jetty.servlet.ServletContextHandler;
  39. import org.eclipse.jetty.util.security.Constraint;
  40. import org.eclipse.jetty.util.security.Password;
  41. import org.eclipse.jetty.util.ssl.SslContextFactory;
  42. import org.eclipse.jgit.transport.URIish;
  43. /**
  44. * Tiny web application server for unit testing.
  45. * <p>
  46. * Tests should start the server in their {@code setUp()} method and stop the
  47. * server in their {@code tearDown()} method. Only while started the server's
  48. * URL and/or port number can be obtained.
  49. */
  50. public class AppServer {
  51. /** Realm name for the secure access areas. */
  52. public static final String realm = "Secure Area";
  53. /** Username for secured access areas. */
  54. public static final String username = "agitter";
  55. /** Password for {@link #username} in secured access areas. */
  56. public static final String password = "letmein";
  57. /** SSL keystore password; must have at least 6 characters. */
  58. private static final String keyPassword = "mykeys";
  59. /** Role for authentication. */
  60. private static final String authRole = "can-access";
  61. static {
  62. // Install a logger that throws warning messages.
  63. //
  64. final String prop = "org.eclipse.jetty.util.log.class";
  65. System.setProperty(prop, RecordingLogger.class.getName());
  66. }
  67. private final Server server;
  68. private final HttpConfiguration config;
  69. private final ServerConnector connector;
  70. private final HttpConfiguration secureConfig;
  71. private final ServerConnector secureConnector;
  72. private final ContextHandlerCollection contexts;
  73. private final TestRequestLog log;
  74. private List<File> filesToDelete = new ArrayList<>();
  75. /**
  76. * Constructor for <code>AppServer</code>.
  77. */
  78. public AppServer() {
  79. this(0, -1);
  80. }
  81. /**
  82. * Constructor for <code>AppServer</code>.
  83. *
  84. * @param port
  85. * the http port number; may be zero to allocate a port
  86. * dynamically
  87. * @since 4.2
  88. */
  89. public AppServer(int port) {
  90. this(port, -1);
  91. }
  92. /**
  93. * Constructor for <code>AppServer</code>.
  94. *
  95. * @param port
  96. * for http, may be zero to allocate a port dynamically
  97. * @param sslPort
  98. * for https,may be zero to allocate a port dynamically. If
  99. * negative, the server will be set up without https support.
  100. * @since 4.9
  101. */
  102. public AppServer(int port, int sslPort) {
  103. server = new Server();
  104. config = new HttpConfiguration();
  105. config.setSecureScheme("https");
  106. config.setSecurePort(0);
  107. config.setOutputBufferSize(32768);
  108. connector = new ServerConnector(server,
  109. new HttpConnectionFactory(config));
  110. connector.setPort(port);
  111. String ip;
  112. String hostName;
  113. try {
  114. final InetAddress me = InetAddress.getByName("localhost");
  115. ip = me.getHostAddress();
  116. connector.setHost(ip);
  117. hostName = InetAddress.getLocalHost().getCanonicalHostName();
  118. } catch (UnknownHostException e) {
  119. throw new RuntimeException("Cannot find localhost", e);
  120. }
  121. if (sslPort >= 0) {
  122. SslContextFactory sslContextFactory = createTestSslContextFactory(
  123. hostName);
  124. secureConfig = new HttpConfiguration(config);
  125. secureConnector = new ServerConnector(server,
  126. new SslConnectionFactory(sslContextFactory,
  127. HttpVersion.HTTP_1_1.asString()),
  128. new HttpConnectionFactory(secureConfig));
  129. secureConnector.setPort(sslPort);
  130. secureConnector.setHost(ip);
  131. } else {
  132. secureConfig = null;
  133. secureConnector = null;
  134. }
  135. contexts = new ContextHandlerCollection();
  136. log = new TestRequestLog();
  137. log.setHandler(contexts);
  138. if (secureConnector == null) {
  139. server.setConnectors(new Connector[] { connector });
  140. } else {
  141. server.setConnectors(
  142. new Connector[] { connector, secureConnector });
  143. }
  144. server.setHandler(log);
  145. }
  146. private SslContextFactory createTestSslContextFactory(String hostName) {
  147. SslContextFactory.Client factory = new SslContextFactory.Client(true);
  148. String dName = "CN=,OU=,O=,ST=,L=,C=";
  149. try {
  150. File tmpDir = Files.createTempDirectory("jks").toFile();
  151. tmpDir.deleteOnExit();
  152. makePrivate(tmpDir);
  153. File keyStore = new File(tmpDir, "keystore.jks");
  154. Runtime.getRuntime().exec(
  155. new String[] {
  156. "keytool", //
  157. "-keystore", keyStore.getAbsolutePath(), //
  158. "-storepass", keyPassword,
  159. "-alias", hostName, //
  160. "-genkeypair", //
  161. "-keyalg", "RSA", //
  162. "-keypass", keyPassword, //
  163. "-dname", dName, //
  164. "-validity", "2" //
  165. }).waitFor();
  166. keyStore.deleteOnExit();
  167. makePrivate(keyStore);
  168. filesToDelete.add(keyStore);
  169. filesToDelete.add(tmpDir);
  170. factory.setKeyStorePath(keyStore.getAbsolutePath());
  171. factory.setKeyStorePassword(keyPassword);
  172. factory.setKeyManagerPassword(keyPassword);
  173. factory.setTrustStorePath(keyStore.getAbsolutePath());
  174. factory.setTrustStorePassword(keyPassword);
  175. } catch (InterruptedException | IOException e) {
  176. throw new RuntimeException("Cannot create ssl key/certificate", e);
  177. }
  178. return factory;
  179. }
  180. private void makePrivate(File file) {
  181. file.setReadable(false);
  182. file.setWritable(false);
  183. file.setExecutable(false);
  184. file.setReadable(true, true);
  185. file.setWritable(true, true);
  186. if (file.isDirectory()) {
  187. file.setExecutable(true, true);
  188. }
  189. }
  190. /**
  191. * Create a new servlet context within the server.
  192. * <p>
  193. * This method should be invoked before the server is started, once for each
  194. * context the caller wants to register.
  195. *
  196. * @param path
  197. * path of the context; use "/" for the root context if binding
  198. * to the root is desired.
  199. * @return the context to add servlets into.
  200. */
  201. public ServletContextHandler addContext(String path) {
  202. assertNotYetSetUp();
  203. if ("".equals(path))
  204. path = "/";
  205. ServletContextHandler ctx = new ServletContextHandler();
  206. ctx.setContextPath(path);
  207. contexts.addHandler(ctx);
  208. return ctx;
  209. }
  210. /**
  211. * Configure basic authentication.
  212. *
  213. * @param ctx
  214. * @param methods
  215. * @return servlet context handler
  216. */
  217. public ServletContextHandler authBasic(ServletContextHandler ctx,
  218. String... methods) {
  219. assertNotYetSetUp();
  220. auth(ctx, new BasicAuthenticator(), methods);
  221. return ctx;
  222. }
  223. static class TestMappedLoginService extends AbstractLoginService {
  224. private String role;
  225. protected final ConcurrentMap<String, UserPrincipal> users = new ConcurrentHashMap<>();
  226. TestMappedLoginService(String role) {
  227. this.role = role;
  228. }
  229. @Override
  230. protected void doStart() throws Exception {
  231. UserPrincipal p = new UserPrincipal(username,
  232. new Password(password));
  233. users.put(username, p);
  234. super.doStart();
  235. }
  236. @Override
  237. protected String[] loadRoleInfo(UserPrincipal user) {
  238. if (users.get(user.getName()) == null) {
  239. return null;
  240. }
  241. return new String[] { role };
  242. }
  243. @Override
  244. protected UserPrincipal loadUserInfo(String user) {
  245. return users.get(user);
  246. }
  247. }
  248. private ConstraintMapping createConstraintMapping() {
  249. ConstraintMapping cm = new ConstraintMapping();
  250. cm.setConstraint(new Constraint());
  251. cm.getConstraint().setAuthenticate(true);
  252. cm.getConstraint().setDataConstraint(Constraint.DC_NONE);
  253. cm.getConstraint().setRoles(new String[] { authRole });
  254. cm.setPathSpec("/*");
  255. return cm;
  256. }
  257. private void auth(ServletContextHandler ctx, Authenticator authType,
  258. String... methods) {
  259. AbstractLoginService users = new TestMappedLoginService(authRole);
  260. List<ConstraintMapping> mappings = new ArrayList<>();
  261. if (methods == null || methods.length == 0) {
  262. mappings.add(createConstraintMapping());
  263. } else {
  264. for (String method : methods) {
  265. ConstraintMapping cm = createConstraintMapping();
  266. cm.setMethod(method.toUpperCase(Locale.ROOT));
  267. mappings.add(cm);
  268. }
  269. }
  270. ConstraintSecurityHandler sec = new ConstraintSecurityHandler();
  271. sec.setRealmName(realm);
  272. sec.setAuthenticator(authType);
  273. sec.setLoginService(users);
  274. sec.setConstraintMappings(
  275. mappings.toArray(new ConstraintMapping[0]));
  276. sec.setHandler(ctx);
  277. contexts.removeHandler(ctx);
  278. contexts.addHandler(sec);
  279. }
  280. /**
  281. * Start the server on a random local port.
  282. *
  283. * @throws Exception
  284. * the server cannot be started, testing is not possible.
  285. */
  286. public void setUp() throws Exception {
  287. RecordingLogger.clear();
  288. log.clear();
  289. server.start();
  290. config.setSecurePort(getSecurePort());
  291. if (secureConfig != null) {
  292. secureConfig.setSecurePort(getSecurePort());
  293. }
  294. }
  295. /**
  296. * Shutdown the server.
  297. *
  298. * @throws Exception
  299. * the server refuses to halt, or wasn't running.
  300. */
  301. public void tearDown() throws Exception {
  302. RecordingLogger.clear();
  303. log.clear();
  304. server.stop();
  305. for (File f : filesToDelete) {
  306. f.delete();
  307. }
  308. filesToDelete.clear();
  309. }
  310. /**
  311. * Get the URI to reference this server.
  312. * <p>
  313. * The returned URI includes the proper host name and port number, but does
  314. * not contain a path.
  315. *
  316. * @return URI to reference this server's root context.
  317. */
  318. public URI getURI() {
  319. assertAlreadySetUp();
  320. String host = connector.getHost();
  321. if (host.contains(":") && !host.startsWith("["))
  322. host = "[" + host + "]";
  323. final String uri = "http://" + host + ":" + getPort();
  324. try {
  325. return new URI(uri);
  326. } catch (URISyntaxException e) {
  327. throw new RuntimeException("Unexpected URI error on " + uri, e);
  328. }
  329. }
  330. /**
  331. * Get port.
  332. *
  333. * @return the local port number the server is listening on.
  334. */
  335. public int getPort() {
  336. assertAlreadySetUp();
  337. return connector.getLocalPort();
  338. }
  339. /**
  340. * Get secure port.
  341. *
  342. * @return the HTTPS port or -1 if not configured.
  343. */
  344. public int getSecurePort() {
  345. assertAlreadySetUp();
  346. return secureConnector != null ? secureConnector.getLocalPort() : -1;
  347. }
  348. /**
  349. * Get requests.
  350. *
  351. * @return all requests since the server was started.
  352. */
  353. public List<AccessEvent> getRequests() {
  354. return new ArrayList<>(log.getEvents());
  355. }
  356. /**
  357. * Get requests.
  358. *
  359. * @param base
  360. * base URI used to access the server.
  361. * @param path
  362. * the path to locate requests for, relative to {@code base}.
  363. * @return all requests which match the given path.
  364. */
  365. public List<AccessEvent> getRequests(URIish base, String path) {
  366. return getRequests(HttpTestCase.join(base, path));
  367. }
  368. /**
  369. * Get requests.
  370. *
  371. * @param path
  372. * the path to locate requests for.
  373. * @return all requests which match the given path.
  374. */
  375. public List<AccessEvent> getRequests(String path) {
  376. ArrayList<AccessEvent> r = new ArrayList<>();
  377. for (AccessEvent event : log.getEvents()) {
  378. if (event.getPath().equals(path)) {
  379. r.add(event);
  380. }
  381. }
  382. return r;
  383. }
  384. private void assertNotYetSetUp() {
  385. assertFalse("server is not running", server.isRunning());
  386. }
  387. private void assertAlreadySetUp() {
  388. assertTrue("server is running", server.isRunning());
  389. }
  390. }