You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

TransportLocal.java 11KB

Client-side protocol V2 support for fetching Make all transports request protocol V2 when fetching. Depending on the transport, set the GIT_PROTOCOL environment variable (file and ssh), pass the Git-Protocol header (http), or set the hidden "\0version=2\0" (git anon). We'll fall back to V0 if the server doesn't reply with a version 2 answer. A user can control which protocol the client requests via the git config protocol.version; if not set, JGit requests protocol V2 for fetching. Pushing always uses protocol V0 still. In the API, there is only a new Transport.openFetch() version that takes a collection of RefSpecs plus additional patterns to construct the Ref prefixes for the "ls-refs" command in protocol V2. If none are given, the server will still advertise all refs, even in protocol V2. BasePackConnection.readAdvertisedRefs() handles falling back to protocol V0. It newly returns true if V0 was used and the advertised refs were read, and false if V2 is used and an explicit "ls-refs" is needed. (This can't be done transparently inside readAdvertisedRefs() because a "stateless RPC" transport like TransportHttp may need to open a new connection for writing.) BasePackFetchConnection implements the changes needed for the protocol V2 "fetch" command (stateless protocol, simplified ACK handling, delimiters, section headers). In TransportHttp, change readSmartHeaders() to also recognize the "version 2" packet line as a valid smart server indication. Adapt tests, and run all the HTTP tests not only with both HTTP connection factories (JDK and Apache HttpClient) but also with both protocol V0 and V2. The SSH tests are much slower and much more focused on the SSH protocol and SSH key handling. Factor out two very simple cloning and pulling tests and make those run with protocol V2. Bug: 553083 Change-Id: I357c7f5daa7efb2872f1c64ee6f6d54229031ae1 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
3 years ago
Client-side protocol V2 support for fetching Make all transports request protocol V2 when fetching. Depending on the transport, set the GIT_PROTOCOL environment variable (file and ssh), pass the Git-Protocol header (http), or set the hidden "\0version=2\0" (git anon). We'll fall back to V0 if the server doesn't reply with a version 2 answer. A user can control which protocol the client requests via the git config protocol.version; if not set, JGit requests protocol V2 for fetching. Pushing always uses protocol V0 still. In the API, there is only a new Transport.openFetch() version that takes a collection of RefSpecs plus additional patterns to construct the Ref prefixes for the "ls-refs" command in protocol V2. If none are given, the server will still advertise all refs, even in protocol V2. BasePackConnection.readAdvertisedRefs() handles falling back to protocol V0. It newly returns true if V0 was used and the advertised refs were read, and false if V2 is used and an explicit "ls-refs" is needed. (This can't be done transparently inside readAdvertisedRefs() because a "stateless RPC" transport like TransportHttp may need to open a new connection for writing.) BasePackFetchConnection implements the changes needed for the protocol V2 "fetch" command (stateless protocol, simplified ACK handling, delimiters, section headers). In TransportHttp, change readSmartHeaders() to also recognize the "version 2" packet line as a valid smart server indication. Adapt tests, and run all the HTTP tests not only with both HTTP connection factories (JDK and Apache HttpClient) but also with both protocol V0 and V2. The SSH tests are much slower and much more focused on the SSH protocol and SSH key handling. Factor out two very simple cloning and pulling tests and make those run with protocol V2. Bug: 553083 Change-Id: I357c7f5daa7efb2872f1c64ee6f6d54229031ae1 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
3 years ago
Client-side protocol V2 support for fetching Make all transports request protocol V2 when fetching. Depending on the transport, set the GIT_PROTOCOL environment variable (file and ssh), pass the Git-Protocol header (http), or set the hidden "\0version=2\0" (git anon). We'll fall back to V0 if the server doesn't reply with a version 2 answer. A user can control which protocol the client requests via the git config protocol.version; if not set, JGit requests protocol V2 for fetching. Pushing always uses protocol V0 still. In the API, there is only a new Transport.openFetch() version that takes a collection of RefSpecs plus additional patterns to construct the Ref prefixes for the "ls-refs" command in protocol V2. If none are given, the server will still advertise all refs, even in protocol V2. BasePackConnection.readAdvertisedRefs() handles falling back to protocol V0. It newly returns true if V0 was used and the advertised refs were read, and false if V2 is used and an explicit "ls-refs" is needed. (This can't be done transparently inside readAdvertisedRefs() because a "stateless RPC" transport like TransportHttp may need to open a new connection for writing.) BasePackFetchConnection implements the changes needed for the protocol V2 "fetch" command (stateless protocol, simplified ACK handling, delimiters, section headers). In TransportHttp, change readSmartHeaders() to also recognize the "version 2" packet line as a valid smart server indication. Adapt tests, and run all the HTTP tests not only with both HTTP connection factories (JDK and Apache HttpClient) but also with both protocol V0 and V2. The SSH tests are much slower and much more focused on the SSH protocol and SSH key handling. Factor out two very simple cloning and pulling tests and make those run with protocol V2. Bug: 553083 Change-Id: I357c7f5daa7efb2872f1c64ee6f6d54229031ae1 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
3 years ago
Capture non-progress side band #2 messages and put in result Any messages received on side band #2 that aren't scraped as a progress message into our ProgressMonitor are now forwarded to a buffer which is later included into the OperationResult object. Application callers can use this buffer to present the additional messages from the remote peer after the push or fetch operation has concluded. The smart push connections using the native send-pack/receive-pack protocol now request side-band-64k capability if it is available and forward any messages received through that channel onto this message buffer. This makes hook messages available over smart HTTP, or even over SSH. The SSH transport was modified to redirect the remote command's stderr stream into the message buffer, interleaved with any data received over side band #2. Due to buffering between these two different channels in the SSH channel mux itself the order of any writes between the two cannot be ensured, but it tries to stay close. The local fork transport was also modified to redirect the local receive-pack's stderr into the message buffer, rather than going to the invoking JVM's System.err. This gives applications a chance to log the local error messages, rather than needing to redirect their JVM's stderr before startup. To keep things simple, the application has to wait for the entire operation to complete before it can see the messages. This may be a downside if the user is trying to debug a remote hook that is blocking indefinitely, the user would need to abort the connection before they can inspect the message buffer in any sort of UI built on top of JGit. Change-Id: Ibc215f4569e63071da5b7e5c6674ce924ae39e11 Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
14 years ago
Client-side protocol V2 support for fetching Make all transports request protocol V2 when fetching. Depending on the transport, set the GIT_PROTOCOL environment variable (file and ssh), pass the Git-Protocol header (http), or set the hidden "\0version=2\0" (git anon). We'll fall back to V0 if the server doesn't reply with a version 2 answer. A user can control which protocol the client requests via the git config protocol.version; if not set, JGit requests protocol V2 for fetching. Pushing always uses protocol V0 still. In the API, there is only a new Transport.openFetch() version that takes a collection of RefSpecs plus additional patterns to construct the Ref prefixes for the "ls-refs" command in protocol V2. If none are given, the server will still advertise all refs, even in protocol V2. BasePackConnection.readAdvertisedRefs() handles falling back to protocol V0. It newly returns true if V0 was used and the advertised refs were read, and false if V2 is used and an explicit "ls-refs" is needed. (This can't be done transparently inside readAdvertisedRefs() because a "stateless RPC" transport like TransportHttp may need to open a new connection for writing.) BasePackFetchConnection implements the changes needed for the protocol V2 "fetch" command (stateless protocol, simplified ACK handling, delimiters, section headers). In TransportHttp, change readSmartHeaders() to also recognize the "version 2" packet line as a valid smart server indication. Adapt tests, and run all the HTTP tests not only with both HTTP connection factories (JDK and Apache HttpClient) but also with both protocol V0 and V2. The SSH tests are much slower and much more focused on the SSH protocol and SSH key handling. Factor out two very simple cloning and pulling tests and make those run with protocol V2. Bug: 553083 Change-Id: I357c7f5daa7efb2872f1c64ee6f6d54229031ae1 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
3 years ago
Client-side protocol V2 support for fetching Make all transports request protocol V2 when fetching. Depending on the transport, set the GIT_PROTOCOL environment variable (file and ssh), pass the Git-Protocol header (http), or set the hidden "\0version=2\0" (git anon). We'll fall back to V0 if the server doesn't reply with a version 2 answer. A user can control which protocol the client requests via the git config protocol.version; if not set, JGit requests protocol V2 for fetching. Pushing always uses protocol V0 still. In the API, there is only a new Transport.openFetch() version that takes a collection of RefSpecs plus additional patterns to construct the Ref prefixes for the "ls-refs" command in protocol V2. If none are given, the server will still advertise all refs, even in protocol V2. BasePackConnection.readAdvertisedRefs() handles falling back to protocol V0. It newly returns true if V0 was used and the advertised refs were read, and false if V2 is used and an explicit "ls-refs" is needed. (This can't be done transparently inside readAdvertisedRefs() because a "stateless RPC" transport like TransportHttp may need to open a new connection for writing.) BasePackFetchConnection implements the changes needed for the protocol V2 "fetch" command (stateless protocol, simplified ACK handling, delimiters, section headers). In TransportHttp, change readSmartHeaders() to also recognize the "version 2" packet line as a valid smart server indication. Adapt tests, and run all the HTTP tests not only with both HTTP connection factories (JDK and Apache HttpClient) but also with both protocol V0 and V2. The SSH tests are much slower and much more focused on the SSH protocol and SSH key handling. Factor out two very simple cloning and pulling tests and make those run with protocol V2. Bug: 553083 Change-Id: I357c7f5daa7efb2872f1c64ee6f6d54229031ae1 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
3 years ago
Client-side protocol V2 support for fetching Make all transports request protocol V2 when fetching. Depending on the transport, set the GIT_PROTOCOL environment variable (file and ssh), pass the Git-Protocol header (http), or set the hidden "\0version=2\0" (git anon). We'll fall back to V0 if the server doesn't reply with a version 2 answer. A user can control which protocol the client requests via the git config protocol.version; if not set, JGit requests protocol V2 for fetching. Pushing always uses protocol V0 still. In the API, there is only a new Transport.openFetch() version that takes a collection of RefSpecs plus additional patterns to construct the Ref prefixes for the "ls-refs" command in protocol V2. If none are given, the server will still advertise all refs, even in protocol V2. BasePackConnection.readAdvertisedRefs() handles falling back to protocol V0. It newly returns true if V0 was used and the advertised refs were read, and false if V2 is used and an explicit "ls-refs" is needed. (This can't be done transparently inside readAdvertisedRefs() because a "stateless RPC" transport like TransportHttp may need to open a new connection for writing.) BasePackFetchConnection implements the changes needed for the protocol V2 "fetch" command (stateless protocol, simplified ACK handling, delimiters, section headers). In TransportHttp, change readSmartHeaders() to also recognize the "version 2" packet line as a valid smart server indication. Adapt tests, and run all the HTTP tests not only with both HTTP connection factories (JDK and Apache HttpClient) but also with both protocol V0 and V2. The SSH tests are much slower and much more focused on the SSH protocol and SSH key handling. Factor out two very simple cloning and pulling tests and make those run with protocol V2. Bug: 553083 Change-Id: I357c7f5daa7efb2872f1c64ee6f6d54229031ae1 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
3 years ago
Client-side protocol V2 support for fetching Make all transports request protocol V2 when fetching. Depending on the transport, set the GIT_PROTOCOL environment variable (file and ssh), pass the Git-Protocol header (http), or set the hidden "\0version=2\0" (git anon). We'll fall back to V0 if the server doesn't reply with a version 2 answer. A user can control which protocol the client requests via the git config protocol.version; if not set, JGit requests protocol V2 for fetching. Pushing always uses protocol V0 still. In the API, there is only a new Transport.openFetch() version that takes a collection of RefSpecs plus additional patterns to construct the Ref prefixes for the "ls-refs" command in protocol V2. If none are given, the server will still advertise all refs, even in protocol V2. BasePackConnection.readAdvertisedRefs() handles falling back to protocol V0. It newly returns true if V0 was used and the advertised refs were read, and false if V2 is used and an explicit "ls-refs" is needed. (This can't be done transparently inside readAdvertisedRefs() because a "stateless RPC" transport like TransportHttp may need to open a new connection for writing.) BasePackFetchConnection implements the changes needed for the protocol V2 "fetch" command (stateless protocol, simplified ACK handling, delimiters, section headers). In TransportHttp, change readSmartHeaders() to also recognize the "version 2" packet line as a valid smart server indication. Adapt tests, and run all the HTTP tests not only with both HTTP connection factories (JDK and Apache HttpClient) but also with both protocol V0 and V2. The SSH tests are much slower and much more focused on the SSH protocol and SSH key handling. Factor out two very simple cloning and pulling tests and make those run with protocol V2. Bug: 553083 Change-Id: I357c7f5daa7efb2872f1c64ee6f6d54229031ae1 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
3 years ago
Capture non-progress side band #2 messages and put in result Any messages received on side band #2 that aren't scraped as a progress message into our ProgressMonitor are now forwarded to a buffer which is later included into the OperationResult object. Application callers can use this buffer to present the additional messages from the remote peer after the push or fetch operation has concluded. The smart push connections using the native send-pack/receive-pack protocol now request side-band-64k capability if it is available and forward any messages received through that channel onto this message buffer. This makes hook messages available over smart HTTP, or even over SSH. The SSH transport was modified to redirect the remote command's stderr stream into the message buffer, interleaved with any data received over side band #2. Due to buffering between these two different channels in the SSH channel mux itself the order of any writes between the two cannot be ensured, but it tries to stay close. The local fork transport was also modified to redirect the local receive-pack's stderr into the message buffer, rather than going to the invoking JVM's System.err. This gives applications a chance to log the local error messages, rather than needing to redirect their JVM's stderr before startup. To keep things simple, the application has to wait for the entire operation to complete before it can see the messages. This may be a downside if the user is trying to debug a remote hook that is blocking indefinitely, the user would need to abort the connection before they can inspect the message buffer in any sort of UI built on top of JGit. Change-Id: Ibc215f4569e63071da5b7e5c6674ce924ae39e11 Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
14 years ago
Client-side protocol V2 support for fetching Make all transports request protocol V2 when fetching. Depending on the transport, set the GIT_PROTOCOL environment variable (file and ssh), pass the Git-Protocol header (http), or set the hidden "\0version=2\0" (git anon). We'll fall back to V0 if the server doesn't reply with a version 2 answer. A user can control which protocol the client requests via the git config protocol.version; if not set, JGit requests protocol V2 for fetching. Pushing always uses protocol V0 still. In the API, there is only a new Transport.openFetch() version that takes a collection of RefSpecs plus additional patterns to construct the Ref prefixes for the "ls-refs" command in protocol V2. If none are given, the server will still advertise all refs, even in protocol V2. BasePackConnection.readAdvertisedRefs() handles falling back to protocol V0. It newly returns true if V0 was used and the advertised refs were read, and false if V2 is used and an explicit "ls-refs" is needed. (This can't be done transparently inside readAdvertisedRefs() because a "stateless RPC" transport like TransportHttp may need to open a new connection for writing.) BasePackFetchConnection implements the changes needed for the protocol V2 "fetch" command (stateless protocol, simplified ACK handling, delimiters, section headers). In TransportHttp, change readSmartHeaders() to also recognize the "version 2" packet line as a valid smart server indication. Adapt tests, and run all the HTTP tests not only with both HTTP connection factories (JDK and Apache HttpClient) but also with both protocol V0 and V2. The SSH tests are much slower and much more focused on the SSH protocol and SSH key handling. Factor out two very simple cloning and pulling tests and make those run with protocol V2. Bug: 553083 Change-Id: I357c7f5daa7efb2872f1c64ee6f6d54229031ae1 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
3 years ago
Capture non-progress side band #2 messages and put in result Any messages received on side band #2 that aren't scraped as a progress message into our ProgressMonitor are now forwarded to a buffer which is later included into the OperationResult object. Application callers can use this buffer to present the additional messages from the remote peer after the push or fetch operation has concluded. The smart push connections using the native send-pack/receive-pack protocol now request side-band-64k capability if it is available and forward any messages received through that channel onto this message buffer. This makes hook messages available over smart HTTP, or even over SSH. The SSH transport was modified to redirect the remote command's stderr stream into the message buffer, interleaved with any data received over side band #2. Due to buffering between these two different channels in the SSH channel mux itself the order of any writes between the two cannot be ensured, but it tries to stay close. The local fork transport was also modified to redirect the local receive-pack's stderr into the message buffer, rather than going to the invoking JVM's System.err. This gives applications a chance to log the local error messages, rather than needing to redirect their JVM's stderr before startup. To keep things simple, the application has to wait for the entire operation to complete before it can see the messages. This may be a downside if the user is trying to debug a remote hook that is blocking indefinitely, the user would need to abort the connection before they can inspect the message buffer in any sort of UI built on top of JGit. Change-Id: Ibc215f4569e63071da5b7e5c6674ce924ae39e11 Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
14 years ago
Client-side protocol V2 support for fetching Make all transports request protocol V2 when fetching. Depending on the transport, set the GIT_PROTOCOL environment variable (file and ssh), pass the Git-Protocol header (http), or set the hidden "\0version=2\0" (git anon). We'll fall back to V0 if the server doesn't reply with a version 2 answer. A user can control which protocol the client requests via the git config protocol.version; if not set, JGit requests protocol V2 for fetching. Pushing always uses protocol V0 still. In the API, there is only a new Transport.openFetch() version that takes a collection of RefSpecs plus additional patterns to construct the Ref prefixes for the "ls-refs" command in protocol V2. If none are given, the server will still advertise all refs, even in protocol V2. BasePackConnection.readAdvertisedRefs() handles falling back to protocol V0. It newly returns true if V0 was used and the advertised refs were read, and false if V2 is used and an explicit "ls-refs" is needed. (This can't be done transparently inside readAdvertisedRefs() because a "stateless RPC" transport like TransportHttp may need to open a new connection for writing.) BasePackFetchConnection implements the changes needed for the protocol V2 "fetch" command (stateless protocol, simplified ACK handling, delimiters, section headers). In TransportHttp, change readSmartHeaders() to also recognize the "version 2" packet line as a valid smart server indication. Adapt tests, and run all the HTTP tests not only with both HTTP connection factories (JDK and Apache HttpClient) but also with both protocol V0 and V2. The SSH tests are much slower and much more focused on the SSH protocol and SSH key handling. Factor out two very simple cloning and pulling tests and make those run with protocol V2. Bug: 553083 Change-Id: I357c7f5daa7efb2872f1c64ee6f6d54229031ae1 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
3 years ago
Capture non-progress side band #2 messages and put in result Any messages received on side band #2 that aren't scraped as a progress message into our ProgressMonitor are now forwarded to a buffer which is later included into the OperationResult object. Application callers can use this buffer to present the additional messages from the remote peer after the push or fetch operation has concluded. The smart push connections using the native send-pack/receive-pack protocol now request side-band-64k capability if it is available and forward any messages received through that channel onto this message buffer. This makes hook messages available over smart HTTP, or even over SSH. The SSH transport was modified to redirect the remote command's stderr stream into the message buffer, interleaved with any data received over side band #2. Due to buffering between these two different channels in the SSH channel mux itself the order of any writes between the two cannot be ensured, but it tries to stay close. The local fork transport was also modified to redirect the local receive-pack's stderr into the message buffer, rather than going to the invoking JVM's System.err. This gives applications a chance to log the local error messages, rather than needing to redirect their JVM's stderr before startup. To keep things simple, the application has to wait for the entire operation to complete before it can see the messages. This may be a downside if the user is trying to debug a remote hook that is blocking indefinitely, the user would need to abort the connection before they can inspect the message buffer in any sort of UI built on top of JGit. Change-Id: Ibc215f4569e63071da5b7e5c6674ce924ae39e11 Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
14 years ago
Client-side protocol V2 support for fetching Make all transports request protocol V2 when fetching. Depending on the transport, set the GIT_PROTOCOL environment variable (file and ssh), pass the Git-Protocol header (http), or set the hidden "\0version=2\0" (git anon). We'll fall back to V0 if the server doesn't reply with a version 2 answer. A user can control which protocol the client requests via the git config protocol.version; if not set, JGit requests protocol V2 for fetching. Pushing always uses protocol V0 still. In the API, there is only a new Transport.openFetch() version that takes a collection of RefSpecs plus additional patterns to construct the Ref prefixes for the "ls-refs" command in protocol V2. If none are given, the server will still advertise all refs, even in protocol V2. BasePackConnection.readAdvertisedRefs() handles falling back to protocol V0. It newly returns true if V0 was used and the advertised refs were read, and false if V2 is used and an explicit "ls-refs" is needed. (This can't be done transparently inside readAdvertisedRefs() because a "stateless RPC" transport like TransportHttp may need to open a new connection for writing.) BasePackFetchConnection implements the changes needed for the protocol V2 "fetch" command (stateless protocol, simplified ACK handling, delimiters, section headers). In TransportHttp, change readSmartHeaders() to also recognize the "version 2" packet line as a valid smart server indication. Adapt tests, and run all the HTTP tests not only with both HTTP connection factories (JDK and Apache HttpClient) but also with both protocol V0 and V2. The SSH tests are much slower and much more focused on the SSH protocol and SSH key handling. Factor out two very simple cloning and pulling tests and make those run with protocol V2. Bug: 553083 Change-Id: I357c7f5daa7efb2872f1c64ee6f6d54229031ae1 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
3 years ago
Capture non-progress side band #2 messages and put in result Any messages received on side band #2 that aren't scraped as a progress message into our ProgressMonitor are now forwarded to a buffer which is later included into the OperationResult object. Application callers can use this buffer to present the additional messages from the remote peer after the push or fetch operation has concluded. The smart push connections using the native send-pack/receive-pack protocol now request side-band-64k capability if it is available and forward any messages received through that channel onto this message buffer. This makes hook messages available over smart HTTP, or even over SSH. The SSH transport was modified to redirect the remote command's stderr stream into the message buffer, interleaved with any data received over side band #2. Due to buffering between these two different channels in the SSH channel mux itself the order of any writes between the two cannot be ensured, but it tries to stay close. The local fork transport was also modified to redirect the local receive-pack's stderr into the message buffer, rather than going to the invoking JVM's System.err. This gives applications a chance to log the local error messages, rather than needing to redirect their JVM's stderr before startup. To keep things simple, the application has to wait for the entire operation to complete before it can see the messages. This may be a downside if the user is trying to debug a remote hook that is blocking indefinitely, the user would need to abort the connection before they can inspect the message buffer in any sort of UI built on top of JGit. Change-Id: Ibc215f4569e63071da5b7e5c6674ce924ae39e11 Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
14 years ago
Capture non-progress side band #2 messages and put in result Any messages received on side band #2 that aren't scraped as a progress message into our ProgressMonitor are now forwarded to a buffer which is later included into the OperationResult object. Application callers can use this buffer to present the additional messages from the remote peer after the push or fetch operation has concluded. The smart push connections using the native send-pack/receive-pack protocol now request side-band-64k capability if it is available and forward any messages received through that channel onto this message buffer. This makes hook messages available over smart HTTP, or even over SSH. The SSH transport was modified to redirect the remote command's stderr stream into the message buffer, interleaved with any data received over side band #2. Due to buffering between these two different channels in the SSH channel mux itself the order of any writes between the two cannot be ensured, but it tries to stay close. The local fork transport was also modified to redirect the local receive-pack's stderr into the message buffer, rather than going to the invoking JVM's System.err. This gives applications a chance to log the local error messages, rather than needing to redirect their JVM's stderr before startup. To keep things simple, the application has to wait for the entire operation to complete before it can see the messages. This may be a downside if the user is trying to debug a remote hook that is blocking indefinitely, the user would need to abort the connection before they can inspect the message buffer in any sort of UI built on top of JGit. Change-Id: Ibc215f4569e63071da5b7e5c6674ce924ae39e11 Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
14 years ago
Capture non-progress side band #2 messages and put in result Any messages received on side band #2 that aren't scraped as a progress message into our ProgressMonitor are now forwarded to a buffer which is later included into the OperationResult object. Application callers can use this buffer to present the additional messages from the remote peer after the push or fetch operation has concluded. The smart push connections using the native send-pack/receive-pack protocol now request side-band-64k capability if it is available and forward any messages received through that channel onto this message buffer. This makes hook messages available over smart HTTP, or even over SSH. The SSH transport was modified to redirect the remote command's stderr stream into the message buffer, interleaved with any data received over side band #2. Due to buffering between these two different channels in the SSH channel mux itself the order of any writes between the two cannot be ensured, but it tries to stay close. The local fork transport was also modified to redirect the local receive-pack's stderr into the message buffer, rather than going to the invoking JVM's System.err. This gives applications a chance to log the local error messages, rather than needing to redirect their JVM's stderr before startup. To keep things simple, the application has to wait for the entire operation to complete before it can see the messages. This may be a downside if the user is trying to debug a remote hook that is blocking indefinitely, the user would need to abort the connection before they can inspect the message buffer in any sort of UI built on top of JGit. Change-Id: Ibc215f4569e63071da5b7e5c6674ce924ae39e11 Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
14 years ago
Capture non-progress side band #2 messages and put in result Any messages received on side band #2 that aren't scraped as a progress message into our ProgressMonitor are now forwarded to a buffer which is later included into the OperationResult object. Application callers can use this buffer to present the additional messages from the remote peer after the push or fetch operation has concluded. The smart push connections using the native send-pack/receive-pack protocol now request side-band-64k capability if it is available and forward any messages received through that channel onto this message buffer. This makes hook messages available over smart HTTP, or even over SSH. The SSH transport was modified to redirect the remote command's stderr stream into the message buffer, interleaved with any data received over side band #2. Due to buffering between these two different channels in the SSH channel mux itself the order of any writes between the two cannot be ensured, but it tries to stay close. The local fork transport was also modified to redirect the local receive-pack's stderr into the message buffer, rather than going to the invoking JVM's System.err. This gives applications a chance to log the local error messages, rather than needing to redirect their JVM's stderr before startup. To keep things simple, the application has to wait for the entire operation to complete before it can see the messages. This may be a downside if the user is trying to debug a remote hook that is blocking indefinitely, the user would need to abort the connection before they can inspect the message buffer in any sort of UI built on top of JGit. Change-Id: Ibc215f4569e63071da5b7e5c6674ce924ae39e11 Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
14 years ago
Capture non-progress side band #2 messages and put in result Any messages received on side band #2 that aren't scraped as a progress message into our ProgressMonitor are now forwarded to a buffer which is later included into the OperationResult object. Application callers can use this buffer to present the additional messages from the remote peer after the push or fetch operation has concluded. The smart push connections using the native send-pack/receive-pack protocol now request side-band-64k capability if it is available and forward any messages received through that channel onto this message buffer. This makes hook messages available over smart HTTP, or even over SSH. The SSH transport was modified to redirect the remote command's stderr stream into the message buffer, interleaved with any data received over side band #2. Due to buffering between these two different channels in the SSH channel mux itself the order of any writes between the two cannot be ensured, but it tries to stay close. The local fork transport was also modified to redirect the local receive-pack's stderr into the message buffer, rather than going to the invoking JVM's System.err. This gives applications a chance to log the local error messages, rather than needing to redirect their JVM's stderr before startup. To keep things simple, the application has to wait for the entire operation to complete before it can see the messages. This may be a downside if the user is trying to debug a remote hook that is blocking indefinitely, the user would need to abort the connection before they can inspect the message buffer in any sort of UI built on top of JGit. Change-Id: Ibc215f4569e63071da5b7e5c6674ce924ae39e11 Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
14 years ago
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361
  1. /*
  2. * Copyright (C) 2007, Dave Watson <dwatson@mimvista.com>
  3. * Copyright (C) 2008, 2010 Google Inc.
  4. * Copyright (C) 2008, Marek Zawirski <marek.zawirski@gmail.com>
  5. * Copyright (C) 2008, Robin Rosenberg <robin.rosenberg@dewire.com>
  6. * Copyright (C) 2008, 2020 Shawn O. Pearce <spearce@spearce.org> and others
  7. *
  8. * This program and the accompanying materials are made available under the
  9. * terms of the Eclipse Distribution License v. 1.0 which is available at
  10. * https://www.eclipse.org/org/documents/edl-v10.php.
  11. *
  12. * SPDX-License-Identifier: BSD-3-Clause
  13. */
  14. package org.eclipse.jgit.transport;
  15. import java.io.BufferedInputStream;
  16. import java.io.BufferedOutputStream;
  17. import java.io.File;
  18. import java.io.IOException;
  19. import java.io.InputStream;
  20. import java.io.OutputStream;
  21. import java.util.Collection;
  22. import java.util.Collections;
  23. import java.util.Map;
  24. import java.util.Set;
  25. import org.eclipse.jgit.errors.NoRemoteRepositoryException;
  26. import org.eclipse.jgit.errors.NotSupportedException;
  27. import org.eclipse.jgit.errors.TransportException;
  28. import org.eclipse.jgit.internal.JGitText;
  29. import org.eclipse.jgit.lib.Repository;
  30. import org.eclipse.jgit.lib.RepositoryBuilder;
  31. import org.eclipse.jgit.lib.RepositoryCache;
  32. import org.eclipse.jgit.transport.resolver.ReceivePackFactory;
  33. import org.eclipse.jgit.transport.resolver.UploadPackFactory;
  34. import org.eclipse.jgit.util.FS;
  35. import org.eclipse.jgit.util.io.MessageWriter;
  36. import org.eclipse.jgit.util.io.StreamCopyThread;
  37. /**
  38. * Transport to access a local directory as though it were a remote peer.
  39. * <p>
  40. * This transport is suitable for use on the local system, where the caller has
  41. * direct read or write access to the "remote" repository.
  42. * <p>
  43. * By default this transport works by spawning a helper thread within the same
  44. * JVM, and processes the data transfer using a shared memory buffer between the
  45. * calling thread and the helper thread. This is a pure-Java implementation
  46. * which does not require forking an external process.
  47. * <p>
  48. * However, during {@link #openFetch()}, if the Transport has configured
  49. * {@link Transport#getOptionUploadPack()} to be anything other than
  50. * <code>"git-upload-pack"</code> or <code>"git upload-pack"</code>, this
  51. * implementation will fork and execute the external process, using an operating
  52. * system pipe to transfer data.
  53. * <p>
  54. * Similarly, during {@link #openPush()}, if the Transport has configured
  55. * {@link Transport#getOptionReceivePack()} to be anything other than
  56. * <code>"git-receive-pack"</code> or <code>"git receive-pack"</code>, this
  57. * implementation will fork and execute the external process, using an operating
  58. * system pipe to transfer data.
  59. */
  60. class TransportLocal extends Transport implements PackTransport {
  61. static final TransportProtocol PROTO_LOCAL = new TransportProtocol() {
  62. @Override
  63. public String getName() {
  64. return JGitText.get().transportProtoLocal;
  65. }
  66. @Override
  67. public Set<String> getSchemes() {
  68. return Collections.singleton("file"); //$NON-NLS-1$
  69. }
  70. @Override
  71. public boolean canHandle(URIish uri, Repository local, String remoteName) {
  72. if (uri.getPath() == null
  73. || uri.getPort() > 0
  74. || uri.getUser() != null
  75. || uri.getPass() != null
  76. || uri.getHost() != null
  77. || (uri.getScheme() != null && !getSchemes().contains(uri.getScheme())))
  78. return false;
  79. return true;
  80. }
  81. @Override
  82. public Transport open(URIish uri, Repository local, String remoteName)
  83. throws NoRemoteRepositoryException {
  84. File localPath = local.isBare() ? local.getDirectory() : local.getWorkTree();
  85. File path = local.getFS().resolve(localPath, uri.getPath());
  86. // If the reference is to a local file, C Git behavior says
  87. // assume this is a bundle, since repositories are directories.
  88. if (path.isFile())
  89. return new TransportBundleFile(local, uri, path);
  90. File gitDir = RepositoryCache.FileKey.resolve(path, local.getFS());
  91. if (gitDir == null)
  92. throw new NoRemoteRepositoryException(uri, JGitText.get().notFound);
  93. return new TransportLocal(local, uri, gitDir);
  94. }
  95. @Override
  96. public Transport open(URIish uri) throws NotSupportedException,
  97. TransportException {
  98. File path = FS.DETECTED.resolve(new File("."), uri.getPath()); //$NON-NLS-1$
  99. // If the reference is to a local file, C Git behavior says
  100. // assume this is a bundle, since repositories are directories.
  101. if (path.isFile())
  102. return new TransportBundleFile(uri, path);
  103. File gitDir = RepositoryCache.FileKey.resolve(path, FS.DETECTED);
  104. if (gitDir == null)
  105. throw new NoRemoteRepositoryException(uri,
  106. JGitText.get().notFound);
  107. return new TransportLocal(uri, gitDir);
  108. }
  109. };
  110. private final File remoteGitDir;
  111. TransportLocal(Repository local, URIish uri, File gitDir) {
  112. super(local, uri);
  113. remoteGitDir = gitDir;
  114. }
  115. TransportLocal(URIish uri, File gitDir) {
  116. super(uri);
  117. remoteGitDir = gitDir;
  118. }
  119. UploadPack createUploadPack(Repository dst) {
  120. return new UploadPack(dst);
  121. }
  122. ReceivePack createReceivePack(Repository dst) {
  123. return new ReceivePack(dst);
  124. }
  125. private Repository openRepo() throws TransportException {
  126. try {
  127. return new RepositoryBuilder()
  128. .setFS(local != null ? local.getFS() : FS.DETECTED)
  129. .setGitDir(remoteGitDir).build();
  130. } catch (IOException err) {
  131. TransportException te = new TransportException(uri,
  132. JGitText.get().notAGitDirectory);
  133. te.initCause(err);
  134. throw te;
  135. }
  136. }
  137. /** {@inheritDoc} */
  138. @Override
  139. public FetchConnection openFetch() throws TransportException {
  140. return openFetch(Collections.emptyList());
  141. }
  142. @Override
  143. public FetchConnection openFetch(Collection<RefSpec> refSpecs,
  144. String... additionalPatterns) throws TransportException {
  145. final String up = getOptionUploadPack();
  146. if (!"git-upload-pack".equals(up) //$NON-NLS-1$
  147. && !"git upload-pack".equals(up)) {//$NON-NLS-1$
  148. return new ForkLocalFetchConnection(refSpecs, additionalPatterns);
  149. }
  150. UploadPackFactory<Void> upf = (Void req,
  151. Repository db) -> createUploadPack(db);
  152. return new InternalFetchConnection<>(this, upf, null, openRepo());
  153. }
  154. /** {@inheritDoc} */
  155. @Override
  156. public PushConnection openPush() throws TransportException {
  157. final String rp = getOptionReceivePack();
  158. if (!"git-receive-pack".equals(rp) //$NON-NLS-1$
  159. && !"git receive-pack".equals(rp)) //$NON-NLS-1$
  160. return new ForkLocalPushConnection();
  161. ReceivePackFactory<Void> rpf = (Void req,
  162. Repository db) -> createReceivePack(db);
  163. return new InternalPushConnection<>(this, rpf, null, openRepo());
  164. }
  165. /** {@inheritDoc} */
  166. @Override
  167. public void close() {
  168. // Resources must be established per-connection.
  169. }
  170. /**
  171. * Spawn process
  172. *
  173. * @param cmd
  174. * command
  175. * @return a {@link java.lang.Process} object.
  176. * @throws org.eclipse.jgit.errors.TransportException
  177. * if any.
  178. */
  179. protected Process spawn(String cmd)
  180. throws TransportException {
  181. return spawn(cmd, null);
  182. }
  183. /**
  184. * Spawn process
  185. *
  186. * @param cmd
  187. * command
  188. * @param protocolVersion
  189. * to use
  190. * @return a {@link java.lang.Process} object.
  191. * @throws org.eclipse.jgit.errors.TransportException
  192. * if any.
  193. */
  194. private Process spawn(String cmd,
  195. TransferConfig.ProtocolVersion protocolVersion)
  196. throws TransportException {
  197. try {
  198. String[] args = { "." }; //$NON-NLS-1$
  199. ProcessBuilder proc = local.getFS().runInShell(cmd, args);
  200. proc.directory(remoteGitDir);
  201. // Remove the same variables CGit does.
  202. Map<String, String> env = proc.environment();
  203. env.remove("GIT_ALTERNATE_OBJECT_DIRECTORIES"); //$NON-NLS-1$
  204. env.remove("GIT_CONFIG"); //$NON-NLS-1$
  205. env.remove("GIT_CONFIG_PARAMETERS"); //$NON-NLS-1$
  206. env.remove("GIT_DIR"); //$NON-NLS-1$
  207. env.remove("GIT_WORK_TREE"); //$NON-NLS-1$
  208. env.remove("GIT_GRAFT_FILE"); //$NON-NLS-1$
  209. env.remove("GIT_INDEX_FILE"); //$NON-NLS-1$
  210. env.remove("GIT_NO_REPLACE_OBJECTS"); //$NON-NLS-1$
  211. if (TransferConfig.ProtocolVersion.V2.equals(protocolVersion)) {
  212. env.put(GitProtocolConstants.PROTOCOL_ENVIRONMENT_VARIABLE,
  213. GitProtocolConstants.VERSION_2_REQUEST);
  214. }
  215. return proc.start();
  216. } catch (IOException err) {
  217. throw new TransportException(uri, err.getMessage(), err);
  218. }
  219. }
  220. class ForkLocalFetchConnection extends BasePackFetchConnection {
  221. private Process uploadPack;
  222. private Thread errorReaderThread;
  223. ForkLocalFetchConnection() throws TransportException {
  224. this(Collections.emptyList());
  225. }
  226. ForkLocalFetchConnection(Collection<RefSpec> refSpecs,
  227. String... additionalPatterns) throws TransportException {
  228. super(TransportLocal.this);
  229. final MessageWriter msg = new MessageWriter();
  230. setMessageWriter(msg);
  231. TransferConfig.ProtocolVersion gitProtocol = protocol;
  232. if (gitProtocol == null) {
  233. gitProtocol = TransferConfig.ProtocolVersion.V2;
  234. }
  235. uploadPack = spawn(getOptionUploadPack(), gitProtocol);
  236. final InputStream upErr = uploadPack.getErrorStream();
  237. errorReaderThread = new StreamCopyThread(upErr, msg.getRawStream());
  238. errorReaderThread.start();
  239. InputStream upIn = uploadPack.getInputStream();
  240. OutputStream upOut = uploadPack.getOutputStream();
  241. upIn = new BufferedInputStream(upIn);
  242. upOut = new BufferedOutputStream(upOut);
  243. init(upIn, upOut);
  244. if (!readAdvertisedRefs()) {
  245. lsRefs(refSpecs, additionalPatterns);
  246. }
  247. }
  248. @Override
  249. public void close() {
  250. super.close();
  251. if (uploadPack != null) {
  252. try {
  253. uploadPack.waitFor();
  254. } catch (InterruptedException ie) {
  255. // Stop waiting and return anyway.
  256. } finally {
  257. uploadPack = null;
  258. }
  259. }
  260. if (errorReaderThread != null) {
  261. try {
  262. errorReaderThread.join();
  263. } catch (InterruptedException e) {
  264. // Stop waiting and return anyway.
  265. } finally {
  266. errorReaderThread = null;
  267. }
  268. }
  269. }
  270. }
  271. class ForkLocalPushConnection extends BasePackPushConnection {
  272. private Process receivePack;
  273. private Thread errorReaderThread;
  274. ForkLocalPushConnection() throws TransportException {
  275. super(TransportLocal.this);
  276. final MessageWriter msg = new MessageWriter();
  277. setMessageWriter(msg);
  278. receivePack = spawn(getOptionReceivePack());
  279. final InputStream rpErr = receivePack.getErrorStream();
  280. errorReaderThread = new StreamCopyThread(rpErr, msg.getRawStream());
  281. errorReaderThread.start();
  282. InputStream rpIn = receivePack.getInputStream();
  283. OutputStream rpOut = receivePack.getOutputStream();
  284. rpIn = new BufferedInputStream(rpIn);
  285. rpOut = new BufferedOutputStream(rpOut);
  286. init(rpIn, rpOut);
  287. readAdvertisedRefs();
  288. }
  289. @Override
  290. public void close() {
  291. super.close();
  292. if (receivePack != null) {
  293. try {
  294. receivePack.waitFor();
  295. } catch (InterruptedException ie) {
  296. // Stop waiting and return anyway.
  297. } finally {
  298. receivePack = null;
  299. }
  300. }
  301. if (errorReaderThread != null) {
  302. try {
  303. errorReaderThread.join();
  304. } catch (InterruptedException e) {
  305. // Stop waiting and return anyway.
  306. } finally {
  307. errorReaderThread = null;
  308. }
  309. }
  310. }
  311. }
  312. }