summaryrefslogtreecommitdiffstats
path: root/org.eclipse.jgit.lfs.server/src
Commit message (Collapse)AuthorAgeFilesLines
* LFS: Add remote download to SmudgeFilterMarkus Duft2018-02-161-1/+18
| | | | | | | | | | | | | | | Transfer data in chunks of 8k Transferring data byte per byte is slow, running checkout with CleanFilter on a 2.9MB file takes 20 seconds. Using a buffer of 8k shrinks this time to 70ms. Also register the filter commands in a way that the native GIT LFS can be used alongside with JGit. Implements auto-discovery of LFS server URL when cloning from a Gerrit LFS server. Change-Id: I452a5aa177dcb346d92af08b27c2e35200f246fd Also-by: Christian Halstrick <christian.halstrick@sap.com> Signed-off-by: Markus Duft <markus.duft@ssi-schaefer.com>
* Revert usage of TYPE_USE in Nullable and NonNull annotationsDavid Pursehouse2018-01-191-1/+1
| | | | | | | | | | | | Using TYPE_USE causes compilation errors in Eclipse Neon.3 (JDT 3.12.3) and Eclipse Oxygen.2 (JDT 3.13.2). This reverts commit 8e217517e2c515032dd0d661535d2133cd80123a. This reverts commit 55eba8d0f55464ca84d676828f67a6fe14b2454d. Reported-by: Thomas Wolf <thomas.wolf@paranor.ch> Change-Id: I96869f80dd11ee238911706581b224bca4fb12cd Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* Nullable: Switch to TYPE_USEDavid Pursehouse2018-01-181-1/+1
| | | | | | | | | | | | | | | Since JGit now requires Java 8, we can switch to TYPE_USE instead of explicitly specifying the target type. Some of the existing uses of Nullable need to be reworked slightly as described in [1] to prevent the compilation error: scoping construct cannot be annotated with type-use annotation [1] https://stackoverflow.com/a/21385939/381622 Change-Id: Idba48f67a09353b5237685996ce828c8ca398168 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* Fix remaining javadoc errors raised by doclintMatthias Sohn2017-12-211-4/+3
| | | | | | | For now ignore doclint "missing" warnings. Change-Id: I0e5af7a757f4d92ffeeb113f30576a35414d6781 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* Fix javadoc in org.eclipse.jgit.lfs.serverMatthias Sohn2017-12-1711-66/+98
| | | | Change-Id: I0c52536ebeeb8e71d95c6420f6db7055ff082dfa Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* Use constants from StandardCharsets instead of hard-coded stringsDavid Pursehouse2017-12-071-4/+4
| | | | | | | | | | | | | | | | Instead of hard-coding the charset strings "US-ASCII", "UTF-8", and "ISO-8859-1", use the corresponding constants from StandardCharsets. UnsupportedEncodingException is not thrown when the StandardCharset constants are used, so remove the now redundant handling. Because the encoding names are no longer hard-coded strings, also remove redundant $NON-NLS warning suppressions. Also replace existing usages of the constants with static imports. Change-Id: I0a4510d3d992db5e277f009a41434276f95bda4e Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* Move repeated LFS Gson handling to common classDavid Pursehouse2017-12-073-46/+123
| | | | | | | | | | LfsProtocolServlet and FileLfsServlet both implement the same setup of the Gson object. Factor it out to a common class and reuse it. Change-Id: I5696404fad140cbff1b712ebb04a7e8bba60e4b4 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* FileLfsServlet: Do not pretty print json responseDavid Pursehouse2017-12-071-5/+4
| | | | | | | | | | | | | | | | Pretty printing the response is useful for human readers, but most (if not all) of the time, the response will be read by programs. Remove it to avoid the additional overhead of the formatting and extra bytes in the response. Adjust the test accordingly. Note that LfsProtocolServlet already doesn't use pretty printing, so this change makes FileLfsServlet's behavior consistent. In fact, both classes now have duplicate Gson handling; this will be cleaned up in a separate change. Change-Id: I113a23403f9222f16e2c0ddf39461398b721d064 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* Add missing newlines at ends of Java filesDavid Pursehouse2017-07-251-1/+1
| | | | | Change-Id: Iead36f53d57ead0eb3edd3f9efb63b6630c9c20c Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* LfsProtocolServlet: Pass HTTP Authorization header to getLargeFileRepositoryDavid Pursehouse2017-03-081-2/+49
| | | | | | | | This allows implementations to reject operations that do not include proper authentication. Change-Id: If301476d8fb56a0899e424be3789c7576097d185 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* Enable and fix warnings about redundant specification of type argumentsDavid Pursehouse2017-02-202-8/+8
| | | | | | | | | | Since the introduction of generic type parameter inference in Java 7, it's not necessary to explicitly specify the type of generic parameters. Enable the warning in Eclipse, and fix all occurrences. Change-Id: I9158caf1beca5e4980b6240ac401f3868520aad0 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* Don't rely on default locale when using toUpperCase() and toLowerCase()Matthias Sohn2017-01-281-2/+4
| | | | | | | | | | | | | | | | Otherwise these methods may produce unexpected results if used for strings that are intended to be interpreted locale independently. Examples are programming language identifiers, protocol keys, and HTML tags. For instance, "TITLE".toLowerCase() in a Turkish locale returns "t\u0131tle", where '\u0131' is the LATIN SMALL LETTER DOTLESS I character. See https://docs.oracle.com/javase/8/docs/api/java/lang/String.html#toLowerCase-- http://blog.thetaphi.de/2012/07/default-locales-default-charsets-and.html Bug: 511238 Change-Id: Id8d8f37d84d62239c918b81f8d883ed798d87656 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* Make ObjectDownloadListener publicDavid Pursehouse2017-01-191-2/+4
| | | | | | | The same was already done for ObjectUploadListener in I5b0fb1220. Change-Id: Ie8a79f715fe69bed9331962fb478f7e35acf8680 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* LfsProtocolServlet#LfsRequest: Add operation type helper methodsDavid Pursehouse2017-01-141-0/+27
| | | | | | | | Server implementations may need to check what type of operation is being performed. Add helper methods to make it a bit simpler. Change-Id: Ia33ed6699ebcb9000ef514ce79bcddbebf94e1c5 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* Expose LFS operation strings as public constantsDavid Pursehouse2017-01-131-7/+6
| | | | | | | | So that they can be used either by third party LFS server implementations or from within other parts of JGit's internal LFS implementation. Change-Id: I58da6230247204588e017c010ce5b14e4615448d Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* LfsProtocolServlet: Improve error on getLargeFileRepository failureDavid Pursehouse2017-01-141-1/+10
| | | | | | | | Externalize the error message and make it more specific. Also emit an error log. Change-Id: Ie7b1c90c54673bfb8e133fb0aa19a117d4ca6587 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* Add support for refusing LFS request due to invalid authorizationDavid Pursehouse2017-01-131-0/+4
| | | | | | | | | | | | | | | | | | | Add a new exception type that server implementations can throw when a client attempts to make an unauthorized LFS operation, which will result in HTTP 401 Unauthorized being returned to the client. An example of this is a Gerrit server that rejects a request to perform an LFS operation on a ref that is not visible to the caller. As defined in the LFS spec [1] the request may include authentication, and per RFC 2616 [2], "401 response indicates that authorization has been refused for those credentials". [1] https://github.com/git-lfs/git-lfs/blob/master/docs/api/batch.md [2] https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html Change-Id: I2aa22e2144df5fb7972df0e3285b77b08ecc63f2 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* Expose getObjectToTransfer method of FileLfsServletJacek Centkowski2016-12-061-1/+14
| | | | | | | | | Providing own implementation to doGet/doPut methods is troublesome when this method is private. Change-Id: I098cdc5cb90410eaaebc56c88c2d9e168584dd6d Signed-off-by: Jacek Centkowski <geminica.programs@gmail.com> Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* Merge branch 'stable-4.5'David Pursehouse2016-12-061-2/+3
|\ | | | | | | | | | | | | | | * stable-4.5: Use the same variable to check and extract LFS object id Change-Id: I314bd4373f40843c68853b3999f60d85e08628d9 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
| * Use the same variable to check and extract LFS object idJacek Centkowski2016-11-241-2/+3
| | | | | | | | | | | | | | | | It is easier to maintain when the same variable is used for both check and extraction of LFS object id. Change-Id: I5406f9bc4a085aa164c4565a9667ad2925105190 Signed-off-by: Jacek Centkowski <geminica.programs@gmail.com>
* | Add missing @since tags for new APIMatthias Sohn2016-11-141-0/+1
| | | | | | | | Change-Id: Iaf83f66637d6a13e4a6d096ba8529553af7e30ed Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* | Organize importsDavid Pursehouse2016-11-141-1/+1
| | | | | | | | | | Change-Id: I7c545d06b1bced678c020fab9af1382bc4416b6e Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* | FileLfsServlet: Make sendError protectedDavid Pursehouse2016-11-131-1/+13
| | | | | | | | | | Change-Id: Id89b3cd0da86a535946feb15254737bbf8d855d4 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* | Make ObjectUploadListener publicDavid Pursehouse2016-11-131-3/+8
| | | | | | | | | | Change-Id: I5b0fb1220bc0e5c660f1b5373abc5868d985c60b Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* | Move AtomicObjectOutputStream to lfs/internalChristian Halstrick2016-09-204-122/+2
|/ | | | | | | | | The class AtomicObjectOutputStream should be available to all lfs related classes, not only to the server side. Move the class from org.eclipse.jgit.lfs.server.fs to org.eclipse.jgit.lfs.internal to achieve that. Change-Id: I028e1c9ec7c21f316340b21d558b9a6b77e2060d
* Improve JavaDoc for LfsProtocolServlet.getLargeFileRepositoryMatthias Sohn2016-09-011-0/+23
| | | | | | Guide implementors which exception to throw in case of errors. Change-Id: I74fb76cdf6b7cdef513f3fe8c144572e869cc533 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* Add specific exception for LFS unavailableDavid Pursehouse2016-08-301-10/+11
| | | | | | | | | | | | Instead of returning null, LfsProtocolServlet#getLargeFileRepository should throw LfsUnavailable. If null is returned, throw a generic LfsException. Handle LfsException as an internal server error and return HTTP 500. Change-Id: I33e2a19fcc0fde8aaf0f703860c8fa8ce2de2db5 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* LfsProtocolServlet: Add support for insufficient storage errorDavid Pursehouse2016-08-241-0/+4
| | | | | | | | | | | | | Since [1], the git-lfs specification allows the server to return HTTP 507 if there is insufficient storage for the uploaded object(s). Add a new exception class, which implementations may throw from the getRepository() method, causing HTTP 507 to be returned to the client. [1] https://github.com/github/git-lfs/pull/1473 Change-Id: If5bc0a35fcf870d4216af6ca2f7c8924689ef9c5 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* LfsProtocolServlet: Add support for rate limit and bandwidth limit errorsDavid Pursehouse2016-08-201-0/+10
| | | | | | | | | | | | | | | | | | | | | | The git-lfs specification [1] describes the following optional status codes that may be returned: 429 - The user has hit a rate limit with the server. Though the API does not specify any rate limits, implementors are encouraged to set some for availability reasons. 509 - Returned if the bandwidth limit for the user or repository has been exceeded. The API does not specify any bandwidth limit, but implementors may track usage. Add two new exception classes to represent these cases. Implementations may throw these from #getLargeFileRepository(), causing the corresponding HTTP status codes to be returned to the client. [1] https://github.com/github/git-lfs/blob/master/docs/api/v1/http-v1-batch.md Change-Id: I7b93f3cf90f7344c90b1587e07927fdeb167097e Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* LfsProtocolServlet: Always include message in error responseDavid Pursehouse2016-08-201-1/+2
| | | | | | | | | If the message is not sent, the client shows: Unable to parse HTTP response for POST http://admin@localhost:8080/test-project/info/lfs/objects/batch Change-Id: I8b72d1aded2bcd41b7389676e2373034625a1379 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* LfsProtocolServlet: Always set the Content-Type header on responseDavid Pursehouse2016-08-161-2/+3
| | | | | | | | | | | If the Content-Type is not set on error responses, the git-lfs client does not read the body which contains the error message, and instead just displays a generic error message. Also set the charset on the Content-Type header. Change-Id: I88e6f07f20b622a670e7c5063145dffb8b630aee Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* LfsProtocolServlet: Don't set pretty printing on GsonDavid Pursehouse2016-08-161-5/+4
| | | | | | | | Pretty printing is only used for outputting json content, which is interpreted by the client and does not need to be pretty printed. Change-Id: I48e0280241b6b0f5706300ae0f4c9bc461a89110 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* LfsProtocolServlet: Reuse existing Writer when sending error responseDavid Pursehouse2016-08-091-27/+19
| | | | | | | | Trying to open a new writer on the response causes an illegal state exception and the response is not sent. Change-Id: Ic718d23cfb3e74f5691cc2aea7283003af7df207 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* Add missing @since tags for new APIMatthias Sohn2016-08-092-1/+7
| | | | Change-Id: I8db29a0313fbc476152cef47f2eaa76954f1e280 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* LfsProtocolServlet: Allow access to objects in requestDavid Pursehouse2016-08-052-1/+28
| | | | | | | | | | | Classes implementing the LFS servlet should be able to inspect the objects given in the request. Add a getObjects method. Make the LfsObject class public, and add accessor methods. Change-Id: I27961679f620eb3a89dc8521aadd4ea2f936c60e Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* LfsProtocolServlet: Allow getLargeFileRepository to raise exceptionsDavid Pursehouse2016-08-051-6/+49
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | According to the specification [1] the server may return the following HTTP error responses: - 403: The user has read, but not write access. - 404: The repository does not exist for the user. - 422: Validation error with one or more of the objects in the request. In the current implementation, however, getLargeFileRepository can only return null to indicate an error. This results in the error code: - 503: Service Unavailable being returned to the client regardless of what the actual reason was. Add exception classes to cover these cases, derived from a common base exception, and change the specification of getLargeFileRepository to throw the base exception. In LfsProtocolServlet#post, handle the new exceptions and send back the appropriate HTTP responses as mentioned above. The specification also mentions several other optional response codes (406, 429, 501, and 509) but these are not implemented in this commit. It should be trivial to implement them in follow-up commits. [1] https://github.com/github/git-lfs/blob/master/docs/api/v1/http-v1-batch.md#response-errors Change-Id: I91be6165bcaf856d0cefc533882330962e2fc9b2 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* Fix typo in email address in copyright headersDavid Pursehouse2016-07-282-2/+2
| | | | | Change-Id: Ib7b73d7d37574682bb58f969822c842e4e579233 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* LfsProtocolServlet: Pass request and path to getLargeFileRepositoryDavid Pursehouse2016-07-271-12/+29
| | | | | | | | | | | | Passing the request and path to the method will allow implementations to have more control over determination of the backend, for example: - return different backends for different requests - accept or refuse requests based on request characteristics - etc Change-Id: I1ec6ec54c91a5f0601b620ed18846eb4a3f46783 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* FileLfsServlet: Include error message in response bodyDavid Pursehouse2016-07-271-3/+29
| | | | | | | | | | According to the specification [1], the error response body must include the error message in json format. [1] https://github.com/github/git-lfs/blob/master/docs/api/v1/http-v1-batch.md#response-errors Change-Id: I79e7a841d230fdedefa53b9c6d2d477e81e1f9e6 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* FileLfsServlet: Return HTTP 422 instead of 400David Pursehouse2016-07-271-2/+2
| | | | | | | | | | | According to the specification [1], the error response status code should be 422 when there is a validation error with one or more of the objects in the request [1] https://github.com/github/git-lfs/blob/master/docs/api/v1/http-v1-batch.md#response-errors Change-Id: Id03fe00a2109b896d9a154228a14a33bce5accc3 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* Update links to LFS API documentationDavid Pursehouse2016-07-262-2/+2
| | | | | | | | The location of the API v1 documentation has changed. Update the links accordingly. Change-Id: If0148a0b573c474bbe157fcb7e6674c0055fe8b4 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* [findBugs] Prevent potential NPE in FileLfsRepository.getOutputStream()Matthias Sohn2016-05-301-1/+4
| | | | Change-Id: I6f91997e8a976e5f4ace91c082d8b9adf1451adc Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* [findBugs] Fix calculation of host header in SignerV4Matthias Sohn2016-05-191-3/+3
| | | | | | | | We ignored the returned concatenation of host name and port number. Fix this and use a StringBuilder to avoid creation of unnecessary String objects. Change-Id: I61fac639d4a4c95412eb41a0f9131d0c38aca794 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* Make the FileLfsRepository thread safeSaša Živkov2016-03-222-25/+12
| | | | | | | | | | | | | | The FileLfsRepository.out member could have been accessed from multiple threads which would corrupt the content. Don't store the AtomicObjectOutputStream in the FileLfsRepository.out but move it to the ObjectUploadListener which is instantiated per-request. Add a parallel upload test. Change-Id: I62298630e99c46b500d376843ffcde934436215b Signed-off-by: Saša Živkov <sasa.zivkov@sap.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* Support Amazon S3 based storage for LFSMatthias Sohn2016-02-174-0/+786
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add a storage implementation storing large objects in Amazon S3. The AmazonS3Repository pre-signs download and upload requests. AWS access and secret key are expected to be in the $HOME/.aws/credentials file in the following format: [default] accessKey = ... secretKey = ... Use AWS version 4 request signing [1] because it is more secure and supported by all regions. The version 3 signing is not supported in newer regions. In follow up changes we should: - implement getVerifyAction() and do actual verification. Subclasses of S3Repository can implement caching for object meta data (size) in order to avoid extra roundtrips to S3. Verification should ensure that meta data store and content of S3 storage are in sync - HEAD request used in S3Repository.getSize() seems to always return Content-length 0 in contrast to the documentation [2]. So getSize() does detect if the object exists in S3 or not but in case the object exists it always returns size 0 [1] http://docs.aws.amazon.com/general/latest/gr/signature-version-4.html [2] https://forums.aws.amazon.com/thread.jspa?threadID=223616 Change-Id: Ic47f094928a259e5264c92b3aacf6d90210907a8 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Signed-off-by: Sasa Zivkov <sasa.zivkov@sap.com>
* Support LFS protocol and a file system based LFS storageMatthias Sohn2016-02-0411-0/+1378
Implement LfsProtocolServlet handling the "Git LFS v1 Batch API" protocol [1]. Add a simple file system based LFS content store and the debug-lfs-store command to simplify testing. Introduce a LargeFileRepository interface to enable additional storage implementation while reusing the same protocol implementation. At the client side we have to configure the lfs.url, specify that we use the batch API and we don't use authentication: [lfs] url = http://host:port/lfs batch = true [lfs "http://host:port/lfs"] access = none the git-lfs client appends the "objects/batch" to the lfs.url. Hard code an Authorization header in the FileLfsRepository.getAction because then git-lfs client will skip asking for credentials. It will just forward the Authorization header from the response to the download/upload request. The FileLfsServlet supports file content storage for "Large File Storage" (LFS) server as defined by the Github LFS API [2]. - upload and download of large files is probably network bound hence use an asynchronous servlet for good scalability - simple object storage in file system with 2 level fan-out - use LockFile to protect writing large objects against multiple concurrent uploads of the same object - to prevent corrupt uploads the uploaded file is rejected if its hash doesn't match id given in URL The debug-lfs-store command is used to run the LfsProtocolServlet and, optionally, the FileLfsServlet which makes it easier to setup a local test server. [1] https://github.com/github/git-lfs/blob/master/docs/api/http-v1-batch.md [2] https://github.com/github/git-lfs/tree/master/docs/api Bug: 472961 Change-Id: I7378da5575159d2195138d799704880c5c82d5f3 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Signed-off-by: Sasa Zivkov <sasa.zivkov@sap.com>
generated by cgit v1.2.3 (git 2.39.1) at 2025-01-15 15:09:54 +0000