Set missingOverrideAnnotation=warning in Eclipse compiler preferences
which enables the warning:

  The method <method> of type <type> should be tagged with @Override
  since it actually overrides a superclass method

Justification for this warning is described in:

  http://stackoverflow.com/a/94411/381622

Enabling this causes in excess of 1000 warnings across the entire
code-base. They are very easy to fix automatically with Eclipse's
"Quick Fix" tool.

Fix all of them except 2 which cause compilation failure when the
project is built with mvn; add TODO comments on those for further
investigation.

Change-Id: I5772061041fd361fe93137fd8b0ad356e748a29c
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>

Otherwise these methods may produce unexpected results if used for
strings that are intended to be interpreted locale independently.
Examples are programming language identifiers, protocol keys, and HTML
tags. For instance, "TITLE".toLowerCase() in a Turkish locale returns
"t\u0131tle", where '\u0131' is the LATIN SMALL LETTER DOTLESS I
character.

See
https://docs.oracle.com/javase/8/docs/api/java/lang/String.html#toLowerCase--
http://blog.thetaphi.de/2012/07/default-locales-default-charsets-and.html

Bug: 511238
Change-Id: Id8d8f37d84d62239c918b81f8d883ed798d87656
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

So that they can be used either by third party LFS server implementations
or from within other parts of JGit's internal LFS implementation.

Change-Id: I58da6230247204588e017c010ce5b14e4615448d
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>

Externalize the error message and make it more specific. Also emit
an error log.

Change-Id: Ie7b1c90c54673bfb8e133fb0aa19a117d4ca6587
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

Add a new exception type that server implementations can throw when a
client attempts to make an unauthorized LFS operation, which will result
in HTTP 401 Unauthorized being returned to the client.

An example of this is a Gerrit server that rejects a request to perform
an LFS operation on a ref that is not visible to the caller.

As defined in the LFS spec [1] the request may include authentication,
and per RFC 2616 [2], "401 response indicates that authorization has been
refused for those credentials".

[1] https://github.com/git-lfs/git-lfs/blob/master/docs/api/batch.md
[2] https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html

Change-Id: I2aa22e2144df5fb7972df0e3285b77b08ecc63f2
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>

Add new variation of TreeFilter in order to detect LFS pointer files in
the repository.

Additionally, update LfsPointer to support the legacy version URL [1] as
described in [2], and to allow arbitrary fields in the pointer file.

[1] https://hawser.github.com/spec/v1
[2] https://github.com/git-lfs/git-lfs/blob/master/docs/spec.md

Change-Id: I621eb058619fb1b78888a54c4b60bb110a722fc3
Signed-off-by: Dariusz Luksza <dariusz@luksza.org>
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

Change-Id: Ibc20e9fc16be62f61748b3f59f6d70c1ebf63671
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

LFS pointer files have to be UTF-8 with \n as line ending character.
That is described in [1]. Fix JGit to follow this rules.

[1] https://github.com/github/git-lfs/blob/master/docs/spec.md

Bug: 507120
Change-Id: Ib6bd13f1cc17f1a3de125249b4f250b7b0692396

Enhance and use AtomicObjectOutputStream to write temporary files in
CleanFilter.

Change-Id: I28987dad18255a9067344f94b4e836cbd183e4b1
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

Change-Id: I227a0ed6e4e15ac3d96f96a6cefcaf55680ad8bb
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

The CleanFilter leaked temporary files when a media file already existed
before running clean filter.

Change-Id: Ie20fce3f40d34095ce58e596d25d8d64fe0cde99
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

Change-Id: I083ad1ea3e8d3685df7c306854c2498c92b05ffb
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

Transferring data byte per byte is slow, running add with CleanFilter on
a 2.9MB file takes 20 seconds. Using a buffer of 8k shrinks this time to
70ms.

Change-Id: I3bc2d8c11fe6cfaffcc99dc2a00643e01ac4e9cc
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

Change-Id: Ia56a9f58e336c25255a81d9402e198e5ce5da442
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

The class AtomicObjectOutputStream should be available to all lfs
related classes, not only to the server side. Move the class from
org.eclipse.jgit.lfs.server.fs to org.eclipse.jgit.lfs.internal to
achieve that.

Change-Id: I028e1c9ec7c21f316340b21d558b9a6b77e2060d

Adds a JGit built-in implementation of the "git lfs smudge" filter. This
filter should do the same as the one described in [1] besides that it
only supports the local case when the lfs objects are already present in
the media directory. Remote cases where download of LFS objects from an
LFS server is needed will be done in a later commit.

[1] https://github.com/github/git-lfs/blob/master/docs/man/git-lfs-smudge.1.ronn

Change-Id: I8ff661d4edd3667ef7f86f3b4fa33e568eb4c8f4

Adds a JGit built-in implementation of the "git lfs clean" filter. This
filter should do the same as the one described in [1]. But since this
filter is written in Java and can be called by JGit without forking new
processes it should be much faster

[1]
https://github.com/github/git-lfs/blob/master/docs/man/git-lfs-clean.1.ronn

Change-Id: If60e387e97870245b4bd765eda6717eb84cffb1d

Instead of returning null, LfsProtocolServlet#getLargeFileRepository
should throw LfsUnavailable.

If null is returned, throw a generic LfsException.

Handle LfsException as an internal server error and return HTTP 500.

Change-Id: I33e2a19fcc0fde8aaf0f703860c8fa8ce2de2db5
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>

Change-Id: I4967b5f7a9a7e9488d20f60aaa949efe97442429
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>

Since [1], the git-lfs specification allows the server to return
HTTP 507 if there is insufficient storage for the uploaded object(s).

Add a new exception class, which implementations may throw from the
getRepository() method, causing HTTP 507 to be returned to the client.

[1] https://github.com/github/git-lfs/pull/1473

Change-Id: If5bc0a35fcf870d4216af6ca2f7c8924689ef9c5
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>

The git-lfs specification [1] describes the following optional status codes
that may be returned:

429 - The user has hit a rate limit with the server. Though the API does
      not specify any rate limits, implementors are encouraged to set some
      for availability reasons.

509 - Returned if the bandwidth limit for the user or repository has been
      exceeded. The API does not specify any bandwidth limit, but implementors
      may track usage.

Add two new exception classes to represent these cases. Implementations may
throw these from #getLargeFileRepository(), causing the corresponding HTTP
status codes to be returned to the client.

[1] https://github.com/github/git-lfs/blob/master/docs/api/v1/http-v1-batch.md

Change-Id: I7b93f3cf90f7344c90b1587e07927fdeb167097e
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>

Instead of hard-coding the message strings, define them in a properties
file. This will allow them to be translated.

Change-Id: I77556881579e66b2c13d187759c7efdddfee87ae
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>

According to the specification [1] the server may return the following
HTTP error responses:

- 403: The user has read, but not write access.
- 404: The repository does not exist for the user.
- 422: Validation error with one or more of the objects in the request.

In the current implementation, however, getLargeFileRepository can only
return null to indicate an error. This results in the error code:

- 503: Service Unavailable

being returned to the client regardless of what the actual reason was.

Add exception classes to cover these cases, derived from a common base
exception, and change the specification of getLargeFileRepository to throw
the base exception.

In LfsProtocolServlet#post, handle the new exceptions and send back the
appropriate HTTP responses as mentioned above.

The specification also mentions several other optional response codes (406,
429, 501, and 509) but these are not implemented in this commit. It should
be trivial to implement them in follow-up commits.

[1] https://github.com/github/git-lfs/blob/master/docs/api/v1/http-v1-batch.md#response-errors

Change-Id: I91be6165bcaf856d0cefc533882330962e2fc9b2
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>

The location of the API v1 documentation has changed. Update the
links accordingly.

Change-Id: If0148a0b573c474bbe157fcb7e6674c0055fe8b4
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>

Implement LfsProtocolServlet handling the "Git LFS v1 Batch API"
protocol [1]. Add a simple file system based LFS content store and the
debug-lfs-store command to simplify testing.

Introduce a LargeFileRepository interface to enable additional storage
implementation while reusing the same protocol implementation.

At the client side we have to configure the lfs.url, specify that
we use the batch API and we don't use authentication:

  [lfs]
	  url = http://host:port/lfs
	  batch = true
  [lfs "http://host:port/lfs"]
	  access = none

the git-lfs client appends the "objects/batch" to the lfs.url.

Hard code an Authorization header in the FileLfsRepository.getAction
because then git-lfs client will skip asking for credentials. It will
just forward the Authorization header from the response to the
download/upload request.

The FileLfsServlet supports file content storage for "Large File
Storage" (LFS) server as defined by the Github LFS API [2].

- upload and download of large files is probably network bound hence use
  an asynchronous servlet for good scalability
- simple object storage in file system with 2 level fan-out
- use LockFile to protect writing large objects against multiple
  concurrent uploads of the same object
- to prevent corrupt uploads the uploaded file is rejected if its hash
  doesn't match id given in URL

The debug-lfs-store command is used to run the LfsProtocolServlet and,
optionally, the FileLfsServlet which makes it easier to setup a
local test server.

[1]
https://github.com/github/git-lfs/blob/master/docs/api/http-v1-batch.md
[2] https://github.com/github/git-lfs/tree/master/docs/api

Bug: 472961
Change-Id: I7378da5575159d2195138d799704880c5c82d5f3
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: Sasa Zivkov <sasa.zivkov@sap.com>

The Large File Storage extension specified by GitHub [1] uses SHA-256 to
compute the ID of large files stored by the extension. Hence implement a
SHA-256 abstraction similar to the SHA-1 abstraction used by JGit.

[1] https://git-lfs.github.com/

Bug: 470333
Change-Id: I3a95954543c8570d73929e55f4a884b55dbf1b7a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>