Support LFS protocol and a file system based LFS storage
Implement LfsProtocolServlet handling the "Git LFS v1 Batch API"
protocol [1]. Add a simple file system based LFS content store and the
debug-lfs-store command to simplify testing.
Introduce a LargeFileRepository interface to enable additional storage
implementation while reusing the same protocol implementation.
At the client side we have to configure the lfs.url, specify that
we use the batch API and we don't use authentication:
[lfs]
url = http://host:port/lfs
batch = true
[lfs "http://host:port/lfs"]
access = none
the git-lfs client appends the "objects/batch" to the lfs.url.
Hard code an Authorization header in the FileLfsRepository.getAction
because then git-lfs client will skip asking for credentials. It will
just forward the Authorization header from the response to the
download/upload request.
The FileLfsServlet supports file content storage for "Large File
Storage" (LFS) server as defined by the Github LFS API [2].
- upload and download of large files is probably network bound hence use
an asynchronous servlet for good scalability
- simple object storage in file system with 2 level fan-out
- use LockFile to protect writing large objects against multiple
concurrent uploads of the same object
- to prevent corrupt uploads the uploaded file is rejected if its hash
doesn't match id given in URL
The debug-lfs-store command is used to run the LfsProtocolServlet and,
optionally, the FileLfsServlet which makes it easier to setup a
local test server.
[1]
https://github.com/github/git-lfs/blob/master/docs/api/http-v1-batch.md
[2] https://github.com/github/git-lfs/tree/master/docs/api
Bug: 472961
Change-Id: I7378da5575159d2195138d799704880c5c82d5f3
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: Sasa Zivkov <sasa.zivkov@sap.com>
Add com.google.gson 2.2.4.v201311231704 to target platform
We will use gson to implement the LFS REST API.
CQ: 9966
Change-Id: I7feb6b2b8f7b736888370aad94e4c436ae6014bb
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
The Large File Storage extension specified by GitHub [1] uses SHA-256 to
compute the ID of large files stored by the extension. Hence implement a
SHA-256 abstraction similar to the SHA-1 abstraction used by JGit.
[1] https://git-lfs.github.com/
Bug: 470333
Change-Id: I3a95954543c8570d73929e55f4a884b55dbf1b7a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-4.2:
DirCacheCheckoutTest: Open Git and TreeWalk in try-with-resource
CommitCommand: Remove declaration of unthrown exception
Branch: Fix variable hiding warning
ApplyCommandTest: Open Git in try-with-resource
PackFileTest: Open ObjectInserter.Formatter in try-with-resource
Change-Id: I8484b10fad5a4c35fcfaedc1cdf8ccf97471618e
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
The Branch class has a member named 'branch', which was being
hidden by the local variable of the same name used in a for-loop.
Change-Id: I334092010a9c80686fb79713852d4bfa166ce12f
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
* stable-4.2:
FileTreeIteratorTest: Open Git and RevWalk in try-with-resource
RebaseCommandTest: Open RevWalk in try-with-resource
PullCommandTest: Open RevWalk in try-with-resource
BlameGeneratorTest: Create Git instances in try-with-resource
Remove unnecessary suppression of deprecation warnings in tests
DiffEntryTest: Open Git and TreeWalk in try-with-resource
DiffCommandTest: Open Git and RevWalk in try-with-resource
CommitCommandTest: Open Git and TreeWalk in try-with-resource
Change-Id: Ic886ec9b1a4b3b46f9fa14188b4df832ce36cfa6
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
Repo manifest file allows revision attribute in <remote> tag. This change
teaches JGit to read that information.
Change-Id: I1c878a2505b9d09fa09fbd404a119b71f2fb8fdb
Signed-off-by: Yuxuan 'fishy' Wang <fishywang@google.com>
* stable-4.2:
ConcurrentRepackTest: Don't use deprecated WindowCache.reconfigure
ConcurrentRepackTest: Open RevWalk in try-with-resource
CommitOnlyTest: Open RevWalk in try-with-resource
UnpackedObjectTest: Create ObjectInserter.Formatter in try-with-resource
FileRepositoryBuilderTest: Use try-with-resource for auto-closeables
RepositorySetupWorkDirTest: Fix "resource leak" warnings
Remove java7 bundle from Maven central scripts
Prepare 4.2.1-SNAPSHOT builds
JGit v4.2.0.201601211800-r
Add progress monitor to Merger
Fix TransportException when reading bundle
Fix unused throws CorruptObjectException from addTree
Change-Id: I2325fb995561a6249b7b5e82fa413dfd34ef6007
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Merge changes Ifdb33501,Idc7b7bbd,Ia3db8696,I4ef82311,I3ad58d4c, ... into stable-4.2
* changes:
ConcurrentRepackTest: Don't use deprecated WindowCache.reconfigure
ConcurrentRepackTest: Open RevWalk in try-with-resource
CommitOnlyTest: Open RevWalk in try-with-resource
UnpackedObjectTest: Create ObjectInserter.Formatter in try-with-resource
FileRepositoryBuilderTest: Use try-with-resource for auto-closeables
RepositorySetupWorkDirTest: Fix "resource leak" warnings
ConcurrentRepackTest: Don't use deprecated WindowCache.reconfigure
Replace with calls to WindowCacheConfig.install() as mentioned in
WindowCache.reconfigure's deprecation notice.
Change-Id: Ifdb33501a2209239029c815b1e4e844ea5b56075
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
UnpackedObjectTest: Create ObjectInserter.Formatter in try-with-resource
The ObjectInserter.Formatter instance is only used to call idFor.
Factor out a utility method to do that.
Change-Id: I4ef823110c2152ac7905681df3217eb8001f5bd9
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
FileRepositoryBuilderTest: Use try-with-resource for auto-closeables
Use try-with-resource to create instances of FileRepository and
FileWriter.
"resource" and "unused" warnings no longer occur, so remove the
suppression annotations.
Change-Id: I3ad58d4cc2d4c019cd8edda7cb401e9d9f3fb790
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
Use FileRepositoryBuilder to create the Repository, except in cases
where the creation was already in a try-block. Convert those to use
a try-with-resource.
Change-Id: I7d7adeee81bda6e80d91a119c7d690de3d00dc2b
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
The bundle org.eclipse.jgit.java7 was deleted in 4.0
Change-Id: I57dbc0b18eaf164c8f53cc2b776ecb80481a993d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Monitoring progress of merges can be useful for users for large
repositories or complex merge processes that take some time.
This enables setting a monitor. Existing merge implementations in jgit
do not yet report progress if a monitor is set. This will be added in a
later change.
Change-Id: I17b978b3fc91750dd88649638b90a46820a0877c
Signed-off-by: Laurent Delaigue <laurent.delaigue@obeo.fr>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
When reading a bundle file, commit messages who's oneline format is
longer than 982 characters caused JGit to treat subsequent text in
the commit as a SHA, then throw a TransportException because it's
not a valid SHA.
Now the readLine method will read all the way to the end of the
line, not just the first 1024 characters of it.
Change-Id: If15b491aa9a1e4fd9b8bbed2dd9e6be47a64ccb7
Signed-off-by: Chris Gavin <chris@chrisgavin.me>
Fix unused throws CorruptObjectException from addTree
2262a794b4 removed throws CorruptObjectException, but was not
compiled under Eclipse and did not see errors generated by Eclipse.
Maven build silently ignored the unnecessary code.
Change-Id: I6e9014adcc604588e10e5963d13d0c6ef0e1c617
* stable-4.2:
BundleWriterTest: Open RevWalk in try-with-resource
DiffFormatterTest: Remove accidentally added trailing whitespace
CherryPickCommandTest: Create Git instances in try-with-resource
DiffFormatterTest: Create auto-closeable instances in try-with-resource
ConfigTest: Create Git instance in try-with-resource
CommitAndLogCommandTest: Use assumeFalse to skip test on Windows
CommitAndLogCommandTest: Create Git instances in try-with-resource
AddCommandTest: Create Git instances in try-with-resource
ArchiveCommandTest: Create Git instances in try-with-resource
TagCommandTest: Instantiate Git and RevWalk objects in try-with-resource
BlameCommandTest: Instantiate Git objects in try-with-resource
SideBandOutputStreamTest: Use try-with-resource
FileTreeIteratorJava7Test: Create Git instances in try-with-resource
Change-Id: Ib572e98e6117b70442aee9cd7e7b8c3cf65562a7
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Monitoring progress of merges can be useful for users for large
repositories or complex merge processes that take some time.
This enables setting a monitor. Existing merge implementations in jgit
do not yet report progress if a monitor is set. This will be added in a
later change.
Change-Id: I17b978b3fc91750dd88649638b90a46820a0877c
Signed-off-by: Laurent Delaigue <laurent.delaigue@obeo.fr>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
DiffFormatterTest: Create auto-closeable instances in try-with-resource
Git, ByteArrayOutputStream, and DiffFormatter are auto-closeable and
should be managed in try-with-resource.
Change-Id: I83395116acb4b4f7cd4300fd69564355bc07e4bb
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
Matthias Sohn
David Pursehouse
Shawn Pearce
Yuxuan 'fishy' Wang
Rüdiger Herrmann
Thomas Wolf
Terry Parker
Laurent Delaigue
Chris Gavin