Matthias Sohn [Tue, 22 Dec 2015 15:11:43 +0000 (16:11 +0100)]
Support LFS protocol and a file system based LFS storage
Implement LfsProtocolServlet handling the "Git LFS v1 Batch API"
protocol [1]. Add a simple file system based LFS content store and the
debug-lfs-store command to simplify testing.
Introduce a LargeFileRepository interface to enable additional storage
implementation while reusing the same protocol implementation.
At the client side we have to configure the lfs.url, specify that
we use the batch API and we don't use authentication:
the git-lfs client appends the "objects/batch" to the lfs.url.
Hard code an Authorization header in the FileLfsRepository.getAction
because then git-lfs client will skip asking for credentials. It will
just forward the Authorization header from the response to the
download/upload request.
The FileLfsServlet supports file content storage for "Large File
Storage" (LFS) server as defined by the Github LFS API [2].
- upload and download of large files is probably network bound hence use
an asynchronous servlet for good scalability
- simple object storage in file system with 2 level fan-out
- use LockFile to protect writing large objects against multiple
concurrent uploads of the same object
- to prevent corrupt uploads the uploaded file is rejected if its hash
doesn't match id given in URL
The debug-lfs-store command is used to run the LfsProtocolServlet and,
optionally, the FileLfsServlet which makes it easier to setup a
local test server.
Matthias Sohn [Fri, 17 Jul 2015 14:22:56 +0000 (16:22 +0200)]
Implement SHA-256 abstraction
The Large File Storage extension specified by GitHub [1] uses SHA-256 to
compute the ID of large files stored by the extension. Hence implement a
SHA-256 abstraction similar to the SHA-1 abstraction used by JGit.
[1] https://git-lfs.github.com/
Bug: 470333
Change-Id: I3a95954543c8570d73929e55f4a884b55dbf1b7a Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Wed, 3 Feb 2016 09:30:38 +0000 (10:30 +0100)]
Merge branch 'stable-4.2'
* stable-4.2:
DirCacheCheckoutTest: Open Git and TreeWalk in try-with-resource
CommitCommand: Remove declaration of unthrown exception
Branch: Fix variable hiding warning
ApplyCommandTest: Open Git in try-with-resource
PackFileTest: Open ObjectInserter.Formatter in try-with-resource
Change-Id: I8484b10fad5a4c35fcfaedc1cdf8ccf97471618e Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-4.2:
FileTreeIteratorTest: Open Git and RevWalk in try-with-resource
RebaseCommandTest: Open RevWalk in try-with-resource
PullCommandTest: Open RevWalk in try-with-resource
BlameGeneratorTest: Create Git instances in try-with-resource
Remove unnecessary suppression of deprecation warnings in tests
DiffEntryTest: Open Git and TreeWalk in try-with-resource
DiffCommandTest: Open Git and RevWalk in try-with-resource
CommitCommandTest: Open Git and TreeWalk in try-with-resource
Change-Id: Ic886ec9b1a4b3b46f9fa14188b4df832ce36cfa6 Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
Matthias Sohn [Sun, 24 Jan 2016 09:59:14 +0000 (10:59 +0100)]
Merge branch 'stable-4.2'
* stable-4.2:
ConcurrentRepackTest: Don't use deprecated WindowCache.reconfigure
ConcurrentRepackTest: Open RevWalk in try-with-resource
CommitOnlyTest: Open RevWalk in try-with-resource
UnpackedObjectTest: Create ObjectInserter.Formatter in try-with-resource
FileRepositoryBuilderTest: Use try-with-resource for auto-closeables
RepositorySetupWorkDirTest: Fix "resource leak" warnings
Remove java7 bundle from Maven central scripts
Prepare 4.2.1-SNAPSHOT builds
JGit v4.2.0.201601211800-r
Add progress monitor to Merger
Fix TransportException when reading bundle
Fix unused throws CorruptObjectException from addTree
Change-Id: I2325fb995561a6249b7b5e82fa413dfd34ef6007 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Sun, 24 Jan 2016 01:11:04 +0000 (20:11 -0500)]
Merge changes Ifdb33501,Idc7b7bbd,Ia3db8696,I4ef82311,I3ad58d4c, ... into stable-4.2
* changes:
ConcurrentRepackTest: Don't use deprecated WindowCache.reconfigure
ConcurrentRepackTest: Open RevWalk in try-with-resource
CommitOnlyTest: Open RevWalk in try-with-resource
UnpackedObjectTest: Create ObjectInserter.Formatter in try-with-resource
FileRepositoryBuilderTest: Use try-with-resource for auto-closeables
RepositorySetupWorkDirTest: Fix "resource leak" warnings
Use FileRepositoryBuilder to create the Repository, except in cases
where the creation was already in a try-block. Convert those to use
a try-with-resource.
Change-Id: I7d7adeee81bda6e80d91a119c7d690de3d00dc2b Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
Laurent Delaigue [Wed, 13 Jan 2016 15:56:00 +0000 (16:56 +0100)]
Add progress monitor to Merger
Monitoring progress of merges can be useful for users for large
repositories or complex merge processes that take some time.
This enables setting a monitor. Existing merge implementations in jgit
do not yet report progress if a monitor is set. This will be added in a
later change.
Change-Id: I17b978b3fc91750dd88649638b90a46820a0877c Signed-off-by: Laurent Delaigue <laurent.delaigue@obeo.fr> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Chris Gavin [Wed, 6 Jan 2016 15:53:22 +0000 (15:53 +0000)]
Fix TransportException when reading bundle
When reading a bundle file, commit messages who's oneline format is
longer than 982 characters caused JGit to treat subsequent text in
the commit as a SHA, then throw a TransportException because it's
not a valid SHA.
Now the readLine method will read all the way to the end of the
line, not just the first 1024 characters of it.
Change-Id: If15b491aa9a1e4fd9b8bbed2dd9e6be47a64ccb7 Signed-off-by: Chris Gavin <chris@chrisgavin.me>
Shawn Pearce [Thu, 21 Jan 2016 06:08:23 +0000 (22:08 -0800)]
Fix unused throws CorruptObjectException from addTree
2262a794b48ea removed throws CorruptObjectException, but was not
compiled under Eclipse and did not see errors generated by Eclipse.
Maven build silently ignored the unnecessary code.
Matthias Sohn [Thu, 21 Jan 2016 13:04:39 +0000 (14:04 +0100)]
Merge branch 'stable-4.2'
* stable-4.2:
BundleWriterTest: Open RevWalk in try-with-resource
DiffFormatterTest: Remove accidentally added trailing whitespace
CherryPickCommandTest: Create Git instances in try-with-resource
DiffFormatterTest: Create auto-closeable instances in try-with-resource
ConfigTest: Create Git instance in try-with-resource
CommitAndLogCommandTest: Use assumeFalse to skip test on Windows
CommitAndLogCommandTest: Create Git instances in try-with-resource
AddCommandTest: Create Git instances in try-with-resource
ArchiveCommandTest: Create Git instances in try-with-resource
TagCommandTest: Instantiate Git and RevWalk objects in try-with-resource
BlameCommandTest: Instantiate Git objects in try-with-resource
SideBandOutputStreamTest: Use try-with-resource
FileTreeIteratorJava7Test: Create Git instances in try-with-resource
Change-Id: Ib572e98e6117b70442aee9cd7e7b8c3cf65562a7 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Laurent Delaigue [Wed, 13 Jan 2016 15:56:00 +0000 (16:56 +0100)]
Add progress monitor to Merger
Monitoring progress of merges can be useful for users for large
repositories or complex merge processes that take some time.
This enables setting a monitor. Existing merge implementations in jgit
do not yet report progress if a monitor is set. This will be added in a
later change.
Change-Id: I17b978b3fc91750dd88649638b90a46820a0877c Signed-off-by: Laurent Delaigue <laurent.delaigue@obeo.fr> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Chris Gavin [Wed, 6 Jan 2016 15:53:22 +0000 (15:53 +0000)]
Fix TransportException when reading bundle
When reading a bundle file, commit messages who's oneline format is
longer than 982 characters caused JGit to treat subsequent text in
the commit as a SHA, then throw a TransportException because it's
not a valid SHA.
Now the readLine method will read all the way to the end of the
line, not just the first 1024 characters of it.
Change-Id: If15b491aa9a1e4fd9b8bbed2dd9e6be47a64ccb7 Signed-off-by: Chris Gavin <chris@chrisgavin.me>
Shawn Pearce [Thu, 21 Jan 2016 06:08:23 +0000 (22:08 -0800)]
Fix unused throws CorruptObjectException from addTree
2262a794b48ea removed throws CorruptObjectException, but was not
compiled under Eclipse and did not see errors generated by Eclipse.
Maven build silently ignored the unnecessary code.
Shawn Pearce [Thu, 21 Jan 2016 05:32:17 +0000 (00:32 -0500)]
Merge changes from topic 'reftree'
* changes:
debug-rebuild-ref-tree: Copy HEAD into RefTree
debug-rebuild-ref-tree: Add --enable flag to turn the database on
RefTreeDatabase: Allow ORIG_HEAD, etc. on non-bare repositories
RefTreeDatabase: Expose bootstrap refs in getAdditionalRefs
Shawn Pearce [Thu, 21 Jan 2016 04:15:09 +0000 (23:15 -0500)]
Merge changes from topic 'ketch'
* changes:
daemon: Add --ketch=LEADER flag
Ketch: Intercept push and route it through Ketch
Ketch: Simple map of running leader instances
Ketch: Basic replication system
Shawn Pearce [Wed, 20 Jan 2016 15:41:08 +0000 (07:41 -0800)]
TreeWalk: Remove CorruptObjectException from addTree(AbstractTreeIterator)
This form of addTree() does not parse any objects and cannot throw the
declared checked exception. Callers are being forced to try-catch
CorruptObjectException that cannot occur when the iterator instance
has already been constructed.
ResetCommand was not returning the updated ref as a result of the call()
method. Since the ResetCommand is always updating the same ref (HEAD)
this should always be the HEAD ref.
Bug: 440750
Change-Id: I7974975c3ab05e68c208384e69cf0692ded6e8db Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Add option to allow empty commits to CommitCommand
CommitCommand should allow to specify whether empty commits (commits
having the same tree as the sole predecessor commit) are allowed or not.
Similar to native git's "--allow-empty" flag.
The defaults differ between JGit and native git even after this change.
When not specifying paths then by default JGit allows to create empty
commits while native git does not. It would be API breaking to change
this now.
Bug: 460301
Change-Id: I88feb0c3ffb2c686b1d0594e669729b065cda4cb Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Shawn Pearce [Wed, 13 Jan 2016 05:55:43 +0000 (21:55 -0800)]
daemon: Add --ketch=LEADER flag
Experimental flag to turn on the KetchLeader within this daemon JVM.
This is a manually elected leader process, set from the command line.
Remote followers for each repository are configured per-repository
using remote sections with ketch-type = FULL. For example:
Manually elected leader's $GIT_DIR/config:
[ketch]
name = A
[remote "A"]
ketch-type = FULL
[remote "B"]
url = git://127.0.0.1:9421/sample.git
ketch-type = FULL
[remote "C"]
url = git://127.0.0.1:9422/sample.git
ketch-type = FULL
Shawn Pearce [Mon, 18 Jan 2016 18:33:31 +0000 (10:33 -0800)]
Ketch: Basic replication system
Git Ketch is a multi-master Git repository management system. Writes
are successful only if a majority of participant servers agree. Acked
writes are durable against server failures as a majority of the
participants store all required objects.
Git Ketch is modeled on the Raft Consensus Algorithm[1]. A ketch
sailing vessel is faster and more nimble than a raft. It can also
carry more source codes.
Git Ketch front-loads replication costs, which vaguely resembles a
ketch sailing vessel's distinguishing feature of the main mast on the
front of the ship.
Dave Borowitz [Tue, 19 Jan 2016 16:13:40 +0000 (11:13 -0500)]
ChangeIdUtil: Don't throw IOException
This could have only happened during the getBytes call. Instead, use
Constants.encode, which is a non-throwing implementation.
This change is binary compatible with existing code compiled against
older versions of JGit, although it might break compilation of
previously compiling code due to dead catch blocks.
Shawn Pearce [Tue, 19 Jan 2016 21:08:38 +0000 (13:08 -0800)]
Transport: Implement AutoCloseable
After creating a Transport instance callers should always call
its close() method. Use AutoCloseable to document this idiom
and allow use of try-with-resources.
Shawn Pearce [Tue, 19 Jan 2016 22:08:53 +0000 (17:08 -0500)]
Merge changes from topic 'deadtree'
* changes:
Ignore API errors about Tree, TreeEntry, FileTreeEntry and friends being removed
Revert "Revert "Remove deprecated Tree, TreeEntry, FileTreeEntry and friends""
Shawn Pearce [Mon, 18 Jan 2016 18:42:00 +0000 (10:42 -0800)]
ReceiveCommand.abort(): Utility to mark batch of commands as failed
If one or more commands is failing the entire group usually has to
also fail with "transaction aborted". Pull this loop into a helper
so the idiom can be easily reused in several places throughout JGit.
Shawn Pearce [Fri, 15 Jan 2016 22:13:25 +0000 (14:13 -0800)]
debug-rebuild-ref-tree: Copy HEAD into RefTree
HEAD is not part of getRefs(ALL) and must be copied explicitly.
This allows `jgit debug-rebuild-ref-tree --enable` to convert an
existing repository to use RefTree with a local working tree:
projects / jgit.git / log