Support LFS protocol and a file system based LFS storage
Implement LfsProtocolServlet handling the "Git LFS v1 Batch API"
protocol [1]. Add a simple file system based LFS content store and the
debug-lfs-store command to simplify testing.
Introduce a LargeFileRepository interface to enable additional storage
implementation while reusing the same protocol implementation.
At the client side we have to configure the lfs.url, specify that
we use the batch API and we don't use authentication:
[lfs]
url = http://host:port/lfs
batch = true
[lfs "http://host:port/lfs"]
access = none
the git-lfs client appends the "objects/batch" to the lfs.url.
Hard code an Authorization header in the FileLfsRepository.getAction
because then git-lfs client will skip asking for credentials. It will
just forward the Authorization header from the response to the
download/upload request.
The FileLfsServlet supports file content storage for "Large File
Storage" (LFS) server as defined by the Github LFS API [2].
- upload and download of large files is probably network bound hence use
an asynchronous servlet for good scalability
- simple object storage in file system with 2 level fan-out
- use LockFile to protect writing large objects against multiple
concurrent uploads of the same object
- to prevent corrupt uploads the uploaded file is rejected if its hash
doesn't match id given in URL
The debug-lfs-store command is used to run the LfsProtocolServlet and,
optionally, the FileLfsServlet which makes it easier to setup a
local test server.
[1]
https://github.com/github/git-lfs/blob/master/docs/api/http-v1-batch.md
[2] https://github.com/github/git-lfs/tree/master/docs/api
Bug: 472961
Change-Id: I7378da5575159d2195138d799704880c5c82d5f3
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: Sasa Zivkov <sasa.zivkov@sap.com>
debug-rebuild-ref-tree: Simple program to build a RefTree
This tool scans all references in the repository and writes out a new
reference pointing to a single commit whose root tree is a RefTree
containing the current refs of this repository.
It alway skips storing the reference it will write to, avoiding the
obvious cycle.
Change-Id: I20b1eeb81c55dc49dd600eac3bf8f90297394113
All bugs reported for gc have been fixed and it seems we reached a
stable implementation.
Change-Id: I78a96ee2103beb48325da0f6ee10b2498bdc0267
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
It stopped working when we moved to the Eclipse foundation's Gerrit
server since it doesn't use the Gerrit internal user store but LDAP.
Instead, since 2.0, we use the Eclipse foundation's automatic IP log
generator [1] to generate IP logs for releasing jgit and egit.
[1] http://www.eclipse.org/projects/ip_log_selector.php
Change-Id: I98dc65efb62909bc0258e6c680df0c93a57e9677
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Add "jgit archive" tool that writes a tree as a ZIP file
C Git's "git archive" command represents a tree object using a
standard archival format like tar, zip, or tgz, ready for consumption
by other, git-unaware users or tools.
Add a bare-bones analagous "jgit archive" command to show what is
possible, supporting only ZIP format for now. It uses java.util.zip
which is not aware of the InfoZIP extensions for representing symlinks
and file permissions, so symlinks, executable files, and submodule
entries are represented as plain text files.
Making this functionality available from the library, improving
handling of special entries, and support for other output formats are
left for later patches. Ultimately the intent is to offer a
TreeArchiveStream class for use by web frontends like Gitiles to offer
"download as zip/tgz/txz" links and use by, for example, code search
tools to get easy access to the content of git tree objects.
Test with "jgit archive my-favorite-tree >out.zip".
Change-Id: Ib590f173ceff3df4b58493cecccd6b9a1b355e3d
This is a first basic implementation that displays current branch and
list of files of various status, but isn't as refined as its native
counterpart (e.g. does not say if we're ahead or behind the remote).
It's been helpful in the diagnostic of bug #347885.
Bug: 348318
CQ: 6769
Change-Id: Ifc35da608fbba652524c1b5b522e3c0d5369ad5e
Signed-off-by: François Rey <eclipse.org@francois.rey.name>
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
Implements a garbage collector for FileRepositories. Main ideas are
copied from the garbage collector for DFS based repos
(DfsGarbageCollector). Added functionalities are
- pruning loose objects
- handling of the index
- packing refs
- handling of reflogs (objects referenced from reflog will not be
pruned/)
These are features of a GC which are not handled in this change and
which should come with subsequent changes:
- unpacking packed objects into loose objects (to support that pruning
packed objects doesn't delete them until they are older than two weeks)
- expiration of reflogs
- support for configuration parameters (e.g. gc.pruneExpire)
Change-Id: I14ea5cb7e0fd1b5c50b994fd77f4e05bfbb9d911
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Currently, only --list option is supported with --global, --system,
--local and --file switches.
Change-Id: I9b179b162996520e95c4e001dccd65c566a4bd27
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Command line options match the C implementation of `git blame` as
closely as possible, making for a pretty complete tool.
Change-Id: Ie1bd172ad9de586c3b60f0ee4a77a8f047364882
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This merge command accepts the merge strategy as option and uses the
resolve strategy as default. It expects exactly one other
revision which is merged with current head.
Change-Id: Ia8c188b93ade4afabe6a9ccf267faf045f359a3a
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
Registering the Checkout command and fixed a typo.
The Checkout command line command was added to JGit but it wasn't
registered in the list of available commands.
Additionally, the 'force' option was named '---force' (triple '-').
Change-Id: I259773932fa9aec3bb29e215740e67c834566f6f
Signed-off-by: Sasa Zivkov <sasa.zivkov@sap.com>
Implementation delegates all work to the AddCommand class and,
therefore, supports only those options currently supported by the
AddCommand which means: --update and the filepattern... arguments.
Change-Id: I4827d37e08b4c988c2458d9ba60a61b6ad414d10
Signed-off-by: Sasa Zivkov <sasa.zivkov@sap.com>
debug-diff-algorithms: Real world performance test implementations
When working on a difference algorithm's implementation, its generally
more important to care about how it behaves on real-world inputs than
it does on fake inputs created for unit test cases. Run each
implementation against a number of real-world repositories, looking at
changes between files in each commit. This gives a better picture of
how a particular algorithm performs.
This test suite run against JGit and linux-2.6 with the current
available algorithms shows HistogramDiff always out-performs
MyersDiff, and by a wide margin on the linux-2.6 sources. As
HistogramDiff has similar output properties as PatienceDiff, the
resulting edits are probably also more human-readable. These test
results show that HistogramDiff is a good choice for the default
implementation, and also show that PatienceDiff isn't worth keeping.
jgit: start at baa83ae
2686 files, 760 commits
N= 3 min lines, 3016 max lines
Algorithm Time(ns) ( Time(ns) on Time(ns) on )
( N=3 N=3016 )
---------------------------------------------------------------------
histogram_myers 314652100 ( 3900 298100 )
histogram 315973000 ( 3800 302100 )
patience 774724900 ( 4500 347900 )
patience_histogram_myers 786332800 ( 3700 351200 )
myers 819359300 ( 4100 379100 )
patience_myers 843416700 ( 3800 348000 )
linux-2.6.git: start at 85a3318
4001 files, 2680 commits
N= 2 min lines, 39098 max lines
Algorithm Time(ns) ( Time(ns) on Time(ns) on )
( N=2 N=39098 )
---------------------------------------------------------------------
histogram_myers 1229870000 ( 5900 2642700 )
histogram 1235654100 ( 6000 2695400 )
patience 3856546000 ( 5900 2627700 )
patience_histogram_myers 3866728100 ( 7000 2624000 )
patience_myers 4004875300 ( 8000 2651700 )
myers 9794679000 ( 7200 2716200 )
Change-Id: I2502684d31f7851e720356820d04d8cf767f7229
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
debug-text-hashfunctions: Test suite for content hashes
This is the test suite I was using to help understand why we had
such a high collision rate with RawTextComparator, and to select
a replacement function.
Since its not something we will run very often, lets make it a
program in the debug package rather than a JUnit test. This way
we can run it on demand against any corpus of files we choose,
but we aren't bottlenecking our daily builds running tests with
no assertions.
Adding a new hash function to this suite is simple, just define
a new instance member of type "Hash" with the logic applied to
the region passed in.
Change-Id: Iec0b176adb464cf95b06cda157932b79c0b59886
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
debug-show-packdelta: Dump a pack delta to the console
This is a horribly crude application, it doesn't even verify that
the object its dumping is delta encoded. Its method of getting the
delta is pretty abusive to the public PackWriter API, because right
now we don't want to expose the real internal low-level methods
actually required to do this.
Change-Id: I437a17ceb98708b5603a2061126eb251e82f4ed4
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
The commit command is added using the new Git class. Currently
this supports only the author and commit-message option.
Change-Id: I13152575b5b03f6f9e816d0747e7a8c5c6fccade
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Generate an Eclipse IP log with jgit eclipse-iplog
The new plugin contains the bulk of the logic to scan a Git repository,
and query IPZilla, in order to produce an XML formatted IP log for the
requested revision of any Git based project. This plugin is suitable
for embedding into a servlet container, or into the Eclipse workbench.
The command line pgm package knows how to invoke this plugin through
the eclipse-iplog subcommand, permitting storage of the resulting
log as a local XML file.
Change-Id: If01d9d98d07096db6980292bd5f91618c55d00be
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This way we depend upon the MANIFEST.MF to define our classpath
and our build will act more like any other OSGI bundle build.
Change-Id: I9e1f1f5a0bccb0ab0e39e49b75fb400fea446619
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Matthias Sohn
Shawn Pearce
Kaloyan Raev
Yuxuan 'fishy' Wang
Tomasz Zarna
Jonathan Nieder
Tomasz Zarna
Robin Rosenberg
Christian Halstrick
Tomasz Zarna
Sasa Zivkov