PackWriter: Only search for base objects on thin packs
A non-thin pack does not need to worry about preferred bases, the pack
will be self-contained and all required delta base objects will appear
within the pack itself. Obtaining the path buffer and length from the
ObjectWalk to build the preferred base table is "expensive", so avoid
the cost unless a thin pack is being constructed.
Change-Id: I16e30cd864f4189d4304e7957a7cd5bdb9e84528
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This flag was not being honored due to a bug in createWalk().
argWalk is always non-null when there are commits passed in
on the command line. If --objects was specified, always make
a new ObjectWalk for the actual execution.
Change-Id: I6e1a1636f2634605d86671a83766cc1c42939821
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
* changes:
PackWriter: Skip progress messages on fast operations
IndexPack: Defer the "Resolving deltas" progress meter
IndexPack: Fix "Resolving deltas" progress meter
The previous comment stated that the value set was used
to keep track of the branch in the remote repository
which was incorrect.
Updated the method comment to match the format used
for the PushCommand.setRemote and FetchCommand.setRemote
methods.
Change-Id: I11b81eb3125958af29247b485da56fd88c3bfdf5
Signed-off-by: Kevin Sawicki <kevin@github.com>
PackWriter: Skip progress messages on fast operations
If the "Finding sources" phase will complete in <1 second with no
delta compression enabled, don't bother showing the progress meter for
this phase. Small repositories on the local filesystem tend to rip
through this phase always subsecond and the ProgressMonitor display
can actually slow the operation down.
If delta compression is enabled, there are two phases that may run
very quickly. Set the timer to 500 milliseconds instead, reducing the
risk that the user has to wait longer than 1 second before any sort of
output from the packer occurs.
Change-Id: I58110f17e2a5ffa0134f9768b94804d16bbb8399
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
IndexPack: Defer the "Resolving deltas" progress meter
If delta resolution completes in < 1000 milliseconds, don't bother
showing the progress meter. This is actually very common for a Gerrit
Code Review server, where the client is probably sending 1 commit and
only a few trees/blobs modified... and the base objects are hot in the
process buffer cache.
The 1000 millisecond delay is just a guess at a reasonable time to wait.
Change-Id: I440baa64ab0dfa21be61deae8dcd3ca061bed8ce
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This progress meter never reached 100% as it did not update while
resolving the external bases in thin packs.
Instead of updating in batches at the top level, update once per delta
that is resolved. The batching progress meter type should smooth out
the frequent updates to an update rate that is more reasonable to send
to the UI, while also ensuring a successful pack parse always reaches
100% deltas resolved.
Change-Id: Ic77dcac542cfa97213a6b0194708f9d3c256d223
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Repository inspection tools may find building a reverse index on a
pack useful, as they can then locate an object by offset. As both
C Git and JGit sometimes produce error messages with the offset
rather than the SHA-1, it may be useful to expose this type.
Change-Id: I487bf32e85a8985cf8ab382d4c82fcbe1fc7da6c
There isn't a good reason to hide all of these as package-private.
Make them public so applications can inspect pack files.
Change-Id: Ia418daf65d63e9e015b8dafdf3d06a1ed91d190b
Some embeddings of UploadPack (e.g. Gerrit Code Review) set their own
PackConfig from a server-wide configuration, overriding any JGit
defaults or settings that may exist at the local repository level.
Make a copy constructor form of PackConfig so this server-wide
configuration object can be copied and then merged with repository
specific configuration data.
Change-Id: I4463c95aeaf7d6536c3ab132dec9c50ee528d9e0
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
The cached object databases should not require a close to release
their cached resources. Most object databases just return their
own reference for newCachedDatabase(), so a close() here kills
the real database's internal caches, and possibly underlying files,
resulting in poor performance for the callers of PackParser like
ReceivePack or FetchProcess trying to then go look up objects that
were just parsed, or that current references point to.
Change-Id: Ia4a239093866e5b9faf82744f729fb73f4373f1a
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Fix reading of ref names containing characters that sort before /
A set of ref names like ('a/b' and 'a+b') would cause the RefDirectory
to think that the set of refs have changed because it traversed the
'a' directory in the subtree before looking at 'a+b', but it then
compared with the know refs which are sorted with 'a+b' first.
Fix this by traversing the refs tree in another order. Treat a directory
as if they ends with a '/' before deciding on the order to traverse
the refs tree.
Bug: 348834
Change-Id: I23377f8df00c7252bf27dbcfba5da193c5403917
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
Repository.writeMergeCommitMsg(null) no longer fails if the MERGE_MSG
file is missing. This was done to avoid CommitCommand to fail in case of
a missing MERGE_MSG file.
Bug: 352243
Change-Id: Iddf43533d133f8f22199ed6e2393a552670e7d1f
Signed-off-by: Jens Baumgart <jens.baumgart@sap.com>
Reset command should works recursively and allows reset all changed
files in given directory.
Bug: 348524
Change-Id: I441db34f226be36548c61cef77958995971498de
Signed-off-by: Dariusz Luksza <dariusz@luksza.org>
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
Creation of a branch X from an annotated tag, as the starting point,
resulted into .git/refs/heads/X containing the ID of the annotated tag
instead of the ID of the tagged commit.
This fix peels the tag ref before using it as the starting point for
the newly created branch.
Bug: 340836
Change-Id: I01c7325770ecb37f5bf8ddb2a22f802466524f24
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
Use the temporary file management from superclass.
Change-Id: I3042951dc21860b4b85dd72a6bf41ee7cfe2aba4
Signed-off-by: Adrian Goerler <adrian.goerler@sap.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Cloning should fail when destination directory exists and is not empty
When trying to clone into a folder that already contains a cloned
repository native git will fail with a message "fatal: destination path
'folder' already exists and is not an empty directory.". Now JGit will
also fail in this situation throwing a JGitInternalException.
The test case was provided by Tomasz Zarna.
Bug: 347852
Change-Id: If9e9919a5f92d13cf038dc470c21ee5967322dac
Also-by: Tomasz Zarna <Tomasz.Zarna@pl.ibm.com>
Signed-off-by: Adrian Goerler <adrian.goerler@sap.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Add lookup to remote config if the refSpec is not explicitly set
If no refSpec is explicitly set, the PushCommand should first check the
remote config and then as a fallback use the current behavior.
Change-Id: I2bc648abc517b1d01b2de15d383423ace2081e72
Signed-off-by: Stefan Lay <stefan.lay@sap.com>
DirCacheCheckout: unlock DirCache in case of failure
DirCacheCheckout did not unlock the index if e.g. an IOException occured
during checkout.
Bug: 350677
Change-Id: Ie9fa09f7a404080da7cdccafb9be3a8c845e4869
Signed-off-by: Jens Baumgart <jens.baumgart@sap.com>
Improve performance when writing trees and small blobs
ObjectDirectoryInserter was always creating a temporary file,
writing the complete compressed contents of a tree, fsync()'ing
that to stable storage, and only then checking to see if there
was already an object with the same SHA-1 in the repository.
For commits this strategy makes some sense, the commit is very
unlikely to exist in the repository, as there are embedded times
and these change with each commit.
However for trees coming out of DirCache, it is more common for the
tree to already exist in the repository. Most subdirectories are
not modified in any given commit. Doing all of this local file IO
for things that already exist is very slow.
Try to detect cases where the object is "small enough" that it can
be processed entirely in memory, and avoid doing disk IO entirely
if the object already exists.
Also increase the size of the output buffer for the deflation.
This should boost the average write(2) syscall size from 512 bytes
to 8192 bytes, making streaming of large compressed contents to
disk slightly more efficient.
Change-Id: I1d40364e8725468522435814631916d73174c92b
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
TemporaryBuffer: Fix reading from in-memory InputStream
I had the conditions wrong here, causing the in-memory InputStream
to always appear to be at EOF.
Change-Id: I6811d6187a34eaf1fd6c5002550d631decdfc391
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Adds a git-reflog command and associated tests.
Bug: 347859
Change-Id: Iba146ac842cc9ca0be43d3381b4082c9e92bf56f
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
It's useful to have ReflogEntry refactored out so it can be
used by clients via the JGit API.
Change-Id: I03044df9af9f9547777545b7c9b93bdf5f8b7cb5
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
This patch possibly ties to a specific version of args4j.
Bug: 318286
Change-Id: I05d4ecf6bd25deec7fb2efbfa61913f4ec4e04e5
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
DHT: Change DhtReadher caches to be dynamic by workload
Instead of fixing the prefetch queue and recent chunk queue as
different sizes, allow these to share the same limit but be scaled
based on the work being performed.
During walks about 20% of the space will be given to the prefetcher,
and the other 80% will be used by the recent chunks cache. This
should improve cases where there is bad locality between chunks.
During writing of a pack stream, 90-100% of the space should be
made available to the prefetcher, as the prefetch plan is usually
very accurate about the order chunks will be needed in.
Change-Id: I1ca7acb4518e66eb9d4138fb753df38e7254704d
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
A linear search is somewhat acceptable for only 4 recent chunks, but
a HashMap based lookup would be better. The table will have 16 slots
by default and given the hashCode() of ChunkKey is derived from the
SHA-1 of the chunk, each chunk will fall into its own bucket within
the table and thus evaluate only 1 entry during lookup instead of 4.
Some users may also want to devote more memory to the recent chunks,
in which case expanding this list to a longer length will help to
reduce chunk faults, but would increase search time. Using a HashMap
will help this code to scale to larger sizes better.
Change-Id: Ia41b7a1cc69ad27b85749e3b74cbf8d0aa338044
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
DHT: Always have at least one recent chunk in DhtReader
The RecentChunks cache assumes there is always at least one recent
chunk in the maxSize that it receives from the DhtReaderOptions.
Ensure that is true by requiring the size to be at least 1.
Running with 0 recent chunk cache is very a bad idea, often
during commit walking the parents of a commit will be found
on the same chunk as the commit that was just accessed. In
these cases its a good idea to keep that last chunk around
so the parents can be quickly accessed.
Change-Id: I33b65286e8a4cbf6ef4ced28c547837f173e065d
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
If an internal exception occurs while packing and the request
needs to abort, the HTTP response might already be committed due
to progress message having already been delivered to the client.
This prevents UploadPackServlet from resetting the response and
sending back an HTTP 500 response.
Try to catch all exceptions and report internal errors over the
sideband stream or as an ERR command during the initial ACK/NAK
negotiation phase. This allows JGit to transmit an error message
that the user will receive on their console without needing to
worry about resetting the (already gone) HTTP response.
Change-Id: Ie393fb8bb55d2b79ab1276adf71c781c1807f9fe
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce
Kevin Sawicki
Matthias Sohn
Tomasz Zarna
Robin Rosenberg
Jens Baumgart
Dariusz Luksza
Sasa Zivkov
Chris Aniszczyk
Adrian Goerler
Stefan Lay
Christian Halstrick
Carsten Pfeiffer
Marc Strapetz