]> source.dussan.org Git - jgit.git/commit
Support large delta packed objects as streams 35/1035/1
authorShawn O. Pearce <spearce@spearce.org>
Fri, 2 Jul 2010 09:19:12 +0000 (02:19 -0700)
committerShawn O. Pearce <spearce@spearce.org>
Fri, 2 Jul 2010 09:19:12 +0000 (02:19 -0700)
commitad68553be4417ec7ac636c3d823fdddced46ecfb
tree456e656c6d46be39df3fccb5e96085b9813b91d6
parentded8f6c72150253c86b9586025da3a3997ccad6c
Support large delta packed objects as streams

Very large delta instruction streams, or deltas which use very large
base objects, are now streamed through as large objects rather than
being inflated into a byte array.

This isn't the most efficient way to access delta encoded content, as
we may need to rewind and reprocess the base object when there was a
block moved within the file, but it will at least prevent the JVM from
having its heap explode.

When streaming a delta we have an inflater open for each level in the
delta chain, to inflate the instruction set of the delta, as well as
an inflater for the base level object.  The base object is buffered,
as is the top level delta requested by the application, but we do not
buffer the intermediate delta streams.  This keeps memory usage lower,
so its closer to 1024 bytes per level in the chain, without having an
adverse impact on raw throughput as the top-level buffer gets pushed
down to the lowest stream that has the next region.

Delta instructions transparently collapse here, if the top level does
not copy a region from its base, the base won't materialize that part
from its own base, etc.  This allows us to avoid copying around a lot
of segments which have been deleted from the final version.

Change-Id: I724d45245cebb4bad2deeae7b896fc55b2dd49b3
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
org.eclipse.jgit.test/tst/org/eclipse/jgit/storage/file/PackFileTest.java
org.eclipse.jgit.test/tst/org/eclipse/jgit/storage/pack/DeltaStreamTest.java [new file with mode: 0644]
org.eclipse.jgit/src/org/eclipse/jgit/storage/file/ByteArrayWindow.java
org.eclipse.jgit/src/org/eclipse/jgit/storage/file/ByteBufferWindow.java
org.eclipse.jgit/src/org/eclipse/jgit/storage/file/ByteWindow.java
org.eclipse.jgit/src/org/eclipse/jgit/storage/file/LargePackedDeltaObject.java [new file with mode: 0644]
org.eclipse.jgit/src/org/eclipse/jgit/storage/file/PackFile.java
org.eclipse.jgit/src/org/eclipse/jgit/storage/file/WindowCursor.java
org.eclipse.jgit/src/org/eclipse/jgit/storage/pack/BinaryDelta.java
org.eclipse.jgit/src/org/eclipse/jgit/storage/pack/DeltaEncoder.java [new file with mode: 0644]
org.eclipse.jgit/src/org/eclipse/jgit/storage/pack/DeltaStream.java [new file with mode: 0644]