Browse Source

Bitmap builder: actually compress EWAH bitmaps in memory

For construction performance each new EWAHBitmap is allocated at the
roughly worst-case size the bitmap would need if all of the words must
be literal and no run length compression is available.  In practice
this is far larger than is required, wasting heap memory while the
bitmaps are computed.

Trim down each bitmap to its minimum required size. This copies the
internal array to a new smaller array, allowing the GC to reclaim the
prior larger array for reuse.

A single bitmap of 5.2M bits is only 79 KiB of memory without this
trim call but 15,000 such bitmaps is 1.1 GiB. Trimming can help fit
a larger number of bitmaps during processing.

Change-Id: I2bd19a786189db5b01c4c96f209b83de50e10c3b
tags/v4.1.0.201509280440-r
Shawn Pearce 8 years ago
parent
commit
88bffda524

+ 5
- 0
org.eclipse.jgit/src/org/eclipse/jgit/internal/storage/file/PackBitmapIndexBuilder.java View File

@@ -115,6 +115,10 @@ public class PackBitmapIndexBuilder extends BasePackBitmapIndex {
JGitText.get().badObjectType, String.valueOf(type)));
}
}
commits.trim();
trees.trim();
blobs.trim();
tags.trim();
}

private ObjectToPack[] sortByOffset(List<ObjectToPack> entries) {
@@ -168,6 +172,7 @@ public class PackBitmapIndexBuilder extends BasePackBitmapIndex {
*/
public void addBitmap(
AnyObjectId objectId, EWAHCompressedBitmap bitmap, int flags) {
bitmap.trim();
StoredBitmap result = new StoredBitmap(objectId, bitmap, null, flags);
getBitmaps().add(result);
byAddOrder.add(result);

Loading…
Cancel
Save