Browse Source

ObjectIdOwnerMap: More lightweight map for ObjectIds

OwnerMap is about 200 ms faster than SubclassMap, more friendly to the
GC, and uses less storage: testing the "Counting objects" part of
PackWriter on 1886362 objects:

  ObjectIdSubclassMap:
    load factor 50%
    table: 4194304 (wasted 2307942)
    ms spent 36998 36009 34795 34703 34941 35070 34284 34511 34638 34256
    ms avg 34800 (last 9 runs)

  ObjectIdOwnerMap:
    load factor 100%
    table: 2097152 (wasted 210790)
    directory: 1024
    ms spent 36842 35112 34922 34703 34580 34782 34165 34662 34314 34140
    ms avg 34597 (last 9 runs)

The major difference with OwnerMap is entries must extend from
ObjectIdOwnerMap.Entry, where the OwnerMap has injected its own
private "next" field into each object. This allows the OwnerMap to use
a singly linked list for chaining collisions within a bucket. By
putting collisions in a linked list, we gain the entire table back for
the SHA-1 bits to index their own "private" slot.

Unfortunately this means that each object can appear in at most ONE
OwnerMap, as there is only one "next" field within the object instance
to thread into the map. For types that are very object map heavy like
RevWalk (entity RevObject) and PackWriter (entity ObjectToPack) this
is sufficient, these entity types are only put into one map by their
container.  By introducing a new map type, we don't break existing
applications that might be trying to use ObjectIdSubclassMap to track
RevCommits they obtained from a RevWalk.

The OwnerMap uses less memory. Each object uses 1 reference more (so
we're up 1,886,362 references), but the table is 1/2 the size (2^20
rather than 2^21). The table itself wastes only 210,790 slots, rather
than 2,307,942. So OwnerMap is wasting 200k fewer references.

OwnerMap is more friendly to the GC, because it hardly ever generates
garbage. As the map reaches its 100% load factor target, it doubles in
size by allocating additional segment arrays of 2048 entries. (So the
first grow allocates 1 segment, second 2 segments, third 4 segments,
etc.)  These segments are hooked into the pre-allocated directory of
1024 spaces. This permits the map to grow to 2 million objects before
the directory itself has to grow. By using segments of 2048 entries,
we are asking the GC to acquire 8,204 bytes in a 32 bit JVM. This is
easier to satisfy then 2,307,942 bytes (for the 512k table that is
just an intermediate step in the SubclassMap). By reusing the
previously allocated segments (they are re-hashed in-place) we don't
release any memory during a table grow.

When the directory grows, it does so by discarding the old one and
using one that is 4x larger (so the directory goes to 4096 entries on
its first grow). A directory of size 4096 can handle up to 8 millon
objects. The second directory grow (16384) goes to 33 million objects.
At that point we're starting to really push the limits of the JVM
heap, but at least its many small arrays. Previously SubclassMap would
need a table of 67108864 entries to handle that object count, which
needs a single contiguous allocation of 256 MiB. That's hard to come
by in a 32 bit JVM. Instead OwnerMap uses 8192 arrays of about 8 KiB
each. This is much easier to fit into a fragmented heap.

Change-Id: Ia4acf5cfbf7e9b71bc7faa0db9060f6a969c0c50
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
tags/v0.12.1
Shawn O. Pearce 13 years ago
parent
commit
bd970007be

+ 217
- 0
org.eclipse.jgit.test/tst/org/eclipse/jgit/lib/ObjectIdOwnerMapTest.java View File

@@ -0,0 +1,217 @@
/*
* Copyright (C) 2011, Google Inc.
* and other copyright owners as documented in the project's IP log.
*
* This program and the accompanying materials are made available
* under the terms of the Eclipse Distribution License v1.0 which
* accompanies this distribution, is reproduced below, and is
* available at http://www.eclipse.org/org/documents/edl-v10.php
*
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or
* without modification, are permitted provided that the following
* conditions are met:
*
* - Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
*
* - Redistributions in binary form must reproduce the above
* copyright notice, this list of conditions and the following
* disclaimer in the documentation and/or other materials provided
* with the distribution.
*
* - Neither the name of the Eclipse Foundation, Inc. nor the
* names of its contributors may be used to endorse or promote
* products derived from this software without specific prior
* written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
* CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
* INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
* OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
* CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
* SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
* NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
* LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
* CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
* STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
* ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/

package org.eclipse.jgit.lib;

import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertNotNull;
import static org.junit.Assert.assertSame;
import static org.junit.Assert.assertTrue;
import static org.junit.Assert.fail;

import java.util.Iterator;
import java.util.NoSuchElementException;

import org.junit.Before;
import org.junit.Test;

public class ObjectIdOwnerMapTest {
private MutableObjectId idBuf;

private SubId id_1, id_2, id_3, id_a31, id_b31;

@Before
public void init() {
idBuf = new MutableObjectId();
id_1 = new SubId(id(1));
id_2 = new SubId(id(2));
id_3 = new SubId(id(3));
id_a31 = new SubId(id(31));
id_b31 = new SubId(id((1 << 8) + 31));
}

@Test
public void testEmptyMap() {
ObjectIdOwnerMap<SubId> m = new ObjectIdOwnerMap<SubId>();
assertTrue(m.isEmpty());
assertEquals(0, m.size());

Iterator<SubId> i = m.iterator();
assertNotNull(i);
assertFalse(i.hasNext());

assertFalse(m.contains(id(1)));
}

@Test
public void testAddGetAndContains() {
ObjectIdOwnerMap<SubId> m = new ObjectIdOwnerMap<SubId>();
m.add(id_1);
m.add(id_2);
m.add(id_3);
m.add(id_a31);
m.add(id_b31);
assertFalse(m.isEmpty());
assertEquals(5, m.size());

assertSame(id_1, m.get(id_1));
assertSame(id_1, m.get(id(1)));
assertSame(id_1, m.get(id(1).copy()));
assertSame(id_2, m.get(id(2).copy()));
assertSame(id_3, m.get(id(3).copy()));
assertSame(id_a31, m.get(id(31).copy()));
assertSame(id_b31, m.get(id_b31.copy()));

assertTrue(m.contains(id_1));
}

@Test
public void testClear() {
ObjectIdOwnerMap<SubId> m = new ObjectIdOwnerMap<SubId>();

m.add(id_1);
assertSame(id_1, m.get(id_1));

m.clear();
assertTrue(m.isEmpty());
assertEquals(0, m.size());

Iterator<SubId> i = m.iterator();
assertNotNull(i);
assertFalse(i.hasNext());

assertFalse(m.contains(id(1)));
}

@Test
public void testAddIfAbsent() {
ObjectIdOwnerMap<SubId> m = new ObjectIdOwnerMap<SubId>();
m.add(id_1);

assertSame(id_1, m.addIfAbsent(new SubId(id_1)));
assertEquals(1, m.size());

assertSame(id_2, m.addIfAbsent(id_2));
assertEquals(2, m.size());
assertSame(id_a31, m.addIfAbsent(id_a31));
assertSame(id_b31, m.addIfAbsent(id_b31));

assertSame(id_a31, m.addIfAbsent(new SubId(id_a31)));
assertSame(id_b31, m.addIfAbsent(new SubId(id_b31)));
assertEquals(4, m.size());
}

@Test
public void testAddGrowsWithObjects() {
int n = 16384;
ObjectIdOwnerMap<SubId> m = new ObjectIdOwnerMap<SubId>();
m.add(id_1);
for (int i = 32; i < n; i++)
m.add(new SubId(id(i)));
assertEquals(n - 32 + 1, m.size());

assertSame(id_1, m.get(id_1.copy()));
for (int i = 32; i < n; i++)
assertTrue(m.contains(id(i)));
}

@Test
public void testAddIfAbsentGrowsWithObjects() {
int n = 16384;
ObjectIdOwnerMap<SubId> m = new ObjectIdOwnerMap<SubId>();
m.add(id_1);
for (int i = 32; i < n; i++)
m.addIfAbsent(new SubId(id(i)));
assertEquals(n - 32 + 1, m.size());

assertSame(id_1, m.get(id_1.copy()));
for (int i = 32; i < n; i++)
assertTrue(m.contains(id(i)));
}

@Test
public void testIterator() {
ObjectIdOwnerMap<SubId> m = new ObjectIdOwnerMap<SubId>();
m.add(id_1);
m.add(id_2);
m.add(id_3);

Iterator<SubId> i = m.iterator();
assertTrue(i.hasNext());
assertSame(id_1, i.next());
assertTrue(i.hasNext());
assertSame(id_2, i.next());
assertTrue(i.hasNext());
assertSame(id_3, i.next());

assertFalse(i.hasNext());
try {
i.next();
fail("did not fail on next with no next");
} catch (NoSuchElementException expected) {
// OK
}

i = m.iterator();
assertSame(id_1, i.next());
try {
i.remove();
fail("did not fail on remove");
} catch (UnsupportedOperationException expected) {
// OK
}
}

private AnyObjectId id(int val) {
idBuf.setByte(0, val & 0xff);
idBuf.setByte(3, (val >>> 8) & 0xff);
return idBuf;
}

private static class SubId extends ObjectIdOwnerMap.Entry {
SubId(AnyObjectId id) {
super(id);
}
}
}

+ 356
- 0
org.eclipse.jgit/src/org/eclipse/jgit/lib/ObjectIdOwnerMap.java View File

@@ -0,0 +1,356 @@
/*
* Copyright (C) 2011, Google Inc.
* and other copyright owners as documented in the project's IP log.
*
* This program and the accompanying materials are made available
* under the terms of the Eclipse Distribution License v1.0 which
* accompanies this distribution, is reproduced below, and is
* available at http://www.eclipse.org/org/documents/edl-v10.php
*
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or
* without modification, are permitted provided that the following
* conditions are met:
*
* - Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
*
* - Redistributions in binary form must reproduce the above
* copyright notice, this list of conditions and the following
* disclaimer in the documentation and/or other materials provided
* with the distribution.
*
* - Neither the name of the Eclipse Foundation, Inc. nor the
* names of its contributors may be used to endorse or promote
* products derived from this software without specific prior
* written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
* CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
* INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
* OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
* CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
* SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
* NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
* LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
* CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
* STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
* ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/

package org.eclipse.jgit.lib;

import java.util.Arrays;
import java.util.Iterator;
import java.util.NoSuchElementException;

/**
* Fast, efficient map for {@link ObjectId} subclasses in only one map.
* <p>
* To use this map type, applications must have their entry value type extend
* from {@link ObjectIdOwnerMap.Entry}, which itself extends from ObjectId.
* <p>
* Object instances may only be stored in <b>ONE</b> ObjectIdOwnerMap. This
* restriction exists because the map stores internal map state within each
* object instance. If an instance is be placed in another ObjectIdOwnerMap it
* could corrupt one or both map's internal state.
* <p>
* If an object instance must be in more than one map, applications may use
* ObjectIdOwnerMap for one of the maps, and {@link ObjectIdSubclassMap} for the
* other map(s). It is encouraged to use ObjectIdOwnerMap for the map that is
* accessed most often, as this implementation runs faster than the more general
* ObjectIdSubclassMap implementation.
*
* @param <V>
* type of subclass of ObjectId that will be stored in the map.
*/
public class ObjectIdOwnerMap<V extends ObjectIdOwnerMap.Entry> implements
Iterable<V> {
/** Size of the initial directory, will grow as necessary. */
private static final int INITIAL_DIRECTORY = 1024;

/** Number of bits in a segment's index. Segments are 2^11 in size. */
private static final int SEGMENT_BITS = 11;

private static final int SEGMENT_SHIFT = 32 - SEGMENT_BITS;

/**
* Top level directory of the segments.
* <p>
* The low {@link #bits} of the SHA-1 are used to select the segment from
* this directory. Each segment is constant sized at 2^SEGMENT_BITS.
*/
private V[][] directory;

/** Total number of objects in this map. */
private int size;

/** The map doubles in capacity when {@link #size} reaches this target. */
private int grow;

/** Number of low bits used to form the index into {@link #directory}. */
private int bits;

/** Low bit mask to index into {@link #directory}, {@code 2^bits-1}. */
private int mask;

/** Create an empty map. */
@SuppressWarnings("unchecked")
public ObjectIdOwnerMap() {
bits = 0;
mask = 0;
grow = computeGrowAt(bits);

directory = (V[][]) new Entry[INITIAL_DIRECTORY][];
directory[0] = newSegment();
}

/** Remove all entries from this map. */
public void clear() {
size = 0;

for (V[] tbl : directory) {
if (tbl == null)
break;
Arrays.fill(tbl, null);
}
}

/**
* Lookup an existing mapping.
*
* @param toFind
* the object identifier to find.
* @return the instance mapped to toFind, or null if no mapping exists.
*/
@SuppressWarnings("unchecked")
public V get(final AnyObjectId toFind) {
int h = toFind.w1;
V obj = directory[h & mask][h >>> SEGMENT_SHIFT];
for (; obj != null; obj = (V) obj.next)
if (equals(obj, toFind))
return obj;
return null;
}

/**
* Returns true if this map contains the specified object.
*
* @param toFind
* object to find.
* @return true if the mapping exists for this object; false otherwise.
*/
public boolean contains(final AnyObjectId toFind) {
return get(toFind) != null;
}

/**
* Store an object for future lookup.
* <p>
* An existing mapping for <b>must not</b> be in this map. Callers must
* first call {@link #get(AnyObjectId)} to verify there is no current
* mapping prior to adding a new mapping, or use {@link #addIfAbsent(Entry)}.
*
* @param newValue
* the object to store.
* @param <Q>
* type of instance to store.
*/
public <Q extends V> void add(final Q newValue) {
if (++size == grow)
grow();

int h = newValue.w1;
V[] table = directory[h & mask];
h >>>= SEGMENT_SHIFT;

newValue.next = table[h];
table[h] = newValue;
}

/**
* Store an object for future lookup.
* <p>
* Stores {@code newValue}, but only if there is not already an object for
* the same object name. Callers can tell if the value is new by checking
* the return value with reference equality:
*
* <pre>
* V obj = ...;
* boolean wasNew = map.addIfAbsent(obj) == obj;
* </pre>
*
* @param newValue
* the object to store.
* @return {@code newValue} if stored, or the prior value already stored and
* that would have been returned had the caller used
* {@code get(newValue)} first.
* @param <Q>
* type of instance to store.
*/
@SuppressWarnings("unchecked")
public <Q extends V> V addIfAbsent(final Q newValue) {
int h = newValue.w1;
V[] table = directory[h & mask];
h >>>= SEGMENT_SHIFT;

for (V obj = table[h]; obj != null; obj = (V) obj.next)
if (equals(obj, newValue))
return obj;

newValue.next = table[h];
table[h] = newValue;

if (++size == grow)
grow();
return newValue;
}

/** @return number of objects in this map. */
public int size() {
return size;
}

/** @return true if {@link #size()} is 0. */
public boolean isEmpty() {
return size == 0;
}

public Iterator<V> iterator() {
return new Iterator<V>() {
private int found;

private int dirIdx;

private int tblIdx;

private V next;

public boolean hasNext() {
return found < size;
}

public V next() {
if (next != null)
return found(next);

for (;;) {
V[] table = directory[dirIdx];
if (tblIdx == table.length) {
if (++dirIdx >= (1 << bits))
throw new NoSuchElementException();
table = directory[dirIdx];
tblIdx = 0;
}

while (tblIdx < table.length) {
V v = table[tblIdx++];
if (v != null)
return found(v);
}
}
}

@SuppressWarnings("unchecked")
private V found(V v) {
found++;
next = (V) v.next;
return v;
}

public void remove() {
throw new UnsupportedOperationException();
}
};
}

@SuppressWarnings("unchecked")
private void grow() {
final int oldDirLen = 1 << bits;
final int s = 1 << bits;

bits++;
mask = (1 << bits) - 1;
grow = computeGrowAt(bits);

// Quadruple the directory if it needs to expand. Expanding the
// directory is expensive because it generates garbage, so try
// to avoid doing it often.
//
final int newDirLen = 1 << bits;
if (directory.length < newDirLen) {
V[][] newDir = (V[][]) new Entry[newDirLen << 1][];
System.arraycopy(directory, 0, newDir, 0, oldDirLen);
directory = newDir;
}

// For every bucket of every old segment, split the chain between
// the old segment and the new segment's corresponding bucket. To
// select between them use the lowest bit that was just added into
// the mask above. This causes the table to double in capacity.
//
for (int dirIdx = 0; dirIdx < oldDirLen; dirIdx++) {
final V[] oldTable = directory[dirIdx];
final V[] newTable = newSegment();

for (int i = 0; i < oldTable.length; i++) {
V chain0 = null;
V chain1 = null;
V next;

for (V obj = oldTable[i]; obj != null; obj = next) {
next = (V) obj.next;

if ((obj.w1 & s) == 0) {
obj.next = chain0;
chain0 = obj;
} else {
obj.next = chain1;
chain1 = obj;
}
}

oldTable[i] = chain0;
newTable[i] = chain1;
}

directory[oldDirLen + dirIdx] = newTable;
}
}

@SuppressWarnings("unchecked")
private final V[] newSegment() {
return (V[]) new Entry[1 << SEGMENT_BITS];
}

private static final int computeGrowAt(int bits) {
return 1 << (bits + SEGMENT_BITS);
}

private static final boolean equals(AnyObjectId firstObjectId,
AnyObjectId secondObjectId) {
return firstObjectId.w2 == secondObjectId.w2
&& firstObjectId.w3 == secondObjectId.w3
&& firstObjectId.w4 == secondObjectId.w4
&& firstObjectId.w5 == secondObjectId.w5
&& firstObjectId.w1 == secondObjectId.w1;
}

/** Type of entry stored in the {@link ObjectIdOwnerMap}. */
public static abstract class Entry extends ObjectId {
Entry next;

/**
* Initialize this entry with a specific ObjectId.
*
* @param id
* the id the entry represents.
*/
public Entry(AnyObjectId id) {
super(id);
}
}
}

+ 2
- 4
org.eclipse.jgit/src/org/eclipse/jgit/lib/ObjectIdSubclassMap.java View File

@@ -54,10 +54,8 @@ import java.util.NoSuchElementException;
* This map provides an efficient translation from any ObjectId instance to a
* cached subclass of ObjectId that has the same value.
* <p>
* Raw value equality is tested when comparing two ObjectIds (or subclasses),
* not reference equality and not <code>.equals(Object)</code> equality. This
* allows subclasses to override <code>equals</code> to supply their own
* extended semantics.
* If object instances are stored in only one map, {@link ObjectIdOwnerMap} is a
* more efficient implementation.
*
* @param <V>
* type of subclass of ObjectId that will be stored in the map.

+ 2
- 1
org.eclipse.jgit/src/org/eclipse/jgit/revwalk/RevObject.java View File

@@ -50,9 +50,10 @@ import org.eclipse.jgit.errors.MissingObjectException;
import org.eclipse.jgit.lib.AnyObjectId;
import org.eclipse.jgit.lib.Constants;
import org.eclipse.jgit.lib.ObjectId;
import org.eclipse.jgit.lib.ObjectIdOwnerMap;

/** Base object type accessed during revision walking. */
public abstract class RevObject extends ObjectId {
public abstract class RevObject extends ObjectIdOwnerMap.Entry {
static final int PARSED = 1;

int flags;

+ 3
- 3
org.eclipse.jgit/src/org/eclipse/jgit/revwalk/RevWalk.java View File

@@ -63,7 +63,7 @@ import org.eclipse.jgit.lib.AsyncObjectLoaderQueue;
import org.eclipse.jgit.lib.Constants;
import org.eclipse.jgit.lib.MutableObjectId;
import org.eclipse.jgit.lib.ObjectId;
import org.eclipse.jgit.lib.ObjectIdSubclassMap;
import org.eclipse.jgit.lib.ObjectIdOwnerMap;
import org.eclipse.jgit.lib.ObjectLoader;
import org.eclipse.jgit.lib.ObjectReader;
import org.eclipse.jgit.lib.Repository;
@@ -170,7 +170,7 @@ public class RevWalk implements Iterable<RevCommit> {

final MutableObjectId idBuffer;

private ObjectIdSubclassMap<RevObject> objects;
private ObjectIdOwnerMap<RevObject> objects;

private int freeFlags = APP_FLAGS;

@@ -220,7 +220,7 @@ public class RevWalk implements Iterable<RevCommit> {
repository = repo;
reader = or;
idBuffer = new MutableObjectId();
objects = new ObjectIdSubclassMap<RevObject>();
objects = new ObjectIdOwnerMap<RevObject>();
roots = new ArrayList<RevCommit>();
queue = new DateRevQueue();
pending = new StartGenerator(this);

+ 11
- 4
org.eclipse.jgit/src/org/eclipse/jgit/storage/file/CachedObjectDirectory.java View File

@@ -55,7 +55,7 @@ import org.eclipse.jgit.lib.Config;
import org.eclipse.jgit.lib.Constants;
import org.eclipse.jgit.lib.ObjectDatabase;
import org.eclipse.jgit.lib.ObjectId;
import org.eclipse.jgit.lib.ObjectIdSubclassMap;
import org.eclipse.jgit.lib.ObjectIdOwnerMap;
import org.eclipse.jgit.lib.ObjectLoader;
import org.eclipse.jgit.storage.pack.CachedPack;
import org.eclipse.jgit.storage.pack.ObjectToPack;
@@ -73,7 +73,7 @@ class CachedObjectDirectory extends FileObjectDatabase {
* The set that contains unpacked objects identifiers, it is created when
* the cached instance is created.
*/
private final ObjectIdSubclassMap<ObjectId> unpackedObjects = new ObjectIdSubclassMap<ObjectId>();
private final ObjectIdOwnerMap<UnpackedObjectId> unpackedObjects = new ObjectIdOwnerMap<UnpackedObjectId>();

private final ObjectDirectory wrapped;

@@ -102,7 +102,8 @@ class CachedObjectDirectory extends FileObjectDatabase {
if (e.length() != Constants.OBJECT_ID_STRING_LENGTH - 2)
continue;
try {
unpackedObjects.add(ObjectId.fromString(d + e));
ObjectId id = ObjectId.fromString(d + e);
unpackedObjects.add(new UnpackedObjectId(id));
} catch (IllegalArgumentException notAnObject) {
// ignoring the file that does not represent loose object
}
@@ -235,7 +236,7 @@ class CachedObjectDirectory extends FileObjectDatabase {
switch (result) {
case INSERTED:
case EXISTS_LOOSE:
unpackedObjects.addIfAbsent(objectId);
unpackedObjects.addIfAbsent(new UnpackedObjectId(objectId));
break;

case EXISTS_PACKED:
@@ -255,4 +256,10 @@ class CachedObjectDirectory extends FileObjectDatabase {
WindowCursor curs) throws IOException {
wrapped.selectObjectRepresentation(packer, otp, curs);
}

private static class UnpackedObjectId extends ObjectIdOwnerMap.Entry {
UnpackedObjectId(AnyObjectId id) {
super(id);
}
}
}

+ 6
- 6
org.eclipse.jgit/src/org/eclipse/jgit/storage/pack/BaseSearch.java View File

@@ -56,7 +56,7 @@ import org.eclipse.jgit.lib.AnyObjectId;
import org.eclipse.jgit.lib.FileMode;
import org.eclipse.jgit.lib.MutableObjectId;
import org.eclipse.jgit.lib.ObjectId;
import org.eclipse.jgit.lib.ObjectIdSubclassMap;
import org.eclipse.jgit.lib.ObjectIdOwnerMap;
import org.eclipse.jgit.lib.ObjectLoader;
import org.eclipse.jgit.lib.ObjectReader;
import org.eclipse.jgit.lib.ProgressMonitor;
@@ -74,20 +74,20 @@ class BaseSearch {

private final ObjectId[] baseTrees;

private final ObjectIdSubclassMap<ObjectToPack> objectsMap;
private final ObjectIdOwnerMap<ObjectToPack> objectsMap;

private final List<ObjectToPack> edgeObjects;

private final IntSet alreadyProcessed;

private final ObjectIdSubclassMap<TreeWithData> treeCache;
private final ObjectIdOwnerMap<TreeWithData> treeCache;

private final CanonicalTreeParser parser;

private final MutableObjectId idBuf;

BaseSearch(ProgressMonitor countingMonitor, Set<RevTree> bases,
ObjectIdSubclassMap<ObjectToPack> objects,
ObjectIdOwnerMap<ObjectToPack> objects,
List<ObjectToPack> edges, ObjectReader or) {
progress = countingMonitor;
reader = or;
@@ -96,7 +96,7 @@ class BaseSearch {
edgeObjects = edges;

alreadyProcessed = new IntSet();
treeCache = new ObjectIdSubclassMap<TreeWithData>();
treeCache = new ObjectIdOwnerMap<TreeWithData>();
parser = new CanonicalTreeParser();
idBuf = new MutableObjectId();
}
@@ -198,7 +198,7 @@ class BaseSearch {
return buf;
}

private static class TreeWithData extends ObjectId {
private static class TreeWithData extends ObjectIdOwnerMap.Entry {
final byte[] buf;

TreeWithData(AnyObjectId id, byte[] buf) {

+ 2
- 2
org.eclipse.jgit/src/org/eclipse/jgit/storage/pack/PackWriter.java View File

@@ -83,7 +83,7 @@ import org.eclipse.jgit.lib.AsyncObjectSizeQueue;
import org.eclipse.jgit.lib.Constants;
import org.eclipse.jgit.lib.NullProgressMonitor;
import org.eclipse.jgit.lib.ObjectId;
import org.eclipse.jgit.lib.ObjectIdSubclassMap;
import org.eclipse.jgit.lib.ObjectIdOwnerMap;
import org.eclipse.jgit.lib.ObjectLoader;
import org.eclipse.jgit.lib.ObjectReader;
import org.eclipse.jgit.lib.ProgressMonitor;
@@ -148,7 +148,7 @@ public class PackWriter {
objectsLists[Constants.OBJ_TAG] = new BlockList<ObjectToPack>();
}

private final ObjectIdSubclassMap<ObjectToPack> objectsMap = new ObjectIdSubclassMap<ObjectToPack>();
private final ObjectIdOwnerMap<ObjectToPack> objectsMap = new ObjectIdOwnerMap<ObjectToPack>();

// edge objects for thin packs
private List<ObjectToPack> edgeObjects = new BlockList<ObjectToPack>();

+ 4
- 3
org.eclipse.jgit/src/org/eclipse/jgit/transport/PackParser.java View File

@@ -68,6 +68,7 @@ import org.eclipse.jgit.lib.NullProgressMonitor;
import org.eclipse.jgit.lib.ObjectChecker;
import org.eclipse.jgit.lib.ObjectDatabase;
import org.eclipse.jgit.lib.ObjectId;
import org.eclipse.jgit.lib.ObjectIdOwnerMap;
import org.eclipse.jgit.lib.ObjectIdSubclassMap;
import org.eclipse.jgit.lib.ObjectInserter;
import org.eclipse.jgit.lib.ObjectLoader;
@@ -152,7 +153,7 @@ public abstract class PackParser {

private int entryCount;

private ObjectIdSubclassMap<DeltaChain> baseById;
private ObjectIdOwnerMap<DeltaChain> baseById;

/**
* Objects referenced by their name from deltas, that aren't in this pack.
@@ -438,7 +439,7 @@ public abstract class PackParser {
readPackHeader();

entries = new PackedObjectInfo[(int) objectCount];
baseById = new ObjectIdSubclassMap<DeltaChain>();
baseById = new ObjectIdOwnerMap<DeltaChain>();
baseByPos = new LongMap<UnresolvedDelta>();
deferredCheckBlobs = new BlockList<PackedObjectInfo>();

@@ -1369,7 +1370,7 @@ public abstract class PackParser {
return inflater;
}

private static class DeltaChain extends ObjectId {
private static class DeltaChain extends ObjectIdOwnerMap.Entry {
UnresolvedDelta head;

DeltaChain(final AnyObjectId id) {

+ 2
- 2
org.eclipse.jgit/src/org/eclipse/jgit/transport/PackedObjectInfo.java View File

@@ -45,7 +45,7 @@
package org.eclipse.jgit.transport;

import org.eclipse.jgit.lib.AnyObjectId;
import org.eclipse.jgit.lib.ObjectId;
import org.eclipse.jgit.lib.ObjectIdOwnerMap;

/**
* Description of an object stored in a pack file, including offset.
@@ -54,7 +54,7 @@ import org.eclipse.jgit.lib.ObjectId;
* (starting position of the object data) to perform random-access reads of
* objects from the pack. This extension of ObjectId includes the offset.
*/
public class PackedObjectInfo extends ObjectId {
public class PackedObjectInfo extends ObjectIdOwnerMap.Entry {
private long offset;

private int crc;

Loading…
Cancel
Save