You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

PackConfig.java 23KB

Support creating pack bitmap indexes in PackWriter. Update the PackWriter to support writing out pack bitmap indexes, a parallel ".bitmap" file to the ".pack" file. Bitmaps are selected at commits every 1 to 5,000 commits for each unique path from the start. The most recent 100 commits are all bitmapped. The next 19,000 commits have a bitmaps every 100 commits. The remaining commits have a bitmap every 5,000 commits. Commits with more than 1 parent are prefered over ones with 1 or less. Furthermore, previously computed bitmaps are reused, if the previous entry had the reuse flag set, which is set when the bitmap was placed at the max allowed distance. Bitmaps are used to speed up the counting phase when packing, for requests that are not shallow. The PackWriterBitmapWalker uses a RevFilter to proactively mark commits with RevFlag.SEEN, when they appear in a bitmap. The walker produces the full closure of reachable ObjectIds, given the collection of starting ObjectIds. For fetch request, two ObjectWalks are executed to compute the ObjectIds reachable from the haves and from the wants. The ObjectIds needed to be written are determined by taking all the resulting wants AND NOT the haves. For clone requests, we get cached pack support for "free" since it is possible to determine if all of the ObjectIds in a pack file are included in the resulting list of ObjectIds to write. On my machine, the best times for clones and fetches of the linux kernel repository (with about 2.6M objects and 300K commits) are tabulated below: Operation Index V2 Index VE003 Clone 37530ms (524.06 MiB) 82ms (524.06 MiB) Fetch (1 commit back) 75ms 107ms Fetch (10 commits back) 456ms (269.51 KiB) 341ms (265.19 KiB) Fetch (100 commits back) 449ms (269.91 KiB) 337ms (267.28 KiB) Fetch (1000 commits back) 2229ms ( 14.75 MiB) 189ms ( 14.42 MiB) Fetch (10000 commits back) 2177ms ( 16.30 MiB) 254ms ( 15.88 MiB) Fetch (100000 commits back) 14340ms (185.83 MiB) 1655ms (189.39 MiB) Change-Id: Icdb0cdd66ff168917fb9ef17b96093990cc6a98d
11 years ago
Support creating pack bitmap indexes in PackWriter. Update the PackWriter to support writing out pack bitmap indexes, a parallel ".bitmap" file to the ".pack" file. Bitmaps are selected at commits every 1 to 5,000 commits for each unique path from the start. The most recent 100 commits are all bitmapped. The next 19,000 commits have a bitmaps every 100 commits. The remaining commits have a bitmap every 5,000 commits. Commits with more than 1 parent are prefered over ones with 1 or less. Furthermore, previously computed bitmaps are reused, if the previous entry had the reuse flag set, which is set when the bitmap was placed at the max allowed distance. Bitmaps are used to speed up the counting phase when packing, for requests that are not shallow. The PackWriterBitmapWalker uses a RevFilter to proactively mark commits with RevFlag.SEEN, when they appear in a bitmap. The walker produces the full closure of reachable ObjectIds, given the collection of starting ObjectIds. For fetch request, two ObjectWalks are executed to compute the ObjectIds reachable from the haves and from the wants. The ObjectIds needed to be written are determined by taking all the resulting wants AND NOT the haves. For clone requests, we get cached pack support for "free" since it is possible to determine if all of the ObjectIds in a pack file are included in the resulting list of ObjectIds to write. On my machine, the best times for clones and fetches of the linux kernel repository (with about 2.6M objects and 300K commits) are tabulated below: Operation Index V2 Index VE003 Clone 37530ms (524.06 MiB) 82ms (524.06 MiB) Fetch (1 commit back) 75ms 107ms Fetch (10 commits back) 456ms (269.51 KiB) 341ms (265.19 KiB) Fetch (100 commits back) 449ms (269.91 KiB) 337ms (267.28 KiB) Fetch (1000 commits back) 2229ms ( 14.75 MiB) 189ms ( 14.42 MiB) Fetch (10000 commits back) 2177ms ( 16.30 MiB) 254ms ( 15.88 MiB) Fetch (100000 commits back) 14340ms (185.83 MiB) 1655ms (189.39 MiB) Change-Id: Icdb0cdd66ff168917fb9ef17b96093990cc6a98d
11 years ago
Support creating pack bitmap indexes in PackWriter. Update the PackWriter to support writing out pack bitmap indexes, a parallel ".bitmap" file to the ".pack" file. Bitmaps are selected at commits every 1 to 5,000 commits for each unique path from the start. The most recent 100 commits are all bitmapped. The next 19,000 commits have a bitmaps every 100 commits. The remaining commits have a bitmap every 5,000 commits. Commits with more than 1 parent are prefered over ones with 1 or less. Furthermore, previously computed bitmaps are reused, if the previous entry had the reuse flag set, which is set when the bitmap was placed at the max allowed distance. Bitmaps are used to speed up the counting phase when packing, for requests that are not shallow. The PackWriterBitmapWalker uses a RevFilter to proactively mark commits with RevFlag.SEEN, when they appear in a bitmap. The walker produces the full closure of reachable ObjectIds, given the collection of starting ObjectIds. For fetch request, two ObjectWalks are executed to compute the ObjectIds reachable from the haves and from the wants. The ObjectIds needed to be written are determined by taking all the resulting wants AND NOT the haves. For clone requests, we get cached pack support for "free" since it is possible to determine if all of the ObjectIds in a pack file are included in the resulting list of ObjectIds to write. On my machine, the best times for clones and fetches of the linux kernel repository (with about 2.6M objects and 300K commits) are tabulated below: Operation Index V2 Index VE003 Clone 37530ms (524.06 MiB) 82ms (524.06 MiB) Fetch (1 commit back) 75ms 107ms Fetch (10 commits back) 456ms (269.51 KiB) 341ms (265.19 KiB) Fetch (100 commits back) 449ms (269.91 KiB) 337ms (267.28 KiB) Fetch (1000 commits back) 2229ms ( 14.75 MiB) 189ms ( 14.42 MiB) Fetch (10000 commits back) 2177ms ( 16.30 MiB) 254ms ( 15.88 MiB) Fetch (100000 commits back) 14340ms (185.83 MiB) 1655ms (189.39 MiB) Change-Id: Icdb0cdd66ff168917fb9ef17b96093990cc6a98d
11 years ago
Support creating pack bitmap indexes in PackWriter. Update the PackWriter to support writing out pack bitmap indexes, a parallel ".bitmap" file to the ".pack" file. Bitmaps are selected at commits every 1 to 5,000 commits for each unique path from the start. The most recent 100 commits are all bitmapped. The next 19,000 commits have a bitmaps every 100 commits. The remaining commits have a bitmap every 5,000 commits. Commits with more than 1 parent are prefered over ones with 1 or less. Furthermore, previously computed bitmaps are reused, if the previous entry had the reuse flag set, which is set when the bitmap was placed at the max allowed distance. Bitmaps are used to speed up the counting phase when packing, for requests that are not shallow. The PackWriterBitmapWalker uses a RevFilter to proactively mark commits with RevFlag.SEEN, when they appear in a bitmap. The walker produces the full closure of reachable ObjectIds, given the collection of starting ObjectIds. For fetch request, two ObjectWalks are executed to compute the ObjectIds reachable from the haves and from the wants. The ObjectIds needed to be written are determined by taking all the resulting wants AND NOT the haves. For clone requests, we get cached pack support for "free" since it is possible to determine if all of the ObjectIds in a pack file are included in the resulting list of ObjectIds to write. On my machine, the best times for clones and fetches of the linux kernel repository (with about 2.6M objects and 300K commits) are tabulated below: Operation Index V2 Index VE003 Clone 37530ms (524.06 MiB) 82ms (524.06 MiB) Fetch (1 commit back) 75ms 107ms Fetch (10 commits back) 456ms (269.51 KiB) 341ms (265.19 KiB) Fetch (100 commits back) 449ms (269.91 KiB) 337ms (267.28 KiB) Fetch (1000 commits back) 2229ms ( 14.75 MiB) 189ms ( 14.42 MiB) Fetch (10000 commits back) 2177ms ( 16.30 MiB) 254ms ( 15.88 MiB) Fetch (100000 commits back) 14340ms (185.83 MiB) 1655ms (189.39 MiB) Change-Id: Icdb0cdd66ff168917fb9ef17b96093990cc6a98d
11 years ago
Support cutting existing delta chains longer than the max depth Some packs built by JGit have incredibly long delta chains due to a long standing bug in PackWriter. Google has packs created by JGit's DfsGarbageCollector with chains of 6000 objects long, or more. Inflating objects at the end of this 6000 long chain is impossible to complete within a reasonable time bound. It could take a beefy system hours to perform even using the heavily optimized native C implementation of Git, let alone with JGit. Enable pack.cutDeltaChains to be set in a configuration file to permit the PackWriter to determine the length of each delta chain and clip the chain at arbitrary points to fit within pack.depth. Delta chain cycles are still possible, but no attempt is made to detect them. A trivial chain of A->B->A will iterate for the full pack.depth configured limit (e.g. 50) and then pick an object to store as non-delta. When cutting chains the object list is walked in reverse to try and take advantage of existing chain computations. The assumption here is most deltas are near the end of the list, and their bases are near the front of the list. Going up from the tail attempts to reuse chainLength computations by relying on the memoized value in the delta base. The chainLength field in ObjectToPack is overloaded into the depth field normally used by DeltaWindow. This is acceptable because the chain cut happens before delta search, and the chainLength is reset to 0 if delta search will follow. Change-Id: Ida4fde9558f3abbbb77ade398d2af3941de9c812
11 years ago
Support creating pack bitmap indexes in PackWriter. Update the PackWriter to support writing out pack bitmap indexes, a parallel ".bitmap" file to the ".pack" file. Bitmaps are selected at commits every 1 to 5,000 commits for each unique path from the start. The most recent 100 commits are all bitmapped. The next 19,000 commits have a bitmaps every 100 commits. The remaining commits have a bitmap every 5,000 commits. Commits with more than 1 parent are prefered over ones with 1 or less. Furthermore, previously computed bitmaps are reused, if the previous entry had the reuse flag set, which is set when the bitmap was placed at the max allowed distance. Bitmaps are used to speed up the counting phase when packing, for requests that are not shallow. The PackWriterBitmapWalker uses a RevFilter to proactively mark commits with RevFlag.SEEN, when they appear in a bitmap. The walker produces the full closure of reachable ObjectIds, given the collection of starting ObjectIds. For fetch request, two ObjectWalks are executed to compute the ObjectIds reachable from the haves and from the wants. The ObjectIds needed to be written are determined by taking all the resulting wants AND NOT the haves. For clone requests, we get cached pack support for "free" since it is possible to determine if all of the ObjectIds in a pack file are included in the resulting list of ObjectIds to write. On my machine, the best times for clones and fetches of the linux kernel repository (with about 2.6M objects and 300K commits) are tabulated below: Operation Index V2 Index VE003 Clone 37530ms (524.06 MiB) 82ms (524.06 MiB) Fetch (1 commit back) 75ms 107ms Fetch (10 commits back) 456ms (269.51 KiB) 341ms (265.19 KiB) Fetch (100 commits back) 449ms (269.91 KiB) 337ms (267.28 KiB) Fetch (1000 commits back) 2229ms ( 14.75 MiB) 189ms ( 14.42 MiB) Fetch (10000 commits back) 2177ms ( 16.30 MiB) 254ms ( 15.88 MiB) Fetch (100000 commits back) 14340ms (185.83 MiB) 1655ms (189.39 MiB) Change-Id: Icdb0cdd66ff168917fb9ef17b96093990cc6a98d
11 years ago
Support cutting existing delta chains longer than the max depth Some packs built by JGit have incredibly long delta chains due to a long standing bug in PackWriter. Google has packs created by JGit's DfsGarbageCollector with chains of 6000 objects long, or more. Inflating objects at the end of this 6000 long chain is impossible to complete within a reasonable time bound. It could take a beefy system hours to perform even using the heavily optimized native C implementation of Git, let alone with JGit. Enable pack.cutDeltaChains to be set in a configuration file to permit the PackWriter to determine the length of each delta chain and clip the chain at arbitrary points to fit within pack.depth. Delta chain cycles are still possible, but no attempt is made to detect them. A trivial chain of A->B->A will iterate for the full pack.depth configured limit (e.g. 50) and then pick an object to store as non-delta. When cutting chains the object list is walked in reverse to try and take advantage of existing chain computations. The assumption here is most deltas are near the end of the list, and their bases are near the front of the list. Going up from the tail attempts to reuse chainLength computations by relying on the memoized value in the delta base. The chainLength field in ObjectToPack is overloaded into the depth field normally used by DeltaWindow. This is acceptable because the chain cut happens before delta search, and the chainLength is reset to 0 if delta search will follow. Change-Id: Ida4fde9558f3abbbb77ade398d2af3941de9c812
11 years ago
Support cutting existing delta chains longer than the max depth Some packs built by JGit have incredibly long delta chains due to a long standing bug in PackWriter. Google has packs created by JGit's DfsGarbageCollector with chains of 6000 objects long, or more. Inflating objects at the end of this 6000 long chain is impossible to complete within a reasonable time bound. It could take a beefy system hours to perform even using the heavily optimized native C implementation of Git, let alone with JGit. Enable pack.cutDeltaChains to be set in a configuration file to permit the PackWriter to determine the length of each delta chain and clip the chain at arbitrary points to fit within pack.depth. Delta chain cycles are still possible, but no attempt is made to detect them. A trivial chain of A->B->A will iterate for the full pack.depth configured limit (e.g. 50) and then pick an object to store as non-delta. When cutting chains the object list is walked in reverse to try and take advantage of existing chain computations. The assumption here is most deltas are near the end of the list, and their bases are near the front of the list. Going up from the tail attempts to reuse chainLength computations by relying on the memoized value in the delta base. The chainLength field in ObjectToPack is overloaded into the depth field normally used by DeltaWindow. This is acceptable because the chain cut happens before delta search, and the chainLength is reset to 0 if delta search will follow. Change-Id: Ida4fde9558f3abbbb77ade398d2af3941de9c812
11 years ago
Support cutting existing delta chains longer than the max depth Some packs built by JGit have incredibly long delta chains due to a long standing bug in PackWriter. Google has packs created by JGit's DfsGarbageCollector with chains of 6000 objects long, or more. Inflating objects at the end of this 6000 long chain is impossible to complete within a reasonable time bound. It could take a beefy system hours to perform even using the heavily optimized native C implementation of Git, let alone with JGit. Enable pack.cutDeltaChains to be set in a configuration file to permit the PackWriter to determine the length of each delta chain and clip the chain at arbitrary points to fit within pack.depth. Delta chain cycles are still possible, but no attempt is made to detect them. A trivial chain of A->B->A will iterate for the full pack.depth configured limit (e.g. 50) and then pick an object to store as non-delta. When cutting chains the object list is walked in reverse to try and take advantage of existing chain computations. The assumption here is most deltas are near the end of the list, and their bases are near the front of the list. Going up from the tail attempts to reuse chainLength computations by relying on the memoized value in the delta base. The chainLength field in ObjectToPack is overloaded into the depth field normally used by DeltaWindow. This is acceptable because the chain cut happens before delta search, and the chainLength is reset to 0 if delta search will follow. Change-Id: Ida4fde9558f3abbbb77ade398d2af3941de9c812
11 years ago
Support cutting existing delta chains longer than the max depth Some packs built by JGit have incredibly long delta chains due to a long standing bug in PackWriter. Google has packs created by JGit's DfsGarbageCollector with chains of 6000 objects long, or more. Inflating objects at the end of this 6000 long chain is impossible to complete within a reasonable time bound. It could take a beefy system hours to perform even using the heavily optimized native C implementation of Git, let alone with JGit. Enable pack.cutDeltaChains to be set in a configuration file to permit the PackWriter to determine the length of each delta chain and clip the chain at arbitrary points to fit within pack.depth. Delta chain cycles are still possible, but no attempt is made to detect them. A trivial chain of A->B->A will iterate for the full pack.depth configured limit (e.g. 50) and then pick an object to store as non-delta. When cutting chains the object list is walked in reverse to try and take advantage of existing chain computations. The assumption here is most deltas are near the end of the list, and their bases are near the front of the list. Going up from the tail attempts to reuse chainLength computations by relying on the memoized value in the delta base. The chainLength field in ObjectToPack is overloaded into the depth field normally used by DeltaWindow. This is acceptable because the chain cut happens before delta search, and the chainLength is reset to 0 if delta search will follow. Change-Id: Ida4fde9558f3abbbb77ade398d2af3941de9c812
11 years ago
Support creating pack bitmap indexes in PackWriter. Update the PackWriter to support writing out pack bitmap indexes, a parallel ".bitmap" file to the ".pack" file. Bitmaps are selected at commits every 1 to 5,000 commits for each unique path from the start. The most recent 100 commits are all bitmapped. The next 19,000 commits have a bitmaps every 100 commits. The remaining commits have a bitmap every 5,000 commits. Commits with more than 1 parent are prefered over ones with 1 or less. Furthermore, previously computed bitmaps are reused, if the previous entry had the reuse flag set, which is set when the bitmap was placed at the max allowed distance. Bitmaps are used to speed up the counting phase when packing, for requests that are not shallow. The PackWriterBitmapWalker uses a RevFilter to proactively mark commits with RevFlag.SEEN, when they appear in a bitmap. The walker produces the full closure of reachable ObjectIds, given the collection of starting ObjectIds. For fetch request, two ObjectWalks are executed to compute the ObjectIds reachable from the haves and from the wants. The ObjectIds needed to be written are determined by taking all the resulting wants AND NOT the haves. For clone requests, we get cached pack support for "free" since it is possible to determine if all of the ObjectIds in a pack file are included in the resulting list of ObjectIds to write. On my machine, the best times for clones and fetches of the linux kernel repository (with about 2.6M objects and 300K commits) are tabulated below: Operation Index V2 Index VE003 Clone 37530ms (524.06 MiB) 82ms (524.06 MiB) Fetch (1 commit back) 75ms 107ms Fetch (10 commits back) 456ms (269.51 KiB) 341ms (265.19 KiB) Fetch (100 commits back) 449ms (269.91 KiB) 337ms (267.28 KiB) Fetch (1000 commits back) 2229ms ( 14.75 MiB) 189ms ( 14.42 MiB) Fetch (10000 commits back) 2177ms ( 16.30 MiB) 254ms ( 15.88 MiB) Fetch (100000 commits back) 14340ms (185.83 MiB) 1655ms (189.39 MiB) Change-Id: Icdb0cdd66ff168917fb9ef17b96093990cc6a98d
11 years ago
Support creating pack bitmap indexes in PackWriter. Update the PackWriter to support writing out pack bitmap indexes, a parallel ".bitmap" file to the ".pack" file. Bitmaps are selected at commits every 1 to 5,000 commits for each unique path from the start. The most recent 100 commits are all bitmapped. The next 19,000 commits have a bitmaps every 100 commits. The remaining commits have a bitmap every 5,000 commits. Commits with more than 1 parent are prefered over ones with 1 or less. Furthermore, previously computed bitmaps are reused, if the previous entry had the reuse flag set, which is set when the bitmap was placed at the max allowed distance. Bitmaps are used to speed up the counting phase when packing, for requests that are not shallow. The PackWriterBitmapWalker uses a RevFilter to proactively mark commits with RevFlag.SEEN, when they appear in a bitmap. The walker produces the full closure of reachable ObjectIds, given the collection of starting ObjectIds. For fetch request, two ObjectWalks are executed to compute the ObjectIds reachable from the haves and from the wants. The ObjectIds needed to be written are determined by taking all the resulting wants AND NOT the haves. For clone requests, we get cached pack support for "free" since it is possible to determine if all of the ObjectIds in a pack file are included in the resulting list of ObjectIds to write. On my machine, the best times for clones and fetches of the linux kernel repository (with about 2.6M objects and 300K commits) are tabulated below: Operation Index V2 Index VE003 Clone 37530ms (524.06 MiB) 82ms (524.06 MiB) Fetch (1 commit back) 75ms 107ms Fetch (10 commits back) 456ms (269.51 KiB) 341ms (265.19 KiB) Fetch (100 commits back) 449ms (269.91 KiB) 337ms (267.28 KiB) Fetch (1000 commits back) 2229ms ( 14.75 MiB) 189ms ( 14.42 MiB) Fetch (10000 commits back) 2177ms ( 16.30 MiB) 254ms ( 15.88 MiB) Fetch (100000 commits back) 14340ms (185.83 MiB) 1655ms (189.39 MiB) Change-Id: Icdb0cdd66ff168917fb9ef17b96093990cc6a98d
11 years ago
Support creating pack bitmap indexes in PackWriter. Update the PackWriter to support writing out pack bitmap indexes, a parallel ".bitmap" file to the ".pack" file. Bitmaps are selected at commits every 1 to 5,000 commits for each unique path from the start. The most recent 100 commits are all bitmapped. The next 19,000 commits have a bitmaps every 100 commits. The remaining commits have a bitmap every 5,000 commits. Commits with more than 1 parent are prefered over ones with 1 or less. Furthermore, previously computed bitmaps are reused, if the previous entry had the reuse flag set, which is set when the bitmap was placed at the max allowed distance. Bitmaps are used to speed up the counting phase when packing, for requests that are not shallow. The PackWriterBitmapWalker uses a RevFilter to proactively mark commits with RevFlag.SEEN, when they appear in a bitmap. The walker produces the full closure of reachable ObjectIds, given the collection of starting ObjectIds. For fetch request, two ObjectWalks are executed to compute the ObjectIds reachable from the haves and from the wants. The ObjectIds needed to be written are determined by taking all the resulting wants AND NOT the haves. For clone requests, we get cached pack support for "free" since it is possible to determine if all of the ObjectIds in a pack file are included in the resulting list of ObjectIds to write. On my machine, the best times for clones and fetches of the linux kernel repository (with about 2.6M objects and 300K commits) are tabulated below: Operation Index V2 Index VE003 Clone 37530ms (524.06 MiB) 82ms (524.06 MiB) Fetch (1 commit back) 75ms 107ms Fetch (10 commits back) 456ms (269.51 KiB) 341ms (265.19 KiB) Fetch (100 commits back) 449ms (269.91 KiB) 337ms (267.28 KiB) Fetch (1000 commits back) 2229ms ( 14.75 MiB) 189ms ( 14.42 MiB) Fetch (10000 commits back) 2177ms ( 16.30 MiB) 254ms ( 15.88 MiB) Fetch (100000 commits back) 14340ms (185.83 MiB) 1655ms (189.39 MiB) Change-Id: Icdb0cdd66ff168917fb9ef17b96093990cc6a98d
11 years ago
Support cutting existing delta chains longer than the max depth Some packs built by JGit have incredibly long delta chains due to a long standing bug in PackWriter. Google has packs created by JGit's DfsGarbageCollector with chains of 6000 objects long, or more. Inflating objects at the end of this 6000 long chain is impossible to complete within a reasonable time bound. It could take a beefy system hours to perform even using the heavily optimized native C implementation of Git, let alone with JGit. Enable pack.cutDeltaChains to be set in a configuration file to permit the PackWriter to determine the length of each delta chain and clip the chain at arbitrary points to fit within pack.depth. Delta chain cycles are still possible, but no attempt is made to detect them. A trivial chain of A->B->A will iterate for the full pack.depth configured limit (e.g. 50) and then pick an object to store as non-delta. When cutting chains the object list is walked in reverse to try and take advantage of existing chain computations. The assumption here is most deltas are near the end of the list, and their bases are near the front of the list. Going up from the tail attempts to reuse chainLength computations by relying on the memoized value in the delta base. The chainLength field in ObjectToPack is overloaded into the depth field normally used by DeltaWindow. This is acceptable because the chain cut happens before delta search, and the chainLength is reset to 0 if delta search will follow. Change-Id: Ida4fde9558f3abbbb77ade398d2af3941de9c812
11 years ago
Support creating pack bitmap indexes in PackWriter. Update the PackWriter to support writing out pack bitmap indexes, a parallel ".bitmap" file to the ".pack" file. Bitmaps are selected at commits every 1 to 5,000 commits for each unique path from the start. The most recent 100 commits are all bitmapped. The next 19,000 commits have a bitmaps every 100 commits. The remaining commits have a bitmap every 5,000 commits. Commits with more than 1 parent are prefered over ones with 1 or less. Furthermore, previously computed bitmaps are reused, if the previous entry had the reuse flag set, which is set when the bitmap was placed at the max allowed distance. Bitmaps are used to speed up the counting phase when packing, for requests that are not shallow. The PackWriterBitmapWalker uses a RevFilter to proactively mark commits with RevFlag.SEEN, when they appear in a bitmap. The walker produces the full closure of reachable ObjectIds, given the collection of starting ObjectIds. For fetch request, two ObjectWalks are executed to compute the ObjectIds reachable from the haves and from the wants. The ObjectIds needed to be written are determined by taking all the resulting wants AND NOT the haves. For clone requests, we get cached pack support for "free" since it is possible to determine if all of the ObjectIds in a pack file are included in the resulting list of ObjectIds to write. On my machine, the best times for clones and fetches of the linux kernel repository (with about 2.6M objects and 300K commits) are tabulated below: Operation Index V2 Index VE003 Clone 37530ms (524.06 MiB) 82ms (524.06 MiB) Fetch (1 commit back) 75ms 107ms Fetch (10 commits back) 456ms (269.51 KiB) 341ms (265.19 KiB) Fetch (100 commits back) 449ms (269.91 KiB) 337ms (267.28 KiB) Fetch (1000 commits back) 2229ms ( 14.75 MiB) 189ms ( 14.42 MiB) Fetch (10000 commits back) 2177ms ( 16.30 MiB) 254ms ( 15.88 MiB) Fetch (100000 commits back) 14340ms (185.83 MiB) 1655ms (189.39 MiB) Change-Id: Icdb0cdd66ff168917fb9ef17b96093990cc6a98d
11 years ago
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571572573574575576577578579580581582583584585586587588589590591592593594595596597598599600601602603604605606607608609610611612613614615616617618619620621622623624625626627628629630631632633634635636637638639640641642643644645646647648649650651652653654655656657658659660661662663664665666667668669670671672673674675676677678679680681682683684685686687688689690691692693694695696697698699700701702703704705706707708709710711712713714715716717718719720721722723724725726727728729730731732733734735736737738739740
  1. /*
  2. * Copyright (C) 2008-2010, Google Inc.
  3. * Copyright (C) 2008, Marek Zawirski <marek.zawirski@gmail.com>
  4. * and other copyright owners as documented in the project's IP log.
  5. *
  6. * This program and the accompanying materials are made available
  7. * under the terms of the Eclipse Distribution License v1.0 which
  8. * accompanies this distribution, is reproduced below, and is
  9. * available at http://www.eclipse.org/org/documents/edl-v10.php
  10. *
  11. * All rights reserved.
  12. *
  13. * Redistribution and use in source and binary forms, with or
  14. * without modification, are permitted provided that the following
  15. * conditions are met:
  16. *
  17. * - Redistributions of source code must retain the above copyright
  18. * notice, this list of conditions and the following disclaimer.
  19. *
  20. * - Redistributions in binary form must reproduce the above
  21. * copyright notice, this list of conditions and the following
  22. * disclaimer in the documentation and/or other materials provided
  23. * with the distribution.
  24. *
  25. * - Neither the name of the Eclipse Foundation, Inc. nor the
  26. * names of its contributors may be used to endorse or promote
  27. * products derived from this software without specific prior
  28. * written permission.
  29. *
  30. * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
  31. * CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
  32. * INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
  33. * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
  34. * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
  35. * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
  36. * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
  37. * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
  38. * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
  39. * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
  40. * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
  41. * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
  42. * ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  43. */
  44. package org.eclipse.jgit.storage.pack;
  45. import java.util.concurrent.Executor;
  46. import java.util.zip.Deflater;
  47. import org.eclipse.jgit.internal.storage.file.PackIndexWriter;
  48. import org.eclipse.jgit.lib.Config;
  49. import org.eclipse.jgit.lib.Repository;
  50. /**
  51. * Configuration used by a pack writer when constructing the stream.
  52. *
  53. * A configuration may be modified once created, but should not be modified
  54. * while it is being used by a PackWriter. If a configuration is not modified it
  55. * is safe to share the same configuration instance between multiple concurrent
  56. * threads executing different PackWriters.
  57. */
  58. public class PackConfig {
  59. /**
  60. * Default value of deltas reuse option: {@value}
  61. *
  62. * @see #setReuseDeltas(boolean)
  63. */
  64. public static final boolean DEFAULT_REUSE_DELTAS = true;
  65. /**
  66. * Default value of objects reuse option: {@value}
  67. *
  68. * @see #setReuseObjects(boolean)
  69. */
  70. public static final boolean DEFAULT_REUSE_OBJECTS = true;
  71. /**
  72. * Default value of delta compress option: {@value}
  73. *
  74. * @see #setDeltaCompress(boolean)
  75. */
  76. public static final boolean DEFAULT_DELTA_COMPRESS = true;
  77. /**
  78. * Default value of delta base as offset option: {@value}
  79. *
  80. * @see #setDeltaBaseAsOffset(boolean)
  81. */
  82. public static final boolean DEFAULT_DELTA_BASE_AS_OFFSET = false;
  83. /**
  84. * Default value of maximum delta chain depth: {@value}
  85. *
  86. * @see #setMaxDeltaDepth(int)
  87. */
  88. public static final int DEFAULT_MAX_DELTA_DEPTH = 50;
  89. /**
  90. * Default window size during packing: {@value}
  91. *
  92. * @see #setDeltaSearchWindowSize(int)
  93. */
  94. public static final int DEFAULT_DELTA_SEARCH_WINDOW_SIZE = 10;
  95. /**
  96. * Default big file threshold: {@value}
  97. *
  98. * @see #setBigFileThreshold(int)
  99. */
  100. public static final int DEFAULT_BIG_FILE_THRESHOLD = 50 * 1024 * 1024;
  101. /**
  102. * Default delta cache size: {@value}
  103. *
  104. * @see #setDeltaCacheSize(long)
  105. */
  106. public static final long DEFAULT_DELTA_CACHE_SIZE = 50 * 1024 * 1024;
  107. /**
  108. * Default delta cache limit: {@value}
  109. *
  110. * @see #setDeltaCacheLimit(int)
  111. */
  112. public static final int DEFAULT_DELTA_CACHE_LIMIT = 100;
  113. /**
  114. * Default index version: {@value}
  115. *
  116. * @see #setIndexVersion(int)
  117. */
  118. public static final int DEFAULT_INDEX_VERSION = 2;
  119. /**
  120. * Default value of the build bitmaps option: {@value}
  121. *
  122. * @see #setBuildBitmaps(boolean)
  123. * @since 3.0
  124. */
  125. public static final boolean DEFAULT_BUILD_BITMAPS = true;
  126. private int compressionLevel = Deflater.DEFAULT_COMPRESSION;
  127. private boolean reuseDeltas = DEFAULT_REUSE_DELTAS;
  128. private boolean reuseObjects = DEFAULT_REUSE_OBJECTS;
  129. private boolean deltaBaseAsOffset = DEFAULT_DELTA_BASE_AS_OFFSET;
  130. private boolean deltaCompress = DEFAULT_DELTA_COMPRESS;
  131. private int maxDeltaDepth = DEFAULT_MAX_DELTA_DEPTH;
  132. private int deltaSearchWindowSize = DEFAULT_DELTA_SEARCH_WINDOW_SIZE;
  133. private long deltaSearchMemoryLimit;
  134. private long deltaCacheSize = DEFAULT_DELTA_CACHE_SIZE;
  135. private int deltaCacheLimit = DEFAULT_DELTA_CACHE_LIMIT;
  136. private int bigFileThreshold = DEFAULT_BIG_FILE_THRESHOLD;
  137. private int threads;
  138. private Executor executor;
  139. private int indexVersion = DEFAULT_INDEX_VERSION;
  140. private boolean buildBitmaps = DEFAULT_BUILD_BITMAPS;
  141. private boolean cutDeltaChains;
  142. /** Create a default configuration. */
  143. public PackConfig() {
  144. // Fields are initialized to defaults.
  145. }
  146. /**
  147. * Create a configuration honoring the repository's settings.
  148. *
  149. * @param db
  150. * the repository to read settings from. The repository is not
  151. * retained by the new configuration, instead its settings are
  152. * copied during the constructor.
  153. */
  154. public PackConfig(Repository db) {
  155. fromConfig(db.getConfig());
  156. }
  157. /**
  158. * Create a configuration honoring settings in a {@link Config}.
  159. *
  160. * @param cfg
  161. * the source to read settings from. The source is not retained
  162. * by the new configuration, instead its settings are copied
  163. * during the constructor.
  164. */
  165. public PackConfig(Config cfg) {
  166. fromConfig(cfg);
  167. }
  168. /**
  169. * Copy an existing configuration to a new instance.
  170. *
  171. * @param cfg
  172. * the source configuration to copy from.
  173. */
  174. public PackConfig(PackConfig cfg) {
  175. this.compressionLevel = cfg.compressionLevel;
  176. this.reuseDeltas = cfg.reuseDeltas;
  177. this.reuseObjects = cfg.reuseObjects;
  178. this.deltaBaseAsOffset = cfg.deltaBaseAsOffset;
  179. this.deltaCompress = cfg.deltaCompress;
  180. this.maxDeltaDepth = cfg.maxDeltaDepth;
  181. this.deltaSearchWindowSize = cfg.deltaSearchWindowSize;
  182. this.deltaSearchMemoryLimit = cfg.deltaSearchMemoryLimit;
  183. this.deltaCacheSize = cfg.deltaCacheSize;
  184. this.deltaCacheLimit = cfg.deltaCacheLimit;
  185. this.bigFileThreshold = cfg.bigFileThreshold;
  186. this.threads = cfg.threads;
  187. this.executor = cfg.executor;
  188. this.indexVersion = cfg.indexVersion;
  189. this.buildBitmaps = cfg.buildBitmaps;
  190. this.cutDeltaChains = cfg.cutDeltaChains;
  191. }
  192. /**
  193. * Check whether to reuse deltas existing in repository.
  194. *
  195. * Default setting: {@value #DEFAULT_REUSE_DELTAS}
  196. *
  197. * @return true if object is configured to reuse deltas; false otherwise.
  198. */
  199. public boolean isReuseDeltas() {
  200. return reuseDeltas;
  201. }
  202. /**
  203. * Set reuse deltas configuration option for the writer.
  204. *
  205. * When enabled, writer will search for delta representation of object in
  206. * repository and use it if possible. Normally, only deltas with base to
  207. * another object existing in set of objects to pack will be used. The
  208. * exception however is thin-packs where the base object may exist on the
  209. * other side.
  210. *
  211. * When raw delta data is directly copied from a pack file, its checksum is
  212. * computed to verify the data is not corrupt.
  213. *
  214. * Default setting: {@value #DEFAULT_REUSE_DELTAS}
  215. *
  216. * @param reuseDeltas
  217. * boolean indicating whether or not try to reuse deltas.
  218. */
  219. public void setReuseDeltas(boolean reuseDeltas) {
  220. this.reuseDeltas = reuseDeltas;
  221. }
  222. /**
  223. * Checks whether to reuse existing objects representation in repository.
  224. *
  225. * Default setting: {@value #DEFAULT_REUSE_OBJECTS}
  226. *
  227. * @return true if writer is configured to reuse objects representation from
  228. * pack; false otherwise.
  229. */
  230. public boolean isReuseObjects() {
  231. return reuseObjects;
  232. }
  233. /**
  234. * Set reuse objects configuration option for the writer.
  235. *
  236. * If enabled, writer searches for compressed representation in a pack file.
  237. * If possible, compressed data is directly copied from such a pack file.
  238. * Data checksum is verified.
  239. *
  240. * Default setting: {@value #DEFAULT_REUSE_OBJECTS}
  241. *
  242. * @param reuseObjects
  243. * boolean indicating whether or not writer should reuse existing
  244. * objects representation.
  245. */
  246. public void setReuseObjects(boolean reuseObjects) {
  247. this.reuseObjects = reuseObjects;
  248. }
  249. /**
  250. * True if writer can use offsets to point to a delta base.
  251. *
  252. * If true the writer may choose to use an offset to point to a delta base
  253. * in the same pack, this is a newer style of reference that saves space.
  254. * False if the writer has to use the older (and more compatible style) of
  255. * storing the full ObjectId of the delta base.
  256. *
  257. * Default setting: {@value #DEFAULT_DELTA_BASE_AS_OFFSET}
  258. *
  259. * @return true if delta base is stored as an offset; false if it is stored
  260. * as an ObjectId.
  261. */
  262. public boolean isDeltaBaseAsOffset() {
  263. return deltaBaseAsOffset;
  264. }
  265. /**
  266. * Set writer delta base format.
  267. *
  268. * Delta base can be written as an offset in a pack file (new approach
  269. * reducing file size) or as an object id (legacy approach, compatible with
  270. * old readers).
  271. *
  272. * Default setting: {@value #DEFAULT_DELTA_BASE_AS_OFFSET}
  273. *
  274. * @param deltaBaseAsOffset
  275. * boolean indicating whether delta base can be stored as an
  276. * offset.
  277. */
  278. public void setDeltaBaseAsOffset(boolean deltaBaseAsOffset) {
  279. this.deltaBaseAsOffset = deltaBaseAsOffset;
  280. }
  281. /**
  282. * Check whether the writer will create new deltas on the fly.
  283. *
  284. * Default setting: {@value #DEFAULT_DELTA_COMPRESS}
  285. *
  286. * @return true if the writer will create a new delta when either
  287. * {@link #isReuseDeltas()} is false, or no suitable delta is
  288. * available for reuse.
  289. */
  290. public boolean isDeltaCompress() {
  291. return deltaCompress;
  292. }
  293. /**
  294. * Set whether or not the writer will create new deltas on the fly.
  295. *
  296. * Default setting: {@value #DEFAULT_DELTA_COMPRESS}
  297. *
  298. * @param deltaCompress
  299. * true to create deltas when {@link #isReuseDeltas()} is false,
  300. * or when a suitable delta isn't available for reuse. Set to
  301. * false to write whole objects instead.
  302. */
  303. public void setDeltaCompress(boolean deltaCompress) {
  304. this.deltaCompress = deltaCompress;
  305. }
  306. /**
  307. * Get maximum depth of delta chain set up for the writer.
  308. *
  309. * Generated chains are not longer than this value.
  310. *
  311. * Default setting: {@value #DEFAULT_MAX_DELTA_DEPTH}
  312. *
  313. * @return maximum delta chain depth.
  314. */
  315. public int getMaxDeltaDepth() {
  316. return maxDeltaDepth;
  317. }
  318. /**
  319. * Set up maximum depth of delta chain for the writer.
  320. *
  321. * Generated chains are not longer than this value. Too low value causes low
  322. * compression level, while too big makes unpacking (reading) longer.
  323. *
  324. * Default setting: {@value #DEFAULT_MAX_DELTA_DEPTH}
  325. *
  326. * @param maxDeltaDepth
  327. * maximum delta chain depth.
  328. */
  329. public void setMaxDeltaDepth(int maxDeltaDepth) {
  330. this.maxDeltaDepth = maxDeltaDepth;
  331. }
  332. /**
  333. * @return true if existing delta chains should be cut at
  334. * {@link #getMaxDeltaDepth()}. Default is false, allowing existing
  335. * chains to be of any length.
  336. * @since 3.0
  337. */
  338. public boolean getCutDeltaChains() {
  339. return cutDeltaChains;
  340. }
  341. /**
  342. * Enable cutting existing delta chains at {@link #getMaxDeltaDepth()}.
  343. *
  344. * By default this is disabled and existing chains are kept at whatever
  345. * length a prior packer was configured to create. This allows objects to be
  346. * packed one with a large depth (for example 250), and later to quickly
  347. * repack the repository with a shorter depth (such as 50), but reusing the
  348. * complete delta chains created by the earlier 250 depth.
  349. *
  350. * @param cut
  351. * true to cut existing chains.
  352. * @since 3.0
  353. */
  354. public void setCutDeltaChains(boolean cut) {
  355. cutDeltaChains = cut;
  356. }
  357. /**
  358. * Get the number of objects to try when looking for a delta base.
  359. *
  360. * This limit is per thread, if 4 threads are used the actual memory used
  361. * will be 4 times this value.
  362. *
  363. * Default setting: {@value #DEFAULT_DELTA_SEARCH_WINDOW_SIZE}
  364. *
  365. * @return the object count to be searched.
  366. */
  367. public int getDeltaSearchWindowSize() {
  368. return deltaSearchWindowSize;
  369. }
  370. /**
  371. * Set the number of objects considered when searching for a delta base.
  372. *
  373. * Default setting: {@value #DEFAULT_DELTA_SEARCH_WINDOW_SIZE}
  374. *
  375. * @param objectCount
  376. * number of objects to search at once. Must be at least 2.
  377. */
  378. public void setDeltaSearchWindowSize(int objectCount) {
  379. if (objectCount <= 2)
  380. setDeltaCompress(false);
  381. else
  382. deltaSearchWindowSize = objectCount;
  383. }
  384. /**
  385. * Get maximum number of bytes to put into the delta search window.
  386. *
  387. * Default setting is 0, for an unlimited amount of memory usage. Actual
  388. * memory used is the lower limit of either this setting, or the sum of
  389. * space used by at most {@link #getDeltaSearchWindowSize()} objects.
  390. *
  391. * This limit is per thread, if 4 threads are used the actual memory limit
  392. * will be 4 times this value.
  393. *
  394. * @return the memory limit.
  395. */
  396. public long getDeltaSearchMemoryLimit() {
  397. return deltaSearchMemoryLimit;
  398. }
  399. /**
  400. * Set the maximum number of bytes to put into the delta search window.
  401. *
  402. * Default setting is 0, for an unlimited amount of memory usage. If the
  403. * memory limit is reached before {@link #getDeltaSearchWindowSize()} the
  404. * window size is temporarily lowered.
  405. *
  406. * @param memoryLimit
  407. * Maximum number of bytes to load at once, 0 for unlimited.
  408. */
  409. public void setDeltaSearchMemoryLimit(long memoryLimit) {
  410. deltaSearchMemoryLimit = memoryLimit;
  411. }
  412. /**
  413. * Get the size of the in-memory delta cache.
  414. *
  415. * This limit is for the entire writer, even if multiple threads are used.
  416. *
  417. * Default setting: {@value #DEFAULT_DELTA_CACHE_SIZE}
  418. *
  419. * @return maximum number of bytes worth of delta data to cache in memory.
  420. * If 0 the cache is infinite in size (up to the JVM heap limit
  421. * anyway). A very tiny size such as 1 indicates the cache is
  422. * effectively disabled.
  423. */
  424. public long getDeltaCacheSize() {
  425. return deltaCacheSize;
  426. }
  427. /**
  428. * Set the maximum number of bytes of delta data to cache.
  429. *
  430. * During delta search, up to this many bytes worth of small or hard to
  431. * compute deltas will be stored in memory. This cache speeds up writing by
  432. * allowing the cached entry to simply be dumped to the output stream.
  433. *
  434. * Default setting: {@value #DEFAULT_DELTA_CACHE_SIZE}
  435. *
  436. * @param size
  437. * number of bytes to cache. Set to 0 to enable an infinite
  438. * cache, set to 1 (an impossible size for any delta) to disable
  439. * the cache.
  440. */
  441. public void setDeltaCacheSize(long size) {
  442. deltaCacheSize = size;
  443. }
  444. /**
  445. * Maximum size in bytes of a delta to cache.
  446. *
  447. * Default setting: {@value #DEFAULT_DELTA_CACHE_LIMIT}
  448. *
  449. * @return maximum size (in bytes) of a delta that should be cached.
  450. */
  451. public int getDeltaCacheLimit() {
  452. return deltaCacheLimit;
  453. }
  454. /**
  455. * Set the maximum size of a delta that should be cached.
  456. *
  457. * During delta search, any delta smaller than this size will be cached, up
  458. * to the {@link #getDeltaCacheSize()} maximum limit. This speeds up writing
  459. * by allowing these cached deltas to be output as-is.
  460. *
  461. * Default setting: {@value #DEFAULT_DELTA_CACHE_LIMIT}
  462. *
  463. * @param size
  464. * maximum size (in bytes) of a delta to be cached.
  465. */
  466. public void setDeltaCacheLimit(int size) {
  467. deltaCacheLimit = size;
  468. }
  469. /**
  470. * Get the maximum file size that will be delta compressed.
  471. *
  472. * Files bigger than this setting will not be delta compressed, as they are
  473. * more than likely already highly compressed binary data files that do not
  474. * delta compress well, such as MPEG videos.
  475. *
  476. * Default setting: {@value #DEFAULT_BIG_FILE_THRESHOLD}
  477. *
  478. * @return the configured big file threshold.
  479. */
  480. public int getBigFileThreshold() {
  481. return bigFileThreshold;
  482. }
  483. /**
  484. * Set the maximum file size that should be considered for deltas.
  485. *
  486. * Default setting: {@value #DEFAULT_BIG_FILE_THRESHOLD}
  487. *
  488. * @param bigFileThreshold
  489. * the limit, in bytes.
  490. */
  491. public void setBigFileThreshold(int bigFileThreshold) {
  492. this.bigFileThreshold = bigFileThreshold;
  493. }
  494. /**
  495. * Get the compression level applied to objects in the pack.
  496. *
  497. * Default setting: {@value java.util.zip.Deflater#DEFAULT_COMPRESSION}
  498. *
  499. * @return current compression level, see {@link java.util.zip.Deflater}.
  500. */
  501. public int getCompressionLevel() {
  502. return compressionLevel;
  503. }
  504. /**
  505. * Set the compression level applied to objects in the pack.
  506. *
  507. * Default setting: {@value java.util.zip.Deflater#DEFAULT_COMPRESSION}
  508. *
  509. * @param level
  510. * compression level, must be a valid level recognized by the
  511. * {@link java.util.zip.Deflater} class.
  512. */
  513. public void setCompressionLevel(int level) {
  514. compressionLevel = level;
  515. }
  516. /**
  517. * Get the number of threads used during delta compression.
  518. *
  519. * Default setting: 0 (auto-detect processors)
  520. *
  521. * @return number of threads used for delta compression. 0 will auto-detect
  522. * the threads to the number of available processors.
  523. */
  524. public int getThreads() {
  525. return threads;
  526. }
  527. /**
  528. * Set the number of threads to use for delta compression.
  529. *
  530. * During delta compression, if there are enough objects to be considered
  531. * the writer will start up concurrent threads and allow them to compress
  532. * different sections of the repository concurrently.
  533. *
  534. * An application thread pool can be set by {@link #setExecutor(Executor)}.
  535. * If not set a temporary pool will be created by the writer, and torn down
  536. * automatically when compression is over.
  537. *
  538. * Default setting: 0 (auto-detect processors)
  539. *
  540. * @param threads
  541. * number of threads to use. If &lt;= 0 the number of available
  542. * processors for this JVM is used.
  543. */
  544. public void setThreads(int threads) {
  545. this.threads = threads;
  546. }
  547. /** @return the preferred thread pool to execute delta search on. */
  548. public Executor getExecutor() {
  549. return executor;
  550. }
  551. /**
  552. * Set the executor to use when using threads.
  553. *
  554. * During delta compression if the executor is non-null jobs will be queued
  555. * up on it to perform delta compression in parallel. Aside from setting the
  556. * executor, the caller must set {@link #setThreads(int)} to enable threaded
  557. * delta search.
  558. *
  559. * @param executor
  560. * executor to use for threads. Set to null to create a temporary
  561. * executor just for the writer.
  562. */
  563. public void setExecutor(Executor executor) {
  564. this.executor = executor;
  565. }
  566. /**
  567. * Get the pack index file format version this instance creates.
  568. *
  569. * Default setting: {@value #DEFAULT_INDEX_VERSION}
  570. *
  571. * @return the index version, the special version 0 designates the oldest
  572. * (most compatible) format available for the objects.
  573. * @see PackIndexWriter
  574. */
  575. public int getIndexVersion() {
  576. return indexVersion;
  577. }
  578. /**
  579. * Set the pack index file format version this instance will create.
  580. *
  581. * Default setting: {@value #DEFAULT_INDEX_VERSION}
  582. *
  583. * @param version
  584. * the version to write. The special version 0 designates the
  585. * oldest (most compatible) format available for the objects.
  586. * @see PackIndexWriter
  587. */
  588. public void setIndexVersion(final int version) {
  589. indexVersion = version;
  590. }
  591. /**
  592. * True if writer is allowed to build bitmaps for indexes.
  593. *
  594. * Default setting: {@value #DEFAULT_BUILD_BITMAPS}
  595. *
  596. * @return true if delta base is the writer can choose to output an index
  597. * with bitmaps.
  598. * @since 3.0
  599. */
  600. public boolean isBuildBitmaps() {
  601. return buildBitmaps;
  602. }
  603. /**
  604. * Set writer to allow building bitmaps for supported pack files.
  605. *
  606. * Index files can include bitmaps to speed up future ObjectWalks.
  607. *
  608. * Default setting: {@value #DEFAULT_BUILD_BITMAPS}
  609. *
  610. * @param buildBitmaps
  611. * boolean indicating whether bitmaps may be included in the
  612. * index.
  613. * @since 3.0
  614. */
  615. public void setBuildBitmaps(boolean buildBitmaps) {
  616. this.buildBitmaps = buildBitmaps;
  617. }
  618. /**
  619. * Update properties by setting fields from the configuration.
  620. *
  621. * If a property's corresponding variable is not defined in the supplied
  622. * configuration, then it is left unmodified.
  623. *
  624. * @param rc
  625. * configuration to read properties from.
  626. */
  627. public void fromConfig(final Config rc) {
  628. setMaxDeltaDepth(rc.getInt("pack", "depth", getMaxDeltaDepth())); //$NON-NLS-1$ //$NON-NLS-2$
  629. setDeltaSearchWindowSize(rc.getInt(
  630. "pack", "window", getDeltaSearchWindowSize())); //$NON-NLS-1$ //$NON-NLS-2$
  631. setDeltaSearchMemoryLimit(rc.getLong(
  632. "pack", "windowmemory", getDeltaSearchMemoryLimit())); //$NON-NLS-1$ //$NON-NLS-2$
  633. setDeltaCacheSize(rc.getLong(
  634. "pack", "deltacachesize", getDeltaCacheSize())); //$NON-NLS-1$ //$NON-NLS-2$
  635. setDeltaCacheLimit(rc.getInt(
  636. "pack", "deltacachelimit", getDeltaCacheLimit())); //$NON-NLS-1$ //$NON-NLS-2$
  637. setCompressionLevel(rc.getInt("pack", "compression", //$NON-NLS-1$ //$NON-NLS-2$
  638. rc.getInt("core", "compression", getCompressionLevel()))); //$NON-NLS-1$ //$NON-NLS-2$
  639. setIndexVersion(rc.getInt("pack", "indexversion", getIndexVersion())); //$NON-NLS-1$ //$NON-NLS-2$
  640. setBigFileThreshold(rc.getInt(
  641. "core", "bigfilethreshold", getBigFileThreshold())); //$NON-NLS-1$ //$NON-NLS-2$
  642. setThreads(rc.getInt("pack", "threads", getThreads())); //$NON-NLS-1$ //$NON-NLS-2$
  643. // These variables aren't standardized
  644. //
  645. setReuseDeltas(rc.getBoolean("pack", "reusedeltas", isReuseDeltas())); //$NON-NLS-1$ //$NON-NLS-2$
  646. setReuseObjects(rc.getBoolean("pack", "reuseobjects", isReuseObjects())); //$NON-NLS-1$ //$NON-NLS-2$
  647. setDeltaCompress(rc.getBoolean(
  648. "pack", "deltacompression", isDeltaCompress())); //$NON-NLS-1$ //$NON-NLS-2$
  649. setCutDeltaChains(rc.getBoolean(
  650. "pack", "cutdeltachains", getCutDeltaChains())); //$NON-NLS-1$ //$NON-NLS-2$
  651. setBuildBitmaps(rc.getBoolean("pack", "buildbitmaps", isBuildBitmaps())); //$NON-NLS-1$ //$NON-NLS-2$
  652. }
  653. public String toString() {
  654. final StringBuilder b = new StringBuilder();
  655. b.append("maxDeltaDepth=").append(getMaxDeltaDepth()); //$NON-NLS-1$
  656. b.append(", deltaSearchWindowSize=").append(getDeltaSearchWindowSize()); //$NON-NLS-1$
  657. b.append(", deltaSearchMemoryLimit=").append(getDeltaSearchMemoryLimit()); //$NON-NLS-1$
  658. b.append(", deltaCacheSize=").append(getDeltaCacheSize()); //$NON-NLS-1$
  659. b.append(", deltaCacheLimit=").append(getDeltaCacheLimit()); //$NON-NLS-1$
  660. b.append(", compressionLevel=").append(getCompressionLevel()); //$NON-NLS-1$
  661. b.append(", indexVersion=").append(getIndexVersion()); //$NON-NLS-1$
  662. b.append(", bigFileThreshold=").append(getBigFileThreshold()); //$NON-NLS-1$
  663. b.append(", threads=").append(getThreads()); //$NON-NLS-1$
  664. b.append(", reuseDeltas=").append(isReuseDeltas()); //$NON-NLS-1$
  665. b.append(", reuseObjects=").append(isReuseObjects()); //$NON-NLS-1$
  666. b.append(", deltaCompress=").append(isDeltaCompress()); //$NON-NLS-1$
  667. b.append(", buildBitmaps=").append(isBuildBitmaps()); //$NON-NLS-1$
  668. return b.toString();
  669. }
  670. }