You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

PackConfig.java 34KB

Support creating pack bitmap indexes in PackWriter. Update the PackWriter to support writing out pack bitmap indexes, a parallel ".bitmap" file to the ".pack" file. Bitmaps are selected at commits every 1 to 5,000 commits for each unique path from the start. The most recent 100 commits are all bitmapped. The next 19,000 commits have a bitmaps every 100 commits. The remaining commits have a bitmap every 5,000 commits. Commits with more than 1 parent are prefered over ones with 1 or less. Furthermore, previously computed bitmaps are reused, if the previous entry had the reuse flag set, which is set when the bitmap was placed at the max allowed distance. Bitmaps are used to speed up the counting phase when packing, for requests that are not shallow. The PackWriterBitmapWalker uses a RevFilter to proactively mark commits with RevFlag.SEEN, when they appear in a bitmap. The walker produces the full closure of reachable ObjectIds, given the collection of starting ObjectIds. For fetch request, two ObjectWalks are executed to compute the ObjectIds reachable from the haves and from the wants. The ObjectIds needed to be written are determined by taking all the resulting wants AND NOT the haves. For clone requests, we get cached pack support for "free" since it is possible to determine if all of the ObjectIds in a pack file are included in the resulting list of ObjectIds to write. On my machine, the best times for clones and fetches of the linux kernel repository (with about 2.6M objects and 300K commits) are tabulated below: Operation Index V2 Index VE003 Clone 37530ms (524.06 MiB) 82ms (524.06 MiB) Fetch (1 commit back) 75ms 107ms Fetch (10 commits back) 456ms (269.51 KiB) 341ms (265.19 KiB) Fetch (100 commits back) 449ms (269.91 KiB) 337ms (267.28 KiB) Fetch (1000 commits back) 2229ms ( 14.75 MiB) 189ms ( 14.42 MiB) Fetch (10000 commits back) 2177ms ( 16.30 MiB) 254ms ( 15.88 MiB) Fetch (100000 commits back) 14340ms (185.83 MiB) 1655ms (189.39 MiB) Change-Id: Icdb0cdd66ff168917fb9ef17b96093990cc6a98d
12 years ago
Support creating pack bitmap indexes in PackWriter. Update the PackWriter to support writing out pack bitmap indexes, a parallel ".bitmap" file to the ".pack" file. Bitmaps are selected at commits every 1 to 5,000 commits for each unique path from the start. The most recent 100 commits are all bitmapped. The next 19,000 commits have a bitmaps every 100 commits. The remaining commits have a bitmap every 5,000 commits. Commits with more than 1 parent are prefered over ones with 1 or less. Furthermore, previously computed bitmaps are reused, if the previous entry had the reuse flag set, which is set when the bitmap was placed at the max allowed distance. Bitmaps are used to speed up the counting phase when packing, for requests that are not shallow. The PackWriterBitmapWalker uses a RevFilter to proactively mark commits with RevFlag.SEEN, when they appear in a bitmap. The walker produces the full closure of reachable ObjectIds, given the collection of starting ObjectIds. For fetch request, two ObjectWalks are executed to compute the ObjectIds reachable from the haves and from the wants. The ObjectIds needed to be written are determined by taking all the resulting wants AND NOT the haves. For clone requests, we get cached pack support for "free" since it is possible to determine if all of the ObjectIds in a pack file are included in the resulting list of ObjectIds to write. On my machine, the best times for clones and fetches of the linux kernel repository (with about 2.6M objects and 300K commits) are tabulated below: Operation Index V2 Index VE003 Clone 37530ms (524.06 MiB) 82ms (524.06 MiB) Fetch (1 commit back) 75ms 107ms Fetch (10 commits back) 456ms (269.51 KiB) 341ms (265.19 KiB) Fetch (100 commits back) 449ms (269.91 KiB) 337ms (267.28 KiB) Fetch (1000 commits back) 2229ms ( 14.75 MiB) 189ms ( 14.42 MiB) Fetch (10000 commits back) 2177ms ( 16.30 MiB) 254ms ( 15.88 MiB) Fetch (100000 commits back) 14340ms (185.83 MiB) 1655ms (189.39 MiB) Change-Id: Icdb0cdd66ff168917fb9ef17b96093990cc6a98d
12 years ago
Support creating pack bitmap indexes in PackWriter. Update the PackWriter to support writing out pack bitmap indexes, a parallel ".bitmap" file to the ".pack" file. Bitmaps are selected at commits every 1 to 5,000 commits for each unique path from the start. The most recent 100 commits are all bitmapped. The next 19,000 commits have a bitmaps every 100 commits. The remaining commits have a bitmap every 5,000 commits. Commits with more than 1 parent are prefered over ones with 1 or less. Furthermore, previously computed bitmaps are reused, if the previous entry had the reuse flag set, which is set when the bitmap was placed at the max allowed distance. Bitmaps are used to speed up the counting phase when packing, for requests that are not shallow. The PackWriterBitmapWalker uses a RevFilter to proactively mark commits with RevFlag.SEEN, when they appear in a bitmap. The walker produces the full closure of reachable ObjectIds, given the collection of starting ObjectIds. For fetch request, two ObjectWalks are executed to compute the ObjectIds reachable from the haves and from the wants. The ObjectIds needed to be written are determined by taking all the resulting wants AND NOT the haves. For clone requests, we get cached pack support for "free" since it is possible to determine if all of the ObjectIds in a pack file are included in the resulting list of ObjectIds to write. On my machine, the best times for clones and fetches of the linux kernel repository (with about 2.6M objects and 300K commits) are tabulated below: Operation Index V2 Index VE003 Clone 37530ms (524.06 MiB) 82ms (524.06 MiB) Fetch (1 commit back) 75ms 107ms Fetch (10 commits back) 456ms (269.51 KiB) 341ms (265.19 KiB) Fetch (100 commits back) 449ms (269.91 KiB) 337ms (267.28 KiB) Fetch (1000 commits back) 2229ms ( 14.75 MiB) 189ms ( 14.42 MiB) Fetch (10000 commits back) 2177ms ( 16.30 MiB) 254ms ( 15.88 MiB) Fetch (100000 commits back) 14340ms (185.83 MiB) 1655ms (189.39 MiB) Change-Id: Icdb0cdd66ff168917fb9ef17b96093990cc6a98d
12 years ago
Support creating pack bitmap indexes in PackWriter. Update the PackWriter to support writing out pack bitmap indexes, a parallel ".bitmap" file to the ".pack" file. Bitmaps are selected at commits every 1 to 5,000 commits for each unique path from the start. The most recent 100 commits are all bitmapped. The next 19,000 commits have a bitmaps every 100 commits. The remaining commits have a bitmap every 5,000 commits. Commits with more than 1 parent are prefered over ones with 1 or less. Furthermore, previously computed bitmaps are reused, if the previous entry had the reuse flag set, which is set when the bitmap was placed at the max allowed distance. Bitmaps are used to speed up the counting phase when packing, for requests that are not shallow. The PackWriterBitmapWalker uses a RevFilter to proactively mark commits with RevFlag.SEEN, when they appear in a bitmap. The walker produces the full closure of reachable ObjectIds, given the collection of starting ObjectIds. For fetch request, two ObjectWalks are executed to compute the ObjectIds reachable from the haves and from the wants. The ObjectIds needed to be written are determined by taking all the resulting wants AND NOT the haves. For clone requests, we get cached pack support for "free" since it is possible to determine if all of the ObjectIds in a pack file are included in the resulting list of ObjectIds to write. On my machine, the best times for clones and fetches of the linux kernel repository (with about 2.6M objects and 300K commits) are tabulated below: Operation Index V2 Index VE003 Clone 37530ms (524.06 MiB) 82ms (524.06 MiB) Fetch (1 commit back) 75ms 107ms Fetch (10 commits back) 456ms (269.51 KiB) 341ms (265.19 KiB) Fetch (100 commits back) 449ms (269.91 KiB) 337ms (267.28 KiB) Fetch (1000 commits back) 2229ms ( 14.75 MiB) 189ms ( 14.42 MiB) Fetch (10000 commits back) 2177ms ( 16.30 MiB) 254ms ( 15.88 MiB) Fetch (100000 commits back) 14340ms (185.83 MiB) 1655ms (189.39 MiB) Change-Id: Icdb0cdd66ff168917fb9ef17b96093990cc6a98d
12 years ago
Support cutting existing delta chains longer than the max depth Some packs built by JGit have incredibly long delta chains due to a long standing bug in PackWriter. Google has packs created by JGit's DfsGarbageCollector with chains of 6000 objects long, or more. Inflating objects at the end of this 6000 long chain is impossible to complete within a reasonable time bound. It could take a beefy system hours to perform even using the heavily optimized native C implementation of Git, let alone with JGit. Enable pack.cutDeltaChains to be set in a configuration file to permit the PackWriter to determine the length of each delta chain and clip the chain at arbitrary points to fit within pack.depth. Delta chain cycles are still possible, but no attempt is made to detect them. A trivial chain of A->B->A will iterate for the full pack.depth configured limit (e.g. 50) and then pick an object to store as non-delta. When cutting chains the object list is walked in reverse to try and take advantage of existing chain computations. The assumption here is most deltas are near the end of the list, and their bases are near the front of the list. Going up from the tail attempts to reuse chainLength computations by relying on the memoized value in the delta base. The chainLength field in ObjectToPack is overloaded into the depth field normally used by DeltaWindow. This is acceptable because the chain cut happens before delta search, and the chainLength is reset to 0 if delta search will follow. Change-Id: Ida4fde9558f3abbbb77ade398d2af3941de9c812
11 years ago
Support creating pack bitmap indexes in PackWriter. Update the PackWriter to support writing out pack bitmap indexes, a parallel ".bitmap" file to the ".pack" file. Bitmaps are selected at commits every 1 to 5,000 commits for each unique path from the start. The most recent 100 commits are all bitmapped. The next 19,000 commits have a bitmaps every 100 commits. The remaining commits have a bitmap every 5,000 commits. Commits with more than 1 parent are prefered over ones with 1 or less. Furthermore, previously computed bitmaps are reused, if the previous entry had the reuse flag set, which is set when the bitmap was placed at the max allowed distance. Bitmaps are used to speed up the counting phase when packing, for requests that are not shallow. The PackWriterBitmapWalker uses a RevFilter to proactively mark commits with RevFlag.SEEN, when they appear in a bitmap. The walker produces the full closure of reachable ObjectIds, given the collection of starting ObjectIds. For fetch request, two ObjectWalks are executed to compute the ObjectIds reachable from the haves and from the wants. The ObjectIds needed to be written are determined by taking all the resulting wants AND NOT the haves. For clone requests, we get cached pack support for "free" since it is possible to determine if all of the ObjectIds in a pack file are included in the resulting list of ObjectIds to write. On my machine, the best times for clones and fetches of the linux kernel repository (with about 2.6M objects and 300K commits) are tabulated below: Operation Index V2 Index VE003 Clone 37530ms (524.06 MiB) 82ms (524.06 MiB) Fetch (1 commit back) 75ms 107ms Fetch (10 commits back) 456ms (269.51 KiB) 341ms (265.19 KiB) Fetch (100 commits back) 449ms (269.91 KiB) 337ms (267.28 KiB) Fetch (1000 commits back) 2229ms ( 14.75 MiB) 189ms ( 14.42 MiB) Fetch (10000 commits back) 2177ms ( 16.30 MiB) 254ms ( 15.88 MiB) Fetch (100000 commits back) 14340ms (185.83 MiB) 1655ms (189.39 MiB) Change-Id: Icdb0cdd66ff168917fb9ef17b96093990cc6a98d
12 years ago
Support cutting existing delta chains longer than the max depth Some packs built by JGit have incredibly long delta chains due to a long standing bug in PackWriter. Google has packs created by JGit's DfsGarbageCollector with chains of 6000 objects long, or more. Inflating objects at the end of this 6000 long chain is impossible to complete within a reasonable time bound. It could take a beefy system hours to perform even using the heavily optimized native C implementation of Git, let alone with JGit. Enable pack.cutDeltaChains to be set in a configuration file to permit the PackWriter to determine the length of each delta chain and clip the chain at arbitrary points to fit within pack.depth. Delta chain cycles are still possible, but no attempt is made to detect them. A trivial chain of A->B->A will iterate for the full pack.depth configured limit (e.g. 50) and then pick an object to store as non-delta. When cutting chains the object list is walked in reverse to try and take advantage of existing chain computations. The assumption here is most deltas are near the end of the list, and their bases are near the front of the list. Going up from the tail attempts to reuse chainLength computations by relying on the memoized value in the delta base. The chainLength field in ObjectToPack is overloaded into the depth field normally used by DeltaWindow. This is acceptable because the chain cut happens before delta search, and the chainLength is reset to 0 if delta search will follow. Change-Id: Ida4fde9558f3abbbb77ade398d2af3941de9c812
11 years ago
Support cutting existing delta chains longer than the max depth Some packs built by JGit have incredibly long delta chains due to a long standing bug in PackWriter. Google has packs created by JGit's DfsGarbageCollector with chains of 6000 objects long, or more. Inflating objects at the end of this 6000 long chain is impossible to complete within a reasonable time bound. It could take a beefy system hours to perform even using the heavily optimized native C implementation of Git, let alone with JGit. Enable pack.cutDeltaChains to be set in a configuration file to permit the PackWriter to determine the length of each delta chain and clip the chain at arbitrary points to fit within pack.depth. Delta chain cycles are still possible, but no attempt is made to detect them. A trivial chain of A->B->A will iterate for the full pack.depth configured limit (e.g. 50) and then pick an object to store as non-delta. When cutting chains the object list is walked in reverse to try and take advantage of existing chain computations. The assumption here is most deltas are near the end of the list, and their bases are near the front of the list. Going up from the tail attempts to reuse chainLength computations by relying on the memoized value in the delta base. The chainLength field in ObjectToPack is overloaded into the depth field normally used by DeltaWindow. This is acceptable because the chain cut happens before delta search, and the chainLength is reset to 0 if delta search will follow. Change-Id: Ida4fde9558f3abbbb77ade398d2af3941de9c812
11 years ago
Support cutting existing delta chains longer than the max depth Some packs built by JGit have incredibly long delta chains due to a long standing bug in PackWriter. Google has packs created by JGit's DfsGarbageCollector with chains of 6000 objects long, or more. Inflating objects at the end of this 6000 long chain is impossible to complete within a reasonable time bound. It could take a beefy system hours to perform even using the heavily optimized native C implementation of Git, let alone with JGit. Enable pack.cutDeltaChains to be set in a configuration file to permit the PackWriter to determine the length of each delta chain and clip the chain at arbitrary points to fit within pack.depth. Delta chain cycles are still possible, but no attempt is made to detect them. A trivial chain of A->B->A will iterate for the full pack.depth configured limit (e.g. 50) and then pick an object to store as non-delta. When cutting chains the object list is walked in reverse to try and take advantage of existing chain computations. The assumption here is most deltas are near the end of the list, and their bases are near the front of the list. Going up from the tail attempts to reuse chainLength computations by relying on the memoized value in the delta base. The chainLength field in ObjectToPack is overloaded into the depth field normally used by DeltaWindow. This is acceptable because the chain cut happens before delta search, and the chainLength is reset to 0 if delta search will follow. Change-Id: Ida4fde9558f3abbbb77ade398d2af3941de9c812
11 years ago
Support cutting existing delta chains longer than the max depth Some packs built by JGit have incredibly long delta chains due to a long standing bug in PackWriter. Google has packs created by JGit's DfsGarbageCollector with chains of 6000 objects long, or more. Inflating objects at the end of this 6000 long chain is impossible to complete within a reasonable time bound. It could take a beefy system hours to perform even using the heavily optimized native C implementation of Git, let alone with JGit. Enable pack.cutDeltaChains to be set in a configuration file to permit the PackWriter to determine the length of each delta chain and clip the chain at arbitrary points to fit within pack.depth. Delta chain cycles are still possible, but no attempt is made to detect them. A trivial chain of A->B->A will iterate for the full pack.depth configured limit (e.g. 50) and then pick an object to store as non-delta. When cutting chains the object list is walked in reverse to try and take advantage of existing chain computations. The assumption here is most deltas are near the end of the list, and their bases are near the front of the list. Going up from the tail attempts to reuse chainLength computations by relying on the memoized value in the delta base. The chainLength field in ObjectToPack is overloaded into the depth field normally used by DeltaWindow. This is acceptable because the chain cut happens before delta search, and the chainLength is reset to 0 if delta search will follow. Change-Id: Ida4fde9558f3abbbb77ade398d2af3941de9c812
11 years ago
Support creating pack bitmap indexes in PackWriter. Update the PackWriter to support writing out pack bitmap indexes, a parallel ".bitmap" file to the ".pack" file. Bitmaps are selected at commits every 1 to 5,000 commits for each unique path from the start. The most recent 100 commits are all bitmapped. The next 19,000 commits have a bitmaps every 100 commits. The remaining commits have a bitmap every 5,000 commits. Commits with more than 1 parent are prefered over ones with 1 or less. Furthermore, previously computed bitmaps are reused, if the previous entry had the reuse flag set, which is set when the bitmap was placed at the max allowed distance. Bitmaps are used to speed up the counting phase when packing, for requests that are not shallow. The PackWriterBitmapWalker uses a RevFilter to proactively mark commits with RevFlag.SEEN, when they appear in a bitmap. The walker produces the full closure of reachable ObjectIds, given the collection of starting ObjectIds. For fetch request, two ObjectWalks are executed to compute the ObjectIds reachable from the haves and from the wants. The ObjectIds needed to be written are determined by taking all the resulting wants AND NOT the haves. For clone requests, we get cached pack support for "free" since it is possible to determine if all of the ObjectIds in a pack file are included in the resulting list of ObjectIds to write. On my machine, the best times for clones and fetches of the linux kernel repository (with about 2.6M objects and 300K commits) are tabulated below: Operation Index V2 Index VE003 Clone 37530ms (524.06 MiB) 82ms (524.06 MiB) Fetch (1 commit back) 75ms 107ms Fetch (10 commits back) 456ms (269.51 KiB) 341ms (265.19 KiB) Fetch (100 commits back) 449ms (269.91 KiB) 337ms (267.28 KiB) Fetch (1000 commits back) 2229ms ( 14.75 MiB) 189ms ( 14.42 MiB) Fetch (10000 commits back) 2177ms ( 16.30 MiB) 254ms ( 15.88 MiB) Fetch (100000 commits back) 14340ms (185.83 MiB) 1655ms (189.39 MiB) Change-Id: Icdb0cdd66ff168917fb9ef17b96093990cc6a98d
12 years ago
Support creating pack bitmap indexes in PackWriter. Update the PackWriter to support writing out pack bitmap indexes, a parallel ".bitmap" file to the ".pack" file. Bitmaps are selected at commits every 1 to 5,000 commits for each unique path from the start. The most recent 100 commits are all bitmapped. The next 19,000 commits have a bitmaps every 100 commits. The remaining commits have a bitmap every 5,000 commits. Commits with more than 1 parent are prefered over ones with 1 or less. Furthermore, previously computed bitmaps are reused, if the previous entry had the reuse flag set, which is set when the bitmap was placed at the max allowed distance. Bitmaps are used to speed up the counting phase when packing, for requests that are not shallow. The PackWriterBitmapWalker uses a RevFilter to proactively mark commits with RevFlag.SEEN, when they appear in a bitmap. The walker produces the full closure of reachable ObjectIds, given the collection of starting ObjectIds. For fetch request, two ObjectWalks are executed to compute the ObjectIds reachable from the haves and from the wants. The ObjectIds needed to be written are determined by taking all the resulting wants AND NOT the haves. For clone requests, we get cached pack support for "free" since it is possible to determine if all of the ObjectIds in a pack file are included in the resulting list of ObjectIds to write. On my machine, the best times for clones and fetches of the linux kernel repository (with about 2.6M objects and 300K commits) are tabulated below: Operation Index V2 Index VE003 Clone 37530ms (524.06 MiB) 82ms (524.06 MiB) Fetch (1 commit back) 75ms 107ms Fetch (10 commits back) 456ms (269.51 KiB) 341ms (265.19 KiB) Fetch (100 commits back) 449ms (269.91 KiB) 337ms (267.28 KiB) Fetch (1000 commits back) 2229ms ( 14.75 MiB) 189ms ( 14.42 MiB) Fetch (10000 commits back) 2177ms ( 16.30 MiB) 254ms ( 15.88 MiB) Fetch (100000 commits back) 14340ms (185.83 MiB) 1655ms (189.39 MiB) Change-Id: Icdb0cdd66ff168917fb9ef17b96093990cc6a98d
12 years ago
Support creating pack bitmap indexes in PackWriter. Update the PackWriter to support writing out pack bitmap indexes, a parallel ".bitmap" file to the ".pack" file. Bitmaps are selected at commits every 1 to 5,000 commits for each unique path from the start. The most recent 100 commits are all bitmapped. The next 19,000 commits have a bitmaps every 100 commits. The remaining commits have a bitmap every 5,000 commits. Commits with more than 1 parent are prefered over ones with 1 or less. Furthermore, previously computed bitmaps are reused, if the previous entry had the reuse flag set, which is set when the bitmap was placed at the max allowed distance. Bitmaps are used to speed up the counting phase when packing, for requests that are not shallow. The PackWriterBitmapWalker uses a RevFilter to proactively mark commits with RevFlag.SEEN, when they appear in a bitmap. The walker produces the full closure of reachable ObjectIds, given the collection of starting ObjectIds. For fetch request, two ObjectWalks are executed to compute the ObjectIds reachable from the haves and from the wants. The ObjectIds needed to be written are determined by taking all the resulting wants AND NOT the haves. For clone requests, we get cached pack support for "free" since it is possible to determine if all of the ObjectIds in a pack file are included in the resulting list of ObjectIds to write. On my machine, the best times for clones and fetches of the linux kernel repository (with about 2.6M objects and 300K commits) are tabulated below: Operation Index V2 Index VE003 Clone 37530ms (524.06 MiB) 82ms (524.06 MiB) Fetch (1 commit back) 75ms 107ms Fetch (10 commits back) 456ms (269.51 KiB) 341ms (265.19 KiB) Fetch (100 commits back) 449ms (269.91 KiB) 337ms (267.28 KiB) Fetch (1000 commits back) 2229ms ( 14.75 MiB) 189ms ( 14.42 MiB) Fetch (10000 commits back) 2177ms ( 16.30 MiB) 254ms ( 15.88 MiB) Fetch (100000 commits back) 14340ms (185.83 MiB) 1655ms (189.39 MiB) Change-Id: Icdb0cdd66ff168917fb9ef17b96093990cc6a98d
12 years ago
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571572573574575576577578579580581582583584585586587588589590591592593594595596597598599600601602603604605606607608609610611612613614615616617618619620621622623624625626627628629630631632633634635636637638639640641642643644645646647648649650651652653654655656657658659660661662663664665666667668669670671672673674675676677678679680681682683684685686687688689690691692693694695696697698699700701702703704705706707708709710711712713714715716717718719720721722723724725726727728729730731732733734735736737738739740741742743744745746747748749750751752753754755756757758759760761762763764765766767768769770771772773774775776777778779780781782783784785786787788789790791792793794795796797798799800801802803804805806807808809810811812813814815816817818819820821822823824825826827828829830831832833834835836837838839840841842843844845846847848849850851852853854855856857858859860861862863864865866867868869870871872873874875876877878879880881882883884885886887888889890891892893894895896897898899900901902903904905906907908909910911912913914915916917918919920921922923924925926927928929930931932933934935936937938939940941942943944945946947948949950951952953954955956957958959960961962963964965966967968969970971972973974975976977978979980981982983984985986987988989990991992993994995996997998999100010011002100310041005100610071008100910101011101210131014101510161017101810191020102110221023102410251026102710281029103010311032103310341035103610371038103910401041104210431044104510461047104810491050105110521053105410551056105710581059106010611062106310641065106610671068106910701071107210731074107510761077
  1. /*
  2. * Copyright (C) 2008-2010, Google Inc.
  3. * Copyright (C) 2008, Marek Zawirski <marek.zawirski@gmail.com>
  4. * and other copyright owners as documented in the project's IP log.
  5. *
  6. * This program and the accompanying materials are made available
  7. * under the terms of the Eclipse Distribution License v1.0 which
  8. * accompanies this distribution, is reproduced below, and is
  9. * available at http://www.eclipse.org/org/documents/edl-v10.php
  10. *
  11. * All rights reserved.
  12. *
  13. * Redistribution and use in source and binary forms, with or
  14. * without modification, are permitted provided that the following
  15. * conditions are met:
  16. *
  17. * - Redistributions of source code must retain the above copyright
  18. * notice, this list of conditions and the following disclaimer.
  19. *
  20. * - Redistributions in binary form must reproduce the above
  21. * copyright notice, this list of conditions and the following
  22. * disclaimer in the documentation and/or other materials provided
  23. * with the distribution.
  24. *
  25. * - Neither the name of the Eclipse Foundation, Inc. nor the
  26. * names of its contributors may be used to endorse or promote
  27. * products derived from this software without specific prior
  28. * written permission.
  29. *
  30. * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
  31. * CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
  32. * INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
  33. * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
  34. * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
  35. * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
  36. * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
  37. * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
  38. * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
  39. * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
  40. * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
  41. * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
  42. * ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  43. */
  44. package org.eclipse.jgit.storage.pack;
  45. import java.util.concurrent.Executor;
  46. import java.util.zip.Deflater;
  47. import org.eclipse.jgit.internal.storage.file.PackIndexWriter;
  48. import org.eclipse.jgit.lib.Config;
  49. import org.eclipse.jgit.lib.Repository;
  50. /**
  51. * Configuration used by a pack writer when constructing the stream.
  52. *
  53. * A configuration may be modified once created, but should not be modified
  54. * while it is being used by a PackWriter. If a configuration is not modified it
  55. * is safe to share the same configuration instance between multiple concurrent
  56. * threads executing different PackWriters.
  57. */
  58. public class PackConfig {
  59. /**
  60. * Default value of deltas reuse option: {@value}
  61. *
  62. * @see #setReuseDeltas(boolean)
  63. */
  64. public static final boolean DEFAULT_REUSE_DELTAS = true;
  65. /**
  66. * Default value of objects reuse option: {@value}
  67. *
  68. * @see #setReuseObjects(boolean)
  69. */
  70. public static final boolean DEFAULT_REUSE_OBJECTS = true;
  71. /**
  72. * Default value of keep old packs option: {@value}
  73. *
  74. * @see #setPreserveOldPacks(boolean)
  75. */
  76. public static final boolean DEFAULT_PRESERVE_OLD_PACKS = false;
  77. /**
  78. * Default value of prune old packs option: {@value}
  79. *
  80. * @see #setPrunePreserved(boolean)
  81. */
  82. public static final boolean DEFAULT_PRUNE_PRESERVED = false;
  83. /**
  84. * Default value of delta compress option: {@value}
  85. *
  86. * @see #setDeltaCompress(boolean)
  87. */
  88. public static final boolean DEFAULT_DELTA_COMPRESS = true;
  89. /**
  90. * Default value of delta base as offset option: {@value}
  91. *
  92. * @see #setDeltaBaseAsOffset(boolean)
  93. */
  94. public static final boolean DEFAULT_DELTA_BASE_AS_OFFSET = false;
  95. /**
  96. * Default value of maximum delta chain depth: {@value}
  97. *
  98. * @see #setMaxDeltaDepth(int)
  99. */
  100. public static final int DEFAULT_MAX_DELTA_DEPTH = 50;
  101. /**
  102. * Default window size during packing: {@value}
  103. *
  104. * @see #setDeltaSearchWindowSize(int)
  105. */
  106. public static final int DEFAULT_DELTA_SEARCH_WINDOW_SIZE = 10;
  107. /**
  108. * Default big file threshold: {@value}
  109. *
  110. * @see #setBigFileThreshold(int)
  111. */
  112. public static final int DEFAULT_BIG_FILE_THRESHOLD = 50 * 1024 * 1024;
  113. /**
  114. * Default delta cache size: {@value}
  115. *
  116. * @see #setDeltaCacheSize(long)
  117. */
  118. public static final long DEFAULT_DELTA_CACHE_SIZE = 50 * 1024 * 1024;
  119. /**
  120. * Default delta cache limit: {@value}
  121. *
  122. * @see #setDeltaCacheLimit(int)
  123. */
  124. public static final int DEFAULT_DELTA_CACHE_LIMIT = 100;
  125. /**
  126. * Default index version: {@value}
  127. *
  128. * @see #setIndexVersion(int)
  129. */
  130. public static final int DEFAULT_INDEX_VERSION = 2;
  131. /**
  132. * Default value of the build bitmaps option: {@value}
  133. *
  134. * @see #setBuildBitmaps(boolean)
  135. * @since 3.0
  136. */
  137. public static final boolean DEFAULT_BUILD_BITMAPS = true;
  138. /**
  139. * Default count of most recent commits to select for bitmaps. Only applies
  140. * when bitmaps are enabled: {@value}
  141. *
  142. * @see #setBitmapContiguousCommitCount(int)
  143. * @since 4.2
  144. */
  145. public static final int DEFAULT_BITMAP_CONTIGUOUS_COMMIT_COUNT = 100;
  146. /**
  147. * Count at which the span between selected commits changes from
  148. * "bitmapRecentCommitSpan" to "bitmapDistantCommitSpan". Only applies when
  149. * bitmaps are enabled: {@value}
  150. *
  151. * @see #setBitmapRecentCommitCount(int)
  152. * @since 4.2
  153. */
  154. public static final int DEFAULT_BITMAP_RECENT_COMMIT_COUNT = 20000;
  155. /**
  156. * Default spacing between commits in recent history when selecting commits
  157. * for bitmaps. Only applies when bitmaps are enabled: {@value}
  158. *
  159. * @see #setBitmapRecentCommitSpan(int)
  160. * @since 4.2
  161. */
  162. public static final int DEFAULT_BITMAP_RECENT_COMMIT_SPAN = 100;
  163. /**
  164. * Default spacing between commits in distant history when selecting commits
  165. * for bitmaps. Only applies when bitmaps are enabled: {@value}
  166. *
  167. * @see #setBitmapDistantCommitSpan(int)
  168. * @since 4.2
  169. */
  170. public static final int DEFAULT_BITMAP_DISTANT_COMMIT_SPAN = 5000;
  171. /**
  172. * Default count of branches required to activate inactive branch commit
  173. * selection. If the number of branches is less than this then bitmaps for
  174. * the entire commit history of all branches will be created, otherwise
  175. * branches marked as "inactive" will have coverage for only partial
  176. * history: {@value}
  177. *
  178. * @see #setBitmapExcessiveBranchCount(int)
  179. * @since 4.2
  180. */
  181. public static final int DEFAULT_BITMAP_EXCESSIVE_BRANCH_COUNT = 100;
  182. /**
  183. * Default age at which a branch is considered inactive. Age is taken as the
  184. * number of days ago that the most recent commit was made to a branch. Only
  185. * affects bitmap processing if bitmaps are enabled and the
  186. * "excessive branch count" has been exceeded: {@value}
  187. *
  188. * @see #setBitmapInactiveBranchAgeInDays(int)
  189. * @since 4.2
  190. */
  191. public static final int DEFAULT_BITMAP_INACTIVE_BRANCH_AGE_IN_DAYS = 90;
  192. private int compressionLevel = Deflater.DEFAULT_COMPRESSION;
  193. private boolean reuseDeltas = DEFAULT_REUSE_DELTAS;
  194. private boolean reuseObjects = DEFAULT_REUSE_OBJECTS;
  195. private boolean preserveOldPacks = DEFAULT_PRESERVE_OLD_PACKS;
  196. private boolean prunePreserved = DEFAULT_PRUNE_PRESERVED;
  197. private boolean deltaBaseAsOffset = DEFAULT_DELTA_BASE_AS_OFFSET;
  198. private boolean deltaCompress = DEFAULT_DELTA_COMPRESS;
  199. private int maxDeltaDepth = DEFAULT_MAX_DELTA_DEPTH;
  200. private int deltaSearchWindowSize = DEFAULT_DELTA_SEARCH_WINDOW_SIZE;
  201. private long deltaSearchMemoryLimit;
  202. private long deltaCacheSize = DEFAULT_DELTA_CACHE_SIZE;
  203. private int deltaCacheLimit = DEFAULT_DELTA_CACHE_LIMIT;
  204. private int bigFileThreshold = DEFAULT_BIG_FILE_THRESHOLD;
  205. private int threads;
  206. private Executor executor;
  207. private int indexVersion = DEFAULT_INDEX_VERSION;
  208. private boolean buildBitmaps = DEFAULT_BUILD_BITMAPS;
  209. private int bitmapContiguousCommitCount = DEFAULT_BITMAP_CONTIGUOUS_COMMIT_COUNT;
  210. private int bitmapRecentCommitCount = DEFAULT_BITMAP_RECENT_COMMIT_COUNT;
  211. private int bitmapRecentCommitSpan = DEFAULT_BITMAP_RECENT_COMMIT_SPAN;
  212. private int bitmapDistantCommitSpan = DEFAULT_BITMAP_DISTANT_COMMIT_SPAN;
  213. private int bitmapExcessiveBranchCount = DEFAULT_BITMAP_EXCESSIVE_BRANCH_COUNT;
  214. private int bitmapInactiveBranchAgeInDays = DEFAULT_BITMAP_INACTIVE_BRANCH_AGE_IN_DAYS;
  215. private boolean cutDeltaChains;
  216. /** Create a default configuration. */
  217. public PackConfig() {
  218. // Fields are initialized to defaults.
  219. }
  220. /**
  221. * Create a configuration honoring the repository's settings.
  222. *
  223. * @param db
  224. * the repository to read settings from. The repository is not
  225. * retained by the new configuration, instead its settings are
  226. * copied during the constructor.
  227. */
  228. public PackConfig(Repository db) {
  229. fromConfig(db.getConfig());
  230. }
  231. /**
  232. * Create a configuration honoring settings in a {@link Config}.
  233. *
  234. * @param cfg
  235. * the source to read settings from. The source is not retained
  236. * by the new configuration, instead its settings are copied
  237. * during the constructor.
  238. */
  239. public PackConfig(Config cfg) {
  240. fromConfig(cfg);
  241. }
  242. /**
  243. * Copy an existing configuration to a new instance.
  244. *
  245. * @param cfg
  246. * the source configuration to copy from.
  247. */
  248. public PackConfig(PackConfig cfg) {
  249. this.compressionLevel = cfg.compressionLevel;
  250. this.reuseDeltas = cfg.reuseDeltas;
  251. this.reuseObjects = cfg.reuseObjects;
  252. this.preserveOldPacks = cfg.preserveOldPacks;
  253. this.prunePreserved = cfg.prunePreserved;
  254. this.deltaBaseAsOffset = cfg.deltaBaseAsOffset;
  255. this.deltaCompress = cfg.deltaCompress;
  256. this.maxDeltaDepth = cfg.maxDeltaDepth;
  257. this.deltaSearchWindowSize = cfg.deltaSearchWindowSize;
  258. this.deltaSearchMemoryLimit = cfg.deltaSearchMemoryLimit;
  259. this.deltaCacheSize = cfg.deltaCacheSize;
  260. this.deltaCacheLimit = cfg.deltaCacheLimit;
  261. this.bigFileThreshold = cfg.bigFileThreshold;
  262. this.threads = cfg.threads;
  263. this.executor = cfg.executor;
  264. this.indexVersion = cfg.indexVersion;
  265. this.buildBitmaps = cfg.buildBitmaps;
  266. this.bitmapContiguousCommitCount = cfg.bitmapContiguousCommitCount;
  267. this.bitmapRecentCommitCount = cfg.bitmapRecentCommitCount;
  268. this.bitmapRecentCommitSpan = cfg.bitmapRecentCommitSpan;
  269. this.bitmapDistantCommitSpan = cfg.bitmapDistantCommitSpan;
  270. this.bitmapExcessiveBranchCount = cfg.bitmapExcessiveBranchCount;
  271. this.bitmapInactiveBranchAgeInDays = cfg.bitmapInactiveBranchAgeInDays;
  272. this.cutDeltaChains = cfg.cutDeltaChains;
  273. }
  274. /**
  275. * Check whether to reuse deltas existing in repository.
  276. *
  277. * Default setting: {@value #DEFAULT_REUSE_DELTAS}
  278. *
  279. * @return true if object is configured to reuse deltas; false otherwise.
  280. */
  281. public boolean isReuseDeltas() {
  282. return reuseDeltas;
  283. }
  284. /**
  285. * Set reuse deltas configuration option for the writer.
  286. *
  287. * When enabled, writer will search for delta representation of object in
  288. * repository and use it if possible. Normally, only deltas with base to
  289. * another object existing in set of objects to pack will be used. The
  290. * exception however is thin-packs where the base object may exist on the
  291. * other side.
  292. *
  293. * When raw delta data is directly copied from a pack file, its checksum is
  294. * computed to verify the data is not corrupt.
  295. *
  296. * Default setting: {@value #DEFAULT_REUSE_DELTAS}
  297. *
  298. * @param reuseDeltas
  299. * boolean indicating whether or not try to reuse deltas.
  300. */
  301. public void setReuseDeltas(boolean reuseDeltas) {
  302. this.reuseDeltas = reuseDeltas;
  303. }
  304. /**
  305. * Checks whether to reuse existing objects representation in repository.
  306. *
  307. * Default setting: {@value #DEFAULT_REUSE_OBJECTS}
  308. *
  309. * @return true if writer is configured to reuse objects representation from
  310. * pack; false otherwise.
  311. */
  312. public boolean isReuseObjects() {
  313. return reuseObjects;
  314. }
  315. /**
  316. * Set reuse objects configuration option for the writer.
  317. *
  318. * If enabled, writer searches for compressed representation in a pack file.
  319. * If possible, compressed data is directly copied from such a pack file.
  320. * Data checksum is verified.
  321. *
  322. * Default setting: {@value #DEFAULT_REUSE_OBJECTS}
  323. *
  324. * @param reuseObjects
  325. * boolean indicating whether or not writer should reuse existing
  326. * objects representation.
  327. */
  328. public void setReuseObjects(boolean reuseObjects) {
  329. this.reuseObjects = reuseObjects;
  330. }
  331. /**
  332. * Checks whether to preserve old packs in a preserved directory
  333. *
  334. * Default setting: {@value #DEFAULT_PRESERVE_OLD_PACKS}
  335. *
  336. * @return true if repacking will preserve old pack files.
  337. * @since 4.7
  338. */
  339. public boolean isPreserveOldPacks() {
  340. return preserveOldPacks;
  341. }
  342. /**
  343. * Set preserve old packs configuration option for repacking.
  344. *
  345. * If enabled, old pack files are moved into a preserved subdirectory instead
  346. * of being deleted
  347. *
  348. * Default setting: {@value #DEFAULT_PRESERVE_OLD_PACKS}
  349. *
  350. * @param preserveOldPacks
  351. * boolean indicating whether or not preserve old pack files
  352. * @since 4.7
  353. */
  354. public void setPreserveOldPacks(boolean preserveOldPacks) {
  355. this.preserveOldPacks = preserveOldPacks;
  356. }
  357. /**
  358. * Checks whether to remove preserved pack files in a preserved directory
  359. *
  360. * Default setting: {@value #DEFAULT_PRUNE_PRESERVED}
  361. *
  362. * @return true if repacking will remove preserved pack files.
  363. * @since 4.7
  364. */
  365. public boolean isPrunePreserved() {
  366. return prunePreserved;
  367. }
  368. /**
  369. * Set prune preserved configuration option for repacking.
  370. *
  371. * If enabled, preserved pack files are removed from a preserved subdirectory
  372. *
  373. * Default setting: {@value #DEFAULT_PRESERVE_OLD_PACKS}
  374. *
  375. * @param prunePreserved
  376. * boolean indicating whether or not preserve old pack files
  377. * @since 4.7
  378. */
  379. public void setPrunePreserved(boolean prunePreserved) {
  380. this.prunePreserved = prunePreserved;
  381. }
  382. /**
  383. * True if writer can use offsets to point to a delta base.
  384. *
  385. * If true the writer may choose to use an offset to point to a delta base
  386. * in the same pack, this is a newer style of reference that saves space.
  387. * False if the writer has to use the older (and more compatible style) of
  388. * storing the full ObjectId of the delta base.
  389. *
  390. * Default setting: {@value #DEFAULT_DELTA_BASE_AS_OFFSET}
  391. *
  392. * @return true if delta base is stored as an offset; false if it is stored
  393. * as an ObjectId.
  394. */
  395. public boolean isDeltaBaseAsOffset() {
  396. return deltaBaseAsOffset;
  397. }
  398. /**
  399. * Set writer delta base format.
  400. *
  401. * Delta base can be written as an offset in a pack file (new approach
  402. * reducing file size) or as an object id (legacy approach, compatible with
  403. * old readers).
  404. *
  405. * Default setting: {@value #DEFAULT_DELTA_BASE_AS_OFFSET}
  406. *
  407. * @param deltaBaseAsOffset
  408. * boolean indicating whether delta base can be stored as an
  409. * offset.
  410. */
  411. public void setDeltaBaseAsOffset(boolean deltaBaseAsOffset) {
  412. this.deltaBaseAsOffset = deltaBaseAsOffset;
  413. }
  414. /**
  415. * Check whether the writer will create new deltas on the fly.
  416. *
  417. * Default setting: {@value #DEFAULT_DELTA_COMPRESS}
  418. *
  419. * @return true if the writer will create a new delta when either
  420. * {@link #isReuseDeltas()} is false, or no suitable delta is
  421. * available for reuse.
  422. */
  423. public boolean isDeltaCompress() {
  424. return deltaCompress;
  425. }
  426. /**
  427. * Set whether or not the writer will create new deltas on the fly.
  428. *
  429. * Default setting: {@value #DEFAULT_DELTA_COMPRESS}
  430. *
  431. * @param deltaCompress
  432. * true to create deltas when {@link #isReuseDeltas()} is false,
  433. * or when a suitable delta isn't available for reuse. Set to
  434. * false to write whole objects instead.
  435. */
  436. public void setDeltaCompress(boolean deltaCompress) {
  437. this.deltaCompress = deltaCompress;
  438. }
  439. /**
  440. * Get maximum depth of delta chain set up for the writer.
  441. *
  442. * Generated chains are not longer than this value.
  443. *
  444. * Default setting: {@value #DEFAULT_MAX_DELTA_DEPTH}
  445. *
  446. * @return maximum delta chain depth.
  447. */
  448. public int getMaxDeltaDepth() {
  449. return maxDeltaDepth;
  450. }
  451. /**
  452. * Set up maximum depth of delta chain for the writer.
  453. *
  454. * Generated chains are not longer than this value. Too low value causes low
  455. * compression level, while too big makes unpacking (reading) longer.
  456. *
  457. * Default setting: {@value #DEFAULT_MAX_DELTA_DEPTH}
  458. *
  459. * @param maxDeltaDepth
  460. * maximum delta chain depth.
  461. */
  462. public void setMaxDeltaDepth(int maxDeltaDepth) {
  463. this.maxDeltaDepth = maxDeltaDepth;
  464. }
  465. /**
  466. * @return true if existing delta chains should be cut at
  467. * {@link #getMaxDeltaDepth()}. Default is false, allowing existing
  468. * chains to be of any length.
  469. * @since 3.0
  470. */
  471. public boolean getCutDeltaChains() {
  472. return cutDeltaChains;
  473. }
  474. /**
  475. * Enable cutting existing delta chains at {@link #getMaxDeltaDepth()}.
  476. *
  477. * By default this is disabled and existing chains are kept at whatever
  478. * length a prior packer was configured to create. This allows objects to be
  479. * packed one with a large depth (for example 250), and later to quickly
  480. * repack the repository with a shorter depth (such as 50), but reusing the
  481. * complete delta chains created by the earlier 250 depth.
  482. *
  483. * @param cut
  484. * true to cut existing chains.
  485. * @since 3.0
  486. */
  487. public void setCutDeltaChains(boolean cut) {
  488. cutDeltaChains = cut;
  489. }
  490. /**
  491. * Get the number of objects to try when looking for a delta base.
  492. *
  493. * This limit is per thread, if 4 threads are used the actual memory used
  494. * will be 4 times this value.
  495. *
  496. * Default setting: {@value #DEFAULT_DELTA_SEARCH_WINDOW_SIZE}
  497. *
  498. * @return the object count to be searched.
  499. */
  500. public int getDeltaSearchWindowSize() {
  501. return deltaSearchWindowSize;
  502. }
  503. /**
  504. * Set the number of objects considered when searching for a delta base.
  505. *
  506. * Default setting: {@value #DEFAULT_DELTA_SEARCH_WINDOW_SIZE}
  507. *
  508. * @param objectCount
  509. * number of objects to search at once. Must be at least 2.
  510. */
  511. public void setDeltaSearchWindowSize(int objectCount) {
  512. if (objectCount <= 2)
  513. setDeltaCompress(false);
  514. else
  515. deltaSearchWindowSize = objectCount;
  516. }
  517. /**
  518. * Get maximum number of bytes to put into the delta search window.
  519. *
  520. * Default setting is 0, for an unlimited amount of memory usage. Actual
  521. * memory used is the lower limit of either this setting, or the sum of
  522. * space used by at most {@link #getDeltaSearchWindowSize()} objects.
  523. *
  524. * This limit is per thread, if 4 threads are used the actual memory limit
  525. * will be 4 times this value.
  526. *
  527. * @return the memory limit.
  528. */
  529. public long getDeltaSearchMemoryLimit() {
  530. return deltaSearchMemoryLimit;
  531. }
  532. /**
  533. * Set the maximum number of bytes to put into the delta search window.
  534. *
  535. * Default setting is 0, for an unlimited amount of memory usage. If the
  536. * memory limit is reached before {@link #getDeltaSearchWindowSize()} the
  537. * window size is temporarily lowered.
  538. *
  539. * @param memoryLimit
  540. * Maximum number of bytes to load at once, 0 for unlimited.
  541. */
  542. public void setDeltaSearchMemoryLimit(long memoryLimit) {
  543. deltaSearchMemoryLimit = memoryLimit;
  544. }
  545. /**
  546. * Get the size of the in-memory delta cache.
  547. *
  548. * This limit is for the entire writer, even if multiple threads are used.
  549. *
  550. * Default setting: {@value #DEFAULT_DELTA_CACHE_SIZE}
  551. *
  552. * @return maximum number of bytes worth of delta data to cache in memory.
  553. * If 0 the cache is infinite in size (up to the JVM heap limit
  554. * anyway). A very tiny size such as 1 indicates the cache is
  555. * effectively disabled.
  556. */
  557. public long getDeltaCacheSize() {
  558. return deltaCacheSize;
  559. }
  560. /**
  561. * Set the maximum number of bytes of delta data to cache.
  562. *
  563. * During delta search, up to this many bytes worth of small or hard to
  564. * compute deltas will be stored in memory. This cache speeds up writing by
  565. * allowing the cached entry to simply be dumped to the output stream.
  566. *
  567. * Default setting: {@value #DEFAULT_DELTA_CACHE_SIZE}
  568. *
  569. * @param size
  570. * number of bytes to cache. Set to 0 to enable an infinite
  571. * cache, set to 1 (an impossible size for any delta) to disable
  572. * the cache.
  573. */
  574. public void setDeltaCacheSize(long size) {
  575. deltaCacheSize = size;
  576. }
  577. /**
  578. * Maximum size in bytes of a delta to cache.
  579. *
  580. * Default setting: {@value #DEFAULT_DELTA_CACHE_LIMIT}
  581. *
  582. * @return maximum size (in bytes) of a delta that should be cached.
  583. */
  584. public int getDeltaCacheLimit() {
  585. return deltaCacheLimit;
  586. }
  587. /**
  588. * Set the maximum size of a delta that should be cached.
  589. *
  590. * During delta search, any delta smaller than this size will be cached, up
  591. * to the {@link #getDeltaCacheSize()} maximum limit. This speeds up writing
  592. * by allowing these cached deltas to be output as-is.
  593. *
  594. * Default setting: {@value #DEFAULT_DELTA_CACHE_LIMIT}
  595. *
  596. * @param size
  597. * maximum size (in bytes) of a delta to be cached.
  598. */
  599. public void setDeltaCacheLimit(int size) {
  600. deltaCacheLimit = size;
  601. }
  602. /**
  603. * Get the maximum file size that will be delta compressed.
  604. *
  605. * Files bigger than this setting will not be delta compressed, as they are
  606. * more than likely already highly compressed binary data files that do not
  607. * delta compress well, such as MPEG videos.
  608. *
  609. * Default setting: {@value #DEFAULT_BIG_FILE_THRESHOLD}
  610. *
  611. * @return the configured big file threshold.
  612. */
  613. public int getBigFileThreshold() {
  614. return bigFileThreshold;
  615. }
  616. /**
  617. * Set the maximum file size that should be considered for deltas.
  618. *
  619. * Default setting: {@value #DEFAULT_BIG_FILE_THRESHOLD}
  620. *
  621. * @param bigFileThreshold
  622. * the limit, in bytes.
  623. */
  624. public void setBigFileThreshold(int bigFileThreshold) {
  625. this.bigFileThreshold = bigFileThreshold;
  626. }
  627. /**
  628. * Get the compression level applied to objects in the pack.
  629. *
  630. * Default setting: {@value java.util.zip.Deflater#DEFAULT_COMPRESSION}
  631. *
  632. * @return current compression level, see {@link java.util.zip.Deflater}.
  633. */
  634. public int getCompressionLevel() {
  635. return compressionLevel;
  636. }
  637. /**
  638. * Set the compression level applied to objects in the pack.
  639. *
  640. * Default setting: {@value java.util.zip.Deflater#DEFAULT_COMPRESSION}
  641. *
  642. * @param level
  643. * compression level, must be a valid level recognized by the
  644. * {@link java.util.zip.Deflater} class.
  645. */
  646. public void setCompressionLevel(int level) {
  647. compressionLevel = level;
  648. }
  649. /**
  650. * Get the number of threads used during delta compression.
  651. *
  652. * Default setting: 0 (auto-detect processors)
  653. *
  654. * @return number of threads used for delta compression. 0 will auto-detect
  655. * the threads to the number of available processors.
  656. */
  657. public int getThreads() {
  658. return threads;
  659. }
  660. /**
  661. * Set the number of threads to use for delta compression.
  662. *
  663. * During delta compression, if there are enough objects to be considered
  664. * the writer will start up concurrent threads and allow them to compress
  665. * different sections of the repository concurrently.
  666. *
  667. * An application thread pool can be set by {@link #setExecutor(Executor)}.
  668. * If not set a temporary pool will be created by the writer, and torn down
  669. * automatically when compression is over.
  670. *
  671. * Default setting: 0 (auto-detect processors)
  672. *
  673. * @param threads
  674. * number of threads to use. If &lt;= 0 the number of available
  675. * processors for this JVM is used.
  676. */
  677. public void setThreads(int threads) {
  678. this.threads = threads;
  679. }
  680. /** @return the preferred thread pool to execute delta search on. */
  681. public Executor getExecutor() {
  682. return executor;
  683. }
  684. /**
  685. * Set the executor to use when using threads.
  686. *
  687. * During delta compression if the executor is non-null jobs will be queued
  688. * up on it to perform delta compression in parallel. Aside from setting the
  689. * executor, the caller must set {@link #setThreads(int)} to enable threaded
  690. * delta search.
  691. *
  692. * @param executor
  693. * executor to use for threads. Set to null to create a temporary
  694. * executor just for the writer.
  695. */
  696. public void setExecutor(Executor executor) {
  697. this.executor = executor;
  698. }
  699. /**
  700. * Get the pack index file format version this instance creates.
  701. *
  702. * Default setting: {@value #DEFAULT_INDEX_VERSION}
  703. *
  704. * @return the index version, the special version 0 designates the oldest
  705. * (most compatible) format available for the objects.
  706. * @see PackIndexWriter
  707. */
  708. public int getIndexVersion() {
  709. return indexVersion;
  710. }
  711. /**
  712. * Set the pack index file format version this instance will create.
  713. *
  714. * Default setting: {@value #DEFAULT_INDEX_VERSION}
  715. *
  716. * @param version
  717. * the version to write. The special version 0 designates the
  718. * oldest (most compatible) format available for the objects.
  719. * @see PackIndexWriter
  720. */
  721. public void setIndexVersion(int version) {
  722. indexVersion = version;
  723. }
  724. /**
  725. * True if writer is allowed to build bitmaps for indexes.
  726. *
  727. * Default setting: {@value #DEFAULT_BUILD_BITMAPS}
  728. *
  729. * @return true if delta base is the writer can choose to output an index
  730. * with bitmaps.
  731. * @since 3.0
  732. */
  733. public boolean isBuildBitmaps() {
  734. return buildBitmaps;
  735. }
  736. /**
  737. * Set writer to allow building bitmaps for supported pack files.
  738. *
  739. * Index files can include bitmaps to speed up future ObjectWalks.
  740. *
  741. * Default setting: {@value #DEFAULT_BUILD_BITMAPS}
  742. *
  743. * @param buildBitmaps
  744. * boolean indicating whether bitmaps may be included in the
  745. * index.
  746. * @since 3.0
  747. */
  748. public void setBuildBitmaps(boolean buildBitmaps) {
  749. this.buildBitmaps = buildBitmaps;
  750. }
  751. /**
  752. * Get the count of most recent commits for which to build bitmaps.
  753. *
  754. * Default setting: {@value #DEFAULT_BITMAP_CONTIGUOUS_COMMIT_COUNT}
  755. *
  756. * @return the count of most recent commits for which to build bitmaps
  757. * @since 4.2
  758. */
  759. public int getBitmapContiguousCommitCount() {
  760. return bitmapContiguousCommitCount;
  761. }
  762. /**
  763. * Set the count of most recent commits for which to build bitmaps.
  764. *
  765. * Default setting: {@value #DEFAULT_BITMAP_CONTIGUOUS_COMMIT_COUNT}
  766. *
  767. * @param count
  768. * the count of most recent commits for which to build bitmaps
  769. * @since 4.2
  770. */
  771. public void setBitmapContiguousCommitCount(int count) {
  772. bitmapContiguousCommitCount = count;
  773. }
  774. /**
  775. * Get the count at which to switch from "bitmapRecentCommitSpan" to
  776. * "bitmapDistantCommitSpan".
  777. *
  778. * Default setting: {@value #DEFAULT_BITMAP_RECENT_COMMIT_COUNT}
  779. *
  780. * @return the count for switching between recent and distant spans
  781. * @since 4.2
  782. */
  783. public int getBitmapRecentCommitCount() {
  784. return bitmapRecentCommitCount;
  785. }
  786. /**
  787. * Set the count at which to switch from "bitmapRecentCommitSpan" to
  788. * "bitmapDistantCommitSpan".
  789. *
  790. * Default setting: {@value #DEFAULT_BITMAP_RECENT_COMMIT_COUNT}
  791. *
  792. * @param count
  793. * the count for switching between recent and distant spans
  794. * @since 4.2
  795. */
  796. public void setBitmapRecentCommitCount(int count) {
  797. bitmapRecentCommitCount = count;
  798. }
  799. /**
  800. * Get the span of commits when building bitmaps for recent history.
  801. *
  802. * Default setting: {@value #DEFAULT_BITMAP_RECENT_COMMIT_SPAN}
  803. *
  804. * @return the span of commits when building bitmaps for recent history
  805. * @since 4.2
  806. */
  807. public int getBitmapRecentCommitSpan() {
  808. return bitmapRecentCommitSpan;
  809. }
  810. /**
  811. * Set the span of commits when building bitmaps for recent history.
  812. *
  813. * Default setting: {@value #DEFAULT_BITMAP_RECENT_COMMIT_SPAN}
  814. *
  815. * @param span
  816. * the span of commits when building bitmaps for recent history
  817. * @since 4.2
  818. */
  819. public void setBitmapRecentCommitSpan(int span) {
  820. bitmapRecentCommitSpan = span;
  821. }
  822. /**
  823. * Get the span of commits when building bitmaps for distant history.
  824. *
  825. * Default setting: {@value #DEFAULT_BITMAP_DISTANT_COMMIT_SPAN}
  826. *
  827. * @return the span of commits when building bitmaps for distant history
  828. * @since 4.2
  829. */
  830. public int getBitmapDistantCommitSpan() {
  831. return bitmapDistantCommitSpan;
  832. }
  833. /**
  834. * Set the span of commits when building bitmaps for distant history.
  835. *
  836. * Default setting: {@value #DEFAULT_BITMAP_DISTANT_COMMIT_SPAN}
  837. *
  838. * @param span
  839. * the span of commits when building bitmaps for distant history
  840. * @since 4.2
  841. */
  842. public void setBitmapDistantCommitSpan(int span) {
  843. bitmapDistantCommitSpan = span;
  844. }
  845. /**
  846. * Get the count of branches deemed "excessive". If the count of branches in
  847. * a repository exceeds this number and bitmaps are enabled, "inactive"
  848. * branches will have fewer bitmaps than "active" branches.
  849. *
  850. * Default setting: {@value #DEFAULT_BITMAP_EXCESSIVE_BRANCH_COUNT}
  851. *
  852. * @return the count of branches deemed "excessive"
  853. * @since 4.2
  854. */
  855. public int getBitmapExcessiveBranchCount() {
  856. return bitmapExcessiveBranchCount;
  857. }
  858. /**
  859. * Set the count of branches deemed "excessive". If the count of branches in
  860. * a repository exceeds this number and bitmaps are enabled, "inactive"
  861. * branches will have fewer bitmaps than "active" branches.
  862. *
  863. * Default setting: {@value #DEFAULT_BITMAP_EXCESSIVE_BRANCH_COUNT}
  864. *
  865. * @param count
  866. * the count of branches deemed "excessive"
  867. * @since 4.2
  868. */
  869. public void setBitmapExcessiveBranchCount(int count) {
  870. bitmapExcessiveBranchCount = count;
  871. }
  872. /**
  873. * Get the the age in days that marks a branch as "inactive".
  874. *
  875. * Default setting: {@value #DEFAULT_BITMAP_INACTIVE_BRANCH_AGE_IN_DAYS}
  876. *
  877. * @return the age in days that marks a branch as "inactive"
  878. * @since 4.2
  879. */
  880. public int getBitmapInactiveBranchAgeInDays() {
  881. return bitmapInactiveBranchAgeInDays;
  882. }
  883. /**
  884. * Set the the age in days that marks a branch as "inactive".
  885. *
  886. * Default setting: {@value #DEFAULT_BITMAP_INACTIVE_BRANCH_AGE_IN_DAYS}
  887. *
  888. * @param ageInDays
  889. * the age in days that marks a branch as "inactive"
  890. * @since 4.2
  891. */
  892. public void setBitmapInactiveBranchAgeInDays(int ageInDays) {
  893. bitmapInactiveBranchAgeInDays = ageInDays;
  894. }
  895. /**
  896. * Update properties by setting fields from the configuration.
  897. *
  898. * If a property's corresponding variable is not defined in the supplied
  899. * configuration, then it is left unmodified.
  900. *
  901. * @param rc
  902. * configuration to read properties from.
  903. */
  904. public void fromConfig(final Config rc) {
  905. setMaxDeltaDepth(rc.getInt("pack", "depth", getMaxDeltaDepth())); //$NON-NLS-1$ //$NON-NLS-2$
  906. setDeltaSearchWindowSize(rc.getInt(
  907. "pack", "window", getDeltaSearchWindowSize())); //$NON-NLS-1$ //$NON-NLS-2$
  908. setDeltaSearchMemoryLimit(rc.getLong(
  909. "pack", "windowmemory", getDeltaSearchMemoryLimit())); //$NON-NLS-1$ //$NON-NLS-2$
  910. setDeltaCacheSize(rc.getLong(
  911. "pack", "deltacachesize", getDeltaCacheSize())); //$NON-NLS-1$ //$NON-NLS-2$
  912. setDeltaCacheLimit(rc.getInt(
  913. "pack", "deltacachelimit", getDeltaCacheLimit())); //$NON-NLS-1$ //$NON-NLS-2$
  914. setCompressionLevel(rc.getInt("pack", "compression", //$NON-NLS-1$ //$NON-NLS-2$
  915. rc.getInt("core", "compression", getCompressionLevel()))); //$NON-NLS-1$ //$NON-NLS-2$
  916. setIndexVersion(rc.getInt("pack", "indexversion", getIndexVersion())); //$NON-NLS-1$ //$NON-NLS-2$
  917. setBigFileThreshold(rc.getInt(
  918. "core", "bigfilethreshold", getBigFileThreshold())); //$NON-NLS-1$ //$NON-NLS-2$
  919. setThreads(rc.getInt("pack", "threads", getThreads())); //$NON-NLS-1$ //$NON-NLS-2$
  920. // These variables aren't standardized
  921. //
  922. setReuseDeltas(rc.getBoolean("pack", "reusedeltas", isReuseDeltas())); //$NON-NLS-1$ //$NON-NLS-2$
  923. setReuseObjects(
  924. rc.getBoolean("pack", "reuseobjects", isReuseObjects())); //$NON-NLS-1$ //$NON-NLS-2$
  925. setDeltaCompress(
  926. rc.getBoolean("pack", "deltacompression", isDeltaCompress())); //$NON-NLS-1$ //$NON-NLS-2$
  927. setCutDeltaChains(
  928. rc.getBoolean("pack", "cutdeltachains", getCutDeltaChains())); //$NON-NLS-1$ //$NON-NLS-2$
  929. setBuildBitmaps(
  930. rc.getBoolean("pack", "buildbitmaps", isBuildBitmaps())); //$NON-NLS-1$ //$NON-NLS-2$
  931. setBitmapContiguousCommitCount(
  932. rc.getInt("pack", "bitmapcontiguouscommitcount", //$NON-NLS-1$ //$NON-NLS-2$
  933. getBitmapContiguousCommitCount()));
  934. setBitmapRecentCommitCount(rc.getInt("pack", "bitmaprecentcommitcount", //$NON-NLS-1$ //$NON-NLS-2$
  935. getBitmapRecentCommitCount()));
  936. setBitmapRecentCommitSpan(rc.getInt("pack", "bitmaprecentcommitspan", //$NON-NLS-1$ //$NON-NLS-2$
  937. getBitmapRecentCommitSpan()));
  938. setBitmapDistantCommitSpan(rc.getInt("pack", "bitmapdistantcommitspan", //$NON-NLS-1$ //$NON-NLS-2$
  939. getBitmapDistantCommitSpan()));
  940. setBitmapExcessiveBranchCount(rc.getInt("pack", //$NON-NLS-1$
  941. "bitmapexcessivebranchcount", getBitmapExcessiveBranchCount())); //$NON-NLS-1$
  942. setBitmapInactiveBranchAgeInDays(
  943. rc.getInt("pack", "bitmapinactivebranchageindays", //$NON-NLS-1$ //$NON-NLS-2$
  944. getBitmapInactiveBranchAgeInDays()));
  945. }
  946. public String toString() {
  947. final StringBuilder b = new StringBuilder();
  948. b.append("maxDeltaDepth=").append(getMaxDeltaDepth()); //$NON-NLS-1$
  949. b.append(", deltaSearchWindowSize=").append(getDeltaSearchWindowSize()); //$NON-NLS-1$
  950. b.append(", deltaSearchMemoryLimit=") //$NON-NLS-1$
  951. .append(getDeltaSearchMemoryLimit());
  952. b.append(", deltaCacheSize=").append(getDeltaCacheSize()); //$NON-NLS-1$
  953. b.append(", deltaCacheLimit=").append(getDeltaCacheLimit()); //$NON-NLS-1$
  954. b.append(", compressionLevel=").append(getCompressionLevel()); //$NON-NLS-1$
  955. b.append(", indexVersion=").append(getIndexVersion()); //$NON-NLS-1$
  956. b.append(", bigFileThreshold=").append(getBigFileThreshold()); //$NON-NLS-1$
  957. b.append(", threads=").append(getThreads()); //$NON-NLS-1$
  958. b.append(", reuseDeltas=").append(isReuseDeltas()); //$NON-NLS-1$
  959. b.append(", reuseObjects=").append(isReuseObjects()); //$NON-NLS-1$
  960. b.append(", deltaCompress=").append(isDeltaCompress()); //$NON-NLS-1$
  961. b.append(", buildBitmaps=").append(isBuildBitmaps()); //$NON-NLS-1$
  962. b.append(", bitmapContiguousCommitCount=") //$NON-NLS-1$
  963. .append(getBitmapContiguousCommitCount());
  964. b.append(", bitmapRecentCommitCount=") //$NON-NLS-1$
  965. .append(getBitmapRecentCommitCount());
  966. b.append(", bitmapRecentCommitSpan=") //$NON-NLS-1$
  967. .append(getBitmapRecentCommitSpan());
  968. b.append(", bitmapDistantCommitSpan=") //$NON-NLS-1$
  969. .append(getBitmapDistantCommitSpan());
  970. b.append(", bitmapExcessiveBranchCount=") //$NON-NLS-1$
  971. .append(getBitmapExcessiveBranchCount());
  972. b.append(", bitmapInactiveBranchAge=") //$NON-NLS-1$
  973. .append(getBitmapInactiveBranchAgeInDays());
  974. return b.toString();
  975. }
  976. }