You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

MANIFEST.MF 7.7KB

Move reachability checker generation into the ObjectReader object Reachability checkers are retrieved from RevWalk and ObjectWalk objects: * RevWalk.createReachabilityChecker() * ObjectWalk.createObjectReachabilityChecker() Since RevWalks and ObjectWalks are themselves directly instantiated in hundreds of places (e.g. UploadPack...) overriding them in a consistent way requires overloading 100s of methods, which isn't feasible. Moving reachability checker generation to a more central place solves that problem. The ObjectReader object seems a good place from which to get reachability checkers, because reachability checkers return information about relationships between objects. ObjectDatabases delegate many operations to ObjectReaders, and reachability bitmaps are attached to ObjectReaders. The Bitmapped and Pedestrian reachability checker objects were package private in the org.eclipse.jgit.revwalk package. This change makes them public and moves them to the org.eclipse.jgit.internal.revwalk package. Corresponding tests are also moved. Motivation: 1) Reachability checking algorithms need to scale. One of the internal Android repositories has ~2.4 million refs/changes/* references, causing bad long tail performance in reachability checks. 2) Reachability check performance is impacted by repository topography: number of refs, number of objects, amounts of related vs. unrelated history. 3) Reachability check performance is also affected by per-branch access (Gerrit branch permissions) since different users can see different branches. 4) Reachability check performance isn't affected by any state in a RevWalk or ObjectWalk. I don't yet know if a single algorithm will work for all cases in #2 and #3. We may need to evolve the ReachabilityChecker interfaces over time to solve the Gerrit branch permissions case, or use Gerrit-specific identity information to solve that in an efficient way. This change takes the existing public API and moves it to the ObjectReader/whole repository level, which is where we can do consistent customizations for #2 and #3. We intend to upstream the best of whatever works, but anticipate the need for multiple rounds of experimentation. Change-Id: I9185feff43551fb387957c436112d5250486833d Signed-off-by: Terry Parker <tparker@google.com>
3 years ago
Support LFS protocol and a file system based LFS storage Implement LfsProtocolServlet handling the "Git LFS v1 Batch API" protocol [1]. Add a simple file system based LFS content store and the debug-lfs-store command to simplify testing. Introduce a LargeFileRepository interface to enable additional storage implementation while reusing the same protocol implementation. At the client side we have to configure the lfs.url, specify that we use the batch API and we don't use authentication: [lfs] url = http://host:port/lfs batch = true [lfs "http://host:port/lfs"] access = none the git-lfs client appends the "objects/batch" to the lfs.url. Hard code an Authorization header in the FileLfsRepository.getAction because then git-lfs client will skip asking for credentials. It will just forward the Authorization header from the response to the download/upload request. The FileLfsServlet supports file content storage for "Large File Storage" (LFS) server as defined by the Github LFS API [2]. - upload and download of large files is probably network bound hence use an asynchronous servlet for good scalability - simple object storage in file system with 2 level fan-out - use LockFile to protect writing large objects against multiple concurrent uploads of the same object - to prevent corrupt uploads the uploaded file is rejected if its hash doesn't match id given in URL The debug-lfs-store command is used to run the LfsProtocolServlet and, optionally, the FileLfsServlet which makes it easier to setup a local test server. [1] https://github.com/github/git-lfs/blob/master/docs/api/http-v1-batch.md [2] https://github.com/github/git-lfs/tree/master/docs/api Bug: 472961 Change-Id: I7378da5575159d2195138d799704880c5c82d5f3 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Signed-off-by: Sasa Zivkov <sasa.zivkov@sap.com>
8 years ago
Added read/write support for pack bitmap index. A pack bitmap index is an additional index of compressed bitmaps of the object graph. Furthermore, a logical API of the index functionality is included, as it is expected to be used by the PackWriter. Compressed bitmaps are created using the javaewah library, which is a word-aligned compressed variant of the Java bitset class based on run-length encoding. The library only works with positive integer values. Thus, the maximum number of ObjectIds in a pack file that this index can currently support is limited to Integer.MAX_VALUE. Every ObjectId is given an integer mapping. The integer is the position of the ObjectId in the complete ObjectId list, sorted by offset, for the pack file. That integer is what the bitmaps use to reference the ObjectId. Currently, the new index format can only be used with pack files that contain a complete closure of the object graph e.g. the result of a garbage collection. The index file includes four bitmaps for the Git object types i.e. commits, trees, blobs, and tags. In addition, a collection of bitmaps keyed by an ObjectId is also included. The bitmap for each entry in the collection represents the full closure of ObjectIds reachable from the keyed ObjectId (including the keyed ObjectId itself). The bitmaps are further compressed by XORing the current bitmaps against prior bitmaps in the index, and selecting the smallest representation. The XOR'd bitmap and offset from the current entry to the position of the bitmap to XOR against is the actual representation of the entry in the index file. Each entry contains one byte, which is currently used to note whether the bitmap should be blindly reused. Change-Id: Id328724bf6b4c8366a088233098c18643edcf40f
11 years ago
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226
  1. Manifest-Version: 1.0
  2. Bundle-ManifestVersion: 2
  3. Bundle-Name: %Bundle-Name
  4. Automatic-Module-Name: org.eclipse.jgit
  5. Bundle-SymbolicName: org.eclipse.jgit
  6. Bundle-Version: 5.13.0.qualifier
  7. Bundle-Localization: plugin
  8. Bundle-Vendor: %Bundle-Vendor
  9. Eclipse-ExtensibleAPI: true
  10. Export-Package: org.eclipse.jgit.annotations;version="5.13.0",
  11. org.eclipse.jgit.api;version="5.13.0";
  12. uses:="org.eclipse.jgit.transport,
  13. org.eclipse.jgit.notes,
  14. org.eclipse.jgit.dircache,
  15. org.eclipse.jgit.lib,
  16. org.eclipse.jgit.revwalk,
  17. org.eclipse.jgit.treewalk.filter,
  18. org.eclipse.jgit.diff,
  19. org.eclipse.jgit.treewalk,
  20. org.eclipse.jgit.util,
  21. org.eclipse.jgit.submodule,
  22. org.eclipse.jgit.api.errors,
  23. org.eclipse.jgit.revwalk.filter,
  24. org.eclipse.jgit.blame,
  25. org.eclipse.jgit.merge",
  26. org.eclipse.jgit.api.errors;version="5.13.0";
  27. uses:="org.eclipse.jgit.lib,
  28. org.eclipse.jgit.errors",
  29. org.eclipse.jgit.attributes;version="5.13.0";
  30. uses:="org.eclipse.jgit.lib,
  31. org.eclipse.jgit.treewalk",
  32. org.eclipse.jgit.blame;version="5.13.0";
  33. uses:="org.eclipse.jgit.lib,
  34. org.eclipse.jgit.revwalk,
  35. org.eclipse.jgit.treewalk.filter,
  36. org.eclipse.jgit.diff",
  37. org.eclipse.jgit.diff;version="5.13.0";
  38. uses:="org.eclipse.jgit.lib,
  39. org.eclipse.jgit.attributes,
  40. org.eclipse.jgit.revwalk,
  41. org.eclipse.jgit.patch,
  42. org.eclipse.jgit.treewalk.filter,
  43. org.eclipse.jgit.treewalk,
  44. org.eclipse.jgit.util",
  45. org.eclipse.jgit.dircache;version="5.13.0";
  46. uses:="org.eclipse.jgit.events,
  47. org.eclipse.jgit.lib,
  48. org.eclipse.jgit.attributes,
  49. org.eclipse.jgit.treewalk,
  50. org.eclipse.jgit.util",
  51. org.eclipse.jgit.errors;version="5.13.0";
  52. uses:="org.eclipse.jgit.transport,
  53. org.eclipse.jgit.dircache,
  54. org.eclipse.jgit.lib,
  55. org.eclipse.jgit.internal.storage.pack",
  56. org.eclipse.jgit.events;version="5.13.0";
  57. uses:="org.eclipse.jgit.lib",
  58. org.eclipse.jgit.fnmatch;version="5.13.0",
  59. org.eclipse.jgit.gitrepo;version="5.13.0";
  60. uses:="org.xml.sax.helpers,
  61. org.eclipse.jgit.api,
  62. org.eclipse.jgit.lib,
  63. org.eclipse.jgit.revwalk,
  64. org.xml.sax",
  65. org.eclipse.jgit.gitrepo.internal;version="5.13.0";x-internal:=true,
  66. org.eclipse.jgit.hooks;version="5.13.0";uses:="org.eclipse.jgit.lib",
  67. org.eclipse.jgit.ignore;version="5.13.0",
  68. org.eclipse.jgit.ignore.internal;version="5.13.0";
  69. x-friends:="org.eclipse.jgit.test",
  70. org.eclipse.jgit.internal;version="5.13.0";
  71. x-friends:="org.eclipse.jgit.test,
  72. org.eclipse.jgit.http.test",
  73. org.eclipse.jgit.internal.fsck;version="5.13.0";
  74. x-friends:="org.eclipse.jgit.test",
  75. org.eclipse.jgit.internal.revwalk;version="5.13.0";
  76. x-friends:="org.eclipse.jgit.test",
  77. org.eclipse.jgit.internal.storage.dfs;version="5.13.0";
  78. x-friends:="org.eclipse.jgit.test,
  79. org.eclipse.jgit.http.server,
  80. org.eclipse.jgit.http.test,
  81. org.eclipse.jgit.lfs.test",
  82. org.eclipse.jgit.internal.storage.file;version="5.13.0";
  83. x-friends:="org.eclipse.jgit.test,
  84. org.eclipse.jgit.junit,
  85. org.eclipse.jgit.junit.http,
  86. org.eclipse.jgit.http.server,
  87. org.eclipse.jgit.lfs,
  88. org.eclipse.jgit.pgm,
  89. org.eclipse.jgit.pgm.test,
  90. org.eclipse.jgit.ssh.apache",
  91. org.eclipse.jgit.internal.storage.io;version="5.13.0";
  92. x-friends:="org.eclipse.jgit.junit,
  93. org.eclipse.jgit.test,
  94. org.eclipse.jgit.pgm",
  95. org.eclipse.jgit.internal.storage.pack;version="5.13.0";
  96. x-friends:="org.eclipse.jgit.junit,
  97. org.eclipse.jgit.test,
  98. org.eclipse.jgit.pgm",
  99. org.eclipse.jgit.internal.storage.reftable;version="5.13.0";
  100. x-friends:="org.eclipse.jgit.http.test,
  101. org.eclipse.jgit.junit,
  102. org.eclipse.jgit.test,
  103. org.eclipse.jgit.pgm",
  104. org.eclipse.jgit.internal.submodule;version="5.13.0";x-internal:=true,
  105. org.eclipse.jgit.internal.transport.connectivity;version="5.13.0";
  106. x-friends:="org.eclipse.jgit.test",
  107. org.eclipse.jgit.internal.transport.http;version="5.13.0";
  108. x-friends:="org.eclipse.jgit.test",
  109. org.eclipse.jgit.internal.transport.parser;version="5.13.0";
  110. x-friends:="org.eclipse.jgit.http.server,
  111. org.eclipse.jgit.test",
  112. org.eclipse.jgit.internal.transport.ssh;version="5.13.0";
  113. x-friends:="org.eclipse.jgit.ssh.apache,
  114. org.eclipse.jgit.ssh.jsch",
  115. org.eclipse.jgit.lib;version="5.13.0";
  116. uses:="org.eclipse.jgit.transport,
  117. org.eclipse.jgit.util.sha1,
  118. org.eclipse.jgit.dircache,
  119. org.eclipse.jgit.revwalk,
  120. org.eclipse.jgit.internal.storage.file,
  121. org.eclipse.jgit.attributes,
  122. org.eclipse.jgit.events,
  123. com.googlecode.javaewah,
  124. org.eclipse.jgit.treewalk.filter,
  125. org.eclipse.jgit.treewalk,
  126. org.eclipse.jgit.util,
  127. org.eclipse.jgit.submodule,
  128. org.eclipse.jgit.util.time",
  129. org.eclipse.jgit.lib.internal;version="5.13.0";
  130. x-friends:="org.eclipse.jgit.test",
  131. org.eclipse.jgit.logging;version="5.13.0",
  132. org.eclipse.jgit.merge;version="5.13.0";
  133. uses:="org.eclipse.jgit.dircache,
  134. org.eclipse.jgit.lib,
  135. org.eclipse.jgit.revwalk,
  136. org.eclipse.jgit.diff,
  137. org.eclipse.jgit.treewalk,
  138. org.eclipse.jgit.util,
  139. org.eclipse.jgit.api,
  140. org.eclipse.jgit.attributes",
  141. org.eclipse.jgit.nls;version="5.13.0",
  142. org.eclipse.jgit.notes;version="5.13.0";
  143. uses:="org.eclipse.jgit.lib,
  144. org.eclipse.jgit.revwalk,
  145. org.eclipse.jgit.treewalk,
  146. org.eclipse.jgit.merge",
  147. org.eclipse.jgit.patch;version="5.13.0";
  148. uses:="org.eclipse.jgit.lib,
  149. org.eclipse.jgit.diff",
  150. org.eclipse.jgit.revplot;version="5.13.0";
  151. uses:="org.eclipse.jgit.lib,
  152. org.eclipse.jgit.revwalk",
  153. org.eclipse.jgit.revwalk;version="5.13.0";
  154. uses:="org.eclipse.jgit.lib,
  155. org.eclipse.jgit.diff,
  156. org.eclipse.jgit.treewalk.filter,
  157. org.eclipse.jgit.revwalk.filter,
  158. org.eclipse.jgit.treewalk",
  159. org.eclipse.jgit.revwalk.filter;version="5.13.0";
  160. uses:="org.eclipse.jgit.revwalk,
  161. org.eclipse.jgit.lib,
  162. org.eclipse.jgit.util",
  163. org.eclipse.jgit.storage.file;version="5.13.0";
  164. uses:="org.eclipse.jgit.lib,
  165. org.eclipse.jgit.util",
  166. org.eclipse.jgit.storage.pack;version="5.13.0";
  167. uses:="org.eclipse.jgit.lib",
  168. org.eclipse.jgit.submodule;version="5.13.0";
  169. uses:="org.eclipse.jgit.lib,
  170. org.eclipse.jgit.diff,
  171. org.eclipse.jgit.treewalk.filter,
  172. org.eclipse.jgit.treewalk,
  173. org.eclipse.jgit.util",
  174. org.eclipse.jgit.transport;version="5.13.0";
  175. uses:="javax.crypto,
  176. org.eclipse.jgit.util.io,
  177. org.eclipse.jgit.lib,
  178. org.eclipse.jgit.revwalk,
  179. org.eclipse.jgit.transport.http,
  180. org.eclipse.jgit.internal.storage.file,
  181. org.eclipse.jgit.treewalk,
  182. org.eclipse.jgit.util,
  183. org.eclipse.jgit.internal.storage.pack,
  184. org.eclipse.jgit.transport.resolver,
  185. org.eclipse.jgit.storage.pack,
  186. org.eclipse.jgit.errors",
  187. org.eclipse.jgit.transport.http;version="5.13.0";
  188. uses:="javax.net.ssl",
  189. org.eclipse.jgit.transport.resolver;version="5.13.0";
  190. uses:="org.eclipse.jgit.transport,
  191. org.eclipse.jgit.lib",
  192. org.eclipse.jgit.treewalk;version="5.13.0";
  193. uses:="org.eclipse.jgit.dircache,
  194. org.eclipse.jgit.lib,
  195. org.eclipse.jgit.attributes,
  196. org.eclipse.jgit.revwalk,
  197. org.eclipse.jgit.treewalk.filter,
  198. org.eclipse.jgit.util",
  199. org.eclipse.jgit.treewalk.filter;version="5.13.0";
  200. uses:="org.eclipse.jgit.treewalk",
  201. org.eclipse.jgit.util;version="5.13.0";
  202. uses:="org.eclipse.jgit.transport,
  203. org.eclipse.jgit.hooks,
  204. org.eclipse.jgit.revwalk,
  205. org.eclipse.jgit.storage.file,
  206. org.ietf.jgss,
  207. org.eclipse.jgit.attributes,
  208. javax.management,
  209. org.eclipse.jgit.lib,
  210. org.eclipse.jgit.transport.http,
  211. org.eclipse.jgit.treewalk,
  212. javax.net.ssl,
  213. org.eclipse.jgit.util.time",
  214. org.eclipse.jgit.util.io;version="5.13.0";
  215. uses:="org.eclipse.jgit.attributes,
  216. org.eclipse.jgit.lib,
  217. org.eclipse.jgit.treewalk",
  218. org.eclipse.jgit.util.sha1;version="5.13.0",
  219. org.eclipse.jgit.util.time;version="5.13.0"
  220. Bundle-RequiredExecutionEnvironment: JavaSE-1.8
  221. Import-Package: com.googlecode.javaewah;version="[1.1.6,2.0.0)",
  222. javax.crypto,
  223. javax.net.ssl,
  224. org.slf4j;version="[1.7.0,2.0.0)",
  225. org.xml.sax,
  226. org.xml.sax.helpers