aboutsummaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* Builder API to configure SshdSessionFactoriesThomas Wolf2020-05-237-52/+746
| | | | | | | | | | | | | A builder API provides a more convenient way to define a customized SshdSessionFactory by hiding the subclassing. Also provide a new interface SshConfigStore to abstract away the specifics of reading a ssh config file, and provide a way to customize the concrete ssh config implementation to be used. This facilitates using an alternate ssh config implementation that may or may not be based on files. Change-Id: Ib9038e8ff2a4eb3a9ce7b3554d1450befec8e1e1 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* TransportHttp: abort on time-out or on SocketExceptionThomas Wolf2020-05-232-0/+90
| | | | | | | | | | | | | | | | | | Avoid trying other authentication methods on SocketException or on InterruptedIOException. SocketException is rather fatal, such as nothing listening on the peer's port, connection reset, or it could be a connection time-out. Time-outs enforced by Timeout{Input,Output}Stream may result in InterruptedIOException being thrown. In both cases, it makes no sense to try other authentication methods, and doing so may wrongly report "authentication not supported" or "cannot open git-upload-pack" or some such instead of reporting a time-out. Bug: 563138 Change-Id: I0191b1e784c2471035e550205abd06ec9934fd00 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* Merge "Bazel: Remove superfluous dependencies flagged by unused_deps"David Pursehouse2020-05-234-6/+1
|\
| * Bazel: Remove superfluous dependencies flagged by unused_depsDavid Ostrovsky2020-05-224-6/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Bazel buildtools project includes in addition to buildifier also unused deps and buildozer utilities, that detect unused dependencies and fix them by applying the removal to the build files. This change is created by installing unused_deps from buildtools@HEAD and running: $ unused_deps //... and applying the suggested modifications. Change-Id: Iad74ec2fa719475b29391586f40b13ae30477004 Signed-off-by: David Ostrovsky <david@ostrovsky.org>
* | Ignore core.eol if core.autocrlf=inputThomas Wolf2020-05-222-0/+34
| | | | | | | | | | | | | | | | | | | | Config core.eol is to be ignored if core.autocrlf is true or input.[1] JGit didn't do so when core.autocrlf=input was set. [1] https://git-scm.com/docs/git-config#Documentation/git-config.txt-coreeol Bug: 561877 Change-Id: I5e62e0510d160b5113c1090319af09c2bc1bcb59 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* | Attributes: fix handling of text=auto in combination with eolThomas Wolf2020-05-223-7/+117
|/ | | | | | | | | | | | | | | In Git 2.10.0 the interpretation of gitattributes changed or was fixed such that "* text=auto eol=crlf" would indeed still do auto-detection of text vs. binary content.[1] Previously this was identical to "* text eol=crlf", i.e., treating all files as text. JGit still did the latter, which caused surprises because it changed binary files. [1] https://github.com/git/git/blob/master/Documentation/RelNotes/2.10.0.txt#L248 Bug: 561341 Change-Id: I5b6fb97b5e86fd950a98537b6b8574f768ae30e5 Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* Log stack trace if CachingKeyPairProvider hits unexpected exceptionMatthias Sohn2020-05-201-1/+1
| | | | | | | Log the stack trace in order to help understanding the bug 563380 Bug: 563380 Change-Id: If993a63ccec5042b10e1d5e945b18f4b5f06d8ff Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* Update Orbit to S20200519202422 and ant to 1.10.8Matthias Sohn2020-05-2022-55/+55
| | | | Change-Id: I5bc120a495deb2f3b29bd04bb1c9a2058394ba8a Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* Include full IssuerFingerprint in GPG signatureThomas Wolf2020-05-182-11/+17
| | | | | | | | | | Update dependency to Bouncy Castle to 1.65. Add the IssuerFingerprint as a hashed sub-packet in the signature. If added unhashed, GPG ignores it. Bug: 553206 Change-Id: I6807e8e2385e6ec5790f388e4753a44aa9474ebb Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* Bazel: Fix src_sha1 of bcpg-jdk15onDavid Pursehouse2020-05-181-1/+1
| | | | | | | | | Test plan: bazel build //... Change-Id: Ibf1d0b56785d62150bbae49b553c856efbe6d197 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* Suppress API error for new method BitmapIndex.Bitmap#retrieveCompressedMatthias Sohn2020-05-171-0/+8
| | | | | | | OSGi semantic versioning allows breaking implementers in a minor release. Change-Id: Ib55dc43dd3b50b0ef39a7094190f230210aee4b6 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* Fix wrong @since tags added in dcb0265Matthias Sohn2020-05-171-2/+2
| | | | | | This change was introduced in 5.8. Change-Id: Ic74ebff5a0547bb55e0401b38f73ebc6e67cace9 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* Merge changes I39783eee,I874503ec,Ic942a8e4,I6ec2c3e8,I62cb5030, ...Terry Parker2020-05-1311-116/+408
|\ | | | | | | | | | | | | | | | | | | | | | | * changes: PackBitmapIndex: Set distance threshold PackBitmapIndex: Not buffer inflated bitmap in BasePackBitmapIndex PackBitmapIndex: Remove convertedBitmaps in the Remapper PackBitmapIndex: Reduce memory usage in GC PackBitmapIndex: Add AddToBitmapWithCacheFilter class PackBitmapIndex: Add util methods and builder to BitmapCommit PackBitmapIndex: Move BitmapCommit to a top-level class Refactor: Make retriveCompressed an method of the Bitmap class
| * PackBitmapIndex: Set distance thresholdYunjie Li2020-05-121-1/+1
| | | | | | | | | | | | | | | | | | | | Setting the distance threshold to 2000 in PackWriterBitmapPreparer to reduce memory usage in garbage collection. When the threshold is 0, GC for the msm repository would use about 37 GB memory to complete. After setting it to 2000, GC can finish in 75 min with about 10 GB memory. Change-Id: I39783eeecbae58261c883735499e61ee1cac75fe Signed-off-by: Yunjie Li <yunjieli@google.com>
| * PackBitmapIndex: Not buffer inflated bitmap in BasePackBitmapIndexYunjie Li2020-05-121-1/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently we're buffering the inflated bitmap entry in BasePackBitmapIndex to optimize running time. However, this will use lots of memory during the construction of the pack bitmap index file which may cause failure of garbage collection. The running time didn't increase significantly, if there's any increase, after removing the buffering here. The report about usage of time/memory will come in the next commit. Change-Id: I874503ecc85714acab7ca62a6a7968c2dc0b56b3 Signed-off-by: Yunjie Li <yunjieli@google.com>
| * PackBitmapIndex: Remove convertedBitmaps in the RemapperYunjie Li2020-05-121-10/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The convertedBitmaps serves for time-optimization purpose. But it's actually not saving time much but using lots of memory. So remove the field here to save memory. Currently the remapper class is only used in the construction of the bitmap index file. And during the preparation of the file, we're only getting bitmaps from the remapper when finding objects accessible from a commit, so bitmap associated with each commit will only be fetched once and thus the convertedBitmaps would hardly be read, which means that it's not saving time. Change-Id: Ic942a8e485135fb177ec21d09282d08ca6646fdb Signed-off-by: Yunjie Li <yunjieli@google.com>
| * PackBitmapIndex: Reduce memory usage in GCYunjie Li2020-05-124-69/+136
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, the garbage collection is consistently failing for some large repositories in the building bitmap phase, e.g.Linux-MSM project: https://source.codeaurora.org/quic/la/kernel/msm-3.18 Historically, bitmap index creation happened in 3 phases: 1. Select the commits to which bitmaps should be attached. 2. Create all bitmaps for these commits, stored in uncompressed format in the PackBitmapIndexBuilder. 3. Deltify the bitmaps and write them to disk. We investigated the process. For phase 2 it's most efficient to create bitmaps starting with oldest commit and moving to the newest commit, because the newer commits are able to reuse the work for the old ones. But for bitmap deltification in phase 3, it's better when a newer commit's bitmap is the base, and the current disk format writes bitmaps out for the newest commits first. This change introduces a new collection to hold the deltified and compressed representations of the bitmaps, keeping a smaller subset of commits in the PackBitmapIndexBuilder to help make the bitmap index creation more memory efficient. And in this commit, we're setting DISTANCE_THRESHOLD to 0 in the PackWriterBitmapPreparer, which means the garbage collection will not have much behavoir change and will still use as much memory as before. Change-Id: I6ec2c3e8dde11805af47874d67d33cf1ef83660e Signed-off-by: Yunjie Li <yunjieli@google.com>
| * PackBitmapIndex: Add AddToBitmapWithCacheFilter classYunjie Li2020-05-121-0/+91
| | | | | | | | | | | | | | | | | | | | | | | | Add a new revwalk filter, AddToBitmapWithCachedFilter. This filter updates a client-provided {@code BitmapBuilder} as a side effect of a revwalk. Similar to {@code AddToBitmapFilter}, it short circuits the walk when it encounters a commit which is included in the provided bitmap's BitmapIndex. It also short circuits the walk if it encounters the client-provided cached commit. Change-Id: I62cb503016f4d3995d648d92b82baab7f93549a9 Signed-off-by: Yunjie Li <yunjieli@google.com>
| * PackBitmapIndex: Add util methods and builder to BitmapCommitYunjie Li2020-05-121-0/+128
| | | | | | | | | | | | | | | | | | Add some utility methods and a builder class for BitmapCommit class in preparation for improving the memory footprint of GC's bitmap generation phase. Change-Id: Ice3d257fc26f3917a65a64eaf53b508b89043caa Signed-off-by: Yunjie Li <yunjieli@google.com>
| * PackBitmapIndex: Move BitmapCommit to a top-level classYunjie Li2020-05-124-25/+37
| | | | | | | | | | | | | | | | | | Move BitmapCommit from inside the PackWriterBitmapPreparer to a new top-level class in preparation for improving the memory footprint of GC's bitmap generation phase. Change-Id: I4d404a5b3a34998b441d23105197f33d32d39670 Signed-off-by: Yunjie Li <yunjieli@google.com>
| * Refactor: Make retriveCompressed an method of the Bitmap classYunjie Li2020-05-123-13/+18
| | | | | | | | | | | | | | | | | | Make retrieveCompressed() a method of Bitmap interface to avoid type casting and later reuse in improving the memory footprint of GC's bitmap generation phase. Change-Id: I098d85105cf17af845d43b8c71b4ca48b02fd7da Signed-off-by: Yunjie Li <yunjieli@google.com>
* | Fix downloading LFS Object fails behind proxyMatthias Sohn2020-05-123-1/+14
| | | | | | | | | | | | | | | | When downloading LFS objects also accept response code 203 as successful download. This response may be seen when downloading via a proxy. Bug: 563022 Change-Id: Iee85fdb451b33369d08859872e5bfc2a67dffa6d Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* | Merge changes I6543c2e1,I21ed029dTerry Parker2020-05-118-4/+416
|\ \ | | | | | | | | | | | | | | | * changes: ReceivePack: adding IterativeConnectivityChecker Moving transport/internal -> internal/transport
| * | ReceivePack: adding IterativeConnectivityCheckerDemetr Starshov2020-05-083-0/+411
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Introduce an IterativeConnectivityChecker which runs a connectivity check with a filtered set of references, and falls back to using the full set of advertised references. It uses references during first check attempt: - References that are ancestors of an incoming commits (e.g., pushing a commit onto an existing branch or pushing a new branch based on another branch) - Additional list of references we know client can be interested in (e.g. list of open changes for Gerrit) We tested it inside Google and it improves connectivity for certain topologies. For example connectivity counts for chromium.googlesource.com/chromium/src: percentile_50: 1923 (was: 22777) percentile_90: 23272 (was: 353003) percentile_99: 345522 (was: 353435) This saved ~2 seconds on every push to this repository. Signed-off-by: Demetr Starshov <dstarshov@google.com> Change-Id: I6543c2e10ed04622ca795b195665133e690d3b10
| * | Moving transport/internal -> internal/transportDemetr Starshov2020-05-085-4/+5
| | | | | | | | | | | | | | | | | | | | | | | | Moving transport related internal classes into dedicated subpackage in o/e/j/internal package. Signed-off-by: Demetr Starshov <dstarshov@google.com> Change-Id: I21ed029d359f5f7d8298f102efbb4b1dcdf404ad
* | | Allow for using custom s3 host with lfs serverPat Long2020-05-104-4/+53
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | By default, it will generate hostname using the aws region passed to the constructor. This will allow for easier testing, since you can just spin up a local minio (or other s3-compatible storage service) instance and point the application at that for the storage mechanism. It will also allow for storing lfs objects on-prem. Change-Id: I2566b1fcce58f3d306ddd23a8da702ef5a451c7b Signed-off-by: Pat Long <pllong@arista.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* | | Fix error occurring during checkoutNail Samatov2020-05-072-1/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fix NullPointerException occurring when calling CheckoutCommand with forced == true option when the branch isn't changed and there is deleted uncommitted file. Change-Id: I99bf1fc25e6889f07092320d7bc2772ec5d341b5 Signed-off-by: Nail Samatov <sanail@yandex.ru>
* | | Merge branch 'stable-5.8'Matthias Sohn2020-05-070-0/+0
|\ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.8: Prepare 5.8.0-SNAPSHOT builds JGit v5.8.0.202005061305-m2 Change-Id: I7e95ddbedfae23035cc132abdf574545ffcda091
| * | | Prepare 5.8.0-SNAPSHOT buildsMatthias Sohn2020-05-0774-105/+105
| | | | | | | | | | | | | | | | | | | | Change-Id: Ic7d763b3b02b4d2a7a4141d2d9d715855dd69918 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
| * | | JGit v5.8.0.202005061305-m2v5.8.0.202005061305-m2Matthias Sohn2020-05-0674-105/+105
|/ / / | | | | | | | | | | | | Change-Id: I9d9e5a9e000a74513896744cc3646430cb872893 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* | | Update to bouncycastle 1.65 and orbit I20200506000552Matthias Sohn2020-05-0623-106/+106
| | | | | | | | | | | | | | | Bug: 553206 Change-Id: I867bbe88b847bf6d17f17f6cf8c565cd4a2437cc Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* | | ApplyCommand: use Files#copy to copy fileMatthias Sohn2020-05-061-9/+4
| | | | | | | | | | | | | | | | | | This should be faster. Change-Id: I404ec5e66731b3cf7a8e621cf1ff8748d109ea69 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* | | Apply hunks when renaming or copying from patch filesJack Wickham2020-05-068-3/+66
| | | | | | | | | | | | | | | | | | | | | | | | When applying a patch that contains renames or copies using ApplyCommand, also apply all hunks that apply to the renamed or copied file. Change-Id: I9f3fa4370458bd7c14beeb2e2b49e846d70203cb Signed-off-by: Jack Wickham <jwickham@palantir.com>
* | | Create parent directories when renaming a file in ApplyCommandJack Wickham2020-05-065-0/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Before this change, applying a patch will fail if the destination directory doesn't exist; after, the necessary parent directories are created. If renaming the file fails, the directories won't be deleted, so this change isn't atomic. However, ApplyCommand is already not atomic - if one hunk fails to apply, other hunks still get applied - so I don't think that is a blocker. Change-Id: Iea36138b806d4e7012176615bcc673756a82f365 Signed-off-by: Jack Wickham <jwickham@palantir.com>
* | | Reduce BitmappedObjectReachabilityChecker visibilityIvan Frade2020-05-041-3/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | ObjectReachabilityChecker interface is the only public API. The implementation is instantiated by ObjectWalk and doesn't need to be visible outside the package. Change-Id: I5b97bb98990cded637686bdc15c9655330b7780f Signed-off-by: Ivan Frade <ifrade@google.com>
* | | Add missing @since to new APIThomas Wolf2020-05-041-0/+2
| | | | | | | | | | | | Change-Id: Ie7c3481aba515d9c9b0a152db17b5a9dc74e7ede Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* | | Add missing test source file to the buildDavid Ostrovsky2020-05-031-0/+1
| | | | | | | | | | | | | | | Change-Id: I020ff281eaa49794aeb1853711485bc8c7047dd8 Signed-off-by: David Ostrovsky <david@ostrovsky.org>
* | | Upgrade Tycho to 1.7.0Michael Keppler2020-05-012-2/+2
| | | | | | | | | | | | | | | Change-Id: I41d927e53f040750bbf0160dc15d8ca8a3f9f7ca Signed-off-by: Michael Keppler <Michael.Keppler@gmx.de>
* | | ObjectReachabilityCheckers: Make walk member finalIvan Frade2020-04-302-2/+2
|/ / | | | | | | | | | | | | It is only assigned on initialization. Change-Id: I36f68b3f511236e66cc7a4f56ba49252b317276a Signed-off-by: Ivan Frade <ifrade@google.com>
* | Upgrade wagon-ssh to 3.4.0David Pursehouse2020-04-291-1/+1
| | | | | | | | | | Change-Id: Id9a1bf7fcdba7c011ccff54188dfb1bef8bf8a4b Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* | Merge branch 'stable-5.7'Matthias Sohn2020-04-291-1/+1
|\ \ | | | | | | | | | | | | | | | | | | | | | * stable-5.7: Bump Bazel version to 3.1.0 Change-Id: I108815577a5aab605891496c17d6f225d35f7346 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
| * \ Merge branch 'stable-5.6' into stable-5.7Matthias Sohn2020-04-291-1/+1
| |\ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-5.6: Bump Bazel version to 3.1.0 Change-Id: I8a9f39370ab506d9ff5763ed2d05f0d970bbd10b Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
| | * | Bump Bazel version to 3.1.0David Pursehouse2020-04-291-1/+1
| | | | | | | | | | | | | | | | | | | | Change-Id: Id58d1603846a88be3493165583adca481c026eed Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* | | | UploadPack: Use more relevant refs first in object reachability checkIvan Frade2020-04-281-4/+29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The bitmap-bassed object reachability checker, tries to find the objects in the first starter, then adding the second starter... and so on. This rewards passing the most popular refs first. Order the refs with heads first, then tags, then others (e.g. changes) for the object reachability checker. Using streams, delay also the resolution of the ref to RevObject until necessary. Change-Id: I9414b76754d7c0ffee1e2eeed6939895c8e92cbe Signed-off-by: Ivan Frade <ifrade@google.com>
* | | | UploadPack: Refactor to generalize the object reachability checksIvan Frade2020-04-284-33/+34
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ObjectWalk#createObjectReachabilityChecker() returns the best implementation for the repo. UploadPack can use the interface and fold the with/without commits cases in one code path. Change-Id: I857c11735d1d8e36c3ed8185ff11de8a62e86540 Signed-off-by: Ivan Frade <ifrade@google.com>
* | | | UploadPack: Use BitmappedReachabilityChecker for not advertised wantsIvan Frade2020-04-281-18/+12
| | | | | | | | | | | | | | | | | | | | Change-Id: Ifea971d5c0309e28a909441ee8a6f1e62397d6d3 Signed-off-by: Ivan Frade <ifrade@google.com>
* | | | revwalk: Introduce bitmap-based object reachability checkerIvan Frade2020-04-282-0/+110
| | | | | | | | | | | | | | | | | | | | Change-Id: I0b1a2bd21f98894862aab339f8c2e4a417897b89 Signed-off-by: Ivan Frade <ifrade@google.com>
* | | | revwalk: Extract ObjectReachabilityChecker interfaceIvan Frade2020-04-285-193/+231
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Extract ObjectReachabilityChecker interface from the walk-based implementation, to add a bitmapped based implementation later. Refactor the test case to use it for both implementations. Change-Id: Iaac7c6b037723811956ac22625f27d3b4d742139 Signed-off-by: Ivan Frade <ifrade@google.com>
* | | | UploadPack: Extract walk-based reachability checkIvan Frade2020-04-283-33/+272
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Preparing the code to optimize the bitmap-based object reachability checker. We are mirroring first the commit reachability checker structure (interface + 2 implementations). Move the walk-base reachability checker to its own class. This class is public at the moment. Later ObjectWalk will return an interface and this implementation will be package-private. Change-Id: Ifac70094e1af137291c3607d95e689992f814b26 Signed-off-by: Ivan Frade <ifrade@google.com>
* | | | Enable passing java options to jgit command line executableMatthias Sohn2020-04-281-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This allows to pass java options to the jgit command line executable by setting the environment variable "java_args", e.g. $ java_args="-Xmx8g" jgit log Change-Id: Ic7271b104737a8306d9db0f1895079b9f7bb7fd0 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
generated by cgit v1.2.3 (git 2.39.1) at 2025-01-11 03:12:29 +0000