aboutsummaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
...
* | | Merge changes I0d339b9f,I0e6673b8Chris Aniszczyk2010-05-032-10/+9
|\| | | |/ |/| | | | | | | * changes: Favor earlier PackFile instances over later duplicates Cleanup duplicated object reuse code in PackWriter
| * Favor earlier PackFile instances over later duplicatesShawn O. Pearce2010-04-261-3/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There is a potential race condition during insertPack that can lead to us having the same pack file open twice in the same directory. A different thread can miss an object on disk, and trigger a scan of the directory, and notice the pack that was put in by IndexPack. So the pack winds up in the newly created PackList. The IndexPack thread then wakes up and finishes its insertPack by creating a new PackFile and inserting it into position 0 of the list. We now have the same pack listed twice. Readers will favor the earlier PackFile instance, because its the first one they come across as they iterate through the list. Keep that earlier one when we scan the pack directory again, as this will avoid needing to purge out all of the windows that may have been cached. Of course we should also fix that race condition, but this block was taking the wrong resolution if this error ever shows up, so lets first fix the block to use a more sane resolution. Change-Id: I0d339b9fd1dd8012e8fe5a564b893c0f69109e28 Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
| * Cleanup duplicated object reuse code in PackWriterShawn O. Pearce2010-04-261-7/+5
| | | | | | | | | | | | | | | | | | | | This reuse line was identical between the two branches related to reusing a delta, or reusing a whole object. Either way they reuse the body of the object as-is. So just make that a common function after the header is written. Change-Id: I0e6673b8e813c8c08c594ea2ba546fd366339d5d Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
* | Fix ReceivePackRefFilterTest on WindowsShawn O. Pearce2010-04-271-0/+9
| | | | | | | | | | | | | | | | | | | | | | | | The pack files were left open after the test ended, which meant we could not delete them automatically when the test was over. Make sure we close the repositories (and thus their underlying packs) before the tear down finishes. Bug: 310367 Change-Id: I4d2703efa4b2e0c347ea4f4475777899cf71073e Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
* | Cleaning up provider and feature namesChris Aniszczyk2010-04-272-3/+3
|/ | | | | | | | It is incorrect to use Eclipse.org as the providerName now, we'll use Eclipse JGit. Change-Id: I1621b93d4f401176704e7c43935a5ce0c8ee8419 Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
* Merge "Fix NPE during InflaterCache return after corrupt loose object"Robin Rosenberg2010-04-241-1/+0
|\
| * Fix NPE during InflaterCache return after corrupt loose objectShawn O. Pearce2010-04-231-1/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If a corrupt loose object is read, UnpackedObjectLoader was disposing of the Inflater, and then attempting to return the disposed Inflater to the InflaterCache. Since the disposed Inflater had its native libz resource deallocated and its reference cleared out, the Inflater threw NullPointerException and refused to reset itself before being put back into the cache. Instead of disposing of the Inflater when corruption is found, do nothing, and allow it to be returned to the cache. The instance will get reset, and should be usable by a future caller. Bug: 310291 Change-Id: I44f2247c08b6e04fa62f8399609341b07508c096 Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
* | Merge branch 'receive-pack-filter'Shawn O. Pearce2010-04-194-92/+583
|\ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * receive-pack-filter: ReceivePack: Clarify the check reachable option ReceivePack: Micro-optimize object lookup when checking connectivity ReceivePack: Correct type of not provided object IndexPack: Tighten up new and base object bookkeeping ReceivePack: Remove need new,base object id properties ReceivePack: Discard IndexPack as soon as possible ReceivePack: fix ensureProvidedObjectsVisible on thin packs Change-Id: I4ef2fcb931f3219872e0519abfcee220191d5133
| * | ReceivePack: Clarify the check reachable optionShawn O. Pearce2010-04-162-28/+46
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This option was mis-named from day 1. Its not checking that the objects provided by the client are reachable, its actually doing a scan to prove that objects referenced by the client are already reachable through another reference on the server, or were sent as part of the pack from the client. Rename it checkReferencedObjectsAreReachable, since we really are trying to validate that objects referenced by the client's actions are reachable to the client. We also need to ensure we run checkConnectivity() anytime this is enabled, even if the caller didn't turn on fsck for object formats. Otherwise the check would be completely bypassed. Change-Id: Ic352ddb0ca8464d407c6da5c83573093e018af19 Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
| * | ReceivePack: Micro-optimize object lookup when checking connectivityShawn O. Pearce2010-04-161-3/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If we are checking the visibility of everything referenced in the pack that isn't already reachable by a reference, it needs to be in the provided set. Since the provided set lists everything that is in this pack, we can avoid checking to see if the blob exists on disk, because we know it should be there, it was found in the pack we just consumed. Change-Id: Ie3c7746f734d13077242100a68e048f1ac18c34a Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
| * | ReceivePack: Correct type of not provided objectShawn O. Pearce2010-04-161-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | If a tree was referenced but not provided in the pack, report it as a missing tree and not as a missing blob. Change-Id: Iab05705349cdf0d30cc3f8afc6698a8d2a941343 Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
| * | IndexPack: Tighten up new and base object bookkeepingShawn O. Pearce2010-04-162-31/+39
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The only current consumer of these collections is ReceivePack, where it needs to test ObjectId equality between a RevObject and an ObjectId. There we were copying from a traditional HashSet<ObjectId> into an ObjectIdSubclassMap<ObjectId>, as the latter can perform hashing using ObjectId's native value support, bypassing RevObject's override on hashCode() and equals(). Instead of doing that copy, directly create ObjectIdSubclassMap instances inside of ReceivePack. We also only need to record the objects that do not appear in the incoming pack, and were therefore copied from the local repositiory in order to complete delta resolution. Instead of listing everything that used an OBJ_REF_DELTA format, list only the objects that we pulled from the destination repository via a normal ObjectLoader. ReceivePack can now discard the IndexPack object, and all of its other data, as soon as these collections are held by the check connectivity method. This frees up memory for the ObjectWalk's own RevObject pool. Change-Id: I22ef71b45c2045a0202e7fd550a770ee1f6f38a6 Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
| * | ReceivePack: Remove need new,base object id propertiesShawn O. Pearce2010-04-161-49/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | These are more like internal implementation details of how IndexPack works with ReceivePack to validate the incoming object stream. Callers who are embedding the ReceivePack logic in their own application don't really need to know the details of which objects were used for delta bases in the incoming thin pack, or exactly which objects were newly transmitted. Hide these from the API, as exposing them through ReceivePack was an early mistake. Change-Id: I7ee44a314fa19e6a8520472ce05de92c324ad43e Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
| * | ReceivePack: Discard IndexPack as soon as possibleShawn O. Pearce2010-04-161-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The IndexPack object carries a good bit of state within itself about the objects received over the wire. The earlier we can discard it, the sooner the GC is able to reclaim this chunk of memory for other uses. So drop it as soon as we are certain the pack is valid and we have no connectivity concerns. Change-Id: I1e8bc87c2e9183733043622237a064e55957891f Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
| * | ReceivePack: fix ensureProvidedObjectsVisible on thin packsShawn O. Pearce2010-04-163-5/+510
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If ensureProvidedObjectsVisible is enabled we expected any trees or blobs directly reachable from an advertised reference to be marked with UNINTERESTING. Unfortunately ObjectWalk doesn't bother setting this until the traversal is complete. Even then it won't necessarily set it on every tree if the corresponding commit wasn't popped. When we are going to check the base objects for the received pack, ensure the UNINTERESTING flag gets carried into every immediately reachable tree or blob, because these are the ones that the client might try to use as delta bases in a thin pack. Change-Id: I5d5fdcf07e25ac9fc360e79a25dff491925e4101 Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
* | | Merge "ObjectIdSubclassMap: Correct Iterator to throw NoSuchElementException"Matthias Sohn2010-04-171-1/+2
|\| |
| * | ObjectIdSubclassMap: Correct Iterator to throw NoSuchElementExceptionShawn O. Pearce2010-04-161-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | The Iterator contract says next() shall throw NoSuchElementException if there are no more items remaining in the iteration. We got this wrong when I originally wrote the implementation, so fix it. Change-Id: Iea25e6569ead5c8b3128b8a368c5b2caebec7ecc Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
* | | Merge "ObjectIdSubclassMap: Add isEmpty() method"Matthias Sohn2010-04-171-0/+5
|\| |
| * | ObjectIdSubclassMap: Add isEmpty() methodShawn O. Pearce2010-04-161-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This class behaves like a cross between a Set and a Map, sometimes we might expect to use the method isEmpty() to test for size() == 0. So implement it, reducing the surprise folks get when they are given one of these objects. Change-Id: I0d68e1243da8e62edf79c6ba4fd925f643e80a88 Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
* | | Merge "IndexPack: Correct thin pack fix using less than 20 bytes"Robin Rosenberg2010-04-172-0/+67
|\| |
| * | IndexPack: Correct thin pack fix using less than 20 bytesShawn O. Pearce2010-04-162-0/+67
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If we need to append less than 20 bytes in order to fix a thin pack and make it complete, we need to set the length of our file back to the actual number of bytes used because the original SHA-1 footer was not completely overwritten. That extra data will confuse the header and footer fixup logic when it tries to read to the end of the file. This isn't a very common case to occur, which is why we've never seen it before. Getting a delta that requires a whole object which uses less than 20 bytes in pack representation is really hard. Generally a delta generator won't make these, because the delta would be bigger than simply deflating the whole object. I only managed to do this with a hand-crafted pack file where a 1 byte delta was pointed to a 1 byte whole object. Normally we try really hard to avoid truncating, because its typically not safe across network filesystems. But the odds of this occurring are very low. This truncation is done on a file we have open for writing, will append more content onto, and is a temporary file that we won't move into position for others to see until we've validated its SHA-1 is sane. I don't think the truncate on NFS issue is something we need to worry about here. Change-Id: I102b9637dfd048dc833c050890d142f43c1e75ae Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
* | | Fix unit tests using MockSystemReader with user configuationShawn O. Pearce2010-04-141-1/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Since cc905e7d4be "Make Repository.getConfig aware of changed config" its invalid to have a null result from FileBasedConfig.getFile(), as the path is used to stat the location on disk before returning the Config object from Repository.getConfig(). Mock out the isOutdated() method to return false all of the time in the mock test environment, so we don't crash with an NPE when this mock user configuration is being called. Change-Id: I0b4d9cbd346d5dc225ec12674da905c35457fa7c Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
* | | Optimize ref scanningRobin Rosenberg2010-04-131-5/+6
|/ / | | | | | | | | | | | | | | | | | | | | | | | | | | We can avoid one stat call by trying to perform a directory listing without checking if the reference File is a directory. Attempting a directory listing is defined to return. The other case for null returns from list is when an I/O error occcurs. Both cases are now intepreted as a possible plain reference. I/O errors when reading plain references will be handled (ignored) in scanRef(). Change-Id: I9906ed8c42eab4d6029c781aab87b3b07c1a1d2c Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
* | Merge "Make Repository.getConfig aware of changed config"Matthias Sohn2010-04-132-17/+53
|\ \
| * | Make Repository.getConfig aware of changed configJens Baumgart2010-04-122-17/+53
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In the current implementation Repository reads user and repository config only at creation point of time. The new implementatiopn checks in Repository.getConfig if user or repository config have changed on disk and reload the config if required. Change-Id: Ibd97515919ef66c6f8aa1a4fe8a11a6711335dad Signed-off-by: Jens Baumgart <jens.baumgart@sap.com>
* | | Merge "Speed up check for modifications of tracked resources"Shawn Pearce2010-04-101-4/+6
|\ \ \
| * | | Speed up check for modifications of tracked resourcesRobin Rosenberg2010-04-101-4/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We only need to check file existense if some other stat returns a value that may mean that the file does not exist. File.length() == 0 or File.lastModified() == 0 are two such properties. We use length here. Change-Id: If626b12e7bb4da994b5c086f6a5b7a12c187261c Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
* | | | Merge 'Update build to use Tycho 0.8'Shawn O. Pearce2010-04-101-1/+1
|\ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: org.eclipse.jgit.packaging/pom.xml Change-Id: I248a72575ff23fecf7599c06517c909f43f95ee4
| * | | | Update build to use Tycho 0.8Matthias Sohn2010-04-111-1/+1
| |/ / / | | | | | | | | | | | | | | | | Change-Id: I99bac3376d9460ab94b548bd2f83be6fbc6ecbe3 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* / / / JGit plugin not compatible with Eclipse 3.4Robin Rosenberg2010-04-057-12/+18
|/ / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The JSch bundle in Eclipse 3.4 does not export its packages with version numbers. Use Require-Bundle on version 0.1.37 that comes with Eclipse 3.4 There is no 0.1.37 in the maven repositories so the pom still refers to 0.1.41 so the build can get the compile time dependencies right. Bug: 308031 CQ: 3904 jsch Version: 0.1.37 (using Orbit CQ2014) Change-Id: I12eba86bfbe584560c213882ebba58bf1f9fa0c1 Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
* / / Make parsing of PersonIdent from raw byte array fault-tolerant.Marc Strapetz2010-03-232-1/+113
|/ / | | | | | | | | | | RawParseUtils.parsePersonIdent handles now those invalid byte sequences which would result in IndexOutOfBoundsException and returns null in this case.
* | Merge branch 'stable-0.7'Shawn O. Pearce2010-03-220-0/+0
|\ \ | | | | | | | | | | | | | | | * stable-0.7: Qualify post-0.7.1 builds JGit 0.7.1
| * | Qualify post-0.7.1 buildsstable-0.7Shawn O. Pearce2010-03-2223-26/+26
| | | | | | | | | | | | | | | Change-Id: Ifad1a5a6f2909d709fd7834b32b9b9949b2e5633 Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
| * | JGit 0.7.1v0.7.1Shawn O. Pearce2010-03-2223-127/+127
| | | | | | | | | | | | | | | Change-Id: Ica516f1e34335ca7a05b071fd527027b10bb7e73 Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
* | | Merge branch 'stable-0.7'Shawn O. Pearce2010-03-222-1/+26
|\| | | | | | | | | | | | | | * stable-0.7: Fix EGit deadlock listing branches of SSH remote
| * | Fix EGit deadlock listing branches of SSH remoteShawn O. Pearce2010-03-202-1/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When listing branches, EGit only reads the advertisement and then disconnects. When it closes down the pack channel the remote side is waiting for the client to send our list of commands, or a flush-pkt to let it know there is nothing to do. However if an error thread is open watching the SSH stderr stream, we ask for it to finish before we send the flush-pkt. Unfortunately the thread won't terminate until the main output stream closes, which is waiting for the flush-pkt. A classic network deadlock. If the output stream needs a flush-pkt we send it before we wait for the error stream to close. If the flush-pkt is rejected, we close down the output stream early, assuming that the remote side is broken and we will get error information soon. Change-Id: I8d078a339077756220c113f49d206b1bf295d434 Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
* | | Merge branch 'stable-0.7'Shawn O. Pearce2010-03-200-0/+0
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-0.7: Qualify post-0.7.0 builds JGit 0.7.0 This is an 'ours' merge to avoid bringing in the 0.7.0 version numbers in the manifest and pom files. Change-Id: Iad6354af57aaa2f233142fbf679489b08c121a71
| * | Qualify post-0.7.0 buildsShawn O. Pearce2010-03-2023-26/+26
| | | | | | | | | | | | | | | Change-Id: I5afdc624b28fab37b28dd2cc71d334198672eef3 Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
| * | JGit 0.7.0v0.7.0Shawn O. Pearce2010-03-1823-127/+127
| | | | | | | | | | | | Change-Id: I9b00a4041c19115e81326afd2213b98603f789ad
* | | Qualify builds as 0.8.0Shawn O. Pearce2010-03-2023-127/+127
| | | | | | | | | | | | | | | | | | | | | | | | | | | Since the API is changing relative to 0.7.0, we'll call our next release 0.8.1. But until that gets released, builds from master will be 0.8.0.qualifier. Change-Id: I921e984f51ce498610c09e0db21be72a533fee88 Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
* | | Merge branch 'stable-0.7'Shawn O. Pearce2010-03-203-7/+57
|\| | | |/ |/| | | | | | | | | | | * stable-0.7: tools/version.sh: Update OSGi manifest files Drop CQ 3448 from IP log Change-Id: I8d78d27c48c16a70078bf76b255f8ade8e94db2a
| * tools/version.sh: Update OSGi manifest filesShawn O. Pearce2010-03-182-2/+57
| | | | | | | | | | | | | | | | Tag the version number and API range in the OSGi manifest files whenever we bump the pom.xml files. Change-Id: I7c38b51f7139c02bef6b0e67d3f9199cbcdc8a39 Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
| * Drop CQ 3448 from IP logShawn O. Pearce2010-03-181-5/+0
| | | | | | | | | | | | | | | | | | Because this is the original contribution made under the project's official license, EMO has tagged it "epl" and dropped it from the project's IP log. Change-Id: I55a2a57c570a555f4c86903767d60ae7cfddacbe Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
* | Add a paranoid 'must be provided' option to ReceivePackNico Sallembien2010-03-181-3/+55
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | By default a receive pack assumes that its user will only provide references to objects that the user already has access to on their local client. In certain cases, an additional check to verify the references point only to reachable objects is necessary. This additional checking is useful when the code doesn't trust the client not to provide a forged SHA-1 reference to an object, in an attempt to access parts of the DAG that they weren't allowed to see by the configured RefFilter. Change-Id: I3e4b8505cb2992e3e4be253abb14a1501e47b970 Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
* | Merge branch 'stable-0.7'Shawn O. Pearce2010-03-1240-453/+1487
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-0.7: Reuse the line buffer between strings in PacketLineIn http.server: Use TemporaryBuffer and compress some responses Reduce multi-level buffered streams in transport code Fix smart HTTP client buffer alignment Use "ERR message" for early ReceivePack problems Catch and report "ERR message" during remote advertisements Wait for EOF on stderr before finishing SSH channel Capture non-progress side band #2 messages and put in result ReceivePack: Enable side-band-64k capability for status reports Use more restrictive patterns for sideband progress scraping Prefix remote progress tasks with "remote: " Decode side-band channel number as unsigned integer Refactor SideBandInputStream construction Refactor SideBandOutputStream to be buffered
| * Merge branch 'push-sideband' into stable-0.7Shawn O. Pearce2010-03-1240-453/+1487
| |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * push-sideband: Reuse the line buffer between strings in PacketLineIn http.server: Use TemporaryBuffer and compress some responses Reduce multi-level buffered streams in transport code Fix smart HTTP client buffer alignment Use "ERR message" for early ReceivePack problems Catch and report "ERR message" during remote advertisements Wait for EOF on stderr before finishing SSH channel Capture non-progress side band #2 messages and put in result ReceivePack: Enable side-band-64k capability for status reports Use more restrictive patterns for sideband progress scraping Prefix remote progress tasks with "remote: " Decode side-band channel number as unsigned integer Refactor SideBandInputStream construction Refactor SideBandOutputStream to be buffered Change-Id: Ic9689e64e8c87971f2fd402cb619082309d5587f
| | * Reuse the line buffer between strings in PacketLineInShawn O. Pearce2010-03-121-8/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When reading pkt-lines off an InputStream we are quite likely to consume a whole group of fairly short lines in rapid succession, such as in the have exchange that occurs in the fetch-pack/upload-pack protocol. Rather than allocating a throwaway buffer for each line's raw byte sequence, reuse a buffer that is equal to the small side-band packet size, which is 1000 bytes. Text based pkt-lines are required to be less than this size because many widely deployed versions of C Git use a statically allocated array of this length. Change-Id: Ia5c8e95b85020f7f80b6d269dda5059b092d274d Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
| | * http.server: Use TemporaryBuffer and compress some responsesShawn O. Pearce2010-03-127-36/+150
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The HTTP server side code now uses the same approach that the smart HTTP client code uses when preparing a request body. The payload is streamed into a TemporaryBuffer of limited size. If the entire data fits, its compressed with gzip if the user agent supports that, and a Content-Length header is used to transmit the fixed length body to the peer. If however the data overflows the limited memory segment, its streamed uncompressed to the peer. One might initially think that larger contents which overflow the buffer should also be compressed, rather than sent raw, since they were deemed "large". But usually these larger contents are actually a pack file which has been already heavily compressed by Git specific routines. Trying to deflate that with gzip is probably going to take up more space, not less, so the compression overhead isn't worthwhile. This buffer and compress optimization helps repositories with a large number of references, as their text based advertisements compress well. For example jgit's own native repository currently requires 32,628 bytes for its full advertisement of 489 references. Most repositories have fewer references, and thus could compress their entire response in one buffer. Change-Id: I790609c9f763339e0a1db9172aa570e29af96f42 Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
| | * Reduce multi-level buffered streams in transport codeShawn O. Pearce2010-03-1210-44/+71
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Some transports actually provide stream buffering on their own, without needing to be wrapped up inside of a BufferedInputStream in order to smooth out system calls to read or write. A great example of this is the JSch SSH client, or the Apache MINA SSHD server. Both use custom buffering to packetize the streams into the encrypted SSH channel, and wrapping them up inside of a BufferedInputStream or BufferedOutputStream is relatively pointless. Our SideBandOutputStream implementation also provides some fairly large buffering, equal to one complete side-band packet on the main data channel. Wrapping that inside of a BufferedOutputStream just to smooth out small writes from PackWriter causes extra data copies, and provides no advantage. We can save some memory and some CPU cycles by letting PackWriter dump directly into the SideBandOutputStream's internal buffer array. Instead we push the buffering streams down to be as close to the network socket (or operating system pipe) as possible. This allows us to smooth out the smaller reads/writes from pkt-line messages during advertisement and negotation, but avoid copying altogether when the stream switches to larger writes over a side band channel. Change-Id: I2f6f16caee64783c77d3dd1b2a41b3cc0c64c159 Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
| | * Fix smart HTTP client buffer alignmentShawn O. Pearce2010-03-121-32/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This proved to be a pretty difficult to find bug. If we read exactly the number of response bytes from the UnionInputStream and didn't try to read beyond that length, the last connection's InputStream is still inside of the UnionInputStream, and UnionInputStream.isEmpty() returns false. But there is no data present, so the next read request to our UnionInputStream returns EOF at a point where the HTTP client code should have started a new request in order to get more data. Instead of wrapping the UnionInputStream, push an dummy stream onto the end of it which when invoked always starts the next request and then returns EOF. The UnionInputStream will automatically pop that dummy stream out, and then read the next request's stream. This way we never get into the state where we don't think we need to run another request in order to satisfy the current read request, but we really do. The bug was hidden for so long because BasePackConnection.init() was always wrapping the InputStream into a BufferedInputStream with an 8 KiB buffer. This made the odds of us reading from the UnionInputStream the exact number of available bytes quite low, as the BufferedInputStream would always try to read a full buffer size. Change-Id: I02b5ec3ef6853688687d91de000a5fbe2354915d Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
generated by cgit v1.2.3 (git 2.39.1) at 2025-04-13 13:01:12 +0000