Change-Id: I46c39f2eceb4d58e49bd6273b87711f35250ab5c

Java 8 fixed the silent flush during close issue by
FilterOutputStream (base class of BufferedOutputStream)
using try-with-resources to close the stream, getting a
behavior matching what JGit's SafeBufferedOutputStream
was doing:

  try {
    flush();
  } finally {
    out.close();
  }

With Java 8 as the minimum required version to run JGit
it is no longer necessary to override close() or have
this class. Deprecate the class, and use the JRE's version
of close.

Change-Id: Ic0584c140010278dbe4062df2e71be5df9a797b3

Change-Id: I7c545d06b1bced678c020fab9af1382bc4416b6e
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>

Bazel runs ErrorProne by default and ErrorProne rightly complains that
allowing the user to specify any Cipher can lead to insecure code
(in particular, getCipher("AES") operates in ECB mode). Unfortunately
this is required to support existing repositories insecurely stored
on S3.

Extract the insecure factory code to its own class so this can be built
as a java_library() with this check disabled.

Change-Id: I34f381965bdaa25d5aa8ebf6d8d5271b238334e0

Work around issues with JSch not handling interrupts by
isolating the JSch interactions onto another thread.

Run write and flush on a single threaded Executor using
simple Callable operations wrapping the method calls,
waiting on the future to determine the outcome before
allowing the caller to continue.

If any operation was interrupted the state of the stream
becomes fuzzy at close time. The implementation tries to
interrupt the pending write or flush, but this is very
likely to corrupt the stream object, so exceptions are
ignored during such a dirty close.

Change-Id: I42e3ba3d8c35a2e40aad340580037ebefbb99b53

Java 8 is now the minimum for JGit, so Java 7
only paths are not necessary.

Change-Id: I0151625fed4d0da95321ebed5cca648b8c29d5f1

With the auto option, gc checks whether any housekeeping is required; if
not, it exits without performing any work. Some JGit commands run gc
--auto after performing operations that could create many loose objects.
Housekeeping is required if there are too many loose objects or too many
packs in the repository.

If the number of loose objects exceeds the value of the gc.auto option
jgit's GC consolidates all existing packs into a single pack (equivalent
to -A option), whereas git-core would combine all loose objects into a
single pack using repack -d -l. Setting the value of gc.auto to 0
disables automatic packing of loose objects.

If the number of packs exceeds the value of gc.autoPackLimit, then
existing packs (except those marked with a .keep file) are consolidated
into a single pack by using the -A option of repack. Setting
gc.autoPackLimit to 0 disables automatic consolidation of packs.

Like git the following jgit commands run auto gc:
- fetch
- merge
- rebase
- receive-pack

The auto gc for receive-pack can be suppressed by setting the config
option receive.autogc = false

Change-Id: I68a2a051b39ec2c53cb7c4b8f6c596ba65eeba5d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

Remove the assumption that the local repository is a file based one.

Change-Id: I8f10fe7a54e9fc07f2a23d7901e52b65aa570d45
Signed-off-by: Thomas Meyer <thomas.mey@web.de>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

Change-Id: I2abde5dbd4b785d70b7bc0b77188c0a075130eeb
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

Spawn an ObjectReader from the ObjectInserter, so the flush can be
delayed at the end of the fetch.

Change-Id: I35fe8c8370c06c25262645202aec2b1318057c19
Signed-off-by: Zhen Chen <czhen@google.com>

UploadPackLogger is deprecated but will not be removed until
JGit version 5.0.

Suppress the unavoidable deprecation warnings on usages of the
interface that are kept for backwards compatibility.

Add a TODO so that we don't forget to remove it in 5.0.

Change-Id: Id248002b9bdf23db192427196d54c722a012106c
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>

1beb3ccd1e was submitted in 4.6.0-SNAPSHOT

Change-Id: I33e027dbd0995ba36491bd9e2d4b3c84f9f0abd6
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

This is like PackStatistics, but for PackParser.

Change-Id: I854215c0956fd0b36843d631780be303e021b8be
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>

This allows the same try/catch to handle parsing the command list,
push certificate and push options. Any errors will be caught and
handled by the same catch block, as the client is in the same state.

Change-Id: I13a66f9100e2dc8ca8f72cd701a5bd44d093ec84

Checking if the instance allows push options before returning the
collection or null is a bit overkill. Just return the collection
or return null.

Change-Id: Icdc3755194373966e5819284aeb9bfe8dd34de82

Some embeddings of JGit require creating a ReceivePack instance in
another process from the one that handled the network socket with the
client.  Similar to the PushCertificate add a setter to allow the
option list to be supplied.

Change-Id: I303a30e54942ad067c79251eff8b53329c406628

Refactor all of the push option support code to allocate the list
immediately before parsing the options section off the stream.

Move option support down to ReceivePack instead of BaseReceivePack.
Push options are specific to the ReceivePack protocol and are not
likely to appear in the 4 year old subscription proposal.  These
changes are OK before JGit 4.5 ships as no consumer should be relying
on these new APIs.

Change-Id: Ib07d18c877628aba07da07cd91875f918d509c49

Initialize pushOptions when we decide to use them, instead of when we
advertise them.

In the case of HTTP the advertisement is in a different network
request, hence in a different instance of the BaseReceivePack.

Change-Id: I094c60942e04de82cb6d8433c9cd43a46ffae332
Signed-off-by: Stefan Beller <sbeller@google.com>

Imitate the packet tracing feature from C Git v1.7.5-rc0~58^2~1 (add
packet tracing debug code, 2011-02-24).  Unlike C Git, use the log4j
log level setting instead of the GIT_TRACE_PACKET environment variable
to enable tracing.

Tested as follows:

 1. Enable tracing by adding the lines

	log4j.logger.org.eclipse.jgit.transport=DEBUG, stderr
	log4j.additivity.org.eclipse.jgit.transport=false

    to org.eclipse.jgit.pgm/resources/log4j.properties.

 2. mvn package

 3. org.eclipse.jgit.pgm/target/jgit \
	ls-remote git://git.kernel.org/pub/scm/git/git 2>&1 |less

Then the output provides a trace of packets sent and received over
the wire:

  2016-08-24 16:36:42 DEBUG PacketLineOut:145 - git> git-upload-pack /pub/scm/git/git^@host=git.kernel.org^@
  2016-08-24 16:36:42 DEBUG PacketLineIn:165 - git< 2632c897f7 HEAD^@multi_ack thin-pack side-band side-band-64k ofs-delta shallow no-progress include-tag multi_ack_detailed symref=HEAD:refs/heads/master agent=git/2.8.4
  2016-08-24 16:36:42 DEBUG PacketLineIn:165 - git< e0c1ceafc5 refs/heads/maint

Change-Id: I5028c064f3ac090510386057cb4e6d30d4eae232
Signed-off-by: Dan Wang <dwwang@google.com>

The exception can be thrown in a various reason, and sometimes 403
Forbidden is not appropriate. Make the HTTP status code customizable.

Change-Id: If2ef6f454f7479158a4e28a12909837db483521c
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>

Change-Id: I5c6790719991931d615d821c900bfd90a20e540b
Signed-off-by: Dan Wang <dwwang@google.com>

Since 84d2738ff2 (Don't skip want validation when the client sends no
haves, 2013-06-21), this branch is not taken.  Process the
"shallow"s anyway as a defensive measure in case the code path gets
revived.

Change-Id: Idfb834825d77f51e17191c1635c9d78c78738cfd
Signed-off-by: Jonathan Nieder <jrn@google.com>

d385a7a5e5 (Shallow fetch: Respect "shallow" lines, 2016-08-03) forgot
that UploadPack wasn't passing a DepthWalk to PackWriter in the first
place.  As a result, shallow clones fail:

  java.lang.IllegalArgumentException: Shallow packs require a DepthWalk
        at org.eclipse.jgit.internal.storage.pack.PackWriter.preparePack(PackWriter.java:756)
        at org.eclipse.jgit.transport.UploadPack.sendPack(UploadPack.java:1497)
        at org.eclipse.jgit.transport.UploadPack.sendPack(UploadPack.java:1381)
        at org.eclipse.jgit.transport.UploadPack.service(UploadPack.java:774)
        at org.eclipse.jgit.transport.UploadPack.upload(UploadPack.java:667)
        at org.eclipse.jgit.http.server.UploadPackServlet.doPost(UploadPackServlet.java:191)

Change-Id: Ib0d8c2946eebfea910a2b767fb92e23da15d4749

cgit changed the --depth parameter to mean the total depth of history
rather than the depth of ancestors to be returned [1]. JGit still uses
the latter meaning, so update it to match cgit.

depth=0 still means a non-shallow clone. depth=1 now means only the
wants rather than the wants and their direct parents.

This is accomplished by changing the semantic meaning of "depth" in
UploadPack and PackWriter to mean the total depth of history desired,
while keeping "depth" in DepthWalk.{RevWalk,ObjectWalk} to mean
the depth of traversal. Thus UploadPack and PackWriter always
initialize their DepthWalks with "depth-1".

[1] upload-pack: fix off-by-one depth calculation in shallow clone
https://code.googlesource.com/git/+/682c7d2f1a2d1a5443777237450505738af2ff1a

Change-Id: I87ed3c0f56c37e3491e367a41f5e555c4207ff44
Signed-off-by: Terry Parker <tparker@google.com>

We have to be able to access the enum from outside the package as part of
the API.

Change-Id: I4bdc6bd53a14237c5f4fb9397ae850f9a24c4cfb
Signed-off-by: Stefan Beller <sbeller@google.com>

Gerrit's superproject subscription feature uses RefSpecs to formalize
the ACLs of when the superproject subscription feature is allowed.

As this is a slightly different use case than describing a local/remote
pair of refs, we need to be more permissive. Specifically we want to allow:

    refs/heads/*
    refs/heads/*:refs/heads/master
    refs/heads/master:refs/heads/*

Introduce a new constructor, that allows constructing these RefSpecs.

Change-Id: I46c0bea9d876e61eb2c8d50f404b905792bc72b3
Signed-off-by: Stefan Beller <sbeller@google.com>

We had a case in Gerrits superproject subscriptions where
'refs/heads/' was configured with the intention to mean 'refs/heads/*'.

The first expression lacks the '*', which is why it is not considered
a wildcard but it was considered valid and so was not found early to be
a typo.

Refs are not allowed to end with '/' anyway, so add a check for that.

Change-Id: I3ffdd9002146382acafb4fbc310a64af4cc1b7a9
Signed-off-by: Stefan Beller <sbeller@google.com>

Example usage:
$ ./jgit push \
  --push-option "Reviewer=j.doe@example.org" \
  --push-option "<arbitrary string>" \
  origin HEAD:refs/for/master
Stefan Beller has also made an equivalent change to CGit:
http://thread.gmane.org/gmane.comp.version-control.git/299872

Change-Id: I6797e50681054dce3bd179e80b731aef5e200d77
Signed-off-by: Dan Wang <dwwang@google.com>

If the client sent a well-formed enough request to see it wants to use
side-band-64k for status reporting (meaning its a modern client), but
any other command record was somehow invalid (e.g. corrupt SHA-1)
report the parsing exception using channel 3.  This allows clients to
see the failure and know the server will not be continuing.

git-core and JGit clients send all commands and then start a sideband
demux before sending the pack. By consuming all commands first we get
the client into a state where it can see and respond to the channel 3
server failure.

This behavior is useful on HTTPS connections when the client is buggy
and sent a corrupt command, but still managed to request side-band-64k
in the first line.

Change-Id: If385b91ceb9f024ccae2d1645caf15bc6b206130

The "shallow $id" parsing can also throw InvalidObjectIdException,
just like parseCommand. Move it into its own method with a proper
try-catch block to convert to the checked PackProtocolException.

Change-Id: I6839b95f0bc4fbf4d2c213331ebb9bd24ab2af65

This exception is already handled in the outer catch block.

Change-Id: Ifc04b8e138732a97b9c0ee323af923b8e52a7f8e

Instead of deferring until after command parsing, enable the
capabilities after the first pkt-line has been read from the client.
This allows the server to setup the side-band-64k channel immediately.

Change-Id: I141b7fc92e983a41d3a58da8e1464a6917422b6c

Change-Id: I328f5952f23185a7d8f81a144979b4f8a095bd60

If the push client has requested side-band support the server can
signal a fatal error parsing the pack using the error channel (3)
and then hang up. This may cause the PackWriter to fail to write to
data onto the network socket, which throws a misleading error back
up to the application and the user.

During a write failure poll the input to see if the side band system
can parse out an error message off channel 3. This should be fast as
there will either be an error present in the buffer, or the remote will
also have hung-up on the side band channel. In the case of a hang-up
just rethrow the original IOException as its a network error.

This roughly matches what C git does; once commands are sent and the
packer is started a new thread runs in the background to decode any
possible server error during unpacking on the remote peer

Change-Id: Idb37a4a122a565ec4b59130e08c27d963ba09395

The more specific type InvalidObjectIdException is thrown by
ObjectId.fromString().  Use it here in ReceivePack as the more
generic IAE is never thrown by the body of the try-catch block.

Change-Id: I53fc13c561c7d429a50b5eb82773f1a670431c54

A RefAdvertiser writing to the network includes both the reference's
ObjectId and its peeled ObjectId in the advertised set.  In smart HTTP
negotiation requests may bypass the RefAdvertiser and quickly build
the set based on current refs; include the peeled ObjectIds to match
behavior with the normal bidirectional protocols on git:// and SSH.

Change-Id: I5371bed60da36e8d12c4ad9a5c1d91a0f0ad486b

This field was being set twice within the block.  Setting it just once
is sufficient.  writeString() does not examine the field so it is fine
to set it after the call.

Change-Id: Ib4c74df4f1304e9df3015885bf360bf0d7bc6ca2

Change-Id: Ibab89fddda7ccfe3390348aa129ab2e5e3b7f538

Example use case is to set a different IdentityRepository, for example,
a RemoteIdentityRepository to allow SSH Agent supplied credentials.

Change-Id: Id4a4afd64464e452ffe6d6ad49036f9e283b4655
Signed-off-by: markdingram <markdingram@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

The native wire protocol sends ref advertisements in the pkt-line
format, which requires encoding the ObjectId and ref name onto a byte
sequence.  Busy servers show this is a very high source of garbage,
which pushes the garbage collector harder when there are many refs in
the repository (e.g.  70k, in a Gerrit managed repository).

Optimize the side band advertiser by retaining the CharsetEncoder,
minimizing the amount of temporary garbage built during encoding.

Change-Id: I406c654bf82c1eb94b38862da2425e98396134cb

Git servers supporting HTTP transport can send multiple WWW-Authenticate
challenges [1] for different authentication schemes the server supports.
If authentication fails now retry all authentication types proposed by
the server.

[1] https://tools.ietf.org/html/rfc2617#page-3

Bug: 492057
Change-Id: I01d438a5896f9b1008bd6b751ad9c7cbf780af1a
Signed-off-by: Christian Pontesegger <christian.pontesegger@web.de>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

Change-Id: Iecf2b8deafc4991cc3333702fb9fa0638be7b914
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

JGits PushCommand and BasePackPushConnection were throwing a generic
exception when the pushed pack-file was rejected by the server since it
contained too large objects. Teach JGit to better analyze the server's
response to detect this situation and throw a more specific exception.

Detect this situation by parsing the status line sent by the server.
This change only recognizes the response sent by a JGit based server.
All other servers which report such problems in a different way still
lead to a generic TransportExceptions.

Also see https://git.eclipse.org/r/#/c/46348/

Change-Id: I8d6d65e4585ebb3846f7207e7d1a2f82fa9cbd86
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

This makes atomic more consistent with other properties like
allowNonFastForwards.

Change-Id: Ie3b41f14ffba1d0b206ffecc697377417daa6973

Change-Id: I0e4e6cef693e8138562f5ca345e0db61a8abef9c

Now if refs are unreadable when serving an upload pack the handler
will fail due to the actual underlying failure. Previously all wants
would be rejected as invalid because Repository.getAllRefs() returned
an empty map.

Testing this required a new subclass of InMemoryRepository so that
an IOException could be injected at the correct time.

Signed-off-by: Michael Edgar <adgar@google.com>
Change-Id: Iac708b1db9d0ccce08c4ef5ace599ea0b57afdc0

The reason is that URIish(URL) and URIish(String) make different parsing
of path / rawPath with regard to drive letters. /C:/... for URL and
C:/... for String. This patch fixes the issue.

Change-Id: I8e2013fff30b7bb198ff733c038e21366667b8a0
Signed-off-by: Ivan Motsch <ivan.motsch@bsiag.com>

Capture the internal "want X not valid" state as a specific subclass
of PackProtocolException, allowing this to be more easily identified
in server stack traces and wrapper application code.

Change-Id: I4b1adb7497f396432da420b0f600ad25a261f912

If the client sends a SHA-1 that the server does not recognize echo
this back to the client with an explicit error message instead of
the generic "internal server error".

This was always the intent of the implementation but it was being
dropped on smart HTTP due to the UploadPackServlet catching the
PackProtocolException, discarding the buffered message UploadPack
meant to send, and sending along a generic message instead.

Change-Id: I8d96b064ec655aef64ac2ef3e01853625af32cd1

Change-Id: Ibc3e2f3372e1a65732dd6d3c71cec53fb1aa15e2
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>