Report PackProtocolExceptions to client during receive-pack
We have done this since forever with the "wanted old new ref" error,
so let's do it for other such errors thrown in the same block as well.
Change-Id: Ib3b1c7f05e31a5b3e40e85eb07b16736920a033b
Discussion on the git mailing list has concluded[1] that the intended
behavior for all (non-sideband) portions of the receive-pack protocol
is for trailing LFs in pkt-lines to be optional. Go back to using
PacketLineIn#readString() everywhere.
For push certificates specifically, we agreed that the payload signed
by the client is always concatenated with LFs even though the client
MAY omit LFs when framing the certificate for the wire. This is still
reflected in the implementation of PushCertificate#toText().
[1] http://thread.gmane.org/gmane.comp.version-control.git/273175/focus=273412
Change-Id: I817231c4d4defececb8722142fea18ff42e06e44
This may be used by e.g. a custom reflog implementation to record
this information along with the ref update.
Change-Id: I44adbfad704b76f9c1beced6e1ce82eaf71410d2
The default behavior is to read a repository's signed push
configuration from that repo's config file, but this is not very
flexible when it comes to managing groups of repositories (e.g. with
Gerrit). Allow callers to override the configuration using a POJO.
Change-Id: Ib8f33e75daa0b2fbd000a2c4558c01c014ab1ce5
BaseReceivePack: fix reading cert lines in command loop
Add a missing continues to prevent falling through to the command
parsing section. The first continue happens when the command list is
empty, so change the condition to see whether we have read the first
line, rather than any commands.
Fix comparison to BEGIN_SIGNATURE to use raw line with newline.
Change-Id: If3d92f5ceade8ba7605847a4b2bc55ff17d119ac
- Consistently return structured data, such as actual ReceiveCommands,
which is more useful for callers that are doing things other than
verifying the signature, e.g. recording the set of commands.
- Store the certificate version field, as this is required to be part
of the signed payload.
- Add a toText() method to recreate the actual payload for signature
verification. This requires keeping track of the un-chomped command
strings from the original protocol stream.
- Separate the parser from the certificate itself, so the actual
PushCertificate object can be immutable. Make a fair attempt at deep
immutability, but this is not possible with the current mutable
ReceiveCommand structure.
- Use more detailed error messages that don't involve NON-NLS strings.
- Document null return values more thoroughly. Instead of having the
undocumented behavior of throwing NPE from certain methods if they
are not first guarded by enabled(), eliminate enabled() and return
null from those methods.
- Add tests for parsing a push cert from a section of pkt-line stream
using a real live stream captured with Wireshark (which, it should
be noted, uncovered several simply incorrect statements in C git's
Documentation/technical/pack-protocol.txt).
This is a slightly breaking API change to classes that were
technically public and technically released in 4.0. However, it is
highly unlikely that people were actually depending on public
behavior, since there were no public methods to create
PushCertificates with anything other than null field values, or a
PushCertificateParser that did anything other than infinite loop or
throw exceptions when reading.
Change-Id: I5382193347a8eb1811032d9b32af9651871372d0
C git's receive-pack.c strips trailing newlines in command lists when
present[1], although send-pack.c does not send them, at least in the
case of command lists[2]. Change JGit to match this behavior.
Add tests.
This also fixes parsing of commands in the push cert, which, unlike
commands sent in the non-push case, always have trailing newlines.
[1] 7974889a05/builtin/receive-pack.c (L1380)
where packet_read_line chomps newlines:
7974889a05/pkt-line.c (L202)
[2] 7974889a05/send-pack.c (L470)
Change-Id: I4bca6342a7482a53c9a5815a94b3c181a479d04b
Fix that exceptions in ReceivePack cause Invalid Channel 101 exceptions
When during a PushOperation the server hits an exception different from
UnpackException the JGit server behaved wrong. That kind of exceptions
are handled so late that the connection is already released and the
information whether to talk sideband to the client is lost. In detail:
ReceivePack.receive() will call release() and that will reset the
capabilities. But later on the stack in ReceivePackServlet.doPost() it
is tried to send a response to client now with reset capabilities (no
sideband!).
Change-Id: I0a609acc6152ab43b47a93d712deb65bb1105f75
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
git-core has supported this for a long time; allowing clients to
avoid progress messages from the server if they are dumping to a
pipe instead of a tty.
Avoid the two progress monitors going on side-band and expose
isQuiet() method to allow hooks to also reduce their output if
this is sensible for them.
Change-Id: I1df7e38d16765446b441366500b017a90b8ff958
Despite being the primary author of RevWalk and ObjectWalk I still
fail to remember to setRetainBody(false) in application code using
an ObjectWalk to examine the graph.
Document the default for RevWalk is setRetainBody(true), where the
application usually wants the commit bodies to display or inspect.
Change the default for ObjectWalk to setRetainBody(false), as nearly
all callers want only the graph shape and do not need the larger text
inside a commit body. This allows some code in JGit to be simplified.
Change-Id: I367e42209e805bd5e1f41b4072aeb2fa98ec9d99
Do not concatenate strings as arguments to StringBuilder.append()
That more or less defeats the purpose of using a StringBuilder.
Change-Id: I519f7bf1c9b6670e63c3714210f834ee845dc69f
Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>
Since git-core ff5effd (v1.7.12.1) the native wire protocol transmits
the server and client implementation and version strings using
capability "agent=git/1.7.12.1" or similar.
Support this in JGit and hang the implementation data off UploadPack
and ReceivePack. On HTTP transports default to the User-Agent HTTP
header until the client overrides this with the optional capability
string in the first line.
Extract the user agent string into a UserAgent class under transport
where it can be specified to a different value if the application's
build process has broken the Implementation-Version header in the
JGit package.
Change-Id: Icfc6524d84a787386d1786310b421b2f92ae9e65
Add fsck.allowInvalidPersonIdent to accept invalid author/committers
A larger than expected number of real-world repositories found on
the Internet contain invalid author, committer and tagger lines
in their history. Many of these seem to be caused by users misusing
the user.name and user.email fields, e.g.:
[user]
name = Au Thor <author@example.com>
email = author@example.com
that some version of Git (or a reimplementation thereof) copied
directly into the object header. These headers are not valid and
are rejected by a strict fsck, making it impossible to transfer
the repository with JGit/EGit.
Another form is an invalid committer line with double negative for
the time zone, e.g.
committer Au Thor <a@b> 1288373970 --700
The real world is messy. :(
Allow callers and users to weaken the fsck settings to accept these
sorts of breakages if they really want to work on a repo that has
broken history. Most routines will still function fine, however
commit timestamp sorting in RevWalk may become confused by a corrupt
committer line and sort commits out of order. This is mostly fine if
the corrupted chain is shorter than the slop window.
Change-Id: I6d529542c765c131de590f4f7ef8e7c1c8cb9db9
Push certificates ("git push --signed") have been part of
git-core since version 2.2.0 (released Nov 26 2014). We also
want to support that feature.
This is not complete and is lacking the actual functionality
to validate the signature for now.
Change-Id: I249869cadb2d55aef016371b9311b8583591b9cf
Signed-off-by: Stefan Beller <sbeller@google.com>
Implement atomic refs update, if possible by database
Inspired by the series[1], this implements the possibility to
have atomic ref transactions.
If the database supports atomic ref update capabilities, we'll
advertise these. If the client wishes to use this feature, either
all refs will be updated or none at all.
[1] http://thread.gmane.org/gmane.comp.version-control.git/259019/focus=259024
Change-Id: I7b5d19c21f3b5557e41b9bcb5d359a65ff1a493d
Signed-off-by: Stefan Beller <sbeller@google.com>
In Git 1.9 (5dbd767601 "support pushing from a shallow clone")
the git-core project intentionally broke the existing send-pack
protocol from shallow clients.
Shallow clients now transmit their shallow information during push,
ahead of the old-new command sequence. JGit must accept these lines
when presented.
To protect the server against clients sending partial history,
require the connectivity check when pushed to by a shallow client.
Change-Id: I46639366b0900052c376091e1688f07def44ab79
Display progress while checking connectivity on push
Verifying 100 new objects are fully connected to the existing DAG
is usually very cheap. Checking the entire Linux kernel history is
fully connected when pushing it to a new repository can take 30-60
seconds. Display a progress counter during this time so the client
knows the server is still working.
Change-Id: Iababe3ee1d35cb82f2bef2f12da7a2ecd03282b0
Allow configuration of receive pack's ObjectChecker through fsck.*
fsck.allowLeadingZeroFileMode may be set true to permit pushing
broken trees with leading '0' in the file mode.
fsck.safeForWindows may be set true to require new trees to have
only file names that are safe on the Windows platform.
fsck.safeForMacOS may be set true to require new trees to have
only file names that do not cause collisions or confusion on the
Mac OS platform.
Change-Id: I1a225c1b3cd13c0d1a0d43fffe79355c501f49b7
Default receive.fsckObjects to transfer.fsckObjects
ReceivePack should configure itself with receive.fsckObjects,
and if not defined, transfer.fsckObjects. This is the order
used by git-core.
Change-Id: I41f243633dacb606dbcc3132972f63bbaba174d1
Allow ReceivePack callers to configure their own ObjectChecker
PackParser permits supplying a specific ObjectChecker instance.
Allow this to be passed through ReceivePack, giving the caller
more flexibility to configure the implementation.
Change-Id: I9440dd25588008626222f33bfd697f57c05b439e
PostReceiveHooks can make use of this information to, for example,
update a cached size of the Git repository.
Change-Id: I2bf1200959a50531e2155a7609c96035ba45b10d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Revert "Add getPackFile to ReceivePack to make PostReceiveHook more
usable"
This reverts commit 2670fd427c.
By returning an instance of File from the ReceivePack.getPackFile the
abstraction of the persistence implementation was broken.
Change-Id: I28e3ebf3a659a7cbc94be51bba9e1ad338f2b786
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Add getPackFile to ReceivePack to make PostReceiveHook more usable
Having access to the pack file that was created by the ReceivePack
may be useful for post receive hooks. For example, a hook may want
to check the size of the received pack and the created index.
Change-Id: I4d51758e4565d32c9f8892242947eb72644b847d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Possibility to limit the max pack size on receive-pack
The maxPackSizeLimit, when set, will reject a pack if it exceeds
that limit.
This feature is intended to provide a mechanism to control disk space
quota on Git repositories.
Change-Id: I83d8db670875c395f8171461b402083323e623a5
CQ: 7896
JGit 3.0: move internal classes into an internal subpackage
This breaks all existing callers once. Applications are not supposed
to build against the internal storage API unless they can accept API
churn and make necessary updates as versions change.
Change-Id: I2ab1327c202ef2003565e1b0770a583970e432e9
A few classes such as Constanrs are marked with @SuppressWarnings, as are
toString() methods with many liternal, but otherwise $NLS-n$ is used for
string containing text that should not be translated. A few literals may
fall into the gray zone, but mostly I've tried to only tag the obvious
ones.
Change-Id: I22e50a77e2bf9e0b842a66bdf674e8fa1692f590
Simplify push error message when ref already exists
If a client attempts to create a branch that already exists on the
remote side, tell them "already exists" rather than repeat lots of
information about the reference. Previously the error looked like:
! [remote rejected] tags/1.3.1 -> 1.3.1 (Ref Ref[refs/tags/1.3.1=e3857ee05...] already exists)
Now it will simply say:
! [remote rejected] tags/1.3.1 -> 1.3.1 (already exists)
Change-Id: I96fc67ca8b650052de6e662449a3c5bc8bbc010b
None of these should have been exposed to base classes. The majority
of them are private implementation details that are not required by a
subclass in order to interact with the base protocol definition. The
few that are needed should be visible as accessor methods, so the
internals can be modified without breaking the public JGit API.
Change-Id: I874179105c9c37703307facbbf99387c52bf772c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
I have unfortunately introduced a few bugs in the native Git client
over the years. 1.7.5 is unable to send chunked requests correctly,
resulting in corrupt data at the server. Ban this client whenever
it uses chunked encoding with an error message.
Prior to some more recent versions, git push over HTTP failed to
report status information and error messages due to a race within
the client and its helper process. Check for these bad versions and
send errors as messages before the status report, enabling users
to see the failures on their terminal.
Change-Id: Ic62d6591cbd851d21dbb3e9b023d655eaecb0624
Disable PackParser EOF check if more data expected
The PackParser EOF check is incompatible with the expect data after
pack footer flag, so turn off the EOF check if the expecting data
flag is true.
Change-Id: I697ebd9e1d1eed765d00aecaef955cf978cfd0b9
When receiving a pack, data buffered after the pack can restored
to the InputStream if the stream supports mark and reset.
Change-Id: If04915c32c91be28db8df7e8491ed3e9fe0e1608
clone, fetch and push can all update multiple references in a single
command invocation. Rather than performing sequential iteration
of each reference change inside of the application code, push this
down into the reference database where the implementation can take
advantage of the batch size and optimize itself.
For the local filesystem implementation the obvious optimization
is to write a packed-refs file when the repository is completely
empty. The initial clone, fetch or push into the destination may
have hundreds of new references. Writing all of these as loose
files is not efficient. This optimization is not implemented in
this commit and is left as an exercise for the reader to supply
in a future commit to JGit.
To make the API changes simple, define the BatchRefUpdate type and
implementation using the existing sequential behavior.
Change-Id: I8e1674f091e05e24e3ff56ccbc687a6d18a6a61e
Expose ReceiveCommand.updateType to check for UPDATE_NONFASTFORWARD
When a command's type is UPDATE, JGit might not yet be sure if it
is a fast-forward or not. Expose a utility method to compute the
exact type by performing the merge base test, allowing the type
to be switched to UPDATE_NONFASTFORWARD if old ObjectId is not
contained in new ObjectId.
BaseReceivePack already does this test when validating the incoming
command list, so provide a package level backdoor to set the type
and avoid needing to redo the merge test later.
Change-Id: If5a6fcc50dc4d6f96e9bb0bb7bba15ebe8b86377
Reject non-fast-forwards earlier in BaseReceivePack
If BaseReceivePack has setAllowNonFastForwards(false) configured
(such as by receive.denynonfastforwards), automatically reject
any command that attempts a non-fast-forward update before it goes
further in processing.
This matches with other checks in validateCommands(), such as the
early failure of delete attempts when isAllowDeletes() is false.
Change-Id: I3bb28e4dd6d17cb31ede09eb84ceb67cdb17ea5d
We are working on a publish/subscribe based git protocol, and we want to
reuse several parts of the ReceivePack-like code for reading commands
and processing a pack. In this new implementation, the connection
management will be very different, in particular, there may be multiple
packs received on a single open connection. So, hoist out as much as we
can from ReceivePack, mostly just leaving behind the single-connection
version in that class.
Change-Id: I5567aad6ae77951f73f59c1f91996d934ea88334
When a client POSTs to /git-{upload,receive}-pack, the first line
includes their client capabilities. As soon as the C git client sends
side-band(-64k), it goes into a state where it chokes on data not sent
in a valid sideband channel.
GitSmartHttpTools.sendError() is called early in the request, likely
before a {Upload,Receive}Pack handler is assigned or, even so, before it
has read the request. In some cases we must read the first line manually
within sendError() to tell whether sideband is needed.
Change-Id: I8277fd45a4ec3b71fa8f87404b4f5d1a09e0f384
Modify refs in UploadPack/ReceivePack using a hook interface
This is intended to replace the RefFilter interface (but does not yet,
for backwards compatibility). That interface required lots of extra
scanning and copying in filter cases such as only advertising a subtree
of the refs directory. Instead, provide a hook that can be executed
right before ref advertisement, using the public methods on
UploadPack/ReceivePack to explicitly set the map of advertised refs.
Change-Id: I0067019a191c8148af2cfb71a675f2258c5af0ca
Dave Borowitz
Christian Halstrick
Shawn Pearce
Sebastian Schuberth
Matthias Sohn
Stefan Beller
Shawn Pearce
Saša Živkov
Robin Rosenberg
Ian Wetherbee