Dave Borowitz [Wed, 17 Jun 2015 17:12:22 +0000 (13:12 -0400)]
Add a separate type for the identity in a push certificate
These differ subtly from a PersonIdent, because they can contain
anything that is a valid User ID passed to gpg --local-user. Upstream
git push --signed will just take the configuration value from
user.signingkey and pass that verbatim in both --local-user and the
pusher field of the certificate. This does not necessarily contain an
email address, which means the parsing implementation ends up being
substantially different from RawParseUtils.parsePersonIdent.
Nonetheless, we try hard to match PersonIdent behavior in
questionable cases.
Dave Borowitz [Mon, 15 Jun 2015 20:50:22 +0000 (16:50 -0400)]
PushCertificateParser: include begin/end lines in signature
The signature is intended to be passed to a verification library such
as Bouncy Castle, which expects these lines to be present in order to
parse the signature.
Dave Borowitz [Mon, 15 Jun 2015 19:48:22 +0000 (15:48 -0400)]
PushCertificateParser: throw PackProtocolException in more cases
This is the subclass of IOException already thrown by
BaseReceivePack#recvCommands when encountering an invalid value on
the wire. That's what PushCertificateParser is doing too, so use the
same subclass.
Dave Borowitz [Mon, 15 Jun 2015 19:25:14 +0000 (15:25 -0400)]
Extract a class for signed push configuration
The default behavior is to read a repository's signed push
configuration from that repo's config file, but this is not very
flexible when it comes to managing groups of repositories (e.g. with
Gerrit). Allow callers to override the configuration using a POJO.
Dave Borowitz [Mon, 15 Jun 2015 18:54:13 +0000 (14:54 -0400)]
BaseReceivePack: fix reading cert lines in command loop
Add a missing continues to prevent falling through to the command
parsing section. The first continue happens when the command list is
empty, so change the condition to see whether we have read the first
line, rather than any commands.
Fix comparison to BEGIN_SIGNATURE to use raw line with newline.
a85e817d is a slightly breaking API change to classes that were
technically public and technically released in 4.0. However, it is
highly unlikely that people were actually depending on public behavior,
since there were no public methods to create PushCertificates with
anything other than null field values, or a PushCertificateParser that
did anything other than infinite loop or throw exceptions when reading.
Change-Id: I1d0ba9ea0a347e8ff5a0f4af169d9bb18c5838d2 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Terry Parker [Fri, 12 Jun 2015 19:00:36 +0000 (12:00 -0700)]
Introduce PostUploadHook to replace UploadPackLogger
UploadPackLogger is incorrectly named--it can be used to trigger any
post upload action, such as GC/compaction. This change introduces
PostUploadHook/PostUploadHookChain to replace
UploadPackLogger/UploadPackLoggerChain and deprecates the latter.
It also introduces PackStatistics as a replacement for
PackWriter.Statistics, since the latter is not public API.
It changes PackWriter to use PackStatistics and reimplements
PackWriter.Statistics to delegate to PackStatistics.
Change-Id: Ic51df1613e471f568ffee25ae67e118425b38986 Signed-off-by: Terry Parker <tparker@google.com>
Jonathan Nieder [Wed, 10 Jun 2015 22:43:48 +0000 (15:43 -0700)]
Treat CloneCommand.setBranch(null) as setBranch("HEAD")
This method is documented to take a branch name (not a possibly null
string). The only way a caller could have set null without either
re-setting to a sane value afterward or producing NullPointerException
was to also call setNoCheckout(true), in which case there would have
been no reason to set the branch in the first place.
Make setBranch(null) request the default behavior (remote's default
branch) instead, imitating C git's clone --no-branch.
Change-Id: I960e7046b8d5b5bc75c7f3688f3a075d3a951b00 Signed-off-by: Jonathan Nieder <jrn@google.com>
Jonathan Nieder [Wed, 10 Jun 2015 22:43:27 +0000 (15:43 -0700)]
Treat CloneCommand.setRemote(null) as setRemote("origin")
A non-bare clone command with null remote produces a
NullPointerException when trying to produce a refspec to fetch against.
In a bare repository, a null remote name is accepted by mistake,
producing a configuration with items like 'remote.url' instead of
'remote.<remote>.url'. This was never meant to work.
Instead, let's make setRemote(null) undo any previous setRemote calls
and re-set the remote name to DEFAULT_REMOTE, imitating C git clone's
--no-origin option.
While we're here, add some tests for setRemote working normally.
Change-Id: I76f502da5e677df501d3ef387e7f61f42a7ca238 Signed-off-by: Jonathan Nieder <jrn@google.com>
Jonathan Nieder [Wed, 10 Jun 2015 20:59:48 +0000 (13:59 -0700)]
Handle null in ProgressMonitor setters
These commands' monitor fields can never be null unless someone passes
null to setProgressMonitor. Anyone passing null probably meant to
disable the ProgressMonitor, so do that (by falling back to
NullProgressMonitor.INSTANCE) instead of saving a null and eventually
producing NullPointerException.
Change-Id: I63ad93ea8ad669fd333a5fd40880e7583ba24827 Signed-off-by: Jonathan Nieder <jrn@google.com>
Dave Borowitz [Wed, 10 Jun 2015 00:23:03 +0000 (17:23 -0700)]
Rewrite push certificate parsing
- Consistently return structured data, such as actual ReceiveCommands,
which is more useful for callers that are doing things other than
verifying the signature, e.g. recording the set of commands.
- Store the certificate version field, as this is required to be part
of the signed payload.
- Add a toText() method to recreate the actual payload for signature
verification. This requires keeping track of the un-chomped command
strings from the original protocol stream.
- Separate the parser from the certificate itself, so the actual
PushCertificate object can be immutable. Make a fair attempt at deep
immutability, but this is not possible with the current mutable
ReceiveCommand structure.
- Use more detailed error messages that don't involve NON-NLS strings.
- Document null return values more thoroughly. Instead of having the
undocumented behavior of throwing NPE from certain methods if they
are not first guarded by enabled(), eliminate enabled() and return
null from those methods.
- Add tests for parsing a push cert from a section of pkt-line stream
using a real live stream captured with Wireshark (which, it should
be noted, uncovered several simply incorrect statements in C git's
Documentation/technical/pack-protocol.txt).
This is a slightly breaking API change to classes that were
technically public and technically released in 4.0. However, it is
highly unlikely that people were actually depending on public
behavior, since there were no public methods to create
PushCertificates with anything other than null field values, or a
PushCertificateParser that did anything other than infinite loop or
throw exceptions when reading.
Dave Borowitz [Wed, 10 Jun 2015 19:47:04 +0000 (12:47 -0700)]
Allow trailing newlines in receive-pack
C git's receive-pack.c strips trailing newlines in command lists when
present[1], although send-pack.c does not send them, at least in the
case of command lists[2]. Change JGit to match this behavior.
Add tests.
This also fixes parsing of commands in the push cert, which, unlike
commands sent in the non-push case, always have trailing newlines.
The new submodule layout where GITDIR of a submodule is located at
<parent-repo-GITDIR>/modules/<submodule-path> was only used during
clone. Teach SubmoduleAddCommand to use the new layout.
Bug: 469666
Change-Id: Ie97dc0607b71499560444616f362bccee9cce515 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Jonathan Nieder [Wed, 10 Jun 2015 21:48:21 +0000 (14:48 -0700)]
submodule test: Use config.unset instead of setting to null
Most relative-URL tests for SubmoduleInitCommand carry out the following
steps:
1. add a submodule at path "sub" to the index
2. set remote.origin.url in .git/config
3. configure .gitmodules, possibly using relative URLs, and see what
happens
resolveWorkingDirectoryRelativeUrl() is meant to test the fallback when
remote.origin.url is not set, to match C git which treats the URL as
relative to the cwd in that case. To do so, in step (2) it sets
remote.origin.url to null.
However, Config.setString when taking a null value does not actually
unset that value from the configuration --- it sets it to the empty
string. This means we are testing a behavior that C git never
supported. Use Config.unset instead.
Change-Id: I7af29fbbd333a2598843d62c320093c48b2ad972 Signed-off-by: Jonathan Nieder <jrn@google.com>
* changes:
Allow setting detail message and cause when constructing most exceptions
Use message from ServiceNotAuthorizedException, ServiceNotEnabledException
dumb HTTP: Clarify AsIsFilter by introducing req and res locals
Clarify description of ServiceNotAuthorizedException
Jonathan Nieder [Tue, 9 Jun 2015 22:54:52 +0000 (15:54 -0700)]
Revert "Config: Distinguish between empty and null strings"
This reverts commit 96eb3ee3976e7e9e3e118851fa614cce8a1f7d88, which
broke Gerrit tests that set a config value to 'null', serialize the
result, deserialize, and expect 'null' from Config.getString[1].
The intent of that commit was to make it possible to distinguish between
an absent and an empty config value, which we'll have to do with a new
method.
Revert the behavior change. Keep the tests from 428cb23f2de8, since
they test the behavior more precisely than the old tests did.
[1] https://gerrit-review.googlesource.com/68452
Change-Id: Ie8042f380ea0e34e3203e1991aa0feb2e6e44641 Signed-off-by: Jonathan Nieder <jrn@google.com>
Fredrik Medley [Fri, 29 May 2015 18:04:50 +0000 (20:04 +0200)]
UploadPack: Use reachable-sha1-in-want configuration
C git 2.5 supports setting the equivalent of
RequestPolicy.REACHABLE_COMMIT with uploadpack.allowreachablesha1inwant.
Parse this into TransportConfig and use it from UploadPack. An explicitly
set RequestPolicy overrides the config, and the policy may still be
upgraded on a unidirectional connection to avoid races.
Change-Id: Id39771a6e42d8082099acde11249306828a053c0 Signed-off-by: Fredrik Medley <fredrik.medley@gmail.com>
Jonathan Nieder [Mon, 8 Jun 2015 17:47:15 +0000 (10:47 -0700)]
Add missing # to javadoc comment
Change-Id: I351cee0ba2e77e3360846ac0c5368da3a322725c Reported-by: Markus Keller <markus_keller@ch.ibm.com> Signed-off-by: Jonathan Nieder <jrn@google.com>
Jonathan Nieder [Fri, 5 Jun 2015 23:08:55 +0000 (16:08 -0700)]
Allow lookup of multiple exact refs in one shot
exactRef(ref1, ref2, ref3) requests multiple specific refs in a single
lookup, which may be faster in some backends than looking them up one by
one.
firstExactRef generalizes getRef by finding the first existing ref from
the list of refs named. Its main purpose is for the default
implementation of getRef (finding the first existing ref in a search
path). Hopefully it can be useful for other operations that look for
refs in a search path (e.g., git log --notes=<name>), too.
Change-Id: I5c6fcf1d3920f6968b8b97f3d4c3a267258c4b86 Signed-off-by: Jonathan Nieder <jrn@google.com>
Jonathan Nieder [Fri, 5 Jun 2015 21:14:55 +0000 (14:14 -0700)]
Introduce exactRef to read a ref whose exact name is known
Unlike getRef(name), the new exactRef method does not walk the search
path. This should produce a less confusing result than getRef when the
exact ref name is known: it will not try to resolve refs/foo/bar to
refs/heads/refs/foo/bar even when refs/foo/bar does not exist.
It can be faster than both getRefs(ALL).get(name) and getRef(name)
because it only needs to examine a single ref.
A follow-up change will introduce a findRef synonym to getRef and
deprecate getRef to make the choice a caller is making more obvious
(exactRef or findRef, with the same semantics as getRefs(ALL).get and
getRefs(ALL).findRef).
Change-Id: If1bd09bcfc9919e7976a4d77f13184ea58dcda52 Signed-off-by: Jonathan Nieder <jrn@google.com>
David Pletcher [Thu, 4 Jun 2015 18:48:32 +0000 (11:48 -0700)]
Restore checkObjectCollisions flag
I am developing an offline pack verification feature based on
PackParser. The birthday collision check is a prohibitive obstacle
to performance at scale because it interacts with the repository
to perform collision checks. This CL restores the checkObjectCollisions
flag that was removed in 9638e0aa87614a6fb4f109bbeac0cde3462b9769,
while changing the flag getter and setter to protected from public as a
precaution against misuse.
Change-Id: I363cd0c9de57c5e8659cdfe2d51b17823f4fe793 Signed-off-by: David Pletcher <dpletcher@google.com>
Make JGit make the same distinction. This is in line with the current
Javadoc of getString, which claims to return "a String value from the
config, null if not found". It is more reasonable to interpret "x ="
in the above example as "found" rather than "missing".
We need to maintain the special handling of a key name with no "="
resolving to a boolean true, but "=" with an empty string is still not
a valid boolean.
Return -1 from PackWriter.Statistics.getBitmapIndexMises() when no
bitmap indices were found, to differentiate it from the case where
the bitmap indices contained all of the want/have commits.
Change-Id: I78d4600b462c19f62b347217a0b2c19eaaf3a14b Signed-off-by: Terry Parker <tparker@google.com>
Jonathan Nieder [Tue, 2 Jun 2015 23:52:21 +0000 (16:52 -0700)]
Allow setting detail message and cause when constructing most exceptions
In particular, this means a RepositoryResolver, UploadPackFactory, or
ReceivePackFactory can set a detail message for
ServiceNotAuthorizedException or ServiceNotEnabledException with
information for the client about why access is not allowed.
Change-Id: I38e1798e1e9d09b5e75cefacd9d85f25729235a9 Signed-off-by: Jonathan Nieder <jrn@google.com>
Jonathan Nieder [Tue, 2 Jun 2015 23:40:23 +0000 (16:40 -0700)]
Use message from ServiceNotAuthorizedException, ServiceNotEnabledException
When sending an error response due to ServiceNotAuthorizedException or
ServiceNotEnabledException, usually we send a default message. In the
ServiceNotEnabledException case, we use
403 Git access forbidden
except in a dumb-HTTP-specific filter where we use the servlet
container's default 403 response:
403 Forbidden
In the ServiceNotAuthorizedException case, we use the servlet
container's default 401 response:
401 Unauthorized
There is one exception: a ServiceNotEnabledException when handling a
smart HTTP /info/refs request uses the message from the exception:
403 Service not enabled
Be more consistent by always using the message from the exception. This
way, authors of a RepositoryResolver, UploadPackFactory, or
ReceivePackFactory can provide a more detailed message when appropriate.
The defaults are
401 Unauthorized
403 Service not enabled
Change-Id: Id1fe1c2042fb96487c3671c1965c8a65c4b8e1b8 Signed-off-by: Jonathan Nieder <jrn@google.com>
Jonathan Nieder [Tue, 2 Jun 2015 23:13:16 +0000 (16:13 -0700)]
Clarify description of ServiceNotAuthorizedException
This exception's detail message states
Service not permitted
and according to the Javadoc it indicates that the current user does not
have access to the service. In practice, though, callers handle this
exception by presenting a '401 Unauthorized' response to the client,
meaning that the user is unauthenticated and should authenticate.
Clarify the documentation and detail message to match the practice.
The exception message is not used anywhere except logs. No
client-visible effect intended.
Change-Id: I2c6be9cb74af932f0dcb121a381a64f2ad876766 Signed-off-by: Jonathan Nieder <jrn@google.com>
Matthias Sohn [Mon, 1 Jun 2015 22:13:07 +0000 (00:13 +0200)]
Merge branch 'master' into stable-4.0
* master:
Add more reports to maven site
Update project meta-data in pom.xml used for site generation
Update dependencies to the versions used in target platform
Update 4.5 target platform to use final Mars Orbit repository
Compare API changes in clirr report against 3.7.0
Fix CommitCommand.setOnly()
Add "src" folder to source folders of org.eclipse.jgit.test
Improve exception thrown when pull can't find advertised ref
Silence unchecked conversion warning in TransportSftp
Silence deprecation warning in WindowCacheConfig
Silence deprecation warning in DirCacheCheckout
Fix unchecked conversion warning in MergeFormatter
Fix hidden field warnings in bundle org.eclipse.jgit
Close WindowCursor using try-with-resources in UnpackedObject
Fix WindowCursor memory leak.
archive: Drop unnecessary empty comments and 'final' qualifiers on locals
Close 'out' consistently in ArchiveCommand.call
Fix that exceptions in ReceivePack cause Invalid Channel 101 exceptions
Better report too large pack-files from PushCommand
FS: Extract GobblerThread into a private static class
Add bitmap index misses to PackWriter.Statistics
Enable public access to SimilarityIndex scoring function
Add getters to RepoProject.
Silence unused object warning in MyersDiff
Silence resource leak warnings where caller is responsible to close
Silence false potential null pointer access warnings
Fix potential null pointer access in IndexDiffFilter
Add tests for ObjectFilter
Expose Sets helper to tests outside org.eclipse.jgit.api
Change-Id: I34b2bb45f51ed6f52a6bb1215de654ebb2ffde10 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Fri, 29 May 2015 12:39:33 +0000 (14:39 +0200)]
Update project meta-data in pom.xml used for site generation
- Update list of committers
- Update scm information
- Expose jgit hudson for Maven site
- Name our project's organization
- Scm URL should point to a web-enabled view on the repository
- Extract tycho-extras version used during signing to a property
Change-Id: If32bed323ec283b5e83d28ffe8775220f7c4cecd Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
When CommitCommand.setOnly(path) is used a temporary index has to be
created containing only modifications for the specified pathes. The
process to fill this temporary index was broken because filling the
index was stopped when we reached the first untracked file. Instead the
process to fill the temporary index should continue until we processed
all pathes.
Bug: 451465
Change-Id: I22af50d70bd3b81e7c056358724956122b0d158d Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn [Wed, 15 Apr 2015 06:40:05 +0000 (08:40 +0200)]
Improve exception thrown when pull can't find advertised ref
- throw an API exception instead of an internal exception to allow
applications to handle this problem
- improve error message to give hints about possible root causes
Bug: 464660
Change-Id: Ib7d18bb2eeeac0fc218daea375b290ea5034bda1 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Shawn Pearce [Thu, 28 May 2015 19:39:07 +0000 (15:39 -0400)]
Merge changes I7b6d7be4,I63a74651,I39c2ea6b
* changes:
Silence unused object warning in MyersDiff
Silence resource leak warnings where caller is responsible to close
Fix potential null pointer access in IndexDiffFilter
Jonathan Nieder [Wed, 27 May 2015 23:25:49 +0000 (16:25 -0700)]
archive: Drop unnecessary empty comments and 'final' qualifiers on locals
Early JGit code used comments to inform the Eclipse formatter about
where to break lines and used final in the hope of making code faster.
The ArchiveCommand command implementation imitated that style.
Nowadays the project relies less on the Eclipse formatter and relies
more on Java having sane performance with local variables that are not
explicitly marked 'final'. Removing the unnecessary empty comments and
'final' qualifiers makes this code more readable and more consistent
with recent JGit code.
Change-Id: I7a181432eda7e18bd32cf110d89c0efbe490c4f1 Signed-off-by: Jonathan Nieder <jrn@google.com>
Jonathan Nieder [Wed, 27 May 2015 23:25:32 +0000 (16:25 -0700)]
Close 'out' consistently in ArchiveCommand.call
Whether the output stream specified with setOutputStream() is closed by
ArchiveCommand.call() is murky and inconsistent:
- on success, it is closed
- if an exception is encountered when writing the archive, it is closed
- if an exception is encountered when calling createArchiveStream to
open the archive, we forget to close it
Close the output stream consistently to avoid leaks.
Now that the inner try-with-resources doesn't have its own finally
block, this allows us to merge the two try blocks.
It would be even better to never close the output stream. That will
involve more API changes to avoid silently breaking callers, so it is
deferred to a later change.
Change-Id: I0185bdaa60ecee4a541eab5d8ff6c9c4dbe40bf1 Signed-off-by: Jonathan Nieder <jrn@google.com>
Fix that exceptions in ReceivePack cause Invalid Channel 101 exceptions
When during a PushOperation the server hits an exception different from
UnpackException the JGit server behaved wrong. That kind of exceptions
are handled so late that the connection is already released and the
information whether to talk sideband to the client is lost. In detail:
ReceivePack.receive() will call release() and that will reset the
capabilities. But later on the stack in ReceivePackServlet.doPost() it
is tried to send a response to client now with reset capabilities (no
sideband!).
Change-Id: I0a609acc6152ab43b47a93d712deb65bb1105f75 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Better report too large pack-files from PushCommand
JGits PushCommand and BasePackPushConnection were throwing generic
TransportExceptions when the pushed pack-file was rejected by the server
since it was too big. Let JGit better interprete the server's response
to detect this situation and throw a more specific exception.
This detection works by parsing the status line sent by the server. This
change only recognizes the response sent by a JGit based server. All
other servers which report such problems in a different way still lead
to a generic TransportExceptions.
Change-Id: Ic075764ea152939ce72c446252464620dd54edea Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Dave Borowitz [Wed, 27 May 2015 16:40:32 +0000 (09:40 -0700)]
FS: Extract GobblerThread into a private static class
The primary goal is to improve exception readability. Since this is a
standalone thread, just logging the stack trace of the caught
exception is not very useful:
java.io.IOException: Stream closed
at java.io.BufferedInputStream.getBufIfOpen(BufferedInputStream.java:162)
at java.io.BufferedInputStream.read(BufferedInputStream.java:258)
at org.eclipse.jgit.util.FS$2.run(FS.java:451)
Providing a named class eliminates the "FS$2", and including the
command name provides a little more context in the error message.
A future improvement might include the stack trace that created the
GobblerThread as well.
Terry Parker [Tue, 12 May 2015 00:37:02 +0000 (17:37 -0700)]
Add bitmap index misses to PackWriter.Statistics
RevWalks to find commits that are not in bitmap indices are expensive.
Track the count of commits that are enumerated via RevWalks as "bitmap
index misses" in the PackWriter.Statistics class.
Change-Id: Ie0135a0a0aeba2dfb6df78839d545006629f16cb Signed-off-by: Terry Parker <tparker@google.com>
David Pletcher [Mon, 25 May 2015 21:49:58 +0000 (14:49 -0700)]
Enable public access to SimilarityIndex scoring function
The SimilarityIndex class implements the useful capability of scoring
the similarity between two files. That capability is required for a
feature that's being developed in another package, to detect files
derived from a set of potential sources.
This CL adds a public factory method to create a SimilarityIndex from
an ObjectLoader. It grants public access to the SimilarityIndex class,
the score method, an inner exception class and a special marker
instance of that exception class.
Change-Id: I3f72670da643be3bb8e261c5af5e9664bcd0401b Signed-off-by: David Pletcher <dpletcher@google.com>
projects / jgit.git / log