Hoist ObjectIdSet up to lib as part of the public API and add
the interface to some common types like PackIndex and JGit custom
ObjectId map types. This cleans up wrapper code in a number of
places by allowing direct use of the types as an ObjectIdSet.
Future commits can now rely on ObjectIdSet as a simple read-only
type to check a set of objects from a number of storage options.
Change-Id: Ib62b062421d475bd52abd6c84a73916ef36e084b
Neaten up formatting and avoid strings, which prevents the need for
NLS comment tags. Instead check the last character using char
literal, and append a char literal instead of a string.
Change-Id: Ib68e017769a1f5c03200354a805769d585a48c8b
Similar to nested directories, nested copyfiles won't work with git submodule
either.
Change-Id: Idbe965ec20a682fca0432802858162f8238f05de
Signed-off-by: Yuxuan 'fishy' Wang <fishywang@google.com>
Checkout should be able to override modified symbolic links
Handle existing symlink as a file, not as directory if deleting a file
before creating (overriding) a symlink.
Bug: 484491
Change-Id: I29dbf57d1daec2ba98454975b093e1d381d05196
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
Keep a user amused while the server does work by spinning a
little ASCII-art object on a single line.
Change-Id: Ie8f181d1aa606d4ae69e5d3ca4db387cea739f38
Previously, non-reuse deltas were only included in packStatistics if they
were not cached by the deltaWindow.
Change-Id: I7684d8214875f0a7569b34614f8a3ba341dbde9c
Signed-off-by: James Kolb <jkolb@google.com>
When setting timeout on push, BasePackConnection creates a timer, which
will be terminated when push finishes. But, when using
SmartHttpPushConnection, it dropped the first timer created in the
constructor and then created another timer in doPush. If new threads are
created faster than the gc collects then this may stop the service if
it's hitting the max process limit. Hence don't create a new timer if it
already exists.
Bug: 474947
Change-Id: I6746ffe4584ad919369afd5bdbba66fe736be314
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Do not let PathFilter.create("a/b") match 'a' unless 'a' is a subtree
PathFilter and PathFilterGroup form JGit's implementation of git's
path-limiting feature in commands like log and diff. To save time
when traversing trees, a path specification
foo/bar/baz
tells the tree walker not to traverse unrelated trees like qux/. It
does that by returning false from include when the tree walker is
visiting qux and true when it is visiting foo.
Unfortunately that test was implemented to be slightly over-eager: it
doesn't only return true when asked whether to visit a subtree "foo"
but when asked about a plain file "foo" as well. As a result, diffs
and logs restricted to some-file/non-existing-suffix unexpectedly
match against some-file:
$ jgit log -- LICENSE/no-such-file
commit 629fd0d594
Author: Shawn O. Pearce <spearce@spearce.org>
Date: Fri Jul 02 14:52:49 2010 -0700
Clean up LICENSE file
[...]
Fix it by checking against the entry's mode.
Gitiles +log has the same bug and benefits from the same fix.
Callers know not to worry about what subtrees are included in the tree
walk because shouldBeRecursive() returns true in this case, so this
behavior change should be safe. This also better matches the behavior
of C git:
$ empty=$(git mktree </dev/null)
$ git diff-tree --abbrev $empty HEAD -- LICENSE/no-such-file
$ git diff-tree --abbrev $empty HEAD -- tools/no-such-file
:000000 040000 0000000... b62648d... A tools
Bug: 484266
Change-Id: Ib4d53bddd8413a9548622c7b25b338d287d8889d
Fix push with jgit pgm failing with "unauthorized"
Pushing with JGit commandline to e.g. Github failed with "unauthorized"
since HttpUrlConnection calls the configured authenticator implicitly.
The problem is that during a push two requests are sent to the server,
first a GET and then a POST (containing the pack data). The first GET
request sent anonymously is rejected with 401 (unauthorized). When an
Authenticator is installed the java.net classes will use the
Authenticator to ask the user for credentials and retry the request.
But this happens under the hood and JGit level code doesn't see that
this happens.
The next request is the POST but since JGit thinks the first GET request
went through anonymously it doesn't add authentication headers to the
POST request. This POST of course also fails with 401 but since this
request contains a lot of body-data streamed from JGit (the pack file!)
the java.net classes can't simply retry the request with authorization
headers. The whole process fails.
Fix this by using Apache httpclient which doesn't use Authenticator to
retrieve credentials. Instead initialize TransportCommand to use the
default credential provider if no other credentials provider was set
explicitly. org.eclipse.jgit.pgm.Main sets this default for the JGit
command line client.
Change-Id: Ic4e0f8b60d4bd6e69d91eae0c7e1b44cdf851b00
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Fix possible arithmetic overflow when setting a timeout
BasePackPushConnection#readStringLongTimeout() was setting a timeout 10
times bigger than some other timeout or the pack transfer time. This
could lead to negative integer values when we hit an arithmetic
overflow. Add a check for this situation and set the timeout to
Integer.MAX_VALUE when overflow happens.
Bug: 484352
CC: Eugene Petrenko <eugene.petrenko@gmail.com>
Change-Id: Ie2a86312c1bcb1ec3e6388fa490ab3c845d41808
Null-annotated Ref class and fixed related compiler errors
This change fixes all compiler errors in JGit and replaces possible
NPE's with either appropriate exceptions, avoiding multiple "Nullable
return" method calls or early returning from the method.
Change-Id: I24c8a600ec962d61d5f40abf73eac4203e115240
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
If an application uses PushConnection directly on the native Git wire
protocols JGit should send along the application's expected oldId, not
the advertised value. This allows the remote peer to compare-and-swap
since it was not tested inside JGit.
Discovered when I tried to use a PushConnection (bypassing the
standard PushProcess) and the client blindly overwrote the remote
reference, even though my app had supplied the wrong ObjectId for
the expectedOldObjectId. This was not expected and cost me over an
hour of debugging, plus "corruption" in the remote repository.
By passing along the exact expectedOldObjectId from the app the
remote side can do the check that the application skipped, and
avoid data loss.
Change-Id: Id3920837e6c47100376225bb4dd61fa3e88c64db
FileTreeIterator was calling by mistake
WorkingTreeIterator.idSubmodule(Entry). Instead it should always compute
idSubmodule on its own.
Change-Id: Id1b988aded06939b1d7edd2671e34bf756896c0e
In I1f5dc07182dbf6bba2a9f4807fdd25b475da4ead, FileBasedConfig got
support for reading a configuration with UTF8 BOM. Apply the same
support to BlobBasedConfig, to make SubmoduleWalk able to parse
.gitmodules configurations with BOM.
Change-Id: I25b5474779952fe2c076180b96fc2869eef190a8
Signed-off-by: Doug Kelly <dougk.ff7@gmail.com>
The ChainingCredentialsProvider gave up chaining to the next provider if
the first one returned no credentials items for the given URI.
Change-Id: I9539c50db35e564db9d43d8ebb71d7e9c6fdcc19
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
JGit's Nullable type was added[1] in the hope of being able to add
nullness annotations that (a) do not preclude building and running
with Java 7 and (b) could be shared by Gerrit, which uses a custom
Nullable type for other reasons[2]. Sharing a type is useful because
Eclipse's null analysis is only able to use one Nullable type at a
time in a given workspace (so for this analysis to function in a
workspace used to develop Gerrit, JGit and Gerrit would need to use
the same Nullable type).
The new Nullable type has CLASS instead of RUNTIME retention because
there wasn't any obvious use for the annotation at run time.
Gerrit uses the Nullable annotation to communicate with Guice. Guice
injection happens at runtime, so it needs to be able to read the
@Nullable annotations at run time[3]. Otherwise Guice produces
provisioning errors, such as
3) null returned by binding at com.google.gerrit.lucene.LuceneChangeIndex$Factory.create()
but parameter 7 of com.google.gerrit.lucene.LuceneChangeIndex.<init>() is not @Nullable
Switch to RUNTIME retention to avoid this.
While at it, update the javadoc to explain more clearly how this
annotation relates to other Nullable types[4]. This should make it
clearer why JGit needed another Nullable type:
A. Avoiding dependency on Java 8
B. RUNTIME retention to allow Guice to read the annotation at run time
C. Named Nullable so Guice can recognize the annotation
D. Not an addition to Java EE's javax.annotation package, to avoid
the split-package problem[2] that prevents the annotation from
being readable at run time when loaded from an OSGi container
E. Avoiding heavyweight dependencies, deprecated dependencies, and
dependencies on package internals
org.checkerframework.checker.nullness.qual.Nullable: A
com.sun.istack.internal.Nullable: B, E
*.CheckForNull, *.NullAllowed, etc: C
edu.umd.cs.findbugs.annotations.Nullable: B, E
javax.annotation.Nullable: D
org.eclipse.jdt.annotation.Nullable: B
org.jetbrains.annotations.Nullable: B
org.jmlspecs.annotation.Nullable: E
android.annotation.Nullable, android.support.annotation.Nullable: E
[1] https://git.eclipse.org/r/59993
[2] https://gerrit-review.googlesource.com/50112
[3] https://github.com/google/guice/blob/master/core/src/com/google/inject/internal/Nullability.java
[4] https://github.com/typetools/checker-framework/blob/5832a01f1/checker/src/org/checkerframework/checker/nullness/NullnessAnnotatedTypeFactory.java#L118http://types.cs.washington.edu/checker-framework/current/checker-framework-manual.html#nullness-related-work
Change-Id: I6c482653d2b53e3509abb11211b67fc29cf2949c
Signed-off-by: Jonathan Nieder <jrn@google.com>
If the password char array is null constructing a new String from this
array fails with a NPE. Add a null check to fix this.
Change-Id: Ifae6eecca38d5f114861f44658a32521e6e96866
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
This should mirror the behavior of `git push --atomic` where the
client asks the server to apply all-or-nothing. Some JGit servers
already support this based on a custom DFS backend. InMemoryRepository
is extended to support atomic push for unit testing purposes.
Local disk server side support inside of JGit is a more complex animal
due to the excessive amount of file locking required to protect every
reference as a loose reference.
Change-Id: I15083fbe48447678e034afeffb4639572a32f50c
PreReceiveHook is given a Collection<ReceiveCommand> and it can be
very useful here to call ReceiveCommand.filter(cmds, NOT_ATTEMPTED).
Overload filter to accept both Iterable and List.
Keep backwards binary compatibility for List by upcasting to Iterable.
Change-Id: Ib1341876c703670945ef209edc8259715ee86c26
Recursively copying a tree into a DirCache is a bottleneck for some
algorithms like the in memory merge code in Gerrit Code Review. Drop
a layer down in the stack and use CanonicalTreeParser directly as the
addition logic only processes 1 tree at a time and does not need the
merge sorting feature (or overhead) of TreeWalk.
Combined with 761814fe9c ("DirCacheEntry: Speed up creation by
avoiding string cast") tree loading 38,900 entries nearly halves
in running time from 70ms to 36ms on some platforms.
Change-Id: If1490ca25de0679a71cf508f59b486f9cc816165
Application code sometimes wants to read a DirCache from an ObjectId,
but its confusing how to do this because its buried inside the
DirCacheBuilder.
Use this utility in a few places within JGit that also want to read
a DirCache from a tree's ObjectId.
Change-Id: I578b7e18e58753d154937f4ab835012b09e5adca
Instead of checking every entry for .gitattributes only look for the
entry on request by TreeWalk. This avoids impacting uses like RevWalk
filtering history.
When the attrs is requested skip to the start of the tree and look for
.gitattributes until either it is found, or it is impossible to be
present. Due to the sorting rules of tree entries .gitattributes
should be among the first or second entries in the tree so very few
entries will need to be considered.
Waiting to find the .gitattributes file by native ordering may miss
attrs for files like .config, which sorts before .gitattributes.
Starting from the front of the tree on demand ensures the attributes
are parsed as early as necessary to process any entry in the tree.
Due to TreeWalk recursively processing up the tree of iterators we
cannot just reset the current CanonicalTreeParser to the start as
parent parsers share the same path buffer as their children.
Resetting a parent to look for .gitattributes may overwrite path
buffer data used by a child iterator. Work around this by building a
new temporary CanonicalTreeParser instance.
Change-Id: Ife950253b687be325340d27e9915c9a40df2641c
Line breaking before , is ugly to read. Most formatters and humans
expect line break after , so update a few offensive locations.
Use String.format() for the construction of the error message when
a bad DirCachEntry is being failed on. This simplifies the code and
its not a performance critical section.
Change-Id: I5d990389e7ba24ef0861cf8ec0026ed030d4aeda
DirCacheEntry: Speed up creation by avoiding string cast
The checkPath function is available as a byte[] form, in fact the
String form just converts to byte[] to run the algorithm.
Having DirCacheEntry take a byte[] -> String -> byte[] to check if
each path is valid is a huge waste of CPU time. On some systems it
can double the time required to read 38,999 files from trees to the
DirCache. This slows down any operation using a DirCache.
Expose the byte[] form and use it for DirCacheEntry creation.
Change-Id: I6db7bc793ece99ff3c356338d793c07c061aeac7
DirCache: Fix getEntriesWithin("") to not include null entries
The internal array may be longer than entryCnt, in this case the tail
of the array is padded with null entries. Do not return those to the
caller of getEntriesWithin().
Change-Id: I19efb05e103fab6b739ced407f6e28155a48dba6
If defined in .gitattributes call smudge filter during checkout.
To support checkout where current HEAD,index do not contain attributes
we need to also consider attributes from the tree we checkout. Therefore
CanonicalTreeParser has to learn how to provide attributes.
Change-Id: I168fdb81a8e1a9f991587b3e95a36550ea845f0a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
When filters are defined for certain paths in gitattributes make
sure that clean filters are processed when adding new content to the
object database.
Change-Id: Iffd72914cec5b434ba4d0de232e285b7492db868
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Add the new class Attributes holding multiple Attribute(s)
Attributes represents a semantic collector of Attribute(s) and replaces
the anonymous Map<String,Attribute>. This class will be returned by
TreeWalk.getAttributes(). It offers convenient access to the attributes
wrapped in the Attributes object. Adds preparations for a future
Attribute Macro Expansion
Change-Id: I8348c8c457a2a7f1f0c48050e10399b0fa1cdbe1
Signed-off-by: Ivan Motsch <ivan.motsch@bsiag.com>
When FS.runProcess was called and an InputStream was given the method
tried to pump the whole InputStream to the process. When the method
ended the InputStream was not giving any data anymore. Consequently
close the InputStream inside the method.
Change-Id: I0ed738a775e5c977b21447d195acee1ecf5e2cb9
repo: Do not use search path to find refs/remotes/origin/<branch>
When running from a non-bare repository, "jgit repo" checks whether
the rev passed in is a sha1 or branch name and in the latter case will
check out origin/<branch>.
We are expecting refs/remotes/origin/<branch>, but as a side effect of
using getRef we also end up looking for refs/origin/<branch>,
refs/heads/origin/<branch>, and so on. Avoid that by using exactRef
instead.
Signed-off-by: Jonathan Nieder <jrn@google.com>
Change-Id: I670b2e48a88138a1f2104ea201baa958e9edbddb
Repository: Introduce exactRef and findRef, deprecate getRef
The Repository class provides only one method to look up a ref by
name, getRef. If I request refs/heads/master and that ref does not
exist, getRef will look further in the search path:
ref/refs/heads/master
refs/heads/refs/heads/master
refs/remotes/refs/heads/master
This behavior is counterintuitive, needlessly inexpensive, and usually
not what the caller expects.
Allow callers to specify whether to use the search path by providing
two separate methods:
- exactRef, which looks up a ref when its exact name is known
- findRef, which looks for a ref along the search path
For backward compatibility, keep getRef as a deprecated synonym for
findRef.
This change introduces findRef and exactRef but does not update
callers outside tests to use them yet.
Change-Id: I35375d942baeb3ded15520388f8ebb9c0cc86f8c
Signed-off-by: Jonathan Nieder <jrn@google.com>
Null-annotated Repository class and fixed related compiler errors
org.eclipse.jgit.lib.Repository class is an example of the API which
should be written with Java 8 java.util.Optional<T> type. Unfortunately
this API is already released and widely used. The good clients are
currently doing their best with checking return values for null and bad
clients do not know how bad their code is.
I've tried not to change any logic and to be as less intrusive as
possible. Most of the JGit code was well prepared to this, only few
classes needed some smaller fixes.
This change fixes all compiler errors in JGit and replaces possible
NPE's with either appropriate exceptions, avoiding multiple "Nullable
return" method calls or early returning from the method.
Because annotating getDirectory() and getFS() as Nullable would cause
lot of additional changes in JGit and EGit they are postponed.
Change-Id: Ie8369d2c9c5fac5ce83b3b1b9bc217d7b55502a3
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
Raise error if FileNotFoundException is caught for an existing file
File, FileInputStream and friends may throw FileNotFoundException even
if the file is existing e.g. when file permissions don't allow to access
the file content. In most cases this is a severe error we should not
suppress hence rethrow the FileNotFoundException in this case.
This may also fix bug 451508.
Bug: 451508
Change-Id: If4a94217fb5b7cfd4c04d881902f3e86193c7008
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Let FS_Win32_Cygwin detect symlink support by creating temporary symlink
The class FS_Win32 was always trying out to create a temporary symlink
in order to find out whether symlinks are supported. FS_Win32_Cygwin was
overwriting this method and always returned true. But when the user
running JGit does not have administrative rights then the creation of
symlinks is forbidden even if he is running on FS_Win32_Cygwin. A lot of
tests failed only on the Windows platform because of this. It was
correctly detected that FS_Win32_Cygwin is the filesystem abstraction to
be used but creation of symlinks always failed because of lacking
privileges of the user running the tests.
This fix teaches FS_Win32_Cygwin to behave like FS_Win32 and to test
whether symlinks can be created in order to find out whether symlinks
are supported.
Change-Id: Ie2394631ffc4c489bd37c3ec142ed44bbfcac726
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
RefDirectory.getRef: Treat fake missing symrefs like real ones
getRef() loops over its search path to find a ref:
Ref ref = null;
for (String prefix : SEARCH_PATH) {
ref = readRef(prefix + needle, packed);
if (ref != null) {
ref = resolve(ref, 0, null, null, packed);
break;
}
}
fireRefsChanged();
return ref;
If readRef returns null (indicating that the ref does not exist), the
loop continues so we can find the ref later in the search path. And
resolve should never return null, so if we return null it should mean
we exhausted the entire search path and didn't find the ref.
... except that resolve can return null: it does so when it has
followed too many symrefs and concluded that there is a symref loop:
if (MAX_SYMBOLIC_REF_DEPTH <= depth)
return null; // claim it doesn't exist
Continue the loop instead of returning null immediately. This makes
the behavior more consistent.
Arguably getRef should throw an exception when a symref loop is
detected. That would be a more invasive change, so if it's a good
idea it will have to wait for another patch.
Change-Id: Icb1c7fafd4f1e34c9b43538e27ab5bbc17ad9eef
Signed-off-by: Jonathan Nieder <jrn@google.com>
Throw IndexReadException if existing index can't be read
If the index file exists but can't be read for example because of wrong
filesystem permissions we should throw a specific exception. This allows
EGit to handle this error situation.
Bug: 482607
Change-Id: I50bfcb719c45caac3cb5550a8b16307c2ea9def4
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Fix pre-push hook to not set null remoteName as first argument
According to [1] the pre-push hook expects two parameters which provide
the name and location of the destination remote, if a named remote is
not being used both values should be the same.
We did set the first parameter to null in that case which caused
ProcessBuilder to throw a NullPointerException since its start() method
doesn't accept null arguments.
[1] https://git-scm.com/docs/githooks#_pre_push
Bug: 482393
Change-Id: Idb9b0a48cefac01abfcfdf00f6d173f8fa1d9a7b
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Shawn Pearce
Shawn Pearce
Dmitry Neverov
Yuxuan 'fishy' Wang
Andrey Loskutov
James Kolb
Matthias Sohn
Jonathan Nieder
Mike Gilbode
Christian Halstrick
Kaloyan Raev
Doug Kelly
Ivan Motsch