This would run into an endless loop if the offset given was not zero.
Fix the logic to exit the read loop when the buffer is full.

Luckily all existing uses of this method call it only with offset zero.

Change-Id: I0ec2a4fb43efe4a605d06ac2e88cf155d50e2f1e
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>

Reachability checkers are retrieved from RevWalk and ObjectWalk objects:
* RevWalk.createReachabilityChecker()
* ObjectWalk.createObjectReachabilityChecker()

Since RevWalks and ObjectWalks are themselves directly instantiated
in hundreds of places (e.g. UploadPack...) overriding them in a
consistent way requires overloading 100s of methods, which isn't
feasible. Moving reachability checker generation to a more central
place solves that problem.

The ObjectReader object seems a good place from which to get
reachability checkers, because reachability checkers return
information about relationships between objects. ObjectDatabases
delegate many operations to ObjectReaders, and reachability bitmaps
are attached to ObjectReaders.

The Bitmapped and Pedestrian reachability checker objects were
package private in the org.eclipse.jgit.revwalk package. This change
makes them public and moves them to the
org.eclipse.jgit.internal.revwalk package. Corresponding tests are
also moved.

Motivation:
1) Reachability checking algorithms need to scale. One of the
   internal Android repositories has ~2.4 million refs/changes/*
   references, causing bad long tail performance in reachability
   checks.
2) Reachability check performance is impacted by repository
   topography: number of refs, number of objects, amounts of
   related vs. unrelated history.
3) Reachability check performance is also affected by per-branch
   access (Gerrit branch permissions) since different users can
   see different branches.
4) Reachability check performance isn't affected by any state in a
   RevWalk or ObjectWalk.

I don't yet know if a single algorithm will work for all cases in #2
and #3. We may need to evolve the ReachabilityChecker interfaces
over time to solve the Gerrit branch permissions case, or use
Gerrit-specific identity information to solve that in an efficient
way.

This change takes the existing public API and moves it to the
ObjectReader/whole repository level, which is where we can do
consistent customizations for #2 and #3. We intend to upstream the
best of whatever works, but anticipate the need for multiple rounds
of experimentation.

Change-Id: I9185feff43551fb387957c436112d5250486833d
Signed-off-by: Terry Parker <tparker@google.com>

Different archive formats support a compression level in the range
[0-9]. The value 0 is for lowest compressions and 9 for highest. Highest
levels produce output files of smaller sizes but require more memory to
do the compression.

This change allows passing a "compression-level" option to the git
archive command and implements using it for different file formats.

Change-Id: I5758f691c37ba630dbac24db67bb7da827bbc8e1
Signed-off-by: Youssef Elghareeb <ghareeb@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

We sometimes want to get all the refs except specific prefixes,
similarly to getRefsByPrefix that gets all the refs of a specific
prefix.

We now create a new method that gets all refs matching a prefix except a
set of specific prefixes.

One use-case is for Gerrit to be able to get all the refs except
refs/changes; in Gerrit we often have lots of refs/changes, but very
little other refs. Currently, to get all the refs except refs/changes we
need to get all the refs and then filter the refs/changes, which is very
inefficient. With this method, we can simply skip the unneeded prefix so
that we don't have to go over all the elements.

RefDirectory still uses the inefficient implementation, since there
isn't a simple way to use Refcursor to achieve the efficient
implementation (as done in ReftableDatabase).

Signed-off-by: Gal Paikin <paiking@google.com>
Change-Id: I8c5db581acdeb6698e3d3a2abde8da32f70c854c

This method will be used by the follow-up change. This useful if we want
to go over all the changes after a specific ref.

For example, the new method allows us to create a follow-up that would
go over all the refs until we reach a specific ref (e.g refs/changes/),
and then we use seekPastPrefix(refs/changes/) to read the rest of the refs,
thus basically we return all refs except a specific prefix.

When seeking past a prefix, the previous condition that created the
RefCursor still applies. E.g, if the cursor was created by
seekRefsWithPrefix, we can skip some refs but we will not return refs
that are not starting with this prefix.

Signed-off-by: Gal Paikin <paiking@google.com>
Change-Id: I2c02e89c877fe90da8619cb8a4a9a0c865f238ef

Heap always copied whole blocks, which leads to AIOOBEs. LocalFile
didn't overwrite the method and thus caused NPEs.

Change-Id: Ia37d4a875df9f25d4825e6bc95fed7f0dff42afb
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>

Signatures on tags are just tacked onto the end of the message.
Getting the message must not return the signature. Compare [1]
and [2] in C git, which both drop a signature at the end of an
object body.

[1] https://github.com/git/git/blob/21bf933/builtin/tag.c#L173
[2] https://github.com/git/git/blob/21bf933/ref-filter.c#L1276

Change-Id: Ic8a1062b8bc77f2d7c138c3fe8a7fd13b1253f38
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>

Make all transports request protocol V2 when fetching. Depending on
the transport, set the GIT_PROTOCOL environment variable (file and
ssh), pass the Git-Protocol header (http), or set the hidden
"\0version=2\0" (git anon). We'll fall back to V0 if the server
doesn't reply with a version 2 answer.

A user can control which protocol the client requests via the git
config protocol.version; if not set, JGit requests protocol V2 for
fetching. Pushing always uses protocol V0 still.

In the API, there is only a new Transport.openFetch() version that
takes a collection of RefSpecs plus additional patterns to construct
the Ref prefixes for the "ls-refs" command in protocol V2. If none
are given, the server will still advertise all refs, even in protocol
V2.

BasePackConnection.readAdvertisedRefs() handles falling back to
protocol V0. It newly returns true if V0 was used and the advertised
refs were read, and false if V2 is used and an explicit "ls-refs" is
needed. (This can't be done transparently inside readAdvertisedRefs()
because a "stateless RPC" transport like TransportHttp may need to
open a new connection for writing.)

BasePackFetchConnection implements the changes needed for the protocol
V2 "fetch" command (stateless protocol, simplified ACK handling,
delimiters, section headers).

In TransportHttp, change readSmartHeaders() to also recognize the
"version 2" packet line as a valid smart server indication.

Adapt tests, and run all the HTTP tests not only with both HTTP
connection factories (JDK and Apache HttpClient) but also with both
protocol V0 and V2. The SSH tests are much slower and much more
focused on the SSH protocol and SSH key handling. Factor out two
very simple cloning and pulling tests and make those run with
protocol V2.

Bug: 553083
Change-Id: I357c7f5daa7efb2872f1c64ee6f6d54229031ae1
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>

If a file exists in head, merge, and the working tree, but not in
the index, and we're doing a force checkout, the checkout must be
an "update", not a "keep".

This is a follow-up on If3a9b9e60064459d187c7db04eb4471a72c6cece.

Bug: 569962
Change-Id: I59a7ac41898ddc1dd90e86b09b621a41fdf45667
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>

Path#getFileName can return null. Fix the warning by asserting the file
name isn't null.

Change-Id: I7f2fe75b46113d8be1d14e3f18dd77da27df25ed
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

File#listFiles can return null. Use Files#list instead to fix the
problem.

Change-Id: I74e0b49aa6dae370219507c64aa43be4d8aa7b82
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

Path#getParent can return null. Fix the warning by implementing a helper
method which asserts the parent is not null.

Change-Id: Ib4f8dff0674b74bc891f15f08bd9755c5ea728dc
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

File#listFiles can return null, assert it is not null to fix the
warning.

Change-Id: I28fc668fee760d39965e6e039003ac9f85fd461b
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

This was experimental code and never used in production.

Change-Id: Ia3da7f2b82d9e365cec2ccf9397cbc47439cd150
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

Some clients may wish to allow NO_CHANGE lightweight tag updates
without setting the force flag. (For instance EGit does so.)
Command-line git does not allow this.

Propagate the RefUpdate result via the RefAlreadyExistsException.
That way a client has the possibility to catch it and check the
failure reason without having to parse the exception message, and
take appropriate action, like ignoring the exception on NO_CHANGE.

Change-Id: I60e7a15a3c309db4106cab87847a19b6d24866f6
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>

JGit treated a NO_CHANGE RefUpdate as an error in all cases. But when
updating a lightweight tag, this is a successful result if -f was
specified.

Change-Id: Iddfa6d6a6dc8bf8fed81138a008ebc32d5f960bd
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>

Add the two config constants from C git that can switch on signing
of annotated tags. Add them to the GpgConfig, and implement actually
signing a tag in TagCommand.

The interactions between command line options for "git tag" and config
options is a bit murky in C git. There are two config settings for it:

* tag.gpgSign is the main option, if set to true, it kicks in if
  neither -s nor -u are given on the command line.
* tag.forceSignAnnotated signs only tags created via "git tag -m",
  but only if command-line option "-a" is not present. It applies
  even if tag.gpgSign is set explicitly to false.

Giving -s or -u on the command line also forces an annotated tag
since lightweight tags cannot be signed.

Bug: 386908
Change-Id: Ic8a1a44b5f12f47d5cdf3aae2456c1f6ca9ef057
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>

Factor out a common ObjectBuilder as super class of CommitBuilder
and TagBuilder, and make the GpgSigner work on ObjectBuilder.

In order not to break API, add the new method for signing an
ObjectBuilder in a new interface GpgObjectSigner.

The signature for a tag is just tacked onto the end of the tag
message. The message of a signed tag must end in LF.

Bug: 386908
Change-Id: I5e021e3c927f4051825cd7355b129113b949455e
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>

Observed the error when trying to force checkout from a branch
that had no changes on it. When the 'keep()' method from 'DirCacheCheckout'
method was called the 'DirCacheEntry e' was null and was passed like
this to the 'checkoutEntry()' method where the 'getObjectId()' is
being called on the 'e' object

Change-Id: If3a9b9e60064459d187c7db04eb4471a72c6cece

Change-Id: Icd6b876e392c55a7322a990ac7f618250b732b4b
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

Explicitly test that END and DELIM can be distinguished. If not, the
wire protocol V2 breaks down.

Bug: 568950
Change-Id: I5f3496168244303c68893f1c756831dd27440aeb
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>

This would allow other JGit users to access and reuse the constants.

Change-Id: I1608802f45586af5f8582afa592e26679e9cebe3
Signed-off-by: David Ostrovsky <david@ostrovsky.org>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

Change-Id: Id78f4fd7d11a8908f41160a2ab9f3d81b7f771da
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

A SymbolicRef is added to the advertised refs for any symref in
capabilities whose target is an advertised ref; this may replace an
existing entry, such as HEAD.

When cloning, if any advertised HEAD is symbolic then use the target
rather than looking for an advertised ref with a matching objectId.

Add --symref option to LsRemote command.

Bug: 514052
Change-Id: Idfb48e6f6e8dcfe57a6896883fe6d84d533aa9d0
Signed-off-by: Lee Worrall <worrall.la@gmail.com>

If pack or index files are guarded by a pack lock (.keep file)
deleteOrphans() should not touch the respective files protected by the
lock file. Otherwise it may interfere with PackInserter concurrently
inserting a new pack file and its index.

The problem was caused by the following race.

All mentioned files are located in  "objects/pack/".
File endings relevant in "pack" dir:
  .pack
  .keep
  .idx
  .bitmap

When ReceivePack receives a pack file it executes the following steps:

ReceivePack.service():
  receivePackAndCheckConnectivity():
    receivePack():
      receive the pack
      parse the pack, returns packLock (.keep file)
      PackInserter.flush():
        write tmpPck file:  "insert_<random>.pack"
        write tmpIdx file:  "insert_<random>.idx"
        real pack name: "pack-<SHA1>.pack"
        real index name: "pack-<SHA1>.idx"
        atomic rename tmpPack to realPack
        atomic rename tmpIdx to tmpIdx
  execute commands
  unlock pack by removing .keep file
  trigger auto gc if enabled

When PackInserter.flush() renames the temporary pack to the final
"pack-xxx.pack" file  the temporary pack index file "insert_xxx.idx"
has no matching .pack file with the same base name for a short interval.
If deleteOrphans() ran during that interval it deduced the pack index
file was orphaned. Subsequently the missing pack index caused
MissingObjectExceptions since objects contained in the pack couldn't be
looked up anymore.

Bug: https://bugs.chromium.org/p/gerrit/issues/detail?id=13544
Change-Id: I559c81e4b1d7c487f92a751bd78b987d32c98719
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

DirCacheEditor unconditionally applied a PathEdit to all stages in the
index. This gives wrong results if one wants to check out a file from
some commit to resolve a conflict: JGit would update the working tree
file multiple times (once per stage), and set all stages to point to
the checked-out blob.

C git replaces the stages by the entry for the checked-out file.

To support this, add a DirCacheEntry.setStage() method so that
CheckoutCommand can force the stage to zero. In DirCacheEditor, keep
only the zero stage if the PathEdit re-set the stage.

Bug: 568038
Change-Id: Ic7c635bb5aaa06ffaaeed50bc5e45702c56fc6d1
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>

PerformanceLogContext threw NullPointerException when multiple threads
tried to add an event to the PerformanceLogContext. The cause for this
is that the ThreadLocal was initialized only in the class constructor
for the first thread; for subsequent threads it was null.

To fix this initialize eventRecords for each thread.

Change-Id: I18ef67dff8f0488e3ad28c9bbc18ce73d5168cf9
Signed-off-by: Alexa Panfil <alexapizza@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

Fix IOException occurring when calling
GC on a repository with absent objects/pack folder.

Change-Id: I5be1333a0726f4d7491afd25ddac85451686c30a
Signed-off-by: Nail Samatov <sanail@yandex.ru>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

This problem occurred when calling SubmoduleWalk#getModuleName if the
first submodule processed has a name and a path which do not match
SubmoduleWalk#getModuleName returned the module path instead of the
module name. In order to fix this SubmoduleWalk#getModuleName needs to
ensure that the modules config is loaded.

Bug: 565776
Change-Id: I36ce1fbc64c4849f9d8e39864b825c6e28d344f8
Signed-off-by: John Dallaway <john@dallaway.org.uk>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

Add new performance logging to register events of type duration.
The proposed logging is similar to the performance logging
in OS Gerrit https://gerrit-review.googlesource.com/c/gerrit/+/225628:
a global Singleton (LoggingContext in Gerrit) is
collecting the performance logs in a thread-safe events list,
and at the end of the monitored command the list of events is
retrieved and written to a log, after which it is cleared.

What this patch does:
The main component is the Singleton (PerformanceLogContext), which
is used to collect the records (PerformanceLogRecord) in one place
(ThreadLocal eventsList) from anywhere using
PerformanceLogContext.getInstance().addEvent().

Reason why this change is needed:
The current monitoring in JGit has several issues:
1. git fetch and git push events are handled separately
(PackStatistics and ReceivedPackStatistics), with no unified way
of writing or reading the statistics.
2. PostUploadHook is only invoked on the event of sending the
pack, which means that the PackStatistics is not available for
the fetch requests that did not end with sending the pack
(negotiation requests).
3. The way the logs are created is different from the performance
log approach, so the long-running operations need to be collected
from both performance log (for JGit DFS overridden operations and
Gerrit operations) and gitlog (for JGit ones).

The proposed performance logging is going to solve the above
mentioned issues: it collects all of the performance logs in one
place, thus accounting for the commands that do not result in
sending a pack. The logs are compatible with the ones on Gerrit.
Moreover, the Singleton is accessible anywhere in the call stack,
which proved to be successful in other projects like Dapper
(https://research.google/pubs/pub36356/).

Signed-off-by: Alexa Panfil <alexapizza@google.com>
Change-Id: Iabfe667a3412d8a9db94aabb0f39b57f43469c41

This enables jgit to use any refs in the refs/ namespace when describing
commits.

Signed-off-by: Jason Yeo <jasonyeo88@gmail.com>
Change-Id: I1fa22d1c39c0e2f5e4c2938c9751d8556494ac26

According to [1], environment variable GIT_HTTP_USER_AGENT can
override a git config http.userAgent.

[1] https://git-scm.com/docs/git-config#Documentation/git-config.txt-httpuserAgent

Change-Id: I996789dc49faf96339cd7b4e0a682b9bcafb6f70
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>

This fixes a resource leak warning.

Change-Id: I65166eeefc719ea577963c84ce2059e2d7e6c1d7
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

Validate the extra headers and log but otherwise ignore invalid
headers. An empty http.extraHeader starts the list afresh.

The http.userAgent is restricted to printable 7-bit ASCII, other
characters are replaced by '.'.

Moves a support method from the ssh.apache bundle to HttpSupport in
the main JGit bundle.

Bug:541500
Change-Id: Id2d8df12914e2cdbd936ff00dc824d8f871bd580
Signed-off-by: James Wynn <james@jameswynn.com>
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>

Reason why this change is needed:
Getting this metric will help estimate how much time will be saved once
the reachability checks get optimized

What this patch does:
Measure time spent by requestValidator.checkWants() in parseWants() and save
it in an instance of PackStatistics.Accumulator.

Signed-off-by: Alexa Panfil <alexapizza@google.com>
Change-Id: Id7fe4016f96549d9511a2c24052dad93cfbb31a4

When comparing git directory paths to check whether one is a prefix
of another, one must add a slash to avoid false prefix matches when
one directory name is a prefix of another. The path "audio" is not
a prefix of the path "audio-new", but would be a prefix of a path
"audio/new".

Bug: 566799
Change-Id: I6f671ca043c7c2c6044eb05a71dc8cca8d0ee040
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>

Since there is no negotiation for a push, the client is probably sending
redundant objects and bytes which already exist in the server.

Add more metrics in the stats to quantify it. Duplicated size and number
to measure the size and the number of duplicated objects which should
not be pushed.

Change-Id: Iaacd4761ee9366a0a7ec4e26c508eff45c8744de
Signed-off-by: Yunjie Li <yunjieli@google.com>

Change-Id: I4bdf1ab3e7d7e9d96318e2b16b792aa98ebd2ecd
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

DfsBundleWriter writes out the entire repository to a Git bundle file.
It packs all objects included in the packfile by concatenating all pack
files. This makes the bundle creation fast and cheap. Useful for backing
up a repository as-is.

Change-Id: Iee20e4b1ab45b2a178dde8c72093c0dd83f04805
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>

Previously ResolveMerger tried to make a fulltext merge entry in case
one of sides got deleted regardless of file mode. This is not
applicable for GITLINK type of entry. After this change it is
rendering appropriate merge result.

Signed-off-by: Demetr Starshov <dstarshov@google.com>
Change-Id: Ibdb4557bf8781bdb48bcee6529e37dc80582ed7e

Add test cases which cover content-merge resolve logic for deletion.

Sign-off-by: Demetr Starshov <dstarshov@google.com>
Change-Id: I2f2b37e29adc973a5a0cfcc5c8bc32a2c38efdfa

Option ignoreConflicts is used when a caller want to create a virtual
commit and use it in a future merge (recursive merge) or show it on
UI (e.g. Gerrit). According to contract in case of ignoreConflicts
ResolveMerger should populate only stage 0 for entries with merge
conflicts as if there is no conflict. Current implementation breaks
this contract for cases when gitlink revision is ambiguous.

Therefore, always select 'ours' when we merge in ignoreConflicts mode.
This will satisfy the contract contract, so recursive merge can
succeed, however it is an arbitrary decision, so it is not guaranteed
to select best GITLINK in all cases.

GITLINK merging is a special case of recursive merge because of
limitations of GITLINK type of entry. It can't contain more than 1 sha-1
so jgit can't write merge conflicts in place like it can with a blob.
Ideally we could signal the conflict with a special value (like
'0000...'), but that must be supported by all tooling (git fsck, c-git)."

Signed-off-by: Demetr Starshov <dstarshov@google.com>
Change-Id: Id4e9bebc8e828f7a1ef9f83259159137df477d89

Add test cases which cover content-merge resolve logic.

Git clients try to agressively merge blobs by content, but GITLINK types
of entries can't be merged with each other or with blobs. This change
ensures all possible permutations which can trigger blob and GITLINK
content merge are covered.

Signed-off-by: Demetr Starshov <dstarshov@google.com>
Change-Id: I7e83a28a14d4d2f9e0ba2b1cffbf3224fb7f3fef

Due to an integer overflow bug, the current "Index file is too large
for jgit" check did not work properly and subsequently a
NegativeArraySizeException was raised.

Change-Id: I2736efb28987c29e56bc946563b7fa781898a94a
Signed-off-by: Marc Strapetz <marc.strapetz@syntevo.com>

Git never converts line endings if the version in the repository is a
text file with CR/LF and text=auto. See [1]: "When the file has been
committed with CRLF, no conversion is done."

Because the sentence just before is about converting line endings on
check-in, I had understood that in commit 60cf85a [2] to mean that no
conversion on check-in was to be done. However, as bug 565048 and a
code inspection of the C git code showed it really means no conversion
is done on check-in *or check-out*.

If the text attribute is not set but core.autocrlf = true, this is
the same as text=auto eol=crlf. C git does not convert on check-out
even on text=auto eol=lf if the index version is a text file with
CR/LF.

For check-in, one has to look at the intended target, which is done
in WorkingTreeIterator since commit 60cf85a. For check-out, it can
be done by looking at the source and can thus be done in the
AutoLFOutputStream.

Additionally, provide a constructor for AutoLFInputStream to do
the same; for cases where the equivalent of a check-out is done via
an input stream obtained from a blob. (EGit does that in its
GitBlobStorage for the Eclipse compare framework; it's more efficient
than using a TemporaryBuffer and DirCacheCheckout.getContent(), and
it avoids the need for a temporary file.)

Adapt existing tests, and add new checkout and merge tests to verify
the resulting files have the correct line endings.

EGit's GitBlobStorage will need to call the new version of
EolStreamTypeUtil.wrapInputStream().

[1] https://git-scm.com/docs/gitattributes#Documentation/gitattributes.txt-Settostringvalueauto
[2] https://git.eclipse.org/r/c/jgit/jgit/+/127324

Bug: 565048
Change-Id: If1282ef43e2abd00263541bd10a01fe1f5c619fc
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>

Index format version 4 was introduced in C git in 2012. It's about
time that JGit can deal with it.

Version 4 added prefix path compression. Instead of writing the full
path for each index entry to disk, only the difference to the previous
entry's path is written: a variable-encoded int telling how many bytes
to remove from the previous entry's path to get the common prefix,
followed by the new suffix.

Also, cache entries in a version 4 index are not padded anymore.

Internally, version 3 and version 4 index entries are identical; it's
only the stored format that changes.

Implement this path compression, and make sure we write an index file
that we read previously in the same format. (Only changing from version
2 to version 3 if there are extended flags.)

Add support for the "feature.manyFiles" and the "index.version" git
configs, and honor them when writing a new index file.

Add tests, including a compatibility test that verifies that JGit can
read a version 4 index generated by C git and write an identical
version 4 index.

Bug: 565774
Change-Id: Id83241cf009e50f950eb42f8d56b834fb47da1ed
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>

If I create a repository containing an empty file and clone it
with

	git clone --no-checkout --filter=blob:none \
		https://url/of/repository

then I would expect no blobs to be transferred over the wire.  Alas,
JGit rewrites filter=blob:none to filter=blob:limit=0, so if the
repository contains an empty file then the empty blob gets
transferred.

Fix it by teaching JGit about filters based on object type to
complement the existing filters based on object size.  This prepares
us for other future filters such as object:none.

In particular, this means we do not need to look up the size of the
filtered blobs, which should speed up clones.  Noticed by Anna
Pologova and Terry Parker.

Change-Id: Id4b234921a190c108d8be2c87f54dcbfa811602a
Signed-off-by: Jonathan Nieder <jrn@google.com>

Teach the FilterSpec serialization code about tree filters so they can
be communicated over the wire and understood by the server.

While we're here, harden the FilterSpec serialization code to throw
IllegalStateException if we encounter a FilterSpec that cannot be
expressed as a "filter" line.  The only public API for creating a
Filterspec is to pass in a "filter" line to be parsed, so these should
not appear in practice.

Change-Id: I9664844059ffbc9c36eb829e2d860f198b9403a0
Signed-off-by: Jonathan Nieder <jrn@google.com>

Change-Id: I7c44e3603df2dd368cb7c0ba0072413b887b6903
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

In JGit 5.0, the FileTreeIterator was changed to skip ignored folders
by default. To catch tracked files inside ignored folders, the tree
walk needs to have a DirCacheIterator, and the FileTreeIterator has
to know about that DirCacheIterator via setDirCacheIterator(). (Or
the optimization has to be switched off explicitly via
setWalkIgnoredDirectories(true).)

Skipping ignored directories is an important optimization in some
cases, for instance in node.js/npm projects, where we'd otherwise
traverse the whole huge and deep hierarchy of the typically ignored
node_modules folder.

While all uses of WorkingTreeIterator in JGit had been adapted,
DiffFormatter was forgotten. To make it work correctly (again) also
for such cases, make it set up a WorkingTreeeIterator automatically,
and make sure the WorkingTreeSource can find such files, too. Also
pass the repository to the TreeWalks used inside the DiffFormatter
to pick up the correct attributes, filters, and line-ending settings.

Bug: 565081
Change-Id: Ie88ac81166dc396ba28b83313964c1712b6ca199
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>