You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

WorkingTreeIterator.java 48KB

Fix computation of id in WorkingTreeIterator with autocrlf and smudging JGit failed to do checkouts when the index contained smudged entries and autocrlf was on. In such cases the WorkingTreeIterator calculated the SHA1 sometimes on content which was not correctly filtered. The SHA1 was computed on content which two times went through a lf->crlf conversion. We used to tell the treewalk whether it is a checkin or checkout operation and always use the related filters when reading any content. If on windows and autocrlf is true and we do a checkout operation then we always used a lf->crlf conversion on any text content. That's not correct. Even during a checkout we sometimes need the crlf->lf conversion. E.g. when calculating the content-id for working-tree content we need to use crlf->lf filtering although the overall operation type is checkout. Often this bug does not have effects because we seldom compute the content-id of filesystem content during a checkout. But we do need to know whether a file is dirty or not before we overwrite it during a checkout. And if the index entries are smudged we don't trust the index and compute filesystem-content-sha1's explicitly. This caused EGit not to be able to switch branches anymore on Windows when autocrlf was true. EGit denied the checkout because it thought workingtree files are dirty because content-sha1 are computed on wrongly filtered content. Bug: 493360 Change-Id: I1072a57b4c529ba3aaa50b7b02d2b816bb64a9b8 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
8 vuotta sitten
Fix computation of id in WorkingTreeIterator with autocrlf and smudging JGit failed to do checkouts when the index contained smudged entries and autocrlf was on. In such cases the WorkingTreeIterator calculated the SHA1 sometimes on content which was not correctly filtered. The SHA1 was computed on content which two times went through a lf->crlf conversion. We used to tell the treewalk whether it is a checkin or checkout operation and always use the related filters when reading any content. If on windows and autocrlf is true and we do a checkout operation then we always used a lf->crlf conversion on any text content. That's not correct. Even during a checkout we sometimes need the crlf->lf conversion. E.g. when calculating the content-id for working-tree content we need to use crlf->lf filtering although the overall operation type is checkout. Often this bug does not have effects because we seldom compute the content-id of filesystem content during a checkout. But we do need to know whether a file is dirty or not before we overwrite it during a checkout. And if the index entries are smudged we don't trust the index and compute filesystem-content-sha1's explicitly. This caused EGit not to be able to switch branches anymore on Windows when autocrlf was true. EGit denied the checkout because it thought workingtree files are dirty because content-sha1 are computed on wrongly filtered content. Bug: 493360 Change-Id: I1072a57b4c529ba3aaa50b7b02d2b816bb64a9b8 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
8 vuotta sitten
Fix computation of id in WorkingTreeIterator with autocrlf and smudging JGit failed to do checkouts when the index contained smudged entries and autocrlf was on. In such cases the WorkingTreeIterator calculated the SHA1 sometimes on content which was not correctly filtered. The SHA1 was computed on content which two times went through a lf->crlf conversion. We used to tell the treewalk whether it is a checkin or checkout operation and always use the related filters when reading any content. If on windows and autocrlf is true and we do a checkout operation then we always used a lf->crlf conversion on any text content. That's not correct. Even during a checkout we sometimes need the crlf->lf conversion. E.g. when calculating the content-id for working-tree content we need to use crlf->lf filtering although the overall operation type is checkout. Often this bug does not have effects because we seldom compute the content-id of filesystem content during a checkout. But we do need to know whether a file is dirty or not before we overwrite it during a checkout. And if the index entries are smudged we don't trust the index and compute filesystem-content-sha1's explicitly. This caused EGit not to be able to switch branches anymore on Windows when autocrlf was true. EGit denied the checkout because it thought workingtree files are dirty because content-sha1 are computed on wrongly filtered content. Bug: 493360 Change-Id: I1072a57b4c529ba3aaa50b7b02d2b816bb64a9b8 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
8 vuotta sitten
Fix computation of id in WorkingTreeIterator with autocrlf and smudging JGit failed to do checkouts when the index contained smudged entries and autocrlf was on. In such cases the WorkingTreeIterator calculated the SHA1 sometimes on content which was not correctly filtered. The SHA1 was computed on content which two times went through a lf->crlf conversion. We used to tell the treewalk whether it is a checkin or checkout operation and always use the related filters when reading any content. If on windows and autocrlf is true and we do a checkout operation then we always used a lf->crlf conversion on any text content. That's not correct. Even during a checkout we sometimes need the crlf->lf conversion. E.g. when calculating the content-id for working-tree content we need to use crlf->lf filtering although the overall operation type is checkout. Often this bug does not have effects because we seldom compute the content-id of filesystem content during a checkout. But we do need to know whether a file is dirty or not before we overwrite it during a checkout. And if the index entries are smudged we don't trust the index and compute filesystem-content-sha1's explicitly. This caused EGit not to be able to switch branches anymore on Windows when autocrlf was true. EGit denied the checkout because it thought workingtree files are dirty because content-sha1 are computed on wrongly filtered content. Bug: 493360 Change-Id: I1072a57b4c529ba3aaa50b7b02d2b816bb64a9b8 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
8 vuotta sitten
Fix computation of id in WorkingTreeIterator with autocrlf and smudging JGit failed to do checkouts when the index contained smudged entries and autocrlf was on. In such cases the WorkingTreeIterator calculated the SHA1 sometimes on content which was not correctly filtered. The SHA1 was computed on content which two times went through a lf->crlf conversion. We used to tell the treewalk whether it is a checkin or checkout operation and always use the related filters when reading any content. If on windows and autocrlf is true and we do a checkout operation then we always used a lf->crlf conversion on any text content. That's not correct. Even during a checkout we sometimes need the crlf->lf conversion. E.g. when calculating the content-id for working-tree content we need to use crlf->lf filtering although the overall operation type is checkout. Often this bug does not have effects because we seldom compute the content-id of filesystem content during a checkout. But we do need to know whether a file is dirty or not before we overwrite it during a checkout. And if the index entries are smudged we don't trust the index and compute filesystem-content-sha1's explicitly. This caused EGit not to be able to switch branches anymore on Windows when autocrlf was true. EGit denied the checkout because it thought workingtree files are dirty because content-sha1 are computed on wrongly filtered content. Bug: 493360 Change-Id: I1072a57b4c529ba3aaa50b7b02d2b816bb64a9b8 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
8 vuotta sitten
Fix computation of id in WorkingTreeIterator with autocrlf and smudging JGit failed to do checkouts when the index contained smudged entries and autocrlf was on. In such cases the WorkingTreeIterator calculated the SHA1 sometimes on content which was not correctly filtered. The SHA1 was computed on content which two times went through a lf->crlf conversion. We used to tell the treewalk whether it is a checkin or checkout operation and always use the related filters when reading any content. If on windows and autocrlf is true and we do a checkout operation then we always used a lf->crlf conversion on any text content. That's not correct. Even during a checkout we sometimes need the crlf->lf conversion. E.g. when calculating the content-id for working-tree content we need to use crlf->lf filtering although the overall operation type is checkout. Often this bug does not have effects because we seldom compute the content-id of filesystem content during a checkout. But we do need to know whether a file is dirty or not before we overwrite it during a checkout. And if the index entries are smudged we don't trust the index and compute filesystem-content-sha1's explicitly. This caused EGit not to be able to switch branches anymore on Windows when autocrlf was true. EGit denied the checkout because it thought workingtree files are dirty because content-sha1 are computed on wrongly filtered content. Bug: 493360 Change-Id: I1072a57b4c529ba3aaa50b7b02d2b816bb64a9b8 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
8 vuotta sitten
Fix computation of id in WorkingTreeIterator with autocrlf and smudging JGit failed to do checkouts when the index contained smudged entries and autocrlf was on. In such cases the WorkingTreeIterator calculated the SHA1 sometimes on content which was not correctly filtered. The SHA1 was computed on content which two times went through a lf->crlf conversion. We used to tell the treewalk whether it is a checkin or checkout operation and always use the related filters when reading any content. If on windows and autocrlf is true and we do a checkout operation then we always used a lf->crlf conversion on any text content. That's not correct. Even during a checkout we sometimes need the crlf->lf conversion. E.g. when calculating the content-id for working-tree content we need to use crlf->lf filtering although the overall operation type is checkout. Often this bug does not have effects because we seldom compute the content-id of filesystem content during a checkout. But we do need to know whether a file is dirty or not before we overwrite it during a checkout. And if the index entries are smudged we don't trust the index and compute filesystem-content-sha1's explicitly. This caused EGit not to be able to switch branches anymore on Windows when autocrlf was true. EGit denied the checkout because it thought workingtree files are dirty because content-sha1 are computed on wrongly filtered content. Bug: 493360 Change-Id: I1072a57b4c529ba3aaa50b7b02d2b816bb64a9b8 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
8 vuotta sitten
Fix computation of id in WorkingTreeIterator with autocrlf and smudging JGit failed to do checkouts when the index contained smudged entries and autocrlf was on. In such cases the WorkingTreeIterator calculated the SHA1 sometimes on content which was not correctly filtered. The SHA1 was computed on content which two times went through a lf->crlf conversion. We used to tell the treewalk whether it is a checkin or checkout operation and always use the related filters when reading any content. If on windows and autocrlf is true and we do a checkout operation then we always used a lf->crlf conversion on any text content. That's not correct. Even during a checkout we sometimes need the crlf->lf conversion. E.g. when calculating the content-id for working-tree content we need to use crlf->lf filtering although the overall operation type is checkout. Often this bug does not have effects because we seldom compute the content-id of filesystem content during a checkout. But we do need to know whether a file is dirty or not before we overwrite it during a checkout. And if the index entries are smudged we don't trust the index and compute filesystem-content-sha1's explicitly. This caused EGit not to be able to switch branches anymore on Windows when autocrlf was true. EGit denied the checkout because it thought workingtree files are dirty because content-sha1 are computed on wrongly filtered content. Bug: 493360 Change-Id: I1072a57b4c529ba3aaa50b7b02d2b816bb64a9b8 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
8 vuotta sitten
Fix computation of id in WorkingTreeIterator with autocrlf and smudging JGit failed to do checkouts when the index contained smudged entries and autocrlf was on. In such cases the WorkingTreeIterator calculated the SHA1 sometimes on content which was not correctly filtered. The SHA1 was computed on content which two times went through a lf->crlf conversion. We used to tell the treewalk whether it is a checkin or checkout operation and always use the related filters when reading any content. If on windows and autocrlf is true and we do a checkout operation then we always used a lf->crlf conversion on any text content. That's not correct. Even during a checkout we sometimes need the crlf->lf conversion. E.g. when calculating the content-id for working-tree content we need to use crlf->lf filtering although the overall operation type is checkout. Often this bug does not have effects because we seldom compute the content-id of filesystem content during a checkout. But we do need to know whether a file is dirty or not before we overwrite it during a checkout. And if the index entries are smudged we don't trust the index and compute filesystem-content-sha1's explicitly. This caused EGit not to be able to switch branches anymore on Windows when autocrlf was true. EGit denied the checkout because it thought workingtree files are dirty because content-sha1 are computed on wrongly filtered content. Bug: 493360 Change-Id: I1072a57b4c529ba3aaa50b7b02d2b816bb64a9b8 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
8 vuotta sitten
Fix computation of id in WorkingTreeIterator with autocrlf and smudging JGit failed to do checkouts when the index contained smudged entries and autocrlf was on. In such cases the WorkingTreeIterator calculated the SHA1 sometimes on content which was not correctly filtered. The SHA1 was computed on content which two times went through a lf->crlf conversion. We used to tell the treewalk whether it is a checkin or checkout operation and always use the related filters when reading any content. If on windows and autocrlf is true and we do a checkout operation then we always used a lf->crlf conversion on any text content. That's not correct. Even during a checkout we sometimes need the crlf->lf conversion. E.g. when calculating the content-id for working-tree content we need to use crlf->lf filtering although the overall operation type is checkout. Often this bug does not have effects because we seldom compute the content-id of filesystem content during a checkout. But we do need to know whether a file is dirty or not before we overwrite it during a checkout. And if the index entries are smudged we don't trust the index and compute filesystem-content-sha1's explicitly. This caused EGit not to be able to switch branches anymore on Windows when autocrlf was true. EGit denied the checkout because it thought workingtree files are dirty because content-sha1 are computed on wrongly filtered content. Bug: 493360 Change-Id: I1072a57b4c529ba3aaa50b7b02d2b816bb64a9b8 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
8 vuotta sitten
Significantly speed up FileTreeIterator on Windows Getting attributes of files on Windows is an expensive operation. Windows stores file attributes in the directory, so they are basically available "for free" when a directory is listed. The implementation of Java's Files.walkFileTree() takes advantage of that (at least in the OpenJDK implementation for Windows) and provides the attributes from the directory to a FileVisitor. Using Files.walkFileTree() with a maximum depth of 1 is thus a good approach on Windows to get both the file names and the attributes in one go. In my tests, this gives a significant speed-up of FileTreeIterator over the "normal" way: using File.listFiles() and then reading the attributes of each file individually. The speed-up is hard to quantify exactly, but in my tests I've observed consistently 30-40% for staging 500 files one after another, each individually, and up to 50% for individual TreeWalks with a FileTreeIterator. On Unix, this technique is detrimental. Unix stores file attributes differently, and getting attributes of individual files is not costly. On Unix, the old way of doing a listFiles() and getting individual attributes (both native operations) is about three times faster than using walkFileTree, which is implemented in Java. Therefore, move the operation to FS/FS_Win32 and call it from FileTreeIterator, so that we can have different implementations depending on the file system. A little performance test program is included as a JUnit test (to be run manually). While this does speed up things on Windows, it doesn't solve the basic problem of bug 532300: the iterator always gets the full directory listing and the attributes of all files, and the more files there are the longer that takes. Bug: 532300 Change-Id: Ic5facb871c725256c2324b0d97b95e6efc33282a Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
6 vuotta sitten
Fix computation of id in WorkingTreeIterator with autocrlf and smudging JGit failed to do checkouts when the index contained smudged entries and autocrlf was on. In such cases the WorkingTreeIterator calculated the SHA1 sometimes on content which was not correctly filtered. The SHA1 was computed on content which two times went through a lf->crlf conversion. We used to tell the treewalk whether it is a checkin or checkout operation and always use the related filters when reading any content. If on windows and autocrlf is true and we do a checkout operation then we always used a lf->crlf conversion on any text content. That's not correct. Even during a checkout we sometimes need the crlf->lf conversion. E.g. when calculating the content-id for working-tree content we need to use crlf->lf filtering although the overall operation type is checkout. Often this bug does not have effects because we seldom compute the content-id of filesystem content during a checkout. But we do need to know whether a file is dirty or not before we overwrite it during a checkout. And if the index entries are smudged we don't trust the index and compute filesystem-content-sha1's explicitly. This caused EGit not to be able to switch branches anymore on Windows when autocrlf was true. EGit denied the checkout because it thought workingtree files are dirty because content-sha1 are computed on wrongly filtered content. Bug: 493360 Change-Id: I1072a57b4c529ba3aaa50b7b02d2b816bb64a9b8 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
8 vuotta sitten
1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162636465666768697071727374757677787980818283848586878889909192939495969798991001011021031041051061071081091101111121131141151161171181191201211221231241251261271281291301311321331341351361371381391401411421431441451461471481491501511521531541551561571581591601611621631641651661671681691701711721731741751761771781791801811821831841851861871881891901911921931941951961971981992002012022032042052062072082092102112122132142152162172182192202212222232242252262272282292302312322332342352362372382392402412422432442452462472482492502512522532542552562572582592602612622632642652662672682692702712722732742752762772782792802812822832842852862872882892902912922932942952962972982993003013023033043053063073083093103113123133143153163173183193203213223233243253263273283293303313323333343353363373383393403413423433443453463473483493503513523533543553563573583593603613623633643653663673683693703713723733743753763773783793803813823833843853863873883893903913923933943953963973983994004014024034044054064074084094104114124134144154164174184194204214224234244254264274284294304314324334344354364374384394404414424434444454464474484494504514524534544554564574584594604614624634644654664674684694704714724734744754764774784794804814824834844854864874884894904914924934944954964974984995005015025035045055065075085095105115125135145155165175185195205215225235245255265275285295305315325335345355365375385395405415425435445455465475485495505515525535545555565575585595605615625635645655665675685695705715725735745755765775785795805815825835845855865875885895905915925935945955965975985996006016026036046056066076086096106116126136146156166176186196206216226236246256266276286296306316326336346356366376386396406416426436446456466476486496506516526536546556566576586596606616626636646656666676686696706716726736746756766776786796806816826836846856866876886896906916926936946956966976986997007017027037047057067077087097107117127137147157167177187197207217227237247257267277287297307317327337347357367377387397407417427437447457467477487497507517527537547557567577587597607617627637647657667677687697707717727737747757767777787797807817827837847857867877887897907917927937947957967977987998008018028038048058068078088098108118128138148158168178188198208218228238248258268278288298308318328338348358368378388398408418428438448458468478488498508518528538548558568578588598608618628638648658668678688698708718728738748758768778788798808818828838848858868878888898908918928938948958968978988999009019029039049059069079089099109119129139149159169179189199209219229239249259269279289299309319329339349359369379389399409419429439449459469479489499509519529539549559569579589599609619629639649659669679689699709719729739749759769779789799809819829839849859869879889899909919929939949959969979989991000100110021003100410051006100710081009101010111012101310141015101610171018101910201021102210231024102510261027102810291030103110321033103410351036103710381039104010411042104310441045104610471048104910501051105210531054105510561057105810591060106110621063106410651066106710681069107010711072107310741075107610771078107910801081108210831084108510861087108810891090109110921093109410951096109710981099110011011102110311041105110611071108110911101111111211131114111511161117111811191120112111221123112411251126112711281129113011311132113311341135113611371138113911401141114211431144114511461147114811491150115111521153115411551156115711581159116011611162116311641165116611671168116911701171117211731174117511761177117811791180118111821183118411851186118711881189119011911192119311941195119611971198119912001201120212031204120512061207120812091210121112121213121412151216121712181219122012211222122312241225122612271228122912301231123212331234123512361237123812391240124112421243124412451246124712481249125012511252125312541255125612571258125912601261126212631264126512661267126812691270127112721273127412751276127712781279128012811282128312841285128612871288128912901291129212931294129512961297129812991300130113021303130413051306130713081309131013111312131313141315131613171318131913201321132213231324132513261327132813291330133113321333133413351336133713381339134013411342134313441345134613471348134913501351135213531354135513561357135813591360136113621363136413651366136713681369137013711372137313741375137613771378137913801381138213831384138513861387138813891390139113921393139413951396139713981399140014011402140314041405140614071408140914101411141214131414141514161417141814191420142114221423142414251426142714281429143014311432143314341435143614371438143914401441144214431444144514461447144814491450145114521453145414551456145714581459146014611462146314641465146614671468146914701471147214731474147514761477147814791480148114821483148414851486148714881489149014911492149314941495149614971498149915001501150215031504150515061507150815091510151115121513151415151516151715181519152015211522152315241525152615271528152915301531153215331534153515361537153815391540154115421543154415451546154715481549155015511552155315541555155615571558155915601561156215631564156515661567156815691570157115721573157415751576157715781579158015811582
  1. /*
  2. * Copyright (C) 2008, Shawn O. Pearce <spearce@spearce.org>
  3. * Copyright (C) 2010, Christian Halstrick <christian.halstrick@sap.com>
  4. * Copyright (C) 2010, Matthias Sohn <matthias.sohn@sap.com>
  5. * Copyright (C) 2012-2013, Robin Rosenberg
  6. * and other copyright owners as documented in the project's IP log.
  7. *
  8. * This program and the accompanying materials are made available
  9. * under the terms of the Eclipse Distribution License v1.0 which
  10. * accompanies this distribution, is reproduced below, and is
  11. * available at http://www.eclipse.org/org/documents/edl-v10.php
  12. *
  13. * All rights reserved.
  14. *
  15. * Redistribution and use in source and binary forms, with or
  16. * without modification, are permitted provided that the following
  17. * conditions are met:
  18. *
  19. * - Redistributions of source code must retain the above copyright
  20. * notice, this list of conditions and the following disclaimer.
  21. *
  22. * - Redistributions in binary form must reproduce the above
  23. * copyright notice, this list of conditions and the following
  24. * disclaimer in the documentation and/or other materials provided
  25. * with the distribution.
  26. *
  27. * - Neither the name of the Eclipse Foundation, Inc. nor the
  28. * names of its contributors may be used to endorse or promote
  29. * products derived from this software without specific prior
  30. * written permission.
  31. *
  32. * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
  33. * CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
  34. * INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
  35. * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
  36. * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
  37. * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
  38. * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
  39. * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
  40. * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
  41. * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
  42. * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
  43. * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
  44. * ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  45. */
  46. package org.eclipse.jgit.treewalk;
  47. import static java.nio.charset.StandardCharsets.UTF_8;
  48. import java.io.ByteArrayInputStream;
  49. import java.io.File;
  50. import java.io.FileInputStream;
  51. import java.io.FileNotFoundException;
  52. import java.io.IOException;
  53. import java.io.InputStream;
  54. import java.nio.ByteBuffer;
  55. import java.nio.CharBuffer;
  56. import java.nio.charset.CharacterCodingException;
  57. import java.nio.charset.CharsetEncoder;
  58. import java.text.MessageFormat;
  59. import java.time.Instant;
  60. import java.util.Arrays;
  61. import java.util.Collections;
  62. import java.util.Comparator;
  63. import java.util.HashMap;
  64. import java.util.Map;
  65. import org.eclipse.jgit.api.errors.FilterFailedException;
  66. import org.eclipse.jgit.attributes.AttributesNode;
  67. import org.eclipse.jgit.attributes.AttributesRule;
  68. import org.eclipse.jgit.attributes.FilterCommand;
  69. import org.eclipse.jgit.attributes.FilterCommandRegistry;
  70. import org.eclipse.jgit.diff.RawText;
  71. import org.eclipse.jgit.dircache.DirCacheEntry;
  72. import org.eclipse.jgit.dircache.DirCacheIterator;
  73. import org.eclipse.jgit.errors.CorruptObjectException;
  74. import org.eclipse.jgit.errors.MissingObjectException;
  75. import org.eclipse.jgit.errors.NoWorkTreeException;
  76. import org.eclipse.jgit.ignore.FastIgnoreRule;
  77. import org.eclipse.jgit.ignore.IgnoreNode;
  78. import org.eclipse.jgit.internal.JGitText;
  79. import org.eclipse.jgit.lib.Constants;
  80. import org.eclipse.jgit.lib.CoreConfig;
  81. import org.eclipse.jgit.lib.CoreConfig.CheckStat;
  82. import org.eclipse.jgit.lib.CoreConfig.EolStreamType;
  83. import org.eclipse.jgit.lib.CoreConfig.SymLinks;
  84. import org.eclipse.jgit.lib.FileMode;
  85. import org.eclipse.jgit.lib.ObjectId;
  86. import org.eclipse.jgit.lib.ObjectLoader;
  87. import org.eclipse.jgit.lib.ObjectReader;
  88. import org.eclipse.jgit.lib.Repository;
  89. import org.eclipse.jgit.submodule.SubmoduleWalk;
  90. import org.eclipse.jgit.treewalk.TreeWalk.OperationType;
  91. import org.eclipse.jgit.util.FS;
  92. import org.eclipse.jgit.util.FS.ExecutionResult;
  93. import org.eclipse.jgit.util.Holder;
  94. import org.eclipse.jgit.util.IO;
  95. import org.eclipse.jgit.util.Paths;
  96. import org.eclipse.jgit.util.RawParseUtils;
  97. import org.eclipse.jgit.util.TemporaryBuffer;
  98. import org.eclipse.jgit.util.TemporaryBuffer.LocalFile;
  99. import org.eclipse.jgit.util.io.AutoLFInputStream;
  100. import org.eclipse.jgit.util.io.EolStreamTypeUtil;
  101. import org.eclipse.jgit.util.sha1.SHA1;
  102. /**
  103. * Walks a working directory tree as part of a
  104. * {@link org.eclipse.jgit.treewalk.TreeWalk}.
  105. * <p>
  106. * Most applications will want to use the standard implementation of this
  107. * iterator, {@link org.eclipse.jgit.treewalk.FileTreeIterator}, as that does
  108. * all IO through the standard <code>java.io</code> package. Plugins for a Java
  109. * based IDE may however wish to create their own implementations of this class
  110. * to allow traversal of the IDE's project space, as well as benefit from any
  111. * caching the IDE may have.
  112. *
  113. * @see FileTreeIterator
  114. */
  115. public abstract class WorkingTreeIterator extends AbstractTreeIterator {
  116. private static final int MAX_EXCEPTION_TEXT_SIZE = 10 * 1024;
  117. /** An empty entry array, suitable for {@link #init(Entry[])}. */
  118. protected static final Entry[] EOF = {};
  119. /** Size we perform file IO in if we have to read and hash a file. */
  120. static final int BUFFER_SIZE = 2048;
  121. /**
  122. * Maximum size of files which may be read fully into memory for performance
  123. * reasons.
  124. */
  125. private static final long MAXIMUM_FILE_SIZE_TO_READ_FULLY = 65536;
  126. /** Inherited state of this iterator, describing working tree, etc. */
  127. private final IteratorState state;
  128. /** The {@link #idBuffer()} for the current entry. */
  129. private byte[] contentId;
  130. /** Index within {@link #entries} that {@link #contentId} came from. */
  131. private int contentIdFromPtr;
  132. /** List of entries obtained from the subclass. */
  133. private Entry[] entries;
  134. /** Total number of entries in {@link #entries} that are valid. */
  135. private int entryCnt;
  136. /** Current position within {@link #entries}. */
  137. private int ptr;
  138. /** If there is a .gitignore file present, the parsed rules from it. */
  139. private IgnoreNode ignoreNode;
  140. /**
  141. * cached clean filter command. Use a Ref in order to distinguish between
  142. * the ref not cached yet and the value null
  143. */
  144. private Holder<String> cleanFilterCommandHolder;
  145. /**
  146. * cached eol stream type. Use a Ref in order to distinguish between the ref
  147. * not cached yet and the value null
  148. */
  149. private Holder<EolStreamType> eolStreamTypeHolder;
  150. /** Repository that is the root level being iterated over */
  151. protected Repository repository;
  152. /** Cached canonical length, initialized from {@link #idBuffer()} */
  153. private long canonLen = -1;
  154. /** The offset of the content id in {@link #idBuffer()} */
  155. private int contentIdOffset;
  156. /**
  157. * Create a new iterator with no parent.
  158. *
  159. * @param options
  160. * working tree options to be used
  161. */
  162. protected WorkingTreeIterator(WorkingTreeOptions options) {
  163. super();
  164. state = new IteratorState(options);
  165. }
  166. /**
  167. * Create a new iterator with no parent and a prefix.
  168. * <p>
  169. * The prefix path supplied is inserted in front of all paths generated by
  170. * this iterator. It is intended to be used when an iterator is being
  171. * created for a subsection of an overall repository and needs to be
  172. * combined with other iterators that are created to run over the entire
  173. * repository namespace.
  174. *
  175. * @param prefix
  176. * position of this iterator in the repository tree. The value
  177. * may be null or the empty string to indicate the prefix is the
  178. * root of the repository. A trailing slash ('/') is
  179. * automatically appended if the prefix does not end in '/'.
  180. * @param options
  181. * working tree options to be used
  182. */
  183. protected WorkingTreeIterator(final String prefix,
  184. WorkingTreeOptions options) {
  185. super(prefix);
  186. state = new IteratorState(options);
  187. }
  188. /**
  189. * Create an iterator for a subtree of an existing iterator.
  190. *
  191. * @param p
  192. * parent tree iterator.
  193. */
  194. protected WorkingTreeIterator(WorkingTreeIterator p) {
  195. super(p);
  196. state = p.state;
  197. repository = p.repository;
  198. }
  199. /**
  200. * Initialize this iterator for the root level of a repository.
  201. * <p>
  202. * This method should only be invoked after calling {@link #init(Entry[])},
  203. * and only for the root iterator.
  204. *
  205. * @param repo
  206. * the repository.
  207. */
  208. protected void initRootIterator(Repository repo) {
  209. repository = repo;
  210. Entry entry;
  211. if (ignoreNode instanceof PerDirectoryIgnoreNode)
  212. entry = ((PerDirectoryIgnoreNode) ignoreNode).entry;
  213. else
  214. entry = null;
  215. ignoreNode = new RootIgnoreNode(entry, repo);
  216. }
  217. /**
  218. * Define the matching {@link org.eclipse.jgit.dircache.DirCacheIterator},
  219. * to optimize ObjectIds.
  220. *
  221. * Once the DirCacheIterator has been set this iterator must only be
  222. * advanced by the TreeWalk that is supplied, as it assumes that itself and
  223. * the corresponding DirCacheIterator are positioned on the same file path
  224. * whenever {@link #idBuffer()} is invoked.
  225. *
  226. * @param walk
  227. * the walk that will be advancing this iterator.
  228. * @param treeId
  229. * index of the matching
  230. * {@link org.eclipse.jgit.dircache.DirCacheIterator}.
  231. */
  232. public void setDirCacheIterator(TreeWalk walk, int treeId) {
  233. state.walk = walk;
  234. state.dirCacheTree = treeId;
  235. }
  236. /**
  237. * Retrieves the {@link DirCacheIterator} at the current entry if
  238. * {@link #setDirCacheIterator(TreeWalk, int)} was called.
  239. *
  240. * @return the DirCacheIterator, or {@code null} if not set or not at the
  241. * current entry
  242. * @since 5.0
  243. */
  244. protected DirCacheIterator getDirCacheIterator() {
  245. if (state.dirCacheTree >= 0 && state.walk != null) {
  246. return state.walk.getTree(state.dirCacheTree,
  247. DirCacheIterator.class);
  248. }
  249. return null;
  250. }
  251. /**
  252. * Defines whether this {@link WorkingTreeIterator} walks ignored
  253. * directories.
  254. *
  255. * @param includeIgnored
  256. * {@code false} to skip ignored directories, if possible;
  257. * {@code true} to always include them in the walk
  258. * @since 5.0
  259. */
  260. public void setWalkIgnoredDirectories(boolean includeIgnored) {
  261. state.walkIgnored = includeIgnored;
  262. }
  263. /**
  264. * Tells whether this {@link WorkingTreeIterator} walks ignored directories.
  265. *
  266. * @return {@code true} if it does, {@code false} otherwise
  267. * @since 5.0
  268. */
  269. public boolean walksIgnoredDirectories() {
  270. return state.walkIgnored;
  271. }
  272. /** {@inheritDoc} */
  273. @Override
  274. public boolean hasId() {
  275. if (contentIdFromPtr == ptr)
  276. return true;
  277. return (mode & FileMode.TYPE_MASK) == FileMode.TYPE_FILE;
  278. }
  279. /** {@inheritDoc} */
  280. @Override
  281. public byte[] idBuffer() {
  282. if (contentIdFromPtr == ptr)
  283. return contentId;
  284. if (state.walk != null) {
  285. // If there is a matching DirCacheIterator, we can reuse
  286. // its idBuffer, but only if we appear to be clean against
  287. // the cached index information for the path.
  288. DirCacheIterator i = state.walk.getTree(state.dirCacheTree,
  289. DirCacheIterator.class);
  290. if (i != null) {
  291. DirCacheEntry ent = i.getDirCacheEntry();
  292. if (ent != null && compareMetadata(ent) == MetadataDiff.EQUAL
  293. && ((ent.getFileMode().getBits()
  294. & FileMode.TYPE_MASK) != FileMode.TYPE_GITLINK)) {
  295. contentIdOffset = i.idOffset();
  296. contentIdFromPtr = ptr;
  297. return contentId = i.idBuffer();
  298. }
  299. contentIdOffset = 0;
  300. } else {
  301. contentIdOffset = 0;
  302. }
  303. }
  304. switch (mode & FileMode.TYPE_MASK) {
  305. case FileMode.TYPE_SYMLINK:
  306. case FileMode.TYPE_FILE:
  307. contentIdFromPtr = ptr;
  308. return contentId = idBufferBlob(entries[ptr]);
  309. case FileMode.TYPE_GITLINK:
  310. contentIdFromPtr = ptr;
  311. return contentId = idSubmodule(entries[ptr]);
  312. }
  313. return zeroid;
  314. }
  315. /** {@inheritDoc} */
  316. @Override
  317. public boolean isWorkTree() {
  318. return true;
  319. }
  320. /**
  321. * Get submodule id for given entry.
  322. *
  323. * @param e
  324. * a {@link org.eclipse.jgit.treewalk.WorkingTreeIterator.Entry}
  325. * object.
  326. * @return non-null submodule id
  327. */
  328. protected byte[] idSubmodule(Entry e) {
  329. if (repository == null)
  330. return zeroid;
  331. File directory;
  332. try {
  333. directory = repository.getWorkTree();
  334. } catch (NoWorkTreeException nwte) {
  335. return zeroid;
  336. }
  337. return idSubmodule(directory, e);
  338. }
  339. /**
  340. * Get submodule id using the repository at the location of the entry
  341. * relative to the directory.
  342. *
  343. * @param directory
  344. * a {@link java.io.File} object.
  345. * @param e
  346. * a {@link org.eclipse.jgit.treewalk.WorkingTreeIterator.Entry}
  347. * object.
  348. * @return non-null submodule id
  349. */
  350. protected byte[] idSubmodule(File directory, Entry e) {
  351. try (Repository submoduleRepo = SubmoduleWalk.getSubmoduleRepository(
  352. directory, e.getName(),
  353. repository != null ? repository.getFS() : FS.DETECTED)) {
  354. if (submoduleRepo == null) {
  355. return zeroid;
  356. }
  357. ObjectId head = submoduleRepo.resolve(Constants.HEAD);
  358. if (head == null) {
  359. return zeroid;
  360. }
  361. byte[] id = new byte[Constants.OBJECT_ID_LENGTH];
  362. head.copyRawTo(id, 0);
  363. return id;
  364. } catch (IOException exception) {
  365. return zeroid;
  366. }
  367. }
  368. private static final byte[] digits = { '0', '1', '2', '3', '4', '5', '6',
  369. '7', '8', '9' };
  370. private static final byte[] hblob = Constants
  371. .encodedTypeString(Constants.OBJ_BLOB);
  372. private byte[] idBufferBlob(Entry e) {
  373. try {
  374. final InputStream is = e.openInputStream();
  375. if (is == null)
  376. return zeroid;
  377. try {
  378. state.initializeReadBuffer();
  379. final long len = e.getLength();
  380. InputStream filteredIs = possiblyFilteredInputStream(e, is, len,
  381. OperationType.CHECKIN_OP);
  382. return computeHash(filteredIs, canonLen);
  383. } finally {
  384. safeClose(is);
  385. }
  386. } catch (IOException err) {
  387. // Can't read the file? Don't report the failure either.
  388. return zeroid;
  389. }
  390. }
  391. private InputStream possiblyFilteredInputStream(final Entry e,
  392. final InputStream is, final long len) throws IOException {
  393. return possiblyFilteredInputStream(e, is, len, null);
  394. }
  395. private InputStream possiblyFilteredInputStream(final Entry e,
  396. final InputStream is, final long len, OperationType opType)
  397. throws IOException {
  398. if (getCleanFilterCommand() == null
  399. && getEolStreamType(opType) == EolStreamType.DIRECT) {
  400. canonLen = len;
  401. return is;
  402. }
  403. if (len <= MAXIMUM_FILE_SIZE_TO_READ_FULLY) {
  404. ByteBuffer rawbuf = IO.readWholeStream(is, (int) len);
  405. rawbuf = filterClean(rawbuf.array(), rawbuf.limit(), opType);
  406. canonLen = rawbuf.limit();
  407. return new ByteArrayInputStream(rawbuf.array(), 0, (int) canonLen);
  408. }
  409. if (getCleanFilterCommand() == null && isBinary(e)) {
  410. canonLen = len;
  411. return is;
  412. }
  413. final InputStream lenIs = filterClean(e.openInputStream(),
  414. opType);
  415. try {
  416. canonLen = computeLength(lenIs);
  417. } finally {
  418. safeClose(lenIs);
  419. }
  420. return filterClean(is, opType);
  421. }
  422. private static void safeClose(InputStream in) {
  423. try {
  424. in.close();
  425. } catch (IOException err2) {
  426. // Suppress any error related to closing an input
  427. // stream. We don't care, we should not have any
  428. // outstanding data to flush or anything like that.
  429. }
  430. }
  431. private static boolean isBinary(Entry entry) throws IOException {
  432. InputStream in = entry.openInputStream();
  433. try {
  434. return RawText.isBinary(in);
  435. } finally {
  436. safeClose(in);
  437. }
  438. }
  439. private ByteBuffer filterClean(byte[] src, int n, OperationType opType)
  440. throws IOException {
  441. InputStream in = new ByteArrayInputStream(src);
  442. try {
  443. return IO.readWholeStream(filterClean(in, opType), n);
  444. } finally {
  445. safeClose(in);
  446. }
  447. }
  448. private InputStream filterClean(InputStream in) throws IOException {
  449. return filterClean(in, null);
  450. }
  451. private InputStream filterClean(InputStream in, OperationType opType)
  452. throws IOException {
  453. in = handleAutoCRLF(in, opType);
  454. String filterCommand = getCleanFilterCommand();
  455. if (filterCommand != null) {
  456. if (FilterCommandRegistry.isRegistered(filterCommand)) {
  457. LocalFile buffer = new TemporaryBuffer.LocalFile(null);
  458. FilterCommand command = FilterCommandRegistry
  459. .createFilterCommand(filterCommand, repository, in,
  460. buffer);
  461. while (command.run() != -1) {
  462. // loop as long as command.run() tells there is work to do
  463. }
  464. return buffer.openInputStreamWithAutoDestroy();
  465. }
  466. FS fs = repository.getFS();
  467. ProcessBuilder filterProcessBuilder = fs.runInShell(filterCommand,
  468. new String[0]);
  469. filterProcessBuilder.directory(repository.getWorkTree());
  470. filterProcessBuilder.environment().put(Constants.GIT_DIR_KEY,
  471. repository.getDirectory().getAbsolutePath());
  472. ExecutionResult result;
  473. try {
  474. result = fs.execute(filterProcessBuilder, in);
  475. } catch (IOException | InterruptedException e) {
  476. throw new IOException(new FilterFailedException(e,
  477. filterCommand, getEntryPathString()));
  478. }
  479. int rc = result.getRc();
  480. if (rc != 0) {
  481. throw new IOException(new FilterFailedException(rc,
  482. filterCommand, getEntryPathString(),
  483. result.getStdout().toByteArray(MAX_EXCEPTION_TEXT_SIZE),
  484. RawParseUtils.decode(result.getStderr()
  485. .toByteArray(MAX_EXCEPTION_TEXT_SIZE))));
  486. }
  487. return result.getStdout().openInputStreamWithAutoDestroy();
  488. }
  489. return in;
  490. }
  491. private InputStream handleAutoCRLF(InputStream in, OperationType opType)
  492. throws IOException {
  493. return EolStreamTypeUtil.wrapInputStream(in, getEolStreamType(opType));
  494. }
  495. /**
  496. * Returns the working tree options used by this iterator.
  497. *
  498. * @return working tree options
  499. */
  500. public WorkingTreeOptions getOptions() {
  501. return state.options;
  502. }
  503. /** {@inheritDoc} */
  504. @Override
  505. public int idOffset() {
  506. return contentIdOffset;
  507. }
  508. /** {@inheritDoc} */
  509. @Override
  510. public void reset() {
  511. if (!first()) {
  512. ptr = 0;
  513. if (!eof())
  514. parseEntry();
  515. }
  516. }
  517. /** {@inheritDoc} */
  518. @Override
  519. public boolean first() {
  520. return ptr == 0;
  521. }
  522. /** {@inheritDoc} */
  523. @Override
  524. public boolean eof() {
  525. return ptr == entryCnt;
  526. }
  527. /** {@inheritDoc} */
  528. @Override
  529. public void next(int delta) throws CorruptObjectException {
  530. ptr += delta;
  531. if (!eof()) {
  532. parseEntry();
  533. }
  534. }
  535. /** {@inheritDoc} */
  536. @Override
  537. public void back(int delta) throws CorruptObjectException {
  538. ptr -= delta;
  539. parseEntry();
  540. }
  541. private void parseEntry() {
  542. final Entry e = entries[ptr];
  543. mode = e.getMode().getBits();
  544. final int nameLen = e.encodedNameLen;
  545. ensurePathCapacity(pathOffset + nameLen, pathOffset);
  546. System.arraycopy(e.encodedName, 0, path, pathOffset, nameLen);
  547. pathLen = pathOffset + nameLen;
  548. canonLen = -1;
  549. cleanFilterCommandHolder = null;
  550. eolStreamTypeHolder = null;
  551. }
  552. /**
  553. * Get the raw byte length of this entry.
  554. *
  555. * @return size of this file, in bytes.
  556. */
  557. public long getEntryLength() {
  558. return current().getLength();
  559. }
  560. /**
  561. * Get the filtered input length of this entry
  562. *
  563. * @return size of the content, in bytes
  564. * @throws java.io.IOException
  565. */
  566. public long getEntryContentLength() throws IOException {
  567. if (canonLen == -1) {
  568. long rawLen = getEntryLength();
  569. if (rawLen == 0)
  570. canonLen = 0;
  571. InputStream is = current().openInputStream();
  572. try {
  573. // canonLen gets updated here
  574. possiblyFilteredInputStream(current(), is, current()
  575. .getLength());
  576. } finally {
  577. safeClose(is);
  578. }
  579. }
  580. return canonLen;
  581. }
  582. /**
  583. * Get the last modified time of this entry.
  584. *
  585. * @return last modified time of this file, in milliseconds since the epoch
  586. * (Jan 1, 1970 UTC).
  587. * @deprecated use {@link #getEntryLastModifiedInstant()} instead
  588. */
  589. @Deprecated
  590. public long getEntryLastModified() {
  591. return current().getLastModified();
  592. }
  593. /**
  594. * Get the last modified time of this entry.
  595. *
  596. * @return last modified time of this file
  597. * @since 5.1.9
  598. */
  599. public Instant getEntryLastModifiedInstant() {
  600. return current().getLastModifiedInstant();
  601. }
  602. /**
  603. * Obtain an input stream to read the file content.
  604. * <p>
  605. * Efficient implementations are not required. The caller will usually
  606. * obtain the stream only once per entry, if at all.
  607. * <p>
  608. * The input stream should not use buffering if the implementation can avoid
  609. * it. The caller will buffer as necessary to perform efficient block IO
  610. * operations.
  611. * <p>
  612. * The caller will close the stream once complete.
  613. *
  614. * @return a stream to read from the file.
  615. * @throws java.io.IOException
  616. * the file could not be opened for reading.
  617. */
  618. public InputStream openEntryStream() throws IOException {
  619. InputStream rawis = current().openInputStream();
  620. if (getCleanFilterCommand() == null
  621. && getEolStreamType() == EolStreamType.DIRECT)
  622. return rawis;
  623. else
  624. return filterClean(rawis);
  625. }
  626. /**
  627. * Determine if the current entry path is ignored by an ignore rule.
  628. *
  629. * @return true if the entry was ignored by an ignore rule file.
  630. * @throws java.io.IOException
  631. * a relevant ignore rule file exists but cannot be read.
  632. */
  633. public boolean isEntryIgnored() throws IOException {
  634. return isEntryIgnored(pathLen);
  635. }
  636. /**
  637. * Determine if the entry path is ignored by an ignore rule.
  638. *
  639. * @param pLen
  640. * the length of the path in the path buffer.
  641. * @return true if the entry is ignored by an ignore rule.
  642. * @throws java.io.IOException
  643. * a relevant ignore rule file exists but cannot be read.
  644. */
  645. protected boolean isEntryIgnored(int pLen) throws IOException {
  646. return isEntryIgnored(pLen, mode);
  647. }
  648. /**
  649. * Determine if the entry path is ignored by an ignore rule.
  650. *
  651. * @param pLen
  652. * the length of the path in the path buffer.
  653. * @param fileMode
  654. * the original iterator file mode
  655. * @return true if the entry is ignored by an ignore rule.
  656. * @throws IOException
  657. * a relevant ignore rule file exists but cannot be read.
  658. */
  659. private boolean isEntryIgnored(int pLen, int fileMode)
  660. throws IOException {
  661. // The ignore code wants path to start with a '/' if possible.
  662. // If we have the '/' in our path buffer because we are inside
  663. // a sub-directory include it in the range we convert to string.
  664. //
  665. final int pOff = 0 < pathOffset ? pathOffset - 1 : pathOffset;
  666. String pathRel = TreeWalk.pathOf(this.path, pOff, pLen);
  667. String parentRel = getParentPath(pathRel);
  668. // CGit is processing .gitignore files by starting at the root of the
  669. // repository and then recursing into subdirectories. With this
  670. // approach, top-level ignored directories will be processed first which
  671. // allows to skip entire subtrees and further .gitignore-file processing
  672. // within these subtrees.
  673. //
  674. // We will follow the same approach by marking directories as "ignored"
  675. // here. This allows to have a simplified FastIgnore.checkIgnore()
  676. // implementation (both in terms of code and computational complexity):
  677. //
  678. // Without the "ignored" flag, we would have to apply the ignore-check
  679. // to a path and all of its parents always(!), to determine whether a
  680. // path is ignored directly or by one of its parent directories; with
  681. // the "ignored" flag, we know at this point that the parent directory
  682. // is definitely not ignored, thus the path can only become ignored if
  683. // there is a rule matching the path itself.
  684. if (isDirectoryIgnored(parentRel)) {
  685. return true;
  686. }
  687. IgnoreNode rules = getIgnoreNode();
  688. final Boolean ignored = rules != null
  689. ? rules.checkIgnored(pathRel, FileMode.TREE.equals(fileMode))
  690. : null;
  691. if (ignored != null) {
  692. return ignored.booleanValue();
  693. }
  694. return parent instanceof WorkingTreeIterator
  695. && ((WorkingTreeIterator) parent).isEntryIgnored(pLen,
  696. fileMode);
  697. }
  698. private IgnoreNode getIgnoreNode() throws IOException {
  699. if (ignoreNode instanceof PerDirectoryIgnoreNode)
  700. ignoreNode = ((PerDirectoryIgnoreNode) ignoreNode).load();
  701. return ignoreNode;
  702. }
  703. /**
  704. * Retrieves the {@link org.eclipse.jgit.attributes.AttributesNode} for the
  705. * current entry.
  706. *
  707. * @return the {@link org.eclipse.jgit.attributes.AttributesNode} for the
  708. * current entry.
  709. * @throws IOException
  710. */
  711. public AttributesNode getEntryAttributesNode() throws IOException {
  712. if (attributesNode instanceof PerDirectoryAttributesNode)
  713. attributesNode = ((PerDirectoryAttributesNode) attributesNode)
  714. .load();
  715. return attributesNode;
  716. }
  717. private static final Comparator<Entry> ENTRY_CMP = new Comparator<Entry>() {
  718. @Override
  719. public int compare(Entry a, Entry b) {
  720. return Paths.compare(
  721. a.encodedName, 0, a.encodedNameLen, a.getMode().getBits(),
  722. b.encodedName, 0, b.encodedNameLen, b.getMode().getBits());
  723. }
  724. };
  725. /**
  726. * Constructor helper.
  727. *
  728. * @param list
  729. * files in the subtree of the work tree this iterator operates
  730. * on
  731. */
  732. protected void init(Entry[] list) {
  733. // Filter out nulls, . and .. as these are not valid tree entries,
  734. // also cache the encoded forms of the path names for efficient use
  735. // later on during sorting and iteration.
  736. //
  737. entries = list;
  738. int i, o;
  739. final CharsetEncoder nameEncoder = state.nameEncoder;
  740. for (i = 0, o = 0; i < entries.length; i++) {
  741. final Entry e = entries[i];
  742. if (e == null)
  743. continue;
  744. final String name = e.getName();
  745. if (".".equals(name) || "..".equals(name)) //$NON-NLS-1$ //$NON-NLS-2$
  746. continue;
  747. if (Constants.DOT_GIT.equals(name))
  748. continue;
  749. if (Constants.DOT_GIT_IGNORE.equals(name))
  750. ignoreNode = new PerDirectoryIgnoreNode(e);
  751. if (Constants.DOT_GIT_ATTRIBUTES.equals(name))
  752. attributesNode = new PerDirectoryAttributesNode(e);
  753. if (i != o)
  754. entries[o] = e;
  755. e.encodeName(nameEncoder);
  756. o++;
  757. }
  758. entryCnt = o;
  759. Arrays.sort(entries, 0, entryCnt, ENTRY_CMP);
  760. contentIdFromPtr = -1;
  761. ptr = 0;
  762. if (!eof())
  763. parseEntry();
  764. else if (pathLen == 0) // see bug 445363
  765. pathLen = pathOffset;
  766. }
  767. /**
  768. * Obtain the current entry from this iterator.
  769. *
  770. * @return the currently selected entry.
  771. */
  772. protected Entry current() {
  773. return entries[ptr];
  774. }
  775. /**
  776. * The result of a metadata-comparison between the current entry and a
  777. * {@link DirCacheEntry}
  778. */
  779. public enum MetadataDiff {
  780. /**
  781. * The entries are equal by metaData (mode, length,
  782. * modification-timestamp) or the <code>assumeValid</code> attribute of
  783. * the index entry is set
  784. */
  785. EQUAL,
  786. /**
  787. * The entries are not equal by metaData (mode, length) or the
  788. * <code>isUpdateNeeded</code> attribute of the index entry is set
  789. */
  790. DIFFER_BY_METADATA,
  791. /** index entry is smudged - can't use that entry for comparison */
  792. SMUDGED,
  793. /**
  794. * The entries are equal by metaData (mode, length) but differ by
  795. * modification-timestamp.
  796. */
  797. DIFFER_BY_TIMESTAMP
  798. }
  799. /**
  800. * Is the file mode of the current entry different than the given raw mode?
  801. *
  802. * @param rawMode
  803. * an int.
  804. * @return true if different, false otherwise
  805. */
  806. public boolean isModeDifferent(int rawMode) {
  807. // Determine difference in mode-bits of file and index-entry. In the
  808. // bitwise presentation of modeDiff we'll have a '1' when the two modes
  809. // differ at this position.
  810. int modeDiff = getEntryRawMode() ^ rawMode;
  811. if (modeDiff == 0)
  812. return false;
  813. // Do not rely on filemode differences in case of symbolic links
  814. if (getOptions().getSymLinks() == SymLinks.FALSE)
  815. if (FileMode.SYMLINK.equals(rawMode))
  816. return false;
  817. // Ignore the executable file bits if WorkingTreeOptions tell me to
  818. // do so. Ignoring is done by setting the bits representing a
  819. // EXECUTABLE_FILE to '0' in modeDiff
  820. if (!state.options.isFileMode())
  821. modeDiff &= ~FileMode.EXECUTABLE_FILE.getBits();
  822. return modeDiff != 0;
  823. }
  824. /**
  825. * Compare the metadata (mode, length, modification-timestamp) of the
  826. * current entry and a {@link org.eclipse.jgit.dircache.DirCacheEntry}
  827. *
  828. * @param entry
  829. * the {@link org.eclipse.jgit.dircache.DirCacheEntry} to compare
  830. * with
  831. * @return a
  832. * {@link org.eclipse.jgit.treewalk.WorkingTreeIterator.MetadataDiff}
  833. * which tells whether and how the entries metadata differ
  834. */
  835. public MetadataDiff compareMetadata(DirCacheEntry entry) {
  836. if (entry.isAssumeValid())
  837. return MetadataDiff.EQUAL;
  838. if (entry.isUpdateNeeded())
  839. return MetadataDiff.DIFFER_BY_METADATA;
  840. if (isModeDifferent(entry.getRawMode()))
  841. return MetadataDiff.DIFFER_BY_METADATA;
  842. // Don't check for length or lastmodified on folders
  843. int type = mode & FileMode.TYPE_MASK;
  844. if (type == FileMode.TYPE_TREE || type == FileMode.TYPE_GITLINK)
  845. return MetadataDiff.EQUAL;
  846. if (!entry.isSmudged() && entry.getLength() != (int) getEntryLength())
  847. return MetadataDiff.DIFFER_BY_METADATA;
  848. // Git under windows only stores seconds so we round the timestamp
  849. // Java gives us if it looks like the timestamp in index is seconds
  850. // only. Otherwise we compare the timestamp at nanosecond precision,
  851. // unless core.checkstat is set to "minimal", in which case we only
  852. // compare the whole second part.
  853. Instant cacheLastModified = entry.getLastModifiedInstant();
  854. Instant fileLastModified = getEntryLastModifiedInstant();
  855. if ((getOptions().getCheckStat() == CheckStat.MINIMAL)
  856. || (cacheLastModified.getNano() == 0)
  857. // Some Java version on Linux return whole seconds only even
  858. // when the file systems supports more precision.
  859. || (fileLastModified.getNano() == 0)) {
  860. if (fileLastModified.getEpochSecond() != cacheLastModified
  861. .getEpochSecond()) {
  862. return MetadataDiff.DIFFER_BY_TIMESTAMP;
  863. }
  864. }
  865. if (!fileLastModified.equals(cacheLastModified)) {
  866. return MetadataDiff.DIFFER_BY_TIMESTAMP;
  867. } else if (entry.isSmudged()) {
  868. return MetadataDiff.SMUDGED;
  869. }
  870. // The file is clean when when comparing timestamps
  871. return MetadataDiff.EQUAL;
  872. }
  873. /**
  874. * Checks whether this entry differs from a given entry from the
  875. * {@link org.eclipse.jgit.dircache.DirCache}.
  876. *
  877. * File status information is used and if status is same we consider the
  878. * file identical to the state in the working directory. Native git uses
  879. * more stat fields than we have accessible in Java.
  880. *
  881. * @param entry
  882. * the entry from the dircache we want to compare against
  883. * @param forceContentCheck
  884. * True if the actual file content should be checked if
  885. * modification time differs.
  886. * @param reader
  887. * access to repository objects if necessary. Should not be null.
  888. * @return true if content is most likely different.
  889. * @throws java.io.IOException
  890. * @since 3.3
  891. */
  892. public boolean isModified(DirCacheEntry entry, boolean forceContentCheck,
  893. ObjectReader reader) throws IOException {
  894. if (entry == null)
  895. return !FileMode.MISSING.equals(getEntryFileMode());
  896. MetadataDiff diff = compareMetadata(entry);
  897. switch (diff) {
  898. case DIFFER_BY_TIMESTAMP:
  899. if (forceContentCheck)
  900. // But we are told to look at content even though timestamps
  901. // tell us about modification
  902. return contentCheck(entry, reader);
  903. else
  904. // We are told to assume a modification if timestamps differs
  905. return true;
  906. case SMUDGED:
  907. // The file is clean by timestamps but the entry was smudged.
  908. // Lets do a content check
  909. return contentCheck(entry, reader);
  910. case EQUAL:
  911. if (mode == FileMode.SYMLINK.getBits()) {
  912. return contentCheck(entry, reader);
  913. }
  914. return false;
  915. case DIFFER_BY_METADATA:
  916. if (mode == FileMode.TREE.getBits()
  917. && entry.getFileMode().equals(FileMode.GITLINK)) {
  918. byte[] idBuffer = idBuffer();
  919. int idOffset = idOffset();
  920. if (entry.getObjectId().compareTo(idBuffer, idOffset) == 0) {
  921. return true;
  922. } else if (ObjectId.zeroId().compareTo(idBuffer,
  923. idOffset) == 0) {
  924. return new File(repository.getWorkTree(),
  925. entry.getPathString()).list().length > 0;
  926. }
  927. return false;
  928. } else if (mode == FileMode.SYMLINK.getBits())
  929. return contentCheck(entry, reader);
  930. return true;
  931. default:
  932. throw new IllegalStateException(MessageFormat.format(
  933. JGitText.get().unexpectedCompareResult, diff.name()));
  934. }
  935. }
  936. /**
  937. * Get the file mode to use for the current entry when it is to be updated
  938. * in the index.
  939. *
  940. * @param indexIter
  941. * {@link org.eclipse.jgit.dircache.DirCacheIterator} positioned
  942. * at the same entry as this iterator or null if no
  943. * {@link org.eclipse.jgit.dircache.DirCacheIterator} is
  944. * available at this iterator's current entry
  945. * @return index file mode
  946. */
  947. public FileMode getIndexFileMode(DirCacheIterator indexIter) {
  948. final FileMode wtMode = getEntryFileMode();
  949. if (indexIter == null) {
  950. return wtMode;
  951. }
  952. final FileMode iMode = indexIter.getEntryFileMode();
  953. if (getOptions().isFileMode() && iMode != FileMode.GITLINK && iMode != FileMode.TREE) {
  954. return wtMode;
  955. }
  956. if (!getOptions().isFileMode()) {
  957. if (FileMode.REGULAR_FILE == wtMode
  958. && FileMode.EXECUTABLE_FILE == iMode) {
  959. return iMode;
  960. }
  961. if (FileMode.EXECUTABLE_FILE == wtMode
  962. && FileMode.REGULAR_FILE == iMode) {
  963. return iMode;
  964. }
  965. }
  966. if (FileMode.GITLINK == iMode
  967. && FileMode.TREE == wtMode && !getOptions().isDirNoGitLinks()) {
  968. return iMode;
  969. }
  970. if (FileMode.TREE == iMode
  971. && FileMode.GITLINK == wtMode) {
  972. return iMode;
  973. }
  974. return wtMode;
  975. }
  976. /**
  977. * Compares the entries content with the content in the filesystem.
  978. * Unsmudges the entry when it is detected that it is clean.
  979. *
  980. * @param entry
  981. * the entry to be checked
  982. * @param reader
  983. * acccess to repository data if necessary
  984. * @return <code>true</code> if the content doesn't match,
  985. * <code>false</code> if it matches
  986. * @throws IOException
  987. */
  988. private boolean contentCheck(DirCacheEntry entry, ObjectReader reader)
  989. throws IOException {
  990. if (getEntryObjectId().equals(entry.getObjectId())) {
  991. // Content has not changed
  992. // We know the entry can't be racily clean because it's still clean.
  993. // Therefore we unsmudge the entry!
  994. // If by any chance we now unsmudge although we are still in the
  995. // same time-slot as the last modification to the index file the
  996. // next index write operation will smudge again.
  997. // Caution: we are unsmudging just by setting the length of the
  998. // in-memory entry object. It's the callers task to detect that we
  999. // have modified the entry and to persist the modified index.
  1000. entry.setLength((int) getEntryLength());
  1001. return false;
  1002. } else {
  1003. if (mode == FileMode.SYMLINK.getBits()) {
  1004. return !new File(readSymlinkTarget(current())).equals(
  1005. new File(readContentAsNormalizedString(entry, reader)));
  1006. }
  1007. // Content differs: that's a real change, perhaps
  1008. if (reader == null) // deprecated use, do no further checks
  1009. return true;
  1010. switch (getEolStreamType()) {
  1011. case DIRECT:
  1012. return true;
  1013. default:
  1014. try {
  1015. ObjectLoader loader = reader.open(entry.getObjectId());
  1016. if (loader == null)
  1017. return true;
  1018. // We need to compute the length, but only if it is not
  1019. // a binary stream.
  1020. long dcInLen;
  1021. try (InputStream dcIn = new AutoLFInputStream(
  1022. loader.openStream(), true,
  1023. true /* abort if binary */)) {
  1024. dcInLen = computeLength(dcIn);
  1025. } catch (AutoLFInputStream.IsBinaryException e) {
  1026. return true;
  1027. }
  1028. try (InputStream dcIn = new AutoLFInputStream(
  1029. loader.openStream(), true)) {
  1030. byte[] autoCrLfHash = computeHash(dcIn, dcInLen);
  1031. boolean changed = getEntryObjectId()
  1032. .compareTo(autoCrLfHash, 0) != 0;
  1033. return changed;
  1034. }
  1035. } catch (IOException e) {
  1036. return true;
  1037. }
  1038. }
  1039. }
  1040. }
  1041. private static String readContentAsNormalizedString(DirCacheEntry entry,
  1042. ObjectReader reader) throws MissingObjectException, IOException {
  1043. ObjectLoader open = reader.open(entry.getObjectId());
  1044. byte[] cachedBytes = open.getCachedBytes();
  1045. return FS.detect().normalize(RawParseUtils.decode(cachedBytes));
  1046. }
  1047. /**
  1048. * Reads the target of a symlink as a string. This default implementation
  1049. * fully reads the entry's input stream and converts it to a normalized
  1050. * string. Subclasses may override to provide more specialized
  1051. * implementations.
  1052. *
  1053. * @param entry
  1054. * to read
  1055. * @return the entry's content as a normalized string
  1056. * @throws java.io.IOException
  1057. * if the entry cannot be read or does not denote a symlink
  1058. * @since 4.6
  1059. */
  1060. protected String readSymlinkTarget(Entry entry) throws IOException {
  1061. if (!entry.getMode().equals(FileMode.SYMLINK)) {
  1062. throw new java.nio.file.NotLinkException(entry.getName());
  1063. }
  1064. long length = entry.getLength();
  1065. byte[] content = new byte[(int) length];
  1066. try (InputStream is = entry.openInputStream()) {
  1067. int bytesRead = IO.readFully(is, content, 0);
  1068. return FS.detect()
  1069. .normalize(RawParseUtils.decode(content, 0, bytesRead));
  1070. }
  1071. }
  1072. private static long computeLength(InputStream in) throws IOException {
  1073. // Since we only care about the length, use skip. The stream
  1074. // may be able to more efficiently wade through its data.
  1075. //
  1076. long length = 0;
  1077. for (;;) {
  1078. long n = in.skip(1 << 20);
  1079. if (n <= 0)
  1080. break;
  1081. length += n;
  1082. }
  1083. return length;
  1084. }
  1085. private byte[] computeHash(InputStream in, long length) throws IOException {
  1086. SHA1 contentDigest = SHA1.newInstance();
  1087. final byte[] contentReadBuffer = state.contentReadBuffer;
  1088. contentDigest.update(hblob);
  1089. contentDigest.update((byte) ' ');
  1090. long sz = length;
  1091. if (sz == 0) {
  1092. contentDigest.update((byte) '0');
  1093. } else {
  1094. final int bufn = contentReadBuffer.length;
  1095. int p = bufn;
  1096. do {
  1097. contentReadBuffer[--p] = digits[(int) (sz % 10)];
  1098. sz /= 10;
  1099. } while (sz > 0);
  1100. contentDigest.update(contentReadBuffer, p, bufn - p);
  1101. }
  1102. contentDigest.update((byte) 0);
  1103. for (;;) {
  1104. final int r = in.read(contentReadBuffer);
  1105. if (r <= 0)
  1106. break;
  1107. contentDigest.update(contentReadBuffer, 0, r);
  1108. sz += r;
  1109. }
  1110. if (sz != length)
  1111. return zeroid;
  1112. return contentDigest.digest();
  1113. }
  1114. /**
  1115. * A single entry within a working directory tree.
  1116. *
  1117. * @since 5.0
  1118. */
  1119. public static abstract class Entry {
  1120. byte[] encodedName;
  1121. int encodedNameLen;
  1122. void encodeName(CharsetEncoder enc) {
  1123. final ByteBuffer b;
  1124. try {
  1125. b = enc.encode(CharBuffer.wrap(getName()));
  1126. } catch (CharacterCodingException e) {
  1127. // This should so never happen.
  1128. throw new RuntimeException(MessageFormat.format(
  1129. JGitText.get().unencodeableFile, getName()));
  1130. }
  1131. encodedNameLen = b.limit();
  1132. if (b.hasArray() && b.arrayOffset() == 0)
  1133. encodedName = b.array();
  1134. else
  1135. b.get(encodedName = new byte[encodedNameLen]);
  1136. }
  1137. @Override
  1138. public String toString() {
  1139. return getMode().toString() + " " + getName(); //$NON-NLS-1$
  1140. }
  1141. /**
  1142. * Get the type of this entry.
  1143. * <p>
  1144. * <b>Note: Efficient implementation required.</b>
  1145. * <p>
  1146. * The implementation of this method must be efficient. If a subclass
  1147. * needs to compute the value they should cache the reference within an
  1148. * instance member instead.
  1149. *
  1150. * @return a file mode constant from {@link FileMode}.
  1151. */
  1152. public abstract FileMode getMode();
  1153. /**
  1154. * Get the byte length of this entry.
  1155. * <p>
  1156. * <b>Note: Efficient implementation required.</b>
  1157. * <p>
  1158. * The implementation of this method must be efficient. If a subclass
  1159. * needs to compute the value they should cache the reference within an
  1160. * instance member instead.
  1161. *
  1162. * @return size of this file, in bytes.
  1163. */
  1164. public abstract long getLength();
  1165. /**
  1166. * Get the last modified time of this entry.
  1167. * <p>
  1168. * <b>Note: Efficient implementation required.</b>
  1169. * <p>
  1170. * The implementation of this method must be efficient. If a subclass
  1171. * needs to compute the value they should cache the reference within an
  1172. * instance member instead.
  1173. *
  1174. * @return time since the epoch (in ms) of the last change.
  1175. * @deprecated use {@link #getLastModifiedInstant()} instead
  1176. */
  1177. @Deprecated
  1178. public abstract long getLastModified();
  1179. /**
  1180. * Get the last modified time of this entry.
  1181. * <p>
  1182. * <b>Note: Efficient implementation required.</b>
  1183. * <p>
  1184. * The implementation of this method must be efficient. If a subclass
  1185. * needs to compute the value they should cache the reference within an
  1186. * instance member instead.
  1187. *
  1188. * @return time of the last change.
  1189. * @since 5.1.9
  1190. */
  1191. public abstract Instant getLastModifiedInstant();
  1192. /**
  1193. * Get the name of this entry within its directory.
  1194. * <p>
  1195. * Efficient implementations are not required. The caller will obtain
  1196. * the name only once and cache it once obtained.
  1197. *
  1198. * @return name of the entry.
  1199. */
  1200. public abstract String getName();
  1201. /**
  1202. * Obtain an input stream to read the file content.
  1203. * <p>
  1204. * Efficient implementations are not required. The caller will usually
  1205. * obtain the stream only once per entry, if at all.
  1206. * <p>
  1207. * The input stream should not use buffering if the implementation can
  1208. * avoid it. The caller will buffer as necessary to perform efficient
  1209. * block IO operations.
  1210. * <p>
  1211. * The caller will close the stream once complete.
  1212. *
  1213. * @return a stream to read from the file.
  1214. * @throws IOException
  1215. * the file could not be opened for reading.
  1216. */
  1217. public abstract InputStream openInputStream() throws IOException;
  1218. }
  1219. /** Magic type indicating we know rules exist, but they aren't loaded. */
  1220. private static class PerDirectoryIgnoreNode extends IgnoreNode {
  1221. final Entry entry;
  1222. PerDirectoryIgnoreNode(Entry entry) {
  1223. super(Collections.<FastIgnoreRule> emptyList());
  1224. this.entry = entry;
  1225. }
  1226. IgnoreNode load() throws IOException {
  1227. IgnoreNode r = new IgnoreNode();
  1228. try (InputStream in = entry.openInputStream()) {
  1229. r.parse(in);
  1230. }
  1231. return r.getRules().isEmpty() ? null : r;
  1232. }
  1233. }
  1234. /** Magic type indicating there may be rules for the top level. */
  1235. private static class RootIgnoreNode extends PerDirectoryIgnoreNode {
  1236. final Repository repository;
  1237. RootIgnoreNode(Entry entry, Repository repository) {
  1238. super(entry);
  1239. this.repository = repository;
  1240. }
  1241. @Override
  1242. IgnoreNode load() throws IOException {
  1243. IgnoreNode r;
  1244. if (entry != null) {
  1245. r = super.load();
  1246. if (r == null)
  1247. r = new IgnoreNode();
  1248. } else {
  1249. r = new IgnoreNode();
  1250. }
  1251. FS fs = repository.getFS();
  1252. String path = repository.getConfig().get(CoreConfig.KEY)
  1253. .getExcludesFile();
  1254. if (path != null) {
  1255. File excludesfile;
  1256. if (path.startsWith("~/")) //$NON-NLS-1$
  1257. excludesfile = fs.resolve(fs.userHome(), path.substring(2));
  1258. else
  1259. excludesfile = fs.resolve(null, path);
  1260. loadRulesFromFile(r, excludesfile);
  1261. }
  1262. File exclude = fs.resolve(repository.getDirectory(),
  1263. Constants.INFO_EXCLUDE);
  1264. loadRulesFromFile(r, exclude);
  1265. return r.getRules().isEmpty() ? null : r;
  1266. }
  1267. private static void loadRulesFromFile(IgnoreNode r, File exclude)
  1268. throws FileNotFoundException, IOException {
  1269. if (FS.DETECTED.exists(exclude)) {
  1270. try (FileInputStream in = new FileInputStream(exclude)) {
  1271. r.parse(in);
  1272. }
  1273. }
  1274. }
  1275. }
  1276. /** Magic type indicating we know rules exist, but they aren't loaded. */
  1277. private static class PerDirectoryAttributesNode extends AttributesNode {
  1278. final Entry entry;
  1279. PerDirectoryAttributesNode(Entry entry) {
  1280. super(Collections.<AttributesRule> emptyList());
  1281. this.entry = entry;
  1282. }
  1283. AttributesNode load() throws IOException {
  1284. AttributesNode r = new AttributesNode();
  1285. try (InputStream in = entry.openInputStream()) {
  1286. r.parse(in);
  1287. }
  1288. return r.getRules().isEmpty() ? null : r;
  1289. }
  1290. }
  1291. private static final class IteratorState {
  1292. /** Options used to process the working tree. */
  1293. final WorkingTreeOptions options;
  1294. /** File name character encoder. */
  1295. final CharsetEncoder nameEncoder;
  1296. /** Buffer used to perform {@link #contentId} computations. */
  1297. byte[] contentReadBuffer;
  1298. /** TreeWalk with a (supposedly) matching DirCacheIterator. */
  1299. TreeWalk walk;
  1300. /** Position of the matching {@link DirCacheIterator}. */
  1301. int dirCacheTree = -1;
  1302. /** Whether the iterator shall walk ignored directories. */
  1303. boolean walkIgnored = false;
  1304. final Map<String, Boolean> directoryToIgnored = new HashMap<>();
  1305. IteratorState(WorkingTreeOptions options) {
  1306. this.options = options;
  1307. this.nameEncoder = UTF_8.newEncoder();
  1308. }
  1309. void initializeReadBuffer() {
  1310. if (contentReadBuffer == null) {
  1311. contentReadBuffer = new byte[BUFFER_SIZE];
  1312. }
  1313. }
  1314. }
  1315. /**
  1316. * Get the clean filter command for the current entry.
  1317. *
  1318. * @return the clean filter command for the current entry or
  1319. * <code>null</code> if no such command is defined
  1320. * @throws java.io.IOException
  1321. * @since 4.2
  1322. */
  1323. public String getCleanFilterCommand() throws IOException {
  1324. if (cleanFilterCommandHolder == null) {
  1325. String cmd = null;
  1326. if (state.walk != null) {
  1327. cmd = state.walk
  1328. .getFilterCommand(Constants.ATTR_FILTER_TYPE_CLEAN);
  1329. }
  1330. cleanFilterCommandHolder = new Holder<>(cmd);
  1331. }
  1332. return cleanFilterCommandHolder.get();
  1333. }
  1334. /**
  1335. * Get the eol stream type for the current entry.
  1336. *
  1337. * @return the eol stream type for the current entry or <code>null</code> if
  1338. * it cannot be determined. When state or state.walk is null or the
  1339. * {@link org.eclipse.jgit.treewalk.TreeWalk} is not based on a
  1340. * {@link org.eclipse.jgit.lib.Repository} then null is returned.
  1341. * @throws java.io.IOException
  1342. * @since 4.3
  1343. */
  1344. public EolStreamType getEolStreamType() throws IOException {
  1345. return getEolStreamType(null);
  1346. }
  1347. /**
  1348. * @param opType
  1349. * The operationtype (checkin/checkout) which should be used
  1350. * @return the eol stream type for the current entry or <code>null</code> if
  1351. * it cannot be determined. When state or state.walk is null or the
  1352. * {@link TreeWalk} is not based on a {@link Repository} then null
  1353. * is returned.
  1354. * @throws IOException
  1355. */
  1356. private EolStreamType getEolStreamType(OperationType opType)
  1357. throws IOException {
  1358. if (eolStreamTypeHolder == null) {
  1359. EolStreamType type=null;
  1360. if (state.walk != null) {
  1361. type = state.walk.getEolStreamType(opType);
  1362. } else {
  1363. switch (getOptions().getAutoCRLF()) {
  1364. case FALSE:
  1365. type = EolStreamType.DIRECT;
  1366. break;
  1367. case TRUE:
  1368. case INPUT:
  1369. type = EolStreamType.AUTO_LF;
  1370. break;
  1371. }
  1372. }
  1373. eolStreamTypeHolder = new Holder<>(type);
  1374. }
  1375. return eolStreamTypeHolder.get();
  1376. }
  1377. private boolean isDirectoryIgnored(String pathRel) throws IOException {
  1378. final int pOff = 0 < pathOffset ? pathOffset - 1 : pathOffset;
  1379. final String base = TreeWalk.pathOf(this.path, 0, pOff);
  1380. final String pathAbs = concatPath(base, pathRel);
  1381. return isDirectoryIgnored(pathRel, pathAbs);
  1382. }
  1383. private boolean isDirectoryIgnored(String pathRel, String pathAbs)
  1384. throws IOException {
  1385. assert pathRel.length() == 0 || (pathRel.charAt(0) != '/'
  1386. && pathRel.charAt(pathRel.length() - 1) != '/');
  1387. assert pathAbs.length() == 0 || (pathAbs.charAt(0) != '/'
  1388. && pathAbs.charAt(pathAbs.length() - 1) != '/');
  1389. assert pathAbs.endsWith(pathRel);
  1390. Boolean ignored = state.directoryToIgnored.get(pathAbs);
  1391. if (ignored != null) {
  1392. return ignored.booleanValue();
  1393. }
  1394. final String parentRel = getParentPath(pathRel);
  1395. if (parentRel != null && isDirectoryIgnored(parentRel)) {
  1396. state.directoryToIgnored.put(pathAbs, Boolean.TRUE);
  1397. return true;
  1398. }
  1399. final IgnoreNode node = getIgnoreNode();
  1400. for (String p = pathRel; node != null
  1401. && !"".equals(p); p = getParentPath(p)) { //$NON-NLS-1$
  1402. ignored = node.checkIgnored(p, true);
  1403. if (ignored != null) {
  1404. state.directoryToIgnored.put(pathAbs, ignored);
  1405. return ignored.booleanValue();
  1406. }
  1407. }
  1408. if (!(this.parent instanceof WorkingTreeIterator)) {
  1409. state.directoryToIgnored.put(pathAbs, Boolean.FALSE);
  1410. return false;
  1411. }
  1412. final WorkingTreeIterator wtParent = (WorkingTreeIterator) this.parent;
  1413. final String parentRelPath = concatPath(
  1414. TreeWalk.pathOf(this.path, wtParent.pathOffset, pathOffset - 1),
  1415. pathRel);
  1416. assert concatPath(TreeWalk.pathOf(wtParent.path, 0,
  1417. Math.max(0, wtParent.pathOffset - 1)), parentRelPath)
  1418. .equals(pathAbs);
  1419. return wtParent.isDirectoryIgnored(parentRelPath, pathAbs);
  1420. }
  1421. private static String getParentPath(String path) {
  1422. final int slashIndex = path.lastIndexOf('/', path.length() - 2);
  1423. if (slashIndex > 0) {
  1424. return path.substring(path.charAt(0) == '/' ? 1 : 0, slashIndex);
  1425. }
  1426. return path.length() > 0 ? "" : null; //$NON-NLS-1$
  1427. }
  1428. private static String concatPath(String p1, String p2) {
  1429. return p1 + (p1.length() > 0 && p2.length() > 0 ? "/" : "") + p2; //$NON-NLS-1$ //$NON-NLS-2$
  1430. }
  1431. }